An app that uses the `MediaSession` without a `MediaSessionService` may
want to connect a media notification controller on it own. To avoid apps
using a string literal, the key should be exposed as public.
PiperOrigin-RevId: 589875603
The MP4 data in JPEG motion photos can contain multiple `video/hevc` tracks, but only the first is at a playable frame rate while the others are low-fps, high-res tracks designed for specific use-cases (not direct video playback).
ExoPlayer currently selects the unplayable track by default, because it
has a higher resolution. This change introduces a flag to
`Mp4Extractor` that results in the first video track being marked as
`ROLE_FLAG_MAIN`, and all subsequent video tracks `ROLE_FLAG_ALTERNATE`
- this then results in the playable lower-res track being selected by
default.
PiperOrigin-RevId: 589832072
There is a bug in CompositionVideoSinkProvider where the rotation
effect is applied only on the first video and the same effect
is applied on subsequent videos without checking if the next
items in the playlist must be rotated.
The bug applies only on devices with API < 21.
PiperOrigin-RevId: 589823797
We currently pause playback to prevent further progress while the
app thread runs assertion and triggers additional actions. This is
not ideal because we do not actually want to pause playback in
almost all cases where this method used.
This can be improved by keeping the playback thread blocked and only
unblock it the next time the app thread waits for the player (either
via RobolectricUtil methods or by blocking the thread itself). To
add this automatic handling, this change introduces a new util class
for the tests that can keep the list of waiting threads statically
(because the access to this logic is spread across multiple independent
classes).
PiperOrigin-RevId: 589784204
Move the `AnalyticsListener` implementation to a private inner class.
This avoids polluting the public API of `ExoHostedTest`, especially as
it's designed to be extended.
PiperOrigin-RevId: 589188113
Also adds an alternate way to configure the AudioGraph.
Apps should no longer need to ensure that inputs have the same sample
rate.
PiperOrigin-RevId: 588747431
Added a new ABANDONED option so one can tell the difference between when the optimization has been requested but not applied vs not requested at all. also changed the ordering do better represent the hierarchy of failure modes
PiperOrigin-RevId: 588720513
Fix a bug where CompositingVideoSinkProvider.isInitialized() returns
true even after releasing the CompositingVideoSinkProvider.
PiperOrigin-RevId: 588481744
This change moves the instantiation of the CompositingVideoSinkProvider
out of MediaCodecVideoRenderer so that the composition preview player can
re-use the CompositingVideoSinkProvider instance for driving the rendering of
images.
The main point of the change is the ownership of the
VideoFrameReleaseControl, which decides when a frame should be rendered
and so far was owned by the MediaCodecVideoRenderer. With this change,
in the context of composition preview, the VideoFrameReleaseControl
is no longer owned by MediaCodecVideoRenderer, but provided to it.
This way, the CompositingVideoSinkProvider instance, hence the
VideoFrameReleaseControl can be re-used to funnel images into the
video pipeline and render the pipeline from elsewhere (and not
MediaCodecVideoRenderer).
PiperOrigin-RevId: 588459007
The serializable form is used when we need to serialize the result into
bytes in the sample queue. The binder-based (ultimately
filedescriptor-based) form is used for
session/controller IPC, in order to avoid sending the bitmap bytes over
the IPC.
Issue: androidx/media#836
#minor-release
PiperOrigin-RevId: 588420836
This helped me debug which bitmap was actually failing. Otherwise, you need to clear all bitmaps on the device relating to this test, then adb pull all bitmaps and see the last uploaded one to see, which is much more confusing.
PiperOrigin-RevId: 588374081
This image has two video tracks in the MP4 data, one is a 'real' video
which we want to play by default, and the other is a low-fps video track
which isn't intended to be directly played, it's encoded in HEVC for
compression and decoding efficiency.
This test demonstrates ExoPlayer's current behaviour default extraction
and playback, which results in selecting the high-res, low-fps track
(actually single sample in this example), instead of playing the actual
video.
PiperOrigin-RevId: 588068908
As Opus decoders skip some bytes prior to playback during a seek, the renderer for bypass playback should send samples to the decoder even if they would be decode-only. However, the renderer should not send samples with time preceding that range. This change adds that constraint.
#minor-release
PiperOrigin-RevId: 588014983
Both the extension OPUS decoder and the OMX/C2 MediaCodec
implementations for OPUS and VORBIS decode into the channel
layout defined by VORBIS. See
https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-140001.2.3
While this is technically correct for a stand-alone OPUS or VORBIS
decoder, it doesn't match the channel layout expected by Android.
See https://developer.android.com/reference/android/media/AudioFormat#channelMask
The fix is to apply the channel mapping after decoding if needed.
Also add e2e tests with audio dumps for the extension renderer,
including a new 5.1 channel test file.
Issue: google/ExoPlayer#8396
#minor-release
PiperOrigin-RevId: 588004832
`mediaCrypto` is initialized before `codec` in
`maybeInitCodecOrBypass`. Before this change, it was possible for
`maybeInitCodecOrBypass` to complete with `mediaCrypto != null` and
`codec == null`, in particular if it was run as part of clearing the
player surface (since in that case, no video codec is initialized).
This inconsistent state then causes issues during a later invocation of
`maybeInitCodecOrBypass`, when `mediaCrypto` is still non-null, and
`mediaCryptoRequiresSecureDecoder = true`, but the
content has been changed to unencrypted with no associated DRM session.
This results in a playback error, because a secure decoder is
initialized but there's no DRM session available to work with it.
This change ensures that when `maybeInitCodecOrBypass` completes,
either both `mediaCrypto != null` and `codec != null` (i.e. codec
initialization was completed) or `mediaCrypto == null` and
`codec == null` (i.e. codec initialization was not completed). We also
ensure that when nulling out `mediaCrypto` we also set
`maybeInitCodecOrBypass = false`. A later change should be able to
demote `maybeInitCodecOrBypass` from a field to a local in order to
remove any risk of that part of state becoming out of sync. This
resolves the issue, because during the second invocation of
`maybeInitCodecOrBypass` an insecure decoder is now (correctly)
initialized and the unencrypted content is successfully played.
#minor-release
PiperOrigin-RevId: 587713911
We set the `playerId` only in the constructor currently. But in the case of this source doesn't have `preload` called, the player's release of this source will set the playerId to `null`, which makes this source un-reusable with a null `playerId`.
PiperOrigin-RevId: 587698214
* @Before and @After usage. [1].
* static fields before non-static. [2].
@Before method should typically be paired with an @After method,
focused on ensuring that the component is released regardless of what
the test does.
In tests, inlining final class variables is preferrable [1]. In general
things like the file path should be part of the test (the @Rule means
we don't need before/after) if only used once.
Statically importing values and using them directly is preferable to
having a variable declared as non-final that's effectively final,
because from a readability perspective someone can see (the caps) that
the value is final static and immutable, so doesn't have to check if
it's changed/reassigned.
PiperOrigin-RevId: 586697887
Before, a translucent overlay over an opaque video would result in a
translucent output. This is not consistent with physical properties of light
(if putting a translucent object in front of an opaque object, you can't see
behind the opaque object).
Using the mixing properties from DefaultVideoCompositor.
PiperOrigin-RevId: 586636275
`fromBundle` doesn't distinguish between `FIELD_BITMAP` and `FIELD_TEXT`
being present with a null value, or being absent, so we might as well
avoid including them when the value is null.
I've separated this from a later change to add
`Cue.toSerializableBundle` which will also skip setting a bitmap value
into the `Bundle` if `this.bitmap == null`. This is partly because it
results in changes to a lot of extractor test dump files, and it's
easier to review that as a separate change.
PiperOrigin-RevId: 586626141
Add an check when loading progressive media in case the load
is canceled. If the player is released very early, the progressive
media period may carry on with the initial loading unnecessarily.
PiperOrigin-RevId: 586288385
In `TextOverlay` and `DrawableOverlay`, treat `Bitmap` as a buffer, where we
allocate it rarely and reuse it as long as possible before making a new one.
In `BitmapOverlay`, avoid allocating GL textures too often as well.
Strongly reduces allocations and memory usage growth (saving ~100-150 MB on 4k60fps
at high end), at the cost of more code complexity and low-end using 70MB more, on
1/1 comparisons.
PiperOrigin-RevId: 585990602
In `ExternalTextureManager` fields are accessed from the GL thread and the class needs to be constructed on the GL thread.
Also visibly document threading requirement in the parent class.
PiperOrigin-RevId: 585941284
The issue that motivated adding this (frames unexpectedly being dropped by the
decoder) has been addressed, so we can turn off the logging to reduce
unnecessary allocations during transformation. We can easily turn on debug
logging in future as needed by setting `DebugTraceUtil.DEBUG = true`.
Also avoid allocations for string building at logging call sites by passing a
format string for extra info. Formatting the string now only happens when
debugging is turned on.
Tested manually by running transformations in the new state (DEBUG = false) and
with debugging turned on.
PiperOrigin-RevId: 585666349
Serializing bitmap cues is currently broken, but this test is
incorrectly passing. This change makes two changes to introduce the same
failure (both changes are necessary, each one alone still passes):
1. Move from Robolectric to an instrumentation test.
2. Trigger the `Bitmap` to be serialized using a file descriptor, either
by calling `Bitmap.asShared` in the test when constructing the `Cue`,
or constructing the `Bitmap` from a 'real' image byte array instead a
1x1 token image.
Issue: androidx/media#836
PiperOrigin-RevId: 585643486
The decoder and encoder won't accept high values for frame rate, so avoid
setting the key when configuring the decoder, and set a default value for the
encoder (where the key is required).
Also skip SSIM calculation for 4k, where the device lacks concurrent decoding
support.
PiperOrigin-RevId: 585604976
At least some Android 14 devices still have the same invalid URL issue
when using ClearKey DRM, so might as well apply the check to newer
versions of Android as well.
Using `Integer.MAX_VALUE` risks causing arithmetic overflow in the codec
implementation.
Issue: androidx/media#810
#minor-release
PiperOrigin-RevId: 585104621
We don't get the buffer back after decoding to make a proper decision
about whether dropping the buffer is needed, so we do the next best
thing and tell the codec to drop the buffer if its input timestamp is
less than the intended start time.
PiperOrigin-RevId: 584863144
Move the reflective loading of VideoFrameProcessor from
MediaCodecVideoRenderer to the CompositingVideoSinkProvider. This is so
that all reflective code lives in one place. The
CompositingVideoSinkProvider already has reflective code to load the
default PreviewingVideoGraph.Factory.
PiperOrigin-RevId: 584852547
These usages have no need for the double ended input functionality. All
other usages across media3 are ConcurrentLinkedQueue.
PiperOrigin-RevId: 584841104
In OverlayShaderProgram, this method is used quite a lot, and is the only method from Util.java in this file. Marginally reduce complexity by using a static import instead.
PiperOrigin-RevId: 584828455
Reduce short-lived allocations of potentially large objects, like Bitmap.
Unfortunately, this does make the TextureOverlay interface more messy though, requiring a way to signal whether the texture should be flipped vertically.
PiperOrigin-RevId: 584661400
These changes are also compatible with FFmpeg 5.1, which is now minimum supported version.
Also set -Wl,-Bsymbolic flag via target_link_options command which is more correct.
Split CompositingVideoSinkProvider.VideoSinkImpl in two classes:
- VideoSinkImpl now only receives input from MediaCodecVideoRenderer and
forwards frames to its connected VideoFrameProcessor
- VideoFrameRenderControl takes composited frames out of the VideoGraph
and schedules the rendering of those.
- CompositingVideoSinkProvider connects VideoSinkImpl with
VideoFramesRenderer.
PiperOrigin-RevId: 584605078
This change adds `MediaController.getSessionExtras()` through
which a controller can access the session extras.
The session extras can be set for the entire session when
building the session. This can be overridden for specific
controllers in `MediaSession.Callback.onConnect`.
PiperOrigin-RevId: 584430419
This change fixes a bug with seeking forward in MIDI. When seeking forward,
the progressive media period attempts to seek within the sample queue, if a
key-frame exists before the seeking position. With MIDI, however, we can
only skip Note-On and Note-Off samples and all other samples must be sent
to the MIDI decoder.
When seeking outside the sample queue, the MidiExtractor already
instructs the player to start from the beginning of the MIDI input. With
this change, only the first output sample is a key-frame, thus the
progressive media period can no longer seek within the sample queue and
is forced to seek from the MIDI input start always.
Issue: androidx/media#704
#minor-release
PiperOrigin-RevId: 584321443
The live offset override is used to replace the media-defined
live offset after user seeks to ensure the live adjustment adjusts
to the new user-provided live offset and doesn't go back to the
original one.
However, the code currently clips the override to the min/max
live offsets defined in LiveConfiguration. This is useful to
clip the default value (in case of inconsistent values in the media),
but the clipping shouldn't be applied to user overrides as
the player will then adjust the position back to the min/max
and doesn't stay at the desired user position.
See 2416d99857 (r132871601)
#minor-release
PiperOrigin-RevId: 584311004
Composition and EditedMediaItemSequence don't allow empty lists in their main
constructors, so shouldn't the vararg API. This is more inline with Effective
Java item 53.
PiperOrigin-RevId: 583415124
When transmuxing, the `EncodedSampleExporter` maintains a queue of input
buffers that get filled with encoded data by the asset loader. The number of
buffers was limited to avoid using more and more memory if producer (asset
loader) gets far ahead of the consumer (exporter).
Previously this limit was fixed at 10 buffers, but increasing the number of
buffers can make some transmux operations much faster. Allow allocating between
a min and max number of buffers, and also set a target allocation size beyond
which new buffers can't be allocated. This allows audio formats which require
many small buffers to be processed more quickly, while preventing allocating
too much memory for hypothetical very high bitrate formats.
'Remove video' edits on local videos in particular get much faster, because
audio buffers are very short and there are lots of them. With a sample 10
minute video, a 'remove video' edit took 2 seconds (36 seconds before this
change). With a sample 1 minute removing video took 0.25 seconds after this
change (2.5 seconds before).
The speed improvement is smaller for other types of edits that retain the video
track. Transmuxing a 10 minute video retaining the video track took 26 seconds
(40 seconds before).
PiperOrigin-RevId: 583390284
In some contexts (e.g. BottomSheetDialogFrament), Material Design
themes will override the default of singleLine=false to true. This
causes layout problems because the forward/rewind buttons are no
longer visible with singleLine=true.
This problem can be avoided by explicitly requesting the default
value of false in our layout files.
Issue: androidx/media#511
#minor-release
PiperOrigin-RevId: 582604131
This removes the flakiness of the HLS playback test as well.
Previously, this flow only worked for standalone WebVTT subtitles (7b762642db)
PiperOrigin-RevId: 582359383
The test is hidden behind the `@Ignore` annotation due to some
flakiness. However, it will be removed when the subtitle parsing is
moved to a pre-sample-queue architecture.
The test media was created with:
```shell
$ cp ../../dash/standalone-ttml/sample.xml sample.ttml
$ MP4Box -add sample.ttml sample.text.mp4
$ MP4Box -frag 10000 sample.text.mp4
$ rm sample.ttml
```
PiperOrigin-RevId: 582347113
This helps to play ads where the file extension is not sufficient
to deduce the correct source in DefaultMediaSourceFactory.
PiperOrigin-RevId: 582340114
Currently, ads are only defined by a single URL, which makes it
impossible to define additional fields needed to play ads correctly.
This can be fixed by using a full MediaItem in AdPlaybackState,
replacing the previous Uri field.
PiperOrigin-RevId: 582331588
Use renderengine's PQ to SDR tone-mapping implementation instead
of naive implementation from before.
This improves luminance on highlights, as seen in the test image.
PiperOrigin-RevId: 582318045
MIME types are case-insensitive, but none of the many existing
comparisons across our code base take this into account. The
code can be made more robust by normalizing all MIME types at the
moment they are first set into a class/builder and adding toLowerCase
as part of the normalization.
Most concretely, this fixes an issue with playing HLS streams via
the IMA SDK where the stream MIME type is indicated with all lower
case "application/x-mpegurl", which failed the MIME type comparison
in DefaultMediaSourceFactory.
PiperOrigin-RevId: 582317261
All methods in VideoFrameProcessor are expected to be called by the owning thread,
as far as I understand (vs. 10 threads each queuing frames/textures/streams, which
invalidates blocking done by registerInputStream and flush)
PiperOrigin-RevId: 582295240
BundledChunkExtractor creates a JpegExtractor or a PngExtractor if the
chunk's MIME type is JPEG or PNG respectively.
The JpegExtractor is instantiated to load an image track.
PiperOrigin-RevId: 582005828
Skip wearable devices, which don't have the same CDD requirements on codecs from Android 5 (API 21).
Limit an existing skipping case to pre API 33 devices, so that we can catch failures on Android 13 (API 33) onwards, and add OnePlus 9 Pro which also times out calculating SSIM. Remove TODOs for removing test/SSIM skipping now they are restricted to API version.
PiperOrigin-RevId: 581985554
This device over-reports encoding capabilities, so skip the test.
This is a Pixel device that was upgraded to Android 12 (API 31/32) so the issue should only affect devices that didn't take OTAs.
PiperOrigin-RevId: 581981073
Instantiate an ImageRenderer and add it to the list returned by
DefaultRenderersFactory.buildRenderers.
Add Renderer.MessageType.MSG_SET_IMAGE_OUTPUT and
ExoPlayer.setImageOutput to enable setting a custom
ImageRenderer.imageOutput.
Add ImageRenderer.handleMessage to process messages sent to the
renderer.
PiperOrigin-RevId: 581962451
Based on on-device testing, this device seems to have the same issue as Moto G (20) where frames are dropped despite configuring the decoder not to drop frames.
PiperOrigin-RevId: 581943805
The dump file diff as part of this change is because using AudioGraph
means the 2nd item is automatically edited to match the AudioFormat of
the 1st item {44.1KHz mono}, rather than {48KHz stereo}.
Manually verified that for the 2nd item, data output:
* Before: 66936 bytes (16734 frames) output = 348_625us of audio.
* After: 30750 bytes (15375 frames) output = 348_639us of audio.
The small final buffer is caused by SonicAudioProcessor outputting all
pending data when EOS queued, and is WAI.
PiperOrigin-RevId: 580494578
Changes includes:
1. Add resume flow.
2. Change demo app code to resume export.
3. Changes in test infra to trigger resume.
4. E2E Test cases
PiperOrigin-RevId: 579895744
Both interfaces are not really needed as the methods can be called
on the respective classes directly. This also avoid error-prone
situations where classes define to/fromBundle methods with parameters
that should be used instead of the parameter-less version.
Also deprecate all existing CREATOR static instances and make the
corresponding fromBundle method public where needed.
PiperOrigin-RevId: 579189766
* Rename method f(n) to a more helpful name.
* Move the private inner class to the bottom.
* Move public fields before private fields but below the static field.
* Make `private ParsableByteArray data` final.
* Make sure `parseSequenceHeader` can only be called once.
DefaultTrackSelector now has all logic necessary for selecting an
image track.
If isPrioritizeImageOverVideoEnabled is set to true, image track will
try to be selected first and a video track will only be selected if no
image track is available. If isPrioritizeImageOverVideoEnabled is set
to false, image track will be selected only if video track wasn't
selected.
PiperOrigin-RevId: 578806006
The native code can now reallocate the buffer if it needs to grow
its size, so we have to reacquire a reference in the Java code to
avoid accessing a stale instance.
This fixes a bug introduced by 8750ed8de6.
PiperOrigin-RevId: 578799862
BundleableUtil is the only class that really depends on the type
inheritence from Bundleable. However, it only needs this as a way
to define Function<T, Bundle> and Function<Bundle, T>, which could
just be passed in as parameters as it's already done for some of
these methods.
Also rename the class to BundleCollectionUtil accordingly.
PiperOrigin-RevId: 578791946
Android Studio removed some nested imports, but I think the extra
qualification at the usage site is actually mostly helpful, so I'm
leaving it as-is.
PiperOrigin-RevId: 578518880
Unfortunately we can't fail any more obviously at this point, because
manifests often contain config for multiple DRM schemes, and when
parsing the manifest we don't know which scheme is going to be used for
playback. It would be unreasonable to fail playback due to incomplete
ClearKey config if playback was otherwise going to succeed using e.g.
Widevine.
* Issue: androidx/media#777
* Issue: androidx/media#563
* Issue: google/ExoPlayer#9169
#minor-release
PiperOrigin-RevId: 578491484
The toBundle method should only be used for remote processes,
because there is a separate method for in-process bundling.
Renaming the method makes this more explicit and less error-prone.
PiperOrigin-RevId: 578456532
This moves the invocation of `transferInitializing` slightly earlier,
but this is consistent with other `DataSource` implementations like
`OkHttpDataSource`.
PiperOrigin-RevId: 578137236
This has been documented to work since this class was created, but until
now we were always trying to resolve using the current application's
`Resources.getIdentifier` method. This commit changes to resolve to the
other app's `Resources` object if the package name doesn't match the
current package.
This will only work if the current app has package-visibility to the
destination package: http://g.co/dev/packagevisibility
This is hard to test because to do so robustly requires being able to
guaranteed that another test APK will be installed with a known raw
resource inside it.
PiperOrigin-RevId: 577864992
This change applies to standalone WebVTT files linked directly from the manifest.
Since DASH only supports stand-alone IMSC1 (TTML) and WebVTT text files, this change concludes the support extension of text-based subtitle files to be parse during extraction.
PiperOrigin-RevId: 577468830
This change applies to standalone TTML files linked directly from the manifest.
As a result, we no longer have the flakiness in the DashPlaybackTest which uses sidecar-loaded (standalone file) TTML subtitles. We experimentally opt into parsing subtitles during extraction and use SubtitleExtractor in hybrid mode.
PiperOrigin-RevId: 577457256
Throws when calling flush when there's no active input, for example
before an input stream is registered or after all output streams have
ended.
PiperOrigin-RevId: 577165419
This CL adds an isPrioritizeImageOverVideoEnabled flag to
TrackSelectionParameters and an API to set the flag value.
The flag will be used by DefaultTrackSelector to determine whether
to select an image track if both an image track and a video track are available.
PiperOrigin-RevId: 576175162
When transmuxing, we usually only need to offset the timestamp by the position of a mediaItem in a sequence.
Trim optimization introduces another type of offset: for the transmux of the second part of the video we need to offset the timestamps by the total duration already trancoded by transformer.
PiperOrigin-RevId: 576134656
The events happens in the following order, assuming two media items:
1. First media item is fully decoded, record the last frame's pts
- Note frame processing is still ongoing for this media item
2. Renderer sends `onOutputFormatChanged()` to signal the second media item
3. **Block sending the frames of the second media item to the `VideoSink`**
4. Frame processing finishes on the first media item
5. The last frame of the first media item is released
6. **Reconfigure the `VideoSink` to apply new effects**
7. **Start sending the frames of the second media item to the `VideoSink`**
This CL implements the **events in bold**
PiperOrigin-RevId: 576098798
