This refactors the `supportsFormat` method on the MediaCodecVideoRenderer to be a static method that can be called from external components to determine if a video format is supported by the device renderers. Since `context` is the only component that is part of the MCVR, it can easily be obtained externally and passed to the method.
This can help optimize format support decisions ahead of time of a player having been instantiated, such as removing unsupported representations from a dash manifest.
This could also be done for the MCAR but would also require passing in the AudioSink and isn't required at this time.
PlaybackVideoGraphWrapper will soon contain an input and an output video
sink, as the rendering of the VideoGraph output frames will be handled
by a DefaultVideoSink instance.
PiperOrigin-RevId: 683167795
If a codec received the EOS signal and already returned the last
output buffer, we should expect the output EOS very quickly. If
it doesn't arrive within 100ms, we can proceed to end the stream
manually without waiting any further to prevent cases where the
codec is completely stuck otherwise.
PiperOrigin-RevId: 679633116
The buffered position was last updated before the beginning of
the renderer loop in doSomeWork. As the loading happens on a
background thread, it may have progressed further already
depending on how long it took to run the renderer loop.
It's slightly more correct to pass in an updated value to
shouldStartPlayback so that playback can start quicker if the
buffering is particularly fast.
PiperOrigin-RevId: 679203465
We began to use a different preload thread than the main thread in the `DefaultPreloadManagerTest`, then in the test execution, we should also ensure that the events queued on the preload looper have been executed.
Also, there is an edge case also causing flakiness. Assume that `DefaultPreloadManager` is preloading source B, then the app invalidates and triggers another sequence of preloading, say they are A, B and C. There is possibility that source B finishes preloading (started before `invalidate`) when A starting to preload, then `onPreloadSkipped` will be triggered for source B instead of `onPreloadCompleted`. However, the functional block inside of `onPreloadSkipped` is dispatched asynchronously, and by then it's possible that B starts to preload again at the consequence of A being completed, then the functional block just mentioned may think that the current source matches at B, and will advance the current preloading source to C, without informing the `Listener.onPreloadCompleted` for it. As the result, the `Listener.onPreloadCompleted` can never be triggered for B for the second sequence. To fix this, we should prevent the functional block in `onPreloadCompleted`, `onPreloadError`, `onPreloadSkipped` to be even dispatched, when the source doesn't match the current preloading one.
PiperOrigin-RevId: 679145353
This check is also done when setting a surface on ExoPlayerImpl,
but by the time we configure the codec the surface may have become
invalid (e.g. when it is destroyed). Even though we immediately remove
a destroyed surface, we could still accidentally use it before the
removal is processed. To avoid these edge cases, we can simply not
configure the codec with an invalid surface.
PiperOrigin-RevId: 678741425
Allow apps to preload the first period of the next window in
the playlist of `ExoPlayer`. By default playlist preloading is
disabled. To enable preloading,
`ExoPlayer.setPreloadConfiguration(PreloadConfiguration)` can be
called.
`LoadControl` determines when to preload with its implemenation of `shouldContinuePreloading(timeline, mediaPeriodId, bufferedDurationUs)`.
The implementation in `DefaultLoadControl` allows preloading only when
the player isn't currently loading for playback. Apps can override this
behaviour.
Issue: androidx/media#468
PiperOrigin-RevId: 677786017
AudioTrack doesn't automatically ramp up the volume after a flush
(only when resuming with play after a pause), which causes audible
pop sounds in most cases. The issue can be avoided by manually
applying a short 20ms volume ramp, the same duration used by the
platform for the automatic volume ramping where available.
Together with the already submitted 6147050b90, this fixes the
unwanted pop sounds for most cases in the desired way. It only
leaves two cases that are not handled perfectly:
- If the media file itself contains a volume ramp at the beginning,
we wouldn't need this additional ramping. Given the extremely
short duration, this seems ignorable and we can treat it as a
future feature request to mark the beginning of media in a special
way that can then disable the volume ramping.
- For seamless period transitions where we keep using the same
AudioTrack, we may still get a pop sound at the transition. To
solve this, we'd need a dedicated audio processor to either ramp
the end of media down and the beginning of the next item up, or
apply a very short cross-fade. Either way, we need new signalling
to identify cases where the media originates from the same source
and this effect should not be applied (e.g. when re-concatenating
clipped audio snippets from the same file).
PiperOrigin-RevId: 676860234
This method is documented that it may only be called in `STATE_OPENED`
or `STATE_OPENED_WITH_KEYS`. It's possible for it to be called in other
states (like `STATE_ERROR`) without this guard.
Previously this didn't cause issues, but since 9d62845c45
we assume that the `sessionId` is non-null in this method, which results
in an `IllegalStateException` when the documented state restriction is
ignored.
PiperOrigin-RevId: 675969256
This is no longer needed now our `compileSdk` implies a new-enough AGP
which does this out-lining automatically via R8. See also
https://issuetracker.google.com/345472586#comment7
There's no plan to remove the `ApiXXX` classes, but no new ones need
to be added.
PiperOrigin-RevId: 675940634
The Galaxy Tab S7 FE has a device issue that causes 60fps secure H264 streams to be marked as unsupported. This CL adds a workaround for this issue by checking the CDD required support for secure H264 in addition to the current check on standard H264. If the provided performance points do not cover the CDD requirement of support 720p H264 at 60fps, then it falls back to using legacy methods for checking frame rate and resolution support.
Issue: androidx/media#1619
PiperOrigin-RevId: 675920968
Previously, track IDs were added to `trackIndicesPerSampleInQueuedOrder`
even when the sample was not committed. This caused issues where attempts
to read samples from the `SampleQueue` returned `C.RESULT_READ_NOTHING`,
which led to an exception being thrown due to the assumption that samples
were available to read.
This fix updates the logic to track sample commits by comparing the write index before and after calling `SampleQueue.sampleMetadata`. Track indices are only added if the sample was committed, ensuring accurate sample handling and avoiding exceptions.
PiperOrigin-RevId: 675526115
`PreloadMediaSource` allows to have a `startPositionUs` passed when `preload` is called, then in `PreloadControl.onContinueLoadingRequested`, it can be more intuitive to see the buffered duration rather than the absolute buffered position as the preload progress. Similar in `DefaultPreloadManager`, we haven't allowed the apps to set a custom start position for individual sources though, once we add this support, using the "duration from the start position" than the absolute position will be less error-prone, otherwise, it can run into a case that the position that the apps set is smaller than the start position.
PiperOrigin-RevId: 674251362
When processing edit lists in MP4 files, the media start position may be a non-keyframe. To ensure proper playback, the decoder must preroll to the preceding keyframe, as these preroll samples are essential for decoding but are not rendered.
Issue: google/ExoPlayer#1659
#cherrypick
PiperOrigin-RevId: 673457615
Avoids the dispatch of listener notifications of stream type/device volume change during construction of ExoPlayer. That was problematic because we end up blocking on `constructorFinished.blockUninterruptible()`
Issue: androidx/media#1692
PiperOrigin-RevId: 672965552
`Muxer` module needs to use this method, hence moved to common.
This CL also makes `getHevcProfileAndLevel` public because this is used
in `MediaCodecUtil`.
PiperOrigin-RevId: 671739166
Handling for `MediaCodec.CryptoException` was originally added only
around calls to `MediaCodec.queueSecureInputBuffer` and
`queueInputBuffer` (because these are the only methods that can throw
this exception). When asynchronous interaction with `MediaCodec` was
added in <unknown commit>, exceptions from `MediaCodec` started being stored
and bubbled out of **later** interactions with `MediaCodecAdapter`. This
means that `MediaCodecRenderer` can now see `CryptoException` thrown
from a different method, like
`MediaCodecAdapter.dequeueInputBufferIndex()`, and this ends up missing
the `catch (CryptoException)` code in `MediaCodecRenderer`. This results
in an "unexpected runtime error" stack trace like [A].
This change fixes the stack trace to:
1. Make it a "renderer exception" instead of "unexpected runtime error"
2. Include the correct DRM error code -> `@PlaybackException.ErrorCode`
mapping.
You can see the corrected stack trace below [B].
-----
[A] (synthesized from manually throwing a `CryptoException` from
`AsynchronousMediaCodecBufferEnqueuer#doQueueSecureInputBuffer`)
```
playerFailed [eventTime=11.56, mediaPos=10.35, window=0, period=0, errorCode=ERROR_CODE_UNSPECIFIED
androidx.media3.exoplayer.ExoPlaybackException: Unexpected runtime error
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:729)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:85)
Caused by: android.media.MediaCodec$CryptoException: Test error message
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doQueueSecureInputBuffer(AsynchronousMediaCodecBufferEnqueuer.java:232)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doHandleMessage(AsynchronousMediaCodecBufferEnqueuer.java:196)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.access$000(AsynchronousMediaCodecBufferEnqueuer.java:47)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer$1.handleMessage(AsynchronousMediaCodecBufferEnqueuer.java:93)
at android.os.Handler.dispatchMessage(Handler.java:107)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:85)
```
[B]
```
Playback error
androidx.media3.exoplayer.ExoPlaybackException: MediaCodecAudioRenderer error, index=1, format=Format(0, null, null, audio/mp4a-latm, mp4a.40.2, 134359, en, [-1, -1, -1.0, null], [2, 44100]), format_supported=YES
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:649)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:85)
Caused by: android.media.MediaCodec$CryptoException: Test error message
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doQueueSecureInputBuffer(AsynchronousMediaCodecBufferEnqueuer.java:232)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doHandleMessage(AsynchronousMediaCodecBufferEnqueuer.java:196)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.access$000(AsynchronousMediaCodecBufferEnqueuer.java:47)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer$1.handleMessage(AsynchronousMediaCodecBufferEnqueuer.java:93)
at android.os.Handler.dispatchMessage(Handler.java:107)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:85)
```
PiperOrigin-RevId: 670951229
Limit input image size in Transformer to be less than 4096x4096.
For very large images, this can reduce memory usage substantially,
and stays away from `GL_MAX_TEXTURE_SIZE` - often 4096
PiperOrigin-RevId: 670555939
DASH periods don't have to start at the beginning of a segment. In
these cases, they should report an initial discontinuity to let the
player know it needs to expect preroll data (e.g. to flush renderers)
This information is only available in the ChunkSampleStream after
loading the initialization data, so we need to check the sample
streams and tell them to only report discontinuities at the very
beginning of playback. All other position resets are triggered by
the player itself and don't need this method.
Issue: androidx/media#1440
PiperOrigin-RevId: 668831563
In the flaky test `ExoPlayerTest.loading_withLargeAllocationCausingOom_playsRemainingMediaAndThenThrows`, it is indeterministic that when the message `MSG_IO_EXCEPTION` from the loader thread will arrive on the playback looper. If it arrives after `MSG_PERIOD_PREPARED`, then the period can continue loading and get the three samples written to the `SampleQueue` before the intentional OOM surfacing to the `ExoPlayerImplInternal`, otherwise, the OOM will be detected by `ExoPlayerImplInternal` very early and the player disallows to load three samples, which will cause the assertion to fail.
As we are expecting the three samples to play until the playback fails, we should assume that the `Loader` encounters the OOM after those samples loaded, thus we need to put this trigger a bit later until the `SampleStreamItem`s are handled by the `FakeSampleStream`. This could be checked by the return value of `FakeMediaPeriod.continueLoading` (super class implementation). However, `FakeMediaPeriod.continueLoading` originally always returns `true`, which is not aligned with the javadoc of `MediaPeriod.continueLoading`:
"return `true` if progress was made, meaning that `getNextLoadPositionUs()` will return a different value than prior to the call, `false` otherwise."
then we should also modify that logic.
PiperOrigin-RevId: 668438316
This callback allows listeners to track when individual renderers
allow or prevent playback from being ready. For example, this is useful
to figure out which renderer blocked the playback the longest.
PiperOrigin-RevId: 667970933
Before this CL, the video sink was stuck if a flush was executed:
- after VideoSink.onInputStreamChanged, which is setting
pendingInputStreamBufferPresentationTimeUs and
- before the previous stream was fully rendered.
This is because pendingInputStreamBufferPresentationTimeUs was not reset
to TIME_UNSET when flushing the sink, so that the sink was still waiting
for the last frame of the previous stream to be rendered in
handleInputFrame/Bitmap.
PiperOrigin-RevId: 667924517
Per the javadoc, the method `MediaPeriod.maybeThrowPrepareError` is only allowed to be called before the period has completed preparation. For later errors in loading the streams, `SampleStream.maybeThrowError` will be called instead.
PiperOrigin-RevId: 665831430
It is possible for playback to be stuck when there is failure in loading further data, while the player is required to load more due to the buffered duration being under `DefaultLoadControl.bufferForPlayback`. Therefore, we check if there is any loading error in `isLoadingPossible`, so that the player will allow the playback of the existing data rather than waiting forever for the data that can never be loaded.
Issue: androidx/media#1571
#cherrypick
PiperOrigin-RevId: 665801674
This uses a 'bundled' extractor (`SubtitleExtractor`) because
`MediaParser` doesn't support transcoding subtitles to
`application/x-media3-cues`. This transcoding is required to support
non-legacy subtitle handling where they are parsed during extraction,
instead of during rendering.
PiperOrigin-RevId: 665282298
This is only used as part of legacy subtitle decoding. If any apps are
using it directly (rather than via `DashMediaSource` or `SsMediaSource`)
they probably need to enable legacy decoding in `TextRenderer` to avoid
playback failures.
PiperOrigin-RevId: 663740343
The method currently has two ways of operating with a complicated
contract:
1. outputBuffer == null -> sets a new outputBuffer and starts
draining it as far as possible.
2. outputBuffer != null -> continues draining a previous buffer,
requires that the input parameter is the same as outputBuffer.
This contract can be made cleaner by splitting the logic into
(1) setting a new output buffer and (2) draining the output buffer.
Only one parameter is needed for each method and there is no
requirement to pass in the same argument as before because we already
know what we are draining. This also means the caller of the method
doesn't have to keep track of the actual outputBuffer, allowing
further processing steps in the future that change the drained
instance.
PiperOrigin-RevId: 663316251
Previous to this change, `FrameworkMediaDrm.requiresSecureDecoder`
ignores its `sessionId` parameter on API 31+, and uses only the
`mimeType` parameter. This means the result [assumes the session is
opened at the 'default security
level'](https://developer.android.com/reference/android/media/MediaDrm#requiresSecureDecoder(java.lang.String)):
> The default security level is defined as the highest security level
> supported on the device.
This change is a no-op in all (?) cases, because the `ExoMediaDrm`
interface only exposes the zero-arg `openSession()` method, which in the
framework case **also** assumes the highest security level is preferred:
> By default, sessions are opened at the native security level of the
> device.
However, it seems more obviously correct to only make this
"highest/native security level" assumption in one place
(`openSession()`), and check the session's **actual** security level
everywhere else.
Issue: androidx/media#1603
PiperOrigin-RevId: 662872860
This method was added in API 31 (S) but it's non-functional
(incorrectly, silently, returns `false`) on the Widevine plugin version
(`16.0`) from R (API 30), which some devices up to at least API 34 are
still using.
This results in ExoPlayer incorrectly selecting an insecure decoder for
L1 secure content, and subsequently calling
`MediaCodec.queueInputBuffer` instead of `queueSecureInputBuffer`,
which is not supported and generates the following error:
> Operation not supported in this configuration: ERROR_DRM_CANNOT_HANDLE
Issue: androidx/media#1603
#cherrypick
PiperOrigin-RevId: 662852176
When there is an exception thrown from the `LoadTask`, the `Loader` will call `Loader.Callback.onLoadError`. Some implementations of `onLoadError` method may call `MediaPeriod.onContinueLoadingRequested`, and in the `PreloadMediaSource`, its `PreloadMediaPeriodCallback` will be triggered and then it can further call `continueLoading` if it finds needed. However the above process is currently done synchronously, which will cause problem. By calling `continueLoading`, the `Loader` is set with a `currentTask`, and when that long sync logic in `Loader.Callback.onLoadError` ends, the `Loader` will immediately retry, and then a non-null `currentTask` will cause the `IllegalStateException`.
Issue: androidx/media#1568
#cherrypick
PiperOrigin-RevId: 662550622
Check if the output device supports spatialization for the requested output format. If so, return a stream decoded for 6 channels in a 5.1 layout. Otherwise, return a stream decoded for 2 channels in a binaural layout.
PiperOrigin-RevId: 662546818
This is an additional signal that legacy subtitle support needs to be
explicitly enabled, and is going away at some point.
PiperOrigin-RevId: 661305694
The integration with external libraries like Glide or Coil is
currently extremely complicated. Providing the boilerplate code
to handle the ImageDecoder interface and a better hook for custom
image decoders in DefaultRenderersFactory allows apps to easily
inject their own logic for external image loading libraries.
PiperOrigin-RevId: 659508914
This API allows users to retrieve the sample size, ensuring they can allocate sufficient buffer capacity before utilizing the `readSampleData(ByteBuffer buffer, int offset)` method to read data.
PiperOrigin-RevId: 658772408
Refactor to replace instances of `RuntimeException` with `IllegalStateException`.
This change ensures exceptions can be handled more specifically without
unintended catches of unrelated exceptions.
PiperOrigin-RevId: 658337459
Added a new data source which acts an adapter to read media data from platform `MediaDataSource`. This enables adding the `setDataSource(MediaDataSource)` API to `MediaExtractorCompat`.
PiperOrigin-RevId: 657564901
Added three `setDataSource` APIs in `MediaExtractorCompat`:
- `setDataSource(Context context, Uri uri, @Nullable Map<String, String> headers)` to set data source with a content URI and optional headers.
- `setDataSource(String path)` to set data source using a file path or HTTP URL.
- `setDataSource(String path, @Nullable Map<String, String> headers)` to set data source using a file path or HTTP URL with optional headers.
PiperOrigin-RevId: 657563973
The builtin speaker is to be supported as a suitable output when that is deliberately selected for the media playback by the user in Wear OS.
PiperOrigin-RevId: 655950824
The DefaultPreloadManagerTest didn't to catch this because we use main looper as the preload looper in the tests. This CL also improves the tests by assigning the preload looper with one that corresponds to a different thread.
PiperOrigin-RevId: 655664189
