Add missing ones for `ForwardingExtractorInput` and `ForwardingSeekMap`.
`ForwardingTimeline` is a bit more fiddly, so it's left for a follow-up
change.
PiperOrigin-RevId: 701988492
Previously cancelling one FrameExtractor.getFrame
ListenableFuture caused the following requests to fail.
Implement ability to cancel Futures, and correctly issue
next player seek commands after all past requests complete.
Removes uses of SettableFuture, and replaces them with
CallbackToFutureAdapter.
Adds a dependency on androidx.concurrent:concurrent-futures
PiperOrigin-RevId: 701941403
Frame Extractor was getting stuck with SeekParameters.CLOSEST_SYNC.
onPositionDiscontinuity callback was sometimes being called with a
non-adjusted new position.
Fix this by monitoring player state ready.
For the player to become ready, we have to override renderer isReady.
PiperOrigin-RevId: 701924752
When a session sets a custom layout, we currently convert it to the
media button preferences in MediaControllerImplBase and convert it back
to a custom layout for consumers of MediaController.getCustomLayout.
To avoid unnecessary conversions and potential changes, we can
directly use the provided custom layout to get the same logic as
before. This also means we avoid notifying a change to the custom layout
if only the implicit slots changed that weren't relevant before
introducing them (=we can remove some tests for this behavior).
PiperOrigin-RevId: 701903475
The isLastBuffer flag passed to MediaCodecRenderer.processOutputBuffer()
is unreliable. It is true for the last frames (plural) of a video
stream, which makes it possible for the video renderer to end before all
the frames have been rendered to the output surface.
PiperOrigin-RevId: 700941685
This aligns the behavior with `MediaExtractor`, which sets this key in its [`MediaFormat`](https://developer.android.com/reference/android/media/MediaExtractor#getTrackFormat(int)) output.
Additionally, unnecessary `selectTrack` calls have been removed from the existing `getTrackFormat_...` tests, as they are not required to fetch the track format.
PiperOrigin-RevId: 700741326
Currently, `size` is cached in `SampleMetadata` when samples are committed in `SampleQueue`, which worked because we didn't support reading files with `CryptoInfo`. This change prepares for future `CryptoInfo` support, as the size changes when reading samples with encrypted data. Caching the size at commit time could lead to incorrect buffer `size` calculations.
Existing tests verify the correctness of the `getSampleSize()` API.
PiperOrigin-RevId: 700714178
The Android SKD version 30 is stable, allowing us to remove the `@SuppressLint("Override")` annotations from `InputReaderAdapterV30` and `OutputConsumerAdapterV30`.
PiperOrigin-RevId: 700681115
Introduced a `mediaSourceFactory` attribute to CompositionPlayer. This allows for the use of a custom MediaSource.Factory when creating media sources, providing more flexibility and control over media source creation. Previously, the default `MediaSource.Factory` was used in all cases.
PiperOrigin-RevId: 700391911
The previous code assumed that the `VBRI` Table of Contents (ToC)
covers all the MP3 data in the file. In a file with an invalid VBRI ToC
where this isn't the case, this results in playback silently stopping
mid-playback (and either advancing to the next item, or continuing to
count up the playback clock forever). This change considers the `bytes`
field to determine the end of the MP3 data, in addition to deriving it
from the ToC. If they disagree we log a warning and take the max value.
This is because we handle accidentally reading non-MP3 data at the end
(or hitting EoF) better than stopping reading valid MP3 data partway
through.
Issue: androidx/media#1904
#cherrypick
PiperOrigin-RevId: 700319250
The current code assumes that the first Table of Contents segment
includes the `VBRI` frame, but I don't think this is correct and it
should only include real/audible MP3 ata - so this change updates the
logic to assume the first ToC segment starts at the frame **after** the
`VBRI` frame.
Issue: androidx/media#1904
#cherrypick
PiperOrigin-RevId: 700269811
This was hand-crafted with a 4-entry ToC by modifying
`bear-vbr-xing-header.mp3` in a hex editor.
The output difference from 117 samples to 116 samples is due to the
calculation in `VbriSeeker` assuming that the ToC includes the VBRI
frame itself, which I don't think is correct (fix is in a follow-up
change).
Issue: androidx/media#1904
#cherrypick
PiperOrigin-RevId: 700254516
Support extracting frames from HDR input by tone mapping
to SDR (BT.709).
ExperimentalFrameExtractor must be public because HDR tests
live in a different package.
PiperOrigin-RevId: 699994112
Adds a configuration option to Frame Extractor to choose MediaCodecSelector.
Add a MediaCodecSelector that lists software decoders first
PiperOrigin-RevId: 699962365
Renderers join when they can start processing but not produce output.
In CompositionPlayer before this change, the VideoSink will be flushed when
position is reset regardless. This is useful for seeking, as seeking triggers position reset. But with pre-warming, a video renderer can be joining - it is
enabled (which also triggers position reset) before the previous image renderer
is disabled. At this moment, as the image renderer is still producing frames,
we should not flush the video sink just now.
PiperOrigin-RevId: 699943882
We are currently waiting until stream 1 is completely rendered on screen
before registering stream 2 but that is not necessary anymore because
VideoFrameProcessor supports registering stream 2 before stream 1 is
fully processed.
PiperOrigin-RevId: 699929943
Before this CL, the following scenario could happen:
- A new input stream is registered to the DefaultVideoFrameProcessor.
- Before the pipeline is reconfigured, a seek is issued andd the
DefaultVideoFrameProcessor is flushed.
- As a result, the new input stream registration is never taken into
account.
As a result:
- If an input stream is registered after the seek (before queueing any
frame), registerInputStream will block indefinitely because
inputStreamRegisteredCondition will be closed.
- If a frame is queued after the seek, it will be linked to the input
stream information of the previous frames.
This CL makes sure that any pending input stream is registered after a
flush.
PiperOrigin-RevId: 698736866
This call does 2 things:
1. It's used to estimate the frame rate early but this is incorrect as
the frame rate can be changed by effects. I have tested playback on
my device and I don't see any difference.
2. For ExoPlayer.setVideoEffects() only: it allows dropping all the
frames until the next key frame in case the input frame is "very
late". This doesn't seem really necessary though because because we
do the same thing after applying effects.
This change is also a step towards moving all the calls to
VideoFrameRelease/RenderControl to DefaultVideoSink.
PiperOrigin-RevId: 698732161
Changes includes making sure the same `defaultMediaSourceFactory` is used when creating a new media source in `setPrimaryPlayerSequence` and `setSecondaryPlayerSequence`.
This is pre-work for introducing a new attribute `mediaSourceFactory` that can be optionally used instead of the default one. This will allow developers to pass a customized media source factory to CompositionPlayer.
PiperOrigin-RevId: 698721139
This CL also aligns supported color space in `media3 common` and `media3 muxer`.
Earlier `muxer` would take even those values which are considered invalid
in `media3` in general.
Earlier muxer would throw if a given `color standard` is not recognized
but with the new change, it will rather put default `unspecified` value.
#cherrypick
PiperOrigin-RevId: 698683312
Adds a Frame extractor test that verifies decoder
respects rotation metadata from the mp4 container.
Do not rely on the MediaCodec decoder rotate the input.
Rotate via a video effect instead.
PiperOrigin-RevId: 698381282
This non-functional refactor inlines and simplifies the logic of
`BaseAudioProcessor` used by `SpeedChangingAudioProcessor`, and makes
the latter implement `AudioProcessor` directly.
`SpeedChangingAudioProcessor` acts as a wrapper over
`SonicAudioProcessor` and does not actually modify any samples,
so it had very little use for the affordances provided by
`BaseAudioProcessor`.
PiperOrigin-RevId: 698342962
3P app reported it's slow to see the first image frame on the screen, when
`playWhenReady` is false. This is because the player would wake up less
frequently when `playWhenReady` is false. This CL adds a runnable to wake up
the player any time a new frame is made available.
PiperOrigin-RevId: 698066887
SpeedChangingAudioProcessor is redundantly bypassing SonicAudioProcessor
when the speed is set to 1f. SpeedChangingAudioProcessor should not make
assumptions about the underlying audio processing and moreover should
not be bypassing any AudioProcessor based on parameter values. Default
parameter values are a valid state for an active AudioProcessor.
Sonic already handles the "default case" state by just copying the input
buffer onto the output buffer.
This CL also simplifies SonicAudioProcessor, which would mark itself as
inactive when configured with a valid set of default parameters. The API
contract for `isActive()` makes no mention about parameter state, which
makes changes in `isActive()` after applying new valid parameters quite
unintuitive.
PiperOrigin-RevId: 698000500
The following was happening:
- VideoFrameRenderControl.render() was calling
VideoFrameReleaseAction.getFrameReleaseAction().
- VideoFrameReleaseAction.getFrameReleaseAction() was calling
FrameTimingEvaluator.shouldIgnoreFrame(), which is implemented in
MediaCodecVideoRenderer.
- MediaCodecVideoRenderer.shouldIgnoreFrame(), when returning true,
was also flushing the VideoSink, which was flushing the
VideoFrameRenderControl.
- VideoFrameRenderControl.render() was removing the frame from the
presentationTimestampsUs queue, but the frame had already been removed
by the flush operation, causing an exception to be thrown.
This fix removes the last step.
PiperOrigin-RevId: 697915692
To do this, refactor stream start position handling so that
VideoFrameRenderControl is only passed a start position when it should
be applied, and therefore doesn't need to take a timestamp associated
with each start position anymore
PiperOrigin-RevId: 697544416
From [ the last change in `DefaultPreloadManagerTest`](2b54b1ebbe), the preloadManager began to use a separate `preloadThread` in `release_returnZeroCount_sourcesAndRendererCapabilitiesListReleased`, which unveils a bug in `PreloadMediaSource`. When `PreloadMediaSource.releasePreloadMediaSource` is called, `preloadHandler` will post a `Runnable` on the preload looper to release the internal resources. Before this `Runnable` is executed, it is possible that the [`stopPreloading`](https://github.com/androidx/media/blob/main/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreloadMediaSource.java#L442) method is executed just as the result of preloading has completed. This is expected to remove the posted `Runnable`s for further preloading, however, the posted `Runnable` for releasing will also be removed from the message queue.
Ideally we should use `postDelayed(runnable, token, delayMillis)` to post the runnables so that the token will be useful to identify which messages to remove in `removeCallbacksAndMessages(token)`, but that `postDelayed` method is only available from API 28. So in this change we are using a separate handler for releasing, and then the call of `preloadHandler.removeCallbacksAndMessages` won't impact the runnable for releasing.
#cherrypick
PiperOrigin-RevId: 696894483
This method returns a `PersistableBundle` containing metrics data for the current media container. The bundle includes attributes and values for the media container, as described in `MediaExtractor.MetricsConstants`.
PiperOrigin-RevId: 696893276
The new callback allows an app to read data from the content
timeline to populate the `AdPlaybackState`. To avoid a deadlock
between the `AdMediaSource` waiting for the `AdPlaybackState` and
the app waiting for the content `Timeline`, a boolean flag
`useLazyContentSourcePreparation` is introduced to tell the
`AdsMediaSource` to prepare the content source immediately
to make the `Timeline` available.
A unit test verifies that in none of the cases (lazy preparation or
immediate preparation) a `Timeline` without ad data is leaked to the
caller. This ensures that in the case of a preroll, the player won't
initially and accidentally read content media data before starting to
load the preroll ad. While the content source is prepared early, no
content media period must be created before the preroll starts.
PiperOrigin-RevId: 696885392
ISO/IEC 23001-8 has been withdrawn. The corresponding definition for channel configuration is available in ISO/IEC 23091-3. Update the comment to reflect this change.
The content in Issue: androidx/media#1863 has every sample incorrectly marked as a
sync sample in the MP4 metadata, which results in flushing the
re-ordering queue on every sample, so nothing gets re-ordered, so the
subtitles are garbled.
There are currently two "uses" for this call on every keyframe:
1. It offers a safety valve if we don't read a `maxNumReorderSamples`
value from the video. Without this, the queue will just keep growing
and end up swallowing all subtitle data (similar to the bug fixed by
39c734963f).
2. When we do read (or infer) a `maxNumReorderSamples` it means we can
emit samples from the queue slightly earlier - but this is pretty
marginal, given i think the max possible value for
`maxNumReorderSamples` is 16, so the most benefit we would get is 16
frames (~0.53s at 30fps) - in most cases we will have more than 0.5s
of buffer ahead of the playback position, so the subtitles will still
get shown at the right time with no problem.
(1) is resolved in this change by setting the queue size to zero (no
reordering) if we don't have a value for `maxNumReorderSamples`.
(2) has minimal impact, so we just accept it.
We may be able to inspect the NAL unit to determine IDR vs non-IDR
instead - we will consider this as a follow-up change, but given the
minimal impact of (2) we may not pursue this.
#cherrypick
PiperOrigin-RevId: 696583702
- Rename a few methods/variable to improve readability.
- Refactor how video size changes are tracked. This will simplify the
upcoming logic to refactor the timestamp logic in
VideoFrameRenderControl because we will use the same logic for
outputVideoSize and for outputStreamStartPositionUs.
PiperOrigin-RevId: 696026515
Earlier watchdog timer was started only after all the tracks are
added. For the cases where export is stuck before adding tracks,
the export would hang forever and will not timeout automatically.
With the changes, watchdog timer is started as soon as export is
started/resumed. Now if the pipeline is stuck before writing any samples,
it will timeout and report error to the caller.
Existing test case: c35a9d62ba/libraries/transformer/src/test/java/androidx/media3/transformer/MediaItemExportTest.java (L996)
PiperOrigin-RevId: 695781654
The behaviour was changed in 1.4.0 with 0f42dd4752,
so that the buffer timestamp is compared to `outputStartTimeUs` when
deciding whether to discard a "decode only" buffer before decoding
(instead of the deprecated/removed `isDecodeOnly` property). This breaks
when the buffer timestamp is `TIME_END_OF_SOURCE` (which is
`Long.MIN_VALUE`), because `TIME_END_OF_SOURCE < outputStartTimeUs` is
always true, so the end-of-stream buffer is never passed to the decoder
and on to `TextRenderer` where it is used to
[set `inputStreamEnded = true`](40f187e4b4/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/text/TextRenderer.java (L434-L436))
and so playback hangs.
#cherrypick
Issue: androidx/media#1863
PiperOrigin-RevId: 695767247
Implemented `getCachedDuration()` to provide an estimate of cached data in memory and `hasCacheReachedEndOfStream()` to indicate if the cache has reached the end of the stream.
Note: The Javadoc for the newly added methods closely follows that of the platform `MediaExtractor`. While the current implementation always uses a cache and therefore never returns `-1` from `getCachedDuration`, this leaves room for future changes should caching behavior or conditions evolve.
PiperOrigin-RevId: 695694460
A skeleton implementation that doesn't actually return decoded frames.
In the current state, we use ExoPlayer to seek to a position, and
ExoPlayer.setVideoEffects to record the presentation time selected.
Seeking and processing frames are synchronized via ListenableFuture
callbacks.
PiperOrigin-RevId: 695691183
In some cases, callers will need to pass in both BitmapLoader and Codec.DecoderFactory without specifying a custom MediaSource.Factory. Omitting the annotation will result in NULL_FOR_NONNULL_TYPE compilation errors in Kotlin.
PiperOrigin-RevId: 695481606
This is because the method is called for each media item, the current name
implies that the method is only called once for all videos in a sequence.
PiperOrigin-RevId: 695332916
