Commit graph

5350 commits

Author SHA1 Message Date
Ian Baker
2683331299 Add tests and missing overrides to ForwardingExtractor 2024-11-28 14:01:14 +00:00
Ian Baker
01422d8322 Reformat with google-java-format 2024-11-28 11:45:09 +00:00
Colin Kho
6d265476e4 Added Forwarding classes for Extractor and ExtractorOutput 2024-11-28 11:45:09 +00:00
kimvde
4d925bde81 Remove stale TODO
Handling (very) late buffers is already supported.

PiperOrigin-RevId: 700965581
2024-11-28 03:02:59 -08:00
kimvde
852091f2d9 Implement DefaultVideoSink.isEnded
To do that, I had to add VideoFrameRenderControl.signalEndOfInput() and
isEnded() to match the VideoSink interface.

PiperOrigin-RevId: 700957098
2024-11-28 02:27:50 -08:00
kimvde
f3f4646296 Add VideoSink.signalEndOfCurrentInputStream()
The isLastBuffer flag passed to MediaCodecRenderer.processOutputBuffer()
is unreliable. It is true for the last frames (plural) of a video
stream, which makes it possible for the video renderer to end before all
the frames have been rendered to the output surface.

PiperOrigin-RevId: 700941685
2024-11-28 01:23:55 -08:00
kimvde
dfe3c90f9a Fix CompositionPlayerTest.imagePreview_imagePlaysForSetDuration
PiperOrigin-RevId: 700931124
2024-11-28 00:36:46 -08:00
rohks
08619bf6fa Add KEY_TRACK_ID to MediaFormat returned by getTrackFormat(int)
This aligns the behavior with `MediaExtractor`, which sets this key in its [`MediaFormat`](https://developer.android.com/reference/android/media/MediaExtractor#getTrackFormat(int)) output.

Additionally, unnecessary `selectTrack` calls have been removed from the existing `getTrackFormat_...` tests, as they are not required to fetch the track format.

PiperOrigin-RevId: 700741326
2024-11-27 10:26:30 -08:00
dancho
6e29b96337 Add support for extracting HDR frames as HLG Bitmaps
* Add GlUtils to create RGB10_A2 textures.
* Add OETF conversion to FrameReadingGlShaderProgram.
* Add FrameExtractor configuration API to request HDR output: API 34+

PiperOrigin-RevId: 700738895
2024-11-27 10:18:57 -08:00
rohks
270543555d Add getSampleCryptoInfo API to MediaExtractorCompat
This method enables handling encrypted samples by providing the necessary decryption details.

PiperOrigin-RevId: 700729949
2024-11-27 09:48:11 -08:00
kimvde
c861947bee Skip failing test on API 31 emulator
PiperOrigin-RevId: 700722123
2024-11-27 09:17:52 -08:00
rohks
52e45e0864 Prepare for CryptoInfo support by removing cached size
Currently, `size` is cached in `SampleMetadata` when samples are committed in `SampleQueue`, which worked because we didn't support reading files with `CryptoInfo`. This change prepares for future `CryptoInfo` support, as the size changes when reading samples with encrypted data. Caching the size at commit time could lead to incorrect buffer `size` calculations.

Existing tests verify the correctness of the `getSampleSize()` API.

PiperOrigin-RevId: 700714178
2024-11-27 08:47:37 -08:00
Copybara-Service
bff5523bb6 Merge pull request #1823 from MGaetan89:remove_outdated_sdk_check
PiperOrigin-RevId: 700706152
2024-11-27 08:16:02 -08:00
ibaker
60133b0c7e Recommend ForwardingSimpleBasePlayer in ForwardingPlayer javadoc
Also add an explicit warning about how fiddly `ForwardingPlayer` can be
to use correctly.

PiperOrigin-RevId: 700698032
2024-11-27 07:42:24 -08:00
rohks
a371824dbc Remove unnecessary @SuppressLint("Override")
The Android SKD version 30 is stable, allowing us to remove the `@SuppressLint("Override")` annotations from `InputReaderAdapterV30` and `OutputConsumerAdapterV30`.

PiperOrigin-RevId: 700681115
2024-11-27 06:32:24 -08:00
ibaker
6cf3004d62 Bump IMA dependency to 3.35.1
The previous version (3.33.0) is known to have some bugs, and the latest
version (3.36.0) is also known to be buggy.

PiperOrigin-RevId: 700657484
2024-11-27 04:56:28 -08:00
shahddaghash
d5d5771cd2 Add custom MediaSource.Factory attribute to CompositionPlayer
Introduced a `mediaSourceFactory` attribute to CompositionPlayer. This allows for the use of a custom MediaSource.Factory when creating media sources, providing more flexibility and control over media source creation. Previously, the default `MediaSource.Factory` was used in all cases.

PiperOrigin-RevId: 700391911
2024-11-26 10:48:22 -08:00
claincly
96c35966d8 Remove unnecessary parentherses
PiperOrigin-RevId: 700356007
2024-11-26 08:49:20 -08:00
claincly
de76d7932f Add missing test cases in SeekTest
PiperOrigin-RevId: 700354445
2024-11-26 08:43:06 -08:00
ibaker
b19b6ccc60 MP3 VBRI: Use sampleCountToDurationUs() and samplesPerFrame
PiperOrigin-RevId: 700340564
2024-11-26 07:52:18 -08:00
ibaker
46578ee0a6 MP3: Use bytes field from VBRI frame instead of deriving from ToC
The previous code assumed that the `VBRI` Table of Contents (ToC)
covers all the MP3 data in the file. In a file with an invalid VBRI ToC
where this isn't the case, this results in playback silently stopping
mid-playback (and either advancing to the next item, or continuing to
count up the playback clock forever). This change considers the `bytes`
field to determine the end of the MP3 data, in addition to deriving it
from the ToC. If they disagree we log a warning and take the max value.
This is because we handle accidentally reading non-MP3 data at the end
(or hitting EoF) better than stopping reading valid MP3 data partway
through.

Issue: androidx/media#1904

#cherrypick

PiperOrigin-RevId: 700319250
2024-11-26 06:18:29 -08:00
claincly
2a4bae01ef Add two missing test cases in seeking
PiperOrigin-RevId: 700284751
2024-11-26 03:44:23 -08:00
ibaker
f257e5511f MP3: Exclude VBRI frame from ToC position calculations
The current code assumes that the first Table of Contents segment
includes the `VBRI` frame, but I don't think this is correct and it
should only include real/audible MP3 ata - so this change updates the
logic to assume the first ToC segment starts at the frame **after** the
`VBRI` frame.

Issue: androidx/media#1904

#cherrypick

PiperOrigin-RevId: 700269811
2024-11-26 02:34:33 -08:00
Gaëtan Muller
2fc379a61b Fix documentation for Format.Builder.setTileCountHorizontal and Format.Builder.setTileCountVertical 2024-11-26 10:05:16 +00:00
ibaker
3eb36d67bd Add MP3 test asset with VBRI frame
This was hand-crafted with a 4-entry ToC by modifying
`bear-vbr-xing-header.mp3` in a hex editor.

The output difference from 117 samples to 116 samples is due to the
calculation in `VbriSeeker` assuming that the ToC includes the VBRI
frame itself, which I don't think is correct (fix is in a follow-up
change).

Issue: androidx/media#1904

#cherrypick

PiperOrigin-RevId: 700254516
2024-11-26 01:33:39 -08:00
dancho
bb20eb4975 Frame Extractor HDR: tone map to SDR
Support extracting frames from HDR input by tone mapping
to SDR (BT.709).

ExperimentalFrameExtractor must be public because HDR tests
live in a different package.

PiperOrigin-RevId: 699994112
2024-11-25 08:41:42 -08:00
dancho
0d8f1d5ab9 Add Frame Extractor configuration for mediacodecselector
Adds a configuration option to Frame Extractor to choose MediaCodecSelector.
Add a MediaCodecSelector that lists software decoders first

PiperOrigin-RevId: 699962365
2024-11-25 06:40:28 -08:00
claincly
7ef1a7ab8b Support renderer joining in CompositionPlayer
Renderers join when they can start processing but not produce output.

In CompositionPlayer before this change, the VideoSink will be flushed when
position is reset regardless. This is useful for seeking, as seeking triggers position reset. But with pre-warming, a video renderer can be joining -  it is
enabled (which also triggers position reset) before the previous image renderer
is disabled. At this moment, as the image renderer is still producing frames,
we should not flush the video sink just now.

PiperOrigin-RevId: 699943882
2024-11-25 05:22:22 -08:00
kimvde
25a782e10a Stop delaying registering input stream on VideoFrameProcessor
We are currently waiting until stream 1 is completely rendered on screen
before registering stream 2 but that is not necessary anymore because
VideoFrameProcessor supports registering stream 2 before stream 1 is
fully processed.

PiperOrigin-RevId: 699929943
2024-11-25 04:17:32 -08:00
ibaker
827966b7a4 Add pixel aspect ratio to Format.toLogString
#cherrypick

PiperOrigin-RevId: 698770714
2024-11-21 07:21:11 -08:00
shahddaghash
73c4bb6e1f Bump Media3 version to 1.5.0
PiperOrigin-RevId: 698761734
2024-11-21 06:47:36 -08:00
kimvde
52dfe25542 Fix flushing issue in DefaultVideoFrameProcessor
Before this CL, the following scenario could happen:
- A new input stream is registered to the DefaultVideoFrameProcessor.
- Before the pipeline is reconfigured, a seek is issued andd the
  DefaultVideoFrameProcessor is flushed.
- As a result, the new input stream registration is never taken into
  account.

As a result:
- If an input stream is registered after the seek (before queueing any
  frame), registerInputStream will block indefinitely because
  inputStreamRegisteredCondition will be closed.
- If a frame is queued after the seek, it will be linked to the input
  stream information of the previous frames.

This CL makes sure that any pending input stream is registered after a
flush.

PiperOrigin-RevId: 698736866
2024-11-21 05:00:32 -08:00
kimvde
79effe7b60 Remove call to getFrameReleaseAction prior to applying effects
This call does 2 things:
1. It's used to estimate the frame rate early but this is incorrect as
   the frame rate can be changed by effects. I have tested playback on
   my device and I don't see any difference.
2. For ExoPlayer.setVideoEffects() only: it allows dropping all the
   frames until the next key frame in case the input frame is "very
   late". This doesn't seem really necessary though because because we
   do the same thing after applying effects.

This change is also a step towards moving all the calls to
VideoFrameRelease/RenderControl to DefaultVideoSink.

PiperOrigin-RevId: 698732161
2024-11-21 04:38:54 -08:00
michaelkatz
5282fe3125 Rollback of 854566dbfe
PiperOrigin-RevId: 698730105
2024-11-21 04:29:19 -08:00
shahddaghash
754dfd76c8 Simplify use of defaultMediaSourceFactory in CompositionPlayer
Changes includes making sure the same `defaultMediaSourceFactory` is used when creating a new media source in `setPrimaryPlayerSequence` and `setSecondaryPlayerSequence`.
This is pre-work for introducing a new attribute `mediaSourceFactory` that can be optionally used instead of the default one. This will allow developers to pass a customized media source factory to CompositionPlayer.

PiperOrigin-RevId: 698721139
2024-11-21 03:52:58 -08:00
sheenachhabra
407bc4fec9 Manage all color value conversions in ColorInfo class
This CL also aligns supported color space in `media3 common` and `media3 muxer`.

Earlier `muxer` would take even those values which are considered invalid
in `media3` in general.

Earlier muxer would throw if a given `color standard` is not recognized
but with the new change, it will rather put default `unspecified` value.

#cherrypick

PiperOrigin-RevId: 698683312
2024-11-21 01:09:02 -08:00
Copybara-Service
089eaa1d5d Merge pull request #1898 from DolbyLaboratories:dlb/ac4-level4-dash/dev
PiperOrigin-RevId: 698456089
2024-11-20 11:18:59 -08:00
dancho
d92c9aa8a1 Add a Frame Extractor test with rotated input
Adds a Frame extractor test that verifies decoder
respects rotation metadata from the mp4 container.

Do not rely on the MediaCodec decoder rotate the input.
Rotate via a video effect instead.

PiperOrigin-RevId: 698381282
2024-11-20 07:20:50 -08:00
ivanbuper
66e8b53b43 Make SpeedChangingAudioProcessor direct subclass of AudioProcessor
This non-functional refactor inlines and simplifies the logic of
`BaseAudioProcessor` used by `SpeedChangingAudioProcessor`, and makes
the latter implement `AudioProcessor` directly.

`SpeedChangingAudioProcessor` acts as a wrapper over
`SonicAudioProcessor` and does not actually modify any samples,
so it had very little use for the affordances provided by
`BaseAudioProcessor`.

PiperOrigin-RevId: 698342962
2024-11-20 04:27:27 -08:00
claincly
3c01500a4e Use dynamic scheduling to speed up rendering the first image frame
3P app reported it's slow to see the first image frame on the screen, when
`playWhenReady` is false. This is because the player would wake up less
frequently when `playWhenReady` is false. This CL adds a runnable to wake up
the player any time a new frame is made available.

PiperOrigin-RevId: 698066887
2024-11-19 10:33:59 -08:00
dancho
dc7a0ca22f Move SDK == 33 tests to SDK 33+
Ensure tests run on any recent emulator.

PiperOrigin-RevId: 698015529
2024-11-19 07:55:32 -08:00
ivanbuper
646a6352a2 Remove bypass of SonicAudioProcessor in SpeedChangingAudioProcessor
SpeedChangingAudioProcessor is redundantly bypassing SonicAudioProcessor
when the speed is set to 1f. SpeedChangingAudioProcessor should not make
assumptions about the underlying audio processing and moreover should
not be bypassing any AudioProcessor based on parameter values. Default
parameter values are a valid state for an active AudioProcessor.

Sonic already handles the "default case" state by just copying the input
buffer onto the output buffer.

This CL also simplifies SonicAudioProcessor, which would mark itself as
inactive when configured with a valid set of default parameters. The API
contract for `isActive()` makes no mention about parameter state, which
makes changes in `isActive()` after applying new valid parameters quite
unintuitive.

PiperOrigin-RevId: 698000500
2024-11-19 07:02:20 -08:00
rohks
d702e1d496 Add getPsshInfo() API to MediaExtractorCompat
This method retrieves PSSH (Protection System Specific Header) data from the media as a map of UUID-to-bytes.

PiperOrigin-RevId: 697974134
2024-11-19 05:14:55 -08:00
bachinger
b9c9d95b90 Add JSON asset list parser
PiperOrigin-RevId: 697966021
2024-11-19 04:41:17 -08:00
Copybara-Service
98723dc0a8 Merge pull request #1888 from DolbyLaboratories:dlb/dash-parser/dev
PiperOrigin-RevId: 697956473
2024-11-19 04:01:30 -08:00
dancho
ccc7b22ff4 Implement custom Frame Extractor renderer
Render only one frame per seek to reduce the amount of work done

PiperOrigin-RevId: 697946350
2024-11-19 03:14:57 -08:00
dancho
26f10effc2 Add frame extractor tests for decoder counters.
Right now, decoders produce an unspecified (but large)
number of frames. This will be resolved in a follow-up

PiperOrigin-RevId: 697945076
2024-11-19 03:10:17 -08:00
bachinger
c7ca9fbe7d Sanitize duration parsing for HLS interstitials
PiperOrigin-RevId: 697941445
2024-11-19 02:54:53 -08:00
kimvde
b782cdff52 Don't drop frames to ignore in RenderControl
The following was happening:
- VideoFrameRenderControl.render() was calling
VideoFrameReleaseAction.getFrameReleaseAction().
- VideoFrameReleaseAction.getFrameReleaseAction() was calling
FrameTimingEvaluator.shouldIgnoreFrame(), which is implemented in
MediaCodecVideoRenderer.
- MediaCodecVideoRenderer.shouldIgnoreFrame(), when returning true,
was also flushing the VideoSink, which was flushing the
VideoFrameRenderControl.
- VideoFrameRenderControl.render() was removing the frame from the
presentationTimestampsUs queue, but the frame had already been removed
by the flush operation, causing an exception to be thrown.

This fix removes the last step.

PiperOrigin-RevId: 697915692
2024-11-19 01:15:17 -08:00
claincly
9c6d9e9e47 Reduce test name lengths
Some parametrized names are too long (>255 chars). Remove putting parameter
info in the test name and instead log them.

PiperOrigin-RevId: 697677513
2024-11-18 10:34:06 -08:00
bachinger
9ae136becb Add context as a parameter to shouldStartForegroundService
Issue: androidx/media#1887
PiperOrigin-RevId: 697650546
2024-11-18 09:15:46 -08:00
kimvde
8be2556efd Implement DefaultVideoSink.setStreamTimestampInfo
To do this, refactor stream start position handling so that
VideoFrameRenderControl is only passed a start position when it should
be applied, and therefore doesn't need to take a timestamp associated
with each start position anymore

PiperOrigin-RevId: 697544416
2024-11-18 02:01:52 -08:00
kimvde
2568ff73cb Remove unnecessary checkStateNotNull checks
LongArrayQueue.remove() throws a NoSuchElementException if the queue is
empty.

PiperOrigin-RevId: 697531458
2024-11-18 01:05:34 -08:00
ybai001
7b3effb871 Update test code based on the latest code structure 2024-11-18 10:29:40 +08:00
ibaker
cbb8e2f1e6 Bump media3 version to 1.5.0-rc02
PiperOrigin-RevId: 696912494
2024-11-15 09:33:11 -08:00
tianyifeng
0143884cd7 Deflake the DefaultPreloadManagerTest
From [ the last change in `DefaultPreloadManagerTest`](2b54b1ebbe), the preloadManager began to use a separate `preloadThread` in `release_returnZeroCount_sourcesAndRendererCapabilitiesListReleased`, which unveils a bug in `PreloadMediaSource`. When `PreloadMediaSource.releasePreloadMediaSource` is called, `preloadHandler` will post a `Runnable` on the preload looper to release the internal resources. Before this `Runnable` is executed, it is possible that the [`stopPreloading`](https://github.com/androidx/media/blob/main/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreloadMediaSource.java#L442) method is executed just as the result of preloading has completed. This is expected to remove the posted `Runnable`s for further preloading, however, the posted `Runnable` for releasing will also be removed from the message queue.

Ideally we should use `postDelayed(runnable, token, delayMillis)` to post the runnables so that the token will be useful to identify which messages to remove in `removeCallbacksAndMessages(token)`, but that `postDelayed` method is only available from API 28. So in this change we are using a separate handler for releasing, and then the call of `preloadHandler.removeCallbacksAndMessages` won't impact the runnable for releasing.

#cherrypick

PiperOrigin-RevId: 696894483
2024-11-15 08:22:54 -08:00
rohks
ecc5cd889f Add getMetrics() API to MediaExtractorCompat
This method returns a `PersistableBundle` containing metrics data for the current media container. The bundle includes attributes and values for the media container, as described in `MediaExtractor.MetricsConstants`.

PiperOrigin-RevId: 696893276
2024-11-15 08:17:26 -08:00
bachinger
1af0b5b432 Add handleContentTimelineChanged to AdsLoader
The new callback allows an app to read data from the content
timeline to populate the `AdPlaybackState`. To avoid a deadlock
between the `AdMediaSource` waiting for the `AdPlaybackState` and
the app waiting for the content `Timeline`, a boolean flag
`useLazyContentSourcePreparation` is introduced to tell the
`AdsMediaSource` to prepare the content source immediately
to make the `Timeline` available.

A unit test verifies that in none of the cases (lazy preparation or
immediate preparation) a `Timeline` without ad data is leaked to the
caller. This ensures that in the case of a preroll, the player won't
initially and accidentally read content media data before starting to
load the preroll ad. While the content source is prepared early, no
content media period must be created before the preroll starts.

PiperOrigin-RevId: 696885392
2024-11-15 07:49:41 -08:00
rohks
c3d4a3d683 Set container MIME type for extractors in ts module
PiperOrigin-RevId: 696882489
2024-11-15 07:37:43 -08:00
rohks
68c0b8115f Set container MIME type for Flv, Matroska, and image extractors
Image extractors (e.g., `PngExtractor`) use `SingleSampleExtractor`
for extraction.

PiperOrigin-RevId: 696869657
2024-11-15 06:42:12 -08:00
dancho
11fc0871ac Implement video effects for Frame Extraction
Test that downscaled images match MediaMetadataRetriever.

PiperOrigin-RevId: 696862566
2024-11-15 06:09:55 -08:00
ybai001
c503a63878 Update comment for MPEG_CHANNEL_CONFIGURATION_MAPPING
ISO/IEC 23001-8 has been withdrawn. The corresponding definition for channel configuration is available in ISO/IEC 23091-3. Update the comment to reflect this change.
2024-11-15 16:31:52 +08:00
ibaker
e6448f3498 Don't assume MP4 keyframe metadata is correct for CEA re-ordering
The content in Issue: androidx/media#1863 has every sample incorrectly marked as a
sync sample in the MP4 metadata, which results in flushing the
re-ordering queue on every sample, so nothing gets re-ordered, so the
subtitles are garbled.

There are currently two "uses" for this call on every keyframe:
1. It offers a safety valve if we don't read a `maxNumReorderSamples`
value from the video. Without this, the queue will just keep growing
and end up swallowing all subtitle data (similar to the bug fixed by
39c734963f).

2. When we do read (or infer) a `maxNumReorderSamples` it means we can
emit samples from the queue slightly earlier - but this is pretty
marginal, given i think the max possible value for
`maxNumReorderSamples` is 16, so the most benefit we would get is 16
frames (~0.53s at 30fps) - in most cases we will have more than 0.5s
of buffer ahead of the playback position, so the subtitles will still
get shown at the right time with no problem.

(1) is resolved in this change by setting the queue size to zero (no
reordering) if we don't have a value for `maxNumReorderSamples`.

(2) has minimal impact, so we just accept it.

We may be able to inspect the NAL unit to determine IDR vs non-IDR
instead - we will consider this as a follow-up change, but given the
minimal impact of (2) we may not pursue this.

#cherrypick

PiperOrigin-RevId: 696583702
2024-11-14 11:05:02 -08:00
rohks
2379d0f18c Set container MIME type for audio-only extractors
PiperOrigin-RevId: 696560053
2024-11-14 10:03:17 -08:00
rohks
93b4c6ef47 Set container MIME type in Mp4Extractor and FragmentedMp4Extractor
PiperOrigin-RevId: 696513199
2024-11-14 07:23:22 -08:00
bachinger
6453054878 Parse interstitials into HLS media playlist
PiperOrigin-RevId: 696454575
2024-11-14 03:09:12 -08:00
dancho
16a15b94ca Add Configration for Frame Extraction for specific SeekParameters
Expose ExoPlayer seek parameters via FrameExtractor API

PiperOrigin-RevId: 696449874
2024-11-14 02:48:05 -08:00
dancho
301ef207f2 Add pixel comparison to FrameExtractorTest
Add PSNR comparison with the output of MedaiMetadataRetriever.

PiperOrigin-RevId: 696190585
2024-11-13 10:43:36 -08:00
Copybara-Service
74611bbdc0 Merge pull request #1265 from DolbyLaboratories:dlb/ac4-level4/dev_new2
PiperOrigin-RevId: 696157037
2024-11-13 09:08:22 -08:00
ibaker
0f96ad247d Fix 'narrowing conversion' warning in FragmentedMp4Extractor
PiperOrigin-RevId: 696119977
2024-11-13 07:08:12 -08:00
kimvde
9db66d6c6b Simplify bufferTimestampAdjustment handling
PiperOrigin-RevId: 696060496
2024-11-13 03:21:52 -08:00
kimvde
8393c2e445 Implement/document trivial methods in DefaultVideoSink
PiperOrigin-RevId: 696047835
2024-11-13 02:33:16 -08:00
dancho
175dca41df Read Bitmap in ExperimentalFrameExtractor
Add a MatrixTransformation GlEffect to flip between
OpenGL and Bitmap coordinates

PiperOrigin-RevId: 696029842
2024-11-13 01:27:02 -08:00
kimvde
64e92cb8e1 Various refactorings in VideoFrameRenderControl
- Rename a few methods/variable to improve readability.
- Refactor how video size changes are tracked. This will simplify the
  upcoming logic to refactor the timestamp logic in
  VideoFrameRenderControl because we will use the same logic for
  outputVideoSize and for outputStreamStartPositionUs.

PiperOrigin-RevId: 696026515
2024-11-13 01:15:58 -08:00
Googler
838c621d00 Add support to ApvC codec in Mp4Extractor
PiperOrigin-RevId: 695819965
2024-11-12 11:59:56 -08:00
sheenachhabra
59106bba1c Start muxer watchdog timer on export start
Earlier watchdog timer was started only after all the tracks are
added. For the cases where export is stuck before adding tracks,
the export would hang forever and will not timeout automatically.

With the changes, watchdog timer is started as soon as export is
started/resumed. Now if the pipeline is stuck before writing any samples,
it will timeout and report error to the caller.

Existing test case: c35a9d62ba/libraries/transformer/src/test/java/androidx/media3/transformer/MediaItemExportTest.java (L996)

PiperOrigin-RevId: 695781654
2024-11-12 10:19:13 -08:00
ibaker
19b38c83b6 Handle C.TIME_END_OF_SOURCE buffer timestamps in CeaDecoder
The behaviour was changed in 1.4.0 with 0f42dd4752,
so that the buffer timestamp is compared to `outputStartTimeUs` when
deciding whether to discard a "decode only" buffer before decoding
(instead of the deprecated/removed `isDecodeOnly` property). This breaks
when the buffer timestamp is `TIME_END_OF_SOURCE` (which is
`Long.MIN_VALUE`), because `TIME_END_OF_SOURCE < outputStartTimeUs` is
always true, so the end-of-stream buffer is never passed to the decoder
and on to `TextRenderer` where it is used to
[set `inputStreamEnded = true`](40f187e4b4/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/text/TextRenderer.java (L434-L436))
and so playback hangs.

#cherrypick

Issue: androidx/media#1863
PiperOrigin-RevId: 695767247
2024-11-12 09:38:44 -08:00
dancho
40f187e4b4 Block calls to FrameExtractor.release
Block calls to release() method, and handle
calls on the player looper or not.

PiperOrigin-RevId: 695697602
2024-11-12 05:49:19 -08:00
rohks
de0e08397e Add caching status APIs to MediaExtractorCompat
Implemented `getCachedDuration()` to provide an estimate of cached data in memory and `hasCacheReachedEndOfStream()` to indicate if the cache has reached the end of the stream.

Note: The Javadoc for the newly added methods closely follows that of the platform `MediaExtractor`. While the current implementation always uses a cache and therefore never returns `-1` from `getCachedDuration`, this leaves room for future changes should caching behavior or conditions evolve.
PiperOrigin-RevId: 695694460
2024-11-12 05:34:42 -08:00
dancho
ed288fca46 Experimental frame extraction based on ExoPlayer
A skeleton implementation that doesn't actually return decoded frames.
In the current state, we use ExoPlayer to seek to a position, and
ExoPlayer.setVideoEffects to record the presentation time selected.

Seeking and processing frames are synchronized via ListenableFuture
callbacks.

PiperOrigin-RevId: 695691183
2024-11-12 05:18:01 -08:00
Googler
4acd1b970c Make MediaSource.Factory a nullable parameter.
In some cases, callers will need to pass in both BitmapLoader and Codec.DecoderFactory without specifying a custom MediaSource.Factory. Omitting the annotation will result in NULL_FOR_NONNULL_TYPE compilation errors in Kotlin.

PiperOrigin-RevId: 695481606
2024-11-11 14:26:14 -08:00
claincly
01c784775e Rename signalEndOfInputStream()
This is because the method is called for each media item, the current name
implies that the method is only called once for all videos in a sequence.

PiperOrigin-RevId: 695332916
2024-11-11 07:44:15 -08:00
bachinger
fa790bd73c Fix supportedCommands in MediaMetadata
#cherrypick

PiperOrigin-RevId: 695304782
2024-11-11 06:07:02 -08:00
michaelkatz
0c982a7994 Fix error where non-enabled renderers were set to final
If the renderer has no stream, then it should not be called with `setCurrentStreamFinal` on it.

PiperOrigin-RevId: 695284225
2024-11-11 04:35:25 -08:00
ivanbuper
f84f44bf6b Implement static method to return estimated output samples
This method will replace the current estimation mechanism in place on
`getSpeedAdjustedTimeAsync()`, which relies on heuristics from previous
output rates and depends on the state of the `AudioProcessor`. The new
static method should be considerably more accurate.

PiperOrigin-RevId: 694607264
2024-11-08 12:54:47 -08:00
ibaker
53953dd377 Re-define 'max size' of SEI queue to operate on unique timestamps
This ensures it works correctly when there are multiple SEI messages per
sample and the max size is set from e.g. H.264's
`max_num_reorder_frames`.

PiperOrigin-RevId: 694526152
2024-11-08 09:20:57 -08:00
michaelkatz
b329806859 Fix unit test that confirms AAC_LC passthrough is unsupported
PiperOrigin-RevId: 694494792
2024-11-08 07:49:11 -08:00
tianyifeng
2b54b1ebbe Release internal components on preload thread in DefaultPreloadManager
The `RendererCapabilities` and `TrackSelector` objects are accessed on the preload thread during the preloading, when releasing them, they need to be released on the same thread. Otherwise, it is possible that they have already released on the application thread, while the PreloadMediaSource still tries to access them on the preload thread before the source is released.

#cherrypick

PiperOrigin-RevId: 694173131
2024-11-07 10:50:22 -08:00
rohks
91dfaab93f Add getDrmInitData() API to MediaExtractorCompat
Note: `androidx.media3.common.DrmInitData` is returned instead of the platform `android.media.DrmInitData` because the platform class has a package-private constructor, making it impossible to implement the abstract class outside the platform.
PiperOrigin-RevId: 694096542
2024-11-07 06:33:43 -08:00
shahddaghash
d164ce221a Change lib-common dependency scope to api in lib-ui
Changed the dependency of `lib-common` in `lib-ui` from an `implementation` dependency to an `api` dependency. This change ensures that classes and interfaces from `lib-common` used in the public API of `lib-ui` are correctly exposed to consumers of `lib-ui`.

For example, `PlayerView implements AdViewProvider` where the latter is from `androidx.media3.common.AdViewProvider`.

PiperOrigin-RevId: 693734349
2024-11-06 08:21:16 -08:00
rohks
3d51b36e99 Fix wrong class name in error message of MediaExtractorCompatTest
PiperOrigin-RevId: 693685232
2024-11-06 05:08:57 -08:00
dancho
31ae260db4 Support alternative sample MIME types in MuxerWrapper
Enable transformer to transmux into alternative sample MIME types.
For example, some Dolby Vision profiles have a backwards-compatible
AVC or HEVC layer. MV-HEVC is backwards compatible with HEVC.

This change enables Transformer to transmux into the backwards compatible
format to improve compatibility with legacy APIs such as
MediaMetadataRetriever.

PiperOrigin-RevId: 693667597
2024-11-06 03:58:02 -08:00
dancho
f9fd8badec Prevent repeated release of the same EGLContext
All instances of PlaybackVideoGraphWrapper use the same
VIDEO_FRAME_PROCESSOR_FACTORY_SUPPLIER which uses the same
DefaultGlObjectsProvider.

Each call to DefaultGlObjectsProvider.release() releases
all previously created EGLContexts.

Lazily create a new DefaultGlObjectsProvider for each
DefaultVideoFrameProcessor (not one per factory).

PiperOrigin-RevId: 693658458
2024-11-06 03:18:42 -08:00
ibaker
ab723fc8ff Mark every media3-cues sample as a keyframe
This format doesn't require any context from previous samples, so every
sample is a keyframe.

PiperOrigin-RevId: 693658370
2024-11-06 03:15:59 -08:00
tonihei
5336d71c22 Add TestPlayerRunHelper run(player).untilFullyBuffered
This simplifies some common test setup steps that rely on
a fully buffered player before making further test progress.

PiperOrigin-RevId: 693651493
2024-11-06 02:45:51 -08:00
Googler
08470140ac Internal
PiperOrigin-RevId: 693418471
2024-11-05 11:24:39 -08:00
tonihei
d38aba92fe Fix flakiness in HlsPlaybackTest and DashPlaybackTest
The tests became more flaky after 76e4abe428, likely because playback
was able to start slightly earlier, exaggerating any existing race
conditions. Fix the flakiness by letting all tests with subtitle
parsing wait until all data is fully loaded before starting to play.

PiperOrigin-RevId: 693380656
2024-11-05 09:42:53 -08:00
michaelkatz
28b75f7d29 Schedule doSomeWork after setVideoOutput
PiperOrigin-RevId: 693374521
2024-11-05 09:22:17 -08:00
ibaker
286273c10e Enable lint in tests for modules that require no fixes
PiperOrigin-RevId: 693313908
2024-11-05 05:40:03 -08:00
tonihei
12cb803486 Compat logic for MediaController.getCustomLayout
When a new media session sets media button preferences, we need to
"translate" them back to a custom layout to ensure the user preferences
are represented as closely as possible when the controller uses the
old button placement rules.

PiperOrigin-RevId: 693306153
2024-11-05 05:06:47 -08:00
tonihei
9fb4ed91b6 Make getCustomLayoutFromMediaButtonPreferences side-effect free
The method currently modifies the input Bundle, but it's easier to
reason about it if the method is side-effect free and the places
that need to modify a Bundle do this after calling the method.

PiperOrigin-RevId: 693288031
2024-11-05 03:54:40 -08:00
tonihei
76e4abe428 Reduce default values for DefaultLoadControl buffer for playback
PiperOrigin-RevId: 693284512
2024-11-05 03:39:16 -08:00
ibaker
6f81b5792b Add 'session extras' and MediaMetadata.durationMs to the stable API
PiperOrigin-RevId: 693279664
2024-11-05 03:17:06 -08:00
ibaker
212bd943f5 Fix lint violations in MediaExtractorCompatTest
I missed these as part of aebf822c3c

PiperOrigin-RevId: 692982120
2024-11-04 08:51:47 -08:00
rohks
278eaf47ad Add log session ID APIs to MediaExtractorCompat
Implemented `setLogSessionId(LogSessionId)` and `getLogSessionId()` methods.

Note: The `LogSessionId` is currently **not** forwarded to `MediaParser`, but this will be addressed once `MediaParser` can be used to configure `MediaExtractorCompat`.
PiperOrigin-RevId: 692945255
2024-11-04 06:38:38 -08:00
jbibik
0270267e08 Add ShuffleButtonState and RepeatButtonState to ui-compose
* Provide a helper Composable for remembering the state instance and launching the listening coroutine that observes the changes in the Player
* Add an example to demo-compose of using Shuffle- and Repeat- ButtonStates inside a Shuffle- and RepeatButton Composable.
* Reformat the MainActivity usage of `Shuffle` and `Repeat` buttons to form extra Player Controls and combine Prev/Play-Pause/Next with Shuffle/Repeat (Minimal controls + Extra controls = `PlayerControls`)

PiperOrigin-RevId: 692939825
2024-11-04 06:12:57 -08:00
rohks
261ca326c5 Move MediaExtractorCompatTest from test/ to androidTest/
The test has been moved to an instrumentation test as it relies on APIs that vary by SDK version. Robolectric’s emulation lacks sufficient realism in some cases, which impacts test accuracy. By using an instrumentation test, we ensure that the tests run in a real Android environment, providing reliable results for SDK-dependent APIs.

PiperOrigin-RevId: 692933259
2024-11-04 05:42:45 -08:00
ibaker
aebf822c3c Enable lint errors in exoplayer tests
Follow-up to 76db936d68

PiperOrigin-RevId: 692925617
2024-11-04 05:08:34 -08:00
sheenachhabra
c3e72a87e5 Make error messages unique in Boxes.java
PiperOrigin-RevId: 692920946
2024-11-04 04:49:27 -08:00
ibaker
af1c13524c Enable lint in lib-common tests
Also move the `lint.xml` config which disables the `NewApi` check from
`lib-session` to the existing top-level file, and limit it to cover all
Robolectric tests by path matching.

Follow-up to 76db936d68

PiperOrigin-RevId: 692913646
2024-11-04 04:16:39 -08:00
dancho
bd90ef38b0 MCVR.hasSurfaceForCodec considers VideoSink
This change keeps getSurfaceForCodec and hasSurfaceForCodec in sync.
Before this change when ExoPlayer is configured with setVideoEffects
and no display surface, codecs that need Set Output Surface Workaround
would not be initialized because hasSurfaceForCodec always returns false

PiperOrigin-RevId: 692900899
2024-11-04 03:17:54 -08:00
Copybara-Service
aebda58314 Merge pull request #1829 from colinkho:ssms-custom-exec-2
PiperOrigin-RevId: 692897382
2024-11-04 03:02:46 -08:00
rohks
d8ad18383d Optimize sample metadata handling in MediaExtractorCompat
Restructured sample metadata management in `MediaExtractorCompat` to maintain a queue of `SampleMetadata` objects, each containing `timeUs`, `flags`, `size`, and `trackIndex`, rather than storing only track indices.

Introduced a pooling mechanism for `SampleMetadata` to reduce object allocation and improve memory efficiency. The new `SampleMetaDataQueue` centralizes metadata for easier access and management, eliminating the need to allocate a buffer or peek into the `SampleQueue` to retrieve sample metadata.

This change does not change existing behavior, it optimizes the internal structure, and existing tests already verify the relevant APIs.

PiperOrigin-RevId: 692285581
2024-11-01 14:07:11 -07:00
ivanbuper
7edbaa3f2c Use sample-aligned speed changes in SpeedChangingAudioProcessor
This CL adds utility methods to obtain sample-aligned timestamps from a
`SpeedProvider` to solve in a unified way the issues arising from
conversions between sample positions and microseconds.

The new utility methods will also help with the implementation of a
static method like `getDurationAfterProcessorApplied()` that will remove
the need for thread synchronization between the video and audio pipeline
for speed changing effects.

PiperOrigin-RevId: 692233318
2024-11-01 11:09:18 -07:00
ivanbuper
0b1695124b Bump Media3 to 1.5.0-rc01
#cherrypick

PiperOrigin-RevId: 692221696
2024-11-01 10:30:45 -07:00
tonihei
06718c5df3 Fix position tracking bug for inaccurate audio processors
If audio processors report a drifting position, we currently update
the media position parameters to correct this drift. However, this
means we pass in the wrong value to
audioProcessorChain.getMediaDuration, which reuqires the time since
the last flush.

To fix this problem, we can instead save the drift seperately and
apply it where needed.

PiperOrigin-RevId: 692202219
2024-11-01 09:28:15 -07:00
claincly
b0c6106882 Fix trim optimization logic when edit lists exist
Also makes muxer shift the first video timestamp to zero, if it's not.

The trim position should respect the media timeline.

For example in a video that is 10s long (without edit list), if an edit list
adds 1_000ms to each video sample, and trimming 100ms, here's the expected:

- The video duration is 10.9s (`10s + 1s edit - 0.1s trim`)
- The first video frame time would be at 0.9s (`1s edit - 0.1s trim`)

PiperOrigin-RevId: 692187399
2024-11-01 08:36:08 -07:00
Copybara-Service
4910b2cdc0 Merge pull request #1225 from Kekelic:support-for-parsing-rtsp-packets-with-header-extension
PiperOrigin-RevId: 692156233
2024-11-01 06:35:16 -07:00
tonihei
544d7aa2dc Annotate parameters in RepeatModeUtil
PiperOrigin-RevId: 692129684
2024-11-01 04:33:31 -07:00
jbibik
1b302e879a Add PreviousButtonState and NextButtonState to ui-compose
* Provide a helper Composable for remembering the state instance and launching the listening coroutine that observes the changes in the Player
* Add an example to demo-compose of using Previous- and Next- ButtonStates inside a Previous- and NextButton Composable.
* Reformat the MainActivity usage of `Previous`, `PlayPause`, `Next` buttons to form Minimal Player Controls

PiperOrigin-RevId: 691943147
2024-10-31 14:52:38 -07:00
jbibik
676a3872a5 Add PlayPauseButtonState to ui-compose
Provide a helper Composable for remembering the state instance and launching the listening coroutine that observes the changes in the Player

Add an example to demo-compose of using PlayPauseButtonState inside a PlayPauseButton Composable.

The smart State object has been deemed a preferred solution over collecting Flows due to queuing/timing/buffering limitations. Instead, it uses the new `Player.listen` suspending extension function to catch the relevant events.

PiperOrigin-RevId: 691879975
2024-10-31 11:42:14 -07:00
microkatz
0e020f778c Added release note 2024-10-31 12:42:03 +00:00
microkatz
71bc177e39 Format with google-java-format 2024-10-31 11:57:47 +00:00
Kekelic
47f4dabb98 Fix documentation and improve readability of unit test with rtpDataWithHeaderExtension example 2024-10-31 11:57:47 +00:00
Kekelic
84f8beb884 Remove saving unnecessary header extension data 2024-10-31 11:57:47 +00:00
Kekelic
5a8a250a1a Add reading length of extension payload from header extension in RtpPacket 2024-10-31 11:57:47 +00:00
Kekelic
1ecb7c3fbf Format changes 2024-10-31 11:57:46 +00:00
Kekelic
7dd428534e Change RtpPacket class to parse header extension if exists 2024-10-31 11:57:46 +00:00
ibaker
b5db8a6cbe Fix empty LoadEventInfo.uri passed to onLoadStarted
This affects both `AnalyticsListener` and `MediaSourceEventListener`

This was introduced by d051b4b993

Also fix a missing 'load started' event for HLS media playlists (this
was also introduced by d051b4b993).

PiperOrigin-RevId: 691580183
2024-10-30 15:52:48 -07:00
jbibik
6cbf77b3f0 Refactor shouldEnablePlayPauseButton out of PlayerControlView into Util
PiperOrigin-RevId: 691502799
2024-10-30 11:59:25 -07:00
rohks
2b27e33784 Refactor OpusDecoderTest to use OpusLibrary.isAvailable()
Replaced the custom `LibraryLoader` instance with `OpusLibrary.isAvailable()` to verify the library loading. This simplifies the code by leveraging the existing library loading mechanism.

#cherrypick

PiperOrigin-RevId: 691457871
2024-10-30 10:02:46 -07:00
ivanbuper
2a9963424b Refactor readBytes to readFrames in SpeedChangingAudioProcessor
Using frames instead of bytes helps simplify the processing logic for
the component and will help with the move towards sample-based
`SpeedProvider`s. By using frames, we also abstract away the complexity
related to sample encoding.

This is a non-functional refactor.

PiperOrigin-RevId: 691444106
2024-10-30 09:21:54 -07:00
rohks
15583f7c64 Set track duration in AmrExtractor, Mp3Extractor and OggExtractor
PiperOrigin-RevId: 691433024
2024-10-30 08:46:45 -07:00
ibaker
0462349902 Deprecate NalUnitUtil.isNalUnitSei(String, byte)
The overload that takes a `Format` is preferred, because it can detect
SEI NAL units in Dolby Vision tracks too.

Submitting this in a separate change so we can avoid cherry-picking it
into `1.5.0-rc01`, otherwise it would naturally be part of
27371db225.

Issue: androidx/media#1820
PiperOrigin-RevId: 691408725
2024-10-30 07:21:26 -07:00
rohks
df07fa35d8 Populate MediaFormat.KEY_DURATION in MediaExtractorCompat
Previously, `getTrackFormat()` in `MediaExtractorCompat` returned a `MediaFormat` without setting `MediaFormat.KEY_DURATION`. With this change:

- `MediaFormat.KEY_DURATION` is set based on the track's duration, if available.
- If the track duration is unset, the duration from the seek map is used as a fallback.
- When neither duration is set, `MediaFormat.KEY_DURATION` remains unset.

This ensures that `MediaFormat.KEY_DURATION` is populated when possible, enhancing duration information availability.

PiperOrigin-RevId: 691395114
2024-10-30 06:29:47 -07:00
rohks
27de9f02e0 Add missing DefaultRenderersFactoryTest for decoder extensions
Added `DefaultRenderersFactoryTest` for `IAMF`, `AV1`, and `MIDI` decoder extensions.

PiperOrigin-RevId: 691381816
2024-10-30 05:33:38 -07:00
ibaker
27371db225 Support CEA-608 subtitles in Dolby Vision
Issue: androidx/media#1820

#cherrypick

PiperOrigin-RevId: 691378476
2024-10-30 05:18:14 -07:00
ibaker
08a141328d Mark ProgressiveMediaSource.setSuppressPrepareError package-private
This method will likely be removed in the next release, and is currently
only needed from within the `source` package.

#cherrypick

PiperOrigin-RevId: 691351449
2024-10-30 03:19:14 -07:00
rohks
129cf8ea72 Use assumeTrue for libiamf availability check in IamfDecoderTest
This change ensures that the test uses `assumeTrue` to avoid failures when the `libiamf` library is not pre-built.

#cherrypick

PiperOrigin-RevId: 691333564
2024-10-30 02:01:05 -07:00
kimvde
96b923b610 Fix failing portrait export test
The logic in assumeFormatsSupported assumes that portrait videos are
always rotated before encoding but that's not the case when portrait
encoding is enabled.

PiperOrigin-RevId: 691042881
2024-10-29 09:10:59 -07:00
kimvde
14094b5094 Fix offset passed to VideoFrameReleaseControl
In some cases, the streamOffsetUs was passed to
VideoFrameReleaseControl.getFrameReleaseAction() but it should be the
streamStartPositionUs.

PiperOrigin-RevId: 691040172
2024-10-29 09:04:35 -07:00
shahddaghash
7f94aaf49f Remove deprecated SingleFrameGlShaderProgram class.
Class was deprecated in Media3 1.2.0. Use BaseGlShaderProgram class instead.

PiperOrigin-RevId: 691001562
2024-10-29 06:53:45 -07:00
tonihei
7c0cffdca8 Improve position estimate when transitioning to another checkpoint
When transitioning to the next media position parameter checkpoint
we estimate the position because the audio processor chain no longer
provides access to the actual playout duration.

The estimate using the declared speed and the last checkpoint may
have drifted over time, so we currently estimate relative to the
next checkpoint, which is closer and presumably provides a better
estimate. However, this assumes that these checkpoint are perfectly
aligned without any position jumps.

The current approach has two issues:
 - The next checkpoint may include a position jump by design, e.g.
   if it was set for a new item in the playlist and the duration of
   the current item wasn't perfectly accurate.
 - The sudden switch between two estimation methods may cause a jump
   in the output position, which is visible when we add new media
   position checkpoints to the queue, not when we actually reach the
   playback position of the checkpoint.

We can fix both issues by taking a slightly different approach:
 - Continuously monitor the estimate using the current checkpoint. If
   it starts drifting, we can adjust it directly. This way the estimate
   is always aligned with the actual position.
 - The change above means we can safely switch to using the estimate
   based on the previous checkpoint. This way we don't have to make
   assumptions about the next checkpoint and any position jumps will
   only happen when we actually reach this checkpoint (which is more
   what a user expects to see, e.g. at a playlist item transition).

Issue: androidx/media#1698
PiperOrigin-RevId: 690979859
2024-10-29 05:31:25 -07:00
shahddaghash
7a8f05f736 Remove deprecated Transformer methods/variables
Removed deprecated `Transformer.PROGRESS_STATE_NO_TRANSFORMATION`, `Transformer.setListener`, and `Transformer.startTransformation` from Transformer.

PiperOrigin-RevId: 690971992
2024-10-29 05:02:31 -07:00
rohks
b1f2efd218 Fix handling of cues that exceed total duration in MatroskaExtractor
Adjusted logic to accurately calculate sizes and durations for the last valid cue point when cue timestamps are greater than the total duration.

Fixes the issue where the reported duration of the MKV file was greater than the total duration specified by the duration element. Verified this using `mkvinfo` and `mediainfo` tools.

PiperOrigin-RevId: 690961276
2024-10-29 04:18:38 -07:00
shahddaghash
51055d7e79 Remove deprecated TransformationException class.
Class was deprecated in Media3 1.1.0. Use `ExportException` class instead.

PiperOrigin-RevId: 690955853
2024-10-29 03:59:15 -07:00
dancho
b0df3b2da3 DecodeOneFrameUtil: do not access release players
A player that is being released may report an error with
null `player.getPlayerError()`. Do not try to read errors
of players that are released.

PiperOrigin-RevId: 690953083
2024-10-29 03:48:29 -07:00
shahddaghash
dd6e88889a Remove deprecated TransformationResult class.
Class was deprecated in Media3 1.1.0. Use `ExportResult` class instead.

PiperOrigin-RevId: 690694730
2024-10-28 11:51:02 -07:00
ivanbuper
7e7764de5e Implement getExpectedFrameCountAfterProcessorApplied() in Sonic
This method allows `Sonic` to statically and accurately report the
expected number of output frames for any given parameter configuration.

This change is required prework for `SpeedChangingAudioProcessor` to
implement a similar static method and allow precise, non-blocking
timestamp adjustments for the experimental speed changing effect.

PiperOrigin-RevId: 690669627
2024-10-28 10:43:03 -07:00
shahddaghash
772bd20f7d Remove deprecated Transformer.Listener methods
Methods were deprecated between Media3 1.0.0 and Media3 1.1.0.

PiperOrigin-RevId: 690656410
2024-10-28 10:09:20 -07:00