This allows seek operations in files that were previously unseekable, particularly those with variable bitrate (VBR) or constant bitrate (CBR) with silence frames.
New samples for tests were created by adding silence frames to existing narrowband and wideband versions.
PiperOrigin-RevId: 665349241
The CL adds another way of writing editable video
tracks where the samples will be interleaved with the
primary track samples in the "mdat" box.
PiperOrigin-RevId: 665313751
This uses a 'bundled' extractor (`SubtitleExtractor`) because
`MediaParser` doesn't support transcoding subtitles to
`application/x-media3-cues`. This transcoding is required to support
non-legacy subtitle handling where they are parsed during extraction,
instead of during rendering.
PiperOrigin-RevId: 665282298
In order to support building with `gradle` a new `build.gradle` file was added together with `Android.mk`, `Application.mk`, and `libiamf.mk` necessary for local builds with NDK.
After this change, IAMF files may also be played in the ExoPlayer demo built locally (without blaze).
PiperOrigin-RevId: 664841684
This is only used as part of legacy subtitle decoding. If any apps are
using it directly (rather than via `DashMediaSource` or `SsMediaSource`)
they probably need to enable legacy decoding in `TextRenderer` to avoid
playback failures.
PiperOrigin-RevId: 663740343
The method currently has two ways of operating with a complicated
contract:
1. outputBuffer == null -> sets a new outputBuffer and starts
draining it as far as possible.
2. outputBuffer != null -> continues draining a previous buffer,
requires that the input parameter is the same as outputBuffer.
This contract can be made cleaner by splitting the logic into
(1) setting a new output buffer and (2) draining the output buffer.
Only one parameter is needed for each method and there is no
requirement to pass in the same argument as before because we already
know what we are draining. This also means the caller of the method
doesn't have to keep track of the actual outputBuffer, allowing
further processing steps in the future that change the drained
instance.
PiperOrigin-RevId: 663316251
This makes it clearer that the part inside and outside the parentheses
are clearly different, and not "just" a reference to `Player.Events`.
Specifically this syntax is showing that the function has a `Player`
"receiver", **and** is a function from `Player.Events` to `Unit`.
https://kotlinlang.org/docs/lambdas.html#function-types
PiperOrigin-RevId: 663293405
This makes the class available to custom MP4-parsing implementations,
while also allowing it to be used by `muxer` in future.
'Box' is the term used throughout the ISO 14496-12 spec, while the
'Atom' nomenclature was used in an earlier form of the spec
(Quicktime).
This change moves it from `extractor.mp4.Atom` to `container.Mp4Box`,
to be consistent with existing MP4-specific types in the `container`
module like `Mp4TimestampData`.
PiperOrigin-RevId: 663274752
The audio mixing is not deterministic on real device testing because of
threading, but runs deterministically on robolectric tests
PiperOrigin-RevId: 662925912
Use ExoPlayer dynamic scheduling to reduce the render() interval for
older API devices where `DefaultCodec.getMaxPendingFrameCount()` is set
to 1 in order to prevent frame drops.
Controlled via API on DefaultDecoderFactory.
Add TransformerForegroundSpeedTest that mimics transcoding while the app
is in foreground.
PiperOrigin-RevId: 662925764
The parameter class will allow addition of more
parameters (link shouldInterleaveSamples), which are specific to
editable video file format.
PiperOrigin-RevId: 662923844
AMR samples with identical data but different names, previously used to generate uniquely named dump files, have been deleted. Instead, `AssertionConfig` is now used to set the dump file prefix, ensuring files are generated with unique names.
PiperOrigin-RevId: 662883541
Previous to this change, `FrameworkMediaDrm.requiresSecureDecoder`
ignores its `sessionId` parameter on API 31+, and uses only the
`mimeType` parameter. This means the result [assumes the session is
opened at the 'default security
level'](https://developer.android.com/reference/android/media/MediaDrm#requiresSecureDecoder(java.lang.String)):
> The default security level is defined as the highest security level
> supported on the device.
This change is a no-op in all (?) cases, because the `ExoMediaDrm`
interface only exposes the zero-arg `openSession()` method, which in the
framework case **also** assumes the highest security level is preferred:
> By default, sessions are opened at the native security level of the
> device.
However, it seems more obviously correct to only make this
"highest/native security level" assumption in one place
(`openSession()`), and check the session's **actual** security level
everywhere else.
Issue: androidx/media#1603
PiperOrigin-RevId: 662872860
Constant bit rate (CBR) seeking can be enabled even when the length of the file is not known.
Additionally, dump files for these files have been updated to accurately log the `position` when `timeUs` is set to `0`.
PiperOrigin-RevId: 662868607
This method was added in API 31 (S) but it's non-functional
(incorrectly, silently, returns `false`) on the Widevine plugin version
(`16.0`) from R (API 30), which some devices up to at least API 34 are
still using.
This results in ExoPlayer incorrectly selecting an insecure decoder for
L1 secure content, and subsequently calling
`MediaCodec.queueInputBuffer` instead of `queueSecureInputBuffer`,
which is not supported and generates the following error:
> Operation not supported in this configuration: ERROR_DRM_CANNOT_HANDLE
Issue: androidx/media#1603
#cherrypick
PiperOrigin-RevId: 662852176
The TrackToken is primarily for public API.
Using Track object internally will remove unnecessary
type casting at various places.
PiperOrigin-RevId: 662564224
When there is an exception thrown from the `LoadTask`, the `Loader` will call `Loader.Callback.onLoadError`. Some implementations of `onLoadError` method may call `MediaPeriod.onContinueLoadingRequested`, and in the `PreloadMediaSource`, its `PreloadMediaPeriodCallback` will be triggered and then it can further call `continueLoading` if it finds needed. However the above process is currently done synchronously, which will cause problem. By calling `continueLoading`, the `Loader` is set with a `currentTask`, and when that long sync logic in `Loader.Callback.onLoadError` ends, the `Loader` will immediately retry, and then a non-null `currentTask` will cause the `IllegalStateException`.
Issue: androidx/media#1568
#cherrypick
PiperOrigin-RevId: 662550622
Moving this field to `IamfDecoder` instead of `iamf_jni` allows multiple instances of the IAMF decoder with possibly different configurations at the same time.
PiperOrigin-RevId: 662548068
Check if the output device supports spatialization for the requested output format. If so, return a stream decoded for 6 channels in a 5.1 layout. Otherwise, return a stream decoded for 2 channels in a binaural layout.
PiperOrigin-RevId: 662546818
This is groundwork to moving `Atom` to the `container` library, which
we want to do before making it public (so it can be used by `muxer` in
future).
PiperOrigin-RevId: 662453520
After this change, a WakeLock of PowerManager#PARTIAL_WAKE_LOCK level would be acquired when the media is paused due to playback attempt without suitable output.
This WakeLock will be release either when the suitable media output has been connected or the set timeout to do so has expired.
PiperOrigin-RevId: 661570346
This is an additional signal that legacy subtitle support needs to be
explicitly enabled, and is going away at some point.
PiperOrigin-RevId: 661305694
If the length of the `ExtractorInput` is not known then the
`subtitleData` field is re-sized by 1kB each time
(`SubtitleExtractor.DEFAULT_BUFFER_SIZE`), so the end of the array is
often not populated. This change ensures that `length` is propagated to
`SubtitleParser`, so that implementations don't try and parse the
garbage/zero bytes at the end of the array.
Discovered while investigating Issue: androidx/media#1516
#cherrypick
PiperOrigin-RevId: 661195634
Some RTSP servers may provide media descriptions for custom streams that are not supported. ExoPlayer should skip the invalid media description and continues parsing the following media descriptions.
To start, ExoPlayer will still error on malformed SDP lines for media descriptions, but will now skip media descriptions with "non-parsable" formats as described by [RFC 8866 Section 5.14](https://datatracker.ietf.org/doc/html/rfc8866#section-5.14).
Issue: androidx/media#1472
PiperOrigin-RevId: 660826116
App users can choose arbitrary data that might not be
anticipated by developers. Transformer shouldn't `checkState` based on
media data or file type -- report an error for unsupported data instead.
Public API change `ImageAssetLoader` needs to parse MIME type and now accepts
`Context` as parameter.
PiperOrigin-RevId: 660762459
This is caused when the requested "output start time" is equal to or
larger than the last event time in a `Subtitle` object.
This resolves the error in Issue: androidx/media#1516, but subtitles are still not
renderered (probably because the timestamps aren't what we expect
somewhere, but I need to investigate this part further).
#cherrypick
PiperOrigin-RevId: 660462720
Previously in MultiInputVideoGraph, each VFP would destroy the shared
eglContext, such that the same eglContext object is destroyed multiple times.
Adding a flag to disallow this.
The alternative being we could add a flag on the VFP constructor, but I think
that is too subscriptive (meaning if we later might want to add another boolean
to control another GL behaviour, multiple booleans would make the class less
reason-able), and would incur a lot of code changes at places.
PiperOrigin-RevId: 660354367
The method is not supposed to work with any input byte[]
so its best to make it non static and add appropriate validations.
PiperOrigin-RevId: 659906543
The integration with external libraries like Glide or Coil is
currently extremely complicated. Providing the boilerplate code
to handle the ImageDecoder interface and a better hook for custom
image decoders in DefaultRenderersFactory allows apps to easily
inject their own logic for external image loading libraries.
PiperOrigin-RevId: 659508914
There are two ways to write editable tracks samples.
1. In the embedded edit data MP4.
2. Interleaved with primary tracks samples.
Initial plan was to support only option 1 but then the
decision is to support both ways. To identify between these two
an additional key will be required.
Option 2 is yet to be implemented in Mp4Muxer.
PiperOrigin-RevId: 658791214
This API allows users to retrieve the sample size, ensuring they can allocate sufficient buffer capacity before utilizing the `readSampleData(ByteBuffer buffer, int offset)` method to read data.
PiperOrigin-RevId: 658772408
Instead of hard-coding values in multiple files, all default values are declared in `IamfDecoder`. Additionally, the max number of frames used for output buffer initialisation is fetched from `libiamf` native functions.
PiperOrigin-RevId: 658772175
For now, the only extension function on the `Player` has been used in a Compose demo. It can be promoted to a proper module where in the future other extension functions will reside. Given that `Player` is in `androidx.media3.common`, the corresponding KTX library for it is `androidx.media3.common-ktx`
To start using the new `suspend fun listen`, one must add `androidx.media3:media3-common-ktx` as a Gradle dependency and `import androidx.media3.common.listen`
PiperOrigin-RevId: 658771029
This is needed to correctly handle files with trailing non-MP3 data
(which is indicated by the length in the `Info` frame being shorter than
the overall length of the file).
The test file was generated by appending 150kB of `DEADBEEF` onto the
end of `test-cbr-info-header.mp3`, and the test asserts that the
extracted samples are identical.
Issue: androidx/media#1480
#cherrypick
PiperOrigin-RevId: 658727595
It will be used for Kotlin-specific functionality like extension functions on the classes from the `media3-common` module. To import it, add the following to your build.gradle file:
`implementation("androidx.media3:media3-common-ktx:1.X.Y")`
PiperOrigin-RevId: 658492256
The data source may behave differently, depending on the provider,
so we can extend the contract test to check all available providers.
PiperOrigin-RevId: 658457650
Package-private until API is more useable.
Similar to frame analyzer mode: uses ImageReader instead of an encoder,
and no muxer.
PiperOrigin-RevId: 658446675
Transformer.experimentalSetMaxFramesInEncoder controls max number
of frames in encoder.
VideoFrameProcessor now allows delayed releasing of frames to Surface,
while still using the original presentation time.
VideoSampleExporter can now configure video graphs to not render frames
automatically to output surface.
VideoSampleExporter.VideoGraphWrapper tracks how many frames are ready
to be rendered to Surface, and how many frames are already in-use by encoder.
PiperOrigin-RevId: 658429969
Add parameterized test for codecs supported by InAppMuxer.
Split TransformerWithInAppMuxerEndToEndParameterizedTest tests
between parameterized and non parameterized
This will be helpful for adding more tests which need not to
be parameterized.
PiperOrigin-RevId: 658353532
Refactor to replace instances of `RuntimeException` with `IllegalStateException`.
This change ensures exceptions can be handled more specifically without
unintended catches of unrelated exceptions.
PiperOrigin-RevId: 658337459
For some predefined keys the type of value is already defined.
Early validation will help avoiding error when processing this data later.
PiperOrigin-RevId: 658060844
Added a new data source which acts an adapter to read media data from platform `MediaDataSource`. This enables adding the `setDataSource(MediaDataSource)` API to `MediaExtractorCompat`.
PiperOrigin-RevId: 657564901
Added three `setDataSource` APIs in `MediaExtractorCompat`:
- `setDataSource(Context context, Uri uri, @Nullable Map<String, String> headers)` to set data source with a content URI and optional headers.
- `setDataSource(String path)` to set data source using a file path or HTTP URL.
- `setDataSource(String path, @Nullable Map<String, String> headers)` to set data source using a file path or HTTP URL with optional headers.
PiperOrigin-RevId: 657563973
- Modified the logic of `open()` and `read()` methods to handle scenarios where length is unset for the `FileDescriptor` provided.
- Added unit test and contract test to handle this case.
Also used `getDeclaredLength()` instead of `getLength()` to set the length of `AssetFileDescriptor` in unit tests and contract tests.
PiperOrigin-RevId: 657551343
These tests addresses two identified gaps in the contract:
- Ensures that the output buffer offset passed to the `DataSource.read` method is correctly applied.
- Verifies that the position within the input stream is properly incremented when reading in two parts.
PiperOrigin-RevId: 656358935
Public methods either assert they're running GL thread, or
submit a task to run on GL thread.
Move methods to keep interface implementations together.
Add javadoc to VideoFrameProcessingTaskExecutor to clarify which
thread can call each public method.
PiperOrigin-RevId: 655978796
The builtin speaker is to be supported as a suitable output when that is deliberately selected for the media playback by the user in Wear OS.
PiperOrigin-RevId: 655950824
The DefaultPreloadManagerTest didn't to catch this because we use main looper as the preload looper in the tests. This CL also improves the tests by assigning the preload looper with one that corresponds to a different thread.
PiperOrigin-RevId: 655664189
The dependency on the native `opusV2JNI` library doesn't work from
Robolectric, so the `assumeTrue` statements in this test always fail,
and the tests are always skipped. Moving it to an instrumentation test
allows the native library to be successfully loaded, and the test to be
run.
PiperOrigin-RevId: 655570129
This is following a renaming of registerInputFrame to handleInputFrame.
- queueInputBitmap is renamed to handleInputBitmap for consistency with
handleInputFrame.
- registerInputStream is renamed to onInputStreamChanged for consistency
with media3 method names.
PiperOrigin-RevId: 655529699
Build upon Transformer.videoFrameProcessorFactory in MultipleInputVideoGraph
Check that Transformer.videoFrameProcessorFactory is DefaultVideoFrameProcessor
for multi-input video
Fixes https://github.com/androidx/media/issues/1509
PiperOrigin-RevId: 655232381
Aligns `MediaExtractorCompat` with platform behavior by using `getDeclaredLength()`
instead of `getLength()` when setting the data source. This ensures compatibility
with asset files where the length is not predefined, even though it may result
in reading across multiple logical files when backed by the same physical file.
PiperOrigin-RevId: 655153390
eglDestroySurface only destroys the surface when it's made not current.
Pass the placeholder surface in FinalShaderProgramWrapper and use it
when destroying eglSurface.
PiperOrigin-RevId: 655139661
Add VideoFrameProcessingTaskExecutor.invoke() method that blocks until
Task has executed on GL thread.
Use that for FinalShaderProgramWrapper.setOutputSurfaceInfo
PiperOrigin-RevId: 655119768
AudioTrack automatically ramps down volume when pausing. However,
when this happens as part of a AudioSink.flush() operation, we
need to postpone the actual flush() until the ramp down finished
in the audio system. Otherwise audio is just cut off, creating pop
sounds.
Delaying the release is fine now, because DefaultAudioSink starts
creating a new track immediately without waiting for the previous
track to be released.
Also using the opportunity to add more comments about related quirks
of the AudioTrack flush/release handling for more context.
PiperOrigin-RevId: 654794818
This was used in media1 `MediaItemDescription` to indicate the download
status of a media item. When connected to a legacy
`MediaBrowserServiceCompat` the Media3 browsers converts the legacy
media item to a Media3 `MediaItem` and converts the extras of
`MediaDescriptionCompat.extras` to `MediaMetadata.extras`.
#cherrypick
PiperOrigin-RevId: 654625502
We currently wait until a previous AudioTrack from the same
DefaultAudioSink is released on a background thread before attempting
to initialize a new AudioTrack. This is done to avoid issues where
the releasing track blocks some shared audio memory, preventing a new
track from being created.
The current solution has two main shortcomings:
- In most cases, the system can easily handle multiple AudioTracks
and waiting for the release just causes unnecessary delays (e.g.
when seeking).
- It only waits for a previous track from the same DefaultAudioSink,
not accounting for any other tracks that may be in the process of
being released from other players.
To improve on both shortcomings, we can
(1) move the check for "is releasing tracks and thus may block shared
memory" to the static release infrastructure to be shared across all
player instances.
(2) optimistically create a new AudioTrack immediately without waiting
for the previous one to be fully released.
(3) extend the existing retry logic that already retries failed
attempts for 100ms to only start the timer when ongoing releases are
done. This ensures we really waited until we have all shared resources
we can get before giving up completely. This also acts as a replacement
for change (2) to handle situations where creating a second track is
genuinely not possible. Also increase threshold to 200ms as the new
unit test is falky on a real device with just 100ms (highlighting that
the device needed more than 100ms to clean up internal resources).
PiperOrigin-RevId: 654053123
All methods check if the player is currently handling the ad source
by calling isCurrentAdPlaying(). This method was missing a check
for empty timelines that throws an exception when trying to access
a non-existent period.
Also add this check to two methods that assume the current item
is the ads source, but didn't check it yet.
PiperOrigin-RevId: 653963557
Introduced three `setDataSource` APIs in `MediaExtractorCompat`, enabling the use of `AssetFileDescriptor` and `FileDescriptor` to set the data source.
PiperOrigin-RevId: 653957035
This controller connects the audio output to the MediaCodec so
that it can automatically propagate CTA-2075 loudness metadata.
PiperOrigin-RevId: 653628503
Creating a moov box is same as creating any other box so
there is no particular need to have a separate class for this.
In a follow up CL the method will be moved into Boxes.java along with
other box creation methods.
Made nit changes in the final fields ordering to match with
constructor parameter ordering.
PiperOrigin-RevId: 653602558
This test is flaky at p4head, because CompositionPlayer has no logic to
ensure a specific input is used to configure the audio graph. Depending
on which sequence registers input first, it changes the output format
of the media.
By setting effects on the 2nd sequence, both inputs are the same format
and this flakiness is avoided in the test.
PiperOrigin-RevId: 653582441
This replaces the existing PlaceholderSurface mode with a more
efficient solution that doesn't require a GL texture or a new
thread.
PiperOrigin-RevId: 653537596
Boxes.moov() simply wraps the subboxes and this logic can be
put into main method which has all the logic of creating moov box.
This is to eventually move Mp4MoovStucture.moov() into
Boxes.java where all other box creation methods are already present.
PiperOrigin-RevId: 653319292
-----
Context:
* Each sequence is wrapped as a single MediaSource, each being played
by an underlying ExoPlayer.
* Repeat mode is typically implemented in Players as a seek to the next
item in the playlist.
-----
This CL:
Repeat mode is triggered by listening for when the main input player
sees a play when ready change due to the end of the media item.
There is a slight delay at the end of the playback, before it repeats.
Setting repeat mode on the underlying players addresses this, but means
that the players will seek without waiting for the CompositionPlayer,
and as such previewAudioPipeline does not get the correct signals
around blocking/flushing.
PreviewAudioPipeline - The seek position can validly be C.TIME_UNSET,
however preview pipeline did not handle this case.
CompositionPlayer getContentPosition is given (through a lambda) as a
supplier to the State object, which means any comparisons between
previous/new state for this value does not work. In SimpleBasePlayer,
there is logic to use the positionDiscontinuityPositionUs for the
position change (see getPositionInfo called from
updateStateAndInformListeners), however this logic is not considered in
getMediaItemTransitionReason, so a condition needed to be added for
this case.
-----
Tests:
* Dump files clearly show the position and data is repeated.
* Assertions on the reasons for transitions or position
discontinuities.
PiperOrigin-RevId: 653210278
Also resolve some failures.
Lint checks [aren't enabled in tests by default](http://groups.google.com/g/lint-dev/c/rtjVpqHmY0Y).
This change suppresses `NewApi` failures in Robolectric tests because
these tests only run at 'target SDK' by default (currently 30), but the
lint doesn't understand this and so flags spurious issues with API
usages below this.
PiperOrigin-RevId: 653172059
This is a regression test for the bug introduced in bb9ff30c3a
which was manually spotted and fixed in 0d2bf49d6a.
Reverting the fix causes this test to fail.
This test is a bit hacky because we have to munge the stack trace of
the `IllegalStateException` to make it look like it was thrown from
inside `MediaCodec`. We deliberately do this 'badly' (e.g. using
`fakeMethod`) to avoid a future reader being confused by a
fake-but-plausible stack trace.
PiperOrigin-RevId: 652820878
Now the value is guaranteed to be zero (see bb9ff30c3a), we can
remove the rotation handling for it in the UI module. We can also
enforce the documentation more clearly by not even setting the
value to anything other than zero.
PiperOrigin-RevId: 652772091
Added a method to set the timeout for the SNTP request.
Also changed the default timeout to 5s instead of 10s as it seemed quite high.
Issue: androidx/media#1540
PiperOrigin-RevId: 652566008
The change in bb9ff30c3a removed the detection util completely,
but even on API21+, the codec exceptions can be thrown as
IllegalStateException and we should reinstate this check to
correctly classify the exceptions.
PiperOrigin-RevId: 652557091
When removing one listener from the `ListenerSet`, we release the corresponding `ListenerHolder`, which prevents the event queued or sent later than the removal from being invoked. We should also do this in the method `ListenerSet.clear()` where every listener is removed.
PiperOrigin-RevId: 652535216
This removes several workarounds that are no longer needed, including
`codecNeedsMonoChannelCountWorkaround` which has been permanently
disabled since the (incomplete) minSdk 19 clean-up in fb7438378d.
PiperOrigin-RevId: 652495578
Sources (for example media projection) can populate the `Surface` from
`SurfaceAssetLoader` with timestamps that don't start from zero. But
`MuxerWrapper` assumes the latest sample timestamp can be used as the duration
when it calculates average bitrate and notifies its listener.
This can cause a crash because the calculated average bitrate can be zero if
the denominator duration is large enough.
Use the max minus first sample timestamp across tracks instead to get the
duration.
Side note: the large timestamps from the surface texture when using media
projection arrive unchanged (apart from conversion from ns to us) in effect
implementations and in the muxer wrapper (and are passed to the underlying
muxer). The outputs of media3 muxer and the framework muxer are similar.
PiperOrigin-RevId: 652422674
This should have no influence on app behavior and other policies
and just allows code to depend on new API 35 platform symbols.
PiperOrigin-RevId: 652414026
We currently use displaySurface == placeholderSurface IF codec != null
as a signal that the codec is configured with a placeholder. In the
future, this placeholder might not be needed and we can decouple this
state a bit better by leaving displaySurface == null in this case and
only using placeholderSurface instead of null when setting a Surface
to the codec.
PiperOrigin-RevId: 652391729
