These were missing in 5211ff0dc1
Without these, the `": null"` is still logged (but the `"Unexpected
IllegalStateException"` bit is not), because the **whole** string
concatenation is compared to null as the boolean condition of the
ternary, and this condition is always false (the result of a string
concatentation is never `null`) i.e. (with excess parentheses for
clarity):
```
(("Unexpected " + cause.getClass().getSimpleName() + cause.getMessage()) != null)
? ": " + cause.getMessage()
: ""
```
Also add a test because obviously this isn't as simple as I'd thought.
PiperOrigin-RevId: 593079975
The previous code led me to misread this stack trace as a null pointer
exception, but it's really an index out of bounds exception:
```
Caused by: androidx.media3.exoplayer.upstream.Loader$UnexpectedLoaderException: Unexpected IllegalArgumentException: null
at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:435)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at java.lang.Thread.run(Thread.java:1012)
Caused by: java.lang.IllegalArgumentException
at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:40)
at androidx.media3.common.util.ParsableByteArray.setPosition(ParsableByteArray.java:164)
at androidx.media3.extractor.text.cea.Cea608Parser.parse(Cea608Parser.java:440)
```
PiperOrigin-RevId: 592876546
Instead, for input videos, use the colorInfo provided by the extractor. Similarly, for input images, use sRGB, the only color currently in use.
Textures do still need the input ColorInfo provided though.
PiperOrigin-RevId: 592875967
Replace the event for notifying fallback to cover codec initialization in
general (but keeping the list of errors).
Add a flag to control whether to try non-primary codecs, with the same
documentation as the similar flag in ExoPlayer's renderer.
Make the class final as it shouldn't be necessary to subclass it.
PiperOrigin-RevId: 592869868
This is similar to the HLS fix in 770ca66fbc
Similar to HLS, the original problem here was **not** modifying the
`Format` for caption tracks
embedded into the video stream. I tried just updating the format in
both places, but that caused new failures because the new
('transcoded') format was then fed into `FragmentedMp4Extractor` as
part of `closedCaptionFormats`, which resulted in the CEA-608 data
being emitted from `FragmentedMp4Extractor` with the incorrect
`application/x-media3-cues` MIME type (but the bytes were actually
CEA-608), meaning the transcoding wrapper passed it through without
transcoding and decoding failed (because obviously CEA-608 bytes can't
be decoded by `CueDecoder` which is expecting a `Bundle` from
`CuesWithTiming.toBundle`.
To resolve this we keep track of the 'original' caption formats inside
`TrackGroupInfo`, so we can feed them into `FragmentedMp4Extractor`.
For all other usages in `DashMediaPeriod` we use the 'transcoded'
caption formats.
PiperOrigin-RevId: 592866262
With fMP4 implementation there will be two writers `DefaultMp4Writer`
and `FragmentedMp4Writer`.
Changes includes:
1. Make Mp4Writer as an abstract class and keep only common functionality
into it.
2. Create a DefaultMp4Writer which contains existing logic to write MP4.
3. The fMP4 logic needs to access `pending sample buffer info` at various
places, so did refactoring to split List<Pair<BufferInfo, ByteBuffer>>
into two separate lists.
PiperOrigin-RevId: 592861398
Checking the output format's mime type may skip tests more often than we'd like,
because we may desire using a lower-spec output mimetype than what's passed in, if
based on the input's HDR mimetype value.
Therefore, update this output format to null, for tone-mapping tests
PiperOrigin-RevId: 592855713
Also updated the `README` file to accurately specify the use of NDK r23c and the default setting `ANDROID_ABI=21` for NDK r26b.
PiperOrigin-RevId: 592845796
Issues with the current implementation
1. The implementation is unnecessarily complicated and can be
easily simplified.To make all the tracks start from the same time,
its only the first sample that require some timestamp adjustments
but the current implementation shifts all the timestamps. Since method
calculates the `sample duration`, shifting all the timestamps has no effect
as such.
2. The implementation always forced first sample to start at 0. But when we
want to use same method for `Fragmented MP4` then it will look inaccurate
as we will call this method for different `fragments` and each `fragment`
will not start from 0 presentation time. Although the output will be same
since this method returns `duration` and not the `timestamps`.
3. As per previous implementation if there is just one sample then
its duration is made equals to its presentation time, which looks incorrect.
With new changes, if a single sample is passed then its duration will always
be 0 irrespective of specified last sample duration behaviour.
PiperOrigin-RevId: 592826612
getAudioCapabilities currently creates the receiver and returns
the current capabilities. This is error-prone because the
capabilities are also available as a class field.
This can made cleaner by letting the method just create the receiver
and all access to the capabilities can be made via class field.
PiperOrigin-RevId: 592591590
HLS distinguishes between 'subtitles' (WebVTT or TTML distributed in
separate files with their own playlist) and 'captions' (CEA-608 or 708,
distributed muxed into the video file).
The format transformation added in 7b762642db
only applies to subtitles and not captions. This change makes the same
transformation for caption formats.
This resolves an error like:
```
SampleQueueMappingException: Unable to bind a sample queue to TrackGroup with MIME type application/cea-608.
```
Also add two playback tests for HLS CEA-608, one that parses during
decoding (old way) and one during extraction (new way). Adding these
tests is what alerted me to this issue.
PiperOrigin-RevId: 592571284
This was generated by combining the existing `ts/bbb_2500ms.ts` test
asset and a temporary `.srt` file using
https://cloud.google.com/transcoder/docs/how-to/captions-and-subtitles
This doesn't directly reproduce the problem fixed by
7ca26f898d,
because the CEA-608 subs are structured differently to the stream I
discovered the problem with (from Issue: androidx/media#887). However this test
does fail if that fix is reverted after
486230fbd7.
I'm also not able to repro the character duplication reported in
Issue: androidx/media#887 by just changing the manifest in this CL. I'm not yet
sure on the exact differences between the stream provided on GitHub
and this stream.
This stream does provide some regression protection, because it
currently fails with 'new' subtitle parsing
(`DashMediaSource.Factory.experimentalParseSubtitlesDuringExtraction(true)`),
though I'm not sure on the exact reason for that yet.
PiperOrigin-RevId: 592476328
This is a refactoring that allows the `MediaPeriodQueue` to
create media period holders without all the collaborators
being passes in to `enqueueNextMediaPeriodHolder(...)`.
The factory is presumably also helpful in unit test to
know whether and when exactly a holder is created in
the preloading process.
PiperOrigin-RevId: 592301400
We deliberately left `Cea608Decoder` and `Cea708Decoder` intact when
creating the respective `SubtitleParser` implementations (in
27caeb8038
and
94e45eb4ad
respectively).
However we didn't correctly change the behaviour of
`SubtitleDecoderFactory.DEFAULT` in order to use these 'legacy'
implementations. We firstly left the `Cea608Decoder` instantiation
**after** the `DefaultSubtitleParserFactory.supportsFormat()` check,
meaning that code would never be reached (since `supportsFormat` now
returns true for CEA-608). Then in the second change (which was supposed
to only be about CEA-708) we removed all instantiations of **both**
`Cea608Decoder` and `Cea708Decoder`. This change puts the decoder
instantiations back and moves them above the
`DefaultSubtitleParserFactory.supportsFormat` check, meaning they will
be used in preference to `DelegatingSubtitleDecoder`.
This resolves the immediately-disappearing subtitles tracked by
Issue: androidx/media#904.
PiperOrigin-RevId: 592217937
While investigating Issue: androidx/media#887 I naively assumed the CEA-608
captions were in a TS file, but they're actually in an MP4 (which is
possibly obvious given DASH only supports MP4). This change includes
container info in the `EventLogger` `tracks` output.
PiperOrigin-RevId: 592192752
This allows us to remove the additional thread we create
for asynchronous buffer queuing and use a synchronized
queueing approach again.
This API looks strictly beneficial on all tested devices,
but since this code path in MediaCodec has not been used
widely, we leave an opt-out flag for now.
PiperOrigin-RevId: 591867472
The parameters may change the decoding behavior of the
following samples and it's suprising/wrong to apply them
from another thread where we can't guarantee from which
sample they apply.
PiperOrigin-RevId: 591808760
Move `inputColorInfo` from `VideoFrameProcessor`'s `Factory.create` to `FrameInfo`,
input via `registerInputStream`.
Also, manually tested on exoplayer demo that setVideoEffects still works.
PiperOrigin-RevId: 591273545
When an adaptive source has been preloaded, the track selection
of the player should possibly not override the preloaded selection
to avoid discarding preloaded data.
PiperOrigin-RevId: 591256334
The number of empty image byte arrays written is one less than the
total number of tiles in the image. The empty byte arrays act as
placeholders for individual tiles inside ImageRenderer.
PiperOrigin-RevId: 591231432
This is because currently
1. Player sets a surfaceView to render to
2. Player intializes the renderer
3. MCVR initializes the VideoSinkProvider, by extension VideoGraph
But when 1 happens, MCVR doesn't set the surfaceView on the VideoGraph because
it's not initialized. Consequently after VideoGraph is initialized, it doesn't
have a surface to render to, and thus dropping the first a few frames.
Also adds a test for first frame to verify the correct first frame is rendered.
PiperOrigin-RevId: 591228174
When track is changed during playback, `playbackPositionUs` may be in middle of a chunk and `loadPositionUs` should be the start of that chunk. In this situation `loadPositionUs` can be less than the current `playbackPositionUs`, resulting into negative `bufferedDurationUs`. It translates to having no buffer and hence we should send `0` for `bufferedDurationUs` when creating new instances of `CmcdData.Factory`.
Issue: androidx/media#888
#minor-release
PiperOrigin-RevId: 591099785
Otherwise, there's a memory leak of ~30MB, as this is never released.
This likely used to be considered released as part of what now became
`intermediateGlShaderPrograms`, but its release was missed after we split
`finalShaderProgramWrapper` out from the larger glShaderProgram list.
PiperOrigin-RevId: 590954785
To prepare to move `inputColorInfo` from `VFP.Factory.create` to
`VFP.registerInputStream`, move all usage of `inputColorInfo` to be *after*
`registerInputStream`.
To do this, defer creation of `externalShaderProgram` instances, which require
`inputColorInfo`. However, we must still initialize `InputSwitcher` and OpenGL
ES 3.0 contexts in the VFP create, to create and request the input surface from
ExternalTextureManager.
PiperOrigin-RevId: 590937251
The timestamp adjuster also estimates the number of wraparounds
of the 90Khz TS timestamp. It does that by assuming that a new
timestamp is always close to the previous one (in either direction).
This logic doesn't always work for duration estimates because the
timestamp at the end of the media is not close to the one at the
beginning and it may also never be less than the one at the beginning.
This can be fixed by introducing a new estimation model that assumes
the new timestamp is strictly greater than the previous one without
making the assumption that it has to be close to it.
Issue: androidx/media#855
#minor-release
PiperOrigin-RevId: 590936953
When broadcasting a notifyChildrenChanged event, the task for legacy
controllers was sent to the broadcasting callback. This would
technically work, but because the subscription list is maintained
with specific controllers, the broadcast controller isn't subscribed
and hence the call wasn't executed.
This change calls the overloaded method for a specific controller
for each connected controller. Making sure (only) subscribed
controllers are notified.
Issue: androidx/media#644
PiperOrigin-RevId: 590904037
Vertical and horizontal tile counts inside Format used by ImageRenderer
used to be required to equal 1 and were set to 1 regardless of what
the manifest said.
This change removes the above requirement and sets the tile counts to
the values from the manifest.
PiperOrigin-RevId: 590608353
Using 0 as the unset prepare position is the root cause of a number of issues,
as outliine in the ExoPlayer issue https://github.com/google/ExoPlayer/issues/7975
The premise of this fix is that once the prepare override is used (the initial call
to `selectTracks()`) it is never needed again, so simply invalidate it after use.
Despite GL 3.0 not being required on API 29+, it is experimentally
determined to always be supported on our testing devices, on API 29+.
That said, still fall back to OpenGL 2.0 if 3.0 is not supported,
just in case.
PiperOrigin-RevId: 590569772
It's better to use the generic Android resource scheme which
is at least as powerful as our own one.
Issue: androidx/media#868
PiperOrigin-RevId: 590168919
`Mp4Writer` receives all the tracks/samples and then it adds them to
corresponding `Track` object. As the `Track` object is primarily to hold
track specific data and `Mp4Writer` is responsible for interleaving related
logic, the call to `doInterleave` should be with `Mp4Writer`. The `Track`
object should just act like a data holding class.
This will also help in reusing `Track` class for fragmented MP4 implementation.
PiperOrigin-RevId: 590138002
An app that uses the `MediaSession` without a `MediaSessionService` may
want to connect a media notification controller on it own. To avoid apps
using a string literal, the key should be exposed as public.
PiperOrigin-RevId: 589875603
The MP4 data in JPEG motion photos can contain multiple `video/hevc` tracks, but only the first is at a playable frame rate while the others are low-fps, high-res tracks designed for specific use-cases (not direct video playback).
ExoPlayer currently selects the unplayable track by default, because it
has a higher resolution. This change introduces a flag to
`Mp4Extractor` that results in the first video track being marked as
`ROLE_FLAG_MAIN`, and all subsequent video tracks `ROLE_FLAG_ALTERNATE`
- this then results in the playable lower-res track being selected by
default.
PiperOrigin-RevId: 589832072
There is a bug in CompositionVideoSinkProvider where the rotation
effect is applied only on the first video and the same effect
is applied on subsequent videos without checking if the next
items in the playlist must be rotated.
The bug applies only on devices with API < 21.
PiperOrigin-RevId: 589823797
We currently pause playback to prevent further progress while the
app thread runs assertion and triggers additional actions. This is
not ideal because we do not actually want to pause playback in
almost all cases where this method used.
This can be improved by keeping the playback thread blocked and only
unblock it the next time the app thread waits for the player (either
via RobolectricUtil methods or by blocking the thread itself). To
add this automatic handling, this change introduces a new util class
for the tests that can keep the list of waiting threads statically
(because the access to this logic is spread across multiple independent
classes).
PiperOrigin-RevId: 589784204
Move the `AnalyticsListener` implementation to a private inner class.
This avoids polluting the public API of `ExoHostedTest`, especially as
it's designed to be extended.
PiperOrigin-RevId: 589188113
Also adds an alternate way to configure the AudioGraph.
Apps should no longer need to ensure that inputs have the same sample
rate.
PiperOrigin-RevId: 588747431
Added a new ABANDONED option so one can tell the difference between when the optimization has been requested but not applied vs not requested at all. also changed the ordering do better represent the hierarchy of failure modes
PiperOrigin-RevId: 588720513
Fix a bug where CompositingVideoSinkProvider.isInitialized() returns
true even after releasing the CompositingVideoSinkProvider.
PiperOrigin-RevId: 588481744
This change moves the instantiation of the CompositingVideoSinkProvider
out of MediaCodecVideoRenderer so that the composition preview player can
re-use the CompositingVideoSinkProvider instance for driving the rendering of
images.
The main point of the change is the ownership of the
VideoFrameReleaseControl, which decides when a frame should be rendered
and so far was owned by the MediaCodecVideoRenderer. With this change,
in the context of composition preview, the VideoFrameReleaseControl
is no longer owned by MediaCodecVideoRenderer, but provided to it.
This way, the CompositingVideoSinkProvider instance, hence the
VideoFrameReleaseControl can be re-used to funnel images into the
video pipeline and render the pipeline from elsewhere (and not
MediaCodecVideoRenderer).
PiperOrigin-RevId: 588459007
The serializable form is used when we need to serialize the result into
bytes in the sample queue. The binder-based (ultimately
filedescriptor-based) form is used for
session/controller IPC, in order to avoid sending the bitmap bytes over
the IPC.
Issue: androidx/media#836
#minor-release
PiperOrigin-RevId: 588420836
This helped me debug which bitmap was actually failing. Otherwise, you need to clear all bitmaps on the device relating to this test, then adb pull all bitmaps and see the last uploaded one to see, which is much more confusing.
PiperOrigin-RevId: 588374081
This image has two video tracks in the MP4 data, one is a 'real' video
which we want to play by default, and the other is a low-fps video track
which isn't intended to be directly played, it's encoded in HEVC for
compression and decoding efficiency.
This test demonstrates ExoPlayer's current behaviour default extraction
and playback, which results in selecting the high-res, low-fps track
(actually single sample in this example), instead of playing the actual
video.
PiperOrigin-RevId: 588068908
As Opus decoders skip some bytes prior to playback during a seek, the renderer for bypass playback should send samples to the decoder even if they would be decode-only. However, the renderer should not send samples with time preceding that range. This change adds that constraint.
#minor-release
PiperOrigin-RevId: 588014983
Both the extension OPUS decoder and the OMX/C2 MediaCodec
implementations for OPUS and VORBIS decode into the channel
layout defined by VORBIS. See
https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-140001.2.3
While this is technically correct for a stand-alone OPUS or VORBIS
decoder, it doesn't match the channel layout expected by Android.
See https://developer.android.com/reference/android/media/AudioFormat#channelMask
The fix is to apply the channel mapping after decoding if needed.
Also add e2e tests with audio dumps for the extension renderer,
including a new 5.1 channel test file.
Issue: google/ExoPlayer#8396
#minor-release
PiperOrigin-RevId: 588004832
`mediaCrypto` is initialized before `codec` in
`maybeInitCodecOrBypass`. Before this change, it was possible for
`maybeInitCodecOrBypass` to complete with `mediaCrypto != null` and
`codec == null`, in particular if it was run as part of clearing the
player surface (since in that case, no video codec is initialized).
This inconsistent state then causes issues during a later invocation of
`maybeInitCodecOrBypass`, when `mediaCrypto` is still non-null, and
`mediaCryptoRequiresSecureDecoder = true`, but the
content has been changed to unencrypted with no associated DRM session.
This results in a playback error, because a secure decoder is
initialized but there's no DRM session available to work with it.
This change ensures that when `maybeInitCodecOrBypass` completes,
either both `mediaCrypto != null` and `codec != null` (i.e. codec
initialization was completed) or `mediaCrypto == null` and
`codec == null` (i.e. codec initialization was not completed). We also
ensure that when nulling out `mediaCrypto` we also set
`maybeInitCodecOrBypass = false`. A later change should be able to
demote `maybeInitCodecOrBypass` from a field to a local in order to
remove any risk of that part of state becoming out of sync. This
resolves the issue, because during the second invocation of
`maybeInitCodecOrBypass` an insecure decoder is now (correctly)
initialized and the unencrypted content is successfully played.
#minor-release
PiperOrigin-RevId: 587713911
We set the `playerId` only in the constructor currently. But in the case of this source doesn't have `preload` called, the player's release of this source will set the playerId to `null`, which makes this source un-reusable with a null `playerId`.
PiperOrigin-RevId: 587698214
