Reduces flakiness of tests that assert on PCM audio. Tests now have to
clearly choose how they want the capturing muxer to handle pcm audio.
Note that the only dump files that have changed are those that deal
with PCM audio (.wav, sowt, twos, silence). Because of the continuous
nature of PCM, timestamps are not part of the dump.
PiperOrigin-RevId: 623155302
Compared to `release`, the `reset` method doesn't release the preload manager instance. This applies to the use case that an app wants to discard all the sources but keep the preload manager active for later usage.
Also rename the `releaseSourceInternal` to `removeSourceInternal`, as the latter sounds more generic for different preload manager implementations.
PiperOrigin-RevId: 623148723
In offloaded audio playback, the `DefaultAudioSink` should use the `AudioTrack.StreamEventCallback` `onPresentationEnded` to note whether the AudioTrack has completed playing all pending data.
PiperOrigin-RevId: 622885399
The androidx.media library is only used for its compat MediaStyle.
On API 21 and above, the logic can be easily inlined, and only
on API < 21, the androidx.media handling can be used.
This allows to remove the androidx.media dependency completely
once the minSdk has been increased to 21.
PiperOrigin-RevId: 622855469
Both `remove(MediaItem)` and `remove(MediaSource)` return a boolean suggesting that whether the preload manager is holding the corresponding `MediaSource` and it has been removed.
PiperOrigin-RevId: 622185427
This change makes sure the `DefaultLoadControl` would work
when passed to multiple players. It makes sure and unit tests
that the loading state of a player is maintained for each player
that is using `DefaultLoadControl`.
The targetBufferSize of the `DefaultAllocator` is increased
linearly for each player and memory is allocated in a simple
first-come-first-serve manner.
PiperOrigin-RevId: 622126523
The join mode is used for two cases: surface switching and mid-playback
enabling of video.
In both cases, we want to pretend to be ready despite not having rendered
a new "first frame". So far, we also avoided force-rendering the first
frame immediately because it causes a stuttering effect for the
mid-playback enable case. The surface switch case doesn't have this
stuttering issue as the same codec is used without interruption. Not
force-rendering the frame immediately causes the first-frame rendered
callback to arrive too early though, which may lead to cases where
apps hide shutter views too quickly.
This problem can be solved by only avoiding the force-render for the
mid-playback enabling case, but not for the surface switching case.
PiperOrigin-RevId: 622105916
This API additions help an app to implement the lifecycle of a MediaSessionService
properly and in consistency with the `MediaSessionService` being in the foreground
or not.
Not properly implementing `onTaskRemoved` is the main reason for crashes and
confusion. This change provides `MediaSessionService` with a default
implementation that avoids crashes of the service. This default implementation
uses the new API provided with this change just as an app can do.
Issue: androidx/media#1219
PiperOrigin-RevId: 621874838
Makes OverlayFrameAnchor works as described in the OverlaySettings documentation. Currently the code does the opposite e.g setting the anchor to (+1,-1) makes the code anchor to the top left rather than the bottom right.
PiperOrigin-RevId: 621585558
Otherwise, apps overriding BitmapOverlay.getVertexTransformation may not
realize this is being done, and may ask why bitmaps are flipped vertically from
what they expect
Reference: https://github.com/androidx/media/issues/1128
PiperOrigin-RevId: 621544348
Also explicitly use a synchronized list to collect cues. The previous
bare `ArrayList` was probably fine, because the `ConditionVariable`
creates a memory barrier between the main thread and the test thread,
but this seems more explicit (and any performance implications of the
synchronization don't matter for this test).
PiperOrigin-RevId: 621523824
the output format conatins key information about the output of the assetloader being hdr, so we must signal the output format, not the input format to the sample exporter
fixes mh ultraHdr test failures.
Also discovered images created are very device specfic so got rid of the pixel tests, we have pixel tests in the effects library that cover the same case.
PiperOrigin-RevId: 619899249
If the mixer is reset without resetting maxPositionOfRemovedSources and
then reused, the value of maxPositionOfRemovedSources can be outdated,
leading to an incorrect number of bytes being output by the mixer.
PiperOrigin-RevId: 619832502
calculate based on the output of consecutive calls rather than the speed provider speed change point to ensure the timestamps are monotonically increasing.
PiperOrigin-RevId: 619584001
Since the public class has already been split into `Mp4Muxer` and
`FragmentedMp4Muxer`, there is no need for having common interface
for internal implementation.
In the follow up CL `BasicMp4Writer` will be renamed to `Mp4Writer`
which is more appropriate and aligns with public class names.
PiperOrigin-RevId: 619486876
With this CL:
- The large timestamp offset added by ExoPlayer so that decoders
don't see any negative timestamps is removed before passing the
timestamp to the AudioGraph.
- Clipped media timestamps are offset by the clipping start position
before being passed to the AudioGraph.
- The offset added to the audio buffer timestamps is removed when
computing the audio sink position, to convert them back to the
timestamps passed by the player.
PiperOrigin-RevId: 619225990
This CL aims to separate Fragmented MP4 related logic in a separate public class.
Earlier all the logic was in a single class `Mp4Muxer`.
PiperOrigin-RevId: 619206661
The workaround check is now part of CTS and we should be able
to rely on the PerformancePoints values (or at least can be sure
that they cover all CDD requirements)
#minor-release
PiperOrigin-RevId: 619201331
This property is transformer specific and does not belong to
muxer interface.
This is to eventually replace muxer interface in Transformer module with
muxer interface in Muxer module.
PiperOrigin-RevId: 618895836
This ensures MediaMetadata with just non-null extras is not
considered equal to MediaMetadata.EMPTY. This makes sure the
contents are bundled when a controller sets the extras in a
new MediaItem
Issue: androidx/media#1176
#minor-release
PiperOrigin-RevId: 618876642
These TODOs precede <unknown commit> when the group was set in the top-level
`build.gradle` file.
Issue: androidx/media#1215
PiperOrigin-RevId: 618835040
The immediate plan is to replace `Transformer Muxer interface`
with this one.
It was not straight forward to move/change `Transformer Muxer interface`
hence this intermediate step.
The follow up CL will also remove fragmented MP4 related code from
`Mp4Muxer` and move it into `FragmentedMp4Muxer implements Muxer`.
PiperOrigin-RevId: 618789265
This is the most widely-used test-skipping method I'm aware of, so I figured this
would be a great method to scale usage of AssumptionViolatedException.
PiperOrigin-RevId: 618160931
Issue: androidx/media#725
Ideally the test to transmux a `ts` file (having 3 bytes NAL start code)
should go into `Mp4MuxerEndToEndTest`but `Mp4MuxerEndToEndTest` uses `MediaExtractor` and `MediaExtractor` is returning samples with `4 byte NAL start code` which will not exercise the newly added code path.
Hence the test is added in `TransformerWithInAppMuxerEndToEndTest` which
internally uses `media3 extractor` and feeds samples with `3 bytes NAL start code`
only.
PiperOrigin-RevId: 617985866
Seeking will consist of the following steps:
- Block the AudioGraph input data.
- Flush the AudioGraph.
- Seek the ExoPlayers.
- Unblock the AudioGraph input data.
PiperOrigin-RevId: 617868124
The original change did not set the color info in the codec
for standard SDR infos. This was meant to help with consistency
to avoid a situation where the information is partially set and
later the bitstream provides slightly different values (all in
standard SDR though).
We can revert this part because the bitstream may change anyway
and the decoder needs to handle this internally. And more
importantly, it also avoids removing this information when encoding
the format again in Transformer.
PiperOrigin-RevId: 617582066
This refactoring is required to support 3 byte NAL start code, in which
the sample ByteBuffer might change after AnnexB to Avcc conversion.
This would required changes in the corresponding BufferInfo as well.
PiperOrigin-RevId: 617538338
Some media can read color info values from the bitstream
and may partially set some of the SDR default values in
Format.ColorInfo. Setting these default values for SDR can
confuse some codecs and may also prevent adaptive ABR
switches if not all ColorInfo values are set in exactly the
same way.
We can avoid any influence of HDR color info handling by
disabling setting the color info MediaFormat keys for SDR
video and also avoid codec reset at format changes if both
formats are SDR with slightly different ColorInfo settings.
To identify "SDR" ColorInfo instances, we need to do some
fuzzy matching as many of the default values are assumed to
match the SDR profile even if not set.
Issue: androidx/media#1158
PiperOrigin-RevId: 617473937
The AV1 video stream does not contain NAL unit which is a concept
specific to H.264/H.265.
This was not caught before muxer does an in place replacement of
`NAL start code` with `NAL length`. In the absence of `NAL start code`
it was automatically a no-op.
PiperOrigin-RevId: 617193317
nullness checks were removed a while ago from editing, so no need for them now.
Bit neater to have less code to parse too.
Also, removes RequiresNonNull
PiperOrigin-RevId: 616887502
This also makes it more consistent with newer testId tests that don't have nullness, so tests don't look inconsistent, and was pretty easy to do :P
PiperOrigin-RevId: 616856888
This was broken by 2fa38d15dd, which added an assumes
method, but missed the `!` operator for the tests where HDR is
unsupported. Add an assumes method that assumes a lack of support
for HDR.
Tested on the failing device (Pixel 7) and confirmed this fixes
the test on that device, to throw AssumptionViolatedException
instead of allowing test logic to run after failing the
assumption.
PiperOrigin-RevId: 616846588
Add a new method `onUsedByPlayer(PreloadMediaSource)` for `PreloadMediaSource.PreloadControl`, which will be invoked when the player starts to use the `PreloadMediaSource`, or calling `PreloadMediaSource.preload` while the player is already using that source. `DefaultPreloadManager` will immediately preload the next source when receiving `onUsedByPlayer` event.
PiperOrigin-RevId: 616789121
Confirmed that the HLG extension displays properly on Pixel 7 Pro, API 34 and Samsung S24, API 34. On these devices, this fixes the washed out HLG preview issue on API 34, where the PQ ext had a washed out look, and where the HLG ext is supported (this bug didn't occur on API 33, only 34).
More info on manual tests done to sanity-check:
* Test cases: Transformer debug SurfaceView and ExoPlayer.setVideoEffects
* Test inputs: HLG and PQ
* Test devices: Pixel 7 Pro (API 33 & 34), Samsung Galaxy S24 (API 34)
* Added debugging: Logging colorInfo used in GlUtil.createEglSurface
No regressions were seen. HLG extension is used more in API 34, and behavior stays the same on API 33. Only human-visible change without logging is that HLG content looks better on API 34, for Samsung S24 and Pixel 7 Pro.
PiperOrigin-RevId: 616131192
It's a bit arguable whether the `Subtitle` implementation supports
zero-duration events, since `getEventTimeCount` is documented as
effectively "the number of times the cues returns by `getCues(long)`
changes", and zero-duration events violate that. However, the current
`WebvttSubtitle` impl **does** produce zero-duration events, so it
seems safer to handle them gracefully here and then, as a possible
follow-up, fix the `WebvttSubtitle` impl (or remove it completely).
Issue: androidx/media#1177
#minor-release
PiperOrigin-RevId: 616095798
When flushing the AudioGraph, the AudioMixer will be preserved but the
sources will be recreated. This is probably a bit less efficient but
makes the logic simpler. Indeed, if we had to keep the sources alive,
we would need to add a way to reconfigure them with a new timestamp for
seeking. We would also need to change the way sources are ended because
they are currently removed when they are ended. Also, it is acceptable
to have a small delay when seeking, which means that performance is less
critical than for playback.
PiperOrigin-RevId: 615775501
getSupportedEncoderNamesForHdrEditing became getSupportedEncodersForHdrEditing at some point, but this comment wasn't updated before...
PiperOrigin-RevId: 615712707
Renderers may be enabled for subsequent media items as soon as the current media item's renderer's isEnded() returns true. Currently, when a player is set to pause, it stops all renderers that are `STATE_STARTED`. When a player is set to play, it starts all renderers that are enabled. This would include renderers that were enabled early for the subsequent media item. The solution is to only start renderers that are enabled by the current playing period.
Issue: androidx/media#1017
PiperOrigin-RevId: 614734437
This should already be the default, but some devices seem
to not adhere to this contract and assume the default is unset.
Issue: androidx/media#1169
PiperOrigin-RevId: 614697283
Some devices just don't work very well with the synchronous
model, but are currently still excluded from our approximate
API 31 check. This change allows to include additional devices
or device groups by passing in the Context to the default
adapter.
It also adopts the workaround added in ebceee08c6
for Fire TV Smart devices that exhibit this issue.
PiperOrigin-RevId: 614642545
Some FireOS6 devices ask to force the external surround global
flag and ignore any signals from the HDMI connection.
This is the equivalent change of e341944d1e
PiperOrigin-RevId: 614634499
Having a default icon available allows apps to only specify the
icon constant without having to define an icon drawable themselves
as Media3 can fill in the icon resource for backwards compatibility.
The switch util method allows R8 to easily remove unused icons, so
having default icons won't affect APK size unless the constants are
used to set up the CommandButtons.
PiperOrigin-RevId: 614623909
Using the more accurate check available on later API versions
first is likely better than falling back to a fallback solution
from older API versions.
PiperOrigin-RevId: 614612628
minor fix for arithmetic error in calculated added in f4c60c52b9 (speed is outputTime (change in y) divided by inputTime (change in x), this calculation is inverted in code).
Changed test to cover the case.
PiperOrigin-RevId: 613966027
These input types include images, video without audio, and video with audio.
While playing these inputs, the video frame presentation is always synced with
audio.
PiperOrigin-RevId: 613921719
This check is already present in `DataSpec.Builder.build()` but there
are many public constructors which bypass the builder (only some of
which are deprecated), so this adds an additional check.
PiperOrigin-RevId: 613908358
Also remove intermediate object allocations in
`Util.toByteArray(int...)`.
`Util.toByteArray(InputStream)` is kept (but deprecated) because it's
been part of the library for a while and might be in use by some apps.
The others are much newer, so the chance of usages outside the library
is very low, so we just remove them directly.
PiperOrigin-RevId: 613878453
The implementation of these methods is changed in a follow-up CL, and
these tests help to ensure nothing breaks.
This doesn't include tests for `toByteArray(InputStream)` or
`toByteArray(int)` because these implementations are fully replaced by
Guava equivalents in a follow-up CL.
PiperOrigin-RevId: 613581845
This is the only `build.gradle` file that currently doesn't apply this
config, and it seems to lead to desugaring errors when apps depend on
the library locally.
PiperOrigin-RevId: 613559535
Renderers may be enabled for subsequent media items as soon as the current media item's renderer's isEnded() returns true. When a renderer is being enabled and the player is 'playing', that renderer is also started. When playing a mixed playlist of images and content with audio & video, the player may skip some image items because the early-starting of the audio renderer causes a clock update.
A solution is to only start the "early-enabled" renderers at the point of media transition and add a condition on DefaultMediaClock to use the standalone clock when reading-ahead and the renderer clock source is not in a started state.
Issue: androidx/media#1017
PiperOrigin-RevId: 613231227
The old methods are deprecated and are called from the new
method for backwards compatibility of custom implementations.
'DefaultLoadControl' is unchanged, but `ExoPlayerImplInternal`
already calls the new methods passing in the `PlayerId`,
PiperOrigin-RevId: 613197190
Pass firstAssetLoaderOutputFormat to videoSampleExporter for non-video use cases, so that the downstream components like the videoFrameProcessor can be set up with the right output color. Surface creation is still in the VSP so can't do this for all use cases currently.
also moves getDecoderOutputColor() to TransformerUtil, since it is used in multiple places and doesn't make sense for once to have reference to the other.
PiperOrigin-RevId: 613113958
We currently update this value for controllers to match the
availability of the associated command. This however makes it
impossible to mark a button as unavailable if the command is
available. This can be refined by only setting the 'enabled'
field to false if the command is not available, not the other
way round. And we should also enable the button by default as
disabling is the unusual case not many apps will use.
In addition, this change fixes missing update logic when the
player commands changed and it adds some additional test coverage
for all these cases.
PiperOrigin-RevId: 612881016
`BasePreloadManager` coordinates the preloading for multiple sources based on the priorities defined by their `rankingData`. Customization is possible by extending this class. Apps will implement `TargetPreloadStatusControl` to return preload manager the target preload status for a given `rankingData` of the source.
`DefaultPreloadManager` extends from the above base class and uses `PreloadMediaSource` to preload media samples of the sources into memory. It also uses an integer `rankingData` that indicates the index of an item on the UI, and the priority of the items is determined by their adjacency to the current playing item. Apps can set the index of current playing item via `DefaultPreloadManager.setCurrentPlayingIndex` when the user swiping is detected.
PiperOrigin-RevId: 612829642
The removed check searched for a player command inside a list of
session commands, which is not allowed by the IntDef definition
and only worked because both types map to a Java int.
PiperOrigin-RevId: 612758442
Plumbing hdrMode through the default asset loader factory via the constructor is problematic because it breaks API boundaries. It means there is another way to set hdrMode outside of Composition.java and TransformationRequest.java, which is error prone and cause problems if someone an app starts customizing the assetloaderfactory. It also means custom asset loaders can't receive this information without hacking around.
The introduction of the composition-level settings class makes this approach easily extensible for other settings applied on the composition level but use in an individual asset level basis (e.g. ultraHDR support).
PiperOrigin-RevId: 611466920
Add abilitiy to use real surfaces in instrumentation tests
using the ActivityScenarioRule and an activity class for testing
purposes.
PiperOrigin-RevId: 611421490
When the controller replaces the current item, the masking position will be changed to the default position of the new item for a short while, before the correct position comes from the session. This will interrupt the current position fetched from the controller when the playback doesn't interrupted by the item replacing.
Issue: androidx/media#951
#minor-release
PiperOrigin-RevId: 611417539
This implementation generates lint errors because neither the `first`
nor `second` parameters are used, and that's generally
unexpected/incorrect for a `Comparator` implementation since it should
always consider both its parameters.
PiperOrigin-RevId: 611039632
This change aims to prioritise tracks that have a 'smooth enough for
video' frame rate, without always selecting the track with the highest
frame rate.
In particular MP4 files extracted from motion photos sometimes have two
HEVC tracks, with the higher-res one having a very low frame rate (not
intended for use in video playback). Before this change
`DefaultTrackSelector` would pick the low-fps, high-res track.
This change adds a somewhat arbitrary 10fps threshold for "smooth video
playback", meaning any tracks above this threshold are selected in
preference to tracks below it. Within the tracks above the threshold
other attributes are used to select the preferred track. We deliberately
don't pick the highest-fps track (over pixel count and bitrate), because
most users would prefer to see a 30fps 4k track over a 60fps 720p track.
This change also includes a test MP4 file, extracted from the existing
`jpeg/pixel-motion-photo-2-hevc-tracks.jpg` file by logging
`mp4StartPosition` in
[`MotionPhotoDescription.getMotionPhotoMetadata`](b930b40a16/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/MotionPhotoDescription.java (L123))
and then using `dd`:
```
mp4StartPosition=2603594
$ dd if=jpeg/pixel-motion-photo-2-hevc-tracks.jpg \
of=mp4/pixel-motion-photo-2-hevc-tracks.mp4 \
bs=1 \
skip=2603594
```
----
This solution is in addition to the `JpegMotionPhotoExtractor` change
made specifically for these two-track motion photos in
5266c71b3a.
We will keep both changes, even though that change is not strictly
needed after this one, because adding the role flags helps to
communicate more clearly the intended usage of these tracks. This
change to consider FPS seems like a generally useful improvement to
`DefaultTrackSelector`, since it seems unlikely we would prefer a 5fps
video track over a 30fps one.
Issue: androidx/media#1051
PiperOrigin-RevId: 611015459
This new test is for `ExoPlayer.setVideoEffects()`. It plays the
one-second-long video, applies an overlay that prints the video frame timestamp
onto the frame, captures the output frame and compares the captured output
frame with golden.
PiperOrigin-RevId: 610781590
These audio offload failure recovery tests model the DefaultAudioSink failing at audio track init and write operations in offload mode. Playback should recover and try again as DefaultAudioSink will disable offload mode.
PiperOrigin-RevId: 610372935
Some devices supporting Performance Points for decoder coverage are missing coverage over the CDD requirements for H264. For these cases ExoPlayer should fall back to legacy resolution and frame rate support checks. If there is a stream evaluated as a PerformancePointCoverageResult of COVERAGE_RESULT_NO, then ExoPlayer checks for coverage of the 720p H264 CDD requirement.
Issue: google/ExoPlayer#10898
Issue: androidx/media#693
Issue: androidx/media#966
PiperOrigin-RevId: 609740128
Even when there's no display surface, MCVR can render frames to VFP, becuase by
the time the frame is processed:
- If there's still no surface, VFP will drop the frame;
- If there's surface, the processed frame would be rendered.
In short, placeholder surface is not needed in effect enabled playback. FWIW,
it is used to swallow frames directly from MediaCodec when there's no output
surface.
PiperOrigin-RevId: 609705222
This change makes ExoPlayerImplInternal.releaseInternal() unblock the
app thread if a runtime exception is thrown while releasing components
from the playback thread.
Before this change, if a runtime exception occurred during releasing
components in the playback thread, ExoPlayer.release() would wait for
`releaseTimeoutMs` and then raise a player error. With this change,
the player error is reported only when the playback thread is blocked
but if there is a runtime exception, the application thread is
unblocked.
The impact of this change is potentially fewer ANRs on
ExoPlayer.release() at the expense of less error reporting.
PiperOrigin-RevId: 609702549
We forgot to add it when we added AudioSink.release(). The commit
includes a test that ensures ForwardingAudioSink overrides all the
methods defined in the AudioSink interface.
PiperOrigin-RevId: 609402258
If the reading period has already advanced and a track reselection procs that only affects the reading period media, then ExoPlayer may try and apply the reading period's track selection incorrectly unto the playing period. ExoPlayer should apply the playing period's track selection to the playing period instead.
PiperOrigin-RevId: 609375077
If render error occurs due to AudioTrack initialization failure in offload mode, then ExoPlayer should allow retry as subsequent attempt will be with DefaultAudioSink disabling offload.
PiperOrigin-RevId: 609304897
The `PreloadMediaPeriod.selectTracksForPreloading` can be called for multiple times at the preloading stage (before the period is being played). For example, when the period resumes preloading. This change fix the assertion failure in `ProgressiveMediaPeriod.selectTracks` caused by the wrong implementation of `PreloadMediaPeriod.selectTracksForPreloading` when it is trying to retain the previously preloaded streams.
Also the `TrackSelectorResult` parameter is changed to a list of `ExoTrackSelection`. We should compare the selections only rather than considering the `RendererConfiguration` in the `TrackSelectorResult` to decide whether to retain the streams, as for preloading case the renderers haven't consumed the samples yet.
PiperOrigin-RevId: 609126868
Previously, the track format was used in VideoSampleExporter. Now, we use a
simulated decoder output format.
As the last expected change for this bug, also adds release notes
PiperOrigin-RevId: 609080629
Also introduce a fluent API that allows callers to ignore non-fatal
errors (while avoiding adding boolean overloads for every method).
**Most** tests want to fail on non-fatal errors (since they likely
indicate user-visible issues like codec errors etc), only tests
explicitly testing fallback in error scenarios should want to ignore
them.
Before this change there were a few `playUntilXXX` methods. These can
now all be triggered via `play(player).untilXXX`, which means
effectively every 'until' condition is available in a 'play until'
variant that calls `play` just before waiting for the condition.
PiperOrigin-RevId: 608988234
MediaCodec docs already allude to potentially mismatching H.264 level
between container and bitstream. Relax the initialization data check to
reflect this.
PiperOrigin-RevId: 608942322
When applying edit lists, we need to output the last partial samples
to have all the necessary data needed for rendering.
The only case where we can omit the sample is for zero duration
audio data that has no additional information.
The current comment and variable name doesn't make this very clear
and this change improves the naming and the comment.
PiperOrigin-RevId: 608579746
There is no super test which covers whole MP4 structure.
In all the E2E test, it verified against `ExtractorOutput` which
would remain same if there are changes in the box structure.
PiperOrigin-RevId: 608310006
These were previously somewhat supported, but the `package` part was
never read (so it only worked if it was either absent or the same as the
current application's package). This change parses and uses the
`package` part.
This partially reverts 35121a2d3d.
This is hard to test for the same reasons as 88f554c74b:
> This is hard to test because to do so robustly requires being able to guaranteed that another test APK will be installed with a known raw resource inside it.
PiperOrigin-RevId: 608270463
This change:
1. Updates the implementation of
`FrameworkMediaDrm.requiresSecureDecoder` to include the
'force allow insecure decoder' workaround logic.
2. Removes the 'force allow insecure decoder' logic from MCR
3. Removes the `requiresSecureDecoder` field from MCR (it can just
be a local derived from `codecDrmSession` now).
PiperOrigin-RevId: 607664186
The behaviour and docs of `playUntilPosition` were changed in
00c7a9bcbb.
This change also affects `playUntilStartOfMediaItem` (since it delegates
to `playUntilPosition`), so the same doc change should also be made
here.
#minor-release
PiperOrigin-RevId: 607364897
Also put back a comment in DownloadTracker that is still relevant on API 19.
Also deprecate PlaceholderSurface.newInstanceV17() in favour of just
newInstance()
VideoFrameProcessor requires every input frame to be registered before they are
rendered to its input Surface. This CL adds the option to register the input
frame only once. This is useful for camera inputs where not all input frames
come with changing FrameInfo.
PiperOrigin-RevId: 606294894
Our previous test video was difficult to use for testing our tone-mapping
algorithm, because it didn't have many different colors. Use a better
video for tone-map tests, by having one with more different colors
PiperOrigin-RevId: 606274843
These allow to set the icon in a standardized way without needing
custom bitmaps or resources. For now, this is just additional
information without backwards-compatible icons or implications.
The same value gets written to the platform session via a new
extras key that can be read and set from sessions not using Media3
yet.
PiperOrigin-RevId: 605670988
The public APIs of these modules reference symbols in some of their
dependencies, so these should be API dependencies, not implementation:
> An API dependency is one that contains at least one type that is
> exposed in the library binary interface, often referred to as its ABI
> (Application Binary Interface).
https://docs.gradle.org/current/userguide/java_library_plugin.html#sec:java_library_recognizing_dependencies
Transformer also uses symbols from `lib-common`, but these are already
an API dep of `lib-exoplayer` so no need to duplicate that here.
PiperOrigin-RevId: 605660621
TestUtil class is more appropriate for the given method.
In the next CL, the only method in FileUtil.java will be moved back
into transformer library and the FileUtil class will be removed.
PiperOrigin-RevId: 605648034
The new API will take both `creation time` and `modification time`.
Till now, Mp4Muxer wrote `modification time` in both
`creation time` and `modification time` field, which was
incorrect.
PiperOrigin-RevId: 605590623
Previously, we missed the BT2020->BT709 color-space conversion.
A user-visible impact of this is that red and green channels used to be
undersaturated, but now are more correctly saturated.
PiperOrigin-RevId: 605411926
Change the file extension retrieval back to how it was before 5488d33da8 to reduce the change of false negatives in `isImage()`
PiperOrigin-RevId: 605281186
`Uri.appendQueryParameter` is documented to encode its arguments, so
calling `Uri.encode` beforehand results in double-encoding.
Issue: androidx/media#1075
#minor-release
PiperOrigin-RevId: 604995441
The track id must be the index in the list of published tracks
as it's used as such elsewhere. This is currently not true if we
skip an empty track as all subsequent tracks get a wrong or even
invalid id.
#minor-release
PiperOrigin-RevId: 604929178
Relax the regex to only check for hyphen which is required by the specification.
Issue: androidx/media#1028
#minor-release
PiperOrigin-RevId: 604719300
Earlier implementation compared the whole file against a golden
data. The new implementation compares only the metadata being tested.
This will avoid updating the golden data when any unrelated change
(unrelated to scenario being tested) is made.
Added a separate test to compare the whole output file against a golden data.
PiperOrigin-RevId: 604692985
The current implementation of `JpegMotionPhotoExtractor.sniff` returns
`true` for any image with Exif data (not just motion photos). Improving
this is tracked by b/324033919. In the meantime, when we 'extract' a
non-motion photo with `JpegMotionPhotoExtractor`, the result is
currently a single empty image track and no video track (since there's
no video, since this isn't a motion photo). This 'empty' image track
is usually used to transmit metadata about the video parts of the
image file (in the form of `MotionPhotoMetadata`), but this metadata is
also (understandably) absent for non-motion photos. Therefore there's
no need to emit this image track at all, and it's clearer to emit no
tracks at all when extracting a non-motion photo using
`JpegMotionPhotoExtractor`.
This change also removes a `TODO` that is misplaced, since there's no
image bytes being emitted here (and never was).
PiperOrigin-RevId: 604688053
These are often the same for image tracks, since we usually drop the
whole image file (both the container and actual encoded image bytes)
into a single sample, but there are cases where we emit a track with
`containerMimeType=image/jpeg` but **no** samples (from
`JpegMotionPhotoExtractor`, to carry some metadata about the image +
video byte offsets).
It's therefore more correct to implement the `supportsFormat` check
based on `sampleMimeType`, so that these 'empty' image tracks are not
considered 'supported' by `ImageRenderer`.
#minor-release
PiperOrigin-RevId: 604672331
This was already done for the TrackGroup ids in <unknown commit>,
but in some scenarios only the Format instances are known and
it's helpful to be able to identify where they came from.
Issue: androidx/media#883
#minor-release
PiperOrigin-RevId: 604644039
This ensures RequestMetadata with just non-null extras is not
considered equal to RequestMetadata.EMPTY. This makes sure the
contents are bundled when a controller sets the extras in a
new MediaItem.
PiperOrigin-RevId: 604632788
Add a test for this consistency in `CompositeSequenceableLoaderTest`,
and also make the
`CompositeSequenceableLoaderTest.FakeSequenceableLoader` implementation
more realistic.
#minor-release
PiperOrigin-RevId: 604604103
This reverses 27caeb8038
Due to the re-ordering of packets done in `CeaDecoder`, there's no way
to use the current implementation to correctly parse these subtitle
formats during extraction (the `SubtitleParser` interface), so we have
to keep the `SubtitleDecoder` implementations.
#minor-release
PiperOrigin-RevId: 604594837
This reverses 94e45eb4ad
Due to the re-ordering of packets done in `CeaDecoder`, there's no way
to use the current implementation to correctly parse these subtitle
formats during extraction (the `SubtitleParser` interface), so we have
to keep the `SubtitleDecoder` implementations.
#minor-release
PiperOrigin-RevId: 604350951
Before supporting transmuxing when both no op effects and regular rotations are set, move setting the muxerWrapper rotation out of shouldTranscodeVideo() to ensure the muxerWrapper rotation is only set at the appropriate times.
This cl also ensures the state between the muxerWrapper and the list of video effects is consistent by clearing the list of videoEffects in trim optimization. If trim optimisation is being applied, then EditedMediItem.effects.videoEffects only contains no-op effects or regular rotations that get be applied in the muxer wrapper. Therefore, we should clear the list of video effects to ensure that no effect gets applied twice.
PiperOrigin-RevId: 604292052
removing the useHdr parameter from queueInputBitmap() it suggests we support changing between HDR and SDR within a stream, which we don't support. instead, identifying whether to use HDR from the shaderprogram which is informed by the inputColorInfo when the stream is registered.
PiperOrigin-RevId: 603681736
Transformer export and ExoPlayer previewing both read inputColorInfo from
registerInputStream now, instead of maintaining a consistent input color
throughout multiple streams in a sequence.
Therefore, we can remove inputColor-related arguments and methods now.
PiperOrigin-RevId: 603423509
If nullness should result in an exception, we should throw as soon as possible,
so that stack traces are easier to follow.
Did a quick scan of media3.effect+transformer and this covers all uses of
peek that immediately throw there.
PiperOrigin-RevId: 603393366
The new test introduced in 45bd5c6f0a is flaky because we only
wait until the media is fully buffered. However, we can't fully
control how much of this data is initially read by the Robolectric
codec and thus the output dump files (containing these codec
interactions) are flaky.
This can be fixed by fully playing the media once and then seeking
back instead.
#minor-release
PiperOrigin-RevId: 603324068
If seeking between last image sample and end of the file where the current stream is not final, then EoS sample will not be provided to `ImageRenderer`. ImageRenderer must still produce the last image sample.
PiperOrigin-RevId: 603312090
The implementation of fragmented MP4 caused a regression where muxer
started writing empty tracks even for non fragmented MP4.
PiperOrigin-RevId: 603091348
VideoFrameProcessor treats BT601 and BT709 as roughly equivalent now, so we
shouldn't be making checks that assume BT709 <-> requires tone-mapping.
Also, the color transfer is a better determinant for tone-mapping than color range, so use just the transfer to determine if tone-mapping is required.
PiperOrigin-RevId: 603083100
The `MediaMuxer` (FrameworkMuxer) supports AV1 mime type from API 34.
For this to work track `Format/MediaFormat` must have `CSD`
data.
Change also include providing AV1 option in demo app.
Transmuxing of an AV1 mime type input will not work because
`Extractor` does not extract AV1 CSD data.
Verified changes manually via demo app.
PiperOrigin-RevId: 603002380
- Added back parsing of scheme data for version 1 as it's technically
allowed by the spec.
- Made constructor of PsshAtom private to only publish the data class
and not the constructor.
- Formatting and Javadoc adjustments
- Additional tests
FileUtil.java can be extended to add more methods for validating
output file. This will be used in Transformer's robolectric tests and
instrumentation tests hence need to move to a common module.
PiperOrigin-RevId: 602789503
Previously, input assets had to be all SDR or all HDR.
After this CL, if tone-mapping is requested, HDR and SDR may mix in any order. If tone-mapping is not requested, SDR may precede HDR, but not vice versa, until SDR to HDR tone-mapping is implemented
Some changes to accomplish this include:
1. Inputting the decoded format's color to VideoFrameProcessor.registerInputStream
for each stream.
2. Calculating the estimated decoded format's color for each stream, by estimating
it based on MediaCodec tone-mapping.
PiperOrigin-RevId: 602747837
* getPcmFormat Util method already sets the mime type as AUDIO_RAW.
* static final AudioFormat consts improve test case readability.
PiperOrigin-RevId: 602740007
This avoids issues that can arise due to the slight discrepancies between
chunk start times (obtained from the manifest of segment index) and
the timestamps of the samples contained within those chunks.
This happens when using `ExoPlayer.setVideoEffects()`.
This CL also fixes the first frame not rendered problem, originally solved in
7e65cce967, but rolled back in 5056dfaa2b because the solution introduces
the flash that is observed in b/292111083.
Before media3 1.1 release, the output size of `VideoFrameProcessor` is not
reported to the app. This was changed later after introducing
`CompositingVideoSinkProvider`, where the video size after processing **is**
reported to the app. After this CL, the size is again, not reported.
PiperOrigin-RevId: 602345087
The updated algorithm has two main improvements:
- The silence padding is not constant but a ratio of the original
silence (up to a defined max) to more naturally represent the
original gaps.
- The silence is not instantly going to zero, but uses a ramp down
and up for a smooth volume transition and also retains a small
percentage of the original noise for more natural "silences" that
still contain some background noise.
#minor-release
Issue: google/ExoPlayer#7423
PiperOrigin-RevId: 602322442
If trim optimisation is being applied, then `EditedMediItem.effects.videoEffects` only contains no-op effects or regular rotations that get be applied in the muxer wrapper. Therefore, we should clear the list of video effects to ensure that no effect gets applied twice.
This fixes requested rotations being applied twice in trim optimization.
Manually tested to ensure all combinations of trimming+muting+rotating works at intended.
PiperOrigin-RevId: 601853203
The return value of onConfigure must not be ignored as it specifies
the output format of the processor, which may be different from the
input format.
#minor-release
PiperOrigin-RevId: 601799440
Many usages are needed to support other deprecations and some
can be replaced by the recommended direct alternative.
Also replace links to deprecated/redirected dev site
PiperOrigin-RevId: 601795998
since we now can support taking in the inputColor upon registering the stream, there is no need to hardcode the image input color anymore. We should now be able to support switching between texture and image input which we couldn't before, but this is untested and not necessary.
PiperOrigin-RevId: 601784149
This allows us to also plumb whether we're doing MediaCodec tone-mapping,
which will be used in a follow-up CL in the ExoAssetLoaderVideoRenderer.
PiperOrigin-RevId: 601774435
It's likely that we will merge these back into their `XXXDecoder`
implementations, but this smaller change allows us to avoid including
these public symbols in the upcoming release.
#minor-release
PiperOrigin-RevId: 601432629
This method works by reflectively loading classes from the `lib-effect`
module, in order to avoid a hard dependency on this module for ExoPlayer
users that don't want video effect functionality. This change ensures
that a failure to load the necessary classes fails immediately, instead
of on a later thread inside `MediaCodecVideoRenderer` when the
reflection currently happens.
Also update the javadoc to make the dependency requirement clear.
#minor-release
PiperOrigin-RevId: 601387957
DASH: `DashMediaSource.Factory` would only propagate it to `DashChunkSource.Factory` -> `BundledChunkExtractor.Factory`
SS: `SSMediaSource.Factory` -> `SsChunkSource.Factory`
HLS: `HlsMediaSource.Factory` -> `HlsExtractorFactory`
Remove nullability of SubtitleParser.Factory across the stack
#minor-release
PiperOrigin-RevId: 601250013
The `SubtitleParser.Factory` is no longer @Nullable and the experimenting toggle is used to enable/disable the use of this factory for subtitle parsing during extraction.
The three places that will hold the "truth" for the `SubtitleParser.Factory` are: BundledChunkExtractor.Factory, SsChunkSource.Factory, DefaultHlsExtractorFactory
DASH: `DashMediaSource.Factory` would only propagate it to `DashChunkSource.Factory` -> `BundledChunkExtractor.Factory`
SS: `SSMediaSource.Factory` -> `SsChunkSource.Factory`
HLS: `HlsMediaSource.Factory` -> `HlsExtractorFactory`
#minor-release
PiperOrigin-RevId: 601151615
As per MP4 spec ISO 14496-12: 8.7.5 Chunk Offset Box, Both "stco" and
"co64" can be used to store chunk offsets. While "stco" supports 32-bit
offsets, "co64" supports 64-bit offsets.
In non fragmented MP4, the mdat box can be extremely large, hence muxer
uses "co64" box.
But for fragmented MP4, muxer does not write any data in this chunk offset
box (present in "moov" box) because all sample related info is present in
"moof" box.
Technically, "co64" box should also work in fragmented MP4because
its empty only but QuickTime player fails to play video if "co64"
box is present in fragmented MP4 output file.
Testing: Verified that QuickTime player does not play video when "co64"
box is present but is able to play when "stco" box is present.
#minor-release
PiperOrigin-RevId: 601147046
Those classes only needed to have access to a `SubtitleParser.Factory` to get a potentially updated `Format` for TrackGroups. The `SubtitleParser.Factory` was only used to check the support for the `mimeType` and getting some cue-related behaviour.
This introduced complexity in a way that both Periods and Extractors needed to have the same `SubtitleParser.Factory` in their individual stacks. To ensure that the sample queue would get the same transcoded/original format.
Instead, now we expose `getOutputTextFormat` methods on `ChunkExtractor.Factory`, `SsChunkSource.Factory` and `HlsExtractorFactory`. Those are the dependencies that Hls/Ss/DashMediaPeriod can make use of to delegate the format-updating logic to.
#minor-release
PiperOrigin-RevId: 601130714
This fix makes output playable on VLC player.
The output does not play on QuickTime player which is being fixed in
a separate CL.
#minor-release
PiperOrigin-RevId: 601118813
Allow setAdditionalRotationDegrees to be called with same rotation after tracks added. This is needed for processes that mux files partially trim optimization so they don't error out after hitting the check state
Manually tested to ensure trim optimization succeeds, automated test added here as well
PiperOrigin-RevId: 601081778
These symbols in `lib-effect` are referenced via reflection from
`CompositingVideoSinkProvider` in `lib-exoplayer` in order to avoid
a hard dependency from `lib-exoplayer` to `lib-effect`. Without this
keep rule, the symbols can get renamed by R8 resulting in the
invocations failing.
#minor-release
PiperOrigin-RevId: 601074636
Populate both `artworkUri` and `artworkData` in
`MediaMetadata.Builder.populate(MediaMetadata)` when at least one of them is non-null.
Issue: androidx/media#964
PiperOrigin-RevId: 600826103
Earlier implementation processed each track (pending sample's buffer info)
individually when writing their corresponding "traf" box in a fragment.
The change involves processing all tracks before start writing "traf" boxes.
#minor-release
PiperOrigin-RevId: 600811093
The default notification provider was still using the legacy
compat MediaStyle instead of our own Media3 one. They are fully
equivalent in their implementation and API and can be swapped out
easily.
PiperOrigin-RevId: 600797920
We keep the previous parsing-during-rendering tests, even though they
can be a bit flaky, because this is an important regression test. The
regression risk is lower for this instrumentation test compared to
robolectric tests with different `ShadowLooper` behaviour.
#minor-release
PiperOrigin-RevId: 600781035
Due to poor isolation between the session tests, in particular the
static state in `MediaSession.SESSION_ID_TO_SESSION_MAP`, an unreleased
session at the end of one test can cause subsequent tests to fail with
obscure errors like `Session ID must be unique`.
#minor-release
PiperOrigin-RevId: 600737697
We can just continue to assume that we don't know the current device.
This case happens on the latest Robolectric release where this method
call isn't implemented yet. As we not generally assume that the
method can throw, this workaround can be removed once Robolectric
is updated again.
#minor-release
PiperOrigin-RevId: 600426851
Add protected method in SimpleBasePlayer for thread verification to help
subclasses verify thread access for additional methods they define and
still report the same message to the user.
Also, remove the DAC link pointing to the ExoPlayer-specific
documentation from the exception message. Users who extend
SimpleBasePlayer have access to the class' javadoc.
PiperOrigin-RevId: 600426692
[Android best
practices](https://developer.android.com/media/optimize/sharing#android_8_81_and_9)
recommend disabling B-frames on API 27, but some devices output B-frames anyway
when H.264/AVC High profile is selected. Add a workaround to force these
devices not to use high profile, to prevent B-frames being output.
`MediaMuxer` likely handles B-frames on these specific devices, but this change
allows the migration to default to in-app muxing to take place without
introducing errors, and it's a temporary workaround until B-frames are properly
supported in the in-app muxer.
PiperOrigin-RevId: 600422238
Some player method calls sent from MediaControllers accept int
or float values with a very clear API contract that disallows
some values. Filtering by these values early avoids calling a
Player implementation with invalid values.
PiperOrigin-RevId: 600413993
This optimization always reports buffers as 'skipped' (i.e. deliberately
not shown), which makes sense for the target case of high FPS content on
a lower refresh rate screen, when lots of the frames will **never** be
shown.
However the optimization also results in reporting buffers as 'skipped'
when decoding is a bit slow, resulting in a frame being released one
vsync late, which then means we have two frames to release in the same
vsync (when the previous vsync was empty). In this case, it would be
more correct to report this as a 'dropped' frame (since it was due to
slow decoding).
Until we can change the logic to distinguish these cases and report them
separately, this CL disables the optimization completely in GTS tests.
This is needed because we often assert there were zero skipped frames,
so slight decoding slowness can cause spurious/flaky test failures (our
threshold for dropped frames is non-zero).
#minor-release
PiperOrigin-RevId: 600406443
This device doesn't seem to be capable of simultaneous encode/decode at this
resolution. We don't have a good way to check the capability (we are already
checking separate decode/encode capability) so just skip this test to save time
triaging its failures.
PiperOrigin-RevId: 600399564
From API 23, we may have a preferred device that is most likely used
as the output device.
From API 24, the AudioTrack tells us the actual routed device that is
used for output and we can listen to changes happening mid-playback.
From API 33, we can directly query the default device that will
be used for audio output for the current attributes.
If the routed device is known by any of the methods above, we can add
more targeted checks in methods like isBluetoothConnected to avoid
iterating over all devices that are not relevant.
The knowledge about the routed device will also be useful to check
advanced output capabilities in the future (e.g. for lossless
playback)
PiperOrigin-RevId: 600384923
This allows us to inject a videoFrameProcessorFactory into
MediaCodecVideoRenderer, without issues about creating the
VideoFrameReleaseControl in the MediaCodecVideoRenderer.
Unfortunately, this does result in more complex CVSP state, where
VideoFrameReleaseControl is no longer final, may be null, and may potentially
change. However, this tries to be careful with assertions to guarantee good
state, and is cleaner than modifying the long-standing MediaCodecVideoRenderer
interface.
Tested that this works on the ExoPlayer demo with setVideoEffects applied, and
using a playlist with SDR->HDR and HDR->SDR items.
PiperOrigin-RevId: 599823412
Although not public, documenting what happens in each state allows for
better code understanding at a glance.
As part of this, refactored the #getProgress method to highlight that
the other options are "non standard".
PiperOrigin-RevId: 599505369
This also fixes issue introduced by frames being released from a prior version of a
GlShaderProgram
Tested by seeking within a playlist with one SDR then one HDR video.
PiperOrigin-RevId: 599475959
Previously, 8f69bb0d9d updated external input (video input)
but not internal input (image/texture input). Update internal input as
well to match.
PiperOrigin-RevId: 599235813
Since mdat box can be huge so there is a provision to use 64 bit size field.
In case of fragmented MP4, individual fragments should not have large mdat box
so a 32 bit size field should be sufficient.
PiperOrigin-RevId: 599219041
When we default to 'parse during extraction', we will flip the default
of this, to ensure that apps know they are using an
incompatible/deprecated flow for subtitle handling.
PiperOrigin-RevId: 599109304
Imported from GitHub PR https://github.com/androidx/media/pull/275
Added below mentioned features.
- Support for extracting DTS LBR(DTS Express) and DTS UHD Profile 2(DTS:X) descriptor ID from PSI PMT
- The DTSReader class is updated for extracting a DTS LBR.
- Newly added DtsUhdReader class for extracting DTS UHD frame.
- The DTSUtil class is updated to parse the DTS LBR or DTS UHD frame and report the format information.
Feature request for ExoPlayer: https://github.com/google/ExoPlayer/issues/11075
Merge 21efa0810db31550d6b215639f9ca2af6a32139a into 104cfc322c
COPYBARA_INTEGRATE_REVIEW=https://github.com/androidx/media/pull/275 from rahulnmohan:dts-mpeg2ts-update 21efa0810db31550d6b215639f9ca2af6a32139a
PiperOrigin-RevId: 598854998
`Surface`s don't expose their size via Java APIs. Recommend apps pass a
`SurfaceView` (which is preferable to `TextureView` as it's more efficient)
or `SurfaceHolder` so they benefit from the player automatically passing the
size down to the video renderer via `MSG_SET_VIDEO_OUTPUT_RESOLUTION`.
PiperOrigin-RevId: 598804258
MatroskaExtractor will no longer be wrapped in SubtitleTranscodingExtractor, but instead use SubtitleTranscodingExtractorOutput under the hood.
FLAG_EMIT_RAW_SUBTITLE_DATA flag will be used to toggle between subtitle parsing during extraction (before the sample queue) or during decoding (after the sample queue).
The new extractor dump files generated by `MatroskaExtractorTest` now follow the new parsing logic and hence have mimeType as `x-media3-cues`.
PiperOrigin-RevId: 598616231
AviExtractor supports text tracks (`AviExtractor.FOURCC_txts` -> `C.TRACK_TYPE_TEXT`) with subtitles.
AviExtractor will no longer be wrapped in SubtitleTranscodingExtractor, but instead use SubtitleTranscodingExtractorOutput under the hood.
FLAG_EMIT_RAW_SUBTITLE_DATA flag will be used to toggle between subtitle parsing during extraction (before the sample queue) or during decoding (after the sample queue).
PiperOrigin-RevId: 598594981
Moves setting bitmapFactory options from BitmapLoader to DatasourceBitmapLoader
BitmapLoader is a general interface for bitmap loading that could use loading implementations other that BitmapFactory, with the rise of Glide being a loader of choice. It's best to correct this interface so that it remains generic
We can't deprecate easily because the other loadBitmap method in that case has a default implementation that relies on the first one, so the change is still breaking. BitmapLoader is public api in common, but it's @UnstableAPI and hasn't been around for very long (be38670391 added it), so it seems this is the best way forward.
PiperOrigin-RevId: 597897098
Subclasses of this component can customize it by wrapping with the
decorator pattern, and a custom `CompositeSequencableLoaderFactory`
allows access to the list of delegate `SequenceableLoader` instances.
The `final` keyword was removed as part of <unknown commit> but this
component never ended up being subclassed within the library.
Making this class `final` makes upcoming changes easier to reason
about.
PiperOrigin-RevId: 597853429
The seek table in a Xing/Info header is very imprecise (max resolution
of 255 to describe each of 100 byte positions in the file). Seeking
using a constant bitrate assumption is more accurate, especially for
longer files (which exacerbates the imprecision of the Info header).
VBR files should contain an Xing header, while an Info header is
identical but indicates the file is CBR.
Issue: androidx/media#878
PiperOrigin-RevId: 597827891
7e65cce967 introduced a regression on ExoPlayer.setVideoEffects()
where there is flash on the screen after the first few frames are shown.
Before 7e65cce967, the first frames of the content are missed, until
MediaCodecVideoRenderer sends the onVideoSizeChanged() callback. The
first frame is processed but not shown, the onVideoSizeChanged() is
triggered and the renderer receives a video output resolution message as
a response from the UI.
7e65cce967 fixed the missed first frames by setting a surface on the
CompositingVideoSinkProvider before the provider is initialized, and as
as result:
- the first frames are rendered
- the MediaCodecVideoRenderer sends the the onVideoSizeChanged() after
frames are shown
- the UI sends a video output resolution change
- the MediaCodecVideoRenderer updates the CompositingVideoSinkProvider
which causes the flash.
The underlying problem is with onVideoSizeChanged() not being
triggered early enough, before the first frame is shown.
Because the flashing is a regression, this commit is reverting
7e65cce967 but keeps the test that was added as ignored, so that the
test is re-enabled as soon as we address the underlying issue.
PiperOrigin-RevId: 597814013
WebvttExtractor will no longer be wrapped in SubtitleTranscodingExtractor, but instead use SubtitleTranscodingExtractorOutput under the hood.
A new constructor will take a boolean parameter to toggle between subtitle parsing during extraction (before the sample queue) or during decoding (after the sample queue).
PiperOrigin-RevId: 597604942
Partially addresses the following TODO, by simplifying the DefaultShaderProgram
API surface.
```
// TODO(b/274109008): Refactor DefaultShaderProgram to create a class just for sampling.
```
PiperOrigin-RevId: 597575575
Originally a `PlayerId` has to be passed and it should be the same id of the player who is going to play the sources, but it turns out to be unnecessary.
When preloading, we can set a `PlayerId.UNSET` inside of the `PreloadMediaSource`, as there is no ongoing playback. And when the source is handed over to player, player will set the player's `PlayerId`.
PiperOrigin-RevId: 597475119
This involves promoting `setTextTranscodingEnabled` to
`ExtractorsFactory`, and also making it experimental, to indicate it's a
short-lived method.
PiperOrigin-RevId: 597235252
Mp4Extractor will no longer be wrapped in SubtitleTranscodingExtractor, but instead use SubtitleTranscodingExtractorOutput under the hood.
FLAG_EMIT_RAW_SUBTITLE_DATA flag will be used to toggle between subtitle parsing during extraction (before the sample queue) or during decoding (after the sample queue).
PiperOrigin-RevId: 597221831
The capabilities change depending on the attributes, so we should
pass down the actual attributes used during playback and update
the capabilities whenever these change.
PiperOrigin-RevId: 597197507
This means in a later change we can still use some of the info for CBR
files, even if we want to ignore the imprecise table of contents and
seek based on a constant bitrate assumption instead.
PiperOrigin-RevId: 597193997
An audio file can only play sound between two PCM samples (the 'start'
and 'end' of section of a wave form). Therefore when calculating
duration from a count of PCM samples we need to subtract one first (the
'end' sample which has no duration of its own).
This only changes durations by one PCM sample (21us - 22us for 44.1kHz sample
rate).
PiperOrigin-RevId: 596990306
The extractor knows the PCM encoding of the losslessly
encoded data in the samples and should set it in the
Format to allow downstream components to use this information.
PiperOrigin-RevId: 596974863
We introduce SubtitleParser.Factory that supports no formats to be used FragmentedMp4Extractors that will not do any subtitle parsing on the extraction side. We also slowly move away from using SubtitleTranscodingExtractor to SubtitleTranscodingExtractorOutput (hence making it public).
This is required by individual Extractor impls so that they can start using SubtitleTranscodingExtractorOutput rather than be wrapped by SubtitleTranscodingExtractor. The latter is to be deprecated after all the subtitle related Extractors have achieved this migration.
PiperOrigin-RevId: 596942147
Whenever the inputColorInfo updates, update the samplingGlShaderProgram.
Also, allow either SDR or gamma2.2 to be used for HDR->SDR tone-mapping
`outputColorInfo` request. This is required because we can't update the
`outputColorInfo`, but plan to always use gamma2.2 for `outputColorInfo` in the
future.
This allows VideoFrameProcessor to work as is for exoplayer previewing, but
only when not seeking. As we haven't plumbed the per-stream inputColorInfo from
ExoPlayer down to VFP.registerInputStream, follow-up CLs will be needed to
properly support previewing with changing inputColorInfo.
PiperOrigin-RevId: 596627890
This behavior was previously available as opt-in via
`MediaItem.DrmConfiguration.Builder.setPlayClearContentWithoutKey` and
`DefaultDrmSessionManager.Builder.setPlayClearSamplesWithoutKeys`. This
change flips the default of both these properties to true.
This should speed up the time for playback to start when playing DRM
content with a 'clear lead' of unencrypted samples at the start.
Previously playback would wait until the keys for the later encrypted
samples were ready. The new behaviour could result in mid-playback
stalls/rebuffers if the keys are not ready yet by the transition from
clear to encrypted samples, but this is not really a regression since
previously playback wouldn't have started at all at this point.
PiperOrigin-RevId: 595992727
This file is CBR encoded with LAME, so it has an `Info` header (the CBR
equivalent to `Xing`).
A follow-up change will use this file in `Mp3ExtractorTest`.
Issue: androidx/media#878
PiperOrigin-RevId: 595938327
These methods sound similar, but have different behaviour. This change
tries to make the distinction clearer, and sign-post from one to the
other.
#minor-release
Issue: androidx/media#910
PiperOrigin-RevId: 595701540
When the media notification controller is requested for a session
with `getConnectedControllerForSession` and the `Future` is not null
but not yet completed, the `Future` was returned either way. This was
reported as creating a race condition between the notification
being requested for update the very first time, and the media
notification controller having completed connecting to the session.
Returning null from `getConnectedControllerForSession` when the
`Future` is available but not yet done fixes the problem. This is
safe because for the case when a notification update is dropped,
the media notification controller will trigger the update as soon
as the connection completes.
Issue: androidx/media#917
#minor-release
PiperOrigin-RevId: 595699929
`Cea608Parser` and `Cea708Parser` don't currently work correctly on
their own without the re-ordering of input buffers implemented in
`CeaDecoder`, and it's not clear how we can properly do this re-ordering
during extraction. This change ensures that if 'parse subtitles
during extraction' is enabled, CEA-6/708 subs will be passed through
without transcoding and can then be decoded during rendering by
`Cea6/708Decoder`.
PiperOrigin-RevId: 595658628
Invalid frames have no impact on ExoPlayer ability to play the media and should not fail on errors.
Some tools can add 100Mb images in the tags that will trigger recoverable OOM with this fix.
The `Cea608DecoderTest` added here fails if re-ordering is removed from
`CeaDecoder`.
The `Cea608ParserTest` is added with `@Ignore` because there's currently
no re-ordering support in this part of the subtitle handling pipeline
(partly because there's no concept of 'current playback time', meaning
it's hard to know **when** to re-order).
PiperOrigin-RevId: 595320205
When the 'when' timer of the notification is disabled
`DefaultMediaNotificationProvider` may set `C.TIME_UNSET`
as the time. Users reported problems on some devices with
this and the docs ask for an event time that probably
shouldn't be a negative number.
This change sets `0L` instead of `C.TIME_UNSET` when the
timer is disabled.
Issue: androidx/media#903
#minor-release
PiperOrigin-RevId: 594451074
Changes includes;
1. Public API to enable fMP4 and to pass fragment duration.
2. Added `FragmentedMp4Writer`.
3. Added logic to create fragments based on given fragment duration.
4. Write "moov" box only once in the beginning.
3. Add all the required boxes for current implementation.
4. Unit tests for all the new boxes.
5. E2E test for generating fMP4.
Note: The output file is un seek-able with this first implementation.
PiperOrigin-RevId: 594426486
Mp4Muxer does not support out of order B-frames. Currently it
silently writes out of order B-frames, producing an invalid file (with
negative sample durations).
Although `Mp4Extractor` is somehow able to process this invalid file and
`Exoplayer` is able to play it but that is unexpected.
The `sample.mp4` test file contains B frames. Other test files does not
contain `H264 video + AAC audio` format hence created a new test file by
running `sample.mp4` via `Transformer` after applying some effects.
PiperOrigin-RevId: 594016144
This more closely matches the intended, documented behaviour of this
method. The previous implementation was incorrectly checking
`sampleDataEnd + newSampleSize`, but it's more correct to compare
`existingSampleDataLength + newSampleSize`, in order to handle the
case of non-zero `sampleDataStart`. We've already checked above whether
`newSampleSize` fits after `sampleDataEnd` without growing or
reshuffling the array, so no need to basically repeat that check.
In the case of handling one sample at a time, the previous
implementation resulted in `sampleData` growing by `sampleSize`
**every time** the pointers reached the end. With the new check for
`sampleDataStart == sampleDataEnd`, this is avoided by always writing
each new sample at the start of the array (because the previous sample
has already been consumed), meaning `sampleData` remains equal to
`sampleSize` without growing.
PiperOrigin-RevId: 593119927
These were missing in 5211ff0dc1
Without these, the `": null"` is still logged (but the `"Unexpected
IllegalStateException"` bit is not), because the **whole** string
concatenation is compared to null as the boolean condition of the
ternary, and this condition is always false (the result of a string
concatentation is never `null`) i.e. (with excess parentheses for
clarity):
```
(("Unexpected " + cause.getClass().getSimpleName() + cause.getMessage()) != null)
? ": " + cause.getMessage()
: ""
```
Also add a test because obviously this isn't as simple as I'd thought.
PiperOrigin-RevId: 593079975
The previous code led me to misread this stack trace as a null pointer
exception, but it's really an index out of bounds exception:
```
Caused by: androidx.media3.exoplayer.upstream.Loader$UnexpectedLoaderException: Unexpected IllegalArgumentException: null
at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:435)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at java.lang.Thread.run(Thread.java:1012)
Caused by: java.lang.IllegalArgumentException
at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:40)
at androidx.media3.common.util.ParsableByteArray.setPosition(ParsableByteArray.java:164)
at androidx.media3.extractor.text.cea.Cea608Parser.parse(Cea608Parser.java:440)
```
PiperOrigin-RevId: 592876546
Instead, for input videos, use the colorInfo provided by the extractor. Similarly, for input images, use sRGB, the only color currently in use.
Textures do still need the input ColorInfo provided though.
PiperOrigin-RevId: 592875967
Replace the event for notifying fallback to cover codec initialization in
general (but keeping the list of errors).
Add a flag to control whether to try non-primary codecs, with the same
documentation as the similar flag in ExoPlayer's renderer.
Make the class final as it shouldn't be necessary to subclass it.
PiperOrigin-RevId: 592869868
This is similar to the HLS fix in 770ca66fbc
Similar to HLS, the original problem here was **not** modifying the
`Format` for caption tracks
embedded into the video stream. I tried just updating the format in
both places, but that caused new failures because the new
('transcoded') format was then fed into `FragmentedMp4Extractor` as
part of `closedCaptionFormats`, which resulted in the CEA-608 data
being emitted from `FragmentedMp4Extractor` with the incorrect
`application/x-media3-cues` MIME type (but the bytes were actually
CEA-608), meaning the transcoding wrapper passed it through without
transcoding and decoding failed (because obviously CEA-608 bytes can't
be decoded by `CueDecoder` which is expecting a `Bundle` from
`CuesWithTiming.toBundle`.
To resolve this we keep track of the 'original' caption formats inside
`TrackGroupInfo`, so we can feed them into `FragmentedMp4Extractor`.
For all other usages in `DashMediaPeriod` we use the 'transcoded'
caption formats.
PiperOrigin-RevId: 592866262
With fMP4 implementation there will be two writers `DefaultMp4Writer`
and `FragmentedMp4Writer`.
Changes includes:
1. Make Mp4Writer as an abstract class and keep only common functionality
into it.
2. Create a DefaultMp4Writer which contains existing logic to write MP4.
3. The fMP4 logic needs to access `pending sample buffer info` at various
places, so did refactoring to split List<Pair<BufferInfo, ByteBuffer>>
into two separate lists.
PiperOrigin-RevId: 592861398
Checking the output format's mime type may skip tests more often than we'd like,
because we may desire using a lower-spec output mimetype than what's passed in, if
based on the input's HDR mimetype value.
Therefore, update this output format to null, for tone-mapping tests
PiperOrigin-RevId: 592855713
Also updated the `README` file to accurately specify the use of NDK r23c and the default setting `ANDROID_ABI=21` for NDK r26b.
PiperOrigin-RevId: 592845796
Issues with the current implementation
1. The implementation is unnecessarily complicated and can be
easily simplified.To make all the tracks start from the same time,
its only the first sample that require some timestamp adjustments
but the current implementation shifts all the timestamps. Since method
calculates the `sample duration`, shifting all the timestamps has no effect
as such.
2. The implementation always forced first sample to start at 0. But when we
want to use same method for `Fragmented MP4` then it will look inaccurate
as we will call this method for different `fragments` and each `fragment`
will not start from 0 presentation time. Although the output will be same
since this method returns `duration` and not the `timestamps`.
3. As per previous implementation if there is just one sample then
its duration is made equals to its presentation time, which looks incorrect.
With new changes, if a single sample is passed then its duration will always
be 0 irrespective of specified last sample duration behaviour.
PiperOrigin-RevId: 592826612
getAudioCapabilities currently creates the receiver and returns
the current capabilities. This is error-prone because the
capabilities are also available as a class field.
This can made cleaner by letting the method just create the receiver
and all access to the capabilities can be made via class field.
PiperOrigin-RevId: 592591590
HLS distinguishes between 'subtitles' (WebVTT or TTML distributed in
separate files with their own playlist) and 'captions' (CEA-608 or 708,
distributed muxed into the video file).
The format transformation added in 7b762642db
only applies to subtitles and not captions. This change makes the same
transformation for caption formats.
This resolves an error like:
```
SampleQueueMappingException: Unable to bind a sample queue to TrackGroup with MIME type application/cea-608.
```
Also add two playback tests for HLS CEA-608, one that parses during
decoding (old way) and one during extraction (new way). Adding these
tests is what alerted me to this issue.
PiperOrigin-RevId: 592571284
This was generated by combining the existing `ts/bbb_2500ms.ts` test
asset and a temporary `.srt` file using
https://cloud.google.com/transcoder/docs/how-to/captions-and-subtitles
This doesn't directly reproduce the problem fixed by
7ca26f898d,
because the CEA-608 subs are structured differently to the stream I
discovered the problem with (from Issue: androidx/media#887). However this test
does fail if that fix is reverted after
486230fbd7.
I'm also not able to repro the character duplication reported in
Issue: androidx/media#887 by just changing the manifest in this CL. I'm not yet
sure on the exact differences between the stream provided on GitHub
and this stream.
This stream does provide some regression protection, because it
currently fails with 'new' subtitle parsing
(`DashMediaSource.Factory.experimentalParseSubtitlesDuringExtraction(true)`),
though I'm not sure on the exact reason for that yet.
PiperOrigin-RevId: 592476328
This is a refactoring that allows the `MediaPeriodQueue` to
create media period holders without all the collaborators
being passes in to `enqueueNextMediaPeriodHolder(...)`.
The factory is presumably also helpful in unit test to
know whether and when exactly a holder is created in
the preloading process.
PiperOrigin-RevId: 592301400
We deliberately left `Cea608Decoder` and `Cea708Decoder` intact when
creating the respective `SubtitleParser` implementations (in
27caeb8038
and
94e45eb4ad
respectively).
However we didn't correctly change the behaviour of
`SubtitleDecoderFactory.DEFAULT` in order to use these 'legacy'
implementations. We firstly left the `Cea608Decoder` instantiation
**after** the `DefaultSubtitleParserFactory.supportsFormat()` check,
meaning that code would never be reached (since `supportsFormat` now
returns true for CEA-608). Then in the second change (which was supposed
to only be about CEA-708) we removed all instantiations of **both**
`Cea608Decoder` and `Cea708Decoder`. This change puts the decoder
instantiations back and moves them above the
`DefaultSubtitleParserFactory.supportsFormat` check, meaning they will
be used in preference to `DelegatingSubtitleDecoder`.
This resolves the immediately-disappearing subtitles tracked by
Issue: androidx/media#904.
PiperOrigin-RevId: 592217937
While investigating Issue: androidx/media#887 I naively assumed the CEA-608
captions were in a TS file, but they're actually in an MP4 (which is
possibly obvious given DASH only supports MP4). This change includes
container info in the `EventLogger` `tracks` output.
PiperOrigin-RevId: 592192752
This allows us to remove the additional thread we create
for asynchronous buffer queuing and use a synchronized
queueing approach again.
This API looks strictly beneficial on all tested devices,
but since this code path in MediaCodec has not been used
widely, we leave an opt-out flag for now.
PiperOrigin-RevId: 591867472
The parameters may change the decoding behavior of the
following samples and it's suprising/wrong to apply them
from another thread where we can't guarantee from which
sample they apply.
PiperOrigin-RevId: 591808760
Move `inputColorInfo` from `VideoFrameProcessor`'s `Factory.create` to `FrameInfo`,
input via `registerInputStream`.
Also, manually tested on exoplayer demo that setVideoEffects still works.
PiperOrigin-RevId: 591273545
When an adaptive source has been preloaded, the track selection
of the player should possibly not override the preloaded selection
to avoid discarding preloaded data.
PiperOrigin-RevId: 591256334
The number of empty image byte arrays written is one less than the
total number of tiles in the image. The empty byte arrays act as
placeholders for individual tiles inside ImageRenderer.
PiperOrigin-RevId: 591231432
This is because currently
1. Player sets a surfaceView to render to
2. Player intializes the renderer
3. MCVR initializes the VideoSinkProvider, by extension VideoGraph
But when 1 happens, MCVR doesn't set the surfaceView on the VideoGraph because
it's not initialized. Consequently after VideoGraph is initialized, it doesn't
have a surface to render to, and thus dropping the first a few frames.
Also adds a test for first frame to verify the correct first frame is rendered.
PiperOrigin-RevId: 591228174
When track is changed during playback, `playbackPositionUs` may be in middle of a chunk and `loadPositionUs` should be the start of that chunk. In this situation `loadPositionUs` can be less than the current `playbackPositionUs`, resulting into negative `bufferedDurationUs`. It translates to having no buffer and hence we should send `0` for `bufferedDurationUs` when creating new instances of `CmcdData.Factory`.
Issue: androidx/media#888
#minor-release
PiperOrigin-RevId: 591099785
Otherwise, there's a memory leak of ~30MB, as this is never released.
This likely used to be considered released as part of what now became
`intermediateGlShaderPrograms`, but its release was missed after we split
`finalShaderProgramWrapper` out from the larger glShaderProgram list.
PiperOrigin-RevId: 590954785
To prepare to move `inputColorInfo` from `VFP.Factory.create` to
`VFP.registerInputStream`, move all usage of `inputColorInfo` to be *after*
`registerInputStream`.
To do this, defer creation of `externalShaderProgram` instances, which require
`inputColorInfo`. However, we must still initialize `InputSwitcher` and OpenGL
ES 3.0 contexts in the VFP create, to create and request the input surface from
ExternalTextureManager.
PiperOrigin-RevId: 590937251
The timestamp adjuster also estimates the number of wraparounds
of the 90Khz TS timestamp. It does that by assuming that a new
timestamp is always close to the previous one (in either direction).
This logic doesn't always work for duration estimates because the
timestamp at the end of the media is not close to the one at the
beginning and it may also never be less than the one at the beginning.
This can be fixed by introducing a new estimation model that assumes
the new timestamp is strictly greater than the previous one without
making the assumption that it has to be close to it.
Issue: androidx/media#855
#minor-release
PiperOrigin-RevId: 590936953
When broadcasting a notifyChildrenChanged event, the task for legacy
controllers was sent to the broadcasting callback. This would
technically work, but because the subscription list is maintained
with specific controllers, the broadcast controller isn't subscribed
and hence the call wasn't executed.
This change calls the overloaded method for a specific controller
for each connected controller. Making sure (only) subscribed
controllers are notified.
Issue: androidx/media#644
PiperOrigin-RevId: 590904037
Vertical and horizontal tile counts inside Format used by ImageRenderer
used to be required to equal 1 and were set to 1 regardless of what
the manifest said.
This change removes the above requirement and sets the tile counts to
the values from the manifest.
PiperOrigin-RevId: 590608353
Using 0 as the unset prepare position is the root cause of a number of issues,
as outliine in the ExoPlayer issue https://github.com/google/ExoPlayer/issues/7975
The premise of this fix is that once the prepare override is used (the initial call
to `selectTracks()`) it is never needed again, so simply invalidate it after use.
Despite GL 3.0 not being required on API 29+, it is experimentally
determined to always be supported on our testing devices, on API 29+.
That said, still fall back to OpenGL 2.0 if 3.0 is not supported,
just in case.
PiperOrigin-RevId: 590569772
It's better to use the generic Android resource scheme which
is at least as powerful as our own one.
Issue: androidx/media#868
PiperOrigin-RevId: 590168919
`Mp4Writer` receives all the tracks/samples and then it adds them to
corresponding `Track` object. As the `Track` object is primarily to hold
track specific data and `Mp4Writer` is responsible for interleaving related
logic, the call to `doInterleave` should be with `Mp4Writer`. The `Track`
object should just act like a data holding class.
This will also help in reusing `Track` class for fragmented MP4 implementation.
PiperOrigin-RevId: 590138002
An app that uses the `MediaSession` without a `MediaSessionService` may
want to connect a media notification controller on it own. To avoid apps
using a string literal, the key should be exposed as public.
PiperOrigin-RevId: 589875603
The MP4 data in JPEG motion photos can contain multiple `video/hevc` tracks, but only the first is at a playable frame rate while the others are low-fps, high-res tracks designed for specific use-cases (not direct video playback).
ExoPlayer currently selects the unplayable track by default, because it
has a higher resolution. This change introduces a flag to
`Mp4Extractor` that results in the first video track being marked as
`ROLE_FLAG_MAIN`, and all subsequent video tracks `ROLE_FLAG_ALTERNATE`
- this then results in the playable lower-res track being selected by
default.
PiperOrigin-RevId: 589832072
There is a bug in CompositionVideoSinkProvider where the rotation
effect is applied only on the first video and the same effect
is applied on subsequent videos without checking if the next
items in the playlist must be rotated.
The bug applies only on devices with API < 21.
PiperOrigin-RevId: 589823797
We currently pause playback to prevent further progress while the
app thread runs assertion and triggers additional actions. This is
not ideal because we do not actually want to pause playback in
almost all cases where this method used.
This can be improved by keeping the playback thread blocked and only
unblock it the next time the app thread waits for the player (either
via RobolectricUtil methods or by blocking the thread itself). To
add this automatic handling, this change introduces a new util class
for the tests that can keep the list of waiting threads statically
(because the access to this logic is spread across multiple independent
classes).
PiperOrigin-RevId: 589784204
Move the `AnalyticsListener` implementation to a private inner class.
This avoids polluting the public API of `ExoHostedTest`, especially as
it's designed to be extended.
PiperOrigin-RevId: 589188113
Also adds an alternate way to configure the AudioGraph.
Apps should no longer need to ensure that inputs have the same sample
rate.
PiperOrigin-RevId: 588747431
Added a new ABANDONED option so one can tell the difference between when the optimization has been requested but not applied vs not requested at all. also changed the ordering do better represent the hierarchy of failure modes
PiperOrigin-RevId: 588720513
Fix a bug where CompositingVideoSinkProvider.isInitialized() returns
true even after releasing the CompositingVideoSinkProvider.
PiperOrigin-RevId: 588481744
This change moves the instantiation of the CompositingVideoSinkProvider
out of MediaCodecVideoRenderer so that the composition preview player can
re-use the CompositingVideoSinkProvider instance for driving the rendering of
images.
The main point of the change is the ownership of the
VideoFrameReleaseControl, which decides when a frame should be rendered
and so far was owned by the MediaCodecVideoRenderer. With this change,
in the context of composition preview, the VideoFrameReleaseControl
is no longer owned by MediaCodecVideoRenderer, but provided to it.
This way, the CompositingVideoSinkProvider instance, hence the
VideoFrameReleaseControl can be re-used to funnel images into the
video pipeline and render the pipeline from elsewhere (and not
MediaCodecVideoRenderer).
PiperOrigin-RevId: 588459007
The serializable form is used when we need to serialize the result into
bytes in the sample queue. The binder-based (ultimately
filedescriptor-based) form is used for
session/controller IPC, in order to avoid sending the bitmap bytes over
the IPC.
Issue: androidx/media#836
#minor-release
PiperOrigin-RevId: 588420836
This helped me debug which bitmap was actually failing. Otherwise, you need to clear all bitmaps on the device relating to this test, then adb pull all bitmaps and see the last uploaded one to see, which is much more confusing.
PiperOrigin-RevId: 588374081
This image has two video tracks in the MP4 data, one is a 'real' video
which we want to play by default, and the other is a low-fps video track
which isn't intended to be directly played, it's encoded in HEVC for
compression and decoding efficiency.
This test demonstrates ExoPlayer's current behaviour default extraction
and playback, which results in selecting the high-res, low-fps track
(actually single sample in this example), instead of playing the actual
video.
PiperOrigin-RevId: 588068908
As Opus decoders skip some bytes prior to playback during a seek, the renderer for bypass playback should send samples to the decoder even if they would be decode-only. However, the renderer should not send samples with time preceding that range. This change adds that constraint.
#minor-release
PiperOrigin-RevId: 588014983
Both the extension OPUS decoder and the OMX/C2 MediaCodec
implementations for OPUS and VORBIS decode into the channel
layout defined by VORBIS. See
https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-140001.2.3
While this is technically correct for a stand-alone OPUS or VORBIS
decoder, it doesn't match the channel layout expected by Android.
See https://developer.android.com/reference/android/media/AudioFormat#channelMask
The fix is to apply the channel mapping after decoding if needed.
Also add e2e tests with audio dumps for the extension renderer,
including a new 5.1 channel test file.
Issue: google/ExoPlayer#8396
#minor-release
PiperOrigin-RevId: 588004832
`mediaCrypto` is initialized before `codec` in
`maybeInitCodecOrBypass`. Before this change, it was possible for
`maybeInitCodecOrBypass` to complete with `mediaCrypto != null` and
`codec == null`, in particular if it was run as part of clearing the
player surface (since in that case, no video codec is initialized).
This inconsistent state then causes issues during a later invocation of
`maybeInitCodecOrBypass`, when `mediaCrypto` is still non-null, and
`mediaCryptoRequiresSecureDecoder = true`, but the
content has been changed to unencrypted with no associated DRM session.
This results in a playback error, because a secure decoder is
initialized but there's no DRM session available to work with it.
This change ensures that when `maybeInitCodecOrBypass` completes,
either both `mediaCrypto != null` and `codec != null` (i.e. codec
initialization was completed) or `mediaCrypto == null` and
`codec == null` (i.e. codec initialization was not completed). We also
ensure that when nulling out `mediaCrypto` we also set
`maybeInitCodecOrBypass = false`. A later change should be able to
demote `maybeInitCodecOrBypass` from a field to a local in order to
remove any risk of that part of state becoming out of sync. This
resolves the issue, because during the second invocation of
`maybeInitCodecOrBypass` an insecure decoder is now (correctly)
initialized and the unencrypted content is successfully played.
#minor-release
PiperOrigin-RevId: 587713911
We set the `playerId` only in the constructor currently. But in the case of this source doesn't have `preload` called, the player's release of this source will set the playerId to `null`, which makes this source un-reusable with a null `playerId`.
PiperOrigin-RevId: 587698214
* @Before and @After usage. [1].
* static fields before non-static. [2].
@Before method should typically be paired with an @After method,
focused on ensuring that the component is released regardless of what
the test does.
In tests, inlining final class variables is preferrable [1]. In general
things like the file path should be part of the test (the @Rule means
we don't need before/after) if only used once.
Statically importing values and using them directly is preferable to
having a variable declared as non-final that's effectively final,
because from a readability perspective someone can see (the caps) that
the value is final static and immutable, so doesn't have to check if
it's changed/reassigned.
PiperOrigin-RevId: 586697887
Before, a translucent overlay over an opaque video would result in a
translucent output. This is not consistent with physical properties of light
(if putting a translucent object in front of an opaque object, you can't see
behind the opaque object).
Using the mixing properties from DefaultVideoCompositor.
PiperOrigin-RevId: 586636275
`fromBundle` doesn't distinguish between `FIELD_BITMAP` and `FIELD_TEXT`
being present with a null value, or being absent, so we might as well
avoid including them when the value is null.
I've separated this from a later change to add
`Cue.toSerializableBundle` which will also skip setting a bitmap value
into the `Bundle` if `this.bitmap == null`. This is partly because it
results in changes to a lot of extractor test dump files, and it's
easier to review that as a separate change.
PiperOrigin-RevId: 586626141
Add an check when loading progressive media in case the load
is canceled. If the player is released very early, the progressive
media period may carry on with the initial loading unnecessarily.
PiperOrigin-RevId: 586288385
In `TextOverlay` and `DrawableOverlay`, treat `Bitmap` as a buffer, where we
allocate it rarely and reuse it as long as possible before making a new one.
In `BitmapOverlay`, avoid allocating GL textures too often as well.
Strongly reduces allocations and memory usage growth (saving ~100-150 MB on 4k60fps
at high end), at the cost of more code complexity and low-end using 70MB more, on
1/1 comparisons.
PiperOrigin-RevId: 585990602
In `ExternalTextureManager` fields are accessed from the GL thread and the class needs to be constructed on the GL thread.
Also visibly document threading requirement in the parent class.
PiperOrigin-RevId: 585941284
The issue that motivated adding this (frames unexpectedly being dropped by the
decoder) has been addressed, so we can turn off the logging to reduce
unnecessary allocations during transformation. We can easily turn on debug
logging in future as needed by setting `DebugTraceUtil.DEBUG = true`.
Also avoid allocations for string building at logging call sites by passing a
format string for extra info. Formatting the string now only happens when
debugging is turned on.
Tested manually by running transformations in the new state (DEBUG = false) and
with debugging turned on.
PiperOrigin-RevId: 585666349
Serializing bitmap cues is currently broken, but this test is
incorrectly passing. This change makes two changes to introduce the same
failure (both changes are necessary, each one alone still passes):
1. Move from Robolectric to an instrumentation test.
2. Trigger the `Bitmap` to be serialized using a file descriptor, either
by calling `Bitmap.asShared` in the test when constructing the `Cue`,
or constructing the `Bitmap` from a 'real' image byte array instead a
1x1 token image.
Issue: androidx/media#836
PiperOrigin-RevId: 585643486
The decoder and encoder won't accept high values for frame rate, so avoid
setting the key when configuring the decoder, and set a default value for the
encoder (where the key is required).
Also skip SSIM calculation for 4k, where the device lacks concurrent decoding
support.
PiperOrigin-RevId: 585604976
At least some Android 14 devices still have the same invalid URL issue
when using ClearKey DRM, so might as well apply the check to newer
versions of Android as well.
Using `Integer.MAX_VALUE` risks causing arithmetic overflow in the codec
implementation.
Issue: androidx/media#810
#minor-release
PiperOrigin-RevId: 585104621
We don't get the buffer back after decoding to make a proper decision
about whether dropping the buffer is needed, so we do the next best
thing and tell the codec to drop the buffer if its input timestamp is
less than the intended start time.
PiperOrigin-RevId: 584863144
Move the reflective loading of VideoFrameProcessor from
MediaCodecVideoRenderer to the CompositingVideoSinkProvider. This is so
that all reflective code lives in one place. The
CompositingVideoSinkProvider already has reflective code to load the
default PreviewingVideoGraph.Factory.
PiperOrigin-RevId: 584852547
These usages have no need for the double ended input functionality. All
other usages across media3 are ConcurrentLinkedQueue.
PiperOrigin-RevId: 584841104
In OverlayShaderProgram, this method is used quite a lot, and is the only method from Util.java in this file. Marginally reduce complexity by using a static import instead.
PiperOrigin-RevId: 584828455
Reduce short-lived allocations of potentially large objects, like Bitmap.
Unfortunately, this does make the TextureOverlay interface more messy though, requiring a way to signal whether the texture should be flipped vertically.
PiperOrigin-RevId: 584661400
These changes are also compatible with FFmpeg 5.1, which is now minimum supported version.
Also set -Wl,-Bsymbolic flag via target_link_options command which is more correct.
Split CompositingVideoSinkProvider.VideoSinkImpl in two classes:
- VideoSinkImpl now only receives input from MediaCodecVideoRenderer and
forwards frames to its connected VideoFrameProcessor
- VideoFrameRenderControl takes composited frames out of the VideoGraph
and schedules the rendering of those.
- CompositingVideoSinkProvider connects VideoSinkImpl with
VideoFramesRenderer.
PiperOrigin-RevId: 584605078
This change adds `MediaController.getSessionExtras()` through
which a controller can access the session extras.
The session extras can be set for the entire session when
building the session. This can be overridden for specific
controllers in `MediaSession.Callback.onConnect`.
PiperOrigin-RevId: 584430419
This change fixes a bug with seeking forward in MIDI. When seeking forward,
the progressive media period attempts to seek within the sample queue, if a
key-frame exists before the seeking position. With MIDI, however, we can
only skip Note-On and Note-Off samples and all other samples must be sent
to the MIDI decoder.
When seeking outside the sample queue, the MidiExtractor already
instructs the player to start from the beginning of the MIDI input. With
this change, only the first output sample is a key-frame, thus the
progressive media period can no longer seek within the sample queue and
is forced to seek from the MIDI input start always.
Issue: androidx/media#704
#minor-release
PiperOrigin-RevId: 584321443
The live offset override is used to replace the media-defined
live offset after user seeks to ensure the live adjustment adjusts
to the new user-provided live offset and doesn't go back to the
original one.
However, the code currently clips the override to the min/max
live offsets defined in LiveConfiguration. This is useful to
clip the default value (in case of inconsistent values in the media),
but the clipping shouldn't be applied to user overrides as
the player will then adjust the position back to the min/max
and doesn't stay at the desired user position.
See 2416d99857 (r132871601)
#minor-release
PiperOrigin-RevId: 584311004
Composition and EditedMediaItemSequence don't allow empty lists in their main
constructors, so shouldn't the vararg API. This is more inline with Effective
Java item 53.
PiperOrigin-RevId: 583415124
When transmuxing, the `EncodedSampleExporter` maintains a queue of input
buffers that get filled with encoded data by the asset loader. The number of
buffers was limited to avoid using more and more memory if producer (asset
loader) gets far ahead of the consumer (exporter).
Previously this limit was fixed at 10 buffers, but increasing the number of
buffers can make some transmux operations much faster. Allow allocating between
a min and max number of buffers, and also set a target allocation size beyond
which new buffers can't be allocated. This allows audio formats which require
many small buffers to be processed more quickly, while preventing allocating
too much memory for hypothetical very high bitrate formats.
'Remove video' edits on local videos in particular get much faster, because
audio buffers are very short and there are lots of them. With a sample 10
minute video, a 'remove video' edit took 2 seconds (36 seconds before this
change). With a sample 1 minute removing video took 0.25 seconds after this
change (2.5 seconds before).
The speed improvement is smaller for other types of edits that retain the video
track. Transmuxing a 10 minute video retaining the video track took 26 seconds
(40 seconds before).
PiperOrigin-RevId: 583390284
In some contexts (e.g. BottomSheetDialogFrament), Material Design
themes will override the default of singleLine=false to true. This
causes layout problems because the forward/rewind buttons are no
longer visible with singleLine=true.
This problem can be avoided by explicitly requesting the default
value of false in our layout files.
Issue: androidx/media#511
#minor-release
PiperOrigin-RevId: 582604131
This removes the flakiness of the HLS playback test as well.
Previously, this flow only worked for standalone WebVTT subtitles (7b762642db)
PiperOrigin-RevId: 582359383
The test is hidden behind the `@Ignore` annotation due to some
flakiness. However, it will be removed when the subtitle parsing is
moved to a pre-sample-queue architecture.
The test media was created with:
```shell
$ cp ../../dash/standalone-ttml/sample.xml sample.ttml
$ MP4Box -add sample.ttml sample.text.mp4
$ MP4Box -frag 10000 sample.text.mp4
$ rm sample.ttml
```
PiperOrigin-RevId: 582347113
This helps to play ads where the file extension is not sufficient
to deduce the correct source in DefaultMediaSourceFactory.
PiperOrigin-RevId: 582340114
Currently, ads are only defined by a single URL, which makes it
impossible to define additional fields needed to play ads correctly.
This can be fixed by using a full MediaItem in AdPlaybackState,
replacing the previous Uri field.
PiperOrigin-RevId: 582331588
Use renderengine's PQ to SDR tone-mapping implementation instead
of naive implementation from before.
This improves luminance on highlights, as seen in the test image.
PiperOrigin-RevId: 582318045
MIME types are case-insensitive, but none of the many existing
comparisons across our code base take this into account. The
code can be made more robust by normalizing all MIME types at the
moment they are first set into a class/builder and adding toLowerCase
as part of the normalization.
Most concretely, this fixes an issue with playing HLS streams via
the IMA SDK where the stream MIME type is indicated with all lower
case "application/x-mpegurl", which failed the MIME type comparison
in DefaultMediaSourceFactory.
PiperOrigin-RevId: 582317261
All methods in VideoFrameProcessor are expected to be called by the owning thread,
as far as I understand (vs. 10 threads each queuing frames/textures/streams, which
invalidates blocking done by registerInputStream and flush)
PiperOrigin-RevId: 582295240
BundledChunkExtractor creates a JpegExtractor or a PngExtractor if the
chunk's MIME type is JPEG or PNG respectively.
The JpegExtractor is instantiated to load an image track.
PiperOrigin-RevId: 582005828
Skip wearable devices, which don't have the same CDD requirements on codecs from Android 5 (API 21).
Limit an existing skipping case to pre API 33 devices, so that we can catch failures on Android 13 (API 33) onwards, and add OnePlus 9 Pro which also times out calculating SSIM. Remove TODOs for removing test/SSIM skipping now they are restricted to API version.
PiperOrigin-RevId: 581985554
This device over-reports encoding capabilities, so skip the test.
This is a Pixel device that was upgraded to Android 12 (API 31/32) so the issue should only affect devices that didn't take OTAs.
PiperOrigin-RevId: 581981073
Instantiate an ImageRenderer and add it to the list returned by
DefaultRenderersFactory.buildRenderers.
Add Renderer.MessageType.MSG_SET_IMAGE_OUTPUT and
ExoPlayer.setImageOutput to enable setting a custom
ImageRenderer.imageOutput.
Add ImageRenderer.handleMessage to process messages sent to the
renderer.
PiperOrigin-RevId: 581962451
Based on on-device testing, this device seems to have the same issue as Moto G (20) where frames are dropped despite configuring the decoder not to drop frames.
PiperOrigin-RevId: 581943805
The dump file diff as part of this change is because using AudioGraph
means the 2nd item is automatically edited to match the AudioFormat of
the 1st item {44.1KHz mono}, rather than {48KHz stereo}.
Manually verified that for the 2nd item, data output:
* Before: 66936 bytes (16734 frames) output = 348_625us of audio.
* After: 30750 bytes (15375 frames) output = 348_639us of audio.
The small final buffer is caused by SonicAudioProcessor outputting all
pending data when EOS queued, and is WAI.
PiperOrigin-RevId: 580494578
Changes includes:
1. Add resume flow.
2. Change demo app code to resume export.
3. Changes in test infra to trigger resume.
4. E2E Test cases
PiperOrigin-RevId: 579895744
Both interfaces are not really needed as the methods can be called
on the respective classes directly. This also avoid error-prone
situations where classes define to/fromBundle methods with parameters
that should be used instead of the parameter-less version.
Also deprecate all existing CREATOR static instances and make the
corresponding fromBundle method public where needed.
PiperOrigin-RevId: 579189766
* Rename method f(n) to a more helpful name.
* Move the private inner class to the bottom.
* Move public fields before private fields but below the static field.
* Make `private ParsableByteArray data` final.
* Make sure `parseSequenceHeader` can only be called once.
DefaultTrackSelector now has all logic necessary for selecting an
image track.
If isPrioritizeImageOverVideoEnabled is set to true, image track will
try to be selected first and a video track will only be selected if no
image track is available. If isPrioritizeImageOverVideoEnabled is set
to false, image track will be selected only if video track wasn't
selected.
PiperOrigin-RevId: 578806006
The native code can now reallocate the buffer if it needs to grow
its size, so we have to reacquire a reference in the Java code to
avoid accessing a stale instance.
This fixes a bug introduced by 8750ed8de6.
PiperOrigin-RevId: 578799862
BundleableUtil is the only class that really depends on the type
inheritence from Bundleable. However, it only needs this as a way
to define Function<T, Bundle> and Function<Bundle, T>, which could
just be passed in as parameters as it's already done for some of
these methods.
Also rename the class to BundleCollectionUtil accordingly.
PiperOrigin-RevId: 578791946
Android Studio removed some nested imports, but I think the extra
qualification at the usage site is actually mostly helpful, so I'm
leaving it as-is.
PiperOrigin-RevId: 578518880
Unfortunately we can't fail any more obviously at this point, because
manifests often contain config for multiple DRM schemes, and when
parsing the manifest we don't know which scheme is going to be used for
playback. It would be unreasonable to fail playback due to incomplete
ClearKey config if playback was otherwise going to succeed using e.g.
Widevine.
* Issue: androidx/media#777
* Issue: androidx/media#563
* Issue: google/ExoPlayer#9169
#minor-release
PiperOrigin-RevId: 578491484
The toBundle method should only be used for remote processes,
because there is a separate method for in-process bundling.
Renaming the method makes this more explicit and less error-prone.
PiperOrigin-RevId: 578456532
This moves the invocation of `transferInitializing` slightly earlier,
but this is consistent with other `DataSource` implementations like
`OkHttpDataSource`.
PiperOrigin-RevId: 578137236
This has been documented to work since this class was created, but until
now we were always trying to resolve using the current application's
`Resources.getIdentifier` method. This commit changes to resolve to the
other app's `Resources` object if the package name doesn't match the
current package.
This will only work if the current app has package-visibility to the
destination package: http://g.co/dev/packagevisibility
This is hard to test because to do so robustly requires being able to
guaranteed that another test APK will be installed with a known raw
resource inside it.
PiperOrigin-RevId: 577864992
This change applies to standalone WebVTT files linked directly from the manifest.
Since DASH only supports stand-alone IMSC1 (TTML) and WebVTT text files, this change concludes the support extension of text-based subtitle files to be parse during extraction.
PiperOrigin-RevId: 577468830
This change applies to standalone TTML files linked directly from the manifest.
As a result, we no longer have the flakiness in the DashPlaybackTest which uses sidecar-loaded (standalone file) TTML subtitles. We experimentally opt into parsing subtitles during extraction and use SubtitleExtractor in hybrid mode.
PiperOrigin-RevId: 577457256
Throws when calling flush when there's no active input, for example
before an input stream is registered or after all output streams have
ended.
PiperOrigin-RevId: 577165419
This CL adds an isPrioritizeImageOverVideoEnabled flag to
TrackSelectionParameters and an API to set the flag value.
The flag will be used by DefaultTrackSelector to determine whether
to select an image track if both an image track and a video track are available.
PiperOrigin-RevId: 576175162
When transmuxing, we usually only need to offset the timestamp by the position of a mediaItem in a sequence.
Trim optimization introduces another type of offset: for the transmux of the second part of the video we need to offset the timestamps by the total duration already trancoded by transformer.
PiperOrigin-RevId: 576134656
The events happens in the following order, assuming two media items:
1. First media item is fully decoded, record the last frame's pts
- Note frame processing is still ongoing for this media item
2. Renderer sends `onOutputFormatChanged()` to signal the second media item
3. **Block sending the frames of the second media item to the `VideoSink`**
4. Frame processing finishes on the first media item
5. The last frame of the first media item is released
6. **Reconfigure the `VideoSink` to apply new effects**
7. **Start sending the frames of the second media item to the `VideoSink`**
This CL implements the **events in bold**
PiperOrigin-RevId: 576098798
Some devices supporting Performance Points for decoder coverage are missing coverage over the CDD requirements for H264. For these cases ExoPlayer should fall back to legacy resolution and frame rate support checks. If there is an H264 stream evaluated as a `PerformancePointCoverageResult` of `COVERAGE_RESULT_NO`, then ExoPlayer checks for coverage of the [720p CDD requirement](https://source.android.com/docs/compatibility/10/android-10-cdd#5_3_4_h_264).
Issue: google/ExoPlayer#10898
Issue: androidx/media#693
PiperOrigin-RevId: 575768836
This change moves the handling of any media button event into
`MediaSessionImpl.onMediaButtonEvent(intent)`. This includes
the double click handling from `MediaSessionLegacyStub`.
The advantage is that everything is in one place which allows
to offer `MediaSession.Callback.onMediaButtonEvent` with which
an app can override the default implementation and handle media
buttons in a custom way.
Media button events can originate from various places:
- Delivered to `MediaSessionService.onStartCommand(Intent)`
- A `PendingIntent` from the notification below API 33
- An `Intent` sent to the `MediaButtonReceiver` by the system dispatched
to the service
- Delivered to `MediaSessionCompat.Callback.onMediaButtonEvent(Intent)`
implemented by `MediaSessionLegacyStub` during the session is active
- Bluetooth (headset/remote control)
- Apps/system using `AudioManager.dispatchKeyEvent(KeyEvent)`
- Apps/system using `MediaControllerCompat.dispatchKeyEvent(keyEvent)`
Issue: androidx/media#12
Issue: androidx/media#159
Issue: androidx/media#216
Issue: androidx/media#249
#minor-release
PiperOrigin-RevId: 575231251
Player methods shouldn't be called if they are not available and the
entry point to the playback resumption flow only checks
COMMAND_PLAY_PAUSE.
#minor-release
PiperOrigin-RevId: 574834148
If an app rejects the connection of the internal media notification manager
the session should behave like without the the media notification controller.
The legacy System UI controller should not be hidden or even rejected to
connect in such a case.
#minor-release
PiperOrigin-RevId: 574807901
Since DEFAULT_MAX_CHANNEL_COUNT was increased from 8 to 10,
getMaxSupportedChannelCountForPassthrough always throws if its loop
enters its second iteration (channelCount of 9). This is due to
Util.getAudioTrackChannelConfig returning CHANNEL_INVALID when passed a
channelCount of 9, and setting CHANNEL_INVALID as the AudioFormat's
channel mask throws an exception.
This change skips each iteration where CHANNEL_INVALID is returned.
HttpEngine was added in Android SDK 34. This DataSource is preferable to the DefaultHttpDataSource if supported as it offers better performance and more modern features.
PiperOrigin-RevId: 574594553
The current formatting makes the 'scheme' part of the list blend into
the definition, especially when the definition is multi-line.
https://developer.android.com/reference/androidx/media3/datasource/DefaultDataSource
I considered adding another level of nesting, but I think bold will
help distinguish the structure of the list without adding too much HTML
or visual whitespace.
#minor-release
PiperOrigin-RevId: 574514208
As Opus decoders skip some bytes prior to playback during a seek, the renderer for bypass playback should send samples to the decoder even if they would be decode-only.
#minor-release
PiperOrigin-RevId: 574494666
This replaces the SimpleBitmapLoader that can now be deprecated
as it's fully unused and doesn't provide any additional functionality.
#minor-release
PiperOrigin-RevId: 574454636
The `PreloadMediaSource` has below two new public methods that suppose to be called by the app:
* `preload(long)` allows the apps to preload the source at the passed start position before playback. The preload efforts include preparing the source for a `Timeline`, creating and caching a `MediaPeriod`, preparing the period, selecting tracks on the period and continuing loading the data on the period.
* `releasePreloadMediaSource()` allows the apps to release the preloaded progress.
The `PreloadMediaPeriod` is designed to facilitate the `PreloadMediaSource` for the preloading work. It has a new package-private method `selectTracksForPreloading` which will cache the `SampleStream` that corresponds to the track selection made during the preloading, and when the `PreloadMediaPeriod.selectTracks` is called for playback, it will uses the preloaded streams if the new selection is equal to the selection made during the preloading.
Also add a shortform demo module to demo the usage of `PreloadMediaSource` with the short-form content use case.
PiperOrigin-RevId: 574439529
This change moves the release timestamp adjustment logic out of
MediaCodecVideoRenderer and into a standalone component, the
VideoFrameReleaseControl. The plan for VideoFrameReleaseControl is to use
it:
- from MediaCodecVideoRenderer, when ExoPlayer plays video in standalone
mode.
- from the CompositionPlayer's DefaultVideoSink, when CompositionPlayer
supports multiple sequences.
- (temporarily) from the CompositionPlayer's custom ImageRenderer while
we are implementing single-sequence preview, which is an intermediate
milestone for composition preview.
PiperOrigin-RevId: 574420427
Media button event coming from the `MediaSessionService` are delegated
to the `MediaSessionImpl` and then sent to the session by using the
`MediaSessionStub` directly instead of using the `MediaController`
API.
Splitting the `MediaController.Listener` and `Player.Listener` in
`MediaNotificationManager` got reverted, and both listener are set to the
controller as before. This reverts the change that introduced a
different timing behaviour. It still holds, that a listener
registered on a `MediaController` that calls a method like `play()` is
called immediately and before the call has arrived at the player. This
change works around this behaviour from the library side by calling
`MediaSessionStub` directly with a `ControllerInfo`.
#minor-release
PiperOrigin-RevId: 573918850
I couldn't work out a good way to set up a test environment with
an asset in a different package, so I'm not adding a test for the
package overriding.
PiperOrigin-RevId: 573843326
Android Auto shows a queue button when the queue is not empty.
Apps were able to remove this queue button with the legacy API
by not setting the queue of the session.
After this change, removing `COMMAND_GET_TIMELINE` from the commands
of the media notification controller or the session player sets the
queue in the platform session to null.
#minor-release
Issue: androidx/media#339
PiperOrigin-RevId: 573813558
This is the documented format of this URI. The previous test was
exercising an undocumented format. Support for the previous format is
kept (and documented) because it seems to be relatively common (despite
the package name not being needed or used).
PiperOrigin-RevId: 573756190
Previously, we calculated the next playlist reload time by adding the target duration (or half of it, depending on whether there is a real update in the new playlist snapshot) from the last load completion time, which makes the reload interval as long as `targetDuration(or half of it) + lastLoadDuration`. While still complying to the standard that "the client MUST wait for at least the target duration before attempting to reload the Playlist file again", this could cause buffering when the playback position is close to the end of live window. This change is to calculate the reload interval accurately by not adding the term `lastLoadDuration`.
Issue: androidx/media#663
#minor-release
PiperOrigin-RevId: 573300009
All production and test callers of the non-incremental methods are
already migrated, so we can remove them in this change too.
#minor-release
PiperOrigin-RevId: 573207318
If the `Subtitle` has 'active' cues at `OutputOptions.startTimeUs`, this
change ensures these are emitted in a `CuesWithTiming` with
`CuesWithTiming.startTimeUs = OutputOptions.startTimeUs`. If
`OutputOptions.outputAllCues` is also set, then another `CuesWithTiming`
is emitted at the end that covers the 'first part' of the active cues,
and ends at `OutputOptions.startTimeUs`.
As well as adding some more tests to `LegacySubtitleUtilWebvttTest`,
this change also adds more tests for `TtmlParser` handling of
`OutputOptions`, which transitively tests the behaviour of
`LegacySubtitleUtil`.
#minor-release
PiperOrigin-RevId: 573151016
All the production code is already calling these new incremental
methods, migrating the tests allows us to remove the old
`List`-returning methods in a follow-up change.
#minor-release
PiperOrigin-RevId: 572822828
PlayerInfo bundling is costly and we can add a shortcut for
in-process binder calls where we store the direct object
reference in a live Binder object that can be written to the
Bundle instead of the individual data fields.
#minor-release
PiperOrigin-RevId: 572816784
A few methods in PlayerInfo and related classes combine filtering
information with bundling in one method. This makes it impossible
to use just the filtering for example and it's also easier to reason
about than two dedicated methods. This change splits these methods
into two parts accordingly.
PiperOrigin-RevId: 572592458
When the controller of the `MediaNotificationManager` is disconnected,
the session is removed from the service without checking whether the
session hasn't already been removed. This caused flakiness in `MediaSessionServiceTest.addSession()`.
Because there is a public API `MediaSessionService.removeSession()`,
the controller can't make an assumption whether the session is still
contained in the service when being disconnected.
#minor-release
PiperOrigin-RevId: 572568350
Looping through all MIME types is a bit hacky, but seems like a nice way
to check "all" MIME types (or at least more than a random selection).
PiperOrigin-RevId: 572508027
This currently only applies to subtitles muxed into mp4 segments, and
not standalone text files linked directly from the manifest.
Issue: androidx/media#288
#minor-release
PiperOrigin-RevId: 572263764
Interrupting the main thread in particular may be dangerous
as the flag is not cleared after handling the current message.
#minor-release
PiperOrigin-RevId: 572259422
This gives access to the replacement behavior for a particular subtitle
format without needing to instantiate a `SubtitleParser`.
#minor-release
PiperOrigin-RevId: 572226084
In particular:
- Add allowAudioNonSeamlessAdaptiveness parameter (default true, same
as video and as already implemented by default)
- Forward mixedMimeTypeAdaptation support to AudioTrackScore
(as for VideoTrackScore) and adapt mixed MIME type adaptive
support accordingly
- Check adaptive support when deciding whether a track is allowed for
adaptation (also same check as for video). This takes the new
parameter into account.
#minor-release
PiperOrigin-RevId: 572191308
When used within the same process, we don't have to go via the
onTransact method (which includes marshalling and unmarhsalling
the data), but can directly return the list.
#minor-release
PiperOrigin-RevId: 572179846
Future.isDone and getDone doesn't imply the Future was successful
and it may have been cancelled or failed.
In case where we handle failure, we should also handle cancellation
to avoid CancellationException to bubble up unchecked.
In demo app code where we use isDone for field initialization, we
want to crash in the failure case (usually security exception where
the connection is disallowed), but we want to gracefully handle
cancellation. Cancellation of these variables usually happens in
Activity.onDestroy/onStop, but methods may be called after this point.
#minor-release
PiperOrigin-RevId: 572178018
With this change the playback will resume as soon as the suitable device is connected and suppression reason is cleared (within set time out).
#minor-release
PiperOrigin-RevId: 572140309
After 4fad529433, MediaCodecVideoRenderer does not report if frames
are dropped from the VideoSink. This commit fixes this.
#minor-release
PiperOrigin-RevId: 571905721
This is more correct, though in reality all text-based subtitles handled
by `SubtitleExtractor` are implemented with MERGE behavior in media3.
PiperOrigin-RevId: 571888573
Alters RTSP KeepAlive monitor test to just make sure that keep-alive message is sent.
The test was added in 42c1846984
#minor-release
PiperOrigin-RevId: 571349013
In some streaming scenarios, like offload, the sink may finish writing buffers a bit before playback reaches the end of the track. In this case a player may pause while in this 'stopping' state.
The AudioTrackPositionTracker needs to update the cached values it uses to calculate position in the `PLAYSTATE_STOPPED`/`PLAYSTATE_STOPPING` states if pause/play are called during this period.
PiperOrigin-RevId: 571345914
In offload mode, `AudioTrack#stop()` will put the track in `PLAYSTATE_STOPPING` rather than `PLAYSTATE_STOPPED`. The difference in state means that `AudioTrack` can be paused and played during this 'stopping' period.
Currently, if `AudioTrackPositionTracker#handleEndOfStream()` has been called then `DefaultAudioSink` in `pause()` won't call `AudioTrack#pause()`. `AudioTrack#pause()` should be called in this case if in offload mode.
#minor-release
PiperOrigin-RevId: 571335108
This belongs in the resolver, because it depends on the resolution
algorithm (and therefore the logic can't live in `TextRenderer`).
This also fixes a bug in `TextRenderer` where we were doing arithmetic
with `cues.durationUs` without checking if it was `TIME_UNSET` first.
#minor-release
PiperOrigin-RevId: 571332750
The flag is no longer used by our components and only set and checked
in a few places to guarantee compatiblity with existing renderers and
decoders that still use it.
The flag will be removed in the future due to its design limitations.
#minor-release
PiperOrigin-RevId: 571291168
The existing `Subtitle` handling code is left intact to support the
legacy post-`SampleQueue` decoding path for now.
This also includes full support for merging overlapping `CuesWithTiming`
instances, which explains the test dump file changes, and which should
resolve the following issues (if used with the
decoder-before-`SampleQueue` subtitle logic added in
5d453fcf37):
* Issue: google/ExoPlayer#10295
* Issue: google/ExoPlayer#4794
It should also help resolve Issue: androidx/media#288, but that will also require
some changes in the DASH module to enable pre-`SampleQueue` subtitle
parsing (which should happen soon).
#minor-release
PiperOrigin-RevId: 571021417
The aim of this test is to make sure the image is onscreen for the right amount of time, so to drive down flakes from the decoder taking too long, change this to an atLeast check
#minor-release
PiperOrigin-RevId: 570988044
location and index are mostly equivalent, except that it's possible that location is more up-to-date than index, so we can just always use location.
PiperOrigin-RevId: 570950407
Set KeepAliveMonitor to send a keep-alive message at half the timeout value, if provided, by the RTSP Setup response.
Issue: androidx/media#662
PiperOrigin-RevId: 570946237
In most cases this is more useful than `durationUs`.
We will keep `durationUs`, and the constructor will continue to take
`startTimeUs` and `durationUs`, to allow for use-cases where we don't
know the start time but still want to indicate a duration (this will be
used to implement CEA-608 timeout).
#minor-release
PiperOrigin-RevId: 570944449
Much simpler than ExternalTextureManager flushing, because
ExternalTextureManager complexity is due to decoder race condition behavior,
which we won't see for Bitmaps due to a more well-defined interface.
This is needed to test texture output flushing.
PiperOrigin-RevId: 570896363
This gets rid of the reliance on the decode only flag that is still
set on input buffers to the decoder if they are less than the start
time.
We still need to set and check the decode-only flag in SimpleDecoder
to ensure compatbility with custom decoders that use the flag while
it's not fully removed.
PiperOrigin-RevId: 570736692
The interface requires the implementation to return null if the
decode-only flag is set. So instead of setting the flag and returning
null, we can simply not call the method and assume it's null.
The only reason where this wouldn't work is if the metadata format
has keyframe-like logic and requires previous metadata to decode
the next one. This is not something we came across before and it seems
ignorable. If that feature is needed in the future, we should instead
add a method to MetadataDecoder to set the first output timestamp.
#minor-release
PiperOrigin-RevId: 570399838
While sleeping for offload, position is estimated based on time playing. If asleep and AudioTrack is reused, then the position will keep incrementing as the subsequent item plays. That is until wakeup when playing position is updated to the timestamp of the second item. Offload scheduling should be disabled until track transitions fully.
PiperOrigin-RevId: 570397140
The test is hidden behind the Ignore annotation due to some flakiness just like `webvttInMp4`. However, it will be removed when the subtitle parsing is moved to a pre-sample-queue architecture.
#minor-release
PiperOrigin-RevId: 570376275
In some cases, SimpleDecoder output needs to be skipped for rendering
because the decoder produces no data. This is one of the remaining
usages of BUFFER_FLAG_DECODE_ONLY at the moment and can be more
directly solved without using the flag. SimpleDecoder still needs to
check the flag though for backwards compatbility with custom decoders
while the flag is not completely removed.
PiperOrigin-RevId: 570345233
When seeking, we must first flush the video sink so it stops
using any SurfaceTextures before flushing MediaCodec.
#minor-release
PiperOrigin-RevId: 570015998
Move remote 8K file to local and trim to 320ms.
Trim done with ffmpeg:
`ffmpeg -i {remote_file} -t 0.3 -c:v copy -c:a copy 8k24fps_300ms.mp4`
PiperOrigin-RevId: 569449962
It seems likely we will define a new "image decoder" interface that
returns `ListenableFuture<Bitmap>`, and naming that will be
hard/annoying if we need to keep this interface working too.
It's also not really clear what a non-test implementation of this
interface would be expected to do, since `DefaultImageDecoder` is
documented to always decode using `BitmapFactory`.
#minor-release
PiperOrigin-RevId: 569206325
The library already maintains the subscribed controllers internally. This
change adds `MediaLibrarySession.getSubscribedControllers(mediaId)` to
access subscribed controllers for a given media ID.
To accept a subscription, `MediaLibraryService.Callback.onSubscribe` is
required to return `RESULT_SUCCESS`. So far, this isn't the case for the
default implementation of the library.
This change implements `Callback.onSubscribe` to conditionally
provide `RESULT_SUCCESS`. The default calls `Callback.onGetItem(mediaId)` to
assess the availability of the media item. If the app retruns `RESULT_SUCCESS`
with a browsable item, the subscription is accepted. If receiving a valid item
fails, the subscription is rejected.
Issue: androidx/media#561
PiperOrigin-RevId: 568925079
Changes
---
- Added `removeOfflineLicense(byte[])` and `getOfflineLicenseKeySetIds` and consumed them in their implementations
Background
---
- These APIs will help in addressing an increasing amount of `java.lang.IllegalArgumentException: Failed to restore keys: BAD_VALUE` which is our top playback error in our app
- Based on our discussion with Widevine team and [this exoplayer issue](https://github.com/google/ExoPlayer/issues/11202#issuecomment-1708792594)
- TL;DR: The failure occurs on startup if the user has 200+ offline licenses, we would like to add the functionality to remove offline licenses
**Note: Why we want these APIs in ExoMediaDrm and not in OfflineLicenseHelper**
- As per the issue above, we would like to access these 2 public APIs in MediaDrm that don’t exist in `OfflineLicenseHelper` or `ExoMediaDrm`
- APIs interested in:
- [MediaDrm#removeOfflineLicense()](https://developer.android.com/reference/android/media/MediaDrm#removeOfflineLicense(byte%5B%5D)): To remove offline license
- [MediaDrm#getOfflineLicenseKeySetIds()](https://developer.android.com/reference/android/media/MediaDrm#getOfflineLicenseKeySetIds()): To see number of offline licenses on startup
- We use `OfflineLicenseHelper` to download license for L1 and we don't interact with `ExoMediaDrm` directly. But for the alternate Widevine integration, we directly depend on `ExoMediaDrm` APIs to override and call CDM Native APIs.
- We would like to have the functionality of removing offline licenses for both integration which would need access to above APIs in `ExoMediaDrm`.
Links
---
- https://github.com/androidx/media/issues/659
MediaCodecRenderer has already been updated to not rely on the
input stream to mark its samples as decode-only and instead use
a simple time-based comparison to achieve the same effect.
This change makes the same update for all other renderers that
either use the flag directly or forward to a "decoder" instance.
PiperOrigin-RevId: 568232212
Before this change, media button events are routed from `onStartCommand`
of the `MediaSessionService` to the `MediaSessionCompat`, resolved by
the legacy library to a session command called on
`MediaSessionCompat.Callback` from where the command is delegated back
to the Media3 session.
With this change the keycode is resolved directly to a Media3 command
that is sent to the session through the media notification controller
of the session.
After this change, a playback or custom command sent to the session
from a notification, either as a pending intent (before API 33) or as
a legacy session command, look the same and the caller is the
media notification controller on all API levels.
PiperOrigin-RevId: 568224123
With this change, the `ControllerInfo` passed to
`MediaSessionService.onGetSession(ControllerInfo)`
is the same instance that is passed later to all
callback methods of `MediaSession.Callback`.
PiperOrigin-RevId: 568216855
ExoPlayer will not fail playback if an RTSP server responds to the Options request with an unknown RTSP method request type. ExoPlayer will parse the response and just not call methods it does not know how to use.
Issue: androidx/media#613
PiperOrigin-RevId: 568152076
With this change, the notification controller that is connected by
`MediaNotificationManager`, is used as a proxy controller of the
System UI controller. An app can use the proxy at connection time
and during the lifetime of the session for configuration of the
platform session and the media notification on all API levels.
This includes using custom layout and available player and session
commands of the proxy to maintain the platform session (actions,
custom actions, session extras) and the `MediaNotification.Provider`.
The legacy System UI controller is hidden from the public API,
instead the app interacts with the Media3 proxy:
- System UI is hidden from `MediaSession.getConnectedControllers()`.
- Calls from System UI to methods of `MediaSession.Callback`/
`MediaLibrarySession.Callback` are mapped to the `ControllerInfo`
of the proxy controller.
- When `getControllerForCurrentRequest()` is called during an operation of
System UI the proxy `ControllerInfo` is returned.
PiperOrigin-RevId: 567606117
The per-stream methods are generally preferred and having a reference
to them from the device-wide methods may help with discoverability.
Issue: google/ExoPlayer#11295
PiperOrigin-RevId: 567604785
Adjust the Javadoc to highlight that data of this MIME type just contains a URI, not the actual image content.
And also remove the superfluous "key" part of the MIME type string that doesn't
really add information (and it's also really just an URI, not an URI key).
PiperOrigin-RevId: 567560238
The current code only set the videoEffects when CVSP is initialized, which
happens after `player.prepare()`. But it's valid that videoEffects are set
before calling `prepare()`.
PiperOrigin-RevId: 566941216
Move audio offload mode related interfaces and definitions from `TrackSelectionParameters` to a new `AudioOffloadModePreferences` class.
PiperOrigin-RevId: 566905017
This method isn't used by the library (since <unknown commit>).
It doesn't really work well (e.g. arbitrarily defaults to `MUSIC` when
`UNKNOWN` would be a better default). There's no suggested replacement.
PiperOrigin-RevId: 566676744
This change uses the new incremental overloads of `SubtitleParser` to
avoid introducing the performance regression caused by the original
change requiring all cues to be fully parsed before the first could be
shown on screen.
`TtmlDecoder` which used to be `SimpleSubtitleDecoder` will now be
called `TtmlParser` and implement `SubtitleParser` interface. For
backwards compatibility, we will have the same functionality provided
by `DelegatingSubtitleDecoder` backed-up by a new `TtmlParser`
instance.
PiperOrigin-RevId: 566671398
I spotted that this looked wrong while investigating test various
session test failures. However, changing this logic doesn't seem to
affect the tests so I don't know if the change is an improvement or not.
PiperOrigin-RevId: 566655318
This requires adapting the 'to `CuesWithTiming` list' logic to work with
the new partial-output API, and that needs a private method so it's no
longer a good fit for a default method on `Subtitle` - hence moving it
to a new utility class.
Also update the implementation to never return `UNSET` duration (this is
an equivalent change to the `SsaParser` change in 9631923440).
PiperOrigin-RevId: 566598094
If getProgress is blocking whilst the internal thread calls endInternal
(for error or success), the condition is never opened. Related to this,
onCompleted and onError are therefore never surfaced to the app.
progressState is accessed from application and internal threads, so
should be marked volatile to prevent a thread caching the value.
PiperOrigin-RevId: 565720184
When running on emulators, ExternalTextureManager needs a longer timeout for
forcing EOS, but we didn't catch a device name running on blaze: `generic_x86`
PiperOrigin-RevId: 565513152
Organize logic a bit by splitting logic about how we draw using OpenGL onto a
texture, out from the larger class, which has lots of logic discussing how we select
frames and streams.
No functional change intended, but a few method calls are shuffled around to
simplify things.
PiperOrigin-RevId: 565426225
When we switch from MUXER_MODE_MUX_PARTIAL_VIDEO to MUXER_MODE_APPEND_VIDEO
`muxedPartialVideo` will already be `true` so `endTrack` method will pass
through this `if(muxedPartialVideo)` check which is incorrect.
PiperOrigin-RevId: 565398117
These tests were under androidTest because we needed a functional
BitmapFactory. Robolectric now supports decoding bitmaps so moving them
under tests.
PiperOrigin-RevId: 565181239
The logic that handles components' boundaries are grouped together in private
methods, like handling VideoCompositor's output textures.
PiperOrigin-RevId: 565131579
This change introduces two new types of method to `SubtitleParser`:
1. `parse()` methods that take a `Consumer<CuesWithTiming>` and return `void`
2. `parseToLegacySubtitle` method that returns `Subtitle`
(1) ensures that in the new 'parse before SampleQueue' world we can
write cues to the `SampleQueue` as soon as they're ready - this is
especially important when parsing monolithic text files, e.g. for a
whole movie.
(2) ensures that during the transition, the legacy 'parse after
SampleQueue' behaviour doesn't see any regressions in 'time to first
cue being shown'. Previously we had a single implementation to convert
from `List<CuesWithTiming>` to `Subtitle`, but this relies on the
complete list of cues being available, which can take a long time for
large files in some formats (with ExoPlayer's current parsing logic).
By allowing implementations to customise the way they create a
`Subtitle`, we can directly re-use the existing logic, so that the
'time to first cue being shown' should stay the same.
This change migrates all **usages** to the new methods, but doesn't
migrate any **implementations**. I will migrate the implementations in
follow-up CLs before deleting the old list-returning `parse()` methods.
PiperOrigin-RevId: 565057945
As discussed offline, this is important for users of the class, and not all users may choose to read method javadoc, so best to make sure it's visible by leaving it at the class-level.
PiperOrigin-RevId: 565057677
Removed `ExoPlayer.experimentalSetOffloadSchedulingEnabled` as scheduling will be enabled by default when offload is enabled for audio-only playback.
In addition, the `experimental` key word was taken out of the following
method signatures:
* `ExoPlayer.experimentalIsSleepingForOffload`
* `AudioOffloadListener.onExperimentalSleepingForOffloadChanged`
* `AudioOffloadListener.onExperimentalOffloadedPlayback`
PiperOrigin-RevId: 565035289
Previously any `CuesWithTiming.durationUs` could be `TIME_UNSET`,
meaning it should be replaced by the next `CuesWithTiming` instance
(instead of being merged if the durations overlap, which is currently
the expected behavior for all `CuesWithTiming` with a 'real' duration).
This technically allowed a single subtitle track to include a mixture of
`CuesWithTiming` that should be merged, and some that should be
replaced. This is not actually needed for any of the subtitle formats
currently supported by ExoPlayer - in all cases a format expects either
all cues to be merged, or each cue to replace the previous one.
Supporting this mixture of merging and replacing in `TextRenderer` ended
up being very complicated, and it seemed a bit pointless since it's not
actually needed. This change means a given subtitle track either merges
**all** cues (meaning `CuesWithTiming.durationUs = C.TIME_UNSET` is not
allowed), or **every** cue is replaced by the next one (meaning
`CuesWithTiming.durationUs` may be set (to allow for cues to 'time out',
needed for CEA-608), or may be `TIME_UNSET`).
This value will be used in a subsequent change that adds cue-merging
support to `TextRenderer`.
PiperOrigin-RevId: 565028066
This is equivalent to the existing `scaleLargeTimestamp` method with the
following changes/improvements:
* No longer specific to timestamps (there was nothing inherently
time-specific about the logic in `scaleLargeTimestamp`, but the name
and docs suggested it shouldn't be used for non-timestamp use-cases).
* Additional 'perfect division' checks between `value` and `divisor`.
* The caller can now provide a `RoundingMode`.
* Robust against `multiplier == 0`.
* Some extra branches before falling through to (potentially lossy)
floating-point math, including trying to simplify the fraction with
greatest common divisor to reduce the chance of overflowing `long`.
This was discussed during review of 6e91f0d4c5
This change also includes some golden test file updates - these
represent a bug fix where floating-point maths had previously resulted
in a timestamp being incorrectly rounded down to the previous
microsecond. These changes are due to the 'some more branches' mentioned
above.
PiperOrigin-RevId: 564760748
While debugging various session tests failure I found that sometimes
this list was empty (e.g. when a controller had failed to connect, or
hadn't finished connecting yet) - resulting in these methods being
silent no-ops. I think it's basically never expected for there to be no
controllers connected when these methods are invoked, so this logs at
ERROR level to give a clue of what happened when looking at the test
logcat.
PiperOrigin-RevId: 564750969
For pause and resume feature we need to use same `MuxerWrapper`
for `remuxing processed video` and then to `process remaining video`.
In order to use same `MuxerWrapper` across `different Exports`
we need to preserve its state.
PiperOrigin-RevId: 564728396
This aligns the equivalent implementation in `MockMediaSessionService`,
and it looks more correct to me - but the tests seem to pass both with
and without this change (and it doesn't seem to affect flakiness
either).
PiperOrigin-RevId: 564689493
It's much better to merge the windows at source level to reflect the
same state everywhere (e.g. notifications, player state, etc). This
can be done with ConcatenatingMediaSource2.
PiperOrigin-RevId: 563715718
There is no known use case left that needs to be solved by this
class. All other use cases are better solved by alternative
approaches (Player playlist methods or ConcatenatingMediaSource2).
PiperOrigin-RevId: 563713384
The class currently disallows offsets of periods in their windows
except for the very first window. This is not necessary because
we can use TimeOffsetMediaPeriod to eliminate the offset if needed.
This makes the class more useful for many use cases, in particular
for using it with ClippingMediaSource.
Issue: google/ExoPlayer#11226
PiperOrigin-RevId: 563702120
Modifying dumping to not required "released" to be called.
Track index is an arbitrary value based on the order of addTrack calls.
Samples are dumped by track (rather than as soon as they are written),
so it's preferable to use a value that provides more context.
By using the track type as a key, dump files will be more deterministic
and will have more similarities when branched.
PiperOrigin-RevId: 563700982
This callback asks to correct the media time in a composition, but doesn't
provide the MediaPeriodId as the necessary context to do this.
PiperOrigin-RevId: 563699344
That said, only SDR is supported for now, so this will always throw if
HDR is input. This will also throw if different ColorInfo values are input
because color SDR mixing (ex. between sRGB and BT709) is not yet supported.
PiperOrigin-RevId: 563457729
While investigating a different test failure on a pending change I
noticed that sometimes the "testAddSessions_removedWhenClose" session
was being released before it could be fully connected - and this
results in a failure like:
```
expected not to contain: androidx.media3.session.MediaSession@50a9ef1
but was : [androidx.media3.session.MediaSession@50a9ef1, androidx.media3.session.MediaSession@3a6bac]
at androidx.media3.session.MediaSessionServiceTest.addSessions_removedWhenReleased(MediaSessionServiceTest.java:376)
```
It also shows up in the logs like:
```
MediaController future failed (so we couldn't release it)
...
Caused by: java.lang.SecurityException: Session rejected the connection request.
```
Adding some debug logging revealed that the `session.release` call
posted to the test thread was happening before the call to
`MediaSessionStub.connect` on the main thread (this is triggered deep
in prod media3 session logic).
PiperOrigin-RevId: 563431265
Instead of invoking "undefined behavior" may occur when a class is used
incorrectly, which is clear but not very helpful to callers, specify what Apps
or users of a class should do when issues occur.
Apply this to Compositor as well, where this otherwise might seem unclear.
PiperOrigin-RevId: 563406219
For pause and resume feature we will need to build `ExportResult`
from multiple internal Exports. Keeping the `ExportResultBuider` in
`Transformer` class will allow using same builder across different
internal exports.
PiperOrigin-RevId: 563392443
This change ensures that a different package is used for
`test_session_current/src/main:support_app` and
`test_session_current/src/androidTest:test_app` - previously they used
the same package which was unusual (and possibly led to some confusing
behaviour).
PiperOrigin-RevId: 563051818
This change avoids using a framework API that is deprecated since
API 31 and works around difficulties with this deprecated API on
devices from some manufacturers on API 33/34.
Manually tested with BT headphone connected to:
- Samsung SM-G920F (API 24)
- Pixel (API 28)
- Samsung SM-A515W (API 33)
- Pixel 6 Pro (API 33)
(verified manually with devices and adb output of dumpsys)
Issue: androidx/media#167
PiperOrigin-RevId: 562890206
When checking the color transfer, there is no reason that the file should have
exactly two tracks, and this assertion means that this method can't be used
as-is for checking video-only files, for example.
PiperOrigin-RevId: 562813111
We have an existing logic for recovering the `AudioTrack` from the [ERROR_DEAD_OBJECT](https://developer.android.com/reference/android/media/AudioTrack#ERROR_DEAD_OBJECT)
throws. However, this only applies to the situation when the
`writtenEncodedFrames > 0`.
However, we have a case when `ERROR_DEAD_OBJECT` throws while writing
PCM data. When the `tunneling` is turned on, the `AudioTrack` is set the
flag `AUDIO_OUTPUT_FLAG_HW_AV_SYNC`, and further because of this flag,
it is forced the flag `AUDIO_OUTPUT_FLAG_DIRECT`. When the platform
invalidates the `AudioTrack`, the `AudioTrack` won't be restored by the
platform due to `AUDIO_OUTPUT_FLAG_DIRECT` flag, and `ERROR_DEAD_OBJECT`
is thrown.
Issue: androidx/media#431
PiperOrigin-RevId: 562582451
This is so CompositionPlayer can customize the Renderers and figure
out when a transition to a MediaItem is made hence it can configure the
effects pipeline with the effects that are specific to the MediaItem.
PiperOrigin-RevId: 562530774
Created new method `AudioSink#setOffloadDelayPadding` that will set delay and padding data onto an `AudioTrack` using `AudioTrack#setOffloadDelayPadding`. This feature adds support for offloaded, gapless Opus playback as the content requires the capability to set padding data post-`AudioSink#configure`.
PiperOrigin-RevId: 562518193
Overlays may be overlaid over:
* In VideoFrameProcessor, videos or images (or texture input).
* In Compositor, other videos.
In Compositor, Overlays may consist of video, so it could be confusing
for videoFrameAnchor to contrast with overlayAnchor.
Also, rename overlayAnchor to overlayFrameAnchor, since it's modifying
the anchor in the overlay's frame, so this name seems slightly more precise.
PiperOrigin-RevId: 562004292
This change makes capturing components in CapturingRenderersFactory
to dump data only they have previously captured any. This is so we can
extend the CapturingRenderersFactory with more renderers that do not
capture data in pre-existing tests and we don't have to change the
golden files.
PiperOrigin-RevId: 561973645
also improves tests to hash bitmap now that we know the bitmaps are decoded in roboelectic tests,
as well as some minor fixes in the DefaultImageDecoder
PiperOrigin-RevId: 561946073
Implement VideoCompositor support of:
* Different input and output sizes
* CompositorSettings, to customize output size based on input texture sizes
* OverlaySettings, to place an input frame in an arbitrary position on
the output frame.
Also, refactor Overlay's matrix logic to make it more reusable between
Compositor and Overlays
PiperOrigin-RevId: 561931854
This wasn't properly documented yet (unlike the longer setMediaItems
method that already includes this documentation).
Issue: androidx/media#607
PiperOrigin-RevId: 561910073
For pause and resume feature we will remux the previously processed
video in the first export and then in the second export we will resume
processing remaining video. For the second export we will have to
set initial timestamp offset so that video samples from the second
export are in continuation to the previously muxed samples.
PiperOrigin-RevId: 561689651
For pause and resume we need to perform multiple intermediate exports.
So moved start() logic into a separate so that it can be reused from
different places with different Compositions and MuxerWrapper.
PiperOrigin-RevId: 561675472
In most cases, the places that were trying to avoid a transitive
dependency on `lib-exoplayer` when this duplication was introduced [1]
are already depending on it again, except for `lib-container` where the
dep is added in this change.
In general it seems fine for the tests of module A to depend
(transitively or directly) on module B even where the prod code of
module A **does not** depend on module B.
[1] <unknown commit>
PiperOrigin-RevId: 561660371
Switch to SparseArray<List> rather than a Map<Integer, List>.
Track indices now determine sample list dumping, reducing flakiness.
PiperOrigin-RevId: 561658586
Currently, we only support sending Common Media Client Data (CMCD) data through custom HTTP request headers, added capability to configure and transmit it as HTTP query parameters.
PiperOrigin-RevId: 561591246
We're moving towards parsing subtitles before the `SampleQueue`, and we
already have subtitle playback assertions using `Player.Listener.onCues`
which is the most 'complete' place to assert subtitle output from. We
don't need to hook anything inside `TextRenderer`, so we can remove this
TODO.
PiperOrigin-RevId: 561418660
This will help with debugging subtitle sync issues as it will show up in
track selection details emitted to logcat by `EventLogger`.
PiperOrigin-RevId: 561355836
For pause/resume feature, same MuxerWrapper needs to be
used across intermediate exports. So pass the MuxerWrapper
from Transformer.java
More specifically, when resume() is called
1. Create a MuxerWrapper and remux the previous
video samples (Export 1).
2. User the same MuxerWrapper and start processing
remaining video samples (Export 2).
PiperOrigin-RevId: 561325867
If a RtspMediaSource is used within a combining source like MergingMediaSource, then it could become stuck in a loop of calling continueLoading when the source has not actually started loading yet.
Issue: androidx/media#577
PiperOrigin-RevId: 561322900
Also use `@linkplain` in more places: If I was already touching a javadoc
block, I switched to `@linkplain` throughout the whole block.
PiperOrigin-RevId: 561320273
At this stage, the image renderer is designed to be able to render one input stream containing one sample. The renderer supports seeking inside that stream, which is a no-op because the stream comprises for exactly one sample.
PiperOrigin-RevId: 561307859
Also re-jig some `checkNotNull` calls to satisfy the nullness checker
(remove an unneeded one, and add a new one).
Follow-up to discussion in bb214b19f9
PiperOrigin-RevId: 561295724
More specifically, this CL
- Defines a VideoGraph interface
- Adds a factory method, to hide the constructors
- Separate out an initialize method that does the real work (following that constructor should not do real work)
- The VideoGraph takes in a list of composition effects. For now, we only use the `Presentation`, if there is one in the list. This means we can take any number of Presentations now, but only the first one will be used.
- Moves the VideoFrameProcessingWrapper to its own file
PiperOrigin-RevId: 561059653
By doing so, the media notification sent from DefaultMediaNotificationProvider won't be auto-grouped (https://developer.android.com/develop/ui/views/notifications/group#automatic-grouping) with the regular notifications sent from the other places in the app. As the result, when the customer attempts to dismiss the regular notifications (by swiping away), the media notification won't be dismissed as being in the same group.
Issue: androidx/media#549
PiperOrigin-RevId: 561047571
The `controller.prepare()` call can trigger the callback `onPrepare()`, however, this will count down the latch by one before the callback `onStop()` is triggered.
PiperOrigin-RevId: 560988209
Split this logic out from `OverlayShaderProgram`, to share logic like
this with `VideoCompositor`.
Also, remove the unused `context` parameter from `OverlayShaderProgram`
`VideoCompositor` will reuse `OverlayMatrixProvider`'s storing of the matrices,
though it may generate the matrices differently, since `OverlayShaderProgram`
applies this in the fragment shader, whereas `VideoCompositor` applies this in the
vertex shader.
PiperOrigin-RevId: 560142164
The actual logic scales the alpha, instead of setting it, so rename
this to what it does.
We would also prefer alpha to be scaled here, to not lose alpha information
(for example for PNGs with transparent regions).
PiperOrigin-RevId: 560121708
For pause and resume feature we need to find the timestamp of
the last sync sample in an MP4 file. The Mp4ExtractorWrapper
provides an easy to use API on top of Mp4Extractor.
PiperOrigin-RevId: 560113112
If the loading thread of the second media item progresses slow, then when the playback of the first item approaches to the end before the second item is prepared, the stream will be set "final" and the renderers are disabled during such transition. In this case, `AudioSink.handleDiscontinuity` never gets called, which introduces the flakiness into the `test_bypassOffThenOn`.
Similarly for `test_subtitle`, if the loading thread of extracting the subtitle is slow, then when the renderer attempts to queue the input buffer, the data hasn't been available in the stream. And when the extracting finished, the renderer already advanced to the time after the subtitle end time.
To de-flake the tests, we have to make sure that the period of the second item has fully loaded before starting the playback.
PiperOrigin-RevId: 560080119
The period index was calculated relative to the contentTimeline
of the DAI stream, but then used with the full timeline of the
player. This means it currently only works when the stream is the
only or first item in the playlist.
Issue: androidx/media#571
PiperOrigin-RevId: 560023412
The first call to method `registerInputStream` doesn't block.
Later successive calls to the method blocks until the previous register call
finishes.
PiperOrigin-RevId: 559694490
This removes concurrent access from `rtspLoaderWrappers`. Previously
there was a race between the playback thread clearing & re-adding
entries to this list in `retryWithRtpTcp()`, and the loading thread
accessing the entries in `InternalListener.track()` (implemented from
`ExtractorOutput`).
This change means each `ExtractorOutputImpl` uses exactly one
`SampleQueue` for its one `TrackOutput`. When the `RtspLoaderWrapper`
instances are replaced in `retryWithRtpTcp()`, any stale instances will
only be able to access their own (also stale) `SampleQueue` instances
(vs before, where the stale `ExtractorOutput` could accidentally access
'new' `SampleQueue` instances via the `rtspLoaderWrappers` field).
As well as fixing a race condition in the prod code, this also de-flakes
`RtspPlaybackTest`.
#minor-release
PiperOrigin-RevId: 559130479
Replace Queue<Long> with LongArrayQueue which provides queue semantics
for long primitives. LongArrayQueue is forked from IntArrayQueue which
in turn was forked from Androidx CircularIntArray.
IntArrayQueue is deleted and we now use CircularIntArray directly from
Androidx Collection.
PiperOrigin-RevId: 559129744
There are multiple problems in the test:
1. The controller must run on the main thread due to the Player
interface requirement to add surfaces on the thread their
callbacks will be called on.
2. This test made assertions about an expected callback that
aren't true. Setting an anonymous surface will not result in
a size change callback because the size stays unknown.
But clearing a surface (even if never set) ensures the returned
size is guaranteed to be zero.
PiperOrigin-RevId: 559080905
In case BaseShaderProgram implementations have an expensive
configure() call, we can only call configure() when absolutely necessary,
aka when the input size changes.
To reduce scope, this doesn't reduce the amount of
configure() calls that may be made outside this class, in
DefaultFrameDroppingShaderProgram and
FinalShaderProgramWrapper.
PiperOrigin-RevId: 558754314
Covers all permutations&combinations of:
* AV asset.
* AV asset with effects.
* AV asset with silence.
* AV asset with silence and effects.
Note that the video is not relevant (therefore transmuxed), but is
needed for silence generation.
PiperOrigin-RevId: 558734593
Based on the Common Media Client Data (CMCD) specification key-value pairs should be sequenced in alphabetical order of the key name in order to reduce the fingerprinting surface exposed by the player.
PiperOrigin-RevId: 558296264
Added more comprehensive Javadoc around setting custom data and verification on key format.
Changed adding custom data as a `String` to `List<String>`. This would enable us to sort all keys to reduce the fingerprinting surface.
PiperOrigin-RevId: 558291240
We will need to build upon original composition
for pause and resume feature and will have to set modified sequences.
In the next CL, will add method to set sequences on builder and will
probably remove it from the constructor parameter (to avoid providing
two APIs for same thing).
PiperOrigin-RevId: 558127334
I wasn't able to reproduce the error in the bug by running videoTranscoding_withTextureInput_completesWithCorrectFrameCountAndDuration
100 times, but I figured this added debug info may help the next time the
issue is seen.
PiperOrigin-RevId: 558118443
This file decompresses to 3024*4032*4 bytes ~= 48
MB, which leads to OOM errors on some devices.
Use a lower resolution test image for tests, to
still test a portrait image but without hitting
OOM errors.
PiperOrigin-RevId: 557861063
In addition to this being how Alpha should be handled, tests do fail under
API 29 without this CL. Unfortunately while these tests do fail under API 29
without this CL, we currently only run this test on API 33 emulators, so we
didn't catch this failure earlier (until compositor tests on mh failed on all
devices under API 29 and succeeded on all at or over 29).
PiperOrigin-RevId: 557781757
When the foreground service type is set to `mediaPlayiback` this shouldn't
be required. However, some reports from users say that setting the
foreground service behavor fixes some problems on some devices. Setting
it explicitely with this change makes sure users don't have to do that
workaround themselves.
Issue: androidx/media#167
PiperOrigin-RevId: 557608577
In an upcoming change I'm going to mark SSA subtitles as 'merge
replacement behavior', which will mean that
`CuesWithTiming.durationUs=C.TIME_UNSET` will not be permitted. This CL
makes it clear that `SsaParser` obeys that invariant.
PiperOrigin-RevId: 557504633
This `DvbParser` was updated to implement the new `SubtitleParser`
interface in 2261756d3a.
This method used to return a `List<Cue>`, but now it returns a
`List<CuesWithTiming>`, which is closer to `List<List<Cue>>`. The
previous method returned an empty list, meaning a single 'event' with
no cues. After the change above the method still returns an empty list,
but that now means 'no events'. In order to maintain the old behaviour,
this change updates it to return a single-item list containing an empty
list of cues.
PiperOrigin-RevId: 557502261
This test uses an existing CBR test asset, and shows that (as expected)
`isSeekable = true` even in the `unknown_length` dump file.
PiperOrigin-RevId: 557455364
It was moved there temporarily to support another temporary move
of ExoPlaybackException. See <unknown commit>.
Since then, ExoPlaybackException has been moved back to ExoPlayer
and we can do the same with MediaPeriodId, which only makes sense
in the context of the ExoPlayer module.
PiperOrigin-RevId: 557159381
This changes the default logic of shouldShowPlayButton to show a play
button while the playback is temporarily suppressed. This helps to
provide better UI feedback to the fact that playback stopped and
provides a quick way for users to override the suppression and attempt
to restart playback.
Some apps may want to keep the legacy behavior depending on their app's
needs. Hence, we also add a config parameter to set this behavior both
in MediaSession and our default UI components.
Issue: google/ExoPlayer#11213
PiperOrigin-RevId: 557129171
This aligns the bypass code with the MediaCodec logic to not
use buffer.isDecodeOnly and instead rely on a timestamp
comparison with the last reset time. Overall, the change should
not introduce a functional difference and is covered by existing
end-to-end tests with bypass playbacks.
Splitting bypass batch buffers by decoder-only flag is
technically wrong for formats where not every buffer is a
keyframe, and this change also leaves a TODO to explain this
known shortcoming.
PiperOrigin-RevId: 556800038
The method was recently introduced and only searched for matching
samples up to index==length. However, SampleQueue uses a circular
array to index its data and the search should continue until
relativeStartIndex+length, while also handling the overflow
in the circular array.
The tests for seeking didn't cover these overflow cases yet
because they always started writing data in an empty SampleQueue.
This can be fixed by prewarming the queue with placeholder data
to force using the overflow logic in the various seek methods.
PiperOrigin-RevId: 555963011
`BlankFrameProducer` was originally internal to
`DefaultVideoFrameProcessorVideoFrameRenderingTest`. We later copied it in
c221958889. This CL converges the usages.
PiperOrigin-RevId: 555953620
This is leftover from the fact that these constants were used
for ExoPlayer renderers only initially, but are now more generally
used to indicate format playback support.
PiperOrigin-RevId: 555951001
Use `C.RATE_UNSET_INT` instead of introducing `Long.MIN_VALUE` as the default value for rate parameters. Modifies default behaviour of the `TrackSelection.getLatestBitrateEstimate()` method and the `measuredThroughputInKbps` parameter in `CmcdHeadersFactory.CmcdRequest`.
PiperOrigin-RevId: 555939420
This effect can scale Alpha values.
This is needed for testing Compositor, and coincidentally was requested
by a partner in the past. This also will generally be more useful in
full multi-asset, where we may want to have one input have different alpha
values than another
Also, update fragment_shader_copy_es2.glsl to not throw away alpha values.
This means ThumbnailStripShaderProgram (the only place this glsl is used) will
consider input alpha and output alpha appropriately
PiperOrigin-RevId: 555915092
Optimises the construction of Common Media Client Data (CMCD) headers by eliminating the use of trailing commas. Instead of appending values directly to a StringBuilder, it now adds values to a list and joins them using a comma separator. While the primary focus is on enhancing readability and robustness, it's worth noting that in certain instances, removing usages of `formatInvariant`, there are also performance improvements.
PiperOrigin-RevId: 555911498
This is the first iteration for the CompositingVideoSinkProvider which
is basically a copy of MediaCodecVideoRenderer's
VieoFrameProcessorManager.
The MediaCodecVideoRenderer now instantiates a
CompositingVideoSinkProvider instead of a VieoFrameProcessorManager.
PiperOrigin-RevId: 555903928
Rename AudioMixerImpl to DefaultAudioMixer.
Removes the AudioMixerImpl specific getOutputAudioFormat, as the caller
defines and sets this.
PiperOrigin-RevId: 555887722
To ensure that for each output bitmap from the compostor, the right input
timestamps were used.
Only applied on a subset of tests to avoid needing to upload+maintain
too many files/size in the test binary, especially when it would test
duplicate behavior
PiperOrigin-RevId: 555222530
This adds an effect to change the speed of the video track. The speed
of the audio track can already be changed using a SonicAudioProcessor.
Issue: androidx/media#559
PiperOrigin-RevId: 555155418
Modifies the `ChunkSource.getNextChunk(long, long, List, ChunkHolder)` method in the `ChunkSource` interface to `ChunkSource.getNextChunk(LoadingInfo, long, List, ChunkHolder)`.
LoadingInfo contains additional parameters, including `playbackRate` and `lastRebufferRealtimeMs` in addition to the existing `playbackPositionUs`. The additional parameters will allow us to pass these information to Common Media Client Data (CMCD) logging.
PiperOrigin-RevId: 555148768
Gradle Lint doesn't recognise `checkState` assertion and TargetApi should only ever be used for suppressing a bug in Android Lint. Hence, we keep @RequiresApi and add an if-statement explicitly. Also, fixes >26 to >=26 for the version check.
#minor-release
PiperOrigin-RevId: 555144577
Modifies the `SequenceableLoader.continueLoading(long)` method in the `SequenceableLoader` interface to `SequenceableLoader.continueLoading(LoadingInfo)`.
`LoadingInfo` contains additional parameters, including `playbackSpeed` and `lastRebufferRealtimeMs` in addition to the existing `playbackPositionUs`. The additional parameters will allow us to pass these information to Common Media Client Data (CMCD) logging.
By using the `LoadingInfo` object, we ensure future flexibility to add more fields without causing any breaking changes to the API.
PiperOrigin-RevId: 555123961
Updates to the ad playback state are posted on the main handler,
so they may arrive after the source has already been released
(=the internal MediaSource is null). This can cause NPEs.
PiperOrigin-RevId: 555102426
When hardware buttons are used to control the volume of the remote device, the call propagates to `MediaSessionCompat.setPlaybackToRemote(volumeProviderCompat)`. However, `volumeProviderCompat` was created incorrectly when the new device volume commands were present (COMMAND_SET_DEVICE_VOLUME_WITH_FLAGS and COMMAND_ADJUST_DEVICE_VOLUME_WITH_FLAGS), i.e. with volumeControlType = `VOLUME_CONTROL_FIXED`. This resulted in `VolumeProviderCompat` which doesn't call `onSetVolumeTo` or `onAdjustVolume` and hence doesn't propagate the calls to the `Player`. Instead, it only worked with the deprecated commands which ensured the volumeControlType was `VOLUME_CONTROL_ABSOLUTE`.
This bug was introduced in c71e4bf1ff (1.0 media 3 release) when `PlayerWrapper`'s call to `createVolumeProviderCompat` was mostly rewritten to handle the new commands, but the two if-statements were not amended. Note: this change fixes the bug only for Android 11 and below. For 12 and above, there is a tracking bug for the regression that was introduced: https://issuetracker.google.com/issues/201546605http://Issue: androidx/media#554
#minor-release
PiperOrigin-RevId: 554966361
`asReadOnlyBuffer` doesn't copy the original byte order but always sets it to big-endian. Replace calls to it with a utility method that manually sets the byte order after creating the read-only copy.
This fixes `TeeAudioProcessor` providing a `ByteBuffer` always in big-endian and hence causing `AudioBufferSink` to read wrong data.
PiperOrigin-RevId: 554861402
When implementing pause and resume, overlay appears as black frame on resume, suggesting the texture isn't being remade properly due to lack of proper deletion.
PiperOrigin-RevId: 554832580
If we throw away the first keyframe (because it does not contain any bytes to read), we shouldn't behave as if we had written a keyframe to our track. We should instead continue waiting for the first (valid) keyframe.
PiperOrigin-RevId: 554579146
The test file decompresses to about 48 MB, which can cause an
`OutOfMemoryError` on lower end devices.
Also, a test appears to apply a rotation to this bitmap without downscaling
first, leading to exceeding a maximum texture width/height of 4096 on some
devices.
Replace the test file with one that has a quarter of the original's
width/height so that these tests can pass reliably even on lower end
devices.
PiperOrigin-RevId: 554507185
This fixes a bug with playing very short audio files, introduced by
fe710871aa
The existing code using floor integer division results in playback never
transitioning to `STATE_ENDED` because at the end of playback for the
short sample clip provided `currentPositionUs=189937`,
`outputSampleRate=16000` and `(189937 * 16000) / 1000000 = 3038.992`,
while `writtenFrames=3039`. This is fixed by using `Util.ceilDivide`
so we return `3039`, which means
`AudioTrackPositionTracker.hasPendingData()` returns `false` (since
`writtenFrames ==
durationUsToFrames(getCurrentPositionUs(/* sourceEnded= */ false))`).
#minor-release
Issue: androidx/media#538
PiperOrigin-RevId: 554481782
Periods that are not the last in the manifest are always ended
and non-dynamic periods are also always ended by definition.
Also, some periods may not be able to declare their final duration
accurately and we need to account for some inaccuracy. This can be
done by testing whether no further full segment within the duration
can be expected.
If stream is not considered final and `ClippingMediaSource` is read to `endUs`, then `BaseRenderer.readSource` returns `C.RESULT_NOTHING_READ`. In that case, the `lastBufferInStreamPresentationTimeUs` is not set and the last frame is not rendered.
PiperOrigin-RevId: 554418971
This workaround was added for TS streams that do not adjust their
timestamps to start from zero. Over time, the default audio sink
logic has become more robust towards unexpected timestamps and
we no longer need this workaround to jump forward in time.
The workaround also actively caused issues by adjusting the audio
timestamps backwards if the stream starts with large negative
values. See Issue: androidx/media#291. This caused playback to get stuck due
to another bug in the first-frame rendering logic in the video
renderer that is now fixed.
PiperOrigin-RevId: 553493618
This is the same change as 8655429af7, just on the second call site
of readDiscontinuity. The tests didn't cover this case yet because
they never queued more than 2 items in a playlist.
PiperOrigin-RevId: 553485244
This means we now require 2+ input frames per input, and compare the primary
stream timestamp with secondary stream timestamps in order to select the
correct output timestamp. We also must release frames and back-pressure as
soon as possible to avoid blocking upstream VFPs.
Also, improve signalling of VFP onReadyToAcceptInputFrame
PiperOrigin-RevId: 553448965
setInputFormat calls registerInputStream and will cause VideoFrameProcessor to
reconfigure. We don't need to call it when setting up the VFP
(in `onReadyToInitCodec()`). Rather, we wait until `onOutputFormatChanged()`
called (this output format refers to mediaCodec's outptut format) to register
the input stream.
PiperOrigin-RevId: 553448633
ExoPlayer queues the EOS buffer to the decoder with offset/size/timestamp all
equal to zero, and a EOS flag.
69769c77b3 set TIME_END_OF_SOURCE on the EOS buffer from the extractor.
Queueing the EOS buffer to the decoder with TIME_END_OF_SOURCE causes some
decoders to output wrong timestamps in its output.
This CL replicates what ExoPlayer does in Transformer.
PiperOrigin-RevId: 553104213
We currently only force the first frame if the frame timestamp is
greater than the stream *offset*.
This is wrong for two reasons:
1. The timestamp and the offset are not comparable and it should be
the stream start position.
2. The check should only be applied at stream transitions where we
need to make sure that a new first frame isn't rendered until we
passed the transition point.
We have to fix both issues together, because fixing just issue (1)
causes seeks to before the start position to no longer render the
frame (and playback will be stuck). A new test covers this case.
We also amend the stream transition test case to actually test what it
promises to test and add a test for prerolling samples at the
beginning, to ensure the first frame is still renderered.
Issue: androidx/media#291
PiperOrigin-RevId: 552858967
We currently use 3 different booleans to track the state of the first
frame rendering, which implies that there are 8 distinct possible
overall states. However, this is actually a staged process and there
are only 3 different overall states in the current code. This means
it's clearer and easier to reason about if the variables are combined
to a single state value. Overall, this should be a complete no-op.
State mapping:
- rFFAReset=false, rFFAEnable=false, mayRenderFFAEINS=false
=> FIRST_FRAME_NOT_RENDERED_ONLY_ALLOWED_IF_STARTED
- rFFAReset=false and/or rFFAEnable=false, mayRenderFFAEINS=any
=> FIRST_FRAME_NOT_RENDERED
- rFFAReset=true, rFFAEnable=true, mayRenderFFAEINS=any
=> FIRST_FRAME_RENDERED
PiperOrigin-RevId: 552857802
Refactored `CmcdLog` to `CmcdHeadersFactory` for improved representation of its purpose and updated implementations.
#minor-change
PiperOrigin-RevId: 552831995
This allows for custom implementations of this interface, like
a TestVideoCompositor or partner-implemented implementation
PiperOrigin-RevId: 552541631
`SubtitleExtractor` used to rely on a `SubtitleDecoder`. However, now all SubtitleDecoders that are used for side-loaded subtitles have been migrated to a `SubtitleParser` interface. We can therefore refactor the extractor.
The `SubtitleExtractor` is only used for side-loaded subtitles which means we do not require the migration of the CEA-608/708 `SubtitleDecoders`.
PiperOrigin-RevId: 552471710
Fix short term reference picture list parsing. Before this change, `deltaPocS0`
was derived by adding one to the value of the syntax element
`delta_poc_s0_minus1`, but (maybe surprising) the specification actually says
that `DeltaPocS0[stRpsIdx][i]` should be assigned the negation
`-(delta_poc_s0_minus1[i] + 1)` on the first iteration, then that value added
to the previous value on previous iterations. See equations (7-67) to (7-70) in
the 2021-08 version of the H.265/HEVC specification.
Also read the number of long term reference pictures once rather than on every
loop iteration (subsection 7.3.2.2.1).
PiperOrigin-RevId: 551852999
After this change, every queued bitmap is treated as an individual input stream
(like a new MediaItems).
This change merges the FrameDropTest and FrameDropPixelTest into one (while maintaining all the test cases)
- This is accomplished by generating bitmaps with timestamps on it in FrameDropTest and compare them with goldens (one may call this a pixel test, please lmk if you want this to be renamed)
- The most part of the change comes from DefaultVideoFrameProcessorVideoFrameRenderingTest. The overall working is
- We bypass the input manager
- The TestFrameGenerator generates frames based on timestamps. In this case, we generate frames with timestamps on it
- The generated frame is sent to texture output and in turn saved to bitmaps
- We then compare the generated bitmap with the goldens
PiperOrigin-RevId: 551795770
Move shared test logic to the test runner.
This does increase indirection, which isn't usually preferable in tests.
However, we will have many different tests that would use logic
like this, so this allows us to reduce repetition.
PiperOrigin-RevId: 551536438
1. For codecs that change timestamps, we need to use greater
or equal instead of just equal for comparison.
2. We should only make this comparison once a value has been
set.
3. The largest queue timestamp isn't updated before using it
for this detection, meaning that we also mark the
second-to-last sample as the last one.
PiperOrigin-RevId: 550901978
This should make no functional difference because `SampleExporter` always
checks for H.265 and H.264 first. However, in case we ever change that code,
these are used in priority order so it's better to order them accordingly.
PiperOrigin-RevId: 550894935
We currently pass in the time at which the stream originally started,
but newly enabled renderers should get the current playback position
instead.
PiperOrigin-RevId: 550894630
When generating silence for AudioProcessingPipeline, audio never
queued EOS downstream.
Linked to this, when silence followed an item with audio, the silence
was added to SilentAudioGenerator before the mediaItem reconfiguration
occurred. If the silence had effects, the APP would be flushed after
silence queued EOS, resetting APP.isEnded back to false, so AudioGraph
never ended.
Regression tests reproduce failure without fix, but pass with it.
PiperOrigin-RevId: 550853714
For each event, the timestamp and presentation time is logged. The trace can
then be dumped to a tsv file and easily imported in a spreadsheet.
PiperOrigin-RevId: 550839156
Instead of `text/x-exoplayer-cues`, we will use `application/x-media3-cues`.
The prefix should be `application` not `text` since the encoded form is arbitrary bytes, not necessarily text. The name should not reference `exoplayer`, since the Media3 Extractors (which are not part of `exoplayer`) produce this format.
PiperOrigin-RevId: 550181852
signalEndOfInputStream is needed for when streams have different amounts of
frames, so that if the primary stream finishes after a secondary stream, it
can end without waiting indefinitely for the secondary stream's matching
timestamps.
onEnded mirrors this API on the output side, which will be necessary to
know when to call signalEndOfInput on downstream components (ex. on downstream)
VideoFrameProcessors
PiperOrigin-RevId: 549969933
The VideoFrameProcessingTaskExecutor already handles wrapping
GlUtil.GlException errors into VideoFrameProcessingExceptions.
Therefore, stop wrapping this ourselves, so that errors are
attributed to the correct line number with less indirection.
PiperOrigin-RevId: 549959258
Alters the OverlayShaderProgram implementation to support rotations, however we need to apply the transformations separately in order for them to work as expected so the matrix is removed from the interface in favour of explicit methods.
Adds a rotation test to ensure this ability doesn't regress
PiperOrigin-RevId: 549890847
The `MediaNotificationManager` registers an internal controller
to each session. This change marks this controller through its
connection hints and provides an API for apps to hide
implementation details of the marking.
Issue: androidx/media#389
PiperOrigin-RevId: 549712768
`TtmlDecoder` which used to be `SimpleSubtitleDecoder` will now be called `TtmlParser` and implement SubtitleParser interface. For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `TtmlParser` instance.
PiperOrigin-RevId: 549700490
It is not part of the `SubtitleParser` interface's promise to give events for all changes to do with `Cue` objects. So while in the past, we would have `endTimeUs` of one Cue event being the `startTimeUs` of the next one, now we have that encoded in `durationUs` and can skip event with empty `Cues`.
PiperOrigin-RevId: 549629157
This tracker aims to replicate the behavior of a specific codec to
ensure MediaCodecRenderer correctly detects stream and output
format transitions. The class was needed because MediaCodecRenderer
made assumptions about codec behavior this codec did not fulfil (in
particular, changing timestamps and number of samples).
Since then, MediaCodecRenderer was made more robust to this kind of
codec behavior in general and currently has no assumptions that
require any special handling of this codec. This means we can remove
the workaround completely.
PiperOrigin-RevId: 549610989
This is no longer flaky because WebVTT subtitles are decoded on the
loading thread (enabled in this test with
`defaultExractorsFactory.setTextTrackTranscodingEnabled(true)` - and
supported for WebVTT since f0f24aa0d4).
PiperOrigin-RevId: 549594291
The class tries to be flexible to support as many different input
and codec behavior combinations as possible. But so far it didn't
spell out its remaining assumptions and explicit non-assumptions,
making it hard to know which behavior to rely on.
PiperOrigin-RevId: 549589347
Updated `ExoTrackSelection` to provide the most recent bitrate estimate, enabling the inclusion of measured throughput (mtp) as a CMCD-Request field in Common Media Client Data (CMCD) logging.
Additionally, made changes to the `checkArgument` methods in `CmcdLog` to prevent the use of default values in certain cases.
#minor-release
PiperOrigin-RevId: 549369529
`WebvttDecoder` which used to be `SimpleSubtitleDecoder` will now be called `WebvttParser` and implement `SubtitleParser` interface.
For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `WebvttParser` instance. `WebvttSubtitle` will still be used behind the scenes to handle overlapping `Cues`.
PiperOrigin-RevId: 549298733
The end-to-end test output for the overlapping SRT and SSA subtitles
is currently incorrect. They will be fixed in a future change that
updates `TextRenderer` to support this overlap.
The 'extra' samples visible in the extractor test output files are
'empty cue list' samples produced by `SsaParser`. They will go away
when this implementation is updated to remove this behaviour and rely
on `CuesWithTiming.durationUs` instead (the 'empty list' behaviour is
not required by the `SubtitleParser` interface).
PiperOrigin-RevId: 549264593
The main demo app was still targeting API 29 to avoid scoped storage
restrictions. It is now updated to 34 (like the rest of the demo apps)
and handles scoped storage as it should handle it.
More specifically:
- We need to request READ_MEDIA_... permissions instead of
READ_EXTERNAL_STORAGE from API33.
- The legacy scoped storage opt-out can be removed
- READ_MEDIA_... permissions don't allow arbitrary file access
if the file doesn't end in a typical media file extension, so
this change adds a remark on the guide page to place samples
in the app-specific directory.
- We also don't have to request permissions for the app-specific
directories.
- Custom json files can't be placed in arbitray local locations
because they don't end in a media file extension, as there is
no way we can request a permission to load them. This means we
can remove the storage request logic in SampleChooserActivity.
Issue: google/ExoPlayer#6045
PiperOrigin-RevId: 549252474
*** Original commit ***
Rollback of b69b33206e
*** Original commit ***
Mark output sample as decode-only based on start time
We currently do the same check on the input timestamps and
expect the output timestamps to match. Some codecs produce
samples with modified timestamps and the logic is a lot safer
when the comparison with the start time is done on the output
side of the codec.
Issue: google/ExoPlayer#11000
***
***
PiperOrigin-RevId: 549019403
This ensures the DownloadService stays functional on Android 14
where defining this type is required. On Android 14 and above,
the app also needs to define the DATA_SYNC permission, which is
added to the demo app as well. In the future, this service type
will no longer be supported and DownloadService needs to be
rewritten with another background scheduling framework.
Issue: google/ExoPlayer#11239
PiperOrigin-RevId: 548994842
AudioMixingUtil#mix handles input & output in float or Int16 PCM. Given
Float and Int16 use different sample ratnes, this util handles
conversion between the two, based on the encoding being mixed to.
Migrate AudioMixer to use the util, removing AudioMixingAlgorithm
interface and implementation. ChannelMixingAudioProcessor will be
migrated after additional performance checks.
PiperOrigin-RevId: 548994584
This was done because it was deemed correct to only start at timestamp
zero when the code was originally written. However, in case of
prerolling from a keyframe, many samples will get the same timestamp,
which is not correct and interferes with downstream logic
that deals with timestamps.
PiperOrigin-RevId: 548986160
This fixes two things in one go:
1. In order to indicate 'end of a cue' **without** relying on
`CuesWithTiming.durationUs`, `SubripParser` should have been emitting
an empty `List<Cue>` instead of the current behaviour of a
single-item list with `cue.text=""`.
2. There's no need for the empty cue (or cue list), we can use
`durationUs` to indicate the end of each list of cues.
There's no real need to ever have a `Cue` with `text=""`, so also
deprecate `Cue.EMPTY`.
PiperOrigin-RevId: 548938874
Reading a discontinuity from a media period indicates that a position
reset is required. As part of this event, the media period may need
further loading (e.g in a MergingMediaPeriod where one stream reported
a discontinuity and the other need to reload from this position).
This currently fails if the media periods was already fully loaded and
we started loading further items in the playlist. As a result, playback
is stuck forever. We can fix this by detecting that further loading is
needed and resetting the loading period to the current one.
The existing MergingPlaylistPlaybackTest already covers this case
reliably, because it combines all the right preconditions (merging
source, clipping to get a discontinuity and a playlist).
PiperOrigin-RevId: 548735177
This is a step towards adding general support for overlapping
subtitles in these formats (and others), both muxed and sideloaded:
* Issue: google/ExoPlayer#10295
* Issue: google/ExoPlayer#4794
This change adds these files to the end-to-end playback tests too, but
the subtitle track is currently disabled because renderer-side subtitle
parsing causes flaky tests (due to an uncontrolled thread in
`SimpleSubtitleDecoder`). The subtitle track will be re-enabled in
a follow-up change when loading-side subtitle parsing is added (so the
tests will no longer be flaky). At this point the overlapping subtitles
**still** won't be supported end-to-end, but a second change will
resolve this will changes in `TextRenderer` - which will change the
end-to-end playback dumps to reflect the overlapping subtitles.
PiperOrigin-RevId: 548705032
For now, just identical timestamps. A follow-up CL will allow for different
framerates. Prior tests already send identical timestamp frames, so no new
tests were added
PiperOrigin-RevId: 548676759
`EGL14` was added in API 17, so move all references into the `Api17` wrapper
class. `GLES30` requires API 18.
Remove some defensive calls to check that a context is current. Checking for GL
errors after calls should catch the case of calling GL methods without a
context (which is also only likely to come up early in development), and these
checks were not done consistently across all methods anyway. That allows
removing the API 17 requirement from `SceneRenderer` (where previously it
wasn't documented).
PiperOrigin-RevId: 548669946
The streams return end-of-input if they read no samples, but know that
they are fully buffered to at least the clipped end time. This helps to
detect the end of stream even if there are no new buffers after the end
of the clip (e.g. for sparse metadata tracks).
The race condition occurs because the buffered position is evaluated
after reading the sample. So between reading "no sample" and checking
the buffered position, the source may have loaded arbitrary amounts
of data. This may lead to a situation where the source has not read
all samples, reads NOTHING_READ (because the queue is empty) and then
immediately returns end-of-stream (because the buffered position
jumped forward), causing all remaining samples in the stream to be
skipped. This can fixed by moving the buffered position check to
before reading the sample, so that it never exceeds the buffered
position at the time of reading "no sample".
#minor-release
PiperOrigin-RevId: 548646464
Also re-use the `CuesWithTimingSubtitle` implementation (previously a
private class inside `DelegatingSubtitleDecoder`) in `ExoPlayerCuesDecoder`.
PiperOrigin-RevId: 548612040
It will act similarly to `SubtitleDecoderFactory`, but return parsers instead of decoders. In turn `SubtitleDecoderFactory.createDecoder()` cab wrap those parsers with `DelegatingSubtitleDecoder`.
PiperOrigin-RevId: 548528054
`PgsDecoder` which used to be `SimpleSubtitleDecoder` will now be called `PgsParser` and implement `SubtitleParser` interface. For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `PgsParser` instance.
PiperOrigin-RevId: 548520549
This serves as a sort of test with more frames for the compositor for now
(before more varied video system tests come later when integrating with
Transformer), showing it doesn't error out and outputs the right
amount of frames.
Due to the VFPTestRunner having a 5s timeout, and mostly due to presubmit
emulators being very slow with OpenGL, there is a sort of limitation
to how many frames this type of test can have, depending on the test target
PiperOrigin-RevId: 548159762
`DvbDecoder` which used to be `SimpleSubtitleDecoder` will now be called `DvbParser` and implement `SubtitleParser` interface. There was, however, already a `DvbParser`, used by `DvbDecoder` behind the scenes. Hence, the refactoring only requires the existing `DvbParser` to adhere to the new `SubtitleParser` interface.
For backwards compatibility, we will have the same functionality as `DvbDecoder` provided by `DelegatingSubtitleDecoder` backed-up by a new `DvbParser` instance, available from the `SubtitleDecoderFactory`.
PiperOrigin-RevId: 548155759
`Tx3gDecoder` which used to be `SimpleSubtitleDecoder` will now be called `Tx3gParser` and implement `SubtitleParser` interface. For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `Tx3gParser` instance.
PiperOrigin-RevId: 548144492
Before this CL, the text with a scale span would wrap text, with because the scale wasn't taken into account when measuring the width of the overlay.
PiperOrigin-RevId: 548123626
Unlike ExoPlayer, when play/pause is called on MediaController, the playback suppression reason is set to default causing a change in it and subsequent callbacks on the Player.Listener.
PiperOrigin-RevId: 548088931
Add Ogg ID Header and Comment Header Pages to the Ogg encapsulated Opus for offload playback. This further matches the RFC 7845 spec and provides initialization data to decoders.
PiperOrigin-RevId: 548080222
These tests allow to verify the samples sent for decoding,
when decoders are reset, which samples are dropped as decode-only
for video and which samples are sent to the AudioTrack for playback.
The test exercises all combinations of merges where audio or video
is the primary track and where audio, video or both are clipped.
PiperOrigin-RevId: 548061254
These tests allow to verify the samples sent for decoding,
when decoders are reset, which samples are dropped as decode-only
for video and which samples are sent to the AudioTrack for playback.
The test exercises all combinations of clipping transitions for
sources that are either clipped at the start, the end, or both.
PiperOrigin-RevId: 547730824
When a renderer is pre-enabled (while another playback is still
ongoing), we pass mayRenderStartOfStream=false to Renderer.enable.
This ensures we don't show any first frames while the previous media
is still playing.
Currently, we never tell the renderer when we actually stop playing
the previous media so that it could render the start of the stream,
because we allow this as soon as the renderer is in STATE_STARTED and
we assume that we have to be in STATE_STARTED to make this stream
transition.
While this assumption is true, there are also cases where we can't
start the renderers because they are not ready yet and the video
renderer can't become ready because it didn't render its first frame.
This effectively blocks playback forever.
The most direct way of solving this, is to tell the renderer that
playback has transitioned and that it is now allowed to render the
start of the stream. This means it can never get blocked as described
above.
PiperOrigin-RevId: 547727347
SingleInputVideoGraph implements GraphInput now, so the asset loaders would
interface directly with SIVG, rather than VideoSampleExporter. This is to pave
way for multi-asset video processing.
PiperOrigin-RevId: 547561042
The setPlayerCommand and setSessionCommand builder methods contain assertions to ensure that at most one of these fields is set, but this left it possible to create a command button with no command by calling build on an empty builder.
PiperOrigin-RevId: 547488248
This change also marks the buttons of the custom layout as
enabled/disabled according to available commands in the controller.
Accordingly, `CommandButton.Builder.setEnabled(boolean)` is deprecated
because the value is overridden by the library.
Issue: androidx/media#38
#minor-release
PiperOrigin-RevId: 547272588
`Mp4WebvttDecoder` which used to be `SimpleSubtitleDecoder` will now be called `Mp4WebvttParser` and implement `SubtitleParser` interface. For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `Mp4WebvttParser` instance.
PiperOrigin-RevId: 547248157
* Allow more than one input bitmap at a time.
* Allow Compositor to take in and set up an Executor. Otherwise,
Compositor resources may be created on one thread and accessed on another.
* Add a Compositor TestRunner to reuse test code more.
* Update VideoFrameProcessingTaskExecutor to use a new onError listener, so
that it's more reusable in non-DVFP contexts, like for Compositor.
PiperOrigin-RevId: 547206053
durationUs is almost always going to be a larger number than the sample
rate, so pass as the main value, rather than the multiplier.
PiperOrigin-RevId: 547193927
For sync-sample-only formats, we have an optimization to drop all buffers
with less than the start time when writing them to the queue.
For the same formats, if we set a new start time (=seek), we only seek
to the buffer at or before the start time. This means the first sample
in the queue is different depending on whether we seek to a start time
or set a start time and then write samples. This is inconsistent and
effectively means the first sample depends on a race condition between
the Loader thread (writing samples) and the playback thread (attempting
an initial seek in the already loaded samples).
The effect of this inconsistency is that we have to decode one sample
we don't need (and could have skipped) and that some tests become flaky
if the test setup runs into the mentioned race condition.
The fix is to change the SampleQueue seek method to also seek to
a sample at or after the specified time, to align the behavior to the
case where we write the same samples to an empty queue.
The change also clarifies the Javadoc of
MimeTypes.allSamplesAreSyncSamples to note that this should really only
return true if the samples have no "duration" that matters. Otherwise,
we could reasonably return true for most subtitle formats although it
would break subtitle display because we'd remove samples that start
before the seek time.
PiperOrigin-RevId: 547189941
This is needed for several subtitle formats where the `data` passed to
`SubtitleParser` doesn't contain any cue timing, such as PGS, CEA-608,
CEA-708, and DVB.
PiperOrigin-RevId: 547177127
TransformerAndroidTestRunner#run will throw any exception that occurs,
so there is no need to assert the result exception is null.
PiperOrigin-RevId: 546923823
We currently only log the input buffers to the codec, but the test
will be more complete and provide more insights into the playback
behavior if we also dump more information about these input buffers
(timestamps and flags) and also all output buffers with their
timestamps and whether they are rendererd (only relevant for video).
And also use the CapturingAudioSink to output the audio data we produce.
PiperOrigin-RevId: 546898937
Also parse the PCM encoding for lpcm in MP4, and update `MatroskaExtractor`
similarly.
Tested manually in the demo app using an MP4 with 24-bit big endian audio.
PiperOrigin-RevId: 546878505
Some adjustments to the test to make it more correct and prevent it
from becoming flaky:
- Use separate output dump files per test setup. Once we add more data
to these files, they are not guaranteed to be the same anymore.
- Use a seek position that is actually behind the midroll as described
in the test setup.
- Change ad insertion position to ensure the ad group duration doesn't
exceed the underlying media duration.
- Add a wait for isLoading to ensure the late insertion of an ad group
happens consistently at the same processing stage.
PiperOrigin-RevId: 546825183
This changes all MediaSources in our library to allow updates to
their MediaItems (if supported).
Issue: google/ExoPlayer#9978
Issue: androidx/media#33
PiperOrigin-RevId: 546808812
This removes some stack traces in unit tests when a
`RemoveMediaController` was created with `waitForConnection=false`
in which case the missing controller in the
`MediaControllerProviderService` caused a `NullPointerException`.
PiperOrigin-RevId: 546795181
`SubripDecoder` which used to be `SimpleSubtitleDecoder` will now be called `SubripParser` and implement `SubtitleParser` interface. For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `SubripParser` instance.
PiperOrigin-RevId: 546538113
`SsaDecoder` which used to be `SimpleSubtitleDecoder` will now be called `SsaParser` and implement `SubtitleParser` interface. For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `SsaParser` instance.
PiperOrigin-RevId: 546336035
`DelegatingSubtitleDecoder` is a `SimpleSubtitleDecoder` that delegates to a `SubtitleParser` instance (e.g. future implementations like `WebvttParser`, `SubripParser`, `SsaParser`, `DvbParser` etc)
It will be in the `exoplayer` package, rather than `extractor` and only be used in the decoding flow which happens after SampleQueue (and to be deprecated). When this decoding flow is deprecated and removed, the class can be later deleted.
PiperOrigin-RevId: 546317528
Frame buffers are focused, not current. EGL Contexts may be current but this
is a prerequisite for focusing a framebuffer, and this alone isn't
sufficient to make a bitmap.
Clear this up as it can make some complex tests harder to understand, if the
methods needed don't match up in naming.
PiperOrigin-RevId: 546305118
Add documentation for threading requirements at the class level (in
addition to existing documentation on the methods) to improve
discoverablility. Also fix a couple of nits in the javadoc (US English
spelling, avoid passive voice) and in `OnInputFrameProcessedListener`.
PiperOrigin-RevId: 546303732
Otherwise, errors like `GL_INVALID_FRAMEBUFFER_OPERATION` will only
report a `null` error string, instead of the proper error string.
PiperOrigin-RevId: 546273328
This makes tests more realistic because the returned value matches
the rest of the simulated test time.
It also prevents test flakiness in (yet to be written) tests that
may not correctly advance the playback time in the position tracker.
PiperOrigin-RevId: 546011842
When selecting progressive and HLS tracks, we currently check if
we can either seek in the existing samples or are starting from the
beginning of the stream. In both cases, we don't need to reload the
stream and can continue reading.
Seeking to the beginning isn't side-effect free though because it
also sets the startTimeUs to zero. Whether a seek to the beginning
is successful also depends on whether we already loaded a sample or
not. This mean that the startTimeUs value is set (or not set) based
on the speed of the Loader thread even for the same input starting
from the beginning. This race condition means the actual samples
we write to the stream may differ if the stream has initial samples
with negative timestamps.
We can avoid this race condition by checking if the queue is empty
first, so that we only call seekTo if really needed.
PiperOrigin-RevId: 546010951
Robolectric writes warnings to the log output because the Surfaces
we create are not released and pollute the test environment. This
causes performance issues if tests are run repeatedly.
PiperOrigin-RevId: 545906757
We have the optimization to skip buffers if the release time is
exactly the same as the one for the previous buffer. This makes
sense under the assumption that these buffers get released to
a visible Surface and will be ignored anyway.
However, it's also helpful to provide a customization option to not
do this for cases where the outputting MediaCodecAdapter is not
directly talking to a visible Surface or for tests where we can't
fully control the vsync timing of the Surface and want to ensure
we output all samples that are meant to be shown.
PiperOrigin-RevId: 545906113
BaseGlShaderProgram and SingleFrameGlShaderProgram are technically color space agnostic. useHdr is passed to the TexturePool, which only uses the value to choose between high/regular precision. useHdr is therefore a bit misleading and should be generalized to "useHighPrecisionColorComponents."
PiperOrigin-RevId: 545726629
This makes tests more realistic because the returned value matches
the rest of the simulated test time.
It also prevents test flakiness in (yet to be written) tests that
may accidentally drop output buffers or calculate the wrong values.
PiperOrigin-RevId: 545690008
Based on the spec, ETSI TS 102 366 V1.4.1 Annex F, 6 bits should have skipped instead of 6 bytes.
This correction was pointed out in Issue: androidx/media#474.
PiperOrigin-RevId: 545658365
This allows us to simplify and reduce thread initialization, hops and the
synchronization cost associated with them, especially when more and more VFPs
are used.
PiperOrigin-RevId: 545622169
Upon track transition of offloaded playback of gapless tracks, the framework will reset the playback head position. The AudioTrackPositionTracker must be made to expect the reset and cache accumulated sum of rawPlaybackHeadPosition.
#minor-release
PiperOrigin-RevId: 545602979
This allows MediaSources to accept MediaItem updates after creation.
This CL adds the handling and plumbing logic in `ExoPlayerImpl`,
`ExoPlayerImplInternal`, `MediaSourceList` and `MaskingMediaSource`.
It also updates all forwarding/wrapping sources to forward these calls
to their wrapped instance.
The actual functionality is only added to `FakeMediaSource` instances in
tests so far.
PiperOrigin-RevId: 545450210
This plumbs a duration through `SampleQueue` which will make it easier
to handle transcoding muxed subtitles from e.g. SubRip to
`text/x-exoplayer-cues`.
This change is a no-op to the end-to-end behaviour of ExoPlayer because
currently we only support parsing sideloaded subtitles before
`SampleQueue`, and by adding the duration we don't affect the cues that
are ultimately output by `Player.Listener.onCues` (as shown by no change
to the golden files for `WebvttPlaybackTest` in this commit).
I considered making `CuesWithStartTimeAndDuration` implement
`Bundleable` (and deleting `CueEncoder/Decoder`) but decided against
it because we are deliberately not encoding `startTimeUs` (since that's
encoded as the sample time in `SampleQueue`). I also considered
introducing another type that only has `List<Cue>` and `durationUs`
fields, but it didn't seem necessary, since we want `startTimeUs`
everywhere else (except inside `SampleQueue`).
PiperOrigin-RevId: 545226847
Also make some casting that is needed a bit safer.
Error Prone was complaining about casting `long` to `int` before passing
it to `Extractor.seek(long, long)`:
> Converting a `long` or `Long` to an `int` to pass as a `long`
> parameter is usually not necessary. If this conversion is intentional,
> consider `Longs.constrainToRange()` instead.
PiperOrigin-RevId: 545226063
This is a corollary of `SubtitleDecoderFactory`. We may add a
`boolean supportsFormat(Format)` method later if it turns out to be
useful.
PiperOrigin-RevId: 545225489
The timeout time of 5000ms is quite long, and means that for tests with >1
VFPTestRunner, like texture output and compositor tests, we have to wait 10s.
The latch allows us to continue with the test once we're done processing input,
instead of needing to wait longer, until the timeout is reached.
PiperOrigin-RevId: 545159839
The actual errors are all positive hex values. Without this CL, we must first
convert decimal errors to hex ones before figuring out what went wrong.
PiperOrigin-RevId: 544695961
We introduced truncation to 32 chars in <unknown commit>
and included indent and offset in the calculation. I think this is
technically correct, but it causes problems with the content in
Issue: google/ExoPlayer#11019 and it doesn't seem a problem to only truncate actual
cue text (i.e. ignore offset and indent).
#minor-release
PiperOrigin-RevId: 544677965
This avoids issues with potential number clashes with ID_UNSET.
Also fixes some further parsing where adaptation set ids are referenced
from manifest properties.
Auto-resume playback when the removal of playback suppression due to unsuitable output is conveyed via change in playback suppression to Player.PLAYBACK_SUPPRESSION_REASON_NONE within a configurable timeout defaulting to 5 minutes.
PiperOrigin-RevId: 544411987
Instead of playing or pausing itself, the ExoPlayer implementation should only update the playback suppression reason as and when audio outputs are added or removed dynamically.
PiperOrigin-RevId: 544379033
On a MediaItem change, the input Format (and Effects to apply) may be
different. Therefore the AudioProcessingPipeline must be reconfigured
to determine what processing is active, and what the AudioFormat of the
data output is. In the event that it is different, additional
AudioProcessor instances must be used to ensure the encoder will still
be able to accept the audio buffers.
PiperOrigin-RevId: 544338451
Add a fail-fast check in `ExoPlayerImpl` to ensure the equality of the lengths of `ShuffleOrder` and the current playlist. Also improve the documentation of `setShuffleOrder(ShuffleOrder)` with explicit instruction on this.
Issue: androidx/media#480
#minor-release
PiperOrigin-RevId: 544009359
Providing the sync token in the api allows the client to decide which waiting method they would like to use depending on the use case, allowing them to optimise if possible.
PiperOrigin-RevId: 543997311
This fixes a broken javadoc link and makes it possible to customize media
source creation via the asset loader factory without also having to
implementing a decoder factory from scratch.
PiperOrigin-RevId: 543995538
It currently wrongly documents that it is only called before reading
streams (that has never been the case and all MediaPeriods already need
to handle calls after reading samples from the streams).
It was also a bit unclear what a discontinuity implies and the new
Javadoc calls out the main use case for discontinuties and the intended
meaning of returning a discontinuity.
#minor-release
PiperOrigin-RevId: 543989124
We use `@Nullable` for IntDefs in other places and don't use this suffix
elsewhere, so I don't think we need it here either.
PiperOrigin-RevId: 543687285
Goal of tests (SequenceExportTest) that use this media is for the
silence and the media to match exactly with audio format, however
`sample_with_increasing_timestamps.mp4` had a different sample rate.
testvid_1022ms.mp4: channel count = 2, sample rate = 44100.
PiperOrigin-RevId: 543458948
MP4 edit lists sometimes ask to start playback between two samples.
If this happens, we currently change the timestamp of the first
sample to zero to trim it (e.g. to display the first frame for a
slightly shorter period of time). However, we can't do this to audio
samples are they have an inherent duration and trimming them this
way is not possible.
#minor-release
PiperOrigin-RevId: 543420218
*** Original commit ***
Mark output sample as decode-only based on start time
We currently do the same check on the input timestamps and
expect the output timestamps to match. Some codecs produce
samples with modified timestamps and the logic is a lot safer
when the comparison with the start time is done on the output
side of the codec.
Issue: google/ExoPlayer#11000
***
PiperOrigin-RevId: 543379665
FallbackListener.onTransformationRequestFinalized() is called from the
AssetLoader thread for audio, and from the GL thread for video.
PiperOrigin-RevId: 542851284
By passing this class where it's needed, implementations don't need to store it
in a field (reducing boilerplate) and it's clearer that it can't be unset when
needed.
PiperOrigin-RevId: 542823522
`MediaControllerImplBase` has 2 methods for updating listeners about `PlayerInfo` changes - `updatePlayerInfo` (for masking the state) and `onPlayerInfoChanged` (when communicating with the session). There is a set number of listener callbacks related to `PlayerInfo` updates and both methods should go through the same control flow (whether we know that masking will ignore most of them or not).
A unified method `notifyPlayerInfoListenersWithReasons` encapsulates only the shared logic of 2 methods - listeners' callbacks. This ensures that both methods call them in the same order and none are missed out.
PiperOrigin-RevId: 542587879
With the upcoming "handle format changes" CL, stereo -> mono audio
would add an AudioProcessor. Robolectric decodes output encoded data,
which crashes some AudioProcessors because the number of frames may not
be an integer.
PiperOrigin-RevId: 542568875
This change uses this new method everywhere we currently `instanceof`
check an `Extractor` directly. This allows us to introduce
wrapping/delegating `Extractor` instances - because the `instanceof`
checks will continue to operate on the underlying instance.
HLS is a slightly different case, because it directly re-instantiates
`Extractor` instances, which is not compatible with an arbitrary
wrapping structure. Luckily the only `Extractor` instances that HLS
re-instantiates do not support muxed subtitles, so won't be wrapped
in the first place (although future changes might use the
delegating-`Extractor` pattern for other purposes, which might affect
HLS).
PiperOrigin-RevId: 542550928
The callbacks for `PlayerInfo` changes are currently in both `MediaControllerImplBase.updatePlayerInfo` (masking) and `MediaControllerImplBase.onPlayerInfoChanged`. But the order was different between them both and `ExoPlayerImpl.updatePlaybackInfo` which they are trying to mimic.
#minor-release
PiperOrigin-RevId: 542519070
Devices pre-API 33 are not able to comprehend the position reset that occurs by the HAL in offloaded gapless track transitions.
PiperOrigin-RevId: 542503662
The effects pipeline must receive images in the sRGB colorspace due to the color transfers applied in the shaders. Currently the burden to making sure images are in the right colorspaces falls onto apps. This CL ensures that this is not the case anymore.
PiperOrigin-RevId: 542323613
When an app tried to re-prepare a live streeam with server side inserted
ad after a playback exception, the player tried to find the ad group by
its index in the ad playback state of the next timeline when creating
the first period.
If a source that supports server side ad, has removed the ad playback
state when the source has been removed, this causes a crash. For live
streams this is a reasonable thing to do given the exception could be
caused by an invalid ad playback state.
This change removes the ad metadata from the current period for live
streams and the timeline. In case the ad playback state is not reset
by the source, the first timeline refresh would ad the metadata again.
PiperOrigin-RevId: 541959628
Some events may arrive after the playlist is cleared (e.g. load
cancellation). In this case, the DefaultPlaybackSessionManager may
create a new session for the already removed item.
We already have checks in place that ignore events with old
windowSequenceNumbers, but these checks only work if the current
session is set (i.e. the playlist is non-empty). The fix is to add
the same check for empty playlists by keeping note of the last
removed window sequence number.
PiperOrigin-RevId: 541870812
FakeClock currently doesn't work well with Espresso and Compose UI
tests because view interactions in both frameworks intentionally idle
the main looper to handle pending UI effects. However, this also
advances playback progress even though we want to deterministically
trigger progress from the test itself.
To solve this problem, we can detect the idling Robolectric call and
postpone any further updates until we leave this state.
PiperOrigin-RevId: 541831050
Add a Wear OS specific implementation of 'Player.Listener' to help resolving the playback suppression due to unsuitable output by launching a system media output switcher dialog.
PiperOrigin-RevId: 541698125
