Renderers may be enabled for subsequent media items as soon as the current media item's renderer's isEnded() returns true. When a renderer is being enabled and the player is 'playing', that renderer is also started. When playing a mixed playlist of images and content with audio & video, the player may skip some image items because the early-starting of the audio renderer causes a clock update.
A solution is to only start the "early-enabled" renderers at the point of media transition and add a condition on DefaultMediaClock to use the standalone clock when reading-ahead and the renderer clock source is not in a started state.
Issue: androidx/media#1017
PiperOrigin-RevId: 613231227
(cherry picked from commit 638b2a3c86)
This change aims to prioritise tracks that have a 'smooth enough for
video' frame rate, without always selecting the track with the highest
frame rate.
In particular MP4 files extracted from motion photos sometimes have two
HEVC tracks, with the higher-res one having a very low frame rate (not
intended for use in video playback). Before this change
`DefaultTrackSelector` would pick the low-fps, high-res track.
This change adds a somewhat arbitrary 10fps threshold for "smooth video
playback", meaning any tracks above this threshold are selected in
preference to tracks below it. Within the tracks above the threshold
other attributes are used to select the preferred track. We deliberately
don't pick the highest-fps track (over pixel count and bitrate), because
most users would prefer to see a 30fps 4k track over a 60fps 720p track.
This change also includes a test MP4 file, extracted from the existing
`jpeg/pixel-motion-photo-2-hevc-tracks.jpg` file by logging
`mp4StartPosition` in
[`MotionPhotoDescription.getMotionPhotoMetadata`](b930b40a16/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/MotionPhotoDescription.java (L123))
and then using `dd`:
```
mp4StartPosition=2603594
$ dd if=jpeg/pixel-motion-photo-2-hevc-tracks.jpg \
of=mp4/pixel-motion-photo-2-hevc-tracks.mp4 \
bs=1 \
skip=2603594
```
----
This solution is in addition to the `JpegMotionPhotoExtractor` change
made specifically for these two-track motion photos in
5266c71b3a.
We will keep both changes, even though that change is not strictly
needed after this one, because adding the role flags helps to
communicate more clearly the intended usage of these tracks. This
change to consider FPS seems like a generally useful improvement to
`DefaultTrackSelector`, since it seems unlikely we would prefer a 5fps
video track over a 30fps one.
Issue: androidx/media#1051
PiperOrigin-RevId: 611015459
(cherry picked from commit c7e00b12b4)
This change makes ExoPlayerImplInternal.releaseInternal() unblock the
app thread if a runtime exception is thrown while releasing components
from the playback thread.
Before this change, if a runtime exception occurred during releasing
components in the playback thread, ExoPlayer.release() would wait for
`releaseTimeoutMs` and then raise a player error. With this change,
the player error is reported only when the playback thread is blocked
but if there is a runtime exception, the application thread is
unblocked.
The impact of this change is potentially fewer ANRs on
ExoPlayer.release() at the expense of less error reporting.
PiperOrigin-RevId: 609702549
(cherry picked from commit 0480eff6a1)
We forgot to add it when we added AudioSink.release(). The commit
includes a test that ensures ForwardingAudioSink overrides all the
methods defined in the AudioSink interface.
PiperOrigin-RevId: 609402258
(cherry picked from commit 440d2ab162)
If the reading period has already advanced and a track reselection procs that only affects the reading period media, then ExoPlayer may try and apply the reading period's track selection incorrectly unto the playing period. ExoPlayer should apply the playing period's track selection to the playing period instead.
PiperOrigin-RevId: 609375077
(cherry picked from commit 4192924622)
The `PreloadMediaPeriod.selectTracksForPreloading` can be called for multiple times at the preloading stage (before the period is being played). For example, when the period resumes preloading. This change fix the assertion failure in `ProgressiveMediaPeriod.selectTracks` caused by the wrong implementation of `PreloadMediaPeriod.selectTracksForPreloading` when it is trying to retain the previously preloaded streams.
Also the `TrackSelectorResult` parameter is changed to a list of `ExoTrackSelection`. We should compare the selections only rather than considering the `RendererConfiguration` in the `TrackSelectorResult` to decide whether to retain the streams, as for preloading case the renderers haven't consumed the samples yet.
PiperOrigin-RevId: 609126868
(cherry picked from commit d952a06214)
`Uri.appendQueryParameter` is documented to encode its arguments, so
calling `Uri.encode` beforehand results in double-encoding.
Issue: androidx/media#1075
#minor-release
PiperOrigin-RevId: 604995441
(cherry picked from commit 7ebfed505c)
These are often the same for image tracks, since we usually drop the
whole image file (both the container and actual encoded image bytes)
into a single sample, but there are cases where we emit a track with
`containerMimeType=image/jpeg` but **no** samples (from
`JpegMotionPhotoExtractor`, to carry some metadata about the image +
video byte offsets).
It's therefore more correct to implement the `supportsFormat` check
based on `sampleMimeType`, so that these 'empty' image tracks are not
considered 'supported' by `ImageRenderer`.
#minor-release
PiperOrigin-RevId: 604672331
(cherry picked from commit eabba49610)
This was already done for the TrackGroup ids in <unknown commit>,
but in some scenarios only the Format instances are known and
it's helpful to be able to identify where they came from.
Issue: androidx/media#883
#minor-release
PiperOrigin-RevId: 604644039
(cherry picked from commit f8f6d80477)
Add a test for this consistency in `CompositeSequenceableLoaderTest`,
and also make the
`CompositeSequenceableLoaderTest.FakeSequenceableLoader` implementation
more realistic.
#minor-release
PiperOrigin-RevId: 604604103
(cherry picked from commit db74bb9609)
If seeking between last image sample and end of the file where the current stream is not final, then EoS sample will not be provided to `ImageRenderer`. ImageRenderer must still produce the last image sample.
PiperOrigin-RevId: 603312090
(cherry picked from commit 62c7ee0fb0)
The updated algorithm has two main improvements:
- The silence padding is not constant but a ratio of the original
silence (up to a defined max) to more naturally represent the
original gaps.
- The silence is not instantly going to zero, but uses a ramp down
and up for a smooth volume transition and also retains a small
percentage of the original noise for more natural "silences" that
still contain some background noise.
#minor-release
Issue: google/ExoPlayer#7423
PiperOrigin-RevId: 602322442
(cherry picked from commit bb533332f4)
This method works by reflectively loading classes from the `lib-effect`
module, in order to avoid a hard dependency on this module for ExoPlayer
users that don't want video effect functionality. This change ensures
that a failure to load the necessary classes fails immediately, instead
of on a later thread inside `MediaCodecVideoRenderer` when the
reflection currently happens.
Also update the javadoc to make the dependency requirement clear.
#minor-release
PiperOrigin-RevId: 601387957
(cherry picked from commit a6812156e6)
This allows us to inject a videoFrameProcessorFactory into
MediaCodecVideoRenderer, without issues about creating the
VideoFrameReleaseControl in the MediaCodecVideoRenderer.
Unfortunately, this does result in more complex CVSP state, where
VideoFrameReleaseControl is no longer final, may be null, and may potentially
change. However, this tries to be careful with assertions to guarantee good
state, and is cleaner than modifying the long-standing MediaCodecVideoRenderer
interface.
Tested that this works on the ExoPlayer demo with setVideoEffects applied, and
using a playlist with SDR->HDR and HDR->SDR items.
PiperOrigin-RevId: 599823412
(cherry picked from commit cb0f5a7fff)
When we default to 'parse during extraction', we will flip the default
of this, to ensure that apps know they are using an
incompatible/deprecated flow for subtitle handling.
PiperOrigin-RevId: 599109304
Originally a `PlayerId` has to be passed and it should be the same id of the player who is going to play the sources, but it turns out to be unnecessary.
When preloading, we can set a `PlayerId.UNSET` inside of the `PreloadMediaSource`, as there is no ongoing playback. And when the source is handed over to player, player will set the player's `PlayerId`.
PiperOrigin-RevId: 597475119
This behavior was previously available as opt-in via
`MediaItem.DrmConfiguration.Builder.setPlayClearContentWithoutKey` and
`DefaultDrmSessionManager.Builder.setPlayClearSamplesWithoutKeys`. This
change flips the default of both these properties to true.
This should speed up the time for playback to start when playing DRM
content with a 'clear lead' of unencrypted samples at the start.
Previously playback would wait until the keys for the later encrypted
samples were ready. The new behaviour could result in mid-playback
stalls/rebuffers if the keys are not ready yet by the transition from
clear to encrypted samples, but this is not really a regression since
previously playback wouldn't have started at all at this point.
PiperOrigin-RevId: 595992727
This file is CBR encoded with LAME, so it has an `Info` header (the CBR
equivalent to `Xing`).
A follow-up change will use this file in `Mp3ExtractorTest`.
Issue: androidx/media#878
PiperOrigin-RevId: 595938327
These were missing in 5211ff0dc1
Without these, the `": null"` is still logged (but the `"Unexpected
IllegalStateException"` bit is not), because the **whole** string
concatenation is compared to null as the boolean condition of the
ternary, and this condition is always false (the result of a string
concatentation is never `null`) i.e. (with excess parentheses for
clarity):
```
(("Unexpected " + cause.getClass().getSimpleName() + cause.getMessage()) != null)
? ": " + cause.getMessage()
: ""
```
Also add a test because obviously this isn't as simple as I'd thought.
PiperOrigin-RevId: 593079975
This is a refactoring that allows the `MediaPeriodQueue` to
create media period holders without all the collaborators
being passes in to `enqueueNextMediaPeriodHolder(...)`.
The factory is presumably also helpful in unit test to
know whether and when exactly a holder is created in
the preloading process.
PiperOrigin-RevId: 592301400
The parameters may change the decoding behavior of the
following samples and it's suprising/wrong to apply them
from another thread where we can't guarantee from which
sample they apply.
PiperOrigin-RevId: 591808760
When an adaptive source has been preloaded, the track selection
of the player should possibly not override the preloaded selection
to avoid discarding preloaded data.
PiperOrigin-RevId: 591256334
Vertical and horizontal tile counts inside Format used by ImageRenderer
used to be required to equal 1 and were set to 1 regardless of what
the manifest said.
This change removes the above requirement and sets the tile counts to
the values from the manifest.
PiperOrigin-RevId: 590608353
We currently pause playback to prevent further progress while the
app thread runs assertion and triggers additional actions. This is
not ideal because we do not actually want to pause playback in
almost all cases where this method used.
This can be improved by keeping the playback thread blocked and only
unblock it the next time the app thread waits for the player (either
via RobolectricUtil methods or by blocking the thread itself). To
add this automatic handling, this change introduces a new util class
for the tests that can keep the list of waiting threads statically
(because the access to this logic is spread across multiple independent
classes).
PiperOrigin-RevId: 589784204
Fix a bug where CompositingVideoSinkProvider.isInitialized() returns
true even after releasing the CompositingVideoSinkProvider.
PiperOrigin-RevId: 588481744
This change moves the instantiation of the CompositingVideoSinkProvider
out of MediaCodecVideoRenderer so that the composition preview player can
re-use the CompositingVideoSinkProvider instance for driving the rendering of
images.
The main point of the change is the ownership of the
VideoFrameReleaseControl, which decides when a frame should be rendered
and so far was owned by the MediaCodecVideoRenderer. With this change,
in the context of composition preview, the VideoFrameReleaseControl
is no longer owned by MediaCodecVideoRenderer, but provided to it.
This way, the CompositingVideoSinkProvider instance, hence the
VideoFrameReleaseControl can be re-used to funnel images into the
video pipeline and render the pipeline from elsewhere (and not
MediaCodecVideoRenderer).
PiperOrigin-RevId: 588459007
This image has two video tracks in the MP4 data, one is a 'real' video
which we want to play by default, and the other is a low-fps video track
which isn't intended to be directly played, it's encoded in HEVC for
compression and decoding efficiency.
This test demonstrates ExoPlayer's current behaviour default extraction
and playback, which results in selecting the high-res, low-fps track
(actually single sample in this example), instead of playing the actual
video.
PiperOrigin-RevId: 588068908
We set the `playerId` only in the constructor currently. But in the case of this source doesn't have `preload` called, the player's release of this source will set the playerId to `null`, which makes this source un-reusable with a null `playerId`.
PiperOrigin-RevId: 587698214
Move the reflective loading of VideoFrameProcessor from
MediaCodecVideoRenderer to the CompositingVideoSinkProvider. This is so
that all reflective code lives in one place. The
CompositingVideoSinkProvider already has reflective code to load the
default PreviewingVideoGraph.Factory.
PiperOrigin-RevId: 584852547
Split CompositingVideoSinkProvider.VideoSinkImpl in two classes:
- VideoSinkImpl now only receives input from MediaCodecVideoRenderer and
forwards frames to its connected VideoFrameProcessor
- VideoFrameRenderControl takes composited frames out of the VideoGraph
and schedules the rendering of those.
- CompositingVideoSinkProvider connects VideoSinkImpl with
VideoFramesRenderer.
PiperOrigin-RevId: 584605078
The live offset override is used to replace the media-defined
live offset after user seeks to ensure the live adjustment adjusts
to the new user-provided live offset and doesn't go back to the
original one.
However, the code currently clips the override to the min/max
live offsets defined in LiveConfiguration. This is useful to
clip the default value (in case of inconsistent values in the media),
but the clipping shouldn't be applied to user overrides as
the player will then adjust the position back to the min/max
and doesn't stay at the desired user position.
See 2416d99857 (r132871601)
#minor-release
PiperOrigin-RevId: 584311004
Currently, ads are only defined by a single URL, which makes it
impossible to define additional fields needed to play ads correctly.
This can be fixed by using a full MediaItem in AdPlaybackState,
replacing the previous Uri field.
PiperOrigin-RevId: 582331588
Both interfaces are not really needed as the methods can be called
on the respective classes directly. This also avoid error-prone
situations where classes define to/fromBundle methods with parameters
that should be used instead of the parameter-less version.
Also deprecate all existing CREATOR static instances and make the
corresponding fromBundle method public where needed.
PiperOrigin-RevId: 579189766
DefaultTrackSelector now has all logic necessary for selecting an
image track.
If isPrioritizeImageOverVideoEnabled is set to true, image track will
try to be selected first and a video track will only be selected if no
image track is available. If isPrioritizeImageOverVideoEnabled is set
to false, image track will be selected only if video track wasn't
selected.
PiperOrigin-RevId: 578806006
