Commit graph

345 commits

Author SHA1 Message Date
ibaker
3a7a665d5d Rollback of 406c0a15be
PiperOrigin-RevId: 605015994
2024-02-07 09:56:59 -08:00
ibaker
7ebfed505c Stop double-encoding CMCD query parameters
`Uri.appendQueryParameter` is documented to encode its arguments, so
calling `Uri.encode` beforehand results in double-encoding.

Issue: androidx/media#1075

#minor-release

PiperOrigin-RevId: 604995441
2024-02-07 08:44:33 -08:00
ibaker
eabba49610 Check sampleMimeType rather than containerMimeType for images
These are often the same for image tracks, since we usually drop the
whole image file (both the container and actual encoded image bytes)
into a single sample, but there are cases where we emit a track with
`containerMimeType=image/jpeg` but **no** samples (from
`JpegMotionPhotoExtractor`, to carry some metadata about the image +
video byte offsets).

It's therefore more correct to implement the `supportsFormat` check
based on `sampleMimeType`, so that these 'empty' image tracks are not
considered 'supported' by `ImageRenderer`.

#minor-release

PiperOrigin-RevId: 604672331
2024-02-06 09:31:37 -08:00
tonihei
f8f6d80477 Add source prefix to MergingMediaPeriod Format ids
This was already done for the TrackGroup ids in <unknown commit>,
but in some scenarios only the Format instances are known and
it's helpful to be able to identify where they came from.

Issue: androidx/media#883

#minor-release

PiperOrigin-RevId: 604644039
2024-02-06 07:45:42 -08:00
ibaker
db74bb9609 Clearly define the consistency requirements for SequenceableLoader
Add a test for this consistency in `CompositeSequenceableLoaderTest`,
and also make the
`CompositeSequenceableLoaderTest.FakeSequenceableLoader` implementation
more realistic.

#minor-release

PiperOrigin-RevId: 604604103
2024-02-06 04:25:14 -08:00
huangdarwin
a27511cc50 Effect: Remove inputColorInfo from create() methods.
Transformer export and ExoPlayer previewing both read inputColorInfo from
registerInputStream now, instead of maintaining a consistent input color
throughout multiple streams in a sequence.

Therefore, we can remove inputColor-related arguments and methods now.

PiperOrigin-RevId: 603423509
2024-02-01 11:09:57 -08:00
michaelkatz
62c7ee0fb0 Render last image despite not receiving EoS
If seeking between last image sample and end of the file where the current stream is not final, then EoS sample will not be provided to `ImageRenderer`. ImageRenderer must still produce the last image sample.

PiperOrigin-RevId: 603312090
2024-02-01 02:54:36 -08:00
tonihei
bb533332f4 Update skip silence algorithm
The updated algorithm has two main improvements:
 - The silence padding is not constant but a ratio of the original
   silence (up to a defined max) to more naturally represent the
   original gaps.
 - The silence is not instantly going to zero, but uses a ramp down
   and up for a smooth volume transition and also retains a small
   percentage of the original noise for more natural "silences" that
   still contain some background noise.

#minor-release

Issue: google/ExoPlayer#7423
PiperOrigin-RevId: 602322442
2024-01-29 02:33:28 -08:00
ibaker
a6812156e6 Throw immediately from ExoPlayer.setVideoEffects() if dep not found
This method works by reflectively loading classes from the `lib-effect`
module, in order to avoid a hard dependency on this module for ExoPlayer
users that don't want video effect functionality. This change ensures
that a failure to load the necessary classes fails immediately, instead
of on a later thread inside `MediaCodecVideoRenderer` when the
reflection currently happens.

Also update the javadoc to make the dependency requirement clear.

#minor-release

PiperOrigin-RevId: 601387957
2024-01-25 02:43:35 -08:00
michaelkatz
688622eb47 Keep stream offset alive in ImageRenderer until stream transition
Fix modeled after OutputStreamInfo usage for stream offset in `MediaCodecRenderer`

PiperOrigin-RevId: 601109900
2024-01-24 07:10:04 -08:00
tonihei
c6bf380d50 Fix cleared metadata when repeating the same item
Issue: androidx/media#1007

#minor-release

PiperOrigin-RevId: 600477540
2024-01-22 09:20:36 -08:00
huangdarwin
cb0f5a7fff Previewing: set VideoFrameReleaseControl after CVSP is created.
This allows us to inject a videoFrameProcessorFactory into
MediaCodecVideoRenderer, without issues about creating the
VideoFrameReleaseControl in the MediaCodecVideoRenderer.

Unfortunately, this does result in more complex CVSP state, where
VideoFrameReleaseControl is no longer final, may be null, and may potentially
change. However, this tries to be careful with assertions to guarantee good
state, and is cleaner than modifying the long-standing MediaCodecVideoRenderer
interface.

Tested that this works on the ExoPlayer demo with setVideoEffects applied, and
using a playlist with SDR->HDR and HDR->SDR items.

PiperOrigin-RevId: 599823412
2024-01-19 07:08:23 -08:00
ibaker
34a08e13fc Rollback of 406c0a15be
PiperOrigin-RevId: 599546140
2024-01-18 10:21:53 -08:00
ibaker
406c0a15be CompositeSequenceableLoader: Prefer buffered position of A/V tracks
Preferring audio and video tracks matches logic we already have in
[`ProgressiveMediaPeriod.getBufferedPositionUs`](f6fe90f30b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriod.java (L403)).

PiperOrigin-RevId: 599154217
2024-01-17 06:33:14 -08:00
ibaker
80bfa819c0 Add method to TextRenderer to control whether decoding is done or not
When we default to 'parse during extraction', we will flip the default
of this, to ensure that apps know they are using an
incompatible/deprecated flow for subtitle handling.

PiperOrigin-RevId: 599109304
2024-01-17 02:51:48 -08:00
lpribanic
e93188fe7f Crop and pass thumbnails to ImageOutput
Image grids are now cropped into tiles. The tiles are provided to
ImageOutput at their correct timestamps.

PiperOrigin-RevId: 597553029
2024-01-11 07:45:50 -08:00
tianyifeng
733301d562 Remove the parameter PlayerId from PreloadMediaSource.Factory
Originally a `PlayerId` has to be passed and it should be the same id of the player who is going to play the sources, but it turns out to be unnecessary.

When preloading, we can set a `PlayerId.UNSET` inside of the `PreloadMediaSource`, as there is no ongoing playback. And when the source is handed over to player, player will set the player's `PlayerId`.

PiperOrigin-RevId: 597475119
2024-01-11 00:48:58 -08:00
ibaker
77f311917f Play clear samples in DRM content without keys by default
This behavior was previously available as opt-in via
`MediaItem.DrmConfiguration.Builder.setPlayClearContentWithoutKey` and
`DefaultDrmSessionManager.Builder.setPlayClearSamplesWithoutKeys`. This
change flips the default of both these properties to true.

This should speed up the time for playback to start when playing DRM
content with a 'clear lead' of unencrypted samples at the start.
Previously playback would wait until the keys for the later encrypted
samples were ready. The new behaviour could result in mid-playback
stalls/rebuffers if the keys are not ready yet by the transition from
clear to encrypted samples, but this is not really a regression since
previously playback wouldn't have started at all at this point.

PiperOrigin-RevId: 595992727
2024-01-05 07:46:25 -08:00
ibaker
62f6c64a91 Rename test.mp3 test asset to test-cbr-info-header.mp3
This file is CBR encoded with LAME, so it has an `Info` header (the CBR
equivalent to `Xing`).

A follow-up change will use this file in `Mp3ExtractorTest`.

Issue: androidx/media#878
PiperOrigin-RevId: 595938327
2024-01-05 02:31:20 -08:00
ibaker
a496bbd777 Add parentheses to fix UnexpectedLoaderException message logic
These were missing in 5211ff0dc1

Without these, the `": null"` is still logged (but the `"Unexpected
IllegalStateException"` bit is not), because the **whole** string
concatenation is compared to null as the boolean condition of the
ternary, and this condition is always false (the result of a string
concatentation is never `null`) i.e. (with excess parentheses for
clarity):

```
(("Unexpected " + cause.getClass().getSimpleName() + cause.getMessage()) != null)
    ? ": " + cause.getMessage()
    : ""
```

Also add a test because obviously this isn't as simple as I'd thought.

PiperOrigin-RevId: 593079975
2023-12-22 03:42:42 -08:00
bachinger
36b7a49d52 Use factory to create media period holders in the queue
This is a refactoring that allows the `MediaPeriodQueue` to
create media period holders without all the collaborators
being passes in to `enqueueNextMediaPeriodHolder(...)`.

The factory is presumably also helpful in unit test to
know whether and when exactly a holder is created in
the preloading process.

PiperOrigin-RevId: 592301400
2023-12-19 11:55:13 -08:00
tonihei
b7c3d83e51 Forward setParameters to AsynchronousMediaCodecBufferEnqueuer
The parameters may change the decoding behavior of the
following samples and it's suprising/wrong to apply them
from another thread where we can't guarantee from which
sample they apply.

PiperOrigin-RevId: 591808760
2023-12-18 00:37:21 -08:00
tonihei
f47c4e33ec Remove experimental synchronization on async buffer queuing
PiperOrigin-RevId: 591273508
2023-12-15 09:14:52 -08:00
bachinger
29d7dd1ec9 Keep selections for preloaded adaptive sample streams
When an adaptive source has been preloaded, the track selection
of the player should possibly not override the preloaded selection
to avoid discarding preloaded data.

PiperOrigin-RevId: 591256334
2023-12-15 08:09:22 -08:00
christosts
7df3e9e779 Integrate ImageRenderer with composition preview
PiperOrigin-RevId: 591218013
2023-12-15 05:01:50 -08:00
Copybara-Service
ab296ef686 Merge pull request #753 from stevemayhew:p-fix-issue-9347
PiperOrigin-RevId: 590862514
2023-12-14 02:05:35 -08:00
lpribanic
d52772dff8 Set image tile counts to what the manifest says
Vertical and horizontal tile counts inside Format used by ImageRenderer
used to be required to equal 1 and were set to 1 regardless of what
the manifest said.

This change removes the above requirement and sets the tile counts to
the values from the manifest.

PiperOrigin-RevId: 590608353
2023-12-13 08:37:06 -08:00
tonihei
7a702870f6 Add test and release notes 2023-12-13 14:48:35 +00:00
tonihei
00c7a9bcbb Keep playback thread blocked until next action with playUntilPosition
We currently pause playback to prevent further progress while the
app thread runs assertion and triggers additional actions. This is
not ideal because we do not actually want to pause playback in
almost all cases where this method used.

This can be improved by keeping the playback thread blocked and only
unblock it the next time the app thread waits for the player (either
via RobolectricUtil methods or by blocking the thread itself). To
add this automatic handling, this change introduces a new util class
for the tests that can keep the list of waiting threads statically
(because the access to this logic is spread across multiple independent
classes).

PiperOrigin-RevId: 589784204
2023-12-11 04:45:31 -08:00
tianyifeng
b2a673d521 Move preload components to androidx.media3.exoplayer.source.preload
PiperOrigin-RevId: 589455479
2023-12-09 13:41:55 -08:00
christosts
faeff17a4c Fix bug in CompositingVideoSinkProvider.isInitialized()
Fix a bug where CompositingVideoSinkProvider.isInitialized() returns
true even after releasing the CompositingVideoSinkProvider.

PiperOrigin-RevId: 588481744
2023-12-06 11:01:45 -08:00
christosts
073ee8a1d0 Move instantiation of CompositingVideoSinkProvider
This change moves the instantiation of the CompositingVideoSinkProvider
out of MediaCodecVideoRenderer so that the composition preview player can
re-use the CompositingVideoSinkProvider instance for driving the rendering of
images.

The main point of the change is the ownership of the
VideoFrameReleaseControl, which decides when a frame should be rendered
and so far was owned by the MediaCodecVideoRenderer. With this change,
in the context of composition preview, the VideoFrameReleaseControl
is no longer owned by MediaCodecVideoRenderer, but provided to it.

This way, the CompositingVideoSinkProvider instance, hence the
VideoFrameReleaseControl can be re-used to funnel images into the
video pipeline and render the pipeline from elsewhere (and not
MediaCodecVideoRenderer).

PiperOrigin-RevId: 588459007
2023-12-06 09:52:43 -08:00
ibaker
6360082b87 Add extractor and playback tests for Pixel JPEG motion photo
This image has two video tracks in the MP4 data, one is a 'real' video
which we want to play by default, and the other is a low-fps video track
which isn't intended to be directly played, it's encoded in HEVC for
compression and decoding efficiency.

This test demonstrates ExoPlayer's current behaviour default extraction
and playback, which results in selecting the high-res, low-fps track
(actually single sample in this example), instead of playing the actual
video.

PiperOrigin-RevId: 588068908
2023-12-05 07:46:19 -08:00
tianyifeng
f32bdf86bc Set playerId when preloading to make the PreloadMediaSource reusable
We set the `playerId` only in the constructor currently. But in the case of this source doesn't have `preload` called, the player's release of this source will set the playerId to `null`, which makes this source un-reusable with a null `playerId`.

PiperOrigin-RevId: 587698214
2023-12-04 06:31:11 -08:00
jbibik
03564fdbc6 Promote method to control subtitle parsing to MediaSource.Factory
PiperOrigin-RevId: 586361222
2023-11-29 09:12:04 -08:00
christosts
2a74cb7ef2 Move VideoFrameProcessor reflective loading code
Move the reflective loading of VideoFrameProcessor from
MediaCodecVideoRenderer to the CompositingVideoSinkProvider. This is so
that all reflective code lives in one place. The
CompositingVideoSinkProvider already has reflective code to load the
default PreviewingVideoGraph.Factory.

PiperOrigin-RevId: 584852547
2023-11-23 03:54:18 -08:00
christosts
6435ddb89e Create VideoFrameRenderControl
Split CompositingVideoSinkProvider.VideoSinkImpl in two classes:
- VideoSinkImpl now only receives input from MediaCodecVideoRenderer and
  forwards frames to its connected VideoFrameProcessor
- VideoFrameRenderControl takes composited frames out of the VideoGraph
  and schedules the rendering of those.
- CompositingVideoSinkProvider connects VideoSinkImpl with
  VideoFramesRenderer.

PiperOrigin-RevId: 584605078
2023-11-22 06:05:04 -08:00
tonihei
af0282b9db Avoid clipping live offset override to min/max offsets
The live offset override is used to replace the media-defined
live offset after user seeks to ensure the live adjustment adjusts
to the new user-provided live offset and doesn't go back to the
original one.

However, the code currently clips the override to the min/max
live offsets defined in LiveConfiguration. This is useful to
clip the default value (in case of inconsistent values in the media),
but the clipping shouldn't be applied to user overrides as
the player will then adjust the position back to the min/max
and doesn't stay at the desired user position.

See 2416d99857 (r132871601)

#minor-release

PiperOrigin-RevId: 584311004
2023-11-21 06:49:19 -08:00
tianyifeng
89bedf0fb5 Emit onPositionDiscontinuity event when silence is skipped
Issue: androidx/media#765
PiperOrigin-RevId: 584024654
2023-11-20 07:50:13 -08:00
christosts
7dc0654c69 Rollback of eafe2e35f0
PiperOrigin-RevId: 583045154
2023-11-16 08:01:24 -08:00
tonihei
03a23bef8f Use full MediaItem to define ad playbacks
Currently, ads are only defined by a single URL, which makes it
impossible to define additional fields needed to play ads correctly.

This can be fixed by using a full MediaItem in AdPlaybackState,
replacing the previous Uri field.

PiperOrigin-RevId: 582331588
2023-11-14 08:35:14 -08:00
lpribanic
877d26abdc Add ImageRenderer in CapturingRenderersFactory by default
DefaultRenderersFactory now instantiates an ImageRenderer by default,
so CapturingRenderersFactory will too.

PiperOrigin-RevId: 582307243
2023-11-14 06:58:48 -08:00
tonihei
312203d38b Deprecate Bundleable and Bundleable.Creator
Both interfaces are not really needed as the methods can be called
on the respective classes directly. This also avoid error-prone
situations where classes define to/fromBundle methods with parameters
that should be used instead of the parameter-less version.

Also deprecate all existing CREATOR static instances and make the
corresponding fromBundle method public where needed.

PiperOrigin-RevId: 579189766
2023-11-03 08:27:21 -07:00
lpribanic
7387284c1c Add image track selection to DefaultTrackSelector
DefaultTrackSelector now has all logic necessary for selecting an
image track.

If isPrioritizeImageOverVideoEnabled is set to true, image track will
try to be selected first and a video track will only be selected if no
image track is available. If isPrioritizeImageOverVideoEnabled is set
to false, image track will be selected only if video track wasn't
selected.

PiperOrigin-RevId: 578806006
2023-11-02 04:25:05 -07:00
tonihei
d4e5ab2c4d Make BundleableUtil more generic by taking Function parameters
BundleableUtil is the only class that really depends on the type
inheritence from Bundleable. However, it only needs this as a way
to define Function<T, Bundle> and Function<Bundle, T>, which could
just be passed in as parameters as it's already done for some of
these methods.

Also rename the class to BundleCollectionUtil accordingly.

PiperOrigin-RevId: 578791946
2023-11-02 03:17:17 -07:00
Googler
cf3733765c Rollback of eafe2e35f0
PiperOrigin-RevId: 574755143
2023-10-19 00:21:43 -07:00
michaelkatz
00193e0304 Send decode-only Opus samples in bypass mode for seekPreRoll skip
As Opus decoders skip some bytes prior to playback during a seek, the renderer for bypass playback should send samples to the decoder even if they would be decode-only.

#minor-release

PiperOrigin-RevId: 574494666
2023-10-18 09:11:27 -07:00
tianyifeng
43e6882fb4 Add PreloadMediaSource and PreloadMediaPeriod
The `PreloadMediaSource` has below two new public methods that suppose to be called by the app:

* `preload(long)` allows the apps to preload the source at the passed start position before playback. The preload efforts include preparing the source for a `Timeline`, creating and caching a `MediaPeriod`, preparing the period, selecting tracks on the period and continuing loading the data on the period.
* `releasePreloadMediaSource()` allows the apps to release the preloaded progress.

The `PreloadMediaPeriod` is designed to facilitate the `PreloadMediaSource` for the preloading work. It has a new package-private method `selectTracksForPreloading` which will cache the `SampleStream` that corresponds to the track selection made during the preloading, and when the `PreloadMediaPeriod.selectTracks` is called for playback, it will uses the preloaded streams if the new selection is equal to the selection made during the preloading.

Also add a shortform demo module to demo the usage of `PreloadMediaSource` with the short-form content use case.

PiperOrigin-RevId: 574439529
2023-10-18 04:57:06 -07:00
christosts
eafe2e35f0 Move video frame release logic to VideoFrameReleaseControl
This change moves the release timestamp adjustment logic out of
MediaCodecVideoRenderer and into a standalone component, the
VideoFrameReleaseControl. The plan for VideoFrameReleaseControl is to use
it:
- from MediaCodecVideoRenderer, when ExoPlayer plays video in standalone
  mode.
- from the CompositionPlayer's DefaultVideoSink, when CompositionPlayer
  supports multiple sequences.
- (temporarily) from the CompositionPlayer's custom ImageRenderer while
  we are implementing single-sequence preview, which is an intermediate
  milestone for composition preview.
PiperOrigin-RevId: 574420427
2023-10-18 03:17:03 -07:00
microkatz
716240776e Updated golden file tests and readded constructor 2023-10-17 11:42:33 +00:00