Plumbing hdrMode through the default asset loader factory via the constructor is problematic because it breaks API boundaries. It means there is another way to set hdrMode outside of Composition.java and TransformationRequest.java, which is error prone and cause problems if someone an app starts customizing the assetloaderfactory. It also means custom asset loaders can't receive this information without hacking around.
The introduction of the composition-level settings class makes this approach easily extensible for other settings applied on the composition level but use in an individual asset level basis (e.g. ultraHDR support).
PiperOrigin-RevId: 611466920
Add abilitiy to use real surfaces in instrumentation tests
using the ActivityScenarioRule and an activity class for testing
purposes.
PiperOrigin-RevId: 611421490
When the controller replaces the current item, the masking position will be changed to the default position of the new item for a short while, before the correct position comes from the session. This will interrupt the current position fetched from the controller when the playback doesn't interrupted by the item replacing.
Issue: androidx/media#951
#minor-release
PiperOrigin-RevId: 611417539
This implementation generates lint errors because neither the `first`
nor `second` parameters are used, and that's generally
unexpected/incorrect for a `Comparator` implementation since it should
always consider both its parameters.
PiperOrigin-RevId: 611039632
This change aims to prioritise tracks that have a 'smooth enough for
video' frame rate, without always selecting the track with the highest
frame rate.
In particular MP4 files extracted from motion photos sometimes have two
HEVC tracks, with the higher-res one having a very low frame rate (not
intended for use in video playback). Before this change
`DefaultTrackSelector` would pick the low-fps, high-res track.
This change adds a somewhat arbitrary 10fps threshold for "smooth video
playback", meaning any tracks above this threshold are selected in
preference to tracks below it. Within the tracks above the threshold
other attributes are used to select the preferred track. We deliberately
don't pick the highest-fps track (over pixel count and bitrate), because
most users would prefer to see a 30fps 4k track over a 60fps 720p track.
This change also includes a test MP4 file, extracted from the existing
`jpeg/pixel-motion-photo-2-hevc-tracks.jpg` file by logging
`mp4StartPosition` in
[`MotionPhotoDescription.getMotionPhotoMetadata`](b930b40a16/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/MotionPhotoDescription.java (L123))
and then using `dd`:
```
mp4StartPosition=2603594
$ dd if=jpeg/pixel-motion-photo-2-hevc-tracks.jpg \
of=mp4/pixel-motion-photo-2-hevc-tracks.mp4 \
bs=1 \
skip=2603594
```
----
This solution is in addition to the `JpegMotionPhotoExtractor` change
made specifically for these two-track motion photos in
5266c71b3a.
We will keep both changes, even though that change is not strictly
needed after this one, because adding the role flags helps to
communicate more clearly the intended usage of these tracks. This
change to consider FPS seems like a generally useful improvement to
`DefaultTrackSelector`, since it seems unlikely we would prefer a 5fps
video track over a 30fps one.
Issue: androidx/media#1051
PiperOrigin-RevId: 611015459
This new test is for `ExoPlayer.setVideoEffects()`. It plays the
one-second-long video, applies an overlay that prints the video frame timestamp
onto the frame, captures the output frame and compares the captured output
frame with golden.
PiperOrigin-RevId: 610781590
These audio offload failure recovery tests model the DefaultAudioSink failing at audio track init and write operations in offload mode. Playback should recover and try again as DefaultAudioSink will disable offload mode.
PiperOrigin-RevId: 610372935
Some devices supporting Performance Points for decoder coverage are missing coverage over the CDD requirements for H264. For these cases ExoPlayer should fall back to legacy resolution and frame rate support checks. If there is a stream evaluated as a PerformancePointCoverageResult of COVERAGE_RESULT_NO, then ExoPlayer checks for coverage of the 720p H264 CDD requirement.
Issue: google/ExoPlayer#10898
Issue: androidx/media#693
Issue: androidx/media#966
PiperOrigin-RevId: 609740128
Even when there's no display surface, MCVR can render frames to VFP, becuase by
the time the frame is processed:
- If there's still no surface, VFP will drop the frame;
- If there's surface, the processed frame would be rendered.
In short, placeholder surface is not needed in effect enabled playback. FWIW,
it is used to swallow frames directly from MediaCodec when there's no output
surface.
PiperOrigin-RevId: 609705222
This change makes ExoPlayerImplInternal.releaseInternal() unblock the
app thread if a runtime exception is thrown while releasing components
from the playback thread.
Before this change, if a runtime exception occurred during releasing
components in the playback thread, ExoPlayer.release() would wait for
`releaseTimeoutMs` and then raise a player error. With this change,
the player error is reported only when the playback thread is blocked
but if there is a runtime exception, the application thread is
unblocked.
The impact of this change is potentially fewer ANRs on
ExoPlayer.release() at the expense of less error reporting.
PiperOrigin-RevId: 609702549
We forgot to add it when we added AudioSink.release(). The commit
includes a test that ensures ForwardingAudioSink overrides all the
methods defined in the AudioSink interface.
PiperOrigin-RevId: 609402258
If the reading period has already advanced and a track reselection procs that only affects the reading period media, then ExoPlayer may try and apply the reading period's track selection incorrectly unto the playing period. ExoPlayer should apply the playing period's track selection to the playing period instead.
PiperOrigin-RevId: 609375077
If render error occurs due to AudioTrack initialization failure in offload mode, then ExoPlayer should allow retry as subsequent attempt will be with DefaultAudioSink disabling offload.
PiperOrigin-RevId: 609304897
The `PreloadMediaPeriod.selectTracksForPreloading` can be called for multiple times at the preloading stage (before the period is being played). For example, when the period resumes preloading. This change fix the assertion failure in `ProgressiveMediaPeriod.selectTracks` caused by the wrong implementation of `PreloadMediaPeriod.selectTracksForPreloading` when it is trying to retain the previously preloaded streams.
Also the `TrackSelectorResult` parameter is changed to a list of `ExoTrackSelection`. We should compare the selections only rather than considering the `RendererConfiguration` in the `TrackSelectorResult` to decide whether to retain the streams, as for preloading case the renderers haven't consumed the samples yet.
PiperOrigin-RevId: 609126868
Previously, the track format was used in VideoSampleExporter. Now, we use a
simulated decoder output format.
As the last expected change for this bug, also adds release notes
PiperOrigin-RevId: 609080629
Also introduce a fluent API that allows callers to ignore non-fatal
errors (while avoiding adding boolean overloads for every method).
**Most** tests want to fail on non-fatal errors (since they likely
indicate user-visible issues like codec errors etc), only tests
explicitly testing fallback in error scenarios should want to ignore
them.
Before this change there were a few `playUntilXXX` methods. These can
now all be triggered via `play(player).untilXXX`, which means
effectively every 'until' condition is available in a 'play until'
variant that calls `play` just before waiting for the condition.
PiperOrigin-RevId: 608988234
MediaCodec docs already allude to potentially mismatching H.264 level
between container and bitstream. Relax the initialization data check to
reflect this.
PiperOrigin-RevId: 608942322
When applying edit lists, we need to output the last partial samples
to have all the necessary data needed for rendering.
The only case where we can omit the sample is for zero duration
audio data that has no additional information.
The current comment and variable name doesn't make this very clear
and this change improves the naming and the comment.
PiperOrigin-RevId: 608579746
There is no super test which covers whole MP4 structure.
In all the E2E test, it verified against `ExtractorOutput` which
would remain same if there are changes in the box structure.
PiperOrigin-RevId: 608310006
These were previously somewhat supported, but the `package` part was
never read (so it only worked if it was either absent or the same as the
current application's package). This change parses and uses the
`package` part.
This partially reverts 35121a2d3d.
This is hard to test for the same reasons as 88f554c74b:
> This is hard to test because to do so robustly requires being able to guaranteed that another test APK will be installed with a known raw resource inside it.
PiperOrigin-RevId: 608270463
This change:
1. Updates the implementation of
`FrameworkMediaDrm.requiresSecureDecoder` to include the
'force allow insecure decoder' workaround logic.
2. Removes the 'force allow insecure decoder' logic from MCR
3. Removes the `requiresSecureDecoder` field from MCR (it can just
be a local derived from `codecDrmSession` now).
PiperOrigin-RevId: 607664186
The behaviour and docs of `playUntilPosition` were changed in
00c7a9bcbb.
This change also affects `playUntilStartOfMediaItem` (since it delegates
to `playUntilPosition`), so the same doc change should also be made
here.
#minor-release
PiperOrigin-RevId: 607364897
Also put back a comment in DownloadTracker that is still relevant on API 19.
Also deprecate PlaceholderSurface.newInstanceV17() in favour of just
newInstance()
VideoFrameProcessor requires every input frame to be registered before they are
rendered to its input Surface. This CL adds the option to register the input
frame only once. This is useful for camera inputs where not all input frames
come with changing FrameInfo.
PiperOrigin-RevId: 606294894
Our previous test video was difficult to use for testing our tone-mapping
algorithm, because it didn't have many different colors. Use a better
video for tone-map tests, by having one with more different colors
PiperOrigin-RevId: 606274843
These allow to set the icon in a standardized way without needing
custom bitmaps or resources. For now, this is just additional
information without backwards-compatible icons or implications.
The same value gets written to the platform session via a new
extras key that can be read and set from sessions not using Media3
yet.
PiperOrigin-RevId: 605670988
The public APIs of these modules reference symbols in some of their
dependencies, so these should be API dependencies, not implementation:
> An API dependency is one that contains at least one type that is
> exposed in the library binary interface, often referred to as its ABI
> (Application Binary Interface).
https://docs.gradle.org/current/userguide/java_library_plugin.html#sec:java_library_recognizing_dependencies
Transformer also uses symbols from `lib-common`, but these are already
an API dep of `lib-exoplayer` so no need to duplicate that here.
PiperOrigin-RevId: 605660621
TestUtil class is more appropriate for the given method.
In the next CL, the only method in FileUtil.java will be moved back
into transformer library and the FileUtil class will be removed.
PiperOrigin-RevId: 605648034
The new API will take both `creation time` and `modification time`.
Till now, Mp4Muxer wrote `modification time` in both
`creation time` and `modification time` field, which was
incorrect.
PiperOrigin-RevId: 605590623
Previously, we missed the BT2020->BT709 color-space conversion.
A user-visible impact of this is that red and green channels used to be
undersaturated, but now are more correctly saturated.
PiperOrigin-RevId: 605411926
Change the file extension retrieval back to how it was before 5488d33da8 to reduce the change of false negatives in `isImage()`
PiperOrigin-RevId: 605281186
`Uri.appendQueryParameter` is documented to encode its arguments, so
calling `Uri.encode` beforehand results in double-encoding.
Issue: androidx/media#1075
#minor-release
PiperOrigin-RevId: 604995441
The track id must be the index in the list of published tracks
as it's used as such elsewhere. This is currently not true if we
skip an empty track as all subsequent tracks get a wrong or even
invalid id.
#minor-release
PiperOrigin-RevId: 604929178
Relax the regex to only check for hyphen which is required by the specification.
Issue: androidx/media#1028
#minor-release
PiperOrigin-RevId: 604719300
Earlier implementation compared the whole file against a golden
data. The new implementation compares only the metadata being tested.
This will avoid updating the golden data when any unrelated change
(unrelated to scenario being tested) is made.
Added a separate test to compare the whole output file against a golden data.
PiperOrigin-RevId: 604692985
The current implementation of `JpegMotionPhotoExtractor.sniff` returns
`true` for any image with Exif data (not just motion photos). Improving
this is tracked by b/324033919. In the meantime, when we 'extract' a
non-motion photo with `JpegMotionPhotoExtractor`, the result is
currently a single empty image track and no video track (since there's
no video, since this isn't a motion photo). This 'empty' image track
is usually used to transmit metadata about the video parts of the
image file (in the form of `MotionPhotoMetadata`), but this metadata is
also (understandably) absent for non-motion photos. Therefore there's
no need to emit this image track at all, and it's clearer to emit no
tracks at all when extracting a non-motion photo using
`JpegMotionPhotoExtractor`.
This change also removes a `TODO` that is misplaced, since there's no
image bytes being emitted here (and never was).
PiperOrigin-RevId: 604688053
These are often the same for image tracks, since we usually drop the
whole image file (both the container and actual encoded image bytes)
into a single sample, but there are cases where we emit a track with
`containerMimeType=image/jpeg` but **no** samples (from
`JpegMotionPhotoExtractor`, to carry some metadata about the image +
video byte offsets).
It's therefore more correct to implement the `supportsFormat` check
based on `sampleMimeType`, so that these 'empty' image tracks are not
considered 'supported' by `ImageRenderer`.
#minor-release
PiperOrigin-RevId: 604672331
This was already done for the TrackGroup ids in <unknown commit>,
but in some scenarios only the Format instances are known and
it's helpful to be able to identify where they came from.
Issue: androidx/media#883
#minor-release
PiperOrigin-RevId: 604644039
This ensures RequestMetadata with just non-null extras is not
considered equal to RequestMetadata.EMPTY. This makes sure the
contents are bundled when a controller sets the extras in a
new MediaItem.
PiperOrigin-RevId: 604632788
Add a test for this consistency in `CompositeSequenceableLoaderTest`,
and also make the
`CompositeSequenceableLoaderTest.FakeSequenceableLoader` implementation
more realistic.
#minor-release
PiperOrigin-RevId: 604604103
This reverses 27caeb8038
Due to the re-ordering of packets done in `CeaDecoder`, there's no way
to use the current implementation to correctly parse these subtitle
formats during extraction (the `SubtitleParser` interface), so we have
to keep the `SubtitleDecoder` implementations.
#minor-release
PiperOrigin-RevId: 604594837
This reverses 94e45eb4ad
Due to the re-ordering of packets done in `CeaDecoder`, there's no way
to use the current implementation to correctly parse these subtitle
formats during extraction (the `SubtitleParser` interface), so we have
to keep the `SubtitleDecoder` implementations.
#minor-release
PiperOrigin-RevId: 604350951
Before supporting transmuxing when both no op effects and regular rotations are set, move setting the muxerWrapper rotation out of shouldTranscodeVideo() to ensure the muxerWrapper rotation is only set at the appropriate times.
This cl also ensures the state between the muxerWrapper and the list of video effects is consistent by clearing the list of videoEffects in trim optimization. If trim optimisation is being applied, then EditedMediItem.effects.videoEffects only contains no-op effects or regular rotations that get be applied in the muxer wrapper. Therefore, we should clear the list of video effects to ensure that no effect gets applied twice.
PiperOrigin-RevId: 604292052
removing the useHdr parameter from queueInputBitmap() it suggests we support changing between HDR and SDR within a stream, which we don't support. instead, identifying whether to use HDR from the shaderprogram which is informed by the inputColorInfo when the stream is registered.
PiperOrigin-RevId: 603681736
Transformer export and ExoPlayer previewing both read inputColorInfo from
registerInputStream now, instead of maintaining a consistent input color
throughout multiple streams in a sequence.
Therefore, we can remove inputColor-related arguments and methods now.
PiperOrigin-RevId: 603423509
If nullness should result in an exception, we should throw as soon as possible,
so that stack traces are easier to follow.
Did a quick scan of media3.effect+transformer and this covers all uses of
peek that immediately throw there.
PiperOrigin-RevId: 603393366
The new test introduced in 45bd5c6f0a is flaky because we only
wait until the media is fully buffered. However, we can't fully
control how much of this data is initially read by the Robolectric
codec and thus the output dump files (containing these codec
interactions) are flaky.
This can be fixed by fully playing the media once and then seeking
back instead.
#minor-release
PiperOrigin-RevId: 603324068
If seeking between last image sample and end of the file where the current stream is not final, then EoS sample will not be provided to `ImageRenderer`. ImageRenderer must still produce the last image sample.
PiperOrigin-RevId: 603312090
The implementation of fragmented MP4 caused a regression where muxer
started writing empty tracks even for non fragmented MP4.
PiperOrigin-RevId: 603091348
VideoFrameProcessor treats BT601 and BT709 as roughly equivalent now, so we
shouldn't be making checks that assume BT709 <-> requires tone-mapping.
Also, the color transfer is a better determinant for tone-mapping than color range, so use just the transfer to determine if tone-mapping is required.
PiperOrigin-RevId: 603083100
The `MediaMuxer` (FrameworkMuxer) supports AV1 mime type from API 34.
For this to work track `Format/MediaFormat` must have `CSD`
data.
Change also include providing AV1 option in demo app.
Transmuxing of an AV1 mime type input will not work because
`Extractor` does not extract AV1 CSD data.
Verified changes manually via demo app.
PiperOrigin-RevId: 603002380
- Added back parsing of scheme data for version 1 as it's technically
allowed by the spec.
- Made constructor of PsshAtom private to only publish the data class
and not the constructor.
- Formatting and Javadoc adjustments
- Additional tests
FileUtil.java can be extended to add more methods for validating
output file. This will be used in Transformer's robolectric tests and
instrumentation tests hence need to move to a common module.
PiperOrigin-RevId: 602789503
Previously, input assets had to be all SDR or all HDR.
After this CL, if tone-mapping is requested, HDR and SDR may mix in any order. If tone-mapping is not requested, SDR may precede HDR, but not vice versa, until SDR to HDR tone-mapping is implemented
Some changes to accomplish this include:
1. Inputting the decoded format's color to VideoFrameProcessor.registerInputStream
for each stream.
2. Calculating the estimated decoded format's color for each stream, by estimating
it based on MediaCodec tone-mapping.
PiperOrigin-RevId: 602747837
