Also put back a comment in DownloadTracker that is still relevant on API 19.
Also deprecate PlaceholderSurface.newInstanceV17() in favour of just
newInstance()
VideoFrameProcessor requires every input frame to be registered before they are
rendered to its input Surface. This CL adds the option to register the input
frame only once. This is useful for camera inputs where not all input frames
come with changing FrameInfo.
PiperOrigin-RevId: 606294894
Our previous test video was difficult to use for testing our tone-mapping
algorithm, because it didn't have many different colors. Use a better
video for tone-map tests, by having one with more different colors
PiperOrigin-RevId: 606274843
These allow to set the icon in a standardized way without needing
custom bitmaps or resources. For now, this is just additional
information without backwards-compatible icons or implications.
The same value gets written to the platform session via a new
extras key that can be read and set from sessions not using Media3
yet.
PiperOrigin-RevId: 605670988
The public APIs of these modules reference symbols in some of their
dependencies, so these should be API dependencies, not implementation:
> An API dependency is one that contains at least one type that is
> exposed in the library binary interface, often referred to as its ABI
> (Application Binary Interface).
https://docs.gradle.org/current/userguide/java_library_plugin.html#sec:java_library_recognizing_dependencies
Transformer also uses symbols from `lib-common`, but these are already
an API dep of `lib-exoplayer` so no need to duplicate that here.
PiperOrigin-RevId: 605660621
TestUtil class is more appropriate for the given method.
In the next CL, the only method in FileUtil.java will be moved back
into transformer library and the FileUtil class will be removed.
PiperOrigin-RevId: 605648034
The new API will take both `creation time` and `modification time`.
Till now, Mp4Muxer wrote `modification time` in both
`creation time` and `modification time` field, which was
incorrect.
PiperOrigin-RevId: 605590623
Previously, we missed the BT2020->BT709 color-space conversion.
A user-visible impact of this is that red and green channels used to be
undersaturated, but now are more correctly saturated.
PiperOrigin-RevId: 605411926
Change the file extension retrieval back to how it was before 5488d33da8 to reduce the change of false negatives in `isImage()`
PiperOrigin-RevId: 605281186
`Uri.appendQueryParameter` is documented to encode its arguments, so
calling `Uri.encode` beforehand results in double-encoding.
Issue: androidx/media#1075
#minor-release
PiperOrigin-RevId: 604995441
The track id must be the index in the list of published tracks
as it's used as such elsewhere. This is currently not true if we
skip an empty track as all subsequent tracks get a wrong or even
invalid id.
#minor-release
PiperOrigin-RevId: 604929178
Relax the regex to only check for hyphen which is required by the specification.
Issue: androidx/media#1028
#minor-release
PiperOrigin-RevId: 604719300
Earlier implementation compared the whole file against a golden
data. The new implementation compares only the metadata being tested.
This will avoid updating the golden data when any unrelated change
(unrelated to scenario being tested) is made.
Added a separate test to compare the whole output file against a golden data.
PiperOrigin-RevId: 604692985
The current implementation of `JpegMotionPhotoExtractor.sniff` returns
`true` for any image with Exif data (not just motion photos). Improving
this is tracked by b/324033919. In the meantime, when we 'extract' a
non-motion photo with `JpegMotionPhotoExtractor`, the result is
currently a single empty image track and no video track (since there's
no video, since this isn't a motion photo). This 'empty' image track
is usually used to transmit metadata about the video parts of the
image file (in the form of `MotionPhotoMetadata`), but this metadata is
also (understandably) absent for non-motion photos. Therefore there's
no need to emit this image track at all, and it's clearer to emit no
tracks at all when extracting a non-motion photo using
`JpegMotionPhotoExtractor`.
This change also removes a `TODO` that is misplaced, since there's no
image bytes being emitted here (and never was).
PiperOrigin-RevId: 604688053
These are often the same for image tracks, since we usually drop the
whole image file (both the container and actual encoded image bytes)
into a single sample, but there are cases where we emit a track with
`containerMimeType=image/jpeg` but **no** samples (from
`JpegMotionPhotoExtractor`, to carry some metadata about the image +
video byte offsets).
It's therefore more correct to implement the `supportsFormat` check
based on `sampleMimeType`, so that these 'empty' image tracks are not
considered 'supported' by `ImageRenderer`.
#minor-release
PiperOrigin-RevId: 604672331
This was already done for the TrackGroup ids in <unknown commit>,
but in some scenarios only the Format instances are known and
it's helpful to be able to identify where they came from.
Issue: androidx/media#883
#minor-release
PiperOrigin-RevId: 604644039
This ensures RequestMetadata with just non-null extras is not
considered equal to RequestMetadata.EMPTY. This makes sure the
contents are bundled when a controller sets the extras in a
new MediaItem.
PiperOrigin-RevId: 604632788
Add a test for this consistency in `CompositeSequenceableLoaderTest`,
and also make the
`CompositeSequenceableLoaderTest.FakeSequenceableLoader` implementation
more realistic.
#minor-release
PiperOrigin-RevId: 604604103
This reverses 27caeb8038
Due to the re-ordering of packets done in `CeaDecoder`, there's no way
to use the current implementation to correctly parse these subtitle
formats during extraction (the `SubtitleParser` interface), so we have
to keep the `SubtitleDecoder` implementations.
#minor-release
PiperOrigin-RevId: 604594837
This reverses 94e45eb4ad
Due to the re-ordering of packets done in `CeaDecoder`, there's no way
to use the current implementation to correctly parse these subtitle
formats during extraction (the `SubtitleParser` interface), so we have
to keep the `SubtitleDecoder` implementations.
#minor-release
PiperOrigin-RevId: 604350951
Before supporting transmuxing when both no op effects and regular rotations are set, move setting the muxerWrapper rotation out of shouldTranscodeVideo() to ensure the muxerWrapper rotation is only set at the appropriate times.
This cl also ensures the state between the muxerWrapper and the list of video effects is consistent by clearing the list of videoEffects in trim optimization. If trim optimisation is being applied, then EditedMediItem.effects.videoEffects only contains no-op effects or regular rotations that get be applied in the muxer wrapper. Therefore, we should clear the list of video effects to ensure that no effect gets applied twice.
PiperOrigin-RevId: 604292052
removing the useHdr parameter from queueInputBitmap() it suggests we support changing between HDR and SDR within a stream, which we don't support. instead, identifying whether to use HDR from the shaderprogram which is informed by the inputColorInfo when the stream is registered.
PiperOrigin-RevId: 603681736
Transformer export and ExoPlayer previewing both read inputColorInfo from
registerInputStream now, instead of maintaining a consistent input color
throughout multiple streams in a sequence.
Therefore, we can remove inputColor-related arguments and methods now.
PiperOrigin-RevId: 603423509
If nullness should result in an exception, we should throw as soon as possible,
so that stack traces are easier to follow.
Did a quick scan of media3.effect+transformer and this covers all uses of
peek that immediately throw there.
PiperOrigin-RevId: 603393366
The new test introduced in 45bd5c6f0a is flaky because we only
wait until the media is fully buffered. However, we can't fully
control how much of this data is initially read by the Robolectric
codec and thus the output dump files (containing these codec
interactions) are flaky.
This can be fixed by fully playing the media once and then seeking
back instead.
#minor-release
PiperOrigin-RevId: 603324068
If seeking between last image sample and end of the file where the current stream is not final, then EoS sample will not be provided to `ImageRenderer`. ImageRenderer must still produce the last image sample.
PiperOrigin-RevId: 603312090
The implementation of fragmented MP4 caused a regression where muxer
started writing empty tracks even for non fragmented MP4.
PiperOrigin-RevId: 603091348
VideoFrameProcessor treats BT601 and BT709 as roughly equivalent now, so we
shouldn't be making checks that assume BT709 <-> requires tone-mapping.
Also, the color transfer is a better determinant for tone-mapping than color range, so use just the transfer to determine if tone-mapping is required.
PiperOrigin-RevId: 603083100
The `MediaMuxer` (FrameworkMuxer) supports AV1 mime type from API 34.
For this to work track `Format/MediaFormat` must have `CSD`
data.
Change also include providing AV1 option in demo app.
Transmuxing of an AV1 mime type input will not work because
`Extractor` does not extract AV1 CSD data.
Verified changes manually via demo app.
PiperOrigin-RevId: 603002380
- Added back parsing of scheme data for version 1 as it's technically
allowed by the spec.
- Made constructor of PsshAtom private to only publish the data class
and not the constructor.
- Formatting and Javadoc adjustments
- Additional tests
FileUtil.java can be extended to add more methods for validating
output file. This will be used in Transformer's robolectric tests and
instrumentation tests hence need to move to a common module.
PiperOrigin-RevId: 602789503
Previously, input assets had to be all SDR or all HDR.
After this CL, if tone-mapping is requested, HDR and SDR may mix in any order. If tone-mapping is not requested, SDR may precede HDR, but not vice versa, until SDR to HDR tone-mapping is implemented
Some changes to accomplish this include:
1. Inputting the decoded format's color to VideoFrameProcessor.registerInputStream
for each stream.
2. Calculating the estimated decoded format's color for each stream, by estimating
it based on MediaCodec tone-mapping.
PiperOrigin-RevId: 602747837
* getPcmFormat Util method already sets the mime type as AUDIO_RAW.
* static final AudioFormat consts improve test case readability.
PiperOrigin-RevId: 602740007
This avoids issues that can arise due to the slight discrepancies between
chunk start times (obtained from the manifest of segment index) and
the timestamps of the samples contained within those chunks.
This happens when using `ExoPlayer.setVideoEffects()`.
This CL also fixes the first frame not rendered problem, originally solved in
7e65cce967, but rolled back in 5056dfaa2b because the solution introduces
the flash that is observed in b/292111083.
Before media3 1.1 release, the output size of `VideoFrameProcessor` is not
reported to the app. This was changed later after introducing
`CompositingVideoSinkProvider`, where the video size after processing **is**
reported to the app. After this CL, the size is again, not reported.
PiperOrigin-RevId: 602345087
The updated algorithm has two main improvements:
- The silence padding is not constant but a ratio of the original
silence (up to a defined max) to more naturally represent the
original gaps.
- The silence is not instantly going to zero, but uses a ramp down
and up for a smooth volume transition and also retains a small
percentage of the original noise for more natural "silences" that
still contain some background noise.
#minor-release
Issue: google/ExoPlayer#7423
PiperOrigin-RevId: 602322442
If trim optimisation is being applied, then `EditedMediItem.effects.videoEffects` only contains no-op effects or regular rotations that get be applied in the muxer wrapper. Therefore, we should clear the list of video effects to ensure that no effect gets applied twice.
This fixes requested rotations being applied twice in trim optimization.
Manually tested to ensure all combinations of trimming+muting+rotating works at intended.
PiperOrigin-RevId: 601853203
The return value of onConfigure must not be ignored as it specifies
the output format of the processor, which may be different from the
input format.
#minor-release
PiperOrigin-RevId: 601799440
Many usages are needed to support other deprecations and some
can be replaced by the recommended direct alternative.
Also replace links to deprecated/redirected dev site
PiperOrigin-RevId: 601795998
since we now can support taking in the inputColor upon registering the stream, there is no need to hardcode the image input color anymore. We should now be able to support switching between texture and image input which we couldn't before, but this is untested and not necessary.
PiperOrigin-RevId: 601784149
This allows us to also plumb whether we're doing MediaCodec tone-mapping,
which will be used in a follow-up CL in the ExoAssetLoaderVideoRenderer.
PiperOrigin-RevId: 601774435
It's likely that we will merge these back into their `XXXDecoder`
implementations, but this smaller change allows us to avoid including
these public symbols in the upcoming release.
#minor-release
PiperOrigin-RevId: 601432629
This method works by reflectively loading classes from the `lib-effect`
module, in order to avoid a hard dependency on this module for ExoPlayer
users that don't want video effect functionality. This change ensures
that a failure to load the necessary classes fails immediately, instead
of on a later thread inside `MediaCodecVideoRenderer` when the
reflection currently happens.
Also update the javadoc to make the dependency requirement clear.
#minor-release
PiperOrigin-RevId: 601387957
DASH: `DashMediaSource.Factory` would only propagate it to `DashChunkSource.Factory` -> `BundledChunkExtractor.Factory`
SS: `SSMediaSource.Factory` -> `SsChunkSource.Factory`
HLS: `HlsMediaSource.Factory` -> `HlsExtractorFactory`
Remove nullability of SubtitleParser.Factory across the stack
#minor-release
PiperOrigin-RevId: 601250013
The `SubtitleParser.Factory` is no longer @Nullable and the experimenting toggle is used to enable/disable the use of this factory for subtitle parsing during extraction.
The three places that will hold the "truth" for the `SubtitleParser.Factory` are: BundledChunkExtractor.Factory, SsChunkSource.Factory, DefaultHlsExtractorFactory
DASH: `DashMediaSource.Factory` would only propagate it to `DashChunkSource.Factory` -> `BundledChunkExtractor.Factory`
SS: `SSMediaSource.Factory` -> `SsChunkSource.Factory`
HLS: `HlsMediaSource.Factory` -> `HlsExtractorFactory`
#minor-release
PiperOrigin-RevId: 601151615
As per MP4 spec ISO 14496-12: 8.7.5 Chunk Offset Box, Both "stco" and
"co64" can be used to store chunk offsets. While "stco" supports 32-bit
offsets, "co64" supports 64-bit offsets.
In non fragmented MP4, the mdat box can be extremely large, hence muxer
uses "co64" box.
But for fragmented MP4, muxer does not write any data in this chunk offset
box (present in "moov" box) because all sample related info is present in
"moof" box.
Technically, "co64" box should also work in fragmented MP4because
its empty only but QuickTime player fails to play video if "co64"
box is present in fragmented MP4 output file.
Testing: Verified that QuickTime player does not play video when "co64"
box is present but is able to play when "stco" box is present.
#minor-release
PiperOrigin-RevId: 601147046
Those classes only needed to have access to a `SubtitleParser.Factory` to get a potentially updated `Format` for TrackGroups. The `SubtitleParser.Factory` was only used to check the support for the `mimeType` and getting some cue-related behaviour.
This introduced complexity in a way that both Periods and Extractors needed to have the same `SubtitleParser.Factory` in their individual stacks. To ensure that the sample queue would get the same transcoded/original format.
Instead, now we expose `getOutputTextFormat` methods on `ChunkExtractor.Factory`, `SsChunkSource.Factory` and `HlsExtractorFactory`. Those are the dependencies that Hls/Ss/DashMediaPeriod can make use of to delegate the format-updating logic to.
#minor-release
PiperOrigin-RevId: 601130714
This fix makes output playable on VLC player.
The output does not play on QuickTime player which is being fixed in
a separate CL.
#minor-release
PiperOrigin-RevId: 601118813
Allow setAdditionalRotationDegrees to be called with same rotation after tracks added. This is needed for processes that mux files partially trim optimization so they don't error out after hitting the check state
Manually tested to ensure trim optimization succeeds, automated test added here as well
PiperOrigin-RevId: 601081778
These symbols in `lib-effect` are referenced via reflection from
`CompositingVideoSinkProvider` in `lib-exoplayer` in order to avoid
a hard dependency from `lib-exoplayer` to `lib-effect`. Without this
keep rule, the symbols can get renamed by R8 resulting in the
invocations failing.
#minor-release
PiperOrigin-RevId: 601074636
Populate both `artworkUri` and `artworkData` in
`MediaMetadata.Builder.populate(MediaMetadata)` when at least one of them is non-null.
Issue: androidx/media#964
PiperOrigin-RevId: 600826103
Earlier implementation processed each track (pending sample's buffer info)
individually when writing their corresponding "traf" box in a fragment.
The change involves processing all tracks before start writing "traf" boxes.
#minor-release
PiperOrigin-RevId: 600811093
The default notification provider was still using the legacy
compat MediaStyle instead of our own Media3 one. They are fully
equivalent in their implementation and API and can be swapped out
easily.
PiperOrigin-RevId: 600797920
We keep the previous parsing-during-rendering tests, even though they
can be a bit flaky, because this is an important regression test. The
regression risk is lower for this instrumentation test compared to
robolectric tests with different `ShadowLooper` behaviour.
#minor-release
PiperOrigin-RevId: 600781035
Due to poor isolation between the session tests, in particular the
static state in `MediaSession.SESSION_ID_TO_SESSION_MAP`, an unreleased
session at the end of one test can cause subsequent tests to fail with
obscure errors like `Session ID must be unique`.
#minor-release
PiperOrigin-RevId: 600737697
We can just continue to assume that we don't know the current device.
This case happens on the latest Robolectric release where this method
call isn't implemented yet. As we not generally assume that the
method can throw, this workaround can be removed once Robolectric
is updated again.
#minor-release
PiperOrigin-RevId: 600426851
Add protected method in SimpleBasePlayer for thread verification to help
subclasses verify thread access for additional methods they define and
still report the same message to the user.
Also, remove the DAC link pointing to the ExoPlayer-specific
documentation from the exception message. Users who extend
SimpleBasePlayer have access to the class' javadoc.
PiperOrigin-RevId: 600426692
[Android best
practices](https://developer.android.com/media/optimize/sharing#android_8_81_and_9)
recommend disabling B-frames on API 27, but some devices output B-frames anyway
when H.264/AVC High profile is selected. Add a workaround to force these
devices not to use high profile, to prevent B-frames being output.
`MediaMuxer` likely handles B-frames on these specific devices, but this change
allows the migration to default to in-app muxing to take place without
introducing errors, and it's a temporary workaround until B-frames are properly
supported in the in-app muxer.
PiperOrigin-RevId: 600422238
Some player method calls sent from MediaControllers accept int
or float values with a very clear API contract that disallows
some values. Filtering by these values early avoids calling a
Player implementation with invalid values.
PiperOrigin-RevId: 600413993
This optimization always reports buffers as 'skipped' (i.e. deliberately
not shown), which makes sense for the target case of high FPS content on
a lower refresh rate screen, when lots of the frames will **never** be
shown.
However the optimization also results in reporting buffers as 'skipped'
when decoding is a bit slow, resulting in a frame being released one
vsync late, which then means we have two frames to release in the same
vsync (when the previous vsync was empty). In this case, it would be
more correct to report this as a 'dropped' frame (since it was due to
slow decoding).
Until we can change the logic to distinguish these cases and report them
separately, this CL disables the optimization completely in GTS tests.
This is needed because we often assert there were zero skipped frames,
so slight decoding slowness can cause spurious/flaky test failures (our
threshold for dropped frames is non-zero).
#minor-release
PiperOrigin-RevId: 600406443
This device doesn't seem to be capable of simultaneous encode/decode at this
resolution. We don't have a good way to check the capability (we are already
checking separate decode/encode capability) so just skip this test to save time
triaging its failures.
PiperOrigin-RevId: 600399564
From API 23, we may have a preferred device that is most likely used
as the output device.
From API 24, the AudioTrack tells us the actual routed device that is
used for output and we can listen to changes happening mid-playback.
From API 33, we can directly query the default device that will
be used for audio output for the current attributes.
If the routed device is known by any of the methods above, we can add
more targeted checks in methods like isBluetoothConnected to avoid
iterating over all devices that are not relevant.
The knowledge about the routed device will also be useful to check
advanced output capabilities in the future (e.g. for lossless
playback)
PiperOrigin-RevId: 600384923
This allows us to inject a videoFrameProcessorFactory into
MediaCodecVideoRenderer, without issues about creating the
VideoFrameReleaseControl in the MediaCodecVideoRenderer.
Unfortunately, this does result in more complex CVSP state, where
VideoFrameReleaseControl is no longer final, may be null, and may potentially
change. However, this tries to be careful with assertions to guarantee good
state, and is cleaner than modifying the long-standing MediaCodecVideoRenderer
interface.
Tested that this works on the ExoPlayer demo with setVideoEffects applied, and
using a playlist with SDR->HDR and HDR->SDR items.
PiperOrigin-RevId: 599823412
Although not public, documenting what happens in each state allows for
better code understanding at a glance.
As part of this, refactored the #getProgress method to highlight that
the other options are "non standard".
PiperOrigin-RevId: 599505369
This also fixes issue introduced by frames being released from a prior version of a
GlShaderProgram
Tested by seeking within a playlist with one SDR then one HDR video.
PiperOrigin-RevId: 599475959
Previously, 8f69bb0d9d updated external input (video input)
but not internal input (image/texture input). Update internal input as
well to match.
PiperOrigin-RevId: 599235813
Since mdat box can be huge so there is a provision to use 64 bit size field.
In case of fragmented MP4, individual fragments should not have large mdat box
so a 32 bit size field should be sufficient.
PiperOrigin-RevId: 599219041
When we default to 'parse during extraction', we will flip the default
of this, to ensure that apps know they are using an
incompatible/deprecated flow for subtitle handling.
PiperOrigin-RevId: 599109304
Imported from GitHub PR https://github.com/androidx/media/pull/275
Added below mentioned features.
- Support for extracting DTS LBR(DTS Express) and DTS UHD Profile 2(DTS:X) descriptor ID from PSI PMT
- The DTSReader class is updated for extracting a DTS LBR.
- Newly added DtsUhdReader class for extracting DTS UHD frame.
- The DTSUtil class is updated to parse the DTS LBR or DTS UHD frame and report the format information.
Feature request for ExoPlayer: https://github.com/google/ExoPlayer/issues/11075
Merge 21efa0810db31550d6b215639f9ca2af6a32139a into 104cfc322c
COPYBARA_INTEGRATE_REVIEW=https://github.com/androidx/media/pull/275 from rahulnmohan:dts-mpeg2ts-update 21efa0810db31550d6b215639f9ca2af6a32139a
PiperOrigin-RevId: 598854998
`Surface`s don't expose their size via Java APIs. Recommend apps pass a
`SurfaceView` (which is preferable to `TextureView` as it's more efficient)
or `SurfaceHolder` so they benefit from the player automatically passing the
size down to the video renderer via `MSG_SET_VIDEO_OUTPUT_RESOLUTION`.
PiperOrigin-RevId: 598804258
MatroskaExtractor will no longer be wrapped in SubtitleTranscodingExtractor, but instead use SubtitleTranscodingExtractorOutput under the hood.
FLAG_EMIT_RAW_SUBTITLE_DATA flag will be used to toggle between subtitle parsing during extraction (before the sample queue) or during decoding (after the sample queue).
The new extractor dump files generated by `MatroskaExtractorTest` now follow the new parsing logic and hence have mimeType as `x-media3-cues`.
PiperOrigin-RevId: 598616231
AviExtractor supports text tracks (`AviExtractor.FOURCC_txts` -> `C.TRACK_TYPE_TEXT`) with subtitles.
AviExtractor will no longer be wrapped in SubtitleTranscodingExtractor, but instead use SubtitleTranscodingExtractorOutput under the hood.
FLAG_EMIT_RAW_SUBTITLE_DATA flag will be used to toggle between subtitle parsing during extraction (before the sample queue) or during decoding (after the sample queue).
PiperOrigin-RevId: 598594981
Moves setting bitmapFactory options from BitmapLoader to DatasourceBitmapLoader
BitmapLoader is a general interface for bitmap loading that could use loading implementations other that BitmapFactory, with the rise of Glide being a loader of choice. It's best to correct this interface so that it remains generic
We can't deprecate easily because the other loadBitmap method in that case has a default implementation that relies on the first one, so the change is still breaking. BitmapLoader is public api in common, but it's @UnstableAPI and hasn't been around for very long (be38670391 added it), so it seems this is the best way forward.
PiperOrigin-RevId: 597897098
Subclasses of this component can customize it by wrapping with the
decorator pattern, and a custom `CompositeSequencableLoaderFactory`
allows access to the list of delegate `SequenceableLoader` instances.
The `final` keyword was removed as part of <unknown commit> but this
component never ended up being subclassed within the library.
Making this class `final` makes upcoming changes easier to reason
about.
PiperOrigin-RevId: 597853429
The seek table in a Xing/Info header is very imprecise (max resolution
of 255 to describe each of 100 byte positions in the file). Seeking
using a constant bitrate assumption is more accurate, especially for
longer files (which exacerbates the imprecision of the Info header).
VBR files should contain an Xing header, while an Info header is
identical but indicates the file is CBR.
Issue: androidx/media#878
PiperOrigin-RevId: 597827891
7e65cce967 introduced a regression on ExoPlayer.setVideoEffects()
where there is flash on the screen after the first few frames are shown.
Before 7e65cce967, the first frames of the content are missed, until
MediaCodecVideoRenderer sends the onVideoSizeChanged() callback. The
first frame is processed but not shown, the onVideoSizeChanged() is
triggered and the renderer receives a video output resolution message as
a response from the UI.
7e65cce967 fixed the missed first frames by setting a surface on the
CompositingVideoSinkProvider before the provider is initialized, and as
as result:
- the first frames are rendered
- the MediaCodecVideoRenderer sends the the onVideoSizeChanged() after
frames are shown
- the UI sends a video output resolution change
- the MediaCodecVideoRenderer updates the CompositingVideoSinkProvider
which causes the flash.
The underlying problem is with onVideoSizeChanged() not being
triggered early enough, before the first frame is shown.
Because the flashing is a regression, this commit is reverting
7e65cce967 but keeps the test that was added as ignored, so that the
test is re-enabled as soon as we address the underlying issue.
PiperOrigin-RevId: 597814013
WebvttExtractor will no longer be wrapped in SubtitleTranscodingExtractor, but instead use SubtitleTranscodingExtractorOutput under the hood.
A new constructor will take a boolean parameter to toggle between subtitle parsing during extraction (before the sample queue) or during decoding (after the sample queue).
PiperOrigin-RevId: 597604942
Partially addresses the following TODO, by simplifying the DefaultShaderProgram
API surface.
```
// TODO(b/274109008): Refactor DefaultShaderProgram to create a class just for sampling.
```
PiperOrigin-RevId: 597575575
Originally a `PlayerId` has to be passed and it should be the same id of the player who is going to play the sources, but it turns out to be unnecessary.
When preloading, we can set a `PlayerId.UNSET` inside of the `PreloadMediaSource`, as there is no ongoing playback. And when the source is handed over to player, player will set the player's `PlayerId`.
PiperOrigin-RevId: 597475119
This involves promoting `setTextTranscodingEnabled` to
`ExtractorsFactory`, and also making it experimental, to indicate it's a
short-lived method.
PiperOrigin-RevId: 597235252
Mp4Extractor will no longer be wrapped in SubtitleTranscodingExtractor, but instead use SubtitleTranscodingExtractorOutput under the hood.
FLAG_EMIT_RAW_SUBTITLE_DATA flag will be used to toggle between subtitle parsing during extraction (before the sample queue) or during decoding (after the sample queue).
PiperOrigin-RevId: 597221831
The capabilities change depending on the attributes, so we should
pass down the actual attributes used during playback and update
the capabilities whenever these change.
PiperOrigin-RevId: 597197507
This means in a later change we can still use some of the info for CBR
files, even if we want to ignore the imprecise table of contents and
seek based on a constant bitrate assumption instead.
PiperOrigin-RevId: 597193997
An audio file can only play sound between two PCM samples (the 'start'
and 'end' of section of a wave form). Therefore when calculating
duration from a count of PCM samples we need to subtract one first (the
'end' sample which has no duration of its own).
This only changes durations by one PCM sample (21us - 22us for 44.1kHz sample
rate).
PiperOrigin-RevId: 596990306
The extractor knows the PCM encoding of the losslessly
encoded data in the samples and should set it in the
Format to allow downstream components to use this information.
PiperOrigin-RevId: 596974863
We introduce SubtitleParser.Factory that supports no formats to be used FragmentedMp4Extractors that will not do any subtitle parsing on the extraction side. We also slowly move away from using SubtitleTranscodingExtractor to SubtitleTranscodingExtractorOutput (hence making it public).
This is required by individual Extractor impls so that they can start using SubtitleTranscodingExtractorOutput rather than be wrapped by SubtitleTranscodingExtractor. The latter is to be deprecated after all the subtitle related Extractors have achieved this migration.
PiperOrigin-RevId: 596942147
Whenever the inputColorInfo updates, update the samplingGlShaderProgram.
Also, allow either SDR or gamma2.2 to be used for HDR->SDR tone-mapping
`outputColorInfo` request. This is required because we can't update the
`outputColorInfo`, but plan to always use gamma2.2 for `outputColorInfo` in the
future.
This allows VideoFrameProcessor to work as is for exoplayer previewing, but
only when not seeking. As we haven't plumbed the per-stream inputColorInfo from
ExoPlayer down to VFP.registerInputStream, follow-up CLs will be needed to
properly support previewing with changing inputColorInfo.
PiperOrigin-RevId: 596627890
This behavior was previously available as opt-in via
`MediaItem.DrmConfiguration.Builder.setPlayClearContentWithoutKey` and
`DefaultDrmSessionManager.Builder.setPlayClearSamplesWithoutKeys`. This
change flips the default of both these properties to true.
This should speed up the time for playback to start when playing DRM
content with a 'clear lead' of unencrypted samples at the start.
Previously playback would wait until the keys for the later encrypted
samples were ready. The new behaviour could result in mid-playback
stalls/rebuffers if the keys are not ready yet by the transition from
clear to encrypted samples, but this is not really a regression since
previously playback wouldn't have started at all at this point.
PiperOrigin-RevId: 595992727
This file is CBR encoded with LAME, so it has an `Info` header (the CBR
equivalent to `Xing`).
A follow-up change will use this file in `Mp3ExtractorTest`.
Issue: androidx/media#878
PiperOrigin-RevId: 595938327
These methods sound similar, but have different behaviour. This change
tries to make the distinction clearer, and sign-post from one to the
other.
#minor-release
Issue: androidx/media#910
PiperOrigin-RevId: 595701540
When the media notification controller is requested for a session
with `getConnectedControllerForSession` and the `Future` is not null
but not yet completed, the `Future` was returned either way. This was
reported as creating a race condition between the notification
being requested for update the very first time, and the media
notification controller having completed connecting to the session.
Returning null from `getConnectedControllerForSession` when the
`Future` is available but not yet done fixes the problem. This is
safe because for the case when a notification update is dropped,
the media notification controller will trigger the update as soon
as the connection completes.
Issue: androidx/media#917
#minor-release
PiperOrigin-RevId: 595699929
`Cea608Parser` and `Cea708Parser` don't currently work correctly on
their own without the re-ordering of input buffers implemented in
`CeaDecoder`, and it's not clear how we can properly do this re-ordering
during extraction. This change ensures that if 'parse subtitles
during extraction' is enabled, CEA-6/708 subs will be passed through
without transcoding and can then be decoded during rendering by
`Cea6/708Decoder`.
PiperOrigin-RevId: 595658628
Invalid frames have no impact on ExoPlayer ability to play the media and should not fail on errors.
Some tools can add 100Mb images in the tags that will trigger recoverable OOM with this fix.
The `Cea608DecoderTest` added here fails if re-ordering is removed from
`CeaDecoder`.
The `Cea608ParserTest` is added with `@Ignore` because there's currently
no re-ordering support in this part of the subtitle handling pipeline
(partly because there's no concept of 'current playback time', meaning
it's hard to know **when** to re-order).
PiperOrigin-RevId: 595320205
When the 'when' timer of the notification is disabled
`DefaultMediaNotificationProvider` may set `C.TIME_UNSET`
as the time. Users reported problems on some devices with
this and the docs ask for an event time that probably
shouldn't be a negative number.
This change sets `0L` instead of `C.TIME_UNSET` when the
timer is disabled.
Issue: androidx/media#903
#minor-release
PiperOrigin-RevId: 594451074
Changes includes;
1. Public API to enable fMP4 and to pass fragment duration.
2. Added `FragmentedMp4Writer`.
3. Added logic to create fragments based on given fragment duration.
4. Write "moov" box only once in the beginning.
3. Add all the required boxes for current implementation.
4. Unit tests for all the new boxes.
5. E2E test for generating fMP4.
Note: The output file is un seek-able with this first implementation.
PiperOrigin-RevId: 594426486
Mp4Muxer does not support out of order B-frames. Currently it
silently writes out of order B-frames, producing an invalid file (with
negative sample durations).
Although `Mp4Extractor` is somehow able to process this invalid file and
`Exoplayer` is able to play it but that is unexpected.
The `sample.mp4` test file contains B frames. Other test files does not
contain `H264 video + AAC audio` format hence created a new test file by
running `sample.mp4` via `Transformer` after applying some effects.
PiperOrigin-RevId: 594016144
This more closely matches the intended, documented behaviour of this
method. The previous implementation was incorrectly checking
`sampleDataEnd + newSampleSize`, but it's more correct to compare
`existingSampleDataLength + newSampleSize`, in order to handle the
case of non-zero `sampleDataStart`. We've already checked above whether
`newSampleSize` fits after `sampleDataEnd` without growing or
reshuffling the array, so no need to basically repeat that check.
In the case of handling one sample at a time, the previous
implementation resulted in `sampleData` growing by `sampleSize`
**every time** the pointers reached the end. With the new check for
`sampleDataStart == sampleDataEnd`, this is avoided by always writing
each new sample at the start of the array (because the previous sample
has already been consumed), meaning `sampleData` remains equal to
`sampleSize` without growing.
PiperOrigin-RevId: 593119927
These were missing in 5211ff0dc1
Without these, the `": null"` is still logged (but the `"Unexpected
IllegalStateException"` bit is not), because the **whole** string
concatenation is compared to null as the boolean condition of the
ternary, and this condition is always false (the result of a string
concatentation is never `null`) i.e. (with excess parentheses for
clarity):
```
(("Unexpected " + cause.getClass().getSimpleName() + cause.getMessage()) != null)
? ": " + cause.getMessage()
: ""
```
Also add a test because obviously this isn't as simple as I'd thought.
PiperOrigin-RevId: 593079975
The previous code led me to misread this stack trace as a null pointer
exception, but it's really an index out of bounds exception:
```
Caused by: androidx.media3.exoplayer.upstream.Loader$UnexpectedLoaderException: Unexpected IllegalArgumentException: null
at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:435)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at java.lang.Thread.run(Thread.java:1012)
Caused by: java.lang.IllegalArgumentException
at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:40)
at androidx.media3.common.util.ParsableByteArray.setPosition(ParsableByteArray.java:164)
at androidx.media3.extractor.text.cea.Cea608Parser.parse(Cea608Parser.java:440)
```
PiperOrigin-RevId: 592876546
Instead, for input videos, use the colorInfo provided by the extractor. Similarly, for input images, use sRGB, the only color currently in use.
Textures do still need the input ColorInfo provided though.
PiperOrigin-RevId: 592875967
Replace the event for notifying fallback to cover codec initialization in
general (but keeping the list of errors).
Add a flag to control whether to try non-primary codecs, with the same
documentation as the similar flag in ExoPlayer's renderer.
Make the class final as it shouldn't be necessary to subclass it.
PiperOrigin-RevId: 592869868
This is similar to the HLS fix in 770ca66fbc
Similar to HLS, the original problem here was **not** modifying the
`Format` for caption tracks
embedded into the video stream. I tried just updating the format in
both places, but that caused new failures because the new
('transcoded') format was then fed into `FragmentedMp4Extractor` as
part of `closedCaptionFormats`, which resulted in the CEA-608 data
being emitted from `FragmentedMp4Extractor` with the incorrect
`application/x-media3-cues` MIME type (but the bytes were actually
CEA-608), meaning the transcoding wrapper passed it through without
transcoding and decoding failed (because obviously CEA-608 bytes can't
be decoded by `CueDecoder` which is expecting a `Bundle` from
`CuesWithTiming.toBundle`.
To resolve this we keep track of the 'original' caption formats inside
`TrackGroupInfo`, so we can feed them into `FragmentedMp4Extractor`.
For all other usages in `DashMediaPeriod` we use the 'transcoded'
caption formats.
PiperOrigin-RevId: 592866262
With fMP4 implementation there will be two writers `DefaultMp4Writer`
and `FragmentedMp4Writer`.
Changes includes:
1. Make Mp4Writer as an abstract class and keep only common functionality
into it.
2. Create a DefaultMp4Writer which contains existing logic to write MP4.
3. The fMP4 logic needs to access `pending sample buffer info` at various
places, so did refactoring to split List<Pair<BufferInfo, ByteBuffer>>
into two separate lists.
PiperOrigin-RevId: 592861398
Checking the output format's mime type may skip tests more often than we'd like,
because we may desire using a lower-spec output mimetype than what's passed in, if
based on the input's HDR mimetype value.
Therefore, update this output format to null, for tone-mapping tests
PiperOrigin-RevId: 592855713
Also updated the `README` file to accurately specify the use of NDK r23c and the default setting `ANDROID_ABI=21` for NDK r26b.
PiperOrigin-RevId: 592845796
Issues with the current implementation
1. The implementation is unnecessarily complicated and can be
easily simplified.To make all the tracks start from the same time,
its only the first sample that require some timestamp adjustments
but the current implementation shifts all the timestamps. Since method
calculates the `sample duration`, shifting all the timestamps has no effect
as such.
2. The implementation always forced first sample to start at 0. But when we
want to use same method for `Fragmented MP4` then it will look inaccurate
as we will call this method for different `fragments` and each `fragment`
will not start from 0 presentation time. Although the output will be same
since this method returns `duration` and not the `timestamps`.
3. As per previous implementation if there is just one sample then
its duration is made equals to its presentation time, which looks incorrect.
With new changes, if a single sample is passed then its duration will always
be 0 irrespective of specified last sample duration behaviour.
PiperOrigin-RevId: 592826612
getAudioCapabilities currently creates the receiver and returns
the current capabilities. This is error-prone because the
capabilities are also available as a class field.
This can made cleaner by letting the method just create the receiver
and all access to the capabilities can be made via class field.
PiperOrigin-RevId: 592591590
HLS distinguishes between 'subtitles' (WebVTT or TTML distributed in
separate files with their own playlist) and 'captions' (CEA-608 or 708,
distributed muxed into the video file).
The format transformation added in 7b762642db
only applies to subtitles and not captions. This change makes the same
transformation for caption formats.
This resolves an error like:
```
SampleQueueMappingException: Unable to bind a sample queue to TrackGroup with MIME type application/cea-608.
```
Also add two playback tests for HLS CEA-608, one that parses during
decoding (old way) and one during extraction (new way). Adding these
tests is what alerted me to this issue.
PiperOrigin-RevId: 592571284
This was generated by combining the existing `ts/bbb_2500ms.ts` test
asset and a temporary `.srt` file using
https://cloud.google.com/transcoder/docs/how-to/captions-and-subtitles
This doesn't directly reproduce the problem fixed by
7ca26f898d,
because the CEA-608 subs are structured differently to the stream I
discovered the problem with (from Issue: androidx/media#887). However this test
does fail if that fix is reverted after
486230fbd7.
I'm also not able to repro the character duplication reported in
Issue: androidx/media#887 by just changing the manifest in this CL. I'm not yet
sure on the exact differences between the stream provided on GitHub
and this stream.
This stream does provide some regression protection, because it
currently fails with 'new' subtitle parsing
(`DashMediaSource.Factory.experimentalParseSubtitlesDuringExtraction(true)`),
though I'm not sure on the exact reason for that yet.
PiperOrigin-RevId: 592476328
This is a refactoring that allows the `MediaPeriodQueue` to
create media period holders without all the collaborators
being passes in to `enqueueNextMediaPeriodHolder(...)`.
The factory is presumably also helpful in unit test to
know whether and when exactly a holder is created in
the preloading process.
PiperOrigin-RevId: 592301400
We deliberately left `Cea608Decoder` and `Cea708Decoder` intact when
creating the respective `SubtitleParser` implementations (in
27caeb8038
and
94e45eb4ad
respectively).
However we didn't correctly change the behaviour of
`SubtitleDecoderFactory.DEFAULT` in order to use these 'legacy'
implementations. We firstly left the `Cea608Decoder` instantiation
**after** the `DefaultSubtitleParserFactory.supportsFormat()` check,
meaning that code would never be reached (since `supportsFormat` now
returns true for CEA-608). Then in the second change (which was supposed
to only be about CEA-708) we removed all instantiations of **both**
`Cea608Decoder` and `Cea708Decoder`. This change puts the decoder
instantiations back and moves them above the
`DefaultSubtitleParserFactory.supportsFormat` check, meaning they will
be used in preference to `DelegatingSubtitleDecoder`.
This resolves the immediately-disappearing subtitles tracked by
Issue: androidx/media#904.
PiperOrigin-RevId: 592217937
While investigating Issue: androidx/media#887 I naively assumed the CEA-608
captions were in a TS file, but they're actually in an MP4 (which is
possibly obvious given DASH only supports MP4). This change includes
container info in the `EventLogger` `tracks` output.
PiperOrigin-RevId: 592192752
This allows us to remove the additional thread we create
for asynchronous buffer queuing and use a synchronized
queueing approach again.
This API looks strictly beneficial on all tested devices,
but since this code path in MediaCodec has not been used
widely, we leave an opt-out flag for now.
PiperOrigin-RevId: 591867472
The parameters may change the decoding behavior of the
following samples and it's suprising/wrong to apply them
from another thread where we can't guarantee from which
sample they apply.
PiperOrigin-RevId: 591808760
Move `inputColorInfo` from `VideoFrameProcessor`'s `Factory.create` to `FrameInfo`,
input via `registerInputStream`.
Also, manually tested on exoplayer demo that setVideoEffects still works.
PiperOrigin-RevId: 591273545
When an adaptive source has been preloaded, the track selection
of the player should possibly not override the preloaded selection
to avoid discarding preloaded data.
PiperOrigin-RevId: 591256334
The number of empty image byte arrays written is one less than the
total number of tiles in the image. The empty byte arrays act as
placeholders for individual tiles inside ImageRenderer.
PiperOrigin-RevId: 591231432
This is because currently
1. Player sets a surfaceView to render to
2. Player intializes the renderer
3. MCVR initializes the VideoSinkProvider, by extension VideoGraph
But when 1 happens, MCVR doesn't set the surfaceView on the VideoGraph because
it's not initialized. Consequently after VideoGraph is initialized, it doesn't
have a surface to render to, and thus dropping the first a few frames.
Also adds a test for first frame to verify the correct first frame is rendered.
PiperOrigin-RevId: 591228174
When track is changed during playback, `playbackPositionUs` may be in middle of a chunk and `loadPositionUs` should be the start of that chunk. In this situation `loadPositionUs` can be less than the current `playbackPositionUs`, resulting into negative `bufferedDurationUs`. It translates to having no buffer and hence we should send `0` for `bufferedDurationUs` when creating new instances of `CmcdData.Factory`.
Issue: androidx/media#888
#minor-release
PiperOrigin-RevId: 591099785
Otherwise, there's a memory leak of ~30MB, as this is never released.
This likely used to be considered released as part of what now became
`intermediateGlShaderPrograms`, but its release was missed after we split
`finalShaderProgramWrapper` out from the larger glShaderProgram list.
PiperOrigin-RevId: 590954785
To prepare to move `inputColorInfo` from `VFP.Factory.create` to
`VFP.registerInputStream`, move all usage of `inputColorInfo` to be *after*
`registerInputStream`.
To do this, defer creation of `externalShaderProgram` instances, which require
`inputColorInfo`. However, we must still initialize `InputSwitcher` and OpenGL
ES 3.0 contexts in the VFP create, to create and request the input surface from
ExternalTextureManager.
PiperOrigin-RevId: 590937251
The timestamp adjuster also estimates the number of wraparounds
of the 90Khz TS timestamp. It does that by assuming that a new
timestamp is always close to the previous one (in either direction).
This logic doesn't always work for duration estimates because the
timestamp at the end of the media is not close to the one at the
beginning and it may also never be less than the one at the beginning.
This can be fixed by introducing a new estimation model that assumes
the new timestamp is strictly greater than the previous one without
making the assumption that it has to be close to it.
Issue: androidx/media#855
#minor-release
PiperOrigin-RevId: 590936953
When broadcasting a notifyChildrenChanged event, the task for legacy
controllers was sent to the broadcasting callback. This would
technically work, but because the subscription list is maintained
with specific controllers, the broadcast controller isn't subscribed
and hence the call wasn't executed.
This change calls the overloaded method for a specific controller
for each connected controller. Making sure (only) subscribed
controllers are notified.
Issue: androidx/media#644
PiperOrigin-RevId: 590904037
Vertical and horizontal tile counts inside Format used by ImageRenderer
used to be required to equal 1 and were set to 1 regardless of what
the manifest said.
This change removes the above requirement and sets the tile counts to
the values from the manifest.
PiperOrigin-RevId: 590608353
Using 0 as the unset prepare position is the root cause of a number of issues,
as outliine in the ExoPlayer issue https://github.com/google/ExoPlayer/issues/7975
The premise of this fix is that once the prepare override is used (the initial call
to `selectTracks()`) it is never needed again, so simply invalidate it after use.
Despite GL 3.0 not being required on API 29+, it is experimentally
determined to always be supported on our testing devices, on API 29+.
That said, still fall back to OpenGL 2.0 if 3.0 is not supported,
just in case.
PiperOrigin-RevId: 590569772
It's better to use the generic Android resource scheme which
is at least as powerful as our own one.
Issue: androidx/media#868
PiperOrigin-RevId: 590168919
`Mp4Writer` receives all the tracks/samples and then it adds them to
corresponding `Track` object. As the `Track` object is primarily to hold
track specific data and `Mp4Writer` is responsible for interleaving related
logic, the call to `doInterleave` should be with `Mp4Writer`. The `Track`
object should just act like a data holding class.
This will also help in reusing `Track` class for fragmented MP4 implementation.
PiperOrigin-RevId: 590138002
An app that uses the `MediaSession` without a `MediaSessionService` may
want to connect a media notification controller on it own. To avoid apps
using a string literal, the key should be exposed as public.
PiperOrigin-RevId: 589875603
The MP4 data in JPEG motion photos can contain multiple `video/hevc` tracks, but only the first is at a playable frame rate while the others are low-fps, high-res tracks designed for specific use-cases (not direct video playback).
ExoPlayer currently selects the unplayable track by default, because it
has a higher resolution. This change introduces a flag to
`Mp4Extractor` that results in the first video track being marked as
`ROLE_FLAG_MAIN`, and all subsequent video tracks `ROLE_FLAG_ALTERNATE`
- this then results in the playable lower-res track being selected by
default.
PiperOrigin-RevId: 589832072
There is a bug in CompositionVideoSinkProvider where the rotation
effect is applied only on the first video and the same effect
is applied on subsequent videos without checking if the next
items in the playlist must be rotated.
The bug applies only on devices with API < 21.
PiperOrigin-RevId: 589823797
We currently pause playback to prevent further progress while the
app thread runs assertion and triggers additional actions. This is
not ideal because we do not actually want to pause playback in
almost all cases where this method used.
This can be improved by keeping the playback thread blocked and only
unblock it the next time the app thread waits for the player (either
via RobolectricUtil methods or by blocking the thread itself). To
add this automatic handling, this change introduces a new util class
for the tests that can keep the list of waiting threads statically
(because the access to this logic is spread across multiple independent
classes).
PiperOrigin-RevId: 589784204
Move the `AnalyticsListener` implementation to a private inner class.
This avoids polluting the public API of `ExoHostedTest`, especially as
it's designed to be extended.
PiperOrigin-RevId: 589188113
Also adds an alternate way to configure the AudioGraph.
Apps should no longer need to ensure that inputs have the same sample
rate.
PiperOrigin-RevId: 588747431
Added a new ABANDONED option so one can tell the difference between when the optimization has been requested but not applied vs not requested at all. also changed the ordering do better represent the hierarchy of failure modes
PiperOrigin-RevId: 588720513
Fix a bug where CompositingVideoSinkProvider.isInitialized() returns
true even after releasing the CompositingVideoSinkProvider.
PiperOrigin-RevId: 588481744
This change moves the instantiation of the CompositingVideoSinkProvider
out of MediaCodecVideoRenderer so that the composition preview player can
re-use the CompositingVideoSinkProvider instance for driving the rendering of
images.
The main point of the change is the ownership of the
VideoFrameReleaseControl, which decides when a frame should be rendered
and so far was owned by the MediaCodecVideoRenderer. With this change,
in the context of composition preview, the VideoFrameReleaseControl
is no longer owned by MediaCodecVideoRenderer, but provided to it.
This way, the CompositingVideoSinkProvider instance, hence the
VideoFrameReleaseControl can be re-used to funnel images into the
video pipeline and render the pipeline from elsewhere (and not
MediaCodecVideoRenderer).
PiperOrigin-RevId: 588459007
The serializable form is used when we need to serialize the result into
bytes in the sample queue. The binder-based (ultimately
filedescriptor-based) form is used for
session/controller IPC, in order to avoid sending the bitmap bytes over
the IPC.
Issue: androidx/media#836
#minor-release
PiperOrigin-RevId: 588420836
This helped me debug which bitmap was actually failing. Otherwise, you need to clear all bitmaps on the device relating to this test, then adb pull all bitmaps and see the last uploaded one to see, which is much more confusing.
PiperOrigin-RevId: 588374081
This image has two video tracks in the MP4 data, one is a 'real' video
which we want to play by default, and the other is a low-fps video track
which isn't intended to be directly played, it's encoded in HEVC for
compression and decoding efficiency.
This test demonstrates ExoPlayer's current behaviour default extraction
and playback, which results in selecting the high-res, low-fps track
(actually single sample in this example), instead of playing the actual
video.
PiperOrigin-RevId: 588068908
As Opus decoders skip some bytes prior to playback during a seek, the renderer for bypass playback should send samples to the decoder even if they would be decode-only. However, the renderer should not send samples with time preceding that range. This change adds that constraint.
#minor-release
PiperOrigin-RevId: 588014983
Both the extension OPUS decoder and the OMX/C2 MediaCodec
implementations for OPUS and VORBIS decode into the channel
layout defined by VORBIS. See
https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-140001.2.3
While this is technically correct for a stand-alone OPUS or VORBIS
decoder, it doesn't match the channel layout expected by Android.
See https://developer.android.com/reference/android/media/AudioFormat#channelMask
The fix is to apply the channel mapping after decoding if needed.
Also add e2e tests with audio dumps for the extension renderer,
including a new 5.1 channel test file.
Issue: google/ExoPlayer#8396
#minor-release
PiperOrigin-RevId: 588004832
`mediaCrypto` is initialized before `codec` in
`maybeInitCodecOrBypass`. Before this change, it was possible for
`maybeInitCodecOrBypass` to complete with `mediaCrypto != null` and
`codec == null`, in particular if it was run as part of clearing the
player surface (since in that case, no video codec is initialized).
This inconsistent state then causes issues during a later invocation of
`maybeInitCodecOrBypass`, when `mediaCrypto` is still non-null, and
`mediaCryptoRequiresSecureDecoder = true`, but the
content has been changed to unencrypted with no associated DRM session.
This results in a playback error, because a secure decoder is
initialized but there's no DRM session available to work with it.
This change ensures that when `maybeInitCodecOrBypass` completes,
either both `mediaCrypto != null` and `codec != null` (i.e. codec
initialization was completed) or `mediaCrypto == null` and
`codec == null` (i.e. codec initialization was not completed). We also
ensure that when nulling out `mediaCrypto` we also set
`maybeInitCodecOrBypass = false`. A later change should be able to
demote `maybeInitCodecOrBypass` from a field to a local in order to
remove any risk of that part of state becoming out of sync. This
resolves the issue, because during the second invocation of
`maybeInitCodecOrBypass` an insecure decoder is now (correctly)
initialized and the unencrypted content is successfully played.
#minor-release
PiperOrigin-RevId: 587713911
We set the `playerId` only in the constructor currently. But in the case of this source doesn't have `preload` called, the player's release of this source will set the playerId to `null`, which makes this source un-reusable with a null `playerId`.
PiperOrigin-RevId: 587698214
* @Before and @After usage. [1].
* static fields before non-static. [2].
@Before method should typically be paired with an @After method,
focused on ensuring that the component is released regardless of what
the test does.
In tests, inlining final class variables is preferrable [1]. In general
things like the file path should be part of the test (the @Rule means
we don't need before/after) if only used once.
Statically importing values and using them directly is preferable to
having a variable declared as non-final that's effectively final,
because from a readability perspective someone can see (the caps) that
the value is final static and immutable, so doesn't have to check if
it's changed/reassigned.
PiperOrigin-RevId: 586697887
Before, a translucent overlay over an opaque video would result in a
translucent output. This is not consistent with physical properties of light
(if putting a translucent object in front of an opaque object, you can't see
behind the opaque object).
Using the mixing properties from DefaultVideoCompositor.
PiperOrigin-RevId: 586636275
`fromBundle` doesn't distinguish between `FIELD_BITMAP` and `FIELD_TEXT`
being present with a null value, or being absent, so we might as well
avoid including them when the value is null.
I've separated this from a later change to add
`Cue.toSerializableBundle` which will also skip setting a bitmap value
into the `Bundle` if `this.bitmap == null`. This is partly because it
results in changes to a lot of extractor test dump files, and it's
easier to review that as a separate change.
PiperOrigin-RevId: 586626141
Add an check when loading progressive media in case the load
is canceled. If the player is released very early, the progressive
media period may carry on with the initial loading unnecessarily.
PiperOrigin-RevId: 586288385
In `TextOverlay` and `DrawableOverlay`, treat `Bitmap` as a buffer, where we
allocate it rarely and reuse it as long as possible before making a new one.
In `BitmapOverlay`, avoid allocating GL textures too often as well.
Strongly reduces allocations and memory usage growth (saving ~100-150 MB on 4k60fps
at high end), at the cost of more code complexity and low-end using 70MB more, on
1/1 comparisons.
PiperOrigin-RevId: 585990602
In `ExternalTextureManager` fields are accessed from the GL thread and the class needs to be constructed on the GL thread.
Also visibly document threading requirement in the parent class.
PiperOrigin-RevId: 585941284
The issue that motivated adding this (frames unexpectedly being dropped by the
decoder) has been addressed, so we can turn off the logging to reduce
unnecessary allocations during transformation. We can easily turn on debug
logging in future as needed by setting `DebugTraceUtil.DEBUG = true`.
Also avoid allocations for string building at logging call sites by passing a
format string for extra info. Formatting the string now only happens when
debugging is turned on.
Tested manually by running transformations in the new state (DEBUG = false) and
with debugging turned on.
PiperOrigin-RevId: 585666349
Serializing bitmap cues is currently broken, but this test is
incorrectly passing. This change makes two changes to introduce the same
failure (both changes are necessary, each one alone still passes):
1. Move from Robolectric to an instrumentation test.
2. Trigger the `Bitmap` to be serialized using a file descriptor, either
by calling `Bitmap.asShared` in the test when constructing the `Cue`,
or constructing the `Bitmap` from a 'real' image byte array instead a
1x1 token image.
Issue: androidx/media#836
PiperOrigin-RevId: 585643486
The decoder and encoder won't accept high values for frame rate, so avoid
setting the key when configuring the decoder, and set a default value for the
encoder (where the key is required).
Also skip SSIM calculation for 4k, where the device lacks concurrent decoding
support.
PiperOrigin-RevId: 585604976
At least some Android 14 devices still have the same invalid URL issue
when using ClearKey DRM, so might as well apply the check to newer
versions of Android as well.
Using `Integer.MAX_VALUE` risks causing arithmetic overflow in the codec
implementation.
Issue: androidx/media#810
#minor-release
PiperOrigin-RevId: 585104621
We don't get the buffer back after decoding to make a proper decision
about whether dropping the buffer is needed, so we do the next best
thing and tell the codec to drop the buffer if its input timestamp is
less than the intended start time.
PiperOrigin-RevId: 584863144
Move the reflective loading of VideoFrameProcessor from
MediaCodecVideoRenderer to the CompositingVideoSinkProvider. This is so
that all reflective code lives in one place. The
CompositingVideoSinkProvider already has reflective code to load the
default PreviewingVideoGraph.Factory.
PiperOrigin-RevId: 584852547
These usages have no need for the double ended input functionality. All
other usages across media3 are ConcurrentLinkedQueue.
PiperOrigin-RevId: 584841104
In OverlayShaderProgram, this method is used quite a lot, and is the only method from Util.java in this file. Marginally reduce complexity by using a static import instead.
PiperOrigin-RevId: 584828455
Reduce short-lived allocations of potentially large objects, like Bitmap.
Unfortunately, this does make the TextureOverlay interface more messy though, requiring a way to signal whether the texture should be flipped vertically.
PiperOrigin-RevId: 584661400
These changes are also compatible with FFmpeg 5.1, which is now minimum supported version.
Also set -Wl,-Bsymbolic flag via target_link_options command which is more correct.
Split CompositingVideoSinkProvider.VideoSinkImpl in two classes:
- VideoSinkImpl now only receives input from MediaCodecVideoRenderer and
forwards frames to its connected VideoFrameProcessor
- VideoFrameRenderControl takes composited frames out of the VideoGraph
and schedules the rendering of those.
- CompositingVideoSinkProvider connects VideoSinkImpl with
VideoFramesRenderer.
PiperOrigin-RevId: 584605078
This change adds `MediaController.getSessionExtras()` through
which a controller can access the session extras.
The session extras can be set for the entire session when
building the session. This can be overridden for specific
controllers in `MediaSession.Callback.onConnect`.
PiperOrigin-RevId: 584430419
This change fixes a bug with seeking forward in MIDI. When seeking forward,
the progressive media period attempts to seek within the sample queue, if a
key-frame exists before the seeking position. With MIDI, however, we can
only skip Note-On and Note-Off samples and all other samples must be sent
to the MIDI decoder.
When seeking outside the sample queue, the MidiExtractor already
instructs the player to start from the beginning of the MIDI input. With
this change, only the first output sample is a key-frame, thus the
progressive media period can no longer seek within the sample queue and
is forced to seek from the MIDI input start always.
Issue: androidx/media#704
#minor-release
PiperOrigin-RevId: 584321443
The live offset override is used to replace the media-defined
live offset after user seeks to ensure the live adjustment adjusts
to the new user-provided live offset and doesn't go back to the
original one.
However, the code currently clips the override to the min/max
live offsets defined in LiveConfiguration. This is useful to
clip the default value (in case of inconsistent values in the media),
but the clipping shouldn't be applied to user overrides as
the player will then adjust the position back to the min/max
and doesn't stay at the desired user position.
See 2416d99857 (r132871601)
#minor-release
PiperOrigin-RevId: 584311004
Composition and EditedMediaItemSequence don't allow empty lists in their main
constructors, so shouldn't the vararg API. This is more inline with Effective
Java item 53.
PiperOrigin-RevId: 583415124
When transmuxing, the `EncodedSampleExporter` maintains a queue of input
buffers that get filled with encoded data by the asset loader. The number of
buffers was limited to avoid using more and more memory if producer (asset
loader) gets far ahead of the consumer (exporter).
Previously this limit was fixed at 10 buffers, but increasing the number of
buffers can make some transmux operations much faster. Allow allocating between
a min and max number of buffers, and also set a target allocation size beyond
which new buffers can't be allocated. This allows audio formats which require
many small buffers to be processed more quickly, while preventing allocating
too much memory for hypothetical very high bitrate formats.
'Remove video' edits on local videos in particular get much faster, because
audio buffers are very short and there are lots of them. With a sample 10
minute video, a 'remove video' edit took 2 seconds (36 seconds before this
change). With a sample 1 minute removing video took 0.25 seconds after this
change (2.5 seconds before).
The speed improvement is smaller for other types of edits that retain the video
track. Transmuxing a 10 minute video retaining the video track took 26 seconds
(40 seconds before).
PiperOrigin-RevId: 583390284
In some contexts (e.g. BottomSheetDialogFrament), Material Design
themes will override the default of singleLine=false to true. This
causes layout problems because the forward/rewind buttons are no
longer visible with singleLine=true.
This problem can be avoided by explicitly requesting the default
value of false in our layout files.
Issue: androidx/media#511
#minor-release
PiperOrigin-RevId: 582604131
This removes the flakiness of the HLS playback test as well.
Previously, this flow only worked for standalone WebVTT subtitles (7b762642db)
PiperOrigin-RevId: 582359383
The test is hidden behind the `@Ignore` annotation due to some
flakiness. However, it will be removed when the subtitle parsing is
moved to a pre-sample-queue architecture.
The test media was created with:
```shell
$ cp ../../dash/standalone-ttml/sample.xml sample.ttml
$ MP4Box -add sample.ttml sample.text.mp4
$ MP4Box -frag 10000 sample.text.mp4
$ rm sample.ttml
```
PiperOrigin-RevId: 582347113
This helps to play ads where the file extension is not sufficient
to deduce the correct source in DefaultMediaSourceFactory.
PiperOrigin-RevId: 582340114
Currently, ads are only defined by a single URL, which makes it
impossible to define additional fields needed to play ads correctly.
This can be fixed by using a full MediaItem in AdPlaybackState,
replacing the previous Uri field.
PiperOrigin-RevId: 582331588
