Commit graph

3672 commits

Author SHA1 Message Date
Ian Baker
0b482ef58c Remove columnLock, columnCount (unimplemented) and rowLock (always true)
Also add a release note and simplify the test by removing the garbage
prefix and suffix.
2024-01-03 17:49:35 +00:00
Ian Baker
04751a1cdb Format with google-java-format 2024-01-03 17:36:55 +00:00
datdoantelus
a58ddcfa23 Update test comment. 2024-01-03 17:36:55 +00:00
datdoantelus
28eb55c077 Use skipBits(2) instead of readBits for ignored bits.
Add singleServiceAndWindowDefinition_ignoreRowLock test to Cea708ParserTest
2024-01-03 17:36:55 +00:00
datdoantelus
8b39f7e287 Ignore rowLock and columnLock values as specified by ANSI/CTA-708-E S-2023 document (section 8.4.7) 2024-01-03 17:36:55 +00:00
Ian Baker
f935f59eaa Improve logging by including full stack trace, and add release note 2024-01-03 16:14:37 +00:00
Ian Baker
820278c564 Format with google-java-format 2024-01-03 15:49:43 +00:00
Tolriq
de772cfbf0 Catch errors and OOM when decoding ID3 frames.
Invalid frames have no impact on ExoPlayer ability to play the media and should not fail on errors.
Some tools can add 100Mb images in the tags that will trigger recoverable OOM with this fix.
2024-01-03 15:49:43 +00:00
samrobinson
c230414bd3 Set the default audio encoder bitrate in DefaultEncoderFactory.
Matches the approach for video frame rate.

PiperOrigin-RevId: 595385930
2024-01-03 06:37:16 -08:00
sheenachhabra
3ecf8732ba Add comments for skipping input files in Mp4Muxer tests
PiperOrigin-RevId: 595356012
2024-01-03 03:50:46 -08:00
ibaker
258ffa68b5 Add tests with out-of-order CEA-608 input buffers
The `Cea608DecoderTest` added here fails if re-ordering is removed from
`CeaDecoder`.

The `Cea608ParserTest` is added with `@Ignore` because there's currently
no re-ordering support in this part of the subtitle handling pipeline
(partly because there's no concept of 'current playback time', meaning
it's hard to know **when** to re-order).

PiperOrigin-RevId: 595320205
2024-01-03 00:45:31 -08:00
ibaker
7b450f0d0a Fix limit calculation to include offset in Cea6/708Parser.parse
PiperOrigin-RevId: 595007390
2024-01-01 20:22:27 -08:00
sheenachhabra
5149cc60ac Skip Mp4Muxer tests for unwanted input files
PiperOrigin-RevId: 594478028
2023-12-29 11:10:26 -08:00
bachinger
426bc94090 Don't set negative values to setWhen()
When the 'when' timer of the notification is disabled
`DefaultMediaNotificationProvider` may set `C.TIME_UNSET`
as the time. Users reported problems on some devices with
this and the docs ask for an event time that probably
shouldn't be a negative number.

This change sets `0L` instead of `C.TIME_UNSET` when the
timer is disabled.

Issue: androidx/media#903

#minor-release

PiperOrigin-RevId: 594451074
2023-12-29 06:56:02 -08:00
sheenachhabra
27ae6d974e Allow creating fragmented MP4 file via InAppMuxer
PiperOrigin-RevId: 594431665
2023-12-29 04:15:01 -08:00
sheenachhabra
e0257f403f Implement fragmented MP4 (fMP4) in the Mp4Muxer
Changes includes;
1. Public API to enable fMP4 and to pass fragment duration.
2. Added `FragmentedMp4Writer`.
3. Added logic to create fragments based on given fragment duration.
4. Write "moov" box only once in the beginning.
3. Add all the required boxes for current implementation.
4. Unit tests for all the new boxes.
5. E2E test for generating fMP4.

Note: The output file is un seek-able with this first implementation.
PiperOrigin-RevId: 594426486
2023-12-29 03:39:38 -08:00
sheenachhabra
b0e00a7d28 Fix ByteBuffer.array() warning
PiperOrigin-RevId: 594274620
2023-12-28 10:07:47 -08:00
sheenachhabra
1609928242 Throw exception when B-frames are written to Mp4Muxer
Mp4Muxer does not support out of order B-frames. Currently it
silently writes out of order B-frames, producing an invalid file (with
negative sample durations).

Although `Mp4Extractor` is somehow able to process this invalid file and
`Exoplayer` is able to play it but that is unexpected.

The `sample.mp4` test file contains B frames. Other test files does not
contain `H264 video + AAC audio` format hence created a new test file by
running `sample.mp4` via `Transformer` after applying some effects.

PiperOrigin-RevId: 594016144
2023-12-27 08:32:36 -08:00
tofunmi
0ab7bafa87 Migrate Gaussian Blur Effect to media3.
PiperOrigin-RevId: 593164068
2023-12-22 14:13:19 -08:00
tofunmi
e3056dacac Rename Mp4MetadataInfo to mp4Info
PiperOrigin-RevId: 593143940
2023-12-22 11:10:30 -08:00
tofunmi
4fc11a98a1 ignore dropSamplesBeforeFirstVideoSample in audio-only streams
PiperOrigin-RevId: 593126632
2023-12-22 08:54:29 -08:00
ibaker
64231ee079 Tweak SubtitleTranscodingTrackOutput.ensureSampleDataCapacity
This more closely matches the intended, documented behaviour of this
method. The previous implementation was incorrectly checking
`sampleDataEnd + newSampleSize`, but it's more correct to compare
`existingSampleDataLength + newSampleSize`, in order to handle the
case of non-zero `sampleDataStart`. We've already checked above whether
`newSampleSize` fits after `sampleDataEnd` without growing or
reshuffling the array, so no need to basically repeat that check.

In the case of handling one sample at a time, the previous
implementation resulted in `sampleData` growing by `sampleSize`
**every time** the pointers reached the end. With the new check for
`sampleDataStart == sampleDataEnd`, this is avoided by always writing
each new sample at the start of the array (because the previous sample
has already been consumed), meaning `sampleData` remains equal to
`sampleSize` without growing.

PiperOrigin-RevId: 593119927
2023-12-22 08:06:30 -08:00
tofunmi
7e12b9e15f Add rough progress updates in trim optimization
PiperOrigin-RevId: 593116025
2023-12-22 07:37:23 -08:00
tofunmi
1632f37d70 Transformer: Add api to drop audio samples before the first video frame
fix for Issue: androidx/media#829

Manual Testing: Viewed the transformer output file of previously problematic case in Exoplayer, Chrome, VLC, Quicktime and Safari and all showed the issue not to occur anymore. The newly produced output file can be found at https://github.com/androidx/media/assets/42352357/fdf105c1-9550-422f-b088-7900f655ac78

PiperOrigin-RevId: 593104752
2023-12-22 06:18:07 -08:00
ibaker
a496bbd777 Add parentheses to fix UnexpectedLoaderException message logic
These were missing in 5211ff0dc1

Without these, the `": null"` is still logged (but the `"Unexpected
IllegalStateException"` bit is not), because the **whole** string
concatenation is compared to null as the boolean condition of the
ternary, and this condition is always false (the result of a string
concatentation is never `null`) i.e. (with excess parentheses for
clarity):

```
(("Unexpected " + cause.getClass().getSimpleName() + cause.getMessage()) != null)
    ? ": " + cause.getMessage()
    : ""
```

Also add a test because obviously this isn't as simple as I'd thought.

PiperOrigin-RevId: 593079975
2023-12-22 03:42:42 -08:00
simakova
09bde8053d Update method description for setEffects in Composition
PiperOrigin-RevId: 593071081
2023-12-22 02:51:52 -08:00
andrewlewis
ef7842dd88 Fix typo
PiperOrigin-RevId: 593061829
2023-12-22 02:04:30 -08:00
andrewlewis
1845a4ae69 Expand operating rate workaround to T612 chipset
PiperOrigin-RevId: 592916187
2023-12-21 11:52:35 -08:00
ibaker
5211ff0dc1 Only append non-null exception messages in UnexpectedLoaderException
The previous code led me to misread this stack trace as a null pointer
exception, but it's really an index out of bounds exception:

```
Caused by: androidx.media3.exoplayer.upstream.Loader$UnexpectedLoaderException: Unexpected IllegalArgumentException: null
  at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:435)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
  at java.lang.Thread.run(Thread.java:1012)
Caused by: java.lang.IllegalArgumentException
  at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:40)
  at androidx.media3.common.util.ParsableByteArray.setPosition(ParsableByteArray.java:164)
  at androidx.media3.extractor.text.cea.Cea608Parser.parse(Cea608Parser.java:440)
```

PiperOrigin-RevId: 592876546
2023-12-21 09:09:41 -08:00
huangdarwin
0ed19937d3 Test: Remove setInputColorInfo from VFP test runner.
Instead, for input videos, use the colorInfo provided by the extractor. Similarly, for input images, use sRGB, the only color currently in use.

Textures do still need the input ColorInfo provided though.

PiperOrigin-RevId: 592875967
2023-12-21 09:06:55 -08:00
tofunmi
57d00fcca0 Rename crow_emulator_transformer_output
PiperOrigin-RevId: 592873355
2023-12-21 08:56:48 -08:00
Copybara-Service
966b5178b6 Merge pull request #914 from cemrich:366-forward-time-wrong
PiperOrigin-RevId: 592871532
2023-12-21 08:48:24 -08:00
andrewlewis
7d11ee7538 Optionally notify codec init in default decoder factory
Replace the event for notifying fallback to cover codec initialization in
general (but keeping the list of errors).

Add a flag to control whether to try non-primary codecs, with the same
documentation as the similar flag in ExoPlayer's renderer.

Make the class final as it shouldn't be necessary to subclass it.

PiperOrigin-RevId: 592869868
2023-12-21 08:42:44 -08:00
tofunmi
3081ceb179 Remove the matchInitializationData option from TransformerInternal
Essentially a manual revert of ba0724ca78

PiperOrigin-RevId: 592869049
2023-12-21 08:38:58 -08:00
ibaker
f36ab87b38 Fix DASH CEA-608 parsing during extraction
This is similar to the HLS fix in 770ca66fbc

Similar to HLS, the original problem here was **not** modifying the
`Format` for caption tracks
embedded into the video stream. I tried just updating the format in
both places, but that caused new failures because the new
('transcoded') format was then fed into `FragmentedMp4Extractor` as
part of `closedCaptionFormats`, which resulted in the CEA-608 data
being emitted from `FragmentedMp4Extractor` with the incorrect
`application/x-media3-cues` MIME type (but the bytes were actually
CEA-608), meaning the transcoding wrapper passed it through without
transcoding and decoding failed (because obviously CEA-608 bytes can't
be decoded by `CueDecoder` which is expecting a `Bundle` from
`CuesWithTiming.toBundle`.

To resolve this we keep track of the 'original' caption formats inside
`TrackGroupInfo`, so we can feed them into `FragmentedMp4Extractor`.
For all other usages in `DashMediaPeriod` we use the 'transcoded'
caption formats.

PiperOrigin-RevId: 592866262
2023-12-21 08:25:03 -08:00
sheenachhabra
60c8273521 Split Mp4Writer into an abstract class and DefaultMp4Writer
With fMP4 implementation there will be two writers `DefaultMp4Writer`
and `FragmentedMp4Writer`.

Changes includes:
1. Make Mp4Writer as an abstract class and keep only common functionality
into it.
2. Create a DefaultMp4Writer which contains existing logic to write MP4.
3. The fMP4 logic needs to access `pending sample buffer info` at various
places, so did refactoring to split List<Pair<BufferInfo, ByteBuffer>>
into two separate lists.

PiperOrigin-RevId: 592861398
2023-12-21 08:02:01 -08:00
huangdarwin
4774fa1a60 Test: Update tone mapping capabilities check output format.
Checking the output format's mime type may skip tests more often than we'd like,
because we may desire using a lower-spec output mimetype than what's passed in, if
based on the input's HDR mimetype value.

Therefore, update this output format to null, for tone-mapping tests

PiperOrigin-RevId: 592855713
2023-12-21 07:30:07 -08:00
Christine Coenen
793b27af3b Fix forward and rewind button texts cut off with material theme applied 2023-12-21 16:28:38 +01:00
rohks
639d380762 Fix the check for clang complier in FFmpeg build file
Also updated the `README` file to accurately specify the use of NDK r23c and the default setting `ANDROID_ABI=21` for NDK r26b.

PiperOrigin-RevId: 592845796
2023-12-21 06:35:37 -08:00
sheenachhabra
0b88e09a82 Improve durationsVuForStts method implementation
Issues with the current implementation
1. The implementation is unnecessarily complicated and can be
easily simplified.To make all the tracks start from the same time,
its only the first sample that require some timestamp adjustments
but the current implementation shifts all the timestamps. Since method
calculates the `sample duration`, shifting all the timestamps has no effect
as such.
2. The implementation always forced first sample to start at 0. But when we
want to use same method for `Fragmented MP4` then it will look inaccurate
as we will call this method for different `fragments` and each `fragment`
will not start from 0 presentation time. Although the output will be same
since this method returns `duration` and not the `timestamps`.
3. As per previous implementation if there is just one sample then
its duration is made equals to its presentation time, which looks incorrect.
With new changes, if a single sample is passed then its duration will always
be 0 irrespective of specified last sample duration behaviour.

PiperOrigin-RevId: 592826612
2023-12-21 04:45:43 -08:00
okunhardt
250fc80419 Move HttpEngineDataSource to the DataSource library.
Since HttpEngineDataSource does not have any external dependencies, it does not need to be in its own package.

PiperOrigin-RevId: 592623750
2023-12-20 12:10:36 -08:00
Googler
e25e497227 Make DefaultDecoderFactory try multiple formats before giving up
PiperOrigin-RevId: 592622544
2023-12-20 12:07:19 -08:00
tonihei
98519931e7 Split creation of AudioCapabilitiesReceiver from instance access
getAudioCapabilities currently creates the receiver and returns
the current capabilities. This is error-prone because the
capabilities are also available as a class field.

This can made cleaner by letting the method just create the receiver
and all access to the capabilities can be made via class field.

PiperOrigin-RevId: 592591590
2023-12-20 10:12:27 -08:00
ibaker
770ca66fbc Fix HLS CEA-608 when parsing during extraction
HLS distinguishes between 'subtitles' (WebVTT or TTML distributed in
separate files with their own playlist) and 'captions' (CEA-608 or 708,
distributed muxed into the video file).

The format transformation added in 7b762642db
only applies to subtitles and not captions. This change makes the same
transformation for caption formats.

This resolves an error like:

```
SampleQueueMappingException: Unable to bind a sample queue to TrackGroup with MIME type application/cea-608.
```

Also add two playback tests for HLS CEA-608, one that parses during
decoding (old way) and one during extraction (new way). Adding these
tests is what alerted me to this issue.

PiperOrigin-RevId: 592571284
2023-12-20 08:52:39 -08:00
tofunmi
4ce47ccdd3 MuxerWrapper: remove unnecessary volatiles
PiperOrigin-RevId: 592569012
2023-12-20 08:37:03 -08:00
tonihei
59f01388a6 Use platform constants for PCM 24/32 bit encoding
These constants are used when setting up the AudioTrack
and should match the definition in the platform.

PiperOrigin-RevId: 592564644
2023-12-20 08:16:35 -08:00
ibaker
8b9bf5e7cd Add DASH CEA-608 playback test
This was generated by combining the existing `ts/bbb_2500ms.ts` test
asset and a temporary `.srt` file using
https://cloud.google.com/transcoder/docs/how-to/captions-and-subtitles

This doesn't directly reproduce the problem fixed by
7ca26f898d,
because the CEA-608 subs are structured differently to the stream I
discovered the problem with (from Issue: androidx/media#887). However this test
does fail if that fix is reverted after
486230fbd7.

I'm also not able to repro the character duplication reported in
Issue: androidx/media#887 by just changing the manifest in this CL. I'm not yet
sure on the exact differences between the stream provided on GitHub
and this stream.

This stream does provide some regression protection, because it
currently fails with 'new' subtitle parsing
(`DashMediaSource.Factory.experimentalParseSubtitlesDuringExtraction(true)`),
though I'm not sure on the exact reason for that yet.

PiperOrigin-RevId: 592476328
2023-12-20 01:13:21 -08:00
bachinger
36b7a49d52 Use factory to create media period holders in the queue
This is a refactoring that allows the `MediaPeriodQueue` to
create media period holders without all the collaborators
being passes in to `enqueueNextMediaPeriodHolder(...)`.

The factory is presumably also helpful in unit test to
know whether and when exactly a holder is created in
the preloading process.

PiperOrigin-RevId: 592301400
2023-12-19 11:55:13 -08:00
ibaker
486230fbd7 Throw a clear error from Cea608/708Parser.parseToLegacySubtitle
This would have immediately exposed the mistakes fixed by
7ca26f898d

Issue: androidx/media#904
PiperOrigin-RevId: 592244943
2023-12-19 08:34:37 -08:00
ibaker
7ca26f898d Fix 'legacy' CEA-608 and CEA-708 to use the Decoder impls
We deliberately left `Cea608Decoder` and `Cea708Decoder` intact when
creating the respective `SubtitleParser` implementations (in
27caeb8038
and
94e45eb4ad
respectively).

However we didn't correctly change the behaviour of
`SubtitleDecoderFactory.DEFAULT` in order to use these 'legacy'
implementations. We firstly left the `Cea608Decoder` instantiation
**after** the `DefaultSubtitleParserFactory.supportsFormat()` check,
meaning that code would never be reached (since `supportsFormat` now
returns true for CEA-608). Then in the second change (which was supposed
to only be about CEA-708) we removed all instantiations of **both**
`Cea608Decoder` and `Cea708Decoder`. This change puts the decoder
instantiations back and moves them above the
`DefaultSubtitleParserFactory.supportsFormat` check, meaning they will
be used in preference to `DelegatingSubtitleDecoder`.

This resolves the immediately-disappearing subtitles tracked by
Issue: androidx/media#904.
PiperOrigin-RevId: 592217937
2023-12-19 06:32:36 -08:00
tonihei
e10f96d6a6 Make some DefaultRenderersFactory methods final
These methods are just setters and not meant to be overridden.

PiperOrigin-RevId: 592193857
2023-12-19 04:38:57 -08:00
ibaker
6853ffccae Add container to Format.toLogString
While investigating Issue: androidx/media#887 I naively assumed the CEA-608
captions were in a TS file, but they're actually in an MP4 (which is
possibly obvious given DASH only supports MP4). This change includes
container info in the `EventLogger` `tracks` output.

PiperOrigin-RevId: 592192752
2023-12-19 04:33:58 -08:00
christosts
3bb233a911 Misc clean-ups in ImageRenderer
Add some missing empty lines and use checkStateNotNull() everywhere.

PiperOrigin-RevId: 592173300
2023-12-19 03:06:29 -08:00
tofunmi
087c07e596 Rename OptimizationResult options
Rename options to be more accurate of what they capture

PiperOrigin-RevId: 592155563
2023-12-19 01:50:40 -08:00
tonihei
e5aa69237e Use CRYPTO_ASYNC mode for video from API 34
This allows us to remove the additional thread we create
for asynchronous buffer queuing and use a synchronized
queueing approach again.

This API looks strictly beneficial on all tested devices,
but since this code path in MediaCodec has not been used
widely, we leave an opt-out flag for now.

PiperOrigin-RevId: 591867472
2023-12-18 05:32:40 -08:00
sheenachhabra
bb7aa2fb19 Update updateMetadataEntries() method javadoc
PiperOrigin-RevId: 591858840
2023-12-18 04:45:35 -08:00
tonihei
b7c3d83e51 Forward setParameters to AsynchronousMediaCodecBufferEnqueuer
The parameters may change the decoding behavior of the
following samples and it's suprising/wrong to apply them
from another thread where we can't guarantee from which
sample they apply.

PiperOrigin-RevId: 591808760
2023-12-18 00:37:21 -08:00
huangdarwin
236bad5597 Add access control modifier to variable.
PiperOrigin-RevId: 591302601
2023-12-15 10:57:53 -08:00
huangdarwin
bde56b6b43 Effect: Move inputColorInfo javadoc to registerInputStream.
Oops, I forgot to move these in the earlier change, http://<unknown commit>.

PiperOrigin-RevId: 591300964
2023-12-15 10:52:22 -08:00
sheenachhabra
1303bfbc7c Set language field when creating MediaFormat from a Format
Currently Transformer simply omits the language field from the input
media file when writing data to the output file.

PiperOrigin-RevId: 591287481
2023-12-15 10:06:38 -08:00
huangdarwin
7579693739 Effect: Move VideoFrameProcessor inputColorInfo interface to FrameInfo.
Move `inputColorInfo` from `VideoFrameProcessor`'s `Factory.create` to `FrameInfo`,
input via `registerInputStream`.

Also, manually tested on exoplayer demo that setVideoEffects still works.

PiperOrigin-RevId: 591273545
2023-12-15 09:17:39 -08:00
tonihei
f47c4e33ec Remove experimental synchronization on async buffer queuing
PiperOrigin-RevId: 591273508
2023-12-15 09:14:52 -08:00
bachinger
29d7dd1ec9 Keep selections for preloaded adaptive sample streams
When an adaptive source has been preloaded, the track selection
of the player should possibly not override the preloaded selection
to avoid discarding preloaded data.

PiperOrigin-RevId: 591256334
2023-12-15 08:09:22 -08:00
lpribanic
1096ae9145 Write empty image byte arrays to sample queue
The number of empty image byte arrays written is one less than the
total number of tiles in the image. The empty byte arrays act as
placeholders for individual tiles inside ImageRenderer.

PiperOrigin-RevId: 591231432
2023-12-15 06:18:54 -08:00
claincly
7e65cce967 Fix first frame not force rendered
This is because currently

1. Player sets a surfaceView to render to
2. Player intializes the renderer
3. MCVR initializes the VideoSinkProvider, by extension VideoGraph

But when 1 happens, MCVR doesn't set the surfaceView on the VideoGraph because
it's not initialized. Consequently after VideoGraph is initialized, it doesn't
have a surface to render to, and thus dropping the first a few frames.

Also adds a test for first frame to verify the correct first frame is rendered.

PiperOrigin-RevId: 591228174
2023-12-15 06:01:12 -08:00
christosts
7df3e9e779 Integrate ImageRenderer with composition preview
PiperOrigin-RevId: 591218013
2023-12-15 05:01:50 -08:00
rohks
7f6596bab2 Fix sending negative bufferedDurationUs to CmcdData.Factory
When track is changed during playback, `playbackPositionUs` may be in middle of a chunk and `loadPositionUs` should be the start of that chunk. In this situation `loadPositionUs` can be less than the current `playbackPositionUs`, resulting into negative `bufferedDurationUs`. It translates to having no buffer and hence we should send `0` for `bufferedDurationUs` when creating new instances of `CmcdData.Factory`.

Issue: androidx/media#888

#minor-release

PiperOrigin-RevId: 591099785
2023-12-14 17:39:41 -08:00
huangdarwin
0baf777c96 Effect: Release FinalShaderProgramWrapper when releasing DVFP.
Otherwise, there's a memory leak of ~30MB, as this is never released.

This likely used to be considered released as part of what now became
`intermediateGlShaderPrograms`, but its release was missed after we split
`finalShaderProgramWrapper` out from the larger glShaderProgram list.

PiperOrigin-RevId: 590954785
2023-12-14 08:59:38 -08:00
ibaker
7399b7c5ce Remove @Beta comments that should have been removed in 0a016b59f7
PiperOrigin-RevId: 590940876
2023-12-14 08:04:23 -08:00
huangdarwin
97e9ed3f7b Effect: Move inputColorInfo usage to registerInputStream.
To prepare to move `inputColorInfo` from `VFP.Factory.create` to
`VFP.registerInputStream`, move all usage of `inputColorInfo` to be *after*
`registerInputStream`.

To do this, defer creation of `externalShaderProgram` instances, which require
`inputColorInfo`. However, we must still initialize `InputSwitcher` and OpenGL
ES 3.0 contexts in the VFP create, to create and request the input surface from
ExternalTextureManager.

PiperOrigin-RevId: 590937251
2023-12-14 07:50:04 -08:00
tonihei
01578780a6 Use different wraparound assumptions for duration readers
The timestamp adjuster also estimates the number of wraparounds
of the 90Khz TS timestamp. It does that by assuming that a new
timestamp is always close to the previous one (in either direction).

This logic doesn't always work for duration estimates because the
timestamp at the end of the media is not close to the one at the
beginning and it may also never be less than the one at the beginning.

This can be fixed by introducing a new estimation model that assumes
the new timestamp is strictly greater than the previous one without
making the assumption that it has to be close to it.

Issue: androidx/media#855

#minor-release

PiperOrigin-RevId: 590936953
2023-12-14 07:47:11 -08:00
bachinger
4974f960e7 Fix broadcasting notifyChildrenChanged for legacy controllers
When broadcasting a notifyChildrenChanged event, the task for legacy
controllers was sent to the broadcasting callback. This would
technically work, but because the subscription list is maintained
with specific controllers, the broadcast controller isn't subscribed
and hence the call wasn't executed.

This change calls the overloaded method for a specific controller
for each connected controller. Making sure (only) subscribed
controllers are notified.

Issue: androidx/media#644
PiperOrigin-RevId: 590904037
2023-12-14 05:20:39 -08:00
samrobinson
84471813b6 Update TODO internal bug number.
PiperOrigin-RevId: 590884408
2023-12-14 03:43:16 -08:00
Copybara-Service
ab296ef686 Merge pull request #753 from stevemayhew:p-fix-issue-9347
PiperOrigin-RevId: 590862514
2023-12-14 02:05:35 -08:00
lpribanic
d52772dff8 Set image tile counts to what the manifest says
Vertical and horizontal tile counts inside Format used by ImageRenderer
used to be required to equal 1 and were set to 1 regardless of what
the manifest said.

This change removes the above requirement and sets the tile counts to
the values from the manifest.

PiperOrigin-RevId: 590608353
2023-12-13 08:37:06 -08:00
lpribanic
b1d65f6d00 Amend javadocs for ImageOutput methods
PiperOrigin-RevId: 590607537
2023-12-13 08:33:55 -08:00
tonihei
7a702870f6 Add test and release notes 2023-12-13 14:48:35 +00:00
Steve Mayhew
64840159d6 Fixes issue #9347 - jump to live on seek to 0
Using 0 as the unset prepare position is the root cause of a number of issues,
as outliine in the ExoPlayer issue https://github.com/google/ExoPlayer/issues/7975

The premise of this fix is that once the prepare override is used (the initial call
to `selectTracks()`) it is never needed again, so simply invalidate it after use.
2023-12-13 14:48:35 +00:00
huangdarwin
a15dfd75be GL: Request OpenGL ES 3.0 context on API 29+, with fallback to 2.0.
Despite GL 3.0 not being required on API 29+, it is experimentally
determined to always be supported on our testing devices, on API 29+.

That said, still fall back to OpenGL 2.0 if 3.0 is not supported,
just in case.

PiperOrigin-RevId: 590569772
2023-12-13 06:01:08 -08:00
tofunmi
2fa5430417 Update OptimizationResult to specify different reasons for abandonment
PiperOrigin-RevId: 590530694
2023-12-13 03:22:55 -08:00
tofunmi
37def3679f Trim optimization: fallback on format mismatches
Manual testing: tested manually with pixel 4a

PiperOrigin-RevId: 590284361
2023-12-12 11:53:43 -08:00
sheenachhabra
ee1147ffe1 Remove unused method from TrackMetadataProvider interface
PiperOrigin-RevId: 590242668
2023-12-12 10:01:46 -08:00
Copybara-Service
f465efeefd Merge pull request #864 from v-novaltd:dsparano-exo129_2
PiperOrigin-RevId: 590234505
2023-12-12 09:37:41 -08:00
ibaker
5580b78b13 Fix VorbisComment.populateMediaMetadata key comparison to ignore case
Issue: androidx/media#876

#minor-release

PiperOrigin-RevId: 590215918
2023-12-12 08:43:02 -08:00
tonihei
8e33fbd536 Deprecate RAW_RESOURCE_SCHEME and util method
It's better to use the generic Android resource scheme which
is at least as powerful as our own one.

Issue: androidx/media#868
PiperOrigin-RevId: 590168919
2023-12-12 05:34:09 -08:00
Copybara-Service
27f437b65a Merge pull request #867 from equeim:ndk-r26
PiperOrigin-RevId: 590142275
2023-12-12 03:48:08 -08:00
sheenachhabra
0660b55ceb Move doInterleave() call to Mp4Writer
`Mp4Writer` receives all the tracks/samples and then it adds them to
corresponding `Track` object. As the `Track` object is primarily to hold
track specific data and `Mp4Writer` is responsible for interleaving related
logic, the call to `doInterleave` should be with `Mp4Writer`. The `Track`
object should just act like a data holding class.
This will also help in reusing `Track` class for fragmented MP4 implementation.

PiperOrigin-RevId: 590138002
2023-12-12 03:32:18 -08:00
Alexey Rochev
38b5b03ebb Fix error with NDK r26 2023-12-12 10:40:27 +00:00
Alexey Rochev
6874cc80cf ffmpeg: add support of specifying Android ABI version when building FFmpeg
Also add proper error message when using NDK that doesn't support requested ABI version.
2023-12-12 10:40:27 +00:00
samrobinson
90a0cbdf3d Fix handling of repeated EOS in SilenceSkippingAudioProcessor.
Issue: androidx/media#712
PiperOrigin-RevId: 589882412
2023-12-11 11:17:14 -08:00
ibaker
d356d88c4f Improve test dump output for role and selection flags
PiperOrigin-RevId: 589878576
2023-12-11 11:05:40 -08:00
bachinger
7f9b02080a Make the media notification controller marker key visible
An app that uses the `MediaSession` without a `MediaSessionService` may
want to connect a media notification controller on it own. To avoid apps
using a string literal, the key should be exposed as public.

PiperOrigin-RevId: 589875603
2023-12-11 10:57:00 -08:00
ibaker
3e5ef51594 Remove empty container/src/androidTest directory
PiperOrigin-RevId: 589870662
2023-12-11 10:41:51 -08:00
ibaker
5266c71b3a Fix handling of multiple HEVC tracks in JPEG motion photos
The MP4 data in JPEG motion photos can contain multiple `video/hevc` tracks, but only the first is at a playable frame rate while the others are low-fps, high-res tracks designed for specific use-cases (not direct video playback).

ExoPlayer currently selects the unplayable track by default, because it
has a higher resolution. This change introduces a flag to
`Mp4Extractor` that results in the first video track being marked as
`ROLE_FLAG_MAIN`, and all subsequent video tracks `ROLE_FLAG_ALTERNATE`
- this then results in the playable lower-res track being selected by
default.

PiperOrigin-RevId: 589832072
2023-12-11 08:31:13 -08:00
christosts
4c4c5f6a90 CompositionVideoSinkProvider: apply rotation effects per video
There is a bug in CompositionVideoSinkProvider where the rotation
effect is applied only on the first video and the same effect
is applied on subsequent videos without checking if the next
items in the playlist must be rotated.

The bug applies only on devices with API < 21.

PiperOrigin-RevId: 589823797
2023-12-11 07:56:50 -08:00
tonihei
eb2092bc8b Formatting fixes and release notes 2023-12-11 15:12:50 +00:00
Daniele Sparano
6f5187a34b Format to wake up PR 2023-12-11 14:28:11 +00:00
Daniele Sparano
4a17ca7600 Add basic h264 sample unit test to ensure correct sample flags, including key frames 2023-12-11 14:28:11 +00:00
Daniele Sparano
d4af13803a Fix H264 reader key frame detection 2023-12-11 14:28:11 +00:00
ibaker
379cb3ba54 Tighten the handling of validDataChannelTimeoutMs in Cea608Parser
This matches the documented requirements for this parameter.

PiperOrigin-RevId: 589799256
2023-12-11 06:05:53 -08:00
tonihei
00c7a9bcbb Keep playback thread blocked until next action with playUntilPosition
We currently pause playback to prevent further progress while the
app thread runs assertion and triggers additional actions. This is
not ideal because we do not actually want to pause playback in
almost all cases where this method used.

This can be improved by keeping the playback thread blocked and only
unblock it the next time the app thread waits for the player (either
via RobolectricUtil methods or by blocking the thread itself). To
add this automatic handling, this change introduces a new util class
for the tests that can keep the list of waiting threads statically
(because the access to this logic is spread across multiple independent
classes).

PiperOrigin-RevId: 589784204
2023-12-11 04:45:31 -08:00
tofunmi
00d5b6ec99 TransformerEndToEndTest: replace /*testId=*/ with variable
PiperOrigin-RevId: 589765715
2023-12-11 03:14:34 -08:00
huangdarwin
1ebe099685 Test: Make a method scope private.
For clarity.

PiperOrigin-RevId: 589758781
2023-12-11 02:43:25 -08:00
tianyifeng
b2a673d521 Move preload components to androidx.media3.exoplayer.source.preload
PiperOrigin-RevId: 589455479
2023-12-09 13:41:55 -08:00
ibaker
224dad3988 Remove AnalyticsListener as a public super-type of ExoHostedTest
Move the `AnalyticsListener` implementation to a private inner class.

This avoids polluting the public API of `ExoHostedTest`, especially as
it's designed to be extended.

PiperOrigin-RevId: 589188113
2023-12-08 11:13:18 -08:00
samrobinson
1471528176 Change AudioGraph #release to #reset, resetting internal state.
PiperOrigin-RevId: 589113634
2023-12-08 06:36:32 -08:00
tofunmi
c5c8e988e8 Abandon trim optimization when transcoding effects are set
PiperOrigin-RevId: 588839072
2023-12-07 10:39:38 -08:00
samrobinson
ab798659d9 Request specific AudioFormat from AudioGraphInputs on creation.
Also adds an alternate way to configure the AudioGraph.

Apps should no longer need to ensure that inputs have the same sample
rate.

PiperOrigin-RevId: 588747431
2023-12-07 04:42:03 -08:00
andrewlewis
c014ba9d5f Use API 33 constants for DV levels
PiperOrigin-RevId: 588740855
2023-12-07 04:15:19 -08:00
tofunmi
be5b29d92d Update ExportResult.OptimizationResult options
Added a new ABANDONED option so one can tell the difference between when the optimization has been requested but not applied vs not requested at all. also changed the ordering do better represent the hierarchy of failure modes

PiperOrigin-RevId: 588720513
2023-12-07 02:52:16 -08:00
tofunmi
cd346ca14d Transformer: Add support for transmuxing audio in trim optimization
PiperOrigin-RevId: 588711597
2023-12-07 02:18:21 -08:00
christosts
faeff17a4c Fix bug in CompositingVideoSinkProvider.isInitialized()
Fix a bug where CompositingVideoSinkProvider.isInitialized() returns
true even after releasing the CompositingVideoSinkProvider.

PiperOrigin-RevId: 588481744
2023-12-06 11:01:45 -08:00
christosts
073ee8a1d0 Move instantiation of CompositingVideoSinkProvider
This change moves the instantiation of the CompositingVideoSinkProvider
out of MediaCodecVideoRenderer so that the composition preview player can
re-use the CompositingVideoSinkProvider instance for driving the rendering of
images.

The main point of the change is the ownership of the
VideoFrameReleaseControl, which decides when a frame should be rendered
and so far was owned by the MediaCodecVideoRenderer. With this change,
in the context of composition preview, the VideoFrameReleaseControl
is no longer owned by MediaCodecVideoRenderer, but provided to it.

This way, the CompositingVideoSinkProvider instance, hence the
VideoFrameReleaseControl can be re-used to funnel images into the
video pipeline and render the pipeline from elsewhere (and not
MediaCodecVideoRenderer).

PiperOrigin-RevId: 588459007
2023-12-06 09:52:43 -08:00
ibaker
bd19953ac9 Split Cue.toBundle into serializable and binder-based variants
The serializable form is used when we need to serialize the result into
bytes in the sample queue. The binder-based (ultimately
filedescriptor-based) form is used for
session/controller IPC, in order to avoid sending the bitmap bytes over
the IPC.

Issue: androidx/media#836

#minor-release

PiperOrigin-RevId: 588420836
2023-12-06 07:35:34 -08:00
huangdarwin
a98052b3fc Test: Mention expected bitmap in assertion message.
This helped me debug which bitmap was actually failing. Otherwise, you need to clear all bitmaps on the device relating to this test, then adb pull all bitmaps and see the last uploaded one to see, which is much more confusing.

PiperOrigin-RevId: 588374081
2023-12-06 04:17:37 -08:00
samrobinson
757585ce6d Release MuxerWrapper in every test (in an @After block).
PiperOrigin-RevId: 588110566
2023-12-05 10:13:27 -08:00
ibaker
6360082b87 Add extractor and playback tests for Pixel JPEG motion photo
This image has two video tracks in the MP4 data, one is a 'real' video
which we want to play by default, and the other is a low-fps video track
which isn't intended to be directly played, it's encoded in HEVC for
compression and decoding efficiency.

This test demonstrates ExoPlayer's current behaviour default extraction
and playback, which results in selecting the high-res, low-fps track
(actually single sample in this example), instead of playing the actual
video.

PiperOrigin-RevId: 588068908
2023-12-05 07:46:19 -08:00
ibaker
d4fe3fe318 Add Robolectric e2e test support for HEVC content
PiperOrigin-RevId: 588055594
2023-12-05 06:52:39 -08:00
Googler
e0f1783a54 Fix nullability issue in MediaControllerImplLegacy
PiperOrigin-RevId: 588035411
2023-12-05 05:24:57 -08:00
tofunmi
b0e2fd78af Use getter for presentationDurationUs
Delays failures to when the field is used in preview code, so that exports in the demo app can continue succeed

PiperOrigin-RevId: 588022569
2023-12-05 04:43:57 -08:00
michaelkatz
d1e38abf93 Limit processing Opus decode-only frames by seek-preroll in offload
As Opus decoders skip some bytes prior to playback during a seek, the renderer for bypass playback should send samples to the decoder even if they would be decode-only. However, the renderer should not send samples with time preceding that range. This change adds that constraint.

#minor-release

PiperOrigin-RevId: 588014983
2023-12-05 03:44:37 -08:00
tonihei
b1541b096f Map VORBIS channel layout to Android layout
Both the extension OPUS decoder and the OMX/C2 MediaCodec
implementations for OPUS and VORBIS decode into the channel
layout defined by VORBIS. See
https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-140001.2.3

While this is technically correct for a stand-alone OPUS or VORBIS
decoder, it doesn't match the channel layout expected by Android.
See https://developer.android.com/reference/android/media/AudioFormat#channelMask

The fix is to apply the channel mapping after decoding if needed.
Also add e2e tests with audio dumps for the extension renderer,
including a new 5.1 channel test file.

Issue: google/ExoPlayer#8396

#minor-release

PiperOrigin-RevId: 588004832
2023-12-05 03:01:32 -08:00
ibaker
913f6da083 MCR: Ensure mediaCrypto and codec are atomically non-null
`mediaCrypto` is initialized before `codec` in
`maybeInitCodecOrBypass`. Before this change, it was possible for
`maybeInitCodecOrBypass` to complete with `mediaCrypto != null` and
`codec == null`, in particular if it was run as part of clearing the
player surface (since in that case, no video codec is initialized).
This inconsistent state then causes issues during a later invocation of
`maybeInitCodecOrBypass`, when `mediaCrypto` is still non-null, and
`mediaCryptoRequiresSecureDecoder = true`, but the
content has been changed to unencrypted with no associated DRM session.
This results in a playback error, because a secure decoder is
initialized but there's no DRM session available to work with it.

This change ensures that when `maybeInitCodecOrBypass` completes,
either both `mediaCrypto != null` and `codec != null` (i.e. codec
initialization was completed) or `mediaCrypto == null` and
`codec == null` (i.e. codec initialization was not completed). We also
ensure that when nulling out `mediaCrypto` we also set
`maybeInitCodecOrBypass = false`. A later change should be able to
demote `maybeInitCodecOrBypass` from a field to a local in order to
remove any risk of that part of state becoming out of sync. This
resolves the issue, because during the second invocation of
`maybeInitCodecOrBypass` an insecure decoder is now (correctly)
initialized and the unencrypted content is successfully played.

#minor-release

PiperOrigin-RevId: 587713911
2023-12-04 07:32:52 -08:00
tianyifeng
f32bdf86bc Set playerId when preloading to make the PreloadMediaSource reusable
We set the `playerId` only in the constructor currently. But in the case of this source doesn't have `preload` called, the player's release of this source will set the playerId to `null`, which makes this source un-reusable with a null `playerId`.

PiperOrigin-RevId: 587698214
2023-12-04 06:31:11 -08:00
tofunmi
88b15e73b2 fix: update TransformerPauseResumeTest to use isRunningOnEmulator
#minor-release

PiperOrigin-RevId: 586964536
2023-12-01 04:01:52 -08:00
samrobinson
fe272d3c49 Follow practices/guidance in Muxer tests around:
* @Before and @After usage. [1].
* static fields before non-static. [2].

@Before method should typically be paired with an @After method,
focused on ensuring that the component is released regardless of what
the test does.

In tests, inlining final class variables is preferrable [1]. In general
things like the file path should be part of the test (the @Rule means
we don't need before/after) if only used once.

Statically importing values and using them directly is preferable to
having a variable declared as non-final that's effectively final,
because from a readability perspective someone can see (the caps) that
the value is final static and immutable, so doesn't have to check if
it's changed/reassigned.

PiperOrigin-RevId: 586697887
2023-11-30 09:29:18 -08:00
tofunmi
5b3491082b fix: make BitmapOverlay constructor public
So developers can subclass it
#minor-release

PiperOrigin-RevId: 586638134
2023-11-30 04:54:06 -08:00
huangdarwin
a2a4ef5f40 Effect: Fix overlay alpha handling.
Before, a translucent overlay over an opaque video would result in a
translucent output. This is not consistent with physical properties of light
(if putting a translucent object in front of an opaque object, you can't see
behind the opaque object).

Using the mixing properties from DefaultVideoCompositor.

PiperOrigin-RevId: 586636275
2023-11-30 04:43:21 -08:00
ibaker
28c210686f Don't include null text or bitmaps in Cue.toBundle()
`fromBundle` doesn't distinguish between `FIELD_BITMAP` and `FIELD_TEXT`
being present with a null value, or being absent, so we might as well
avoid including them when the value is null.

I've separated this from a later change to add
`Cue.toSerializableBundle` which will also skip setting a bitmap value
into the `Bundle` if `this.bitmap == null`. This is partly because it
results in changes to a lot of extractor test dump files, and it's
easier to review that as a separate change.

PiperOrigin-RevId: 586626141
2023-11-30 03:58:33 -08:00
jbibik
03564fdbc6 Promote method to control subtitle parsing to MediaSource.Factory
PiperOrigin-RevId: 586361222
2023-11-29 09:12:04 -08:00
jbibik
8a8b875c72 Add experimental opt-in to parse SS subtitles during extraction
PiperOrigin-RevId: 586331888
2023-11-29 07:14:14 -08:00
huangdarwin
9add30e582 Effect: Use Bitmap generation ID to detect changes.
This is much simpler than using protected methods that signal updates in
bitmaps.

PiperOrigin-RevId: 586295312
2023-11-29 04:29:49 -08:00
christosts
3d1d8f4439 Exit early progressive loads if the load task is canceled
Add an check when loading progressive media in case the load
is canceled. If the player is released very early, the progressive
media period may carry on with the initial loading unnecessarily.

PiperOrigin-RevId: 586288385
2023-11-29 04:01:06 -08:00
andrewlewis
3204313f13 Remove debug logging
This case is expected and we aren't trying to gather information about it
currently, so we shouldn't log.

PiperOrigin-RevId: 586015969
2023-11-28 10:01:37 -08:00
huangdarwin
e5ef31b277 Effect: Avoid allocating bitmaps and textures in Overlays.
In `TextOverlay` and `DrawableOverlay`, treat `Bitmap` as a buffer, where we
allocate it rarely and reuse it as long as possible before making a new one.

In `BitmapOverlay`, avoid allocating GL textures too often as well.

Strongly reduces allocations and memory usage growth (saving ~100-150 MB on 4k60fps
at high end), at the cost of more code complexity and low-end using 70MB more, on
1/1 comparisons.

PiperOrigin-RevId: 585990602
2023-11-28 08:44:30 -08:00
huangdarwin
eb01c3f440 Add exception message for MediaCodecRenderer exception.
Otherwise, it's difficult to differentiate between different sources of ERROR_CODE_DECODING_FORMAT_UNSUPPORTED.

PiperOrigin-RevId: 585966636
2023-11-28 07:15:52 -08:00
andrewlewis
b8c1e22389 Clarify threading-related comments in texture manager
In `ExternalTextureManager` fields are accessed from the GL thread and the class needs to be constructed on the GL thread.

Also visibly document threading requirement in the parent class.

PiperOrigin-RevId: 585941284
2023-11-28 05:30:15 -08:00
andrewlewis
7bbf72c202 Use JsonWriter to convert to JSON
Before the change the output was JSON-like but not valid because it had commas
after objects/arrays.

PiperOrigin-RevId: 585929345
2023-11-28 04:39:50 -08:00
tofunmi
b598c96c2f Update emulator device names
PiperOrigin-RevId: 585682881
2023-11-27 10:02:02 -08:00
andrewlewis
1c7c06999e Turn off debug trace logging by default
The issue that motivated adding this (frames unexpectedly being dropped by the
decoder) has been addressed, so we can turn off the logging to reduce
unnecessary allocations during transformation. We can easily turn on debug
logging in future as needed by setting `DebugTraceUtil.DEBUG = true`.

Also avoid allocations for string building at logging call sites by passing a
format string for extra info. Formatting the string now only happens when
debugging is turned on.

Tested manually by running transformations in the new state (DEBUG = false) and
with debugging turned on.

PiperOrigin-RevId: 585666349
2023-11-27 08:57:30 -08:00
ibaker
63062a9c10 Make CueSerializationTest more realistic
Serializing bitmap cues is currently broken, but this test is
incorrectly passing. This change makes two changes to introduce the same
failure (both changes are necessary, each one alone still passes):

1. Move from Robolectric to an instrumentation test.
2. Trigger the `Bitmap` to be serialized using a file descriptor, either
   by calling `Bitmap.asShared` in the test when constructing the `Cue`,
   or constructing the `Bitmap` from a 'real' image byte array instead a
   1x1 token image.

Issue: androidx/media#836
PiperOrigin-RevId: 585643486
2023-11-27 07:16:19 -08:00
Copybara-Service
5f27b18210 Merge pull request #837 from superjohan:fix/android-14-clearkey
PiperOrigin-RevId: 585639025
2023-11-27 06:56:45 -08:00
andrewlewis
e84a13fb54 Restrict operating rate workaround to SM8550
PiperOrigin-RevId: 585613041
2023-11-27 04:42:42 -08:00
andrewlewis
8b38b34b9f Work around codec frame rate issues in Redmi Note 9 Pro
The decoder and encoder won't accept high values for frame rate, so avoid
setting the key when configuring the decoder, and set a default value for the
encoder (where the key is required).

Also skip SSIM calculation for 4k, where the device lacks concurrent decoding
support.

PiperOrigin-RevId: 585604976
2023-11-27 04:05:44 -08:00
huangdarwin
c650f05234 Document TimestampWrapper not working with previewing.
Mentioned in https://github.com/androidx/media/issues/821

PiperOrigin-RevId: 585595948
2023-11-27 03:25:31 -08:00
Johan Halin
ad7b12e56f Apply workaround for Android 13 and newer
At least some Android 14 devices still have the same invalid URL issue
when using ClearKey DRM, so might as well apply the check to newer
versions of Android as well.
2023-11-27 10:41:47 +00:00
andrewlewis
ad40db4489 Avoid value close to overflow for KEY_OPERATING_RATE
Using `Integer.MAX_VALUE` risks causing arithmetic overflow in the codec
implementation.

Issue: androidx/media#810

#minor-release

PiperOrigin-RevId: 585104621
2023-11-24 08:57:59 -08:00
siroberts
bc36553e81 Expose isTrusted property of ControllerInfo.
PiperOrigin-RevId: 585052580
2023-11-24 03:37:33 -08:00
jbibik
c0ef5f6de4 Serialize media3 custom Spans for Cue encoding/decoding
PiperOrigin-RevId: 585028521
2023-11-24 01:32:40 -08:00
ibaker
479344d74e Fix typo in DashManifestParser
PiperOrigin-RevId: 585017285
2023-11-24 00:32:26 -08:00