Commit graph

3158 commits

Author SHA1 Message Date
Copybara-Service
bbfba27cac Merge pull request #594 from Tolriq:sink_format
PiperOrigin-RevId: 561611655
2023-08-31 04:08:54 -07:00
rohks
206a663dca Add functionality to transmit CMCD data using query parameters
Currently, we only support sending Common Media Client Data (CMCD) data through custom HTTP request headers, added capability to configure and transmit it as HTTP query parameters.

PiperOrigin-RevId: 561591246
2023-08-31 02:11:12 -07:00
ibaker
89f52dcb93 Rollback of 53b882d803
PiperOrigin-RevId: 561587445
2023-08-31 01:52:13 -07:00
rohks
3539b26e19 Add nullness annotations to SsMediaPeriod
#fixit

PiperOrigin-RevId: 561583025
2023-08-31 01:29:27 -07:00
ibaker
147cab89c3 Remove subtitle TODO from CapturingRenderersFactory
We're moving towards parsing subtitles before the `SampleQueue`, and we
already have subtitle playback assertions using `Player.Listener.onCues`
which is the most 'complete' place to assert subtitle output from. We
don't need to hook anything inside `TextRenderer`, so we can remove this
TODO.

PiperOrigin-RevId: 561418660
2023-08-30 12:38:48 -07:00
tonihei
ba4b96d2c1 Prevent forwarding of zero video size to Leanback
Leanback isn't prepared to handle these values and will throw
ArithmeticExceptions.

Issue: androidx/media#617
PiperOrigin-RevId: 561413740
2023-08-30 12:20:51 -07:00
Tolriq
4e93db4d0a Format passed to audioSink.configure is missing information.
AudioSink might require the identifying data and metadata to be able to take decisions.
2023-08-30 18:23:18 +01:00
ibaker
53b882d803 Add subsampleOffsetUs to Format.toLogString
This will help with debugging subtitle sync issues as it will show up in
track selection details emitted to logcat by `EventLogger`.

PiperOrigin-RevId: 561355836
2023-08-30 09:03:42 -07:00
ibaker
64792dfda2 Rollback of 25253698bc
PiperOrigin-RevId: 561352515
2023-08-30 08:53:49 -07:00
sheenachhabra
b37e37aa3c Pass MuxerWrapper to the TransformerInternal
For pause/resume feature, same MuxerWrapper needs to be
used across intermediate exports. So pass the MuxerWrapper
from Transformer.java
More specifically, when resume() is called
1. Create a MuxerWrapper and remux the previous
video samples (Export 1).
2. User the same MuxerWrapper and start processing
remaining video samples (Export 2).

PiperOrigin-RevId: 561325867
2023-08-30 06:45:39 -07:00
michaelkatz
cf3fd1f4dd Added condition to RtspMediaPeriod.isLoading to check RtspState
If a RtspMediaSource is used within a combining source like MergingMediaSource, then it could become stuck in a loop of calling continueLoading when the source has not actually started loading yet.

Issue: androidx/media#577
PiperOrigin-RevId: 561322900
2023-08-30 06:29:53 -07:00
ibaker
b4ee896847 Remove some spurious } from Player javadoc
Also use `@linkplain` in more places: If I was already touching a javadoc
block, I switched to `@linkplain` throughout the whole block.

PiperOrigin-RevId: 561320273
2023-08-30 06:16:37 -07:00
tofunmi
6a5d5aef92 Add limited ImageRenderer
At this stage, the image renderer is designed to be able to render one input stream containing one sample. The renderer supports seeking inside that stream, which is a no-op because the stream comprises for exactly one sample.

PiperOrigin-RevId: 561307859
2023-08-30 05:15:38 -07:00
ibaker
9ee45fc938 Add TrackOutput field in PngExtractor
Also re-jig some `checkNotNull` calls to satisfy the nullness checker
(remove an unneeded one, and add a new one).

Follow-up to discussion in bb214b19f9

PiperOrigin-RevId: 561295724
2023-08-30 04:08:56 -07:00
claincly
b466b06ace Split out VideoGraph and VideoFrameProcessingWrapper
More specifically, this CL

- Defines a VideoGraph interface
  - Adds a factory method, to hide the constructors
  - Separate out an initialize method that does the real work (following that constructor should not do real work)
  - The VideoGraph takes in a list of composition effects. For now, we only use the `Presentation`, if there is one in the list. This means we can take any number of Presentations now, but only the first one will be used.
- Moves the VideoFrameProcessingWrapper to its own file

PiperOrigin-RevId: 561059653
2023-08-29 09:54:43 -07:00
tofunmi
bb214b19f9 Sample metadata in the png extractor
PiperOrigin-RevId: 561049410
2023-08-29 09:17:05 -07:00
tianyifeng
32e367fabf Set groupKey for media notification in DefaultMediaNotificationProvider
By doing so, the media notification sent from DefaultMediaNotificationProvider won't be auto-grouped (https://developer.android.com/develop/ui/views/notifications/group#automatic-grouping) with the regular notifications sent from the other places in the app. As the result, when the customer attempts to dismiss the regular notifications (by swiping away), the media notification won't be dismissed as being in the same group.

Issue: androidx/media#549
PiperOrigin-RevId: 561047571
2023-08-29 09:10:12 -07:00
tianyifeng
5a6b88d751 De-flake MediaSessionCompatCallbackWithMediaControllerTest.onStop test
The `controller.prepare()` call can trigger the callback `onPrepare()`, however, this will count down the latch by one before the callback `onStop()` is triggered.

PiperOrigin-RevId: 560988209
2023-08-29 04:28:16 -07:00
tofunmi
667103f2bd Fix SequenceAssetLoader signalling End Of Video Input twice
PiperOrigin-RevId: 560170216
2023-08-25 12:46:50 -07:00
huangdarwin
99ae44efa4 Overlays: Create OverlayMatrixProvider.
Split this logic out from `OverlayShaderProgram`, to share logic like
this with `VideoCompositor`.

Also, remove the unused `context` parameter from `OverlayShaderProgram`

`VideoCompositor` will reuse `OverlayMatrixProvider`'s storing of the matrices,
though it may generate the matrices differently, since `OverlayShaderProgram`
applies this in the fragment shader, whereas `VideoCompositor` applies this in the
vertex shader.

PiperOrigin-RevId: 560142164
2023-08-25 11:05:03 -07:00
huangdarwin
6da70c842e Overlay: Rename alpha to alphaScale.
The actual logic scales the alpha, instead of setting it, so rename
this to what it does.

We would also prefer alpha to be scaled here, to not lose alpha information
(for example for PNGs with transparent regions).

PiperOrigin-RevId: 560121708
2023-08-25 10:00:50 -07:00
sheenachhabra
1f013afea8 Add Mp4ExtractorWrapper into Transformer module
For pause and resume feature we need to find the timestamp of
the last sync sample in an MP4 file. The Mp4ExtractorWrapper
provides an easy to use API on top of Mp4Extractor.

PiperOrigin-RevId: 560113112
2023-08-25 09:29:28 -07:00
tianyifeng
86f919e7d4 Fix flaky tests in PlaylistPlaybackTest
If the loading thread of the second media item progresses slow, then when the playback of the first item approaches to the end before the second item is prepared, the stream will be set "final" and the renderers are disabled during such transition. In this case, `AudioSink.handleDiscontinuity` never gets called, which introduces the flakiness into the `test_bypassOffThenOn`.

Similarly for `test_subtitle`, if the loading thread of extracting the subtitle is slow, then when the renderer attempts to queue the input buffer, the data hasn't been available in the stream. And when the extracting finished, the renderer already advanced to the time after the subtitle end time.

To de-flake the tests, we have to make sure that the period of the second item has fully loaded before starting the playback.

PiperOrigin-RevId: 560080119
2023-08-25 07:04:26 -07:00
huangdarwin
dccbc7e459 ExoPlayer: Remove tone-mapping workaround.
PiperOrigin-RevId: 560069496
2023-08-25 06:07:44 -07:00
huangdarwin
9810d6ae92 Overlay: Improve alpha documentation and extract reused variables
PiperOrigin-RevId: 560064663
2023-08-25 05:43:02 -07:00
huangdarwin
01689ba2ec Compositor: Queue bitmaps by timestamp.
This allows us to manage timestamps in one list, instead of both via
offsets/framerates, and a list.

PiperOrigin-RevId: 560053888
2023-08-25 04:44:48 -07:00
tonihei
7899241daf Fix period indexing for multi-period DASH DAI live streams
The period index was calculated relative to the contentTimeline
of the DAI stream, but then used with the full timeline of the
player. This means it currently only works when the stream is the
only or first item in the playlist.

Issue: androidx/media#571
PiperOrigin-RevId: 560023412
2023-08-25 02:05:46 -07:00
claincly
feae0245b9 Remove the textureReleaseCallback from textureOutputListerner
PiperOrigin-RevId: 559817280
2023-08-24 11:33:11 -07:00
claincly
28fd43617e Factor out VideoFrameProcessor logic
PiperOrigin-RevId: 559780905
2023-08-24 09:42:21 -07:00
huangdarwin
cd0b7d9e29 Transformer: Document not supporting setTextureOutput on VFP Factory.
PiperOrigin-RevId: 559760003
2023-08-24 08:25:33 -07:00
claincly
8c1bedb483 Remove unnecessary types in lambda
PiperOrigin-RevId: 559745329
2023-08-24 07:21:52 -07:00
sheenachhabra
a6f9fefb88 Add setSequences method to Composition.Builder
For pause and resume feature, we need to buildUpon the
original composition and then set the modified sequences.

PiperOrigin-RevId: 559723694
2023-08-24 05:35:06 -07:00
claincly
930e538aca Allow re-configuring DVFP in a (mostly) non-blocking manner
The first call to method `registerInputStream` doesn't block.

Later successive calls to the method blocks until the previous register call
finishes.

PiperOrigin-RevId: 559694490
2023-08-24 02:55:54 -07:00
rohks
80495ddf9c Add nullness annotations to Representation
#fixit

PiperOrigin-RevId: 559690973
2023-08-24 02:38:11 -07:00
sheenachhabra
5276f797e5 Add buildUpon() and setMediaItem() method in EditedMediaItem
For pause/resume feature we need to build upon original EditedMediaItem
and set a modified MediaItem to it.

PiperOrigin-RevId: 559484129
2023-08-24 09:32:09 +01:00
rohks
e2ee8ae1b0 Add nullness annotations to DefaultDashChunkSource
#fixit

PiperOrigin-RevId: 559405572
2023-08-24 09:28:36 +01:00
rohks
2ad37d2e37 Add nullness annotations to MediaPeriodQueue
#fixit

PiperOrigin-RevId: 559395027
2023-08-24 09:27:43 +01:00
tofunmi
404a259295 Remove queueInputBitmap by framerate and duration interfaces
PiperOrigin-RevId: 559384011
2023-08-24 09:25:49 +01:00
samrobinson
64c2406c1e Use TestUtil value for frame count.
PiperOrigin-RevId: 559380901
2023-08-24 09:21:32 +01:00
rohks
bfa32b3998 Add nullness annotations to MediaCodeAudioRenderer
#fixit

PiperOrigin-RevId: 559175846
2023-08-24 09:19:34 +01:00
samrobinson
ff39726368 Remove redundant "aac" tag for dump file.
PiperOrigin-RevId: 559147235
2023-08-24 09:17:32 +01:00
ibaker
9533f5cd1c RtspMediaPeriod: Use a new ExtractorOutput for each SampleQueue
This removes concurrent access from `rtspLoaderWrappers`. Previously
there was a race between the playback thread clearing & re-adding
entries to this list in `retryWithRtpTcp()`, and the loading thread
accessing the entries in `InternalListener.track()` (implemented from
`ExtractorOutput`).

This change means each `ExtractorOutputImpl` uses exactly one
`SampleQueue` for its one `TrackOutput`. When the `RtspLoaderWrapper`
instances are replaced in `retryWithRtpTcp()`, any stale instances will
only be able to access their own (also stale) `SampleQueue` instances
(vs before, where the stale `ExtractorOutput` could accidentally access
'new' `SampleQueue` instances via the `rtspLoaderWrappers` field).

As well as fixing a race condition in the prod code, this also de-flakes
`RtspPlaybackTest`.

#minor-release

PiperOrigin-RevId: 559130479
2023-08-24 09:16:36 +01:00
christosts
398809e4e2 Replace Queue<Long> with a queue for long primitives
Replace Queue<Long> with LongArrayQueue which provides queue semantics
for long primitives. LongArrayQueue is forked from IntArrayQueue which
in turn was forked from Androidx CircularIntArray.

IntArrayQueue is deleted and we now use CircularIntArray directly from
Androidx Collection.

PiperOrigin-RevId: 559129744
2023-08-24 09:14:44 +01:00
tonihei
36084eef05 Split test_session_current instrumentation tests by file
PiperOrigin-RevId: 559089467
2023-08-22 15:50:29 +01:00
tonihei
a00bef9e51 Fix MediaControllerSurfaceSizeChangeTest
There are multiple problems in the test:
 1. The controller must run on the main thread due to the Player
    interface requirement to add surfaces on the thread their
    callbacks will be called on.
 2. This test made assertions about an expected callback that
    aren't true. Setting an anonymous surface will not result in
    a size change callback because the size stays unknown.
    But clearing a surface (even if never set) ensures the returned
    size is guaranteed to be zero.

PiperOrigin-RevId: 559080905
2023-08-22 15:49:08 +01:00
samrobinson
2309fc5edb Add AudioGraph test verifying silence outputs correct amount of bytes.
PiperOrigin-RevId: 559072505
2023-08-22 15:47:54 +01:00
samrobinson
b7782b5119 Inline ProgressHolder and ArgumentCaptor test class variables.
PiperOrigin-RevId: 559069221
2023-08-22 15:46:13 +01:00
rohks
0541e1953c Add nullness annotations to SampleQueue
#fixit

PiperOrigin-RevId: 559058211
2023-08-22 15:44:50 +01:00
samrobinson
7346029f72 Use TemporaryFolder @Rule for automated tear down of temp files.
Reduce @Before specific logic, following java dev guide.

PiperOrigin-RevId: 558825609
2023-08-22 15:42:07 +01:00
samrobinson
ae7667783c Split dump file directories based on input file name.
Remove old unused dump files.

PiperOrigin-RevId: 558820926
2023-08-22 15:39:28 +01:00
huangdarwin
350b394596 Compositor: Add tests for 1, 3, and 5 inputs.
With this, Compositor now handles an arbitrary number of inputs!

PiperOrigin-RevId: 558813361
2023-08-22 15:38:05 +01:00
tonihei
fddb09be20 Internal change
PiperOrigin-RevId: 558799565
2023-08-22 15:36:56 +01:00
tianyifeng
15f4ff7487 Remove asserts inside callbacks
PiperOrigin-RevId: 558791481
2023-08-22 15:35:33 +01:00
rohks
91764baa90 Add nullness annotations to SimpleCache
#fixit

PiperOrigin-RevId: 558783228
2023-08-22 15:33:08 +01:00
christosts
4fad529433 Rollback of f5a6ecdda1
PiperOrigin-RevId: 558779068
2023-08-22 15:31:43 +01:00
huangdarwin
59744fe788 Effect: Minimize calls to configure in default impl.
In case BaseShaderProgram implementations have an expensive
configure() call, we can only call configure() when absolutely necessary,
aka when the input size changes.

To reduce scope, this doesn't reduce the amount of
configure() calls that may be made outside this class, in
DefaultFrameDroppingShaderProgram and
FinalShaderProgramWrapper.

PiperOrigin-RevId: 558754314
2023-08-22 15:30:23 +01:00
huangdarwin
153740386a Compositor: Use lists for inputs instead of instances.
This makes the code a bit more flexible for when we allow for an arbitrary
number of inputs.

PiperOrigin-RevId: 558740536
2023-08-22 15:29:04 +01:00
tofunmi
521c210fd1 Create Timestamp iterator
PiperOrigin-RevId: 558738035
2023-08-22 15:27:42 +01:00
samrobinson
352916b182 Add parameterized tests for generating silence.
Covers all permutations&combinations of:
* AV asset.
* AV asset with effects.
* AV asset with silence.
* AV asset with silence and effects.

Note that the video is not relevant (therefore transmuxed), but is
needed for silence generation.

PiperOrigin-RevId: 558734593
2023-08-22 15:26:19 +01:00
rohks
e0d3cad8bb Add fields next object request (nor) and next range request (nrr)
Added this CMCD-Request fields to Common Media Client Data (CMCD) logging.

PiperOrigin-RevId: 558317146
2023-08-22 15:25:03 +01:00
rohks
b6b16b2895 Sort Common Media Client Data keys to reduce fingerprinting surface
Based on the Common Media Client Data (CMCD) specification key-value pairs should be sequenced in alphabetical order of the key name in order to reduce the fingerprinting surface exposed by the player.

PiperOrigin-RevId: 558296264
2023-08-22 15:23:33 +01:00
rohks
afb8d6c9e2 Make custom data adhere to Common Media Client Data(CMCD) specification
Added more comprehensive Javadoc around setting custom data and verification on key format.

Changed adding custom data as a `String` to `List<String>`. This would enable us to sort all keys to reduce the fingerprinting surface.

PiperOrigin-RevId: 558291240
2023-08-22 15:22:08 +01:00
samturci
6566387f70 Add a waveform audio buffer sink.
This can be used together with TeeAudioProcessor to draw an audio waveform.

PiperOrigin-RevId: 558213516
2023-08-22 15:20:47 +01:00
samrobinson
54797d4cc5 Use @Rule and other util to simplify ParameterizedAudioExportTest.
PiperOrigin-RevId: 558168355
2023-08-22 15:19:15 +01:00
samrobinson
99ac6feeee Add varargs convenience constructors to Sequence & Composition.Builder
Improves API ease-of-use & readability and reduces verbose boilerplate.

PiperOrigin-RevId: 558163557
2023-08-22 15:17:55 +01:00
tianyifeng
7f1ee0b390 Use TrackGroup.equals when determining equality of BaseTrackSelection
PiperOrigin-RevId: 558150684
2023-08-22 15:16:38 +01:00
samrobinson
2db6f0aee7 Ensure audio components check incoming data is valid.
Default PCM encoding is only set for decoders outputting raw.

Tests migrated to abide by tighter restrictions.

PiperOrigin-RevId: 558129452
2023-08-18 15:33:44 +01:00
sheenachhabra
15650c6bf3 Add buildUpon method in Composition
We will need to build upon original composition
for pause and resume feature and will have to set modified sequences.
In the next CL, will add method to set sequences on builder and will
probably remove it from the constructor parameter (to avoid providing
two APIs for same thing).

PiperOrigin-RevId: 558127334
2023-08-18 15:32:28 +01:00
huangdarwin
030d8148ba Test: Add debug info to debug test failure
I wasn't able to reproduce the error in the bug by running videoTranscoding_withTextureInput_completesWithCorrectFrameCountAndDuration
100 times, but I figured this added debug info may help the next time the
issue is seen.

PiperOrigin-RevId: 558118443
2023-08-18 15:31:00 +01:00
huangdarwin
39bc92ffcf Compositor: Use multi-stage rendering and occlude background.
PiperOrigin-RevId: 558110739
2023-08-18 15:28:23 +01:00
claincly
a5903f5d3e Init the collection types to the expected size
PiperOrigin-RevId: 558102287
2023-08-18 15:27:02 +01:00
bachinger
2386b7e944 Some JavaDoc fixes in ImaServerSideAdInsertionMediaSource
Issue: androidx/media#555
PiperOrigin-RevId: 558097690
2023-08-18 15:25:41 +01:00
huangdarwin
02b347ac66 Test: Reduce tokyo.jpg test file size.
This file decompresses to 3024*4032*4 bytes ~= 48
MB, which leads to OOM errors on some devices.

Use a lower resolution test image for tests, to
still test a portrait image but without hitting
OOM errors.

PiperOrigin-RevId: 557861063
2023-08-18 15:20:34 +01:00
ibaker
450d1cd3e9 Add MediaParserExtractorAdapter.enableConstantBitrateSeeking
PiperOrigin-RevId: 557859297
2023-08-18 15:19:20 +01:00
huangdarwin
86678b99ce Decoder: Add SM-F721 to HLG blocklist.
Not only restricted to SM-F721B, per other [Models](https://www.gsmarena.com/samsung_galaxy_z_flip4-11538.php#:~:text=SM%2DF721B%2C%20SM%2DF721B/DS%2C%20SM%2DF721N%2C%20SM%2DF721U%2C%20SM%2DF721U1%2C%20SM%2DF721W).

PiperOrigin-RevId: 557857282
2023-08-18 15:17:48 +01:00
tofunmi
dd0f88490c Add ability to queue bitmap by timestamp to the sampleConsumer
PiperOrigin-RevId: 557837922
2023-08-18 15:16:24 +01:00
tofunmi
a8944ef2f0 Add BitmapDecoder interface
PiperOrigin-RevId: 557827954
2023-08-18 15:15:14 +01:00
huangdarwin
b814404c56 Effect: Update rgba matrix reference to rgb.
The 4x4 matrix only updates the RGB channels, so it should be referred to as an rgb matrix, not an rgba matrix.

PiperOrigin-RevId: 557804613
2023-08-18 15:13:44 +01:00
claincly
3ae927525b Add error listener to InputSwitcher's sampling shader program
PiperOrigin-RevId: 557797201
2023-08-18 15:12:32 +01:00
huangdarwin
a3de5978e0 Test: Use unpremultiplied alpha for GL output.
In addition to this being how Alpha should be handled, tests do fail under
API 29 without this CL. Unfortunately while these tests do fail under API 29
without this CL, we currently only run this test on API 33 emulators, so we
didn't catch this failure earlier (until compositor tests on mh failed on all
devices under API 29 and succeeded on all at or over 29).

PiperOrigin-RevId: 557781757
2023-08-18 15:11:12 +01:00
bachinger
6bd3e53e8d Set notification foreground service behaviour starting with API 31
When the foreground service type is set to `mediaPlayiback` this shouldn't
be required. However, some reports from users say that setting the
foreground service behavor fixes some problems on some devices. Setting
it explicitely with this change makes sure users don't have to do that
workaround themselves.

Issue: androidx/media#167
PiperOrigin-RevId: 557608577
2023-08-18 15:09:59 +01:00
samrobinson
74fa0ed6bf Move FakeAssetLoader to inner class of MediaItemExportTest.
MediaItemExportTest is the only class using this FakeAssetLoader, so it
should be moved out of TestUtil.

PiperOrigin-RevId: 557546717
2023-08-16 22:03:14 +01:00
samrobinson
27bc2e6844 Add new effects helper util for unit tests.
PiperOrigin-RevId: 557515564
2023-08-16 22:01:52 +01:00
ibaker
9631923440 Update SsaParser logic to show it never returns duration=UNSET
In an upcoming change I'm going to mark SSA subtitles as 'merge
replacement behavior', which will mean that
`CuesWithTiming.durationUs=C.TIME_UNSET` will not be permitted. This CL
makes it clear that `SsaParser` obeys that invariant.

PiperOrigin-RevId: 557504633
2023-08-16 22:00:38 +01:00
ibaker
d3894e6691 Add a default no-op implementation for SubtitleParser.reset
Most implementations were already no-ops.

PiperOrigin-RevId: 557503628
2023-08-16 21:59:19 +01:00
ibaker
d42c58f152 Fix DvbParser handling of pageComposition = null
This `DvbParser` was updated to implement the new `SubtitleParser`
interface in 2261756d3a.

This method used to return a `List<Cue>`, but now it returns a
`List<CuesWithTiming>`, which is closer to `List<List<Cue>>`. The
previous method returned an empty list, meaning a single 'event' with
no cues. After the change above the method still returns an empty list,
but that now means 'no events'. In order to maintain the old behaviour,
this change updates it to return a single-item list containing an empty
list of cues.

PiperOrigin-RevId: 557502261
2023-08-16 21:57:58 +01:00
tofunmi
a801f8d3f5 Add ImageDecoder.Factory
PiperOrigin-RevId: 557498045
2023-08-16 21:56:47 +01:00
jbibik
2156f94480 Bump media3 version to 1.2.0-alpha01 and update RELEASENOTES
#minor-release

PiperOrigin-RevId: 557487465
2023-08-16 21:55:35 +01:00
ibaker
eb63ed5871 Add test for Mp3Extractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING_ALWAYS
This test uses an existing CBR test asset, and shows that (as expected)
`isSeekable = true` even in the `unknown_length` dump file.

PiperOrigin-RevId: 557455364
2023-08-16 14:34:21 +01:00
jbibik
14e5b5b865 Mark CompositionPlayerTest as ClosedSource and copybara exclusion
#minor-release

PiperOrigin-RevId: 557416683
2023-08-16 14:33:02 +01:00
tonihei
7b663eea2f Remove MediaPeriodId from common module
It was moved there temporarily to support another temporary move
of ExoPlaybackException. See <unknown commit>.

Since then, ExoPlaybackException has been moved back to ExoPlayer
and we can do the same with MediaPeriodId, which only makes sense
in the context of the ExoPlayer module.

PiperOrigin-RevId: 557159381
2023-08-16 14:30:08 +01:00
samrobinson
9670c74044 Add parameterized test for audio export.
Verifies export completes successfully for every permutation.

PiperOrigin-RevId: 557157527
2023-08-16 14:28:35 +01:00
tonihei
597de8706d Show play button during playback suppression by default
This changes the default logic of shouldShowPlayButton to show a play
button while the playback is temporarily suppressed. This helps to
provide better UI feedback to the fact that playback stopped and
provides a quick way for users to override the suppression and attempt
to restart playback.

Some apps may want to keep the legacy behavior depending on their app's
needs. Hence, we also add a config parameter to set this behavior both
in MediaSession and our default UI components.

Issue: google/ExoPlayer#11213
PiperOrigin-RevId: 557129171
2023-08-15 17:15:38 +01:00
samrobinson
86b9fdae63 Merge TransformerAudio E2E tests into Transformer E2E class.
PiperOrigin-RevId: 557113117
2023-08-15 14:38:16 +01:00
tofunmi
cacf9b33b6 Add image decoding interfaces and capabilities
PiperOrigin-RevId: 557108362
2023-08-15 14:36:41 +01:00
tofunmi
e65d104330 Add image package-info.java
PiperOrigin-RevId: 557080845
2023-08-15 14:35:13 +01:00
samrobinson
ab8c5a6b63 Clarify & reduce the ShadowMediaCodec configurations added by default.
PiperOrigin-RevId: 556875714
2023-08-14 20:35:06 +01:00
tonihei
3ccb920930 Update bypass decode-only logic to timestamp-based pattern
This aligns the bypass code with the MediaCodec logic to not
use buffer.isDecodeOnly and instead rely on a timestamp
comparison with the last reset time. Overall, the change should
not introduce a functional difference and is covered by existing
end-to-end tests with bypass playbacks.

Splitting bypass batch buffers by decoder-only flag is
technically wrong for formats where not every buffer is a
keyframe, and this change also leaves a TODO to explain this
known shortcoming.

PiperOrigin-RevId: 556800038
2023-08-14 17:21:10 +01:00
tianyifeng
08067eff14 Fix several API level checks
PiperOrigin-RevId: 556751372
2023-08-14 17:18:41 +01:00
Googler
db6f043cb1 Rollback of f5a6ecdda1
PiperOrigin-RevId: 556718678
2023-08-14 11:06:30 +00:00
huangdarwin
4ce83d88e5 Alpha: Don't use deprecated SingleFrameGlShaderProgram interface.
PiperOrigin-RevId: 555990321
2023-08-11 20:41:50 +00:00
tianyifeng
1e2a5cd06a Bump version numbers to Media3 1.1.1 and ExoPlayer 2.19.1
PiperOrigin-RevId: 555987303
2023-08-11 20:40:35 +00:00
tofunmi
48de9fa916 Add VFP.queueInputBitmap api accepting timestamps
PiperOrigin-RevId: 555970312
2023-08-11 20:39:20 +00:00
tonihei
418edda285 Fix capacity overflow logic in findSampleAfter
The method was recently introduced and only searched for matching
samples up to index==length. However, SampleQueue uses a circular
array to index its data and the search should continue until
relativeStartIndex+length, while also handling the overflow
in the circular array.

The tests for seeking didn't cover these overflow cases yet
because they always started writing data in an empty SampleQueue.
This can be fixed by prewarming the queue with placeholder data
to force using the overflow logic in the various seek methods.

PiperOrigin-RevId: 555963011
2023-08-11 20:38:05 +00:00
claincly
2980c30ace Converging BlankFrameProducer
`BlankFrameProducer` was originally internal to
`DefaultVideoFrameProcessorVideoFrameRenderingTest`. We later copied it in
c221958889. This CL converges the usages.

PiperOrigin-RevId: 555953620
2023-08-11 20:36:44 +00:00
tonihei
1b085474e6 Rephrase format support javadoc to avoid Renderer reference
This is leftover from the fact that these constants were used
for ExoPlayer renderers only initially, but are now more generally
used to indicate format playback support.

PiperOrigin-RevId: 555951001
2023-08-11 20:34:03 +00:00
rohks
45348f1576 Utilise existing rate unset representation C.RATE_UNSET_INT
Use `C.RATE_UNSET_INT` instead of introducing `Long.MIN_VALUE` as the default value for rate parameters. Modifies default behaviour of the `TrackSelection.getLatestBitrateEstimate()` method and the `measuredThroughputInKbps` parameter in `CmcdHeadersFactory.CmcdRequest`.

PiperOrigin-RevId: 555939420
2023-08-11 20:32:41 +00:00
huangdarwin
0466bd7957 Effect: Add alpha effect.
This effect can scale Alpha values.

This is needed for testing Compositor, and coincidentally was requested
by a partner in the past. This also will generally be more useful in
full multi-asset, where we may want to have one input have different alpha
values than another

Also, update fragment_shader_copy_es2.glsl to not throw away alpha values.
This means ThumbnailStripShaderProgram (the only place this glsl is used) will
consider input alpha and output alpha appropriately

PiperOrigin-RevId: 555915092
2023-08-11 20:31:21 +00:00
rohks
1e2e225c01 Refactor CMCD headers construction for efficiency and readability
Optimises the construction of Common Media Client Data (CMCD) headers by eliminating the use of trailing commas. Instead of appending values directly to a StringBuilder, it now adds values to a list and joins them using a comma separator. While the primary focus is on enhancing readability and robustness, it's worth noting that in certain instances, removing usages of `formatInvariant`, there are also performance improvements.

PiperOrigin-RevId: 555911498
2023-08-11 20:30:06 +00:00
christosts
f5a6ecdda1 Implement the CompositingVideoSinkProvider
This is the first iteration for the CompositingVideoSinkProvider which
is basically a copy of MediaCodecVideoRenderer's
VieoFrameProcessorManager.

The MediaCodecVideoRenderer now instantiates a
CompositingVideoSinkProvider instead of a VieoFrameProcessorManager.

PiperOrigin-RevId: 555903928
2023-08-11 20:28:52 +00:00
rohks
171e1a1e42 Allow UNSET values in CmcdHeadersFactory's nested class setters
Also added `checkArgument()` in setters that lacked it.

PiperOrigin-RevId: 555900170
2023-08-11 20:27:32 +00:00
samrobinson
c740b58efa Add API for injecting AudioMixer.Factory.
Rename AudioMixerImpl to DefaultAudioMixer.

Removes the AudioMixerImpl specific getOutputAudioFormat, as the caller
defines and sets this.

PiperOrigin-RevId: 555887722
2023-08-11 20:26:21 +00:00
rohks
a9f9537e5f Use utility method Util.usToMs(timeUs) to convert durations to Ms
PiperOrigin-RevId: 555578906
2023-08-11 20:24:54 +00:00
rohks
4282a6ecd7 Add buffer starvation(bs), deadline(dl), playback rate(pr), startup(su)
Enhanced the Common Media Client Data (CMCD) logging by incorporating additional fields:
* buffer starvation (bs) : CMCD-Status
* deadline (dl) and startup (su) : CMCD-Request
* playback rate (pr) : CMCD-Session

PiperOrigin-RevId: 555553357
2023-08-11 20:23:23 +00:00
claincly
c1913e8d89 Deprecate SingleFrameShaderProgram
PiperOrigin-RevId: 555450958
2023-08-11 20:22:03 +00:00
samrobinson
4d0fb4326a Simplify FakeAudioProcessor by extending BaseAudioProcessor.
Correct createOneSecondDefaultSilenceBuffer - It was returning an
incorrect buffer duration for non-mono channel counts.

PiperOrigin-RevId: 555439387
2023-08-10 12:34:07 +00:00
samrobinson
6045190103 Rename TestMuxer to CapturingMuxer to match other dumpable components.
See CapturingAudioSink, CapturingRenderersFactory etc.

PiperOrigin-RevId: 555437158
2023-08-10 12:32:31 +00:00
tofunmi
7a85236655 Update Mimetypes to include supported image mimetypes
PiperOrigin-RevId: 555433856
2023-08-10 12:31:29 +00:00
tofunmi
d4b452d475 Use ProgressiveMediaSource and period for images
PiperOrigin-RevId: 555424664
2023-08-10 12:30:29 +00:00
huangdarwin
fd5784455c Compositor: Add test for correct input timestamps.
To ensure that for each output bitmap from the compostor, the right input
timestamps were used.

Only applied on a subset of tests to avoid needing to upload+maintain
too many files/size in the test binary, especially when it would test
duplicate behavior

PiperOrigin-RevId: 555222530
2023-08-10 12:29:21 +00:00
samrobinson
79725fd332 Use TestMuxer.Factory as the holder of the Dumpable test muxer.
PiperOrigin-RevId: 555222322
2023-08-10 12:28:18 +00:00
claincly
7e12d7619a Avoid create short lived object
PiperOrigin-RevId: 555204528
2023-08-10 12:27:00 +00:00
huangdarwin
3351f0387f Test: Use larger pixel diff on real devices, smaller on emulators.
PiperOrigin-RevId: 555178875
2023-08-10 12:25:59 +00:00
kimvde
266dc5a6d6 Add video speed change effect
This adds an effect to change the speed of the video track. The speed
of the audio track can already be changed using a SonicAudioProcessor.

Issue: androidx/media#559
PiperOrigin-RevId: 555155418
2023-08-10 12:24:41 +00:00
rohks
cf9f048a3d Enhance getNextChunk() API to allow passing more information
Modifies the `ChunkSource.getNextChunk(long, long, List, ChunkHolder)` method in the `ChunkSource` interface to `ChunkSource.getNextChunk(LoadingInfo, long, List, ChunkHolder)`.

LoadingInfo contains additional parameters, including `playbackRate` and `lastRebufferRealtimeMs` in addition to the existing `playbackPositionUs`. The additional parameters will allow us to pass these information to Common Media Client Data (CMCD) logging.

PiperOrigin-RevId: 555148768
2023-08-10 12:23:37 +00:00
andrewlewis
bf5c4d8078 Use golden data generated on emulator for tests based on london.jpg
PiperOrigin-RevId: 555148673
2023-08-10 12:22:31 +00:00
jbibik
f7c31bd3ef Fix Gradle Lint with @RequiresApi and SDK version
Gradle Lint doesn't recognise `checkState` assertion and TargetApi should only ever be used for suppressing a bug in Android Lint. Hence, we keep @RequiresApi and add an if-statement explicitly. Also, fixes >26 to >=26 for the version check.

#minor-release

PiperOrigin-RevId: 555144577
2023-08-10 12:21:27 +00:00
kimvde
2a645ac2b6 Add CompositionPlayer listener tests
PiperOrigin-RevId: 555139635
2023-08-10 12:20:03 +00:00
tofunmi
7bc213d3ef Set tile count in Pngextractor
Tile count must be set to make sure the image decoder recognises and supports the format.

PiperOrigin-RevId: 555136616
2023-08-10 12:19:03 +00:00
rohks
38eabdc9fc Bump IMA SDK version to 3.30.3
#minor-change

PiperOrigin-RevId: 555130308
2023-08-10 12:18:00 +00:00
rohks
481ea4a274 Enhance continueLoading() API to allow passing more information
Modifies the `SequenceableLoader.continueLoading(long)` method in the `SequenceableLoader` interface to `SequenceableLoader.continueLoading(LoadingInfo)`.

`LoadingInfo` contains additional parameters, including `playbackSpeed` and `lastRebufferRealtimeMs` in addition to the existing `playbackPositionUs`. The additional parameters will allow us to pass these information to Common Media Client Data (CMCD) logging.

By using the `LoadingInfo` object, we ensure future flexibility to add more fields without causing any breaking changes to the API.

PiperOrigin-RevId: 555123961
2023-08-10 12:16:41 +00:00
claincly
2da3cc71e5 Fix mis-placed debug tracing call
PiperOrigin-RevId: 555118817
2023-08-10 12:15:35 +00:00
tonihei
20d2ce7ce8 Verify source is not released before updating ad playback state
Updates to the ad playback state are posted on the main handler,
so they may arrive after the source has already been released
(=the internal MediaSource is null). This can cause NPEs.

PiperOrigin-RevId: 555102426
2023-08-10 12:13:12 +00:00
jbibik
dedccc596e Fix PlayerWrapper's creation of VolumeProviderCompat
When hardware buttons are used to control the volume of the remote device, the call propagates to `MediaSessionCompat.setPlaybackToRemote(volumeProviderCompat)`. However, `volumeProviderCompat` was created incorrectly when the new device volume commands were present (COMMAND_SET_DEVICE_VOLUME_WITH_FLAGS and COMMAND_ADJUST_DEVICE_VOLUME_WITH_FLAGS), i.e. with volumeControlType = `VOLUME_CONTROL_FIXED`. This resulted in `VolumeProviderCompat` which doesn't call `onSetVolumeTo` or `onAdjustVolume` and hence doesn't propagate the calls to the `Player`. Instead, it only worked with the deprecated commands which ensured the volumeControlType was `VOLUME_CONTROL_ABSOLUTE`.

This bug was introduced in c71e4bf1ff (1.0 media 3 release) when `PlayerWrapper`'s call to `createVolumeProviderCompat` was mostly rewritten to handle the new commands, but the two if-statements were not amended. Note: this change fixes the bug only for Android 11 and below. For 12 and above, there is a tracking bug for the regression that was introduced: https://issuetracker.google.com/issues/201546605

http://Issue: androidx/media#554

#minor-release

PiperOrigin-RevId: 554966361
2023-08-10 12:12:09 +00:00
claincly
e8a18e208a Improve DebugTraceUtil logging
PiperOrigin-RevId: 554872827
2023-08-10 12:10:50 +00:00
Tianyi Feng
ef54364478 Merge pull request #528 from zgzong:patch-2
PiperOrigin-RevId: 554869426
2023-08-10 12:09:43 +00:00
samturci
821ec49049 Preserve the byte order when using read-only ByteBuffers.
`asReadOnlyBuffer` doesn't copy the original byte order but always sets it to big-endian. Replace calls to it with a utility method that manually sets the byte order after creating the read-only copy.

This fixes `TeeAudioProcessor` providing a `ByteBuffer` always in big-endian and hence causing `AudioBufferSink` to read wrong data.

PiperOrigin-RevId: 554861402
2023-08-10 12:08:19 +00:00
Googler
4fe55b8b63 Return the correct output buffer from audio processing pipeline
PiperOrigin-RevId: 554851456
2023-08-10 12:07:15 +00:00
rohks
3ec462d1cf Add field object type (ot)
Added this CMCD-Object field to Common Media Client Data (CMCD) logging.

#minor-release

PiperOrigin-RevId: 554843305
2023-08-10 12:06:13 +00:00
christosts
cba027c3b4 Util.getMaxPendingFramesCountForMediaCodecDecoders(): Remove unused arguments
PiperOrigin-RevId: 554838494
2023-08-10 12:05:08 +00:00
tofunmi
7be15b7603 Release overlay resources at end of frame processing
When implementing pause and resume, overlay appears as black frame on resume, suggesting the texture isn't being remade properly due to lack of proper deletion.

PiperOrigin-RevId: 554832580
2023-08-10 12:04:03 +00:00
kimvde
5c78290d5a Delete created GL programs
PiperOrigin-RevId: 554832238
2023-08-10 12:02:59 +00:00
kimvde
57086b6641 Make effect classes final
Making classes final when possible is good practice

PiperOrigin-RevId: 554776293
2023-08-10 12:01:53 +00:00
Googler
7d6f02dc50 Only set hadKeyframe flag to true if we end up muxing the frame.
If we throw away the first keyframe (because it does not contain any bytes to read), we shouldn't behave as if we had written a keyframe to our track. We should instead continue waiting for the first (valid) keyframe.

PiperOrigin-RevId: 554579146
2023-08-10 12:00:51 +00:00
kimvde
f43146718c Add CompositionPlayer unit test
Test the audio side only because the video side won't be testable with
unit tests once we add the OpenGL components.

PiperOrigin-RevId: 554508703
2023-08-10 11:59:42 +00:00
andrewlewis
691bf93032 Use lower resolution bitmap for tests
The test file decompresses to about 48 MB, which can cause an
`OutOfMemoryError` on lower end devices.

Also, a test appears to apply a rotation to this bitmap without downscaling
first, leading to exceeding a maximum texture width/height of 4096 on some
devices.

Replace the test file with one that has a quarter of the original's
width/height so that these tests can pass reliably even on lower end
devices.

PiperOrigin-RevId: 554507185
2023-08-10 11:58:25 +00:00
ibaker
6e91f0d4c5 Use ceiling divide logic in AudioTrackPositionTracker.hasPendingData
This fixes a bug with playing very short audio files, introduced by
fe710871aa

The existing code using floor integer division results in playback never
transitioning to `STATE_ENDED` because at the end of playback for the
short sample clip provided `currentPositionUs=189937`,
`outputSampleRate=16000` and `(189937 * 16000) / 1000000 = 3038.992`,
while `writtenFrames=3039`. This is fixed by using `Util.ceilDivide`
so we return `3039`, which means
`AudioTrackPositionTracker.hasPendingData()` returns `false` (since
`writtenFrames ==
durationUsToFrames(getCurrentPositionUs(/* sourceEnded= */ false))`).

#minor-release

Issue: androidx/media#538
PiperOrigin-RevId: 554481782
2023-08-10 11:57:15 +00:00
tonihei
71591782f0 Adjust periodEnded logic further
Periods that are not the last in the manifest are always ended
and non-dynamic periods are also always ended by definition.

Also, some periods may not be able to declare their final duration
accurately and we need to account for some inaccuracy. This can be
done by testing whether no further full segment within the duration
can be expected.
2023-08-08 17:47:23 +01:00
tonihei
88e921ced1 Format with google-java-format 2023-08-08 17:42:57 +01:00
zgzong
fb847cad66 Update dash chunk source to handle in-progressive recording dash stream.
Using calculated segments timeline duration to compare with period duration to update whether period has reach it the end of stream.
2023-08-08 17:42:57 +01:00
claincly
670658f3ae Use a larger pixel difference on real devices
PiperOrigin-RevId: 554452842
2023-08-07 14:04:36 +00:00
kimvde
117b18f54c Check command availability before getting tracks in PlayerView
#minor-release

PiperOrigin-RevId: 554451569
2023-08-07 14:03:27 +00:00
tianyifeng
ff3902debb Update translations
PiperOrigin-RevId: 554439196
2023-08-07 14:02:25 +00:00
samrobinson
cc141f7bf4 Use specific type for SequenceAssetLoader internal logic.
PiperOrigin-RevId: 554431024
2023-08-07 11:36:43 +00:00
samrobinson
42425334ae Mark always passing test as @Ignore.
Added a TODO inline linking to the created bug to address this.

PiperOrigin-RevId: 554422990
2023-08-07 11:35:34 +00:00
michaelkatz
aed21e464e Fix last sample detection issue for ClippingMediaSource
If stream is not considered final and `ClippingMediaSource` is read to `endUs`, then `BaseRenderer.readSource` returns `C.RESULT_NOTHING_READ`. In that case, the `lastBufferInStreamPresentationTimeUs` is not set and the last frame is not rendered.

PiperOrigin-RevId: 554418971
2023-08-07 11:34:34 +00:00
samrobinson
82387ccfe6 Remove unneeded check against silence generator remaining.
PiperOrigin-RevId: 553771328
2023-08-07 11:33:37 +00:00
huangdarwin
80d63a32d3 Test: Generalize Compositor tests for multi-input, and add more tests.
Tests are added for offset streams with gaps in the middle

PiperOrigin-RevId: 553510171
2023-08-07 11:32:38 +00:00
tonihei
57f327840b Remove old workaround for timestamp jumps
This workaround was added for TS streams that do not adjust their
timestamps to start from zero. Over time, the default audio sink
logic has become more robust towards unexpected timestamps and
we no longer need this workaround to jump forward in time.

The workaround also actively caused issues by adjusting the audio
timestamps backwards if the stream starts with large negative
values. See Issue: androidx/media#291. This caused playback to get stuck due
to another bug in the first-frame rendering logic in the video
renderer that is now fixed.

PiperOrigin-RevId: 553493618
2023-08-07 11:31:34 +00:00
tonihei
a3c1d5be9d Reset loading period after reading discontinuity that requires loading
This is the same change as 8655429af7, just on the second call site
of readDiscontinuity. The tests didn't cover this case yet because
they never queued more than 2 items in a playlist.

PiperOrigin-RevId: 553485244
2023-08-07 11:30:38 +00:00
huangdarwin
05782a7e99 Compositor: Add support for mismatched timestamps.
This means we now require 2+ input frames per input, and compare the primary
stream timestamp with secondary stream timestamps in order to select the
correct output timestamp. We also must release frames and back-pressure as
soon as possible to avoid blocking upstream VFPs.

Also, improve signalling of VFP onReadyToAcceptInputFrame

PiperOrigin-RevId: 553448965
2023-08-07 11:29:44 +00:00
claincly
ed1ff222bb Remove unnecessary setInputFormat
setInputFormat calls registerInputStream and will cause VideoFrameProcessor to
reconfigure. We don't need to call it when setting up the VFP
(in `onReadyToInitCodec()`). Rather, we wait until `onOutputFormatChanged()`
called (this output format refers to mediaCodec's outptut format) to register
the input stream.

PiperOrigin-RevId: 553448633
2023-08-07 11:28:43 +00:00
tofunmi
dfd668238a Create SingleSampleSeekMap and use in pngExtractor
PiperOrigin-RevId: 553191571
2023-08-07 11:27:27 +00:00
sheenachhabra
876b767356 Fix nit issues in muxer tests
PiperOrigin-RevId: 553145104
2023-08-07 11:25:20 +00:00
claincly
af546b8eda Fix decoder is queued buffer with timestamp TIME_END_OF_SOURCE
ExoPlayer queues the EOS buffer to the decoder with offset/size/timestamp all
equal to zero, and a EOS flag.

69769c77b3 set TIME_END_OF_SOURCE on the EOS buffer from the extractor.
Queueing the EOS buffer to the decoder with TIME_END_OF_SOURCE causes some
decoders to output wrong timestamps in its output.

This CL replicates what ExoPlayer does in Transformer.

PiperOrigin-RevId: 553104213
2023-08-07 11:24:11 +00:00
huangdarwin
4302102cf0 Test: Use TestName in media3.effects tests.
Instead of duplicating strings

PiperOrigin-RevId: 552865414
2023-08-07 11:22:02 +00:00
tonihei
69656774f9 Only use first frame stream start position check at stream transitions
We currently only force the first frame if the frame timestamp is
greater than the stream *offset*.

This is wrong for two reasons:
 1. The timestamp and the offset are not comparable and it should be
    the stream start position.
 2. The check should only be applied at stream transitions where we
    need to make sure that a new first frame isn't rendered until we
    passed the transition point.

We have to fix both issues together, because fixing just issue (1)
causes seeks to before the start position to no longer render the
frame (and playback will be stuck). A new test covers this case.

We also amend the stream transition test case to actually test what it
promises to test and add a test for prerolling samples at the
beginning, to ensure the first frame is still renderered.

Issue: androidx/media#291
PiperOrigin-RevId: 552858967
2023-08-07 11:21:08 +00:00
tonihei
79e05ad049 Simplify first frame logic
We currently use 3 different booleans to track the state of the first
frame rendering, which implies that there are 8 distinct possible
overall states. However, this is actually a staged process and there
are only 3 different overall states in the current code. This means
it's clearer and easier to reason about if the variables are combined
to a single state value. Overall, this should be a complete no-op.

State mapping:
 - rFFAReset=false, rFFAEnable=false, mayRenderFFAEINS=false
   => FIRST_FRAME_NOT_RENDERED_ONLY_ALLOWED_IF_STARTED
 - rFFAReset=false and/or rFFAEnable=false, mayRenderFFAEINS=any
   => FIRST_FRAME_NOT_RENDERED
 - rFFAReset=true, rFFAEnable=true, mayRenderFFAEINS=any
   => FIRST_FRAME_RENDERED

PiperOrigin-RevId: 552857802
2023-08-07 11:19:56 +00:00
rohks
11648e6c8e Simplify and accurately compute chunk duration
Refactored `CmcdLog` to `CmcdHeadersFactory` for improved representation of its purpose and updated implementations.

#minor-change

PiperOrigin-RevId: 552831995
2023-08-07 11:19:00 +00:00
claincly
6633b05c0a Reorder DebugTraceEvent
PiperOrigin-RevId: 552818752
2023-08-07 11:18:04 +00:00
Googler
e87f0d5580 Add a thumbnail strip effect that tiles frames horizontally.
The size of the thumbnail strip and the timestamp of the video frames to use must be specified by the user of the effect.

PiperOrigin-RevId: 552809210
2023-08-07 11:15:10 +00:00
rohks
267d4164a9 Fix wrong documentation and add missing annotations
#minor-release

PiperOrigin-RevId: 552749407
2023-08-01 14:36:42 +01:00
huangdarwin
f389a85eea Compositor: Extract interface.
This allows for custom implementations of this interface, like
a TestVideoCompositor or partner-implemented implementation

PiperOrigin-RevId: 552541631
2023-08-01 14:35:29 +01:00
sheenachhabra
8c2d047b71 Add validation for empty csd data
PiperOrigin-RevId: 552541399
2023-08-01 14:34:17 +01:00
tofunmi
87a020a154 Add PngExtractor to DefaultExtractorsFactory
PiperOrigin-RevId: 552506545
2023-08-01 14:32:58 +01:00
jbibik
b8e6f27caf Update SubtitleExtractor to use SubtitleParser directly
`SubtitleExtractor` used to rely on a `SubtitleDecoder`. However, now all SubtitleDecoders that are used for side-loaded subtitles have been migrated to a `SubtitleParser` interface. We can therefore refactor the extractor.

The `SubtitleExtractor` is only used for side-loaded subtitles which means we do not require the migration of the CEA-608/708 `SubtitleDecoders`.

PiperOrigin-RevId: 552471710
2023-08-01 14:31:58 +01:00
samrobinson
9e975b25d1 Define TestMuxer Factory and Holder as inner classes.
PiperOrigin-RevId: 552469061
2023-08-01 14:29:38 +01:00
claincly
c16840573b Format DebugTraceUtil report to output JSON string
PiperOrigin-RevId: 552455533
2023-08-01 14:28:23 +01:00
tofunmi
499dd5b84e remove redundant identity setting in the OverlayShaderProgram
PiperOrigin-RevId: 552416606
2023-08-01 14:26:07 +01:00
tofunmi
bb699e41c9 Add png extractor
Implementing a basic extractor that reads the whole png file into the trackOutput as one sample.

PiperOrigin-RevId: 551897619
2023-08-01 14:24:47 +01:00
ibaker
a700974d8c Mark x-media3-cues MIME type as text type
PiperOrigin-RevId: 551853336
2023-08-01 14:23:44 +01:00
andrewlewis
ddb0f86604 Fix parsing of H.265 sequence parameter sets
Fix short term reference picture list parsing. Before this change, `deltaPocS0`
was derived by adding one to the value of the syntax element
`delta_poc_s0_minus1`, but (maybe surprising) the specification actually says
that `DeltaPocS0[stRpsIdx][i]` should be assigned the negation
`-(delta_poc_s0_minus1[i] + 1)` on the first iteration, then that value added
to the previous value on previous iterations. See equations (7-67) to (7-70) in
the 2021-08 version of the H.265/HEVC specification.

Also read the number of long term reference pictures once rather than on every
loop iteration (subsection 7.3.2.2.1).

PiperOrigin-RevId: 551852999
2023-08-01 14:22:29 +01:00
christosts
444f0023a4 Define the VideoSink interface
PiperOrigin-RevId: 551825940
2023-08-01 14:20:18 +01:00
samrobinson
b15e7553e5 Add the hasSource method to the AudioMixer interface.
PiperOrigin-RevId: 551804976
2023-08-01 14:19:04 +01:00
claincly
c221958889 Remove setInputFrameInfo
After this change, every queued bitmap is treated as an individual input stream
(like a new MediaItems).

This change merges the FrameDropTest and FrameDropPixelTest into one (while maintaining all the test cases)

- This is accomplished by generating bitmaps with timestamps on it in FrameDropTest and compare them with goldens (one may call this a pixel test, please lmk if you want this to be renamed)
- The most part of the change comes from DefaultVideoFrameProcessorVideoFrameRenderingTest. The overall working is
  - We bypass the input manager
  - The TestFrameGenerator generates frames based on timestamps. In this case, we generate frames with timestamps on it
  - The generated frame is sent to texture output and in turn saved to bitmaps
  - We then compare the generated bitmap with the goldens

PiperOrigin-RevId: 551795770
2023-08-01 14:17:59 +01:00
tofunmi
ebbcec9a20 Add imageDurationMs to MediaItem
PiperOrigin-RevId: 551776350
2023-08-01 14:16:48 +01:00
samrobinson
ccdde8806e Account for endTimeUs when outputting with no sources.
When endTimeUs is set, restrict output to that point.

PiperOrigin-RevId: 551555392
2023-08-01 14:15:34 +01:00
claincly
5ba0ec9b40 Remove stale TODO
It says RTSP doesn't support this in the javadoc.

PiperOrigin-RevId: 551541942
2023-08-01 14:14:34 +01:00
huangdarwin
bd6a36033e Compositor: Move shared test logic to TestRunner.
Move shared test logic to the test runner.

This does increase indirection, which isn't usually preferable in tests.
However, we will have many different tests that would use logic
like this, so this allows us to reduce repetition.

PiperOrigin-RevId: 551536438
2023-08-01 14:13:25 +01:00
claincly
e77c318d96 Create Util method for SparseArray.contains()
And fixed usages in Transformer and Effect.

PiperOrigin-RevId: 551484860
2023-08-01 14:12:10 +01:00
bachinger
17ee5277d5 Only use result.sendError where supported by legacy media library
`MediaLibraryServiceLegacyStub` handles various edge cases by calling
`result.sendError(null)` with the intention to send back an error to
the legacy browser [1].

`MediaBrowserServiceCompat` of the legacy media1 Compat library has an
inner base class `Result` that has a default implementation of
`onErrorSent` that throws an `UnsupportedOperationException` [2].
However, most anonymous inner classes for `Result` created in
`MediaBrowserServiceCompat` do not override `onErrorSent` [3].

Hence Media3 must not call `sendError` in these cases. Instead we call
`sendResult(null)` according to what the default implementation of
the callbacks in `MediaBrowserServiceCompat` do ([4] as an example).

Issue: androidx/media#78
Issue: androidx/media#334

[1] https://github.com/androidx/media/blob/release/libraries/session/src/main/java/androidx/media3/session/MediaLibraryServiceLegacyStub.java#L200
[2] https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:media/media/src/main/java/androidx/media/MediaBrowserServiceCompat.java;l=872
[3] https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:media/media/src/main/java/androidx/media/MediaBrowserServiceCompat.java;l=578-604?q=MediaBrowserServiceCompat&ss=androidx
[4] https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:media/media/src/main/java/androidx/media/MediaBrowserServiceCompat.java;l=1395

PiperOrigin-RevId: 551210137
2023-08-01 14:11:08 +01:00
tonihei
f766936140 Fix ms to us conversion bug in DecoderVideoRenderer
The current code multiplies the value by 1000 twice,
effectively converting to nanoseconds.

#minor-release

PiperOrigin-RevId: 551129750
2023-08-01 14:09:56 +01:00
samrobinson
b7d7027424 Update Composition javadoc for expanded audio support.
PiperOrigin-RevId: 550922961
2023-08-01 14:08:46 +01:00
tonihei
782f5b2a0e Fix last sample detection issues
1. For codecs that change timestamps, we need to use greater
   or equal instead of just equal for comparison.
2. We should only make this comparison once a value has been
   set.
3. The largest queue timestamp isn't updated before using it
   for this detection, meaning that we also mark the
   second-to-last sample as the last one.

PiperOrigin-RevId: 550901978
2023-08-01 14:06:55 +01:00
andrewlewis
9a2bfc4042 Prioritize H.265 and H.264 over other formats
This should make no functional difference because `SampleExporter` always
checks for H.265 and H.264 first. However, in case we ever change that code,
these are used in priority order so it's better to order them accordingly.

PiperOrigin-RevId: 550894935
2023-08-01 14:05:42 +01:00
tonihei
79c8b0e0f8 Set correct start time for newly enabled renderers during playback
We currently pass in the time at which the stream originally started,
but newly enabled renderers should get the current playback position
instead.

PiperOrigin-RevId: 550894630
2023-08-01 14:04:35 +01:00
samrobinson
54093a152e Integrate AudioMixer for audio export.
Adds support for compositions with multiple audio sequences.

PiperOrigin-RevId: 550880626
2023-08-01 14:03:22 +01:00
samrobinson
357c458028 Ensure EOS is queued after processing generated silence with effects.
When generating silence for AudioProcessingPipeline, audio never
queued EOS downstream.

Linked to this, when silence followed an item with audio, the silence
was added to SilentAudioGenerator before the mediaItem reconfiguration
occurred. If the silence had effects, the APP would be flushed after
silence queued EOS, resetting APP.isEnded back to false, so AudioGraph
never ended.

Regression tests reproduce failure without fix, but pass with it.

PiperOrigin-RevId: 550853714
2023-08-01 14:02:09 +01:00
huangdarwin
35bee3299c Remove VisibleForTesting visibility in 8k workaround.
PiperOrigin-RevId: 550843914
2023-08-01 14:01:02 +01:00
Googler
465e7c1277 Rewrite tracing to be more flexible and provide more info.
For each event, the timestamp and presentation time is logged. The trace can
then be dumped to a tsv file and easily imported in a spreadsheet.

PiperOrigin-RevId: 550839156
2023-08-01 13:59:51 +01:00