Commit graph

280 commits

Author SHA1 Message Date
samrobinson
fbb64e2db2 Remove experimental keepAudioTrackOnSeek.
PiperOrigin-RevId: 570966027
(cherry picked from commit 068d420ba2)
2023-10-06 15:57:25 +01:00
ibaker
8f5835c51c Rename DefaultImageDecoder to BitmapFactoryImageDecoder
This reflects the documented behaviour of this class.

#minor-release

PiperOrigin-RevId: 569475137
2023-09-29 06:27:24 -07:00
claincly
34dddfe9d5 Add previewing specific video graph.
PiperOrigin-RevId: 569473178
2023-09-29 06:16:16 -07:00
christosts
603d36726f Move DefaultImageDecoderTest in tests
Move DefaultImageDecoderTest in tests since we can decode images with
Robolectric's native graphics mode.

PiperOrigin-RevId: 567259458
2023-09-21 03:53:42 -07:00
michaelkatz
3cf9c08dca Create AudioOffloadPreferences class
Move audio offload mode related interfaces and definitions from `TrackSelectionParameters` to a new `AudioOffloadModePreferences` class.

PiperOrigin-RevId: 566905017
2023-09-20 03:02:51 -07:00
ibaker
694d040d7e Re-land: TtmlParser implementation - moved from TtmlDecoder
This change uses the new incremental overloads of `SubtitleParser` to
avoid introducing the performance regression caused by the original
change requiring all cues to be fully parsed before the first could be
shown on screen.

`TtmlDecoder` which used to be `SimpleSubtitleDecoder` will now be
called `TtmlParser` and implement `SubtitleParser` interface. For
backwards compatibility, we will have the same functionality provided
by `DelegatingSubtitleDecoder` backed-up by a new `TtmlParser`
instance.

PiperOrigin-RevId: 566671398
2023-09-19 10:25:48 -07:00
jbibik
39c1ac7f3a Move setAudioAttributes from ExoPlayer to Player
PiperOrigin-RevId: 566607528
2023-09-19 06:08:37 -07:00
tofunmi
16b0ea850f Support multiple streams in the ImageRenderer
PiperOrigin-RevId: 565410924
2023-09-14 10:37:58 -07:00
michaelkatz
85c944b955 Enable offload scheduling by default for audio-only offload playback
Removed `ExoPlayer.experimentalSetOffloadSchedulingEnabled` as scheduling will be enabled by default when offload is enabled for audio-only playback.

In addition, the `experimental` key word was taken out of the following
method signatures:
* `ExoPlayer.experimentalIsSleepingForOffload`
* `AudioOffloadListener.onExperimentalSleepingForOffloadChanged`
* `AudioOffloadListener.onExperimentalOffloadedPlayback`

PiperOrigin-RevId: 565035289
2023-09-13 06:46:50 -07:00
tofunmi
23665090ed Support Jpeg image track extraction in exoplayer
PiperOrigin-RevId: 563818198
2023-09-08 12:15:10 -07:00
christosts
4afecc9099 Rollback of 51fd06482b
PiperOrigin-RevId: 563714392
2023-09-08 04:39:51 -07:00
tonihei
5a1322c9f9 Remove restriction from ConcatenatingMediaSource2
The class currently disallows offsets of periods in their windows
except for the very first window. This is not necessary because
we can use TimeOffsetMediaPeriod to eliminate the offset if needed.
This makes the class more useful for many use cases, in particular
for using it with ClippingMediaSource.

Issue: google/ExoPlayer#11226
PiperOrigin-RevId: 563702120
2023-09-08 03:31:17 -07:00
tonihei
dd2a373636 Add missing synchronization in ExperimentalBandwidthMeter
Issue: androidx/media#612
PiperOrigin-RevId: 563690229
2023-09-08 02:27:20 -07:00
tonihei
f2b1d0cfa3 Move TimeOffsetMediaPeriod to its own class
This makes it reusable for other MediaSource/Periods in the same
package.

Issue: google/ExoPlayer#11226
PiperOrigin-RevId: 563687935
2023-09-08 02:13:39 -07:00
tofunmi
535af35511 Add heif extractor
PiperOrigin-RevId: 563340591
2023-09-07 00:31:04 -07:00
christosts
5b19e08ea9 Rollback of 51fd06482b
PiperOrigin-RevId: 562550636
2023-09-04 07:13:59 -07:00
christosts
51fd06482b Make Timeline and MediaPeriodId visible to Renderers
This is so CompositionPlayer can customize the Renderers and figure
out when a transition to a MediaItem is made hence it can configure the
effects pipeline with the effects that are specific to the MediaItem.

PiperOrigin-RevId: 562530774
2023-09-04 05:12:13 -07:00
michaelkatz
9d58788d86 Add AudioSink#setOffloadDelayPadding to offload play gapless Opus
Created new method `AudioSink#setOffloadDelayPadding` that will set delay and padding data onto an `AudioTrack` using `AudioTrack#setOffloadDelayPadding`. This feature adds support for offloaded, gapless Opus playback as the content requires the capability to set padding data post-`AudioSink#configure`.

PiperOrigin-RevId: 562518193
2023-09-04 04:02:44 -07:00
tofunmi
e15c05fc3c Flip input buffer in the ImageRenderer
also improves tests to hash bitmap now that we know the bitmaps are decoded in roboelectic tests,
as well as some minor fixes in the DefaultImageDecoder

PiperOrigin-RevId: 561946073
2023-09-01 07:12:57 -07:00
tofunmi
5944adcc78 Add image e2e test
PiperOrigin-RevId: 561887238
2023-09-01 01:31:45 -07:00
rohks
206a663dca Add functionality to transmit CMCD data using query parameters
Currently, we only support sending Common Media Client Data (CMCD) data through custom HTTP request headers, added capability to configure and transmit it as HTTP query parameters.

PiperOrigin-RevId: 561591246
2023-08-31 02:11:12 -07:00
ibaker
64792dfda2 Rollback of 25253698bc
PiperOrigin-RevId: 561352515
2023-08-30 08:53:49 -07:00
tofunmi
6a5d5aef92 Add limited ImageRenderer
At this stage, the image renderer is designed to be able to render one input stream containing one sample. The renderer supports seeking inside that stream, which is a no-op because the stream comprises for exactly one sample.

PiperOrigin-RevId: 561307859
2023-08-30 05:15:38 -07:00
tianyifeng
86f919e7d4 Fix flaky tests in PlaylistPlaybackTest
If the loading thread of the second media item progresses slow, then when the playback of the first item approaches to the end before the second item is prepared, the stream will be set "final" and the renderers are disabled during such transition. In this case, `AudioSink.handleDiscontinuity` never gets called, which introduces the flakiness into the `test_bypassOffThenOn`.

Similarly for `test_subtitle`, if the loading thread of extracting the subtitle is slow, then when the renderer attempts to queue the input buffer, the data hasn't been available in the stream. And when the extracting finished, the renderer already advanced to the time after the subtitle end time.

To de-flake the tests, we have to make sure that the period of the second item has fully loaded before starting the playback.

PiperOrigin-RevId: 560080119
2023-08-25 07:04:26 -07:00
claincly
930e538aca Allow re-configuring DVFP in a (mostly) non-blocking manner
The first call to method `registerInputStream` doesn't block.

Later successive calls to the method blocks until the previous register call
finishes.

PiperOrigin-RevId: 559694490
2023-08-24 02:55:54 -07:00
christosts
398809e4e2 Replace Queue<Long> with a queue for long primitives
Replace Queue<Long> with LongArrayQueue which provides queue semantics
for long primitives. LongArrayQueue is forked from IntArrayQueue which
in turn was forked from Androidx CircularIntArray.

IntArrayQueue is deleted and we now use CircularIntArray directly from
Androidx Collection.

PiperOrigin-RevId: 559129744
2023-08-24 09:14:44 +01:00
christosts
4fad529433 Rollback of f5a6ecdda1
PiperOrigin-RevId: 558779068
2023-08-22 15:31:43 +01:00
rohks
b6b16b2895 Sort Common Media Client Data keys to reduce fingerprinting surface
Based on the Common Media Client Data (CMCD) specification key-value pairs should be sequenced in alphabetical order of the key name in order to reduce the fingerprinting surface exposed by the player.

PiperOrigin-RevId: 558296264
2023-08-22 15:23:33 +01:00
rohks
afb8d6c9e2 Make custom data adhere to Common Media Client Data(CMCD) specification
Added more comprehensive Javadoc around setting custom data and verification on key format.

Changed adding custom data as a `String` to `List<String>`. This would enable us to sort all keys to reduce the fingerprinting surface.

PiperOrigin-RevId: 558291240
2023-08-22 15:22:08 +01:00
samturci
6566387f70 Add a waveform audio buffer sink.
This can be used together with TeeAudioProcessor to draw an audio waveform.

PiperOrigin-RevId: 558213516
2023-08-22 15:20:47 +01:00
ibaker
450d1cd3e9 Add MediaParserExtractorAdapter.enableConstantBitrateSeeking
PiperOrigin-RevId: 557859297
2023-08-18 15:19:20 +01:00
tofunmi
a8944ef2f0 Add BitmapDecoder interface
PiperOrigin-RevId: 557827954
2023-08-18 15:15:14 +01:00
tofunmi
a801f8d3f5 Add ImageDecoder.Factory
PiperOrigin-RevId: 557498045
2023-08-16 21:56:47 +01:00
tofunmi
cacf9b33b6 Add image decoding interfaces and capabilities
PiperOrigin-RevId: 557108362
2023-08-15 14:36:41 +01:00
tonihei
3ccb920930 Update bypass decode-only logic to timestamp-based pattern
This aligns the bypass code with the MediaCodec logic to not
use buffer.isDecodeOnly and instead rely on a timestamp
comparison with the last reset time. Overall, the change should
not introduce a functional difference and is covered by existing
end-to-end tests with bypass playbacks.

Splitting bypass batch buffers by decoder-only flag is
technically wrong for formats where not every buffer is a
keyframe, and this change also leaves a TODO to explain this
known shortcoming.

PiperOrigin-RevId: 556800038
2023-08-14 17:21:10 +01:00
Googler
db6f043cb1 Rollback of f5a6ecdda1
PiperOrigin-RevId: 556718678
2023-08-14 11:06:30 +00:00
tonihei
418edda285 Fix capacity overflow logic in findSampleAfter
The method was recently introduced and only searched for matching
samples up to index==length. However, SampleQueue uses a circular
array to index its data and the search should continue until
relativeStartIndex+length, while also handling the overflow
in the circular array.

The tests for seeking didn't cover these overflow cases yet
because they always started writing data in an empty SampleQueue.
This can be fixed by prewarming the queue with placeholder data
to force using the overflow logic in the various seek methods.

PiperOrigin-RevId: 555963011
2023-08-11 20:38:05 +00:00
christosts
f5a6ecdda1 Implement the CompositingVideoSinkProvider
This is the first iteration for the CompositingVideoSinkProvider which
is basically a copy of MediaCodecVideoRenderer's
VieoFrameProcessorManager.

The MediaCodecVideoRenderer now instantiates a
CompositingVideoSinkProvider instead of a VieoFrameProcessorManager.

PiperOrigin-RevId: 555903928
2023-08-11 20:28:52 +00:00
rohks
4282a6ecd7 Add buffer starvation(bs), deadline(dl), playback rate(pr), startup(su)
Enhanced the Common Media Client Data (CMCD) logging by incorporating additional fields:
* buffer starvation (bs) : CMCD-Status
* deadline (dl) and startup (su) : CMCD-Request
* playback rate (pr) : CMCD-Session

PiperOrigin-RevId: 555553357
2023-08-11 20:23:23 +00:00
tofunmi
d4b452d475 Use ProgressiveMediaSource and period for images
PiperOrigin-RevId: 555424664
2023-08-10 12:30:29 +00:00
rohks
481ea4a274 Enhance continueLoading() API to allow passing more information
Modifies the `SequenceableLoader.continueLoading(long)` method in the `SequenceableLoader` interface to `SequenceableLoader.continueLoading(LoadingInfo)`.

`LoadingInfo` contains additional parameters, including `playbackSpeed` and `lastRebufferRealtimeMs` in addition to the existing `playbackPositionUs`. The additional parameters will allow us to pass these information to Common Media Client Data (CMCD) logging.

By using the `LoadingInfo` object, we ensure future flexibility to add more fields without causing any breaking changes to the API.

PiperOrigin-RevId: 555123961
2023-08-10 12:16:41 +00:00
michaelkatz
aed21e464e Fix last sample detection issue for ClippingMediaSource
If stream is not considered final and `ClippingMediaSource` is read to `endUs`, then `BaseRenderer.readSource` returns `C.RESULT_NOTHING_READ`. In that case, the `lastBufferInStreamPresentationTimeUs` is not set and the last frame is not rendered.

PiperOrigin-RevId: 554418971
2023-08-07 11:34:34 +00:00
tonihei
a3c1d5be9d Reset loading period after reading discontinuity that requires loading
This is the same change as 8655429af7, just on the second call site
of readDiscontinuity. The tests didn't cover this case yet because
they never queued more than 2 items in a playlist.

PiperOrigin-RevId: 553485244
2023-08-07 11:30:38 +00:00
tonihei
69656774f9 Only use first frame stream start position check at stream transitions
We currently only force the first frame if the frame timestamp is
greater than the stream *offset*.

This is wrong for two reasons:
 1. The timestamp and the offset are not comparable and it should be
    the stream start position.
 2. The check should only be applied at stream transitions where we
    need to make sure that a new first frame isn't rendered until we
    passed the transition point.

We have to fix both issues together, because fixing just issue (1)
causes seeks to before the start position to no longer render the
frame (and playback will be stuck). A new test covers this case.

We also amend the stream transition test case to actually test what it
promises to test and add a test for prerolling samples at the
beginning, to ensure the first frame is still renderered.

Issue: androidx/media#291
PiperOrigin-RevId: 552858967
2023-08-07 11:21:08 +00:00
rohks
11648e6c8e Simplify and accurately compute chunk duration
Refactored `CmcdLog` to `CmcdHeadersFactory` for improved representation of its purpose and updated implementations.

#minor-change

PiperOrigin-RevId: 552831995
2023-08-07 11:19:00 +00:00
tonihei
79c8b0e0f8 Set correct start time for newly enabled renderers during playback
We currently pass in the time at which the stream originally started,
but newly enabled renderers should get the current playback position
instead.

PiperOrigin-RevId: 550894630
2023-08-01 14:04:35 +01:00
jbibik
25253698bc TtmlParser implementation - moved from TtmlDecoder
`TtmlDecoder` which used to be `SimpleSubtitleDecoder` will now be called `TtmlParser` and implement SubtitleParser interface. For backwards compatibility, we will have the same functionality provided by `DelegatingSubtitleDecoder` backed-up by a new `TtmlParser` instance.

PiperOrigin-RevId: 549700490
2023-08-01 13:43:12 +01:00
jbibik
1c1b78e635 Test for Decoder with Mp4WebvttParser corrected
PiperOrigin-RevId: 549629289
2023-08-01 13:42:06 +01:00
tonihei
85cde93e6c Remove C2Mp3TimestampTracker
This tracker aims to replicate the behavior of a specific codec to
ensure MediaCodecRenderer correctly detects stream and output
format transitions. The class was needed because MediaCodecRenderer
made assumptions about codec behavior this codec did not fulfil (in
particular, changing timestamps and number of samples).

Since then, MediaCodecRenderer was made more robust to this kind of
codec behavior in general and currently has no assumptions that
require any special handling of this codec. This means we can remove
the workaround completely.

PiperOrigin-RevId: 549610989
2023-08-01 13:39:53 +01:00
ibaker
1cfaae3450 Enable WebVTT subtitles in MkvPlaybackTest
This is no longer flaky because WebVTT subtitles are decoded on the
loading thread (enabled in this test with
`defaultExractorsFactory.setTextTrackTranscodingEnabled(true)` - and
supported for WebVTT since f0f24aa0d4).

PiperOrigin-RevId: 549594291
2023-08-01 13:35:11 +01:00