Commit graph

1433 commits

Author SHA1 Message Date
kimvde
d25a423888 Remove unused parameter from VideoFrameRenderControl.FrameRenderer
PiperOrigin-RevId: 689281689
2024-10-24 01:27:20 -07:00
Copybara-Service
b5615d5e91 Merge pull request #1794 from stevemayhew:p-fix-ntp-time-update-main
PiperOrigin-RevId: 689121191
2024-10-23 15:22:22 -07:00
michaelkatz
f11130d59e Move setVideoOutput message handling to playback thread
PiperOrigin-RevId: 688951964
2024-10-23 07:16:50 -07:00
ibaker
7b66209bca Add missing overrides in DefaultTrackSelector.Parameters.Builder
Also add a test for this to avoid missing any others in future. Also
flesh out the existing test for the deprecated builder, to assert the
return type is correctly updated.

PiperOrigin-RevId: 688948768
2024-10-23 07:08:03 -07:00
kimvde
75f29b6997 Implement DefaultVideoSink.render()
PiperOrigin-RevId: 688948682
2024-10-23 07:05:53 -07:00
michaelkatz
e5133e78f5 Abstract EPII renderer logic into holder class
Move EPII calls to renderers to a separate managing class.

PiperOrigin-RevId: 688932712
2024-10-23 06:04:21 -07:00
kimvde
6c932722e4 Set release control frame rate in DefaultVideoSink
All the calls to the VideoFrameReleaseControl and
VideoFrameRenderControl will be moved to the DefaultVideoSink

PiperOrigin-RevId: 688857699
2024-10-23 01:06:04 -07:00
kimvde
8260bb3d2e Refactor frame rate notification to release control
Frame rate change is currently notified to the release control when the
video frame processor input frame rate changes, but it should be when
the video frame processor output frame rate changes.

PiperOrigin-RevId: 688512300
2024-10-22 05:44:41 -07:00
kimvde
be8c58d51e Use Format object in VideoFrameProcessor
This is to use an existing media3 object rather than creating a new one.

PiperOrigin-RevId: 688481323
2024-10-22 03:46:03 -07:00
Marc Baechinger
70f2d516a0 Add getter/setter and disable re-initialization by default 2024-10-21 19:18:58 +02:00
ibaker
b04b37074b Add video language to DefaultTrackSelector
PiperOrigin-RevId: 688155680
2024-10-21 09:02:19 -07:00
rohks
0ecd35e24c Improve error logging for IllegalClippingException
Added start and end time details to the error message for `REASON_START_EXCEEDS_END`, helping to debug cases where the start time exceeds the end time.

PiperOrigin-RevId: 688117440
2024-10-21 06:45:35 -07:00
Copybara-Service
f2ecca3b6a Merge pull request #1742 from colinkho:trackselection-playwhenready
PiperOrigin-RevId: 688050467
2024-10-21 02:27:20 -07:00
ibaker
49337d9667 Fix some markdown-in-javadoc
PiperOrigin-RevId: 687354846
2024-10-18 10:48:43 -07:00
tonihei
513ebf67b7 Add initial playWhenReady setting and tests
Also includes a new test for a similar logic that already exists
for speed changes.
2024-10-18 18:00:55 +01:00
Colin Kho
e4b32e4e31 Fix javadoc formatting 2024-10-18 18:00:18 +01:00
Colin Kho
943e165f1f Add variable to track playWhenReady status of a TrackSelection instance 2024-10-18 18:00:18 +01:00
ibaker
2f01900e83 Release the Surface at the end of every playback test
Without this an error is logged which obfuscates real test failures.

PiperOrigin-RevId: 687302953
2024-10-18 08:01:50 -07:00
ibaker
64e0397811 De-flake new test for ProgressiveMediaSource.suppressPrepareError
This test was added in b3290eff10

#cherrypick

PiperOrigin-RevId: 686918104
2024-10-17 08:11:01 -07:00
ibaker
49dec5db8b Ignore renderer errors from text/metadata tracks
Before this change:

* With legacy subtitle decoding (at render time), load errors (e.g. HTTP
  404) would result playback completely failing, while parse errors
  (e.g. invalid  WebVTT data) would be silently ignored, so playback
  would continue without subtitles.
* With new subtitle decoding (at extraction time), both load and parse
  errors would result in playback completely failing.

This change means that now neither load nor parse errors in text or
metadata tracks stop playback from continuing. Instead the error'd track
is disabled until the end of the current period.

With new subtitle decoding, both load and parse errors happen during
loading/extraction, and so are emitted to the app via
`MediaSourceEventListener.onLoadError` and
`AnalyticsListener.onLoadError`. With legacy subtitle decoding, only
load errors are emitted via these listeners and parsing errors continue
to be silently ignored.

Issue: androidx/media#1722
PiperOrigin-RevId: 686902979
2024-10-17 07:15:22 -07:00
ibaker
191bc094a5 Propagate events from secondary children in MergingMediaSource
These events are always reported with the primary child period ID,
because this is the same ID used in the parent `MergingMediaSource`'s
Timeline.

This ensures that e.g. loading errors from sideloaded subtitles (which
uses `MergingMediaSource`) are now reported via
`AnalyticsListener.onLoadError`.

It results in non-error events being reported from these children too,
which will result in more `onLoadStarted` and `onLoadCompleted` events
being reported (one for each child).

Issue: androidx/media#1722
PiperOrigin-RevId: 686901439
2024-10-17 07:10:59 -07:00
ibaker
b3290eff10 Allow ProgressiveMediaSource to optionally suppress prepare errors
Use this for sideloaded subtitles, so preparation can still complete
despite an error from e.g. `DataSource.open`. In this case, no subtitle
tracks will be emitted.

Issue: androidx/media#1722
PiperOrigin-RevId: 686888588
2024-10-17 06:20:11 -07:00
michaelkatz
2a1e71b203 Reduce needless loading period resets in clipping mediasource playlists
When a `SampleQueue` is prepared prior to playback, the start position may be less than the timestamp of the first sample in the queue and still be valid. This scenario can come about with specific clipping values and if all samples are sync samples. Currently, with `ClippingMediaPeriods` around `ProgressiveMediaPeriods`, if the `SampleQueue` has already been reset through the seekTo operation in `onPrepared`, then in the aforementioned scenario the seekTo operation in `handleDiscontinuity` will remove all samples and reset the loading periods unnecessarily.

The solution is that if the `ProgressiveMediaPeriod` has already handled a seekTo operation for the same position and the sample queue has not been read yet, then loading does not need to be reset.

The tests in `MergingPlaylistPlaybackTest` were specifically causing this behavior through its setup of `MergingMediaSources` around clipped `FilteringMediaSources`. Since the video content was not 'all sync samples', there would always be a discontinuity to handle and the audio content being 'all sync samples' would start with samples post start time.

These changes also remove the flakiness from the `MergingPlaylistPlaybackTest`.

PiperOrigin-RevId: 686858444
2024-10-17 04:20:50 -07:00
ibaker
d2ccace75c Remove allocator param from MediaSourceTestRunner constructor
Currently every test in the library passes `null` here, which seems to
end up being passed into non-null places in the library. This change
removes the parameter and instead initializes the field to a
`DefaultAllocator` instance in the same way that `DefaultLoadControl`
creates one.

PiperOrigin-RevId: 686823273
2024-10-17 02:00:08 -07:00
kimvde
363f71357b Handle output size changes inside DefaultVideoSink
PiperOrigin-RevId: 686507112
2024-10-16 07:59:35 -07:00
Steve Mayhew
621a9aedba SntpClient periodically re-syncs the offset to NTP server
Many of the Android TV platforms do not report accurate `SystemClock.elapsedRealtime()`
In addition this drift is not consistant accross multiple boxes, so the time offset
between any pair of boxes will vary over time.

This variance will lead to a drift in the Live Offset calculation.

Fix is simple, inValidate the NTP server query after a period of time (10 minutes)
and re-issue the call
2024-10-14 22:03:00 +02:00
kimvde
1084c9ea98 Implement DefaultVideoSink.isReady
PiperOrigin-RevId: 685720088
2024-10-14 08:44:51 -07:00
kimvde
37cd008c01 Remove unnecessary method in VideoFrameRenderControl
PiperOrigin-RevId: 685681192
2024-10-14 06:10:33 -07:00
michaelkatz
1c4ee06ad6 Remove Renderer[] from LoadControl.onTracksSelected
The `DefaultLoadControl` implementation of onTracksSelected only utilizes the `Renderer[]` parameter for use in stream type, of which it can collect from the `ExoTrackSelection[]` parameter.

PiperOrigin-RevId: 685677726
2024-10-14 05:56:24 -07:00
kimvde
638eae44ab Remove unnecessary method in VideoFrameRenderControl
PiperOrigin-RevId: 685632001
2024-10-14 02:34:49 -07:00
kimvde
5eeedeacc6 Implement DefaultVideoSink.flush()
This is part of the effort to delegate the rendering of the VideoGraph
output frames to a DefaultVideoSink.

PiperOrigin-RevId: 685622019
2024-10-14 01:53:52 -07:00
kimvde
7dbacdb011 PlaybackVideoGraphWrapper: simplify flushing logic
PiperOrigin-RevId: 685610127
2024-10-14 01:10:07 -07:00
tianyifeng
98dc7f2def Add DefaultPreloadManager.Builder
The `DefaultPreloadManager.Builder` is able to build the `DefaultPreloadManager` and `ExoPlayer` instances with the consistently shared configurations. Apps can:

* Simply setup the `DefaultPreloadManager` and `ExoPlayer` with all default configurations via `build()` and `buildExoPlayer()`;
* Or customize the shared configurations by the setters on `DefaultPreloadManager.Builder` and setup via `build()` and `buildExoPlayer()`;
* Or customize the player-only configurations for `ExoPlayer` via `buildExoPlayer(ExoPlayer.Builder)`.

PiperOrigin-RevId: 684852808
2024-10-11 08:55:25 -07:00
tonihei
73f97c0371 Allow AudioTrack to be provided by a customizable provider
PiperOrigin-RevId: 684800579
2024-10-11 05:23:07 -07:00
Copybara-Service
019fe0589f Merge pull request #1754 from colinkho:loader-plumbing
PiperOrigin-RevId: 684781854
2024-10-11 04:03:09 -07:00
tonihei
b27cbe60b9 Add release callback and generic class to handle type casting 2024-10-11 11:00:45 +01:00
rohks
c744fe9f8f Recognize IAMF format and enhance channel count constraints
- Updated `DefaultTrackSelector.SpatializerWrapperV32.canBeSpatialized` to handle IAMF format.
- Modified `isAudioFormatWithinAudioChannelCountConstraints` to check for `NO_VALUE` of `channelCount` to improve readability.

Note: `DefaultTrackSelector.SpatializerWrapperV32.canBeSpatialized` is not triggered for the IAMF format due to the unset channel count (`Format.NO_VALUE`). The update ensures completeness.

#cherrypick

PiperOrigin-RevId: 684003980
2024-10-09 05:50:58 -07:00
tonihei
15a6906877 Formatting and javadoc 2024-10-09 13:50:30 +01:00
Colin Kho
2f6d8bf5ba Remove unused import 2024-10-09 13:50:29 +01:00
Colin Kho
10bb2e1501 Allow Injection of custom Executor in ProgressiveMediaSource 2024-10-09 13:50:29 +01:00
Colin Kho
ea837e494b Remove unused Executors import 2024-10-09 13:50:29 +01:00
Colin Kho
f7a1b19001 Allow custom Executor to be supplied to ChunkSampleStream 2024-10-09 13:50:29 +01:00
rohks
4df7216bc0 Prioritize object-based audio in DefaultTrackSelector
Object-based audio is more efficient and flexible than channel-based audio, supporting a broader range of devices. This update makes `DefaultTrackSelector` prefer object-based audio when other factors are equal, ensuring its use whenever possible.

#cherrypick

PiperOrigin-RevId: 683990051
2024-10-09 04:55:36 -07:00
Copybara-Service
5e5d486ef1 Merge pull request #1618 from khouzam:main
PiperOrigin-RevId: 683973733
2024-10-09 03:51:10 -07:00
microkatz
3f44f9a898 Cosmetic changes 2024-10-09 09:41:54 +00:00
kimvde
e234076fdc DefaultVideoSink: implement set/clearOutputSurfaceInfo
This is part of the effort to handle rendering of the frames output by
the VideoGraph in a DefaultVideoSink

PiperOrigin-RevId: 683939818
2024-10-09 01:47:38 -07:00
microkatz
ae363671b5 Moved supportsFormat functional code to private method 2024-10-08 15:55:39 +00:00
microkatz
34f50adcd2 Format with google-java-format 2024-10-08 14:43:55 +00:00
Gilles Khouzam
a772e1525c Add a static rendererSupportsFormat method for MCVR
This refactors the `supportsFormat` method on the MediaCodecVideoRenderer to be a static method that can be called from external components to determine if a video format is supported by the device renderers. Since `context` is the only component that is part of the MCVR, it can easily be obtained externally and passed to the method.

This can help optimize format support decisions ahead of time of a player having been instantiated, such as removing unsupported representations from a dash manifest.

This could also be done for the MCAR but would also require passing in the AudioSink and isn't required at this time.
2024-10-08 14:43:55 +00:00
tianyifeng
fd48dd9ce8 Add PlaybackLooperProvider and make it injectable for ExoPlayer
PiperOrigin-RevId: 683607682
2024-10-08 07:07:13 -07:00
kimvde
5f935ef22e DefaultVideoSink: implement initialize() and isInitialized()
The implementation is straightforward as there is nothing to init.

PiperOrigin-RevId: 683596706
2024-10-08 06:25:47 -07:00
Copybara-Service
62864d5475 Merge pull request #1651 from colinkho:mp-cl
PiperOrigin-RevId: 683548885
2024-10-08 03:28:59 -07:00
kimvde
7c9fede3ad Add DefaultVideoSink and implement straightforward methods
Other methods will be implemented in follow-up CLs

PiperOrigin-RevId: 683538245
2024-10-08 02:49:44 -07:00
tonihei
bf88128383 Formatting and additional call for preloading period 2024-10-08 10:15:04 +01:00
Colin Kho
7b0f83690c Add LoadingInfo as a parameter to continueLoading on MediaPeriodHolder 2024-10-08 10:15:04 +01:00
kimvde
8d0b82dfc3 PlaybackVideoGraphWrapper: rename VideoSinkImpl to InputVideoSink
PlaybackVideoGraphWrapper will soon contain an input and an output video
sink, as the rendering of the VideoGraph output frames will be handled
by a DefaultVideoSink instance.

PiperOrigin-RevId: 683167795
2024-10-07 07:02:12 -07:00
tonihei
f7af58951d Allow signed TTML region origins
The origin may be negative if the subtitle starts off-screen
but extends into the screen for example.

TTML1 defines `tts:origin` in terms of
[`length`](https://www.w3.org/TR/2018/REC-ttml1-20181108/#style-value-length),
which allows a prefix of `+`, `-` or nothing, and it's the
[same in TTML2](https://www.w3.org/TR/2018/REC-ttml2-20181108/#style-value-length).

PiperOrigin-RevId: 682379845
2024-10-04 10:58:42 -07:00
tonihei
4481b3567e Add workaround for codecs not propagating EOS signal.
If a codec received the EOS signal and already returned the last
output buffer, we should expect the output EOS very quickly. If
it doesn't arrive within 100ms, we can proceed to end the stream
manually without waiting any further to prevent cases where the
codec is completely stuck otherwise.

PiperOrigin-RevId: 679633116
2024-09-27 10:07:10 -07:00
tonihei
138a8d65ca Get updated buffered position when calling shouldStartPlayback
The buffered position was last updated before the beginning of
the renderer loop in doSomeWork. As the loading happens on a
background thread, it may have progressed further already
depending on how long it took to run the renderer loop.

It's slightly more correct to pass in an updated value to
shouldStartPlayback so that playback can start quicker if the
buffering is particularly fast.

PiperOrigin-RevId: 679203465
2024-09-26 10:33:40 -07:00
tianyifeng
2dde824bde Fix the flakiness in DefaultPreloadManagerTest
We began to use a different preload thread than the main thread in the `DefaultPreloadManagerTest`, then in the test execution, we should also ensure that the events queued on the preload looper have been executed.

Also, there is an edge case also causing flakiness. Assume that `DefaultPreloadManager` is preloading source B, then the app invalidates and triggers another sequence of preloading, say they are A, B and C. There is possibility that source B finishes preloading (started before `invalidate`) when A starting to preload, then `onPreloadSkipped` will be triggered for source B instead of `onPreloadCompleted`. However, the functional block inside of `onPreloadSkipped` is dispatched asynchronously, and by then it's possible that B starts to preload again at the consequence of A being completed, then the functional block just mentioned may think that the current source matches at B, and will advance the current preloading source to C, without informing the `Listener.onPreloadCompleted` for it. As the result, the `Listener.onPreloadCompleted` can never be triggered for B for the second sequence. To fix this, we should prevent the functional block in `onPreloadCompleted`, `onPreloadError`, `onPreloadSkipped` to be even dispatched, when the source doesn't match the current preloading one.

PiperOrigin-RevId: 679145353
2024-09-26 07:47:35 -07:00
ibaker
2520dd12f9 Fix EMSG typo in HlsChunkSource and ChunkSampleStream
PiperOrigin-RevId: 678870309
2024-09-25 15:33:22 -07:00
tonihei
5879426c07 Check surface validity before configuring it on a codec
This check is also done when setting a surface on ExoPlayerImpl,
but by the time we configure the codec the surface may have become
invalid (e.g. when it is destroyed). Even though we immediately remove
a destroyed surface, we could still accidentally use it before the
removal is processed. To avoid these edge cases, we can simply not
configure the codec with an invalid surface.

PiperOrigin-RevId: 678741425
2024-09-25 09:58:11 -07:00
kimvde
d8dc513431 Apply video composition effects in preview
PiperOrigin-RevId: 678207818
2024-09-24 05:46:33 -07:00
Tianyi Feng
b303498834 Fix the typo in javadoc 2024-09-23 20:53:53 +00:00
Colin Kho
31a540953c Add method to MediaSourceEventListener.EventDispatcher to submit events through a lambda function. This allows clients that implement this interface to submit customized event dispatching logic to the EventDispatcher's listeners 2024-09-23 20:53:52 +00:00
bachinger
ba1cdba403 Preload first period of next window
Allow apps to preload the first period of the next window in
the playlist of `ExoPlayer`. By default playlist preloading is
disabled. To enable preloading,
`ExoPlayer.setPreloadConfiguration(PreloadConfiguration)` can be
called.

`LoadControl` determines when to preload with its implemenation of `shouldContinuePreloading(timeline, mediaPeriodId, bufferedDurationUs)`.
The implementation in `DefaultLoadControl` allows preloading only when
the player isn't currently loading for playback. Apps can override this
behaviour.

Issue: androidx/media#468
PiperOrigin-RevId: 677786017
2024-09-23 07:32:48 -07:00
tonihei
5e3dcea1bf Ramp up volume after AudioTrack flush to avoid pop sound
AudioTrack doesn't automatically ramp up the volume after a flush
(only when resuming with play after a pause), which causes audible
pop sounds in most cases. The issue can be avoided by manually
applying a short 20ms volume ramp, the same duration used by the
platform for the automatic volume ramping where available.

Together with the already submitted 6147050b90, this fixes the
unwanted pop sounds for most cases in the desired way. It only
leaves two cases that are not handled perfectly:
 - If the media file itself contains a volume ramp at the beginning,
   we wouldn't need this additional ramping. Given the extremely
   short duration, this seems ignorable and we can treat it as a
   future feature request to mark the beginning of media in a special
   way that can then disable the volume ramping.
 - For seamless period transitions where we keep using the same
   AudioTrack, we may still get a pop sound at the transition. To
   solve this, we'd need a dedicated audio processor to either ramp
   the end of media down and the beginning of the next item up, or
   apply a very short cross-fade. Either way, we need new signalling
   to identify cases where the media originates from the same source
   and this effect should not be applied (e.g. when re-concatenating
   clipped audio snippets from the same file).

PiperOrigin-RevId: 676860234
2024-09-20 08:55:29 -07:00
ibaker
0b86f89498 Guard DrmSession.requiresSecureDecoder calls with state checks
This method is documented that it may only be called in `STATE_OPENED`
or `STATE_OPENED_WITH_KEYS`. It's possible for it to be called in other
states (like `STATE_ERROR`) without this guard.

Previously this didn't cause issues, but since 9d62845c45
we assume that the `sessionId` is non-null in this method, which results
in an `IllegalStateException` when the documented state restriction is
ignored.

PiperOrigin-RevId: 675969256
2024-09-18 06:41:32 -07:00
ibaker
3e8ecbf564 Remove @DoNotInline annotations
This is no longer needed now our `compileSdk` implies a new-enough AGP
which does this out-lining automatically via R8. See also
https://issuetracker.google.com/345472586#comment7

There's no plan to remove the `ApiXXX` classes, but no new ones need
to be added.

PiperOrigin-RevId: 675940634
2024-09-18 04:47:39 -07:00
michaelkatz
f0fb386224 Add workaround for Galaxy Tab S7 FE device PerformancePoint issue
The Galaxy Tab S7 FE has a device issue that causes 60fps secure H264 streams to be marked as unsupported. This CL adds a workaround for this issue by checking the CDD required support for secure H264 in addition to the current check on standard H264. If the provided performance points do not cover the CDD requirement of support 720p H264 at 60fps, then it falls back to using legacy methods for checking frame rate and resolution support.

Issue: androidx/media#1619
PiperOrigin-RevId: 675920968
2024-09-18 03:29:10 -07:00
Googler
e1c4ecf2d3 Add suitable output checker tests relevant for API 35+ in ExoPlayerTest
PiperOrigin-RevId: 675547156
2024-09-17 06:45:15 -07:00
rohks
9bc89ae989 Ensure track indices are only added when samples are committed
Previously, track IDs were added to `trackIndicesPerSampleInQueuedOrder`
even when the sample was not committed. This caused issues where attempts
to read samples from the `SampleQueue` returned `C.RESULT_READ_NOTHING`,
which led to an exception being thrown due to the assumption that samples
were available to read.

This fix updates the logic to track sample commits by comparing the write index before and after calling `SampleQueue.sampleMetadata`. Track indices are only added if the sample was committed, ensuring accurate sample handling and avoiding exceptions.

PiperOrigin-RevId: 675526115
2024-09-17 05:27:51 -07:00
Rohit Singh
c8aa122e8a Change from internal review 2024-09-16 13:34:21 +01:00
Rohit Singh
9975175700 Forward default methods 2024-09-16 13:07:36 +01:00
Rohit Singh
3c6f1f1e77 Add copyright text and move test to correct package 2024-09-16 12:36:03 +01:00
Rohit Singh
8ca12338f6 Format with google-java-format 2024-09-16 12:31:48 +01:00
Colin Kho
62aef96b7d Format code using google java format 2024-09-16 12:31:48 +01:00
Colin Kho
72e39c91c4 Denote ForwardingRenderer as subject to change 2024-09-16 12:31:48 +01:00
Colin Kho
72f26d79f6 Added a forwarding class for Renderer to support composing custom behavior on Metadata & Text Renderers 2024-09-16 12:31:48 +01:00
tianyifeng
72ae454f67 Use buffered duration from start position to control preload progress
`PreloadMediaSource` allows to have a `startPositionUs` passed when `preload` is called, then in `PreloadControl.onContinueLoadingRequested`, it can be more intuitive to see the buffered duration rather than the absolute buffered position as the preload progress. Similar in `DefaultPreloadManager`, we haven't allowed the apps to set a custom start position for individual sources though, once we add this support, using the "duration from the start position" than the absolute position will be less error-prone, otherwise, it can run into a case that the position that the apps set is smaller than the start position.

PiperOrigin-RevId: 674251362
2024-09-13 05:05:39 -07:00
claincly
bc8d82355f Block secondary audio processing until primary format is configured
PiperOrigin-RevId: 673961235
2024-09-12 12:42:09 -07:00
rohks
f133e8d1f2 Fix preroll sample handling for non-keyframe media start positions
When processing edit lists in MP4 files, the media start position may be a non-keyframe. To ensure proper playback, the decoder must preroll to the preceding keyframe, as these preroll samples are essential for decoding but are not rendered.

Issue: google/ExoPlayer#1659

#cherrypick

PiperOrigin-RevId: 673457615
2024-09-11 11:02:39 -07:00
kimvde
8271a5f920 Rename CompositingVideoSinkProvider and PreviewAudioPipeline
The components are mirror components for video and audio so they should
have a matching name

PiperOrigin-RevId: 673357081
2024-09-11 05:49:35 -07:00
jbibik
8bfa7e2de1 Make StreamVolumeManager take streamType in constructor
Avoids the dispatch of listener notifications of stream type/device volume change during construction of ExoPlayer. That was problematic because we end up blocking on `constructorFinished.blockUninterruptible()`

Issue: androidx/media#1692
PiperOrigin-RevId: 672965552
2024-09-10 08:05:21 -07:00
sheenachhabra
327b1c8ad8 Move getCodecProfileAndLevel to CodecSpecificDataUtil
`Muxer` module needs to use this method, hence moved to common.

This CL also makes `getHevcProfileAndLevel` public because this is used
in `MediaCodecUtil`.

PiperOrigin-RevId: 671739166
2024-09-06 06:52:29 -07:00
ibaker
0933f561b7 Move MCR CryptoException handling to top-level render() method
Handling for `MediaCodec.CryptoException` was originally added only
around calls to `MediaCodec.queueSecureInputBuffer` and
`queueInputBuffer` (because these are the only methods that can throw
this exception). When asynchronous interaction with `MediaCodec` was
added in <unknown commit>, exceptions from `MediaCodec` started being stored
and bubbled out of **later** interactions with `MediaCodecAdapter`. This
means that `MediaCodecRenderer` can now see `CryptoException` thrown
from a different method, like
`MediaCodecAdapter.dequeueInputBufferIndex()`, and this ends up missing
the `catch (CryptoException)` code in `MediaCodecRenderer`. This results
in an "unexpected runtime error" stack trace like [A].

This change fixes the stack trace to:
1. Make it a "renderer exception" instead of "unexpected runtime error"
2. Include the correct DRM error code -> `@PlaybackException.ErrorCode`
   mapping.

You can see the corrected stack trace below [B].

-----

[A] (synthesized from manually throwing a `CryptoException` from
`AsynchronousMediaCodecBufferEnqueuer#doQueueSecureInputBuffer`)

```
playerFailed [eventTime=11.56, mediaPos=10.35, window=0, period=0, errorCode=ERROR_CODE_UNSPECIFIED
  androidx.media3.exoplayer.ExoPlaybackException: Unexpected runtime error
      at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:729)
      at android.os.Handler.dispatchMessage(Handler.java:103)
      at android.os.Looper.loopOnce(Looper.java:232)
      at android.os.Looper.loop(Looper.java:317)
      at android.os.HandlerThread.run(HandlerThread.java:85)
  Caused by: android.media.MediaCodec$CryptoException: Test error message
      at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doQueueSecureInputBuffer(AsynchronousMediaCodecBufferEnqueuer.java:232)
      at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doHandleMessage(AsynchronousMediaCodecBufferEnqueuer.java:196)
      at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.access$000(AsynchronousMediaCodecBufferEnqueuer.java:47)
      at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer$1.handleMessage(AsynchronousMediaCodecBufferEnqueuer.java:93)
      at android.os.Handler.dispatchMessage(Handler.java:107)
      at android.os.Looper.loopOnce(Looper.java:232) 
      at android.os.Looper.loop(Looper.java:317) 
      at android.os.HandlerThread.run(HandlerThread.java:85) 
```

[B]

```
Playback error
  androidx.media3.exoplayer.ExoPlaybackException: MediaCodecAudioRenderer error, index=1, format=Format(0, null, null, audio/mp4a-latm, mp4a.40.2, 134359, en, [-1, -1, -1.0, null], [2, 44100]), format_supported=YES
      at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:649)
      at android.os.Handler.dispatchMessage(Handler.java:103)
      at android.os.Looper.loopOnce(Looper.java:232)
      at android.os.Looper.loop(Looper.java:317)
      at android.os.HandlerThread.run(HandlerThread.java:85)
  Caused by: android.media.MediaCodec$CryptoException: Test error message
      at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doQueueSecureInputBuffer(AsynchronousMediaCodecBufferEnqueuer.java:232)
      at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doHandleMessage(AsynchronousMediaCodecBufferEnqueuer.java:196)
      at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.access$000(AsynchronousMediaCodecBufferEnqueuer.java:47)
      at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer$1.handleMessage(AsynchronousMediaCodecBufferEnqueuer.java:93)
      at android.os.Handler.dispatchMessage(Handler.java:107)
      at android.os.Looper.loopOnce(Looper.java:232) 
      at android.os.Looper.loop(Looper.java:317) 
      at android.os.HandlerThread.run(HandlerThread.java:85) 
```

PiperOrigin-RevId: 670951229
2024-09-04 06:44:21 -07:00
rohks
6e1bab03bd Add Mp4PlaybackTest for sample with edit list (edts box)
PiperOrigin-RevId: 670572646
2024-09-03 08:43:33 -07:00
dancho
1c61fbadf7 Downscale bitmaps during decoding in Transformer
Limit input image size in Transformer to be less than 4096x4096.
For very large images, this can reduce memory usage substantially,
and stays away from `GL_MAX_TEXTURE_SIZE` - often 4096

PiperOrigin-RevId: 670555939
2024-09-03 07:53:13 -07:00
Copybara-Service
854566dbfe Merge pull request #1371 from v-novaltd:dsparano-exo207
PiperOrigin-RevId: 669352191
2024-08-30 09:35:32 -07:00
microkatz
f1654732a6 Added release note for change 2024-08-30 13:47:43 +00:00
tonihei
41a56daab7 Code formatting 2024-08-29 13:48:41 +01:00
Colin Kho
a509033a5c Correct grammar and format javadoc on DrmUtil.executePost 2024-08-29 13:19:27 +01:00
Colin Kho
4ee34cc00e Refactor HttpMediaDrmCallback's executePost into a shared utility method within DrmUtil 2024-08-29 13:19:27 +01:00
tonihei
e8664dbc8e Report initial discontinuity for DASH periods that require preroll
DASH periods don't have to start at the beginning of a segment. In
these cases, they should report an initial discontinuity to let the
player know it needs to expect preroll data (e.g. to flush renderers)

This information is only available in the ChunkSampleStream after
loading the initialization data, so we need to check the sample
streams and tell them to only report discontinuities at the very
beginning of playback. All other position resets are triggered by
the player itself and don't need this method.

Issue: androidx/media#1440
PiperOrigin-RevId: 668831563
2024-08-29 01:57:15 -07:00
kimvde
8367e420ad Import correct nullable annotation
PiperOrigin-RevId: 668521379
2024-08-28 10:54:29 -07:00
tianyifeng
c2e81052e8 Fix flakiness in ExoPlayerTest
In the flaky test `ExoPlayerTest.loading_withLargeAllocationCausingOom_playsRemainingMediaAndThenThrows`, it is indeterministic that when the message `MSG_IO_EXCEPTION` from the loader thread will arrive on the playback looper. If it arrives after `MSG_PERIOD_PREPARED`, then the period can continue loading and get the three samples written to the `SampleQueue` before the intentional OOM surfacing to the `ExoPlayerImplInternal`, otherwise, the OOM will be detected by `ExoPlayerImplInternal` very early and the player disallows to load three samples, which will cause the assertion to fail.

As we are expecting the three samples to play until the playback fails, we should assume that the `Loader` encounters the OOM after those samples loaded, thus we need to put this trigger a bit later until the `SampleStreamItem`s are handled by the `FakeSampleStream`. This could be checked by the return value of `FakeMediaPeriod.continueLoading` (super class implementation). However, `FakeMediaPeriod.continueLoading` originally always returns `true`, which is not aligned with the javadoc of `MediaPeriod.continueLoading`:

"return `true` if progress was made, meaning that `getNextLoadPositionUs()` will return a different value than prior to the call, `false` otherwise."

then we should also modify that logic.

PiperOrigin-RevId: 668438316
2024-08-28 07:16:29 -07:00
microkatz
0ac26c4004 Format with google-java-format 2024-08-28 12:41:49 +00:00
Daniele Sparano
7473156853 Fix output pixel aspect ratio and add test for it 2024-08-28 12:41:49 +00:00
Daniele Sparano
f0463bff8a Set pixel aspect ratio from output media format if set 2024-08-28 12:41:49 +00:00
tonihei
d0676245b5 Add AnalyticsListener.onRendererReadyChanged
This callback allows listeners to track when individual renderers
allow or prevent playback from being ready. For example, this is useful
to figure out which renderer blocked the playback the longest.

PiperOrigin-RevId: 667970933
2024-08-27 07:08:42 -07:00
kimvde
a4d0735d4c Fix player hanging when seeking at MediaItem transition
Before this CL, the video sink was stuck if a flush was executed:
- after VideoSink.onInputStreamChanged, which is setting
  pendingInputStreamBufferPresentationTimeUs and
- before the previous stream was fully rendered.

This is because pendingInputStreamBufferPresentationTimeUs was not reset
to TIME_UNSET when flushing the sink, so that the sink was still waiting
for the last frame of the previous stream to be rendered in
handleInputFrame/Bitmap.

PiperOrigin-RevId: 667924517
2024-08-27 04:00:46 -07:00
tonihei
6f0cb539d0 Add util method BaseRenderer.getStreamOffsetUs
This simplifies the common pattern of saving this offset in
onStreamChanged to use it at a later point.

PiperOrigin-RevId: 666226263
2024-08-22 00:53:04 -07:00
tianyifeng
410b26fba1 Detect SampleStream error after PreloadMediaPeriod has prepared
Per the javadoc, the method `MediaPeriod.maybeThrowPrepareError` is only allowed to be called before the period has completed preparation. For later errors in loading the streams, `SampleStream.maybeThrowError` will be called instead.

PiperOrigin-RevId: 665831430
2024-08-21 05:16:42 -07:00
tianyifeng
351593a250 Allow playback regardless buffered duration when loading fails
It is possible for playback to be stuck when there is failure in loading further data, while the player is required to load more due to the buffered duration being under `DefaultLoadControl.bufferForPlayback`. Therefore, we check if there is any loading error in `isLoadingPossible`, so that the player will allow the playback of the existing data rather than waiting forever for the data that can never be loaded.

Issue: androidx/media#1571
#cherrypick
PiperOrigin-RevId: 665801674
2024-08-21 03:37:42 -07:00
Copybara-Service
1bb48b1c47 Merge pull request #1616 from colinkho:chmain
PiperOrigin-RevId: 665779059
2024-08-21 02:32:16 -07:00
Copybara-Service
5304f227a7 Merge pull request #1624 from colinkho:comain
PiperOrigin-RevId: 665430514
2024-08-20 10:47:43 -07:00
Ian Baker
9c25845cd7 Restore a couple of blank lines 2024-08-20 17:47:48 +01:00
Ian Baker
a6f5d3daf5 Format with google-java-format 2024-08-20 17:47:48 +01:00
Colin Kho
0a0444b9a5 Use Suppliers.memoize for lazy instantiation of AudioManager 2024-08-20 17:47:47 +01:00
Colin Kho
6af92b0af3 Differ AudioManager retrieval to whenever AudioFocusManagement is required 2024-08-20 17:47:47 +01:00
Copybara-Service
24f3856ea4 Merge pull request #1580 from colinkho:main
PiperOrigin-RevId: 665330328
2024-08-20 06:17:20 -07:00
Ian Baker
41430aaa0c Fix annotation position and clarify javadoc 2024-08-20 13:42:00 +01:00
ibaker
b04b270c2b Use SubtitleExtractor from MediaParserChunkExtractor
This uses a 'bundled' extractor (`SubtitleExtractor`) because
`MediaParser` doesn't support transcoding subtitles to
`application/x-media3-cues`. This transcoding is required to support
non-legacy subtitle handling where they are parsed during extraction,
instead of during rendering.

PiperOrigin-RevId: 665282298
2024-08-20 03:54:17 -07:00
Ian Baker
af5df6e700 Add retryCount parameter to make it clear onLoadStarted is called for retries too 2024-08-20 11:28:18 +01:00
Colin Kho
33d9b78e21 Fix javadoc of onLoadStarted & remove error information as fields 2024-08-20 11:13:41 +01:00
Colin Kho
1cc1bf02ef Added onLoadStarted callback to Loader's callback 2024-08-20 11:13:41 +01:00
kimvde
266f16823f Remove VideoSink.isFrameDropAllowedOnInput
This is an unnecessary layer

PiperOrigin-RevId: 665267029
2024-08-20 03:09:13 -07:00
Colin Kho
f9f3e67ee2 Include initialization chunk wording in getChunkIndex method 2024-08-19 10:31:34 -07:00
Colin Kho
4a6c5c7598 Cache ChunkIndex once load is done 2024-08-19 10:28:14 -07:00
ibaker
3600d04719 Deprecate SingleSampleMediaChunk
This is only used as part of legacy subtitle decoding. If any apps are
using it directly (rather than via `DashMediaSource` or `SsMediaSource`)
they probably need to enable legacy decoding in `TextRenderer` to avoid
playback failures.

PiperOrigin-RevId: 663740343
2024-08-16 08:36:38 -07:00
Colin Kho
1c1fd4d8b8 Add getter for ChunkIndex on InitializationChunk 2024-08-15 10:00:56 -07:00
tonihei
b58a1d16f7 Split DefaultAudioSink.writeBuffer in two seperate steps
The method currently has two ways of operating with a complicated
contract:
 1. outputBuffer == null -> sets a new outputBuffer and starts
    draining it as far as possible.
 2. outputBuffer != null -> continues draining a previous buffer,
    requires that the input parameter is the same as outputBuffer.

This contract can be made cleaner by splitting the logic into
(1) setting a new output buffer and (2) draining the output buffer.

Only one parameter is needed for each method and there is no
requirement to pass in the same argument as before because we already
know what we are draining. This also means the caller of the method
doesn't have to keep track of the actual outputBuffer, allowing
further processing steps in the future that change the drained
instance.

PiperOrigin-RevId: 663316251
2024-08-15 08:33:31 -07:00
ktrajkovski
fc18ad3135 Add LibiamfAudioRenderer's constructor to proguard rules.
Making a change to the `proguard-rules.txt` also requires a test added as `IamfModuleProguard`.

PiperOrigin-RevId: 663239150
2024-08-15 02:55:38 -07:00
ibaker
9d62845c45 Pass explicit securityLevel into MediaDrm.requiresSecureDecoder
Previous to this change, `FrameworkMediaDrm.requiresSecureDecoder`
ignores its `sessionId` parameter on API 31+, and uses only the
`mimeType` parameter. This means the result [assumes the session is
opened at the 'default security
level'](https://developer.android.com/reference/android/media/MediaDrm#requiresSecureDecoder(java.lang.String)):
> The default security level is defined as the highest security level
> supported on the device.

This change is a no-op in all (?) cases, because the `ExoMediaDrm`
interface only exposes the zero-arg `openSession()` method, which in the
framework case **also** assumes the highest security level is preferred:
> By default, sessions are opened at the native security level of the
> device.

However, it seems more obviously correct to only make this
"highest/native security level" assumption in one place
(`openSession()`), and check the session's **actual** security level
everywhere else.

Issue: androidx/media#1603
PiperOrigin-RevId: 662872860
2024-08-14 04:52:33 -07:00
ibaker
ca455ee858 Check WV version before relying on MediaDrm.requiresSecureDecoder
This method was added in API 31 (S) but it's non-functional
(incorrectly, silently, returns `false`) on the Widevine plugin version
(`16.0`) from R (API 30), which some devices up to at least API 34 are
still using.

This results in ExoPlayer incorrectly selecting an insecure decoder for
L1 secure content, and subsequently calling
`MediaCodec.queueInputBuffer` instead of `queueSecureInputBuffer`,
which is not supported and generates the following error:
> Operation not supported in this configuration: ERROR_DRM_CANNOT_HANDLE

Issue: androidx/media#1603

#cherrypick

PiperOrigin-RevId: 662852176
2024-08-14 03:27:50 -07:00
kak
e7f037bff5 Automated Code Change
PiperOrigin-RevId: 662793513
2024-08-13 23:32:49 -07:00
tianyifeng
cd532c5fb2 Handle preload callbacks asynchronously in PreloadMediaSource
When there is an exception thrown from the `LoadTask`, the `Loader` will call `Loader.Callback.onLoadError`. Some implementations of `onLoadError` method may call `MediaPeriod.onContinueLoadingRequested`, and in the `PreloadMediaSource`, its `PreloadMediaPeriodCallback` will be triggered and then it can further call `continueLoading` if it finds needed. However the above process is currently done synchronously, which will cause problem. By calling `continueLoading`, the `Loader` is set with a `currentTask`, and when that long sync logic in `Loader.Callback.onLoadError` ends, the `Loader` will immediately retry, and then a non-null `currentTask` will cause the `IllegalStateException`.

Issue: androidx/media#1568

#cherrypick

PiperOrigin-RevId: 662550622
2024-08-13 09:45:35 -07:00
ktrajkovski
92cff64321 Add spatial effects to IAMF support in Exoplayer.
Check if the output device supports spatialization for the requested output format. If so, return a stream decoded for 6 channels in a 5.1 layout. Otherwise, return a stream decoded for 2 channels in a binaural layout.

PiperOrigin-RevId: 662546818
2024-08-13 09:34:50 -07:00
ibaker
0411e1937b Deprecate SingleSampleMediaSource
This is an additional signal that legacy subtitle support needs to be
explicitly enabled, and is going away at some point.

PiperOrigin-RevId: 661305694
2024-08-09 10:07:14 -07:00
kak
2202397758 Automated Code Change
PiperOrigin-RevId: 660491742
2024-08-07 12:28:25 -07:00
tonihei
b00e018697 Add ImageDecoder for external image loading libraries
The integration with external libraries like Glide or Coil is
currently extremely complicated. Providing the boilerplate code
to handle the ImageDecoder interface and a better hook for custom
image decoders in DefaultRenderersFactory allows apps to easily
inject their own logic for external image loading libraries.

PiperOrigin-RevId: 659508914
2024-08-05 04:36:16 -07:00
rohks
8aab324fb4 Add the getSampleSize() API to MediaExtractorCompat
This API allows users to retrieve the sample size, ensuring they can allocate sufficient buffer capacity before utilizing the `readSampleData(ByteBuffer buffer, int offset)` method to read data.

PiperOrigin-RevId: 658772408
2024-08-02 06:33:06 -07:00
ktrajkovski
01593a9c1f Replace RuntimeException with IllegalStateException.
Refactor to replace instances of `RuntimeException` with `IllegalStateException`.
This change ensures exceptions can be handled more specifically without
unintended catches of unrelated exceptions.

PiperOrigin-RevId: 658337459
2024-08-01 03:20:54 -07:00
Ian Baker
c637774cc2 Take Executor instead of ExecutorService
This also avoids shutting down the externally-provided ExecutorService,
which we shouldn't do (since we don't really own it, and didn't create
it).
2024-07-31 10:37:14 +01:00
Ian Baker
ff22838c0d Add javadoc summary fragment 2024-07-31 10:00:01 +01:00
Colin Kho
68e65ec884 Allow custom ExecutorService to be supplied to the Loader 2024-07-30 13:20:30 -07:00
ktrajkovski
04bfeec751 Add decoding functions to IamfDecoder and LibiamfAudioRender.
PiperOrigin-RevId: 657621223
2024-07-30 09:33:31 -07:00
Copybara-Service
e9787c4196 Merge pull request #1566 from colinkho:main
PiperOrigin-RevId: 657571792
2024-07-30 06:42:22 -07:00
rohks
8b7b1b51a9 Add MediaDataSourceAdapter
Added a new data source which acts an adapter to read media data from platform `MediaDataSource`. This enables adding the `setDataSource(MediaDataSource)` API to `MediaExtractorCompat`.

PiperOrigin-RevId: 657564901
2024-07-30 06:16:30 -07:00
rohks
867e9ea2da Add APIs to set data source using content URI, file path or HTTP URL
Added three `setDataSource` APIs in `MediaExtractorCompat`:
- `setDataSource(Context context, Uri uri, @Nullable Map<String, String> headers)` to set data source with a content URI and optional headers.
- `setDataSource(String path)` to set data source using a file path or HTTP URL.
- `setDataSource(String path, @Nullable Map<String, String> headers)` to set data source using a file path or HTTP URL with optional headers.

PiperOrigin-RevId: 657563973
2024-07-30 06:12:56 -07:00
Ian Baker
335d5e7a1d Add some tests to TrackSelectionUtilTest 2024-07-29 15:12:23 +01:00
Colin Kho
0ead7bb221 Refactor getMaxVideoSizeInViewport into TrackSelectionUtil so it can be reused in external code 2024-07-29 15:12:23 +01:00
Googler
f6dc02fa6a Unsuppress/suppress playback on suitable media output updates
PiperOrigin-RevId: 657111555
2024-07-29 01:43:46 -07:00
Copybara-Service
ccf704b30b Merge pull request #1549 from MGaetan89:min_sdk_21
PiperOrigin-RevId: 656358426
2024-07-26 05:11:46 -07:00
Googler
300453820c Selectable builtin speaker support for Wear OS
The builtin speaker is to be supported as a suitable output when that is deliberately selected for the media playback by the user in Wear OS.

PiperOrigin-RevId: 655950824
2024-07-25 07:42:31 -07:00
rohks
80202bc9f2 Use getDeclaredLength() for setting length of AssetFileDescriptor
Should have been part of the change: 0ac90855b4.

PiperOrigin-RevId: 655873821
2024-07-25 02:05:14 -07:00
tianyifeng
edd3a3f349 Fix bug where BasePreloadManager.Listener invokes from incorrect thread
The DefaultPreloadManagerTest didn't to catch this because we use main looper as the preload looper in the tests. This CL also improves the tests by assigning the preload looper with one that corresponds to a different thread.

PiperOrigin-RevId: 655664189
2024-07-24 12:41:21 -07:00
Gaëtan Muller
28edfcbb69 Remove unnecessary SDK_INT checks 2024-07-24 16:17:01 +01:00
Gaëtan Muller
7289764a65 Remove unnecessary @RequiresApi 2024-07-24 16:17:00 +01:00
Gaëtan Muller
71b8c32a6f Remove Multidex 2024-07-24 16:17:00 +01:00