Initialising the fields as Integer and then getting a String on compute time is slow. Instead we directly initialise these fields as String. Improves the time taken in bundling PlayerInfo further to less than 200ms from ~300ms.
Also modified a test to improve productive coverage.
PiperOrigin-RevId: 500003935
This brings together the multiple details about a muxer track, and
reduces the need for additional variables and more complicated track
tracking.
PiperOrigin-RevId: 499872145
Improves the time taken to construct playerInfo from its bundle from ~400 ms to ~300 ms.
Also made `Timeline.Window.toBundle(boolean excludeMediaItem)` public as it was required to assert a condition in tests.
PiperOrigin-RevId: 499512353
The old names are not really correct anymore because:
- The Audio/VideoTranscodingSamplePipelines do not decode anymore.
- The pipelines now mux the encoded data.
PiperOrigin-RevId: 499498446
Whilst testing fallback functionality, I found that we were
aggressively reducing the resolution if it was not supported. A quick
test found that we could reduce by a much smaller increments.
Performance wise it appears these checks are incredibly quick.
The code for checking supported sizes was duplicated, with one case
having a bug because of this duplication (2/3 case). This CL abstracts
this into a loop.
PiperOrigin-RevId: 499497646
Added another check in each of these tests to make sure we don't add keys to bundle for fields with default values.
Also fixed comments of similar changes in `AdPlaybackStateTest` and `MediaMetadataTest`.
PiperOrigin-RevId: 499463581
Based on experimentation it seems that buffers can occasionally (roughly 1% of test runs) be dropped when rendering off-screen from EGL on the emulator. Specifically, in this test, sometimes after rendering three buffers with distinct timestamps only the first and third buffers' timestamps are handled in the `ImageReader`'s image available callback causing the assertion checking all frames rendered to fail. This behavior seems to be independent of the nanosecond presentation time attached to the buffers (as expected for off-screen rendering).
Introducing a pause of 1 second between rendering each frame reduces the flake rate to around 1/2000. This increases the run time of some of the tests, so this change also removes the 5 second `FRAME_PROCESSING_WAIT_MS` (it seems to be unnecessary when rendering off-screen) and instead uses a latch to wait until the frame processor has handled 'end of stream'.
PiperOrigin-RevId: 499440591
When a listener method is deprecated, the new method should (by
default) called through to the deprecated one. This is because any
class that implements the method that is now deprecated needs to still
receive that callback.
It appears when onTransformationError(MediaItem, Exception) was
deprecated in favour of onTransformationError(MediaItem,
TransformationException), this deprecation was the wrong way round, and
the newer callback - onTransformationError(MediaItem,
TransformationResult, TransformationException) continued this mistake.
This CL now corrects this.
PiperOrigin-RevId: 498221504
Events on the wrapper should be propagated to TransformerInternal as
soon as they occur, switching round the process so TransformerInternal
does not have to query MuxerWrapper.
This CL is a prerequisite for the child CL, where MuxerWrapper can
simplify the internal state and logic.
PiperOrigin-RevId: 497267202
Before, it was possible for onDurationUs() and onAllTracksRegistered()
to be called before onTrackAdded() because they are called from
different threads. onDurationUs() and onAllTracksRegistered() are called
from the Transformer internal thread, and onTrackAdded() is called from
the playback thread.
PiperOrigin-RevId: 497102556
Did not do this optimisation for `AdPlaybackState.AdGroup` as its length is zero for `AdPlaybackState` with no ads.
No need to pass default values while fetching keys, which we always set in `AdPlaybackState.AdGroup.toBundle()`.
PiperOrigin-RevId: 496995048
AssetLoader declares the tracks with a setTrackCount() method. Setting
the track count on the MuxerWrapper is easier than calling
registerTrack() as many times as the number of tracks.
PiperOrigin-RevId: 496933501
AssetLoader declares the tracks with a setTrackCount() method. Setting
the track count on the FallbackListener is easier than calling
registerTrack() as many times as the number of tracks.
PiperOrigin-RevId: 496919969
The TransformationRequest passed to FallbackListener (and
createSupportedTransformationRequest) can have null for values that
are inferred from source. Within fallback, this can be height, width,
video mime type and audio mime type (HDR mode is not linked to source).
requestedFormat has these values populated from source, and
supportedFormat then finds the closest supported values to the
requested.
If any of the values in supportedFormat do not match the
requestedFormat, then this method would build upon the
TransformationRequest and update ALL possible fallback fields. This is
a problem because the fallback listener compares the original request
to the fallback one and notifies about all the fields that have
changed.
This CL changes this so that only the values that are not the same as
requested are changed in the supported request that is given to the
fallback listener.
PiperOrigin-RevId: 496908492
When trying to run the test on Android Studio, error "incompatible
types: Buffer cannot be converted to ByteBuffer" is logged. This is
because ByteBuffer.flip() returns a Buffer (and not a ByteBuffer).
Annotation @CovariantReturnType on ByteBuffer.flip() should resolve this
automatically but it doesn't seem supported at the moment.
PiperOrigin-RevId: 496894723
There are two overloads of this method due to a type 'rename' from
`PlayerControlView.VisibilityListener` to
`PlayerView.ControllerVisibilityListener`. Currently when you call one
overload it passes `null` to the other one (to clear the other listener).
Unfortunately this results in it clearing itself, because it receives
a null call back!
This change tweaks the documentation to clarify that the 'other'
listener is only cleared if you pass a non-null listener in. This solves
the recursive problem, and allows the 'legacy' visibility listener to be
successfully registered.
Issue: androidx/media#229
#minor-release
PiperOrigin-RevId: 496876397
Users of this class may run into these assertions when creating the
State and they need to check the source code to understand why
the State is invalid. Adding error messages to all our correctness
assertions helps to understand the root cause more easily.
PiperOrigin-RevId: 496875109
This CL replaces concurrent collections and atomic primitives with a single
lock, this way the code is easier to reason about.
PiperOrigin-RevId: 496718057
Note that we simply use GlEffectsFrameProcessor in-app / GL tone-mapping, so PQ->SDR tone-mapping isn't yet implemented.
Tested manually using the demo on Pixel 7, to confirm that device and in-app tone
mapping behave similarly.
PiperOrigin-RevId: 496700231
Adds the AudioMixerAlgorithm interface which allows for specialized
implementations of audio mixing that also efficiently convert between
source and mixing formats.
Initial implementation has two algorithms:
1. Float -> float (with channel mixing)
2. S16 -> float (with channel mixing)
PiperOrigin-RevId: 496686805
To do that, rename PROGRESS_STATE_NO_TRANSFORMATION to
PROGRESS_STATE_NOT_STARTED and update Javadoc of ProgressState to not be
Transformer specific.
PiperOrigin-RevId: 496653460
This should allow us to focus on HDR failures instead of network buffering
failures when debugging HDR issues.
These files are each used on several files, so it should be more worth the
test binary impact to move these files to local first.
Locally, tests did take less time after this diff
PiperOrigin-RevId: 496398130
The MediaSourceFactory won't be used by the other AssetLoaders
In order to do that, ExoPlayerAssetLoader has been made public, and the
DefaultAssetLoaderFactory has become a wrapper around
ExoPlayerAssetLoader.
PiperOrigin-RevId: 496386853
These are the remaining setter operations. They all share the same
logic that handles playlist and/or position changes. The logic to
create the placeholder state is mostly copied from ExoPlayerImpl's
maskTimelineAndPosition and getPeriodPositonUsAfterTimelineChanged.
PiperOrigin-RevId: 496364712
FrameProcessor already support using different transfer function for input and
output color. This CL has two major changes:
- Create an eglSurface that recognizes BT.2020 PQ
- This requires a separate extension that works only after 33
- So we current throw, if input is HDR, and this extension doesn't work
- Create FrameProcessor with PQ output transfer function
PiperOrigin-RevId: 496023758
The tunneling callbacks are sent via Handler messages and may be
handled after the codec/surface was changed or released.
We already guard against the codec/surface change condition by
creating a new listener and verifying that the current callback
happens for the correct listener instance, but we don't guard
against a released codec yet.
PiperOrigin-RevId: 495882353
Some Player methods operate relative to existing indices in the
playlist (add,remove,move,seek). As these operations may be issued
from a place with a stale playlist (e.g. a controller that sends
a command while the playlist is changing), we have to handle out-
of-bounds indices gracefully. In most cases this is already
documented and implemented correctly. However, some cases are not
documented and the existing player implementations don't handle
these cases consistently (or in some cases not even correctly).
PiperOrigin-RevId: 495856295
The `MediaItem` instances in the following cases are not actually empty but acts as a placeholder. `EMPTY_MEDIA_ITEM` can also be confused with `MediaItem.EMPTY`.
PiperOrigin-RevId: 495843012
Improves the time taken to construct `playerInfo` from its bundle from ~450 ms to ~400 ms. Each `MediaItem` inside `Timeline.Window` contains `MediaMetadata` and hence is a good candidate for bundling optimisations. There already exists a test to check all parameters for null values when unset.
PiperOrigin-RevId: 495614719
It covers the following cases:
| From/To | `null` | `surface 0` | `surface 1` |
|-------------|--------|-------------|-------------|
| `null` | 🆖 | 📺 | 📺 |
| `surface 0` | ❌ | 🔁 | 📺 |
| `surface 1` | ❌ | 📺 | 🔁 |
Where
- 🆖 means NOP
- ❌ means
- Set `null` on FrameProcessor, effectively dropping all frames
- 📺 means
- Notify the listener of video size
- Set FrameProcessor output surface and size when MSG_SET_VIDEO_OUTPUT_SIZE is received
- 🔁 means
- Notify the listener of video size
PiperOrigin-RevId: 495477620
This simplifies some position tracking needs for an app implementing
SimpleBasePlayer.
- The period index can always be derived from the media item index
and the position. So there is no need to set it separately.
- The media item index can be left unset in the State in case the app
doesn't care about the value or wants to set it the default start
index (e.g. while the playlist is still empty where UNSET is
different from zero).
- Similarly, we should allow to set the content position (and buffered
position) to C.TIME_UNSET to let the app ignore it or indicate the
default position explictly.
PiperOrigin-RevId: 495352633
Otherwise, the decoders are not captured. It works at the moment for the
video decoder because decoding is still done on the sample pipeline but
it will moved to the AssetLoader soon.
PiperOrigin-RevId: 495275575
`AudioProcessor`s expect direct buffers. This shouldn't make any functional difference in our code, but a custom audio processor might try to access the buffer from JNI in which case a direct byte buffer is more efficient.
PiperOrigin-RevId: 495241669
`SilentAudioGenerator` could output a fractional audio frame, and this could cause downstream components to throw because of trying to read a complete audio frame but only seeing a partial one.
Calculate the output buffer size based on the frame size (which is a no-op for stereo 16-bit audio) and calculate a total number of frames to output then multiple by the frame size.
PiperOrigin-RevId: 494992941
isLoading is not allowed to be true when IDLE, so we have to set to
false when stopping in case it was set to true before.
PiperOrigin-RevId: 494975405
BasePlayer simplifies implementations by handling all the various
seek methods and forwarding to a single method that can then be
implemented by subclasses. However, this loses the information about
the concrete entry point used for seeking, which is relevant when
the subclass wants to verify or filter by Player.Command. This
can be improved by adding the command as a new parameter. Since
we have to change the method anyway, we can also incorporate the
boolean flag about whether the current item is repeated to avoid
the separate method.
PiperOrigin-RevId: 494948094
- Use a single `VideoSize` instance instead of four primitive fields.
- Clarify that the reported size is the decoded size, that is the encoded video
size.
PiperOrigin-RevId: 494148190
The duration in TransformerInternal.ComponentListener is set on the
Transformer internal thread, and is read on the playback thread. Making
this field volatile ensures that the playback thread reads the updated
value.
PiperOrigin-RevId: 493908385
Some Player methods like getting the Looper and adding listeners
were always allowed to be called from any thread, but this is
undocumented. This change makes the threading rules of these
methods more explicit.
Removing listeners was never meant to be called from another thread
and we also don't support it safely because final callbacks may
be triggered from the wrong thread. To find potential issues, we
can assert the correct thread when releasing listeners.
Finally, there is a potential race condition when calling addListener
from a different thread at the same time as release, which may lead to
a registered listener that could receive callbacks after the player is
released.
PiperOrigin-RevId: 493843981
An AndroidTest is needed to test the message sending from ExoPlayerImpl to
a video renderer (MCVR in this case). The test will be added later.
PiperOrigin-RevId: 493602259
The folder type has a mix of information about the item. It shows
whether the item is browsable (type != FOLDER_TYPE_NONE) and
which Bluetooth folder type to set for legacy session information.
It's a lot clearer to split this into a boolean isBrowsable and
use the existing mediaType to map back to the bluetooth folder type
where required.
folderType is not marked as deprecated yet as this would be an API
change, which will be done later.
PiperOrigin-RevId: 493544589
This adds support for the release handling. To align with the
established behavior in ExoPlayer, the player can only call
listeners from within the release methods (and not afterwards)
and automatically enforces an IDLE state (without listener call)
in case getters of the player are used after release.
PiperOrigin-RevId: 493543958
Rename ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED to
ERROR_CODE_ENCODING_FORMAT_UNSUPPORTED.
This makes the error code more consistent with ERROR_CODE_DECODING_FAILED on the
decoding side. Also, the error code is in the "Encoding errors (4xxx)" section,
so muxer errors probably should be in the "Muxer errors (7xxx)" section instead.
Additionally, no muxer errors currently seem to use
ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED, so this should be a safe change.
PiperOrigin-RevId: 493322880
This is necessary to move video slow motion flattening to the
AssetLoader because this step can change the duration. As we use the
duration before flattening to calculate the progress, we must also use
the position before flattening.
PiperOrigin-RevId: 493291990
These terms are easier to understand, and make sense in the context of
MatrixTextureProcessor now that a MatrixTextureProcessor may have a
different input and output transfer.
PiperOrigin-RevId: 493265980
Modify FrameProcessor and MatrixTextureProcessor interfaces to support
different input and output color transfers. Does not implement conversion between
color ranges (ex. HDR and SDR), but should allow for conversion between color
transfers of the same color range (ex. HLG and PQ).
This supports in-app tone mapping, where we need a single FrameProcessor to
input HDR color transfers (ex. HLG/PQ) and output SDR (ex. gamma2.2). This also
supports previewing, where we need a single FrameProcessor to be able to input HLG
and output PQ.
Manually tested by confirming colors still look right on SDR and HDR videos
with a rotation and color affect applied.
PiperOrigin-RevId: 493108678
From this CL on, FrameProcessor listeners will be invoked from an Executor that
is passed in when creating the FrameProcessor.
GlTextureProcessor needs to invoke the ErrorListener on the said Executor too.
PiperOrigin-RevId: 493018583
These have the same value (`-1`), and basically the same meaning (offset
in an array/list/file/byte stream/etc), but 'position' is an overloaded
term in a media playback library, and there's a risk people assume that
methods like `Player.getCurrentPosition()` may return
`C.POSITION_UNSET`, when in fact unset media times (whether duration or
position) are always represented by `C.TIME_UNSET` which is a) a `long`
(not `int`) and b) a different underlying value. (aside:
`getCurrentPosition()` never returns an unset value, but it's a good
example of the ambiguity of the word 'position' between 'byte offset'
and 'media timestamp'.)
PiperOrigin-RevId: 492493102
This inconsistency was exposed by an upcoming change to deprecate
`POSITION_UNSET` in favour of `INDEX_UNSET` because position is an
ambiguous term between 'byte offset' and 'media position', as shown
here.
PiperOrigin-RevId: 492470241
Discovered while investigating Issue: google/ExoPlayer#10823
Example stack trace with the previous code (I added the index value for
debugging):
```
playerFailed [eventTime=44.07, mediaPos=44.01, window=0, period=0, errorCode=ERROR_CODE_FAILED_RUNTIME_CHECK
androidx.media3.exoplayer.ExoPlaybackException: Unexpected runtime error
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:635)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loopOnce(Looper.java:202)
at android.os.Looper.loop(Looper.java:291)
at android.os.HandlerThread.run(HandlerThread.java:67)
Caused by: java.lang.IllegalArgumentException: index=-1
at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:55)
at androidx.media3.extractor.text.webvtt.WebvttSubtitle.getEventTime(WebvttSubtitle.java:62)
at androidx.media3.extractor.text.SubtitleOutputBuffer.getEventTime(SubtitleOutputBuffer.java:56)
at androidx.media3.exoplayer.text.TextRenderer.getCurrentEventTimeUs(TextRenderer.java:435)
at androidx.media3.exoplayer.text.TextRenderer.render(TextRenderer.java:268)
at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1008)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:509)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loopOnce(Looper.java:202)
at android.os.Looper.loop(Looper.java:291)
at android.os.HandlerThread.run(HandlerThread.java:67)
]
```
#minor-release
PiperOrigin-RevId: 492464180
This is necessary to move video decoding to the AssetLoader. Otherwise,
if the decoder max pending frame count is reached, the AssetLoader will
stop queuing frames to the pipeline, and process data will not be called
anymore.
PiperOrigin-RevId: 492392621
In some cases we split a test method, and in other cases we just add
line breaks to make the separation between arrange/act/assert more
clear.
PiperOrigin-RevId: 492182769
This adds the forwarding logic for most setters in SimpleExoPlayer
in the same style as the existing logic for setPlayWhenReady.
This change doesn't implement the setters for modifying media items,
seeking and releasing yet as they require additional handling that
goes beyond the repeated implementation pattern in this change.
PiperOrigin-RevId: 492124399
EncoderUtil using a static list to cache encoders means it's not possible to
use a different set of encoders for different tests when running all
robolectric tests together.
PiperOrigin-RevId: 491959350
This better matches the terminology we use elsewhere in the Player
interface, where items inside the playlist are referred to as
"media item" and only the entire list is called "playlist".
PiperOrigin-RevId: 491882849
Bypass Z Fold 4 HDR10 tone-mapping bug by limiting the max frame count to 12.
This passed with a value of 14, and failed with a value of 15, but I figured I'd use 12 just to be safe.
PiperOrigin-RevId: 491684058
Use the bitrate of the audio format (when available) in
DefaultAudioSink.AudioTrackBufferSizeProvider.getBufferSizeInBytes() to
calculate accurate buffer sizes for direct (passthrough) playbacks.
#minor-release
PiperOrigin-RevId: 491628530
To support OPUS offload, we need to provide a few configuration values
that are currently not set due to the lack of devices supporting
OPUS offload.
PiperOrigin-RevId: 491613716
Main change:
- Removed `Codec.EncoderFactory.createForVideoEncoding`'s argument of a list
of allowed MIME types
- Moved the check for whether a video MIME type is supported to VTSP
PiperOrigin-RevId: 491611799
The list of charsets is already hard-coded, and using `Charset` types
ensures they will all be present at run-time, hence we will never
encounter an 'unsupported' charset.
PiperOrigin-RevId: 491324466
This thread just starts the player and handles the player callbacks for
now. Sample pipelines are still run on the playback thread.
PiperOrigin-RevId: 491299671
This is more clear than using Format.NO_VALUE, when we do actually intend for an
output value.
Also, fix @see formatting by using summary fragments instead, and add an error
output for OETF and EOTF transfer functions.
PiperOrigin-RevId: 490910229
Our FakeClock generally makes sure that playback tests are fully
deterministic. However, this fails if the test uses blocking waits
with clock.onThreadBlocked and where relevant Handlers are created
without using the clock.
To fix the flakiness, we can make the following adjustments:
- Use TestExoPlayerBuilder instead of legacy ExoPlayerTestRunner
to avoid onThreadBlocked calls. This also makes the tests more
readable.
- Use clock to create Handler for FakeVideoRenderer and
FakeAudioRenderer. Ideally, this should be passed through
RenderersFactory, but it's too disruptive given this is a
public API.
- Use clock for MediaSourceList and MediaPeriodQueue update
handler.
PiperOrigin-RevId: 490907495
