Based on
https://developer.android.com/reference/android/media/MediaCodec#using-an-output-surface,
frame dropping behaviour depends on the target SDK version.
After this change transformer will only use
MediaFormat#KEY_ALLOW_FRAME_DROP if both the target and system SDK
version are at least 29 and default to its pre 29 behaviour where each
decoder output frame must be processed before a new one is rendered
to prevent frame dropping otherwise.
Also remove deprecated Transformer.Builder constructor without a
context and the context setter.
PiperOrigin-RevId: 453971097
Transformer always enabled glAssertionsEnabled, so there should
be no functional change.
ExoPlayer previously disabled glAssertionsEnabled, so GlUtil logged
GlExceptions instead of throwing them. The GlExceptions are now
caught and logged by the callers so that there should also be no
functional change overall.
This change also replaces EGLSurfaceTexture#GlException with
GlUtil#GlException.
PiperOrigin-RevId: 453963741
The track selector will select multi-channel formats when those can be
spatialized, otherwise the selector will prefer stereo/mono audio
tracks. When the device supports audio spatialization (Android 12L+),
the DefaultTrackSelector will monitor for changes in the platform
Spatializer and trigger a new track selection upon a
Spatializer change event.
Devices with a `television` UI mode are excluded from audio channel
count constraints.
#minor-release
PiperOrigin-RevId: 453957269
This change fixes a bug where the player is frozen with HLS chunkless
preparation because the audio stream wrappers are not marked as master
timestamp sources before preparation.
#minor-release
PiperOrigin-RevId: 453941815
Some Player implementations have no playlist capability but can still
set a MediaItem for playback. Examples are a MediaController connected
to a legacy MediaSession, ExoPlayer up to 2.12 or MediaPlayer.
To indicate this capability, we need an allowed command in addition
to COMMAND_CHANGE_MEDIA_ITEMS that just allows to set a single item
that replaces everything that is currently played.
#minor-release
PiperOrigin-RevId: 453879626
This removes the prior restriction of needing to remember not to crop and set aspect ratio in the same Presentation.Builder, and makes each class a bit more targeted.
This is partially made feasible by the past work to merge consecutive
MatrixTransformations into a single MatrixTransformationFrameProcessor, which
ensures that there's no loss in quality between successive MatrixTransformations.
PiperOrigin-RevId: 453660582
With HLS chunkless preparation, audio formats may have no value
for channel count. In this case, the DefaultAudioSink will either query
the platform for a supported channel count (API 29+) or assume a max
channel count based on the encoding spec in order to decide whether the
audio format can be played with audio passthrough.
Issue: google/ExoPlayer#10204
#minor-release
PiperOrigin-RevId: 453644548
SingleFrameGlTextureProcessor is now an abstract class containing a
default implementation of the more flexible GlTextureProcessor interface
while still exposing the same simple abstract methods for single frame
processing it previously did.
FrameProcessorChain and GlEffect will be changed to use
GlTextureProcessor in follow-ups.
PiperOrigin-RevId: 453633000
The API 32 SDK has incorrect versioning metadata for Spatializer. It
reports the whole class has only been present since API 33 (which is
surely impossible given it's present in the API 32 SDK):
https://issuetracker.google.com/234009300
The metadata seems to be correct in the API 33 SDK, so this baseline
will no longer be needed when we bump to `compileSdkVersion = 33`.
Fixing lint errors in the string.xml files makes no sense because these are
overridden with the next automated string import. Adding a lint-baseline.xml
instead for the ui module.
See https://issuetracker.google.com/208178382
#minor-release
PiperOrigin-RevId: 455354304
(cherry picked from commit 495480a66a)
The default constructor is only allowed to be called on
API < 32 and the test should use the defined UNSET constant
to be API independent.
#minor-release
PiperOrigin-RevId: 454568893
(cherry picked from commit 0ed53215c2)
`codecDrainAction` is set to `DRAIN_ACTION_NONE` in 3 places in
`MediaCodecRenderer`:
* The constructor (so there's no prior state to worry about)
* `updateDrmSessionV23()`: Where `mediaCrypto` is reconfigured based
on `sourceDrmSession` and `codecDrmSession` is also updated to
`sourceDrmSession`.
* `resetCodecStateForFlush()`: Where (before this change) the action
is unconditionally set back to `DRAIN_ACTION_NONE` and so any
required updated implied by
`DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION` is not done.
This change ensures that `flushOrReleaseCodec()` handles
`DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION` before calling .
This probably also resolves Issue: google/ExoPlayer#10274
#minor-release
PiperOrigin-RevId: 454114428
(cherry picked from commit c736a72c75)
The track selector will select multi-channel formats when those can be
spatialized, otherwise the selector will prefer stereo/mono audio
tracks. When the device supports audio spatialization (Android 12L+),
the DefaultTrackSelector will monitor for changes in the platform
Spatializer and trigger a new track selection upon a
Spatializer change event.
Devices with a `television` UI mode are excluded from audio channel
count constraints.
#minor-release
PiperOrigin-RevId: 453957269
(cherry picked from commit 9f3c595e02)
This change fixes a bug where the player is frozen with HLS chunkless
preparation because the audio stream wrappers are not marked as master
timestamp sources before preparation.
#minor-release
PiperOrigin-RevId: 453941815
(cherry picked from commit 917484100d)
Some Player implementations have no playlist capability but can still
set a MediaItem for playback. Examples are a MediaController connected
to a legacy MediaSession, ExoPlayer up to 2.12 or MediaPlayer.
To indicate this capability, we need an allowed command in addition
to COMMAND_CHANGE_MEDIA_ITEMS that just allows to set a single item
that replaces everything that is currently played.
#minor-release
PiperOrigin-RevId: 453879626
(cherry picked from commit 2056152824)
With HLS chunkless preparation, audio formats may have no value
for channel count. In this case, the DefaultAudioSink will either query
the platform for a supported channel count (API 29+) or assume a max
channel count based on the encoding spec in order to decide whether the
audio format can be played with audio passthrough.
Issue: google/ExoPlayer#10204
#minor-release
PiperOrigin-RevId: 453644548
(cherry picked from commit b3b57bc93d)
The current setup with distinct, private `keyForField` implementations,
leaves open the (theoretical) possibility of a clash in the `Bundle`
keys used by the superclass and subclass. This change brings
consistency with our only other extensible `Bundleable` type
(`PlaybackException`).
#minor-release
PiperOrigin-RevId: 453385875
(cherry picked from commit a056f08a3d)
We generally nest the `Builder` for `Foo` inside `Foo`. In this case,
there's already a `DefaultTrackSelector.Parameters.Builder` type visible
to a developer, it just happens to be the 'common'
`TrackSelectorParameters.Builder`, so using it is a bit weird. For
example this code snippet doesn't compile because
`DefaultTrackSelector.Parameters.Builder#build()` returns
`TrackSelectionParameters`. This CL fixes that problem and the code
snippet now compiles.
```java
DefaultTrackSelector.Parameters params =
new DefaultTrackSelector.Parameters.Builder(context).build()
```
#minor-release
PiperOrigin-RevId: 453215702
(cherry picked from commit 881622385d)
This is done by removing the `@FieldNumber` IntDef completely. It's not
really adding much value anyway, because it's `open` so there's no real
enforcement to prevent passing 'incorrect' values.
#minor-release
PiperOrigin-RevId: 452108972
(cherry picked from commit 97210b5aa0)
The current setup with distinct, private `keyForField` implementations,
leaves open the (theoretical) possibility of a clash in the `Bundle`
keys used by the superclass and subclass. This change brings
consistency with our only other extensible `Bundleable` type
(`PlaybackException`).
#minor-release
PiperOrigin-RevId: 453385875
We generally nest the `Builder` for `Foo` inside `Foo`. In this case,
there's already a `DefaultTrackSelector.Parameters.Builder` type visible
to a developer, it just happens to be the 'common'
`TrackSelectorParameters.Builder`, so using it is a bit weird. For
example this code snippet doesn't compile because
`DefaultTrackSelector.Parameters.Builder#build()` returns
`TrackSelectionParameters`. This CL fixes that problem and the code
snippet now compiles.
```java
DefaultTrackSelector.Parameters params =
new DefaultTrackSelector.Parameters.Builder(context).build()
```
#minor-release
PiperOrigin-RevId: 453215702
Implementations of this interface will be able to drop or add frames,
change timestamps, accept multiple input frames before producing
output, and process frames on their own background thread.
A default implementation of this interface will be added to SingleFrameGlTextureProcessor in a follow-up.
PiperOrigin-RevId: 453159835
This is done by removing the `@FieldNumber` IntDef completely. It's not
really adding much value anyway, because it's `open` so there's no real
enforcement to prevent passing 'incorrect' values.
#minor-release
PiperOrigin-RevId: 452108972
This internal listener avoids wrapping the TransformationExceptions
in PlaybackExceptions that are handled via the Player.Listener and
is also used for FrameProcessingExceptions which already avoided
the PlaybackException layer previously.
This listener will also be useful in follow-ups for encoder-related
TransformationExceptions that are thrown in the SurfaceProvider that
will be called on the GL thread.
PiperOrigin-RevId: 452074575
This reinstates the permissive behaviour removed by
fe7e5b8181
Test file created by opening bear.opus in a hex editor and naively
duplicating the two header packets, starting at (and including) the
first `OggS` in the file and ending just before the third `OggS`.
#minor-release
Issue: google/ExoPlayer#10038
PiperOrigin-RevId: 452015662
(cherry picked from commit b6b282672c)
This reinstates the permissive behaviour removed by
fe7e5b8181
Test file created by opening bear.opus in a hex editor and naively
duplicating the two header packets, starting at (and including) the
first `OggS` in the file and ending just before the third `OggS`.
#minor-release
Issue: google/ExoPlayer#10038
PiperOrigin-RevId: 452015662
Once the more advanced GlTextureProcessor interface exists,
it will be possible to change the output size of a GlTextureProcessor
between frames. To keep the re-configuration based on the frame sizes
minimal, things indepedent of the frame size, such as the GlProgram,
can be initialized in the constructor.
PiperOrigin-RevId: 451997584
This value only existed to allow setting media URLs from external sources
(e.g. in a MediaController) so that a player can start playing this item.
Now that we have MediaItem.RequestMetadata.mediaUrl we can remove this value
from MediaMetadata because it's request metadata, not media metadata.
PiperOrigin-RevId: 451857413
These fields can be used to transport additional request properties
when the requester doesn't know the details needed for the actual
playback (i.e. the LocalConfiguration).
PiperOrigin-RevId: 451857093
ExoPlayer applies a large time offset to buffers so that, if the input has negative timestamps, generally buffers seen by the decoders should have positive timestamps. Modify how the offset is handled in `Transformer` so that decoders and encoders generally see positive timestamps, by leaving the offset on samples when reading them in the base renderer (remove the code that removed the offset), and then removing the offset when muxing. Also update the frame processor chain and slow motion flattening code to retain the existing behavior after this change (these both need original media presentation timestamps)
Tested via existing end-to-end tests and manually verified that the overlay frame processor shows the expected original media timestamps.
Aside: we don't need the same logic as ExoPlayer to track stream offsets across the decoder yet, because we don't try to handle stream changes during playback in single asset editing. (There is an edge case of multi-period DASH that may not work but I doubt anyone will use that as input to `Transformer` before we change the code to handle multi-asset properly.) In future we should try to refactor interaction with the decoder to use the same code for Transformer and ExoPlayer.
PiperOrigin-RevId: 451846055
Most devices won't support 8k decoding, so they'll skip this test entirely.
As the video is quite short, this test shouldn't be any longer than the nearby,
long-running 4k60 test.
PiperOrigin-RevId: 451423368
Unconditionally sleep for offload, if the audio buffer is full.
Previously ExoPlayer would not sleep if the expected wake-up was
in 2s. This was to prevent underrun if the wake-up was delayed.
Experiments have shown that the wakup audio buffer is far more
than 2s (around 1min). Additionally,
the metric was incorrect because it measured both,
AudioTrack + DSP.
Finally, this metric was erroneous after a gapless transition,
when the head position would reset to 0 and thus the computed
delay until next wakeup was too large.
PiperOrigin-RevId: 451383701
SSIM calculation requires the input and output dimensions to be identical.
For devices that can't encode the input dimensions, skip SSIM calculations and
log the cause. Only apply this on tests where the encoder may not support the
input file dimensions.
PiperOrigin-RevId: 451364904
Decode-only video frames (needed when the frame at / first frame after the
clipping start is not a key frame) need to be decoded but not passed to
the frame processor chain or encoder.
The clipping start offset needs to be removed from the frame timestamps
in the passthrough and video pipelines.
There are no changes needed for this in the audio pipeline, as it doesn't
use the input timestamps -- it uses its own timestamps derived from the
buffer sizes instead.
Also add demo option to try this out.
#minor-release
PiperOrigin-RevId: 451353609
Also update names of implementations to match design doc.
In follow-ups, SingleFrameGlTextureProcessor will become
an abstract implementation of a new GlTextureProcessor
interface.
Texture processor makes sense as it processes OpenGL textures.
The term frame processor will be used for something else in
follow-ups.
PiperOrigin-RevId: 451142085
* Rename (via deprecation)
`MediaItem.DrmConfiguration.Builder#forceSessionsForAudioAndVideoTracks`
to `setForceSessionsForAudioAndVideoTracks`. This is more consistent
with existing 'force' method names both in this class and in
`TrackSelectionParameters.Builder`.
* Add missing `@Nullable` annotation to the parameter for
`MediaItem.SubtitleConfiguration.Builder#setMimeType`. This annotation
is already present on the `MediaItem.SubtitleConfiguration#mimeType`
field that this setter corresponds to.
PiperOrigin-RevId: 450941336
There is a problem with the ImageReader formats used by the
SSIM helper that only occurs for Nexus 5 API 21, so as a workaround
we can skip the SSIM calculation on Nexus 5 API 21.
This skips just the SSIM calculation (by setting the value to
1.0 instead and logging). The tests still run when SSIM is skipped
so that we can detect other failures.
PiperOrigin-RevId: 450903183
This value is only needed by subclasses of `Rating`, all of which are
in this package (the `Rating` constructor is already package-private to
ensure this).
PiperOrigin-RevId: 450886872
*** Original commit ***
Rollback of 57182ac7bd
*** Original commit ***
Remove `@Nullable` from `MediaSource.Factory` setters
The null-behaviour of these methods creates a minimization footgun,
because **any** call to these setters will prevent R8 from removing
the default implementation (even if it's never used by the app) - this
is because R8 can't tell the default imple...
***
PiperOrigin-RevId: 450453325
AudioTrack.setOffloadEndOfStream should be called after a track
has been buffered. Additionally, the AudioTrack must be playing.
It has been observed that for very short media (<1s), the AudioTrack
might not have started immediately after the read that buffered
the audio.
In such a situation, calling AudioTrack.setOffloadEndOfStream throws
and playback fails.
Avoid this failure by checking that the AudioTrack is playing before
calling setOffloadEndOfStream.
This means that very short gapless media will not be gapless, this was
deemed acceptable given that such very short media should be very rare
in offload.
PiperOrigin-RevId: 450431146
*** Original commit ***
Remove `@Nullable` from `MediaSource.Factory` setters
The null-behaviour of these methods creates a minimization footgun,
because **any** call to these setters will prevent R8 from removing
the default implementation (even if it's never used by the app) - this
is because R8 can't tell the default implementation is only used if the
parameter is `null`.
******
PiperOrigin-RevId: 450410833
The null-behaviour of these methods creates a minimization footgun,
because **any** call to these setters will prevent R8 from removing
the default implementation (even if it's never used by the app) - this
is because R8 can't tell the default implementation is only used if the
parameter is `null`.
Follow-up to 57182ac7bd
PiperOrigin-RevId: 450395941
The null-behaviour of these methods creates a minimization footgun,
because **any** call to these setters will prevent R8 from removing
the default implementation (even if it's never used by the app) - this
is because R8 can't tell the default implementation is only used if the
parameter is `null`.
PiperOrigin-RevId: 450386627
This detection relies on an unsupported workaround and may trigger
permission warnings in tools analyzing permission usage although
no permission is needed or requested by app code.
Given the majority of 5G-NSA playbacks are on API 31+ by now,
we can remove this path to avoid the permission confusion and the
unsupported detection workaround.
PiperOrigin-RevId: 450382586
The debug surface view's output surface can become invalid during a transformation due to the parent activity pausing, for example. This can currently cause a crash when backing out of the `TransformerActivity` in the demo because the surface can be destroyed before the transformer has fully canceled.
Also clarify naming of the outputSurface and inline the private method that created `EGLSurface`s (it was shorter after removing the debug preview).
PiperOrigin-RevId: 449963440
This causes a bug where the forwarded selections are no longer
assumed equal and the child MediaPeriods will think they need
to reset streams even though the selection stayed the same.
Issue: Issue: google/ExoPlayer#10248
PiperOrigin-RevId: 449454038
We need to pass timestamp for the list of cues so we are defining a new class CueGroup which will store both cues and timestamp.
PiperOrigin-RevId: 449212054
The old reference was just for prototyping HDR. The new reference is for planned work to use the correct formats for input and output for HDR editing in GL.
PiperOrigin-RevId: 449211792
The MMWAVE constant was deprecated in favour of a new constant
with a better name. Thus, we need to check for both constants now.
PiperOrigin-RevId: 449018959
This is an individual language (ISO 639-3) part of the Arabic
macrolanguage ("ar" in ISO 639-1). Add this mapping to our
existing list similar to other individual to macrolanguage
mappings we have already.
Issue: Issue: google/ExoPlayer#10255
PiperOrigin-RevId: 448911950
When using a MatrixTransformationFrameProcessor per transformation
matrix, each frame processor's shader applies the matrix to the
vertices and clips the result to the NDC range when drawing the
output frame.
This change combines consecutive MatrixTransformations into a single
MatrixTransformationFrameProcessor by multiplying the individual
matrices while updating and clipping the visible polygon after
each matrix and mapping the resulting visible polygon back to the
input space so that its vertices and the combined transformation
matrix can be used in the shader.
PiperOrigin-RevId: 448521068
Network type detection on these API levels couldn't be tested
yet because of a missing Robolectric feature. This was fixed by
the recent Robolectric upgrade and the restrictions can be removed.
This also requires to replicate the platform hack we rely on on
these API levels.
PiperOrigin-RevId: 448240431
Some RTP foramts are statically assigned, so they don't have the rtpmap
attribute. Create the missing rtpmap attribute in this case.
PiperOrigin-RevId: 448239724
This listener replaces
FrameProcessorChain#getAndRethrowBackgroundExceptions.
The listener uses a new exception type FrameProcessingException
separate from TransformationException as the frame processing
components will be made reusable outside of transformer soon.
PiperOrigin-RevId: 447455746
This constant is used for https://docs.gl/es2/glVertexAttribPointer
which takes the number of components per generic vertex attribute
(meaning the size of the individual coordinate vectors here) not the
number of attributes (the number of vertices that the old constant
name referred to).
PiperOrigin-RevId: 447427241
Before this change, we list the formats for which we don't
want transcoding. This change disables transcoding altogether.
This was tested by checking that transcoding takes place on a
short camera recording only when the added flag is not present
(and AndroidManifest does not declare support for HEVC).
PiperOrigin-RevId: 446986580
We won't try to provide/rethrow helpful error messages that are already
provided by GL, as this sort of task would expand into writing a GL verifier.
A Gl verifier is unnecessarily complex for minimal payoff, especially as Apps
expected to read GL error messages would mostly be those writing custom
GlFrameProcessors, who should be already be familiar with reading GL error
messages anyways.
PiperOrigin-RevId: 446950837
ScaleToFitFrameProcessor, PresentationFrameProcessor,
and EncoderCompatibilityFrameProcessor now each implement
MatrixTransformation instead of wrapping
MatrixTransformationFrameProcessor.
PiperOrigin-RevId: 446480286
The TrackSelector is released when the player is released. The
TrackSelector can be reused if TrackSelector.init() is called again.
PiperOrigin-RevId: 446439717
This change splits AdvancedFrameProcessor into 4 files:
- MatrixTransformationFrameProcessor for the GlFrameProcessor
implementation
- MatrixTransformation and GlMatrixTransformation for the GlEffect
specification
- MatrixUtils for the static matrix helpers
PiperOrigin-RevId: 446236384
Some APIs from Android 12L were used either via reflection or
constants values were hard-coded. We can now use these APIs directly
since we upgraded the compile SDK version to 32.
PiperOrigin-RevId: 446167543
Also add warning for what can happen if this is ignored.
Previously the return value was overridden by setOutputSurface, now
setOutputSurface only overrides the values using internally.
PiperOrigin-RevId: 445377036
Developers are expected to (eventually) only use methods on
PlayerView (and not PlayerControlView) to interact with the UI
controller.
PiperOrigin-RevId: 445361488
This is consistent with the IntDef name, and frees up the CONTENT_TYPE_
prefix for the @ContentType values (which are currently just TYPE_*,
and therefore ambiguous with lots of other 'type' values in C).
PiperOrigin-RevId: 445356476
This method was introduced in 754eb1527a
as a replacement for Util.inferContentType(String) but it incorrectly
didn't return TYPE_SS when passed "ism" or "isml".
PiperOrigin-RevId: 445217167
We add an entire class like we do for parsing other codec initialization formats; it's currently not doing any parsing though (... initialization data is really simple for AV1 though: just the entire contents of the box).
For testing, we add the sample file, having been re-encoded with ffmpeg (and we also happen to have another av1 file, too).
PiperOrigin-RevId: 444890282
This fixes some small niggles:
1. `inferContentType(String)` is documented to take a path, but in the
tests we're passing full URIs.
2. A `String` parameter is usually a path, but also a MIME type or an
extension. In the new methods, the meaning of a `String` parameter
is always clear from the name of the method.
3. `inferContentType(String)` is always passed an extension in
'production' code (which has to be manually prefixed with a dot).
4. `inferContentType(Uri, @Nullable String)` always ignores the Uri if
the String is non-null. IMO this logic is clearer to a reader if it's
just in-lined at the call-site.
These methods are used from the demo apps, so will be part of the stable
API.
PiperOrigin-RevId: 444826053
To ensure frame processor operations operate on square pixels,
make the frame taller or wider for non-square input pixels.
In addition to automated tests, this was tested by changing the
inputFormat.pixelWidthHeightRatio in the TransformerVideoRenderer.
PiperOrigin-RevId: 444553517
This allows the actual bitmap to be saved, even if output
dimensions are different than expected. Otherwise, differing
output dimensions would throw an exception, preventing the bitmap
from being saved.
PiperOrigin-RevId: 444512210
The last frame processor could use a different drawing command
than a four vertex triangle strip. So we need call its drawing
method again instead of assuming the shape.
PiperOrigin-RevId: 444217274
Avoid no-op FrameProcessors in VideoTranscodingSamplePipeline, to avoid
creating resources for and executing GL for a no-op vertex+fragment shader.
EncoderCompatibilityFrameProcessor will still always be exercised.
PiperOrigin-RevId: 443675833
This test should run on all devices from API 21 (the media uses Baseline
profile level 3.0 H.264) to give us coverage of the full pipeline (forcing
re-encoding) and SSIM calculation on all devices.
PiperOrigin-RevId: 443650002
Split rotationDegrees changes to EncoderCompatibilityFrameProcessor, a new
FrameProcessor.
This removes automatic rotationDegrees adjustments from Presentation, which
allows Presentation to be used for changes before the end of a
FrameProcessorChain pipeline.
PiperOrigin-RevId: 443387226
The issue with decoding getting stuck is still reproducible on `samsung/beyond1lteeea/beyond1:12/SP1A.210812.016/G973FXXUEGULB:user/release-keys`.
PiperOrigin-RevId: 443368546
Some devices under report their resolution support, like 2144 for 2160 in
H265, 1072 for 1080 in H264. This CL only takes care of these two cases,
- reporting 1920x1080 is supported when the device reports 1920x1072, and
- reporting 3840x2160 is supported when the device reports 3840x2144
PiperOrigin-RevId: 443095042
When downlading an adaptive asset, if an ExoPlaybackException happens
during track selection, the player raises an
UnsupportedOperationException which is not handled gracefully and can
crash the app main thread.
This change catches the error and forwards it to
DownloadHelper.Callback.onPrepareError() as an IOException.
PiperOrigin-RevId: 443015332
This is safer because it will prevent any future problems with
creating a new FrameProcessorChain before the previous one has
completed its async release.
From [eglDestroyContext documentation](https://www.khronos.org/registry/EGL/sdk/docs/man/html/eglDestroyContext.xhtml):
"If the EGL rendering context is not current to any thread,
eglDestroyContext destroys it immediately."
The context isn't current to any thread here because GlUtil calls
eglMakeCurrent with EGL_NO_CONTEXT before calling eglDestroyContext.
So everthing should be released once the FrameProcessorChain's
release task terminates.
PiperOrigin-RevId: 442807484
* Group what's now many related test PNGs by moving them to their own directory.
* Move bitmap references to files where they're used, as each bitmap is only
used once each, except the original bitmap.
PiperOrigin-RevId: 441485489
Scaling and rotation using ScaleToFitFrameProcessor may change the
the resolution and aspect ratio, so defaulting to the same as the
input is no longer accurate.
PiperOrigin-RevId: 441463349
What a minimal implementation should include is now explained in the
interface javadoc while the method name reflects what the method does.
PiperOrigin-RevId: 441432059
The stream with the mp3 track is added because
- We only encode to AAC
- We only encode when the source track is not AAC
Now that we have a way to force encoding, we no longer need the mp3 track.
The test asset is kept for later parameterized testing.
PiperOrigin-RevId: 440876080
The MIME type was set to H265 to force transcoding. Now that we have an encoder
factory that forces encoding, switching back to H264 ensures the quality test
is conducted on more devices (those don't support H265 can be tested now).
However, H265 should be part of the quality test after we have proper mechanism
to skip test based on device capability.
PiperOrigin-RevId: 440132471
The DefaultMediaNotificationProvider checks if a command is available
before putting the respective action in the notification.
PiperOrigin-RevId: 440114422
We don't currently have enough understanding of the correlation between a
specific SSIM score and video quality. Dropping to .90 to make most tests pass.
Especially when there's no discernible difference from the videos with .9 and
.95 SSIM.
PiperOrigin-RevId: 440047551
PeriodicDimmingFrameProcessor is an example of how a custom fragment
shader can be used to apply color changes that change over time.
PiperOrigin-RevId: 439840609
The matrix provider allows the transformation matrix to be updated
for each frame based on the timestamp.
The following example effects using this were added to the demo:
* a zoom-in transition for the start of the video,
* cropping a rotating rectangular frame portion,
* rotating the frame around the y-axis in 3D.
PiperOrigin-RevId: 439791592
We add an entire class like we do for parsing other codec initialization formats; it's currently not doing any parsing though (... initialization data is really simple for AV1 though: just the entire contents of the box).
For testing, we add the sample file, having been re-encoded with ffmpeg (and we also happen to have another av1 file, too).
PiperOrigin-RevId: 439453823
This allows apps to use AdvancedFrameProcessor to apply transformations
in 3D space. This functionality is not used in transformer otherwise.
PiperOrigin-RevId: 439313406
The encoder surface is no longer needed for the OpenGL setup and frame
processor initialization, as a placeholder surface is used instead. So
all of the setup can now be done in the factory method.
PiperOrigin-RevId: 438844450
App code should get all of this information from TrackGroupInfo,
and should only need TrackGroup as a key to use for overrides.
PiperOrigin-RevId: 438840925
This allows the MuxerWrapper to keep using trackTypeToTimeUs for
calculating the video duration but slightly changes the meaning of
its interleaving constraints.
PiperOrigin-RevId: 438780686
This provides better compatibility with MediaExtractor, which does read these fields; we also need them for being able to mux file contents into another mp4 file.
Also, there is a minor refactor included so that we have an actual type for esds box contents instead of a pair.
PiperOrigin-RevId: 438673825
The placeholder surface is either EGL_NO_SURFACE or a 1x1 pbuffer
depending on whether the device supports EGL_KHR_surfaceless_context.
PiperOrigin-RevId: 438541846
Since the output textures and surfaces are managed by the
FrameProcessorChain, clearing them there makes sense.
This is also less error-prone as it might not be obvious to
someone implementing a GlFrameProcessor that they need to
glClear. (Clearing twice won't cause any problems.)
PiperOrigin-RevId: 438532247
This requires an additional nanos to micros conversion because
the SurfaceTexture uses nanos. But as the timestamps from the
MediaCodec decoder (propagated in DefaultCodec#releaseOutputBuffer) are
in microseconds no precision is lost here.
Also add test that checks output video duration.
PiperOrigin-RevId: 438010490
MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE was copied from a test
class, but BitmapTestUtil isn't a test. So the javadoc needs
rewording to reflect that.
PiperOrigin-RevId: 438001833
The problem is not the IntDef array, it's the fact the lint tool
is unable to correctly infer the annotations on the lambda parameters
without them being explicitly annotated. It seems explicitly annotating
is better than suppressing all IntDef warnings in the whole method.
PiperOrigin-RevId: 437969271
From Android T onwards `MediaCodec` supports requesting tone-mapping down to
SDR. Add an option to request this behavior and document that it isn't
supported before T. Also add an option in the demo app to try it out.
Tested manually on a prerelease build.
PiperOrigin-RevId: 437765325
The thread name is used to verify the thread in both createOpenGlObjectsAndInitializeFrameProcessors() and processFrame().
Also remove glThread field that was only used for this verification.
PiperOrigin-RevId: 437730804
Since the output size can be overridden, the viewport should be
ouputWidth/Height and NOT the ExternalCopyFrameProcessor's output size
which matches the input size.
PiperOrigin-RevId: 437256635
OutlineColour should be treated as the background color if BorderStyle=3. Since currently BorderStyle is ignored, we can always treat OutlineColor as the background color.
* The AdvancedFrameProcessor calls use() in updateProgramAndDraw().
* The AdvancedFrameProcessor has the same input and output dimensions.
PiperOrigin-RevId: 437231350
On some devices, decoding gets stuck when the number of frames pending at the
`SurfaceTexture` is too high. We added a workaround that only allows one frame
to be pending at a time. That fixed the issue, however, based on on-device
testing it seems that it's safe to queue more than one frame.
Add a method that returns a safe estimate of the number of frames that can be
pending at a time, and use this to limit the number of frames that can be
released from the decoder but not processed by the frame processor chain.
PiperOrigin-RevId: 437057075
Configuring the frame sizes between frame processors is now the
FrameProcessorChain's rather than the caller's responsibility.
The caller can getOutputSize() and override it for encoder fallback
in configure().
PiperOrigin-RevId: 437048436
The factory method is replaced by a public constructor and
configure() method which configures the input/output surfaces
and handles the OpenGL setup.
This is a prerequisite for removing the responsibility of the
caller to configureSizes() before creating the chain in a follow-up.
PiperOrigin-RevId: 437028882
This allows us to bypass many device-specific issues, that only occur when
decoding directly to an encoder surface, without OpenGL. This also allows us
to maintain fewer code branches, which require additional testing to verify
correctness.
PiperOrigin-RevId: 437003138
Allow apps to modify how frames are presented, via modifying resolution.
A follow-up CL will provide aspect ratio, cropping, etc.
PiperOrigin-RevId: 436963312
* These grouping parentheses are unnecessary; it is unlikely the code will be misinterpreted without them
(see http://go/bugpattern/UnnecessaryParentheses) (11 times)
This CL looks good? Just LGTM and Approve it!
This CL doesn’t look good? This is what you can do:
* Revert this CL, by replying "REVERT: <provide reason>"
* File a bug under go/error-prone-bug for category ErrorProneStyle if there's an issue with the CL content.
* File a bug under go/rosie-bug if there's an issue with how the CL was managed.
* Revert this CL and not get a CL that cleans up these paths in the future by
replying "BLOCKLIST: <provide reason>". This is not reversible! We recommend to
opt out the respective paths in your CL Robot configuration instead:
go/clrobot-opt-out.
This CL was generated by CL Robot - a tool that cleans up code findings
(go/clrobot). The affected code paths have been enabled for CL Robot in //depot/google3/java/com/google/android/libraries/media/METADATA which is reachable following include_presubmits from //depot/google3/third_party/java_src/android_libs/media/METADATA.
Anything wrong with the signup? File a bug at go/clrobot-bug.
#codehealth
Tested:
Local presubmit tests passed.
PiperOrigin-RevId: 436726763
SEF is similar to frame editing, where the input and output videos are
not intended to match, so SSIM will not provide a reliable value. To
check SSIM correctly in future, we would need to provide
golden/expected video files to compare output against.
PiperOrigin-RevId: 436707240
The FrameProcessorChain manages a List<GlFrameProcessor>.
FrameProcessorChainDataProcessingTest now tests chaining ScaleToFit-
and AdvancedFrameProcessors.
PiperOrigin-RevId: 436468037
This currently causes the test to fail on Pixel 6 Pro running a recent S build
SQ1D.220205.004.
There is no need to test audio transcoding while we are measuring video
quality.
PiperOrigin-RevId: 435635314
ExternalCopyFrameProcessor's output dimensions match the input
size not the output size. So the intermediate texture size
should match the input size.
Also rename configureOutputDimensions to configureOutputSize.
PiperOrigin-RevId: 435058789
More info: go/lsc-assertthrows and go/assertthrows
NOTE: if the source of truth for this code is _NOT_ `//third_party/`, please ask for this CL to be reverted.
Tested:
TAP --sample ran all affected tests and none failed
http://test/OCL:434925976:BASE:434869111:1647399186064:de338189
PiperOrigin-RevId: 435047509
Some infra thinks the if does not protect against API incompatibilities
(example: Android's soong build system). AndroidStudio 2021.3.1 also
signals a warning.
#minor-release
PiperOrigin-RevId: 435027073
Use android.util.Size, whose naming is much easier to understand than Pair<Integer, Integer>, in both FrameProcessor and EncoderUtil.
PiperOrigin-RevId: 434813986
* Move auto-adjustments for transformation matrices from the
VideoTranscodingSamplePipeline constructor to the new
ScaleToFitFrameProcessor.
* Add GlFrameProcessor#getOutputDimensions() to allow for GlFrameProcessors with
different input and output dimensions. This is a prerequisite for
Presentation.
* Tested with unit tests (and manually just in case).
* A follow up CL will implement change the FrameProcessor input to be scale and
rotate values as requested by the user. This was kept out of this CL to
reduce CL review size. Presentation will also be implemented in a follow up
CL.
PiperOrigin-RevId: 434774854
As pointed out in a previous review, createFallbackTransformationRequest
can be a bit confusing to parse.
Added javadocs and renamed parameters appropriately, to make it slightly
more easy to understand.
PiperOrigin-RevId: 434733313
*** Original commit ***
Rollback of 20282151b5
*** Original commit ***
Fix PlayerView touch handling
Overriding onTouchEvent was causing multiple issues, and
appears to be unnecessary. Removing the override fixes:
1. StyledPlayerView accessibility issue where "hide player
controls" actually toggled play/pause.
2. Delivery of events to a registered OnClick...
***
PiperOrigin-RevId: 434502423
*** Original commit ***
Don't call MediaDrm.setLogSessionId in FrameworkMediaDrm
This method throws an UnsupportedOperationException on some Android 12
devices.
***
PiperOrigin-RevId: 433708582
The variable marks the coordinates used to sample from a texture, so hopefully
this makes the naming a bit more descriptive.
This renames vTexCoords and aTexCoords. No functional changes intended.
PiperOrigin-RevId: 433499934
This fixes an exception thrown when parceling becasue the type can not be found
(expects the AdsPlaybackState to be Serializable). Transforming the map and the
ad playback states to a Bundle fixes the problem.
#minor-release
PiperOrigin-RevId: 433491993
When the start position of a MediaPeriodInfo is equal or higher than the duration,
we set the start position to `duration - 1` to end on the last frame. With server
side inserted ad streams, this has the effect that we actually need to seek back to
the last content frame after a post-roll.
This is desirable when actually ending on that frame but produces a BUFFERING event
when transitioning from an SSAI stream with a post-roll to the next media item in
the playlist. This change sets the start position to the duration when we are
clipping the last content period of an SSAI stream that is played in a playlist.
PiperOrigin-RevId: 433445680
If an OpenGL call blocks because the encoder's input surface is full,
this will now block the background thread while the main thread can
continue querying encoder output and free up encoder capacity until
it accepts more input unblocking the background thread.
PiperOrigin-RevId: 433283287
*** Original commit ***
Fix PlayerView touch handling
Overriding onTouchEvent was causing multiple issues, and
appears to be unnecessary. Removing the override fixes:
1. StyledPlayerView accessibility issue where "hide player
controls" actually toggled play/pause.
2. Delivery of events to a registered OnClickListener when
useController is false.
3. Delivery of events to a registered OnLongClickListener
in all configurations.
4. Incorrectly treating a sequence of touch events that
exit the bounds of the vi...
***
PiperOrigin-RevId: 433262414
With the new version, we try the following before fixing resolution:
- Fix size alignment
- Try 3/4 the width and height
- Try 2/3 the width and height
- Try 1/2 the width and height
Also: align the resolution ends in 1 or 9 to 0.
PiperOrigin-RevId: 433206358
Overriding onTouchEvent was causing multiple issues, and
appears to be unnecessary. Removing the override fixes:
1. StyledPlayerView accessibility issue where "hide player
controls" actually toggled play/pause.
2. Delivery of events to a registered OnClickListener when
useController is false.
3. Delivery of events to a registered OnLongClickListener
in all configurations.
4. Incorrectly treating a sequence of touch events that
exit the bounds of the view before ACTION_UP as a click,
both for delivery to OnClickListener and for toggling
the controls.
Note: After this change, control visibility will not be
toggled if the application developer explicitly sets the
view to be non-clickable. I think that's probably working
as intended though. It seems correct that a non-clickable
view would not respond to clicks.
Issue: google/ExoPlayer#8627
Issue: google/ExoPlayer#9605
Issue: google/ExoPlayer#9861
PiperOrigin-RevId: 433016626
This change rewrites the UI module's track selection
components to depend on the Player API, allowing us to
finally remove the UI module's dependency on ExoPlayer
as a concrete player implementation.
PiperOrigin-RevId: 432989318
Previously, we've used getSupportedHeights/Widths() to find the supported
resolution. However, the height/width can be over-reported when using these
APIs. For example, getSupportedWidths and getSupportedHeights can both return
3840, but the supported height when using 3840 as width is only 2160.
PiperOrigin-RevId: 432926192
This makes the reading period advance early as expected at the end of an ad
period. Before this change the reading position of the metadata renderer
prevented advancing the period until metadata arrived after the start position of
the following period. Only then the reading position of the metadata renderer
is updated and beyond the start position of the following period which is a
condition to advance the reading period.
Because transitioning to the next period is a virtual transition and the
SharedMediaPeriod keeps reading from the same underlying sample streams, the
metadata renderer can safely be ignored for this check.
#minor-release
PiperOrigin-RevId: 432646037
We will be migrating our track selection UI components to be
based on TracksInfo. We need DownloadHelper to expose TracksInfo
to make it compatible with such components.
PiperOrigin-RevId: 432474487
addTrackSelectionForSingleRenderer takes a list of legacy overrides,
which are then set on the supplied parameters one at a time to run
track selection. This allows multiple overrides for a single track
type to be applied in the download use case, despite it not being
possible to place such overrides directly into a single parameters.
For new style overrides, multiple overrides for the same track type
can be placed directly into a single parameters. Therefore we'll be
able to replace use of addTrackSelectionForSingleRenderer with use
of addTrackSelection, which is a much cleaner API. For this to work,
we need to make DownloadHelper apply multiple overrides in this case.
PiperOrigin-RevId: 432459834
This will allow for easier customisation of the additional tasks
performed by the test runner, such as calculating metrics like SSIM.
PiperOrigin-RevId: 432434850
These should have been removed as part of 1391b7c65d, since we no
longer officially support overriding the layout file for this class.
This class is known as StyledPlayerView in exoplayer2.
#minor-release
PiperOrigin-RevId: 432411322
This method is no longer needed since we added SubtitleConfiguration#id
in 59d98b9a4e.
Issue: google/ExoPlayer#10016
#minor-release
PiperOrigin-RevId: 432169262
DownloadHelper is in the ExoPlayer module, so there's no reason
why it can't use ExoPlayer specific track selections. That said,
we want our UI components to operate on generic
TrackSelectionParameters, and we want such UI components to be
useful for selecting tracks for download. To keep this interop,
it's necessary to have DownloadHelper accept generic
TrackSelectionParameters, or to require application code to
convert them. The first approach seems preferable!
PiperOrigin-RevId: 432158846
This constructor always does the wrong thing for non-adaptive groups
containing more than 1 track, because it'll incorrectly generate an
adaptive selection. Replace it with a constructor for specifying a
single track within the group instead.
PiperOrigin-RevId: 431673458
Only log GL exceptions in GlUtil if they aren't thrown. Otherwise,
it's up to the caller whether or not to log them to avoid logging them
twice.
PiperOrigin-RevId: 431657397
This test tests the same cases as the FrameEditorDataProcessingTest
as currently the main FrameEditor functionality is to apply a
transformation matrix using a TransformationFrameProcessor.
PiperOrigin-RevId: 431642066
As evidenced by the somewhat awkward logic in PlayerControlView, the
previous design wasn't very friendly to expected usage. There will be
more usage when the track selection dialog components are migrated,
which would be similarly awkward without this change.
PiperOrigin-RevId: 431407675
We use SSIM to measure the transcoding quality between. SSIM is a widely used
tool that compares the luma channel between two images, and generates a score
from 0 to 1 that indicates "how similar" the two images are.
In `SsimHelper`, we decode the two videos, extract matching frames and
calculates the mean SSIM (SSIM averaged all matching frames) for both videos.
Matching frames are referred to as "comparisonFrame" in the CL, which is
selected based on the frame number and a user-set comparison interval.
For instance, if the interval is 7, then every seventh frames are compared.
We use MediaCodec/MediaExtractor to decode the video, and use ImageReader to
extract the decoded frame.
The SSIM calculation logic is a inspired by and modified from the CTS
[MSSIMMatcher](https://cs.android.com/android/platform/superproject/+/master:cts/tests/tests/uirendering/src/android/uirendering/cts/bitmapcomparers/MSSIMComparer.java;l=1?q=mssimcom)
that has some errors and extra features we don't need (like handling RGB
images).
Adds TranscodeQualityTest to ensure high quality transcoding.
PiperOrigin-RevId: 430951206
*** Original commit ***
Rollback of 8d9c4f4774
*** Original commit ***
Rollback of 43b796b64d
*** Original commit ***
Rollback of 08c5b1cb0a
*** Original commit ***
PiperOrigin-RevId: 430905772
The AsyncronousMediaCodecAdapter should call MediaCodec.start()
on the same thread it calls MediaCodec.flush(), i.e. the playback
thread. This change removes the experimental flag that allowed
calling MediaCodec.start() from the callback thread.
The flag was flipped to true already.
PiperOrigin-RevId: 430689665
This can fix the following warning caused by commit a7aa84a:
Warning: com.google.android.exoplayer2.drm.UnsupportedDrmException.Reason:
The typedef annotation should have @Retention(RetentionPolicy.SOURCE)
Author: Tony Guo <tony.guo.peng@gmail.com>
Date: Mon Feb 28 10:56:52 2022 +0800
We need to access internal state to work around resources not being released on
old API versions. Add a reference to the bug about this and suppress the lint
warning.
#minor-release
PiperOrigin-RevId: 430190794
All (later customizable) GlFrameProcessors after the
ExternalCopyFrameProcessor receive their input from a normal OpenGL
texture not an external texture, so they won't need to worry about
the textureTransformMatrix.
PiperOrigin-RevId: 430165652
getCurrentTrackGroups and getCurrentTrackSelections are
retained for now, but moved from Player to ExoPlayer, to
ease the transition for some application code that currently
uses these methods.
PiperOrigin-RevId: 430036355
There are two major blockers to this test:
- H265 muxing is not available for API<24, so setting video mimeType to H265
will fail on those devices.
- AMR audio encoding is buggy on some device and it's not a widely used format.
The solution: use a video that is encoded with AVC/MP3, to ensure transcoding
to AVC/AAC.
PiperOrigin-RevId: 429648598
Re-enable tests that have no muxer support for timestamps going backwards.
Tests running on the B-frame sample will be added in a future commit.
#mse-bug-week
PiperOrigin-RevId: 429599177
*** Original commit ***
Rollback of 43b796b64d
*** Original commit ***
Rollback of 08c5b1cb0a
*** Original commit ***
Wire up MediaMetricsListener and add configuration to...
***
PiperOrigin-RevId: 429585773
We use the `createForCodec` method that does not take a `MediaFormat` during
transformation, the error message always includes "no configured MediaFormat",
which is false.
PiperOrigin-RevId: 429553573
isTypeSupportedOrEmpty is very specific and a little hard to
understand unless you know the one thing it's useful for. This
commit replaces it with isTypeSupported, which can be used in
conjunction with the recently added containsType method.
PiperOrigin-RevId: 429312712
The track type is derived solely from the content. It does
not depend on any runtime properties such as the player's
capabilities of user track selection. Hence it belongs in
TrackGroup rather than TrackGroupInfo.
Note that this avoids TrackSelectionOverride from having to
re-derive the track type internally.
PiperOrigin-RevId: 429303312
Calling `MediaMuxer.writeSampleData` can block indefinitely on old API versions. It is better not to call this method to fail quickly with an exception rather than getting stuck.
Based on on-device testing media muxer doesn't generally handle out of order samples before API 25. There are a small number of devices where this does succeed but it seems preferable to turn this off everywhere to keep the code simple and have consistent behavior. Once we switch to in-app muxing this limitation will no longer apply.
#mse-bug-week
PiperOrigin-RevId: 429070255
Tested:
Verified that the additional information is available through
instrumentation tests, as well as via manual testing.
#mse-bug-week
PiperOrigin-RevId: 429038695
- The resources were released twice before, which is not necessary since
the MSG_RELEASE message is already in the internal player queue.
- The demo app was failing because the stop watch was stopped in
onTransformationError after being reset.
#minor-release
#mse-bug-week
PiperOrigin-RevId: 428794426
We need TracksInfo.hasTracksOfType to determine which tabs to
display in TrackSelectionDialog.
We need TrackGroupInfo.isAdaptiveSupported to determine whether
to allow multiple selection (check boxes) or not (radio buttons).
PiperOrigin-RevId: 428793739
The GlFrameProcessor handles everything related to the GLSL program,
the FrameEditor manages the GL context and the data flow including
the input SurfaceTexture and output EGLSurface.
This will be split up further in follow-up CLs so that
GlFrameProcessors can be chained. At this CL, there are no
functional changes intended.
PiperOrigin-RevId: 428779179
This change makes sure played server side ads are skipped in a single period
timeline. It avoids creating an ad-MediaPeriodInfo for played postrolls and
creates a content info instead. It also sets the end position for content infos
that terminate the stream before the stream is actually finished. This prevents
the player from continue playing the remaining media delivered by the
MediaPeriod.
We also make sure that the discontinuity of played ads are not reported because
there is actually no discontinuity.
#minor-release
PiperOrigin-RevId: 428734387
The last used shared media period is reused after all media periods have been
released. In case the sample streams are already filled up, they need to be
reset or they download samples from the current position up to the seek
position. This causes long buffering states or load stuck exceptions.
A seek when reusing the shared period takes care for reseting the period or
internally seeks to the current position in the already available samples.
#minor-release
PiperOrigin-RevId: 428484187
We have two ways to choose the minDurationForQualityIncreaseMs value in
AdaptiveTrackSelection: use the configured value for non-live or when
enough buffered data is available, or use a fraction of the available
duration to allow switching when playing close to the live edge.
The decision point when to use which value isn't quite consistent because
we compare against availableDurationUs before making the adjustments. This
means there is range of values where no up-switching is possible despite
perfect buffering. Fix this by choosing the minimum of both values.
Issue: google/ExoPlayer#9784
#minor-release
PiperOrigin-RevId: 428474332
Ignorable ad periods are skipped to resolve the media period id with the
ad playback state of the resulting period. In case of a change in the period
position un-played ad periods are rolled forward to be played.
PiperOrigin-RevId: 428011116
This adds some missing calls to verifyApplicationThread to
ExoPlayerImpl.
Now all public methods start with this call, except listener
registrations because registration after construction on a background
thread is allowed and supported.
PiperOrigin-RevId: 428009498
All the functionality of SimpleExoPlayer has moved to ExoPlayerImpl.
Hence, ExoPlayerImpl can fulfil its own name and become an ExoPlayer
implementation. As a result, ExoPlayer.Builder can return ExoPlayerImpl
directly without using SimpleExoPlayer at all.
#minor-release
PiperOrigin-RevId: 427947028
For when a track is both encrypted and has supplemental data, the sample size will be equal to `block sample size - encryption data size`.
PiperOrigin-RevId: 427807612
We have seen devices running on API21/23 fail transcoding because of setting
encoding profile/level.
Some devices (ale-123/nexus 7) on API21 returns ENOSYS (Function not
implemented) when being configured with a profile setting. (although API21
introduced the capability of setting encoding profile)
Some devices (nexus 5) on API23 fails configuration with a specific parameter
set, despite advertising support for it.
Not setting the baseline profile has no effect on encoding, because when not
set, the encoding will pick a suitable profile to use. Since baseline is
the lowest possible profile, the auto-picked value can't be worse than
baseline.
Ref: b/218696352
PiperOrigin-RevId: 427792124
On some Android devices, the return value of
```
MediaCodecInfo.getCapabilitiesForType(mimeType).profileLevels
```
contains one entry for each encoding profile, like <profile, maxSupportedLevel>
but on some other devices, there are multiple entries for the same profile,
like <HIGH_PROFILE, LEVEL1>, <HIGH_PROFILE, LEVEL2>, <HIGH_PROFILE, LEVEL3>,
where we need to iterate through all the entries and find the max.
PiperOrigin-RevId: 427727030
And also add a test that all Player.Listener events are forwarded
to AnalyticsListener.
The AnalyticsCollector also needlessly implemented
Video/AudioRendererEventListener, which is not needed because all of
the equivalent methods are called directly and never through the
interface.
#minor-release
PiperOrigin-RevId: 427478000
When a asset contains multiple cc embedded tracks, current Cea708Decoder service
block parsing logic is discarding all the data in that frame.
For example, a manifest with 2 embedded CEA-708 close captions tracks:
<Accessibility schemeIdUri="urn:scte:dash:cc:cea-708:2015" value="1=lang:eng;2=lang:spa"/>
when the spanish track is selected (service number 2), when processing the
following CEA-708 frame:
FC9420FD2062FF0829FE8CFCFE9818FEE332FE731FFE1042FE2062FE0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
the frame can be descomposed as:
FF0829 FE8CFC FE9818 FEE332 FE731F FE1042 FE2062 FE0000
08 (00 001000) 0 (sequence) 8 (frame size)
298CFC9818E332731F10 422062 0000
=====
29 (001 01001) 1 (service number) 9 (service block length)
8CFC9818E332731F10
42 (010 00010) 2 (service number) 2 (service block length)
2062
0000 Null block service
The current processCurrentPacket logic will discard the whole frame is discarded
because the first service number found in the frame belongs to service number 1,
which is not the one we are currently looking for.
This commit modifies the processCurrentPacket decoding logic, to take into
account all service blocks available in the frame, by iteraring over the
full frame data and skipping those service blocks we are not interesting in.
Signed-off-by: Jorge Ruesga <jorge@ruesga.com>
There are two sets of listeners in ExoPlayerImpl at the moment,
which can be merged together to use a single ListenerSet. This has
the added advantage that the events that were previously sent
through the ArraySet get additional guarantees provided by ListenerSet
(e.g. correct event ordering and onEvents triggered).
Also add missing constants for onEvents to ensure all Player.Listener
methods have an corresponding constant.
#minor-release
PiperOrigin-RevId: 427415349
This brings listener invocations closer together and removes
unnecessary methods.
Also fixes a bug where a change in track selection parameters only
queued a callback but never flushed it to actually inform the
listeners.
#minor-release
PiperOrigin-RevId: 427201691
In some cases (whose where we previously used EventListener),
AnalyticsCollector is registered as a listener to receive updates,
in other cases it is called directly.
Avoid this inconsistent handling by registering it as normal listener
and removing all callbacks that are handled by the normal listener flow.
The remaining direct usages of AnalyticsCollector calls are those
callbacks that have no equivalent in Player.Listener.
#minor-release
PiperOrigin-RevId: 427201525
With this change, MediaCodecAudioRenderer always configures MediaCodec
with max output channels set to 99 on API 32+.
#minor-release
PiperOrigin-RevId: 427192801
SimpleExoPlayer used to register a listener on ExoPlayerImpl for
the old EventListener callbacks. Now both classes are merged, this is
no longer needed and should be removed in favor of calling methods
directly.
#minor-release
PiperOrigin-RevId: 427187875
We have logic to not immediately interrupt playback when an ad group
fails to load and instead let the current content play and transition
at the point where the ad group should have been.
This logic was broken by dcbdbe5341 because of one of the conditions
used MediaPeriodId.adGroupIndex, which is always -1 for content ids.
It still worked for the last ad group because the next ad group index
was C.INDEX_UNSET.
Fix the issue and amend the test that was meant to catch this to test
the ad failures for the last ad and previous ads.
Also fix the PositionInfo reported in such a case, which was also wrong.
Issue: google/ExoPlayer#9929
#minor-release
PiperOrigin-RevId: 427143223
*** Original commit ***
Rollback of 9b8ed581d0
*** Original commit ***
Add missing imports to ExoPlayerImpl
These were missed in 3bb0210d22
#minor-release
***
***
PiperOrigin-RevId: 427131696
*** Original commit ***
Rollback of 08c5b1cb0a
*** Original commit ***
Wire up MediaMetricsListener and add configuration to disable it.
The listener will automatically forward diagnostics info to the
Android platform. ExoPlayer.Builder gets a new setter that allows
to disable this feature if required.
#...
***
PiperOrigin-RevId: 427131438
*** Original commit ***
Rollback of 3bb0210d22
*** Original commit ***
Move SimpleExoPlayer logic into ExoPlayerImpl
This makes SimpleExoPlayer a simple forwarding wrapper which can be
removed in the future.
The changes are all purely mechanical with none of the potential further
simplifications made yet...
***
PiperOrigin-RevId: 427131338
On some old devices, the encoding level needs to be set with the encoding
profile, but not on newer devices.
The profile/level override is applied by following
https://developer.android.com/guide/topics/media/sharing-video
PiperOrigin-RevId: 427008536
*** Original commit ***
Move SimpleExoPlayer logic into ExoPlayerImpl
This makes SimpleExoPlayer a simple forwarding wrapper which can be
removed in the future.
The changes are all purely mechanical with none of the potential further
simplifications made yet. The only exceptions are name clashes where
either EPI or SEP was calling a method in one of the classes and both
classes had different implementations for the same method name. In these
cases we needed to disambiguate between the two different
implementations (e
***
PiperOrigin-RevId: 426997821
*** Original commit ***
Wire up MediaMetricsListener and add configuration to disable it.
The listener will automatically forward diagnostics info to the
Android platform. ExoPlayer.Builder gets a new setter that allows
to disable this feature if required.
#minor-release
***
PiperOrigin-RevId: 426997342
Transformer's minimum API level is 21, where-as the
full library is still targeting 16. Hence we should
no longer include the transformer module in the
full library dependency.
#minor-release
PiperOrigin-RevId: 426958045
Some strings didn't mention the context in which they are used
(for example as item in a list, or for accessibility). This makes it
harder for translators to choose the most appropriate translation and
grammar.
Also fix repeat and shuffle mode button accessibility descriptions to
indicate the action, not the current state.
PiperOrigin-RevId: 426924163
This ensures Kotlin usages of these IntDef annotations in the 'old'
position will continue to compile.
'Frequently used' is a subjective judgement. I have a parallel change
that marks all the other public IntDefs in the library as TYPE_USE
(those that I've judged to be 'rarely used' by apps).
A follow-up change will fix the positions of existing usages to be as if
they're only TYPE_USE.
#minor-release
PiperOrigin-RevId: 426427334
This is a breaking change if the annotation itself is in use in Kotlin
code. It's judged that the IntDefs in this commit are unlikely to be
referred to often in Kotlin code. This is because they're either:
- Related to esoteric parts of the library, or
- In a common part of the library but only returned from methods (and
never passed to callback methods).
A follow-up change will fix the positions of existing usages to match
this new config.
#minor-release
PiperOrigin-RevId: 426410237
This is not backwards compatible if the @SelectionReason annotation is
used in Kotlin code, but before this change there aren't many library
surfaces that return a value annotated with @SelectionReason, so it
seems relatively unlikely that it is in use in any/many apps.
A follow-up change will fix the positions of existing usages to match
this new config.
#minor-release
PiperOrigin-RevId: 426409877
This is only used inside AudioFocusManager, it doesn't need to public.
Also mark it TYPE_USE and update the position to match.
#minor-release
PiperOrigin-RevId: 426407790
This is only used in DefaultAudioSink, so we could move it there and
make it private - but at that point we might as well refer to the
underlying AudioManager constants instead.
#minor-release
PiperOrigin-RevId: 426407661
Keep values related to LegacyPlayerView in attrs_legacy_player_view.xml
and put all values related to LegacyPlayerControlView back in their
original locations. We plan to remove LegacyPlayerView (and
attrs_legacy_player_view.xml) from media3, but will keep
LegacyPlayerControlView - so the separation of the XML files needs to
reflect this split.
#minor-release
PiperOrigin-RevId: 426406973
Fallback is only disabled for robolectric and instrumentation tests.
For MH tests, fallback is not disabled, as it may be needed due to
the broad range of devices available.
PiperOrigin-RevId: 426403167
This only changes IntDefs that cannot be used by apps because they're
either private or package-private.
A follow-up change will fix the positions of existing usages to match
this new config.
#minor-release
PiperOrigin-RevId: 426372273
Reason for not rolling back the rollback 9151103968: file name changed and
file content moved, the automated tool is unable to correctly apply the change.
Apply suggested AVC profile depending on the API version.
Use `AVCProfileHigh` only when there's encoder support.
PiperOrigin-RevId: 426363780
This makes the delegation model more explicit, and prevents the javadoc
compiler from just pulling in the Player javadoc automatically - which
can lead to some confusion when some method definitions in Player depend
on other methods (e.g. seekForward() is defined in terms of
getSeekForwardIncrement()).
Issue: google/ExoPlayer#9897
#minor-release
PiperOrigin-RevId: 426359004
These were messed up in 74c6ef9ba0
Also suppress deprecation warnings when we're just forwarding a
deprecated method to the delegate.
#minor-release
PiperOrigin-RevId: 426351791
This change makes GlUtil.Program an outer class named GlProgram,
and also moves private static helpers as well as the inner classes
Attribute and Uniform which were only used by GlUtil.Program to
GlProgram. Other static utility methods remain in GlUtil.
No functional changes intended.
PiperOrigin-RevId: 426119299
The longer list of targets is only necessary for backwards
compatibility with existing Kotlin code that will stop compiling
if the position of the annotation becomes 'wrong' by marking it only
TYPE_USE. Since none of these IntDefs have been released (except in
media3 alpha1) we don't need to maintain this compatibility.
Also add a comment to all the places that *do* need the longer list of
targets, in order to explain why it's there and discourage copy-pasting
when defining new IntDefs in future.
Also fix some single-element arrays to remove the array notation.
#minor-release
PiperOrigin-RevId: 426108537
The listener will automatically forward diagnostics info to the
Android platform. ExoPlayer.Builder gets a new setter that allows
to disable this feature if required.
#minor-release
PiperOrigin-RevId: 426099872
Remove most of the customisation documentation, since StyledPlayerView
isn't really designed to be customised as deeply as PlayerView.
Also remove most documentation around StyledPlayerControlView,
especially as a standalone controller class - since it doesn't work
well for this use-case.
#minor-release
PiperOrigin-RevId: 426090762
- The MIME type should ideally default to HEVC if there is an encoder for it.
- Next, check if AVC is supported.
- If there is no encoder for AVC, then we should pick an encoder in the list of
existing encoders instead of abandoning the transformation.
PiperOrigin-RevId: 425900638
This allows the same DefaultMediaSourceFactory instance to be used as
the contentMediaSourceFactory inside
ImaServerSideAdInsertionMediaSource.
PiperOrigin-RevId: 425846609
The current implementation of ExoPlayer Cea708 is processing the whole service block without
taking into account the defined service block size, which could cause the execution of
unwanted command.
The following set of Cea708 represents a real use case of the above.
``` hex
FC9420FD152FFF0929FE8CFCFE9818FEE332FE731FFE1044FE8B03FE8CFCFE0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FC942CFD1520FF4649FE8CFEFE9918FEE332FE731FFE1000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FC942FFD9470FF8924FE8B03FE8CFCFE4A92FE0300FE9005FE0091FE2A00FE0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FC9420FD97A1FFC829FE8CFEFE9918FEE332FE731FFE1043FE9203FE0100FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FC94D6FDCEEFFF082AFE9202FE0C90FE0500FE912AFE0000FE424EFE6F00FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FCC168FD20F4FF4422FE4168FE4220FE7400FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FC2C20FDE520FF8422FE2C20FE4265FE2000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FC64EFFD70EFFFC422FE646FFE4270FE6F00FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FC6EA7FD6E67FF0422FE6E27FE426EFE6700FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FCF420FD6173FF4422FE7420FE4261FE7300FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FC67E5FD206DFF8422FE6765FE4220FE6D00FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
FCF480FD7579FFC422FE7403FE4275FE7900FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000FA0000
```
The above frames should be displaying the following text on the screen (or at least the first part of it).
```
Ah, don't get
all sappy about it
```
ExoPlayer is currently parsing this block as follow (in parentheses the defined service block size):
```
(2) 22416842207400
G0 A
G0 h
G0 B
G0 \u0020
G0 t
C0 COMMAND_NUL
(2) 222C2042652000
G0 ,
G0 \u0020
G0 B
G0 e
G0 \u0020
C0 COMMAND_NUL
(2) 22646F42706F00
G0 d
G0 o
G0 B
G0 p
G0 o
C0 COMMAND_NUL
(2) 226E27426E6700
G0 n
G0 '
G0 B
G0 n
G0 g
C0 COMMAND_NUL
(2) 22742042617300
G0 t
G0 \u0020
G0 B
G0 a
G0 s
C0 COMMAND_NUL
(2) 22676542206D00
G0 g
G0 e
G0 B
G0 \u0020
G0 m
C0 COMMAND_NUL
(2) 22740342757900
G0 t
C0 COMMAND_ETX
G0 B
G0 u
G0 y
C0 COMMAND_NUL
(2) 2392030C4220730000
C1 COMMAND_SPL
G0 B
G0 \u0020
G0 s
C0 COMMAND_NUL
C0 COMMAND_NUL
(2) 22616C42656E00
G0 a
G0 l
G0 B
G0 e
G0 n
C0 COMMAND_NUL
```
So it ended up processing the following cue text (additional unwanted commands could be executed as well)
```
AhB t, Be doBpon'Bngt BasgeB mt
BuyB salBen
```
If instead the parsing logic take into account the service block size
```
(2) 22416842207400
G0 A
G0 h
(2) 222C2042652000
G0 ,
G0 \u0020
(2) 22646F42706F00
G0 d
G0 o
(2) 226E27426E6700
G0 n
G0 '
(2) 22742042617300
G0 t
G0 \u0020
(2) 22676542206D00
G0 g
G0 e
(2) 22740342757900
G0 t
C0 COMMAND_ETX
(2) 2392030C4220730000
C1 COMMAND_SPL
(2) 22616C42656E00
G0 a
G0 l
```
which is translated to (again I didn't processed all the frames, just a few of them)
```
Ah, don't get
al
```
which is what we are looking for.
This PR modifies service block parsing logic to honor service block size instead of read the full service block buffer.
Signed-off-by: Jorge Ruesga <jorge@ruesga.com>
This makes SimpleExoPlayer a simple forwarding wrapper which can be
removed in the future.
The changes are all purely mechanical with none of the potential further
simplifications made yet. The only exceptions are name clashes where
either EPI or SEP was calling a method in one of the classes and both
classes had different implementations for the same method name. In these
cases we needed to disambiguate between the two different
implementations (example: ExoPlayerImpl.setListener was renamed to
setEventListener).
#minor-release
PiperOrigin-RevId: 425823095
Tested by confirming transformations still work and write to a output file in a
scoped-storage directory on a:
* Nexus 6P API 23 emulator
* Google Pixel 4 API 31 physical device
PiperOrigin-RevId: 425644266
In some cases we create empty playback metrics with no corresponding
events (e.g. when an app seeks to a new media item and immediately
releases the player). There is no benefit in having completely empty
metrics entries, so it's cleaner to not report them in such cases.
#minor-release
PiperOrigin-RevId: 425609010
- Add a checkbox in the demo app to enable experimental HDR editing.
- Add an `experimental_` method to `TransformationRequest` to enable HDR editing.
- Add fragment/vertex shaders for the experimental HDR pipeline. The main difference compared to the existing shaders is that we sample from the decoder in YUV rather than RGB (because the YUV -> RGB conversion in the graphics driver is not precisely defined, so we need to do this to get consistent results), which requires the use of ES 3, and then do a crude YUV -> RGB conversion in the shader (ignoring the input color primaries for now).
- When HDR editing is enabled, we force using `FrameEditor` (no passthrough) to avoid the need to select another edit operation, and use the new shaders. The `EGLContext` and `EGLSurface` also need to be set up differently for this path.
PiperOrigin-RevId: 425570639
TransformerTest sounds like a unit test for Transformer but these
tests test behaviour that involves multiple stages of the pipeline.
PiperOrigin-RevId: 425378369
Add a constructor that takes a DrmSessionManagerProvider.
This allows R8 to strip the default implementation.
#minor-release
PiperOrigin-RevId: 425330083
Some phone with limited memory can't allocate bigger
shared memory buffers.
This might or might not be related to Binder's 1M
transaction limit.
Tested on Pixel 4 by setting the minimum buffer size to
1h.
https://github.com/google/ExoPlayer/issues/9712
#minor-release
PiperOrigin-RevId: 425324536
If
a) the end of stream buffer arrives with a frame rather than an
empty buffer or
b) processDataV29() renders several decoder output buffers to the
FrameEditor's input Surface immediately before encountering the
EOS flag
these frames were previously stuck in the FrameEditor's input Surface
and never fed to the encoder.
PiperOrigin-RevId: 424898820
This test tests that all frames are processed when transcoding
video to a different sample MIME type (and that the transformation
completes successfully).
PiperOrigin-RevId: 424896014
Enforcing the correct thread usage has been enabled since 2.13.0.
Opting-out of this enforement is dangerous as it can hide very hard
to debug bugs.
PiperOrigin-RevId: 424815808
If muxerWrapper.release() was throwing an exception, the progress state
was not updated and getProgress could throw an exception.
#minor-release
PiperOrigin-RevId: 424696783
When the decoder output buffer was partially read, a call to
Codec.getOutputBuffer() was returning the same buffer, but with the
position reset to 0. The reason was that, in
Codec.maybeDequeueAndSetOutputBuffer(), mediaCodec.getOutputBuffer()
was called with the same buffer index (L350 in old rev), even though
there was already a buffer available (outputBufferIndex >=0). This
change avoids calling mediaCodec.getOutputBuffer() if the previous
buffer has not been released.
#minor-release
PiperOrigin-RevId: 424612197
If the encoder picks a fallback resolution the video pipeline needs
to take this into account when configuring the frameEditor and when
setting up the fallback TransformationRequest that's passed to the
fallbackListener.
PiperOrigin-RevId: 424611290
Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.
The new class is easily configurable and highly tested.
Manual test was used to catch any regression.
https://github.com/google/ExoPlayer/issues/8891
PiperOrigin-RevId: 424602011
AnalyticsCollector can't be null when passed into ExoPlayerImplInternal,
so there is no need to pass it around as nullable.
PiperOrigin-RevId: 424594031
Two of the sessions are finished at the same time in the test
and the order of the corresponding callbacks depends on the randomly
generated session string and the order these strings are stored in a
HashSet.
Update test to assert both callbacks are called and they contain the
right arguments, but don't assert on the order of these two callbacks.
PiperOrigin-RevId: 424548819
Ad playback shouldn't be affected by manual speed adjustments set
by the user. This change enforces unit speed for ad playback.
Issue: google/ExoPlayer#9018
PiperOrigin-RevId: 424546258
This only affects playbacks using the
experimentalUseProgressiveMediaSourceForSubtitles method.
Also update the SingleSampleMediaSource instantiation to be more
similar, to try and highlight differences like this in future.
PiperOrigin-RevId: 424545980
In some cases we need to update the PlaybackParameters at period
boundaries, for example when switching from live to VOD and live
playback speed adjustment was still active at the point of switching.
Currently, we send the update when the playing MediaPeriod changes in
EPII, which is slightly too late because the new speed gets only applied
after the entire existing AudioTrack buffer has been played out.
We can time the update slightly better by updating the values at the
point where we change the reading period. This makes the update slightly
too early because it also applies to all samples currently in the
decoder. But generally, this is a lot better because the time spent
in the decoder is likely to be considerably lower than the duration of
the AudioTrack buffer.
Note that this change isn't perfectly aligning to the period boundary,
but reduces the number of samples with the wrong speed to a minimum.
If we are getting around to add additional code that allows updating
the speed at exactly the boundary, it also needs to be triggered from
the reading period update, so the new code location is also helpful in
the future.
Issue: google/ExoPlayer#9018
PiperOrigin-RevId: 424540551
Issue: google/ExoPlayer#9775
We got a few issues for this on GH already. Some RTSP servers do not provide
track timing in PLAY responses, or the timings are invalid.
Missing timing means the RTSP stream is not seekable. Added method to
1. Update the timeline that seek is not possible
2. Report read discontinuity so that playback can start from the beginning.
PiperOrigin-RevId: 423281439
If the output sample MIME type is inferred from the input
but is not supported by the muxer, we fallback to transcoding
to a supported sample MIME type.
The audio and video renderers need to make sure not to select the PassthroughSamplePipeline for this case. Which sample MIME type
to choose is decided by the EncoderFactory.
PiperOrigin-RevId: 423272812
To avoid the `MediaPeriodQueue`to discard the reading period, we can set the next ad of an ad group early and then (possibly) only change it's duration once we receive the actual duration. This way we avoid a rebuffering as a result of the reading period being discarded.
The change also takes care to properly set ad break and their durations when we join the live stream at the moment when an ad is playing.
PiperOrigin-RevId: 423163467
We currently only end sessions on Timeline updates if the associated media
is no longer in the playlist. But we should also end all sessions that are
finished as a result of the timeline update (similar to how this is done for
discontinuities). This issue was introduced by 394ab7bcfd
PiperOrigin-RevId: 423075855
We currently don't check which session is causing a network transfer
(it could be a preloaded item in a playlist). To clearly associate
network transfer data with sessions, we need to keep track of
transferred bytes and transfer time per session.
PiperOrigin-RevId: 422867845
We may fall back to a different sample MIME type because
a) the sample MIME type inferred from the input is not supported
by the muxer or b) no encoders are available for the the requested
sample MIME type.
PiperOrigin-RevId: 422849036
With this change, the MediaCodecAudioRenderer configures the MediaCodec
to not downmix audio only if spatialization can be applied. This way,
decoders who are downmixing by default are left doing so when
spatialization cannot be applied. The renderer re-initializes the codec
when spatialization properties change mid-playback.
PiperOrigin-RevId: 422822952
TransformationRequest is otherwise immutable, so if we modify the
transformationMatrix in place (done before this cl) this may cause
confusing behaviour for apps when they reuse a TransformationRequest.
PiperOrigin-RevId: 422822916
Currently, we keep the values for dropped/played frames, audio underruns
and current formats from the last session, causing double reporting
of counters and wrong track change reasons for formats. All these
values should be reset when the active session is finished, so that the
new session can start from scratch.
PiperOrigin-RevId: 422798406
Currently, we always end the current session if onSessionFinished is
called. However, the finished session may not be the active one (for
example when discarding prebuffered items in a playlist). To make this
code more robust, we can save the active session id explicitly and only
end this session.
PiperOrigin-RevId: 422788542
After implementing fallback, it won't always be possible to
differentiate between muxer and encoder as the cause of an output
format not being supported.
PiperOrigin-RevId: 422780443
This change enables the ImaServerSideAdInsertionMediaSource for multi-period content. The global ad playback state is split into pieces for each period and the window and period durations are calculated accordingly in the ServerSideAdInsertionTimeline.
For multi-period live content (DASH), the ad playback state is not set with this change. This is deferred to a follow up CL. Splitting is very tricky. For each timeline update the windowStartTimeUs may vary for some milliseconds relative to the start of the period.positionInWindowUs. This requires to either introduce some fuzzy logic or to choose a different approach than for multi-period VOD. Because mistakes within the playback states of subsequent moving live windows produces crashes, it seems sensible to defer this for now and keep this change in a separate future CL (unblock further work, easy to rollback).
In this state, live DASH stream are working and the ad overlay is placed over the player correctly bu the SDK. However, ads are not reported by the position discontinuity event. Similarly, the player.isPlayingAd() does never returns true when a ad period is playing.
PiperOrigin-RevId: 422539770
This CL implements fixing the input format to the encoder spec. Fixed
parameters include:
- MIME type
- Profile & level
- Resolution
- frame rate, and
- bitrate
PiperOrigin-RevId: 422513738
The MediaMetricsListener currently just looks at the mime type and
doesn't use the inference based on the URI if no mime type is set.
Also change default type to OTHER to avoid classifying streams from
URLs without clear file extension as progressive.
PiperOrigin-RevId: 422373381
Instead of having types in flac inherit types in vorbis, make types in
vorbis inherit types in flac. This is a bit of a hack and somewhat
messy, but it retains backwards compatibility.
Collapse the two variations of `VorbisUtil.buildMetadata` into a single
method called `VorbisUtil.parseVorbisComments` that only takes a list
of vorbis strings, compared to previously where it would take strings
and picture frame instance. Any code that relied on the old signature
now either concatenates picture frames and vorbis comments or copies
vorbis comments into an existing metadata instance.
Simplify how the comment header is parsed and eliminate a few possible
bugs in the process, such as:
- Metadata being overwritten directly by the comments header.
- The packet being rewound to 0 if it cannot find a comment header,
which might result in the cursor being moved to a bad position.
In the old `flac` module, superclass the deprecated types under the
moved types in the `vorbis` module. This ensures backwards compat
with existing library users.
Adaptive video and audio selections will be limited to formats with
the same level of DecoderSupport and HardwareAccelatationSupport, unless
specifically allowed by new flags.
If different levels of decoder support are available, prefer primary
over fallback decoders and hardware-accelerated over software decoders
(in this order). For video, also prefer more efficient codecs, if both
are supported by hardware-accelerated primary decoders.
Issue: google/ExoPlayer#4835
Issue: google/ExoPlayer#9565
PiperOrigin-RevId: 422345048
This change moves the video track selection to the generic
selection method introcuced for audio and text. This ensures
we can apply the same criteria for fixed and adaptive video
track selections. Implicitly, this reorders the preferences
for adaptive tracks to give non-quality preferences (like
preferred MIME type or preferred role flags) a higher priority
than number of tracks in the selection.
Issue: google/ExoPlayer#9519
PiperOrigin-RevId: 422310902
This does currently only happen by chance in replaceStream (called from
enable) if the stream previosly played read until C.TIME_END_OF_SOURCE.
enable already makes all changes done in resetPosition (except resetting
the reading position), so it's less error-prone and makes the intention
clearer if the same code is called from both enable and resetPosition.
The effect of this bug was quite limited because the numerical value
of readingPositionUs was only relevant for periods with changing
durations and server-side inserted ads.
PiperOrigin-RevId: 422300690
The app will be notified about fallback using a callback on
Transformer.Listener. Fallback may be applied separately for
the audio and video options, so an intermediate internal
FallbackListener is needed to accumulate and merge the track-specific
changes to the TransformationRequest.
PiperOrigin-RevId: 421839991
We currently run (almost) the same code for all track types.
De-duplicate this by using a single method that takes functional
interfaces for track-type dependent logic.
This has the benefit that all track-type dependent logic is
contained within their subclasses and the generic logic doesn't
need to make any assumption about the eligibility of tracks for
selection or adaptation, and doesn't need to access Parameters.
Make this change for audio and text only for now. Video can
be updated in a subsequent change.
PiperOrigin-RevId: 421811411
Simplifying and clarifying variables, and adding comments.
Tested by confirming demo-gl and demo-transformer both
correctly display videos
PiperOrigin-RevId: 421792079
The existing wording would be correct if prefixed with
"Returns false if [...]", but it seems confusing to a document a boolean
method in terms the condition it returns false - so I reworded it in
terms of when it returns true.
#minor-release
PiperOrigin-RevId: 421682584
After this change exceptions throw by MediaCodec during
encoding/decoding will result in TransformationExceptions with
ERROR_CODE_ENCODING_FAILED/ERROR_CODE_DECODING_FAILED.
Before this change ERROR_CODE_FAILED_RUNTIME_CHECK was used.
PiperOrigin-RevId: 421560396
Expected images are taken on emulators, so a larger acceptable
difference from expected images must be accepted on physical devices.
PiperOrigin-RevId: 421543441
Live speed adjustment is used for all live playback at the moment,
but has no user visible effect if the media is not played with low
latency. To avoid unnecessary adjustment during playback without
benefit, this change restricts the live speed adjustment to cases
where either the user requested a speed value in the MediaItem or the
media specifically defined a low-latency stream.
Issue: google/ExoPlayer#9329
PiperOrigin-RevId: 421514283
This ensures we test the API level specific logic, in particular
around 5G-NSA detection.
Robolectric has a remaining bug that it doesn't support listening
to service state changes. Hence, we need to ignore some tests on
these API levels still until this is fixed.
PiperOrigin-RevId: 421505951
Codec and its factories can use MediaCodec directly as for API >= 21,
the SynchronousMediaCodecAdapter methods used in Codec just correspond
to a single MediaCodec call each so there is no reason to have another
wrapping layer.
PiperOrigin-RevId: 421041177
Exceptions thrown by MediaMuxer are converted MuxerExceptions
and later to TransformationExceptions with ERROR_CODE_MUXING_FAILED.
PiperOrigin-RevId: 421033721
The media codec renderers have fallback logic in getDecoderInfos
to assume that E-AC3 decoders can handle the 2D version of E-AC3-JOC and
that H264/H265 decoders can handle some base layer of Dolby Vision
content. Both fallbacks are useful if there is no decoder for the
enhanced Dolby formats.
Both fallbacks are not applied during track selection at the moment
because the separate MediaCodecInfo.isCodecSupported method verifies
that the mime type corresponding to format.codecs is the same as the
decoder mime type (which isn't true for the fallback case).
To fix the fallback logic, we can just completely remove this additional
check because it's not needed in the context of this method that is only
called after we already established that the decoder can handle the
format.sampleMimeType.
In addition, we need to map the Dolby Vision profiles to the equivalent
H264/H265 profile to make the codec profile comparison sensible again.
PiperOrigin-RevId: 420959104
SampleQueues may be released in the context of a finally block
after an out of memory error. Allocating in that scenario can
throw yet a new OutOfMemoryError. By safely releasing SampleQueue
memory, we increase the possibility of handling the error
gracefully.
PiperOrigin-RevId: 420859022
This method has been deprecated since 2.12.0 ([commit](d1bbd3507a)).
Also remove
DashMediaSource.Factory#setLivePresentationDelayMs(long, boolean), this
method has been deprecated since 2.13.0 ([commit](41b58d503a)).
PiperOrigin-RevId: 420719877
This class is already called StyledPlayerControlViewLayoutManager, it
seems unecessary to repeate the 'styled' word again in this context.
PiperOrigin-RevId: 420711161
These were missed in 46ab94bd41
These references will be re-written to PlayerView when exporting to
exoplayer2, so this commit results in some small reformatting changes.
Also fix a reference to LegacyPlayerControlView that should be
StyledPlayerControlView.
PiperOrigin-RevId: 420707706
The draw method was disabling vertex attrib arrays but not re-enabling them. Remove the call to disable the vertex attrib arrays so that then remain enabled after the program is created.
Manually verified by setting the surface type to spherical in the demo app and playing a spherical sample video.
Issue: google/ExoPlayer#9782
PiperOrigin-RevId: 420707503
Refactor the overall module to place the unified vorbis tags into a
single package called `vorbis`. Also re-intoduce the vorbis tags
in their original `flac` module, but deprecate them.
The encoder and sonic are now set up in the constructor rather
than in a configuration method called from processData(). This
is more similar to VideoSamplePipeline and reduces null checks.
PiperOrigin-RevId: 420260526
Only allowing TransformationExceptions to be created using the factory methods helps keeping error messages consistent. This is consistent with ExoPlaybackException.
PiperOrigin-RevId: 419841025
The UI component will be switched over to use them in a
subsequent commit, once the translations have been
imported.
Issue: google/ExoPlayer#9811
PiperOrigin-RevId: 419632617
Compensate for aspect ratio of input frames, so that they're applied on
rectangular frames instead of square normalized-device-coordinate frames.
This fixes distortion most visible when rotating any GL video 45°
(non-rectangular frames) or 90° (stretched frames)
Tested by rotating several landscape/portrait demo videos.
(Automated tests will follow in <unknown commit>)
PiperOrigin-RevId: 419619743
* Move checking that the output format is supported by the muxer
from supportsFormat (which deals with the input format) to
ensureConfigured.
* Add maps for the supported MIME types so that the muxer can
return what MIME types it supports rather than just check a
MIME type.
PiperOrigin-RevId: 419578165
The `main` role distinguishes a track from an `alternate`, but unlike
`SELECTION_FLAG_DEFAULT` it doesn't imply the track should be selected
unless user preferences state otherwise. e.g. in the case of a text
track, the player shouldn't enable subtitle rendering just because a
`main` text track is present in the manifest.
The `main`/`alternate` distinction is still available through
`Format.roleFlags` and the `ROLE_FLAG_MAIN` and `ROLE_FLAG_ALTERNATE`
values.
This behaviour was originally [added in 2.2.0](7f967f3057),
however at the time the `C.RoleFlags` IntDef did not exist. The IntDef
was [added in 2.10.0](a86a9137be).
PiperOrigin-RevId: 418937747
* @Override is not a TYPE_USE annotation, so should appear before any modifiers and after Javadocs. @CryptoType is a TYPE_USE annotation, so should appear after modifiers and directly before the type.
PiperOrigin-RevId: 418811744
This is better than silently dropping tracks as done previously. Later,
we will implement fallback to transcoding to a supported MIME type.
PiperOrigin-RevId: 418006258
This more closely matches the pattern we have for all implementations
except DefaultMediaSourceFactory (e.g. ProgressiveMediaSource.Factory)
and other factory interfaces like (Http)DataSource.Factory.
PiperOrigin-RevId: 417826803
Switch to using sentence-case naming convention but with one character prefixes for different types.
This is a no-op change.
PiperOrigin-RevId: 417791624
Some have been deprecated since 2.13.0
([commit](5b9fa7d7d9)):
* `setDrmSessionManager(DrmSessionManager)`
* `setDrmHttpDataSourceFactory(HttpDataSource.Factory)`
* `setDrmUserAgent(String)`
And the rest have been deprecated since 2.12.0
([commit](d1bbd3507a)):
* `setStreamKeys(List<String>)`
* `createMediaSource(Uri)`
PiperOrigin-RevId: 417622794
Move static factories into a separate class and make it implement an interface
that will let tests customize encoder/decoder creation.
PiperOrigin-RevId: 417610825
To support multi-period content we need to store AdPlaybackStates and
SharedMediaPeriod by the periodUid as a key. While after this no-op CL,
we still only support single-period content, storing these resources by
periodUid is the ground work for multi-period support being added in an
follow-up CL.
PiperOrigin-RevId: 416836445
Keep the old interface deprecated so any app code implementing it by
name (rather than with a lambda) will continue to work.
PiperOrigin-RevId: 416816566
Transformer uses ExoPlayer for reading input. Apps using Transformer
do not need to know this. So, PlaybackExceptions are converted to
TransformationExceptions with the same message, cause and error code.
The corresponding IO error codes are copied from PlaybackException.
PiperOrigin-RevId: 416793741
We check the fraction of the available duration we have already
buffered for live streams to see if we can increase the quality.
This fraction compares against the overall available media duration
at the time of the track selection, which by definition can't include
one of the availabe chunks (as this is the one we want to load next).
That means, for example, that for a reasonable live offset of 3 segments
we can at most reach a fraction of 0.66, which is less than our default
threshold of 0.75, meaning we can never switch up.
By subtracting one chunk duration from the available duration, we make
this comparison fair again and allow all live streams (regardless of
live offset) to reach up to 100% buffered data (which is above our
default value of 75%), so that they can increase the quality.
Issue: google/ExoPlayer#9784
PiperOrigin-RevId: 416791033
Before the introduction of the MediaCodecAdapter, users could get
access directly to the MediaCodec instance from
MediaCodecRenderer.getCodec() and then retrieve the codec metrics.
This change exposes MediaCodec.getMetrics() on the MediaCodecAdapter.
Issue: google/ExoPlayer#9766
#minor-release
PiperOrigin-RevId: 416343023
This is a no-op because all the <resource> elements from these XML
files are effectively concatenated together during building.
PiperOrigin-RevId: 416326534
*** Original commit ***
Rollback of 0aa23b08b1
*** Original commit ***
Add capability flags for hardware and decoder support
Issue: google/ExoPlayer#9565
***
***
PiperOrigin-RevId: 416285603
*** Original commit ***
Rollback of 27f905f571
*** Original commit ***
Don't sort decoders by format support in supportsFormat
This is a no-op change that updates supportsFormat to use the
decoder list before it's reordered by format support. Instead,
supportsFormat iterates through the decoders listed in their
original priority order as specified by the MediaCodecSelector.
The end result is identical.
This is n...
***
PiperOrigin-RevId: 416269130
*** Original commit ***
Don't sort decoders by format support in supportsFormat
This is a no-op change that updates supportsFormat to use the
decoder list before it's reordered by format support. Instead,
supportsFormat iterates through the decoders listed in their
original priority order as specified by the MediaCodecSelector.
The end result is identical.
This is necessary groundwork for a subsequent change that will
indicate in Capabilities whether the decoder that suppports the
format is the primary one as specifi
***
PiperOrigin-RevId: 416170612
This change adds a delegate class that loads and forwards calls
to a Spatializer with reflection, so that we can use the Spatializer
API before we update the compile SDK target to 32.
PiperOrigin-RevId: 416027289
The Robolectric implementation of HttpURLConnection forwards to the JRE
implementation [1], which behaves differently to the Android one
available on devices and emulators. For these tests to be a realistic
test of the HTTP stack used in real playbacks we can't use Robolectric.
Similar to df0e89c167
[1] https://github.com/robolectric/robolectric/issues/6769#issuecomment-943556156
PiperOrigin-RevId: 416013662
Opening the `DataSource` is one of the longer operations in the `Loader` sequence, as it requires a round trip to the origin server. This change allows all the potential `Loader` threads to perform this operation before one of them is forced to wait on shared TimestampAdjuster initialization.
Also, the initialization segment load will never produce media samples, so there is no need for it to wait for `TimestampAdjuster` initialization.
UnsupportedEglVersionException() is only used once, and seems a bit too
specific for Transformer. Also, it's possible for eglCreateContext to fail for
other reasons besides lack of support, so it wasn't always accurate when
thrown.
It is possible for devices not to support EGL version 2.0 though, per
https://source.android.com/devices/graphics/implement-opengl-es, which doesn't
specify the EGL version that must be supported.
PiperOrigin-RevId: 415489396
tl;dr:
In the previous transformer, the transcoding flow is
- If a the GL's input surface (from decoder) does not have data, wait 10ms
(DO_SOME_WORK)
- Else, make the decoder render **ONE** frame to the GL's input surface
- Wait at least 10ms, until the frame's texture is available
- Then process the texture
The process is quite slow, so in the new version, we do:
- If a the GL's input surface (from decoder) does not have data, wait 10ms
(DO_SOME_WORK) **same**
- Else, make the decoder render **as many frames** to the GL's input surface
- Process **as many** available textures in this DO_SOME_WORK cycle
PiperOrigin-RevId: 415474722
*** Original commit ***
Make audio track min buffer size configurable.
Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.
The new class is easily configurable and highly tested.
Manual test was used to catch any regression.
https://github.com/google/ExoPlayer/issues/8891
***
PiperOrigin-RevId: 415469179
The input rotation is used to rotate the video during decoding, the
video is rotated so that it is in landscape orientation before encoding
and a rotation is added to the output format where necessary so that
the output video has the same orientation as the input.
PiperOrigin-RevId: 415301328
Refactor GlUtil.java to be a bit more readable. Also, reorder, inline, and
rename a few things. Refactoring change only. No functional changes intended.
PiperOrigin-RevId: 415283874
For DASH manifests, we merge min/max live latency values from various
sources and they may not be consistent with each other. To ensure we
use a sensible configuration in all cases, we can add more correctness
checks:
1. Limit the min/max values to fall into the available live window.
2. Ensure that maxLatency >= minLatency in all cases.
Issue: google/ExoPlayer#9750
PiperOrigin-RevId: 415282938
Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.
The new class is easily configurable and highly tested.
Manual test was used to catch any regression.
https://github.com/google/ExoPlayer/issues/8891
PiperOrigin-RevId: 415268938
Being able to see the output of the GL pipeline is useful for debugging. For
example, when we previously saw flakiness it would have been useful to be able
to tell quickly whether the output looked wrong without needing to run a
transformation to the end then inspect the output file, and when working on
support for HDR editing it's useful to be able to do manual testing on devices
that don't support HDR encoding (but do support decoding/processing it with
GL).
Also change the progress indicator to be linear as this looks better in the
demo app when shown next to the debug preview.
PiperOrigin-RevId: 414999491
This is a no-op change that updates supportsFormat to use the
decoder list before it's reordered by format support. Instead,
supportsFormat iterates through the decoders listed in their
original priority order as specified by the MediaCodecSelector.
The end result is identical.
This is necessary groundwork for a subsequent change that will
indicate in Capabilities whether the decoder that suppports the
format is the primary one as specified by the MediaCodecSelector
(i.e., the one at index=0 in the lists that are now used).
Issue: google/ExoPlayer#9565
PiperOrigin-RevId: 414971986
The color set via textAppearance is overridden by any non-null
textColor set directly on the style. We always want the specific
properties the textAppearance specifies, so set them directly to
prevent them from being overridden.
#minor-release
Issue: google/ExoPlayer#9765
PiperOrigin-RevId: 414967143
Configure MediaCodec in API 32+ to always output 99 channels
so that we use the audio is spatialized, if the platform can apply
spatialization to it.
In a follow-up change, the output channel count will be set based on the
device's spatialization capabilities.
PiperOrigin-RevId: 414751543
`DefaultAudioSink` already has 3 telescoping
constructors and an other one would be have been
needed to add a buffer size tuning option.
PiperOrigin-RevId: 414703366
The existing code creates an imbalance between `inputBufferCount` and
`droppedBufferCount` by adding 'dropped source buffers' to
`droppedBufferCount` but not to `inputBufferCount`. This results in
assertion failures in `DashTestRunner`.
PiperOrigin-RevId: 414672175
It's been observed that some devices fail when releasing a secure codec
attached to a surface and immediately trying to create a new codec
(secure or insecure) attached to the same surface. This change catches
all exceptions thrown during codec creation, sleeps for a short time,
and then retries the codec creation. This is observed to fix the problem
(we believe this is because it allows enough time for some background
part of the previous codec release operation to complete).
This change should have no effect on the control flow when codec
creation succeeds first time. It will introduce a slight delay when
creating the preferred codec fails (while we sleep and retry), which
will either delay propagating a permanent error or attempting to
initialize a fallback decoder. We can't avoid the extra delay to
instantiating the fallback decoder because we can't know whether we
expect the second attempt to create the preferred decoder to succeed or
fail. The benefit to always retrying the preferred decoder creation
(fixing playback failures) outweighs the unfortunate additional delay
to instantiating fallback decoders.
Issue: google/ExoPlayer#8696
#minor-release
PiperOrigin-RevId: 414671743
The TrackSelection API returns two indexes, the index of the selected
track (`getSelectedIndex()`) and the index in the `TrackGroup`. The `HlsChunkSource`
and `HlsSampleStreamWrapper` should only work with the later.
Actually the `getSelectedIndex()`, current selected track index is really only useful on
the API to determine if there is a valid selection or not. The index is
really internal to `TrackSelection`
Also, add 144p as an acceptable output resolution, to allow for
a more obvious resolution difference when running the demo.
PiperOrigin-RevId: 414406664
The new field matches the platform's
AudioAttributes.getSpatializationBehavior() API added in Sv2. At the
moment, the platform API is called via reflection, until Sv2 is released
and the compile SDK target can be increased to 32.
PiperOrigin-RevId: 414406126
outputHeight is the actual output height while
transformation.outputHeight could be Format.NO_VALUE
causing the FrameEditor to be used more often than
necessary in the old version.
PiperOrigin-RevId: 414304251
When calling Android's Log class directly, there's a LongLogTag
lint check that detects tags over the 23 char limit, however it
cannot detect long log tags in ExoPlayer due to the way that we
log via our own Log class. This commit adds @Size annotations to
enforce the same rule.
PiperOrigin-RevId: 413976364
On the Sony Android TV device where this was originally reproducible on Android L, on Android N there is an E-AC3 decoder listed which handles the stream correctly. The workaround is harmless anyway but adding the API version restriction means it will be obvious it can be removed once we bump our min API to 24 or above in the future.
#minor-release
PiperOrigin-RevId: 413967443
Using chunkless preparation greatly improves start up time if the master
playlist declares CODECS for the renditions. Hence, we turn this on
by default as it benefits most well-defined HLS master playlists.
The only known reason why developers may want to turn this feature off is
when the renditions contain muxed closed-caption tracks that are not
declared in the master playlist. So this change also updates the documentation
and RELEASENOTES to point out this caveat.
PiperOrigin-RevId: 413950036
