We already have logic to end all session except the current one if the
current one doesn't have a MediaPeriodId yet. This is assuming that this
only happens after a seek on the app side where the player doesn't have
detailled knowledge about the MediaPeriodIds yet.
Currently this logic isn't triggered if the window we are coming from
doesn't have its MediaPeriodId either as we run into another check that
keeps sessions around until we have a valid windowSequenceNumber.
Swapping both conditions fixes this case without breaking any of the
other known transition scenarios.
Issue: androidx/media#180
PiperOrigin-RevId: 480866465
When debugging and fixing Issue: google/ExoPlayer#10666 I wanted to write a regression
test, but needed to add a test first... This is just a small bit of
coverage to start with. It checks the field/channel filtering works
correctly, but doesn't check any styling info. It also doesn't test
'pop on' subtitles (i.e. when the subtitle isn't shown until a 'end of
subtitle' signal is received).
PiperOrigin-RevId: 480644568
We currently use the literal -1 (=NO_VALUE) when adding up the
total. Tracks without known bitrate can be ignored in the
calculation, but we should use an explicit value of 0.
#minor-release
Issue: google/ExoPlayer#10664
PiperOrigin-RevId: 480048126
If the sample type is Dolby Vision and the display does not support Dolby Vision, then the capabilities DecoderSupport flag is set to DECODER_SUPPORT_FALLBACK_MIMETYPE. This denotes that the renderer will use a decoder for a fallback mimetype if possible. This alters track selection as tracks with DecoderSupport DECODER_SUPPORT_PRIMARY are preferred.
UnitTests included
-DefaultTrackSelector test that checks track selection reordering with DECODER_SUPPORT_FALLBACK_MIMETYPE
-MediaCodecVideoRenderer test that checks setting of DecoderSupport flag based on Display's Dolby Vision support
Issue: google/ExoPlayer#8944
PiperOrigin-RevId: 480040876
Currently, a frame is dropped if it's requested release time is in the past.
This mode was added to support previewing. However, in normal ExoPlayer
playback, slightly late frames (<30ms late) are also rendered. On MediaCodec
side, this means calling `releaseOutputBuffer` with a release time in the
past.
PiperOrigin-RevId: 479615291
This mode is supported by using `C.TIME_UNSET` (which is a negative value). The
new logic decouples the value of `C.TIME_UNSET` and the frame dropping
behaviour.
PiperOrigin-RevId: 479368880
* Add `setOutputStreamOffsetUs(long)` method in `AudioSink`.
* Add private methods `setOutputStreamOffsetUs(long)` method in `MediaCodecRenderer` and `DecoderAudioRenderer`.
* Add protected method `onOutputStreamOffsetUs(long)` method in `MediaCodecRenderer`, in which:
* `MediaCodecRenderer` itself will be no-op for this method.
* `MediaCodecAudioRenderer` will propagate this value to its `audioSink`.
* Add logics in `DecoderAudioRenderer` to calculate `outputStreamOffsetUs`.
PiperOrigin-RevId: 479265429
Currently `FrameProcessor.releaseOutputFrame()` method supports
Release at a specific system time
Drops the frame
This API is not that convenient to use when the caller wants to release a frame, now, regardless of the release time. A use case is to release (present) a frame when no frame is shown for a while, and it's thus better to just release the frame, now.
Currently if MCVR wants a frame to be rendered now, MCVR calls release frame with a set offset like 10us: `releaseOutputFrame(System.nanoTime() + 10_000)`. The 10us offset is to prevent the frame processor dropping the frame, due to thread hopping delays.
To make the API better usable, consider adding a mode for releasing the frame now, like (bold marks the new mode)
- Use C.TIME_UNSET to drop
- **Use -1 to release the frame immediately, or**
- Use an actual release time.
PiperOrigin-RevId: 479044215
A follow up to stopping speaker playback with a Player decorator from
https://github.com/androidx/media/issues/15.
It looks like we will need to change to using playback suppression to avoid
errors like https://github.com/androidx/media/issues/167, when we don't start
a foreground service.
We may not have this implemented by 1.0, but would like it in the API and it seems to be appropriate.
PiperOrigin-RevId: 478835686
Assert that tone mapping is applied when an HDR edit cannot be HDR, but is successfully tone mapped. Meanwhile, assert that fallback, which is applied after codec configuration (which throws the "Tone-mapping requested but not supported by the decoder" error) is not applied when that error is called.
PiperOrigin-RevId: 478762951
"Final" was likely added to reference the FinalMatrixTextureProcessorWrapper,
which is a package-private class. However, I think more clear to express that
this is the input size, which then has all effects applied, to get the output
size.
PiperOrigin-RevId: 477975358
Rename test files to avoid substrings that can be implied by the directory name,
like "Transformation" and "Test"
No functional changes. Renaming-only.
PiperOrigin-RevId: 477724724
This allows to access the associated functionality of AudioTrack and
fills a feature gap to MediaPlayer, which has a similar method.
Issue: androidx/media#135
PiperOrigin-RevId: 476398964
* Before this CL, the texture was stored during the construction of the LUT processor. This failed since if one creates a list of GlEffects on the application thread, the texture will get stored in the application thread during the effect creation and not on the GL thread, which executes the FrameProcessors.
* This is an issue since the executing thread then can't index from the texture stored on a different thread.
PiperOrigin-RevId: 476388021
If the sample type is dolby vision and the following conditions match
a)There is a supported alternative codec mimetype
b)Display does not support Dolby Vision
Then getDecoderInfos will return the alternative types.
Issue: google/ExoPlayer#9794
PiperOrigin-RevId: 476356223
* Transform the intermediate color space to linear SDR by applying the SMPTE 170M EOTF and OETF.
* Use linear colors for the color filter pixel tests and update all golden bitmaps.
PiperOrigin-RevId: 476124592
The ClearKey CDM will attach an 'invalid' URL in `KeyRequest` objects,
when the documentation states this should be an empty string if a
default URL is not known.
#minor-release
PiperOrigin-RevId: 476113513
*** Original commit ***
Handle int instead of byte in SSIM.
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.
This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.
Tested manually and SSIM values match for the same video
before and after this change.
***
PiperOrigin-RevId: 473259446
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.
This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.
Tested manually and SSIM values match for the same video
before and after this change.
PiperOrigin-RevId: 473231779
Test that HDR editing succeeds on devices supporting HDR editing, tone maps on
devices supporting tone mapping, and throws exceptions on all other devices.
Also, only restrict HDR editing and tone mapping support to API 31+ only when
transcoding, not for all transformations.
PiperOrigin-RevId: 472958965
If the back buffer is using too much memory, there is a risk
playback could get stuck because LoadControl refuses to load
further data. This eventually results in a stuck-buffering
playback error.
We can detect this case, clear the back buffer and then ask
the LoadControl again to avoid failing playback in such a case.
PiperOrigin-RevId: 472679797
The stream offset is used to calculate the presentation time of
a metadata object when reading and later when playing, to calculate
the current presentation time to decide whether to send the metadata
to the output.
Accordingly, the presentation time of a pending metadata that has been
calculated with a given offset needs to be recalculated when the
stream offset changes.
#minor-release
PiperOrigin-RevId: 472499943
* Non-standard parameter comment; prefer `/* paramName= */ arg`
(see http://go/bugpattern/ParameterComment) (3 times)
This CL looks good? Just LGTM and Approve it!
This CL doesn’t look good? This is what you can do:
* Revert this CL, by replying "REVERT: <provide reason>"
* File a bug under go/error-prone-bug for category ErrorProneStyle if there's an issue with the CL content.
* File a bug under go/rosie-bug if there's an issue with how the CL was managed.
* Revert this CL and not get a CL that cleans up these paths in the future by
replying "BLOCKLIST: <provide reason>". This is not reversible! We recommend to
opt out the respective paths in your CL Robot configuration instead:
go/clrobot-opt-out.
This CL was generated by CL Robot - a tool that cleans up code findings
(go/clrobot). The affected code paths have been enabled for CL Robot in //depot/google3/java/com/google/android/libraries/media/METADATA which is reachable following include_presubmits from //depot/google3/third_party/java_src/android_libs/media/METADATA.
Anything wrong with the signup? File a bug at go/clrobot-bug.
#codehealth
Tested:
Local presubmit tests passed.
PiperOrigin-RevId: 472254253
shouldPassthrough's internal checks seem to be check whether we should *not*
pass through, which seemed a bit like a confusing double-negative to me.
shouldTranscode is slightly more clear, by instead returning true when we do
want to transcode.
No functional changes intended.
PiperOrigin-RevId: 471753771
3b0d2c1586 made `shouldPassthrough` always return false for `enableHdrVideoEditing`:
>We force using `FrameEditor` (no passthrough) to avoid the need to select another edit operation, and use the new shaders. The `EGLContext` and `EGLSurface` also need to be set up differently for this path.
However, this was introduced before the `videoNeedsEncoding` setting was introduced in 3f615040c0. That setting should apply to HDR videos as much as SDR videos.
PiperOrigin-RevId: 471569853
* Non-standard parameter comment; prefer `/* paramName= */ arg`
(see http://go/bugpattern/ParameterComment) (3 times)
This CL looks good? Just LGTM and Approve it!
This CL doesn’t look good? This is what you can do:
* Revert this CL, by replying "REVERT: <provide reason>"
* File a bug under go/error-prone-bug for category ErrorProneStyle if there's an issue with the CL content.
* File a bug under go/rosie-bug if there's an issue with how the CL was managed.
* Revert this CL and not get a CL that cleans up these paths in the future by
replying "BLOCKLIST: <provide reason>". This is not reversible! We recommend to
opt out the respective paths in your CL Robot configuration instead:
go/clrobot-opt-out.
This CL was generated by CL Robot - a tool that cleans up code findings
(go/clrobot). The affected code paths have been enabled for CL Robot in //depot/google3/java/com/google/android/libraries/media/METADATA which is reachable following include_presubmits from //depot/google3/third_party/java_src/android_libs/media/METADATA.
Anything wrong with the signup? File a bug at go/clrobot-bug.
#codehealth
Tested:
Local presubmit tests passed.
PiperOrigin-RevId: 471198016
In particular, make it a bit more clear that "rendering" and "releasing" frames are
related concepts, and how they differ from one another in conjunction with frame
dropping.
PiperOrigin-RevId: 471037733
* Non-standard parameter comment; prefer `/* paramName= */ arg`
(see http://go/bugpattern/ParameterComment) (19 times)
This CL looks good? Just LGTM and Approve it!
This CL doesn’t look good? This is what you can do:
* Revert this CL, by replying "REVERT: <provide reason>"
* File a bug under go/error-prone-bug for category ErrorProneStyle if there's an issue with the CL content.
* File a bug under go/rosie-bug if there's an issue with how the CL was managed.
* Revert this CL and not get a CL that cleans up these paths in the future by
replying "BLOCKLIST: <provide reason>". This is not reversible! We recommend to
opt out the respective paths in your CL Robot configuration instead:
go/clrobot-opt-out.
This CL was generated by CL Robot - a tool that cleans up code findings
(go/clrobot). The affected code paths have been enabled for CL Robot in //depot/google3/java/com/google/android/libraries/media/METADATA which is reachable following include_presubmits from //depot/google3/third_party/java_src/android_libs/media/METADATA.
Anything wrong with the signup? File a bug at go/clrobot-bug.
#codehealth
Tested:
Local presubmit tests passed.
PiperOrigin-RevId: 471022923
Use the PQ OETF and EOTF to ensure that intermediate fragment shader operations
using PQ are in linear BT.2020 rather than PQ and HLG-1 BT.2020.
Also, swap the OETF and EOTF in shaders, as they were used incorrectly before
Manually tested by verifying transformer demo HLG and PQ videos look the same with and without this CL, including with a BitmapOverlayProcessor enabled to test flows both with one MatrixTransformationProcessor that skips HDR TFs, and with one that doesn't.
PiperOrigin-RevId: 469736067
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.
More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.
PiperOrigin-RevId: 468473072
We create an empty CueGroup in many places as default or
where none is needed. Instead, we can define a constant
for this purpose and reuse it.
PiperOrigin-RevId: 467944841
Increase the estimated max sample size for HEVC by 2x, and set a minimum
size of 2MB. The 2MB will be applied for resolutions up to 1080p, after
which the new calculation takes effect. This is in par with the
platform's HEVC software decoder.
PiperOrigin-RevId: 467641494
This base class will simplify the implementation of custom
Player classes. The current version only supports
available commands and playWhenReady handling.
PiperOrigin-RevId: 467618021
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.
PiperOrigin-RevId: 466641277
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).
PiperOrigin-RevId: 466425738
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.
Therefore, it's more specific to mention just that the transfer is HDR.
PiperOrigin-RevId: 466316960
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.
No functional change intended in this CL.
PiperOrigin-RevId: 466070764
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.
PiperOrigin-RevId: 465584814
`requestCalculateSsim` more clearly represents the intention of the caller.
Also rephrase the javadoc to simplify it and make it more precise.
PiperOrigin-RevId: 465575578
Adds a new event to AudioOffloadListener to get the offload state of the track, which indicates when software decoding is taking place.
PiperOrigin-RevId: 465264362
Avoids disabling Offload on a write error, and instead relies on this being disabled on the AudioTrack init. It will no longer recover by disabling offload.
PiperOrigin-RevId: 465248917
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.
When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.
Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.
PiperOrigin-RevId: 464135080
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.
PiperOrigin-RevId: 464060047
* Sets KEY_HDR_STATIC_INFO from MediaFormat in the DefaultCodec.
* Adds checks in mediaparser to ensure color space, range, and transfer are valid
values.
PiperOrigin-RevId: 463921325
Despite unregistering the callback and clearing pending Handler
messages, the callback may still receive pending calls if they
are already triggered by the AudioTrack. Instead of asserting
that the track is correct, we should gracefully ignore stale
events.
PiperOrigin-RevId: 463851393
TransformationException error codes were previously tied to OpenGL
but other FrameProcessor implementations are possible. So this CL
renames the error codes.
Also, remove GL_INIT_FAILED error code, as FrameProcessor
implemenations may initialize resources on a background thread
after the factory method returns, so it's not obvious how to
distinguish between initialization failures and processing failures.
PiperOrigin-RevId: 463704902
Just a misc nit, since I found the name a bit confusing, and figured
findEncoderWithClosestSupportedFormat might be more descriptive.
PiperOrigin-RevId: 463433646
Extract a FrameProcessor.Factory interface from GlEffectsFrameProcessor
and allow it to be customized using a setter on Transformer.Builder.
PiperOrigin-RevId: 463433438
Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.
PiperOrigin-RevId: 463206474
Add support for multi-value tags based on null terminators.
These are specific to ID3v2.4, but is backwards compatible with
ID3v2.3, so no version checks are needed.
This is needed for applying effects to a playlist.
The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.
Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.
PiperOrigin-RevId: 462714966
This allows us to use BT.2020 RGB linear for intermediate shaders, which also
allows us to re-enable PeriodicVignetteProcessor, which should work properly in
linear color-spaces.
Manually tested by adding a GlEffectsWrapper, and confirming that HLG HDR editing still looks correct.
PiperOrigin-RevId: 462265821
This adds two methods that are helpful when working with Futures.
One is a version of postOrRun that can indicate completion by a
Future and the other is a simplified version of Guava's
Futures.transformAsync (which can't be used as it's in Beta).
PiperOrigin-RevId: 461896598
Otherwise, invalid ColorInfo instances generated using faulty
MediaFormat#getInteger values could cause exceptions.
Confirmed that b/239435670 reproduces without this CL, and does not reproduce
with this CL.
PiperOrigin-RevId: 461862191
This will help developers self-diagnose issues like Issue: google/ExoPlayer#10392
where the NPE occurs far from the original null value because a field
gets assigned to null.
This change aims to ensure that every stable method on Player,
ExoPlayer and ExoPlayer.Builder that takes a non-null type will fail
with an NPE before returning.
#minor-release
PiperOrigin-RevId: 461846580
* Introduced `useHdr` for `GlEffect#toGlTextureProcessor`, so
`TextureProcessor` implementations can decide how to handle HDR.
* Creating FP16 color textures for HDR input.
Tested via manual testing, adding a no-op GlEffectWrapper to the transformation to
force use of intermediate textures, adding a linear ramp to the fragment shader,
and trying to ascertain that there's a real reduction in posterization when
switching from 4-bit to 8-bit unsigned bytes, and again from 8-bit unsigned bytes
to 16-bit floating point.
PiperOrigin-RevId: 461613117
This may happen when a containers' color transfer incorrectly does not match
the video's color transfer.
An example of a file with such a mismatch is the current Transformer demo HDR10
sample file.
Manually tested by confirming that no errors are emitted for SDR and HLG sample
files, and that errors are emitted for our incorrect HDR10 sample file.
PiperOrigin-RevId: 461583532
- Update profile selection logic to pick an HDR-compatible profile when doing HDR editing on H.264/AVC videos.
- Handle doing the capabilities check for all MIME types that support HDR (not just H.265/HEVC).
- Fix a bug where we would pass an HDR input color format to the encoder when using tone-mapping.
- Tweak how `EncoderWrapper` works so decisions at made at construction time.
Manually tested cases:
- Transformation of an SDR video.
- Transformation of an HDR video to AVC (which triggers fallback/tone-mapping on a device that doesn't support HDR editing for AVC).
- Transformation of an HDR video with HDR editing.
PiperOrigin-RevId: 461572973
Also remove VideoEncoderSettings.colorProfile as there are no
concrete use cases for customizing this and it clashes with picking
the color format automatically based on SDR vs. HDR.
PiperOrigin-RevId: 460746987
The call doesn't currently reset the already loaded suppliers and
factories. Also fix the supplier loading code to use a local copy
of the current dataSourceFactory to avoid leaking an updated
instance to a later invocation.
Issue: androidx/media#116
#minor-release
PiperOrigin-RevId: 460721541
We currently start a simple Thread to release AudioTracks
asynchronously. If many AudioTracks are released at the same
time, this may lead to OOM situations because we attempt to
create multiple new threads.
This can be improved by using a shared SingleThreadExecutor.
In the simple case of one simmultaneous release, it's exactly
the same behavior as before: create a thread and release it
as soon as it's done. For multiple simultanous releases we
get the advantage of sharing a single thread to avoid creating
more than one at the same time.
Issue: google/ExoPlayer#10057
PiperOrigin-RevId: 460698942
`HevcConfig.parse` misreads reserved bit to determine NAL unit type. This is currently meant to be always set to 0, but could be given some kind of meaning in a future revision.
Issue: google/ExoPlayer#10366
PiperOrigin-RevId: 460487613
Some calls to handleBuffer return false while a previous
flush is still handled in the background.
Fix this by either asserting the method returns true if
we don't expect any delay, or calling it repeatedly until
it returns true (within a timeout).
PiperOrigin-RevId: 460474419
The call doesn't currently reset the already loaded suppliers and
factories. Also fix the supplier loading code to use a local copy
of the current dataSourceFactory to avoid leaking an updated
instance to a later invocation.
Issue: androidx/media#116
#minor-release
PiperOrigin-RevId: 460721541
(cherry picked from commit 6be0d6ea25)
I don't think it's useful to keep these in numerical order, it makes
more sense to keep them grouped into a 'logical' ordering.
#minor-release
PiperOrigin-RevId: 460453464
This extension is needed for editing HDR input with OpenGL, as the
ExternalTextureProcessor samples raw YUV values from the
external texture for HDR and converts them to RGB itself rather than
relying on the OpenGL driver to do this automatically as for SDR.
PiperOrigin-RevId: 460424154
This saves an intermediate texture copy step for use-cases
where matrix transformations are the first or only effects
in the chain.
PiperOrigin-RevId: 460239403
- Added setter to disable this feature.
- Added accompanying tests.
- Plan to run tests on the same set of settings on H265.
PiperOrigin-RevId: 460238673
I don't think it's useful to keep these in numerical order, it makes
more sense to keep them grouped into a 'logical' ordering.
#minor-release
PiperOrigin-RevId: 460453464
(cherry picked from commit 1b1e0f890d)
We wait until a previous AudioTrack has been released before
creating a new one. This is currently done with a thread
block operation, which may cause ANRs in the extreme case
when someone attempts to release the player while this is
still blocked.
The problem can be avoided by just returning false from
DefaultAudioSink.handleBuffer to try again until the previous
AudioTrack is released.
Reproduction steps to force the issue:
1. Add Thread.sleep(10000); to the AudioTrack release thread.
2. Add this to the demo app:
private int positionMs = 0;
Handler handler = new Handler();
handler.post(new Runnable() {
@Override
public void run() {
player.seekTo(positionMs++);
if (positionMs == 10) {
player.release();
} else {
handler.postDelayed(this, 1000);
}
}
3. Observe Player release timeout exception.
These steps can't be easily captured in a unit test as we can't
artifically delay the AudioTrack release from the test.
Issue: google/ExoPlayer#10057
PiperOrigin-RevId: 459468912
The SDR constant also specified a color space and range, in addition to
C.COLOR_TRANSFER_SDR. However, it turns out that SDR videos may use different color
space and range values, so following prior ExoPlayer conventions to have `null`
mean "generic SDR" is preferable here.
PiperOrigin-RevId: 459296746
If the input is HDR (HLG), check encoder capabilities for HDR support
and request tone-mapping to SDR during decoder configuration otherwise.
Capabilities are only checked for API 31 and above, as HDR editing is
not supported before.
As the encoder capabilities check needs to happen before selecting the
encoder to use (as this may depend on the resolution output by the
effects chain), the EncoderWrapper checks all candidate encoders
for the MIME type for HDR capabilities and only requests fallback to
SDR if none of them support it.
When the actual encoder is selected, the wrapper checks that it matches
one of the encoders is checked capabilities for.
PiperOrigin-RevId: 458511599
Configure the GL shaders and encoder to take in HDR metadata.
This mostly just consists of passing the Format.colorInfo through
the VideoTranscodingSamplePipeline down to the encoder, rather than passing
the PQ-ness down to the GL step.
Due to b/237674316, this will remove HDR10+ support temporarily to introduce
support for HLG10.
Manually tested to confirm that HLG10 operations that don't affect color display
correctly after this CL with "HDR editing" in the demo checked, and continue to display incorrectly (as before this CL) without the option unchecked.
PiperOrigin-RevId: 458490810
ProgressiveMediaPeriod loads all available tracks into SampleStreams
(because it needs to read the data anyway and it allows easy activation
of tracks without reloading). However, the SampleStreams for disabled
tracks are not read and no one if waiting for them.
The buffered position is used for user-visible state (e.g. in the UI)
and to check how much data is already buffered to decide when to stop
buffering (using LoadControl). Both values benefit from only
using the actually enabled tracks to better reflect what is available
for playback at the moment.
Issue:Issue: google/ExoPlayer#10361
PiperOrigin-RevId: 458475038
We used "ALL_COOECS" previously, and it is not necessary because "ALL_CODECS"
additionally the codecs that support tunneling/secure decoding, which there
is no use case in Transformer.
PiperOrigin-RevId: 458470278
Although MediaCodec claims supporting float frame rate, encoder init failed on
API21 Nexus 5. Since it's just a performance hint to the codec, it's OK to
generalize it to other API versions.
PiperOrigin-RevId: 458434650
As per MP4 spec, bitrates in esds boxes can be a 32 bit number which doesn't fits in Java int type, so now reading it as a long value. Our class for holding media format, only allows bitrates value to be an int as we don't expect the bitrates to be greater than or equal to 2^31. So we're limiting the values for bitrates to Integer.MAX_VALUE.
#minor-release
PiperOrigin-RevId: 458423162
The GlEffectsFrameProcessor that will be part of the effects module
uses the DebugViewProvider. So it does not make sense for it
to be an inner interface of Transformer.
PiperOrigin-RevId: 458014932
The FinalMatrixTransformationProcessorWrapper ensures that the
surface is only replaced when it is not being rendered to and vice
versa.
PiperOrigin-RevId: 458007639
The outputHeight in the TransformationRequest is the height of
the frame as it would be displayed (i.e., after applying any
rotation specified in the format). So pass-through should only
be used if the requested outputHeight matches the input
format's height after applying the rotation.
PiperOrigin-RevId: 457934867
Previously two timelines that differed only in shuffle order were
considered equal, which resulted in no call to
Player.Listener.onTimelineChanged when calling
ExoPlayer.setShuffleOrder. This in turn resulted in no call to
MediaControllerCompat.Callback.onQueueChanged.
Also make a small fix inside ExoPlayerImpl.setShuffleOrder, to ensure
that the new shuffle order is used when constructing the masked
timeline.
Issue: google/ExoPlayer#9889
#minor-release
PiperOrigin-RevId: 457703727
NoUidTimeline still exists as a private detail of TestUtil, but it no
longer extends ForwardingTimeline because the interactions are quite
hard to reason about.
#minor-release
PiperOrigin-RevId: 457703593
We wait until a previous AudioTrack has been released before
creating a new one. This is currently done with a thread
block operation, which may cause ANRs in the extreme case
when someone attempts to release the player while this is
still blocked.
The problem can be avoided by just returning false from
DefaultAudioSink.handleBuffer to try again until the previous
AudioTrack is released.
Reproduction steps to force the issue:
1. Add Thread.sleep(10000); to the AudioTrack release thread.
2. Add this to the demo app:
private int positionMs = 0;
Handler handler = new Handler();
handler.post(new Runnable() {
@Override
public void run() {
player.seekTo(positionMs++);
if (positionMs == 10) {
player.release();
} else {
handler.postDelayed(this, 1000);
}
}
3. Observe Player release timeout exception.
These steps can't be easily captured in a unit test as we can't
artifically delay the AudioTrack release from the test.
Issue: google/ExoPlayer#10057
PiperOrigin-RevId: 459468912
(cherry picked from commit a80dd60449)
ProgressiveMediaPeriod loads all available tracks into SampleStreams
(because it needs to read the data anyway and it allows easy activation
of tracks without reloading). However, the SampleStreams for disabled
tracks are not read and no one if waiting for them.
The buffered position is used for user-visible state (e.g. in the UI)
and to check how much data is already buffered to decide when to stop
buffering (using LoadControl). Both values benefit from only
using the actually enabled tracks to better reflect what is available
for playback at the moment.
Issue:Issue: google/ExoPlayer#10361
PiperOrigin-RevId: 458475038
(cherry picked from commit 577e19168d)
As per MP4 spec, bitrates in esds boxes can be a 32 bit number which doesn't fits in Java int type, so now reading it as a long value. Our class for holding media format, only allows bitrates value to be an int as we don't expect the bitrates to be greater than or equal to 2^31. So we're limiting the values for bitrates to Integer.MAX_VALUE.
#minor-release
PiperOrigin-RevId: 458423162
(cherry picked from commit 9e10286b5e)
Previously two timelines that differed only in shuffle order were
considered equal, which resulted in no call to
Player.Listener.onTimelineChanged when calling
ExoPlayer.setShuffleOrder. This in turn resulted in no call to
MediaControllerCompat.Callback.onQueueChanged.
Also make a small fix inside ExoPlayerImpl.setShuffleOrder, to ensure
that the new shuffle order is used when constructing the masked
timeline.
Issue: google/ExoPlayer#9889
#minor-release
PiperOrigin-RevId: 457703727
(cherry picked from commit 5c7ec13e85)
NoUidTimeline still exists as a private detail of TestUtil, but it no
longer extends ForwardingTimeline because the interactions are quite
hard to reason about.
#minor-release
PiperOrigin-RevId: 457703593
(cherry picked from commit 292f6de630)
1. The offloadSchedulingEnabled value doesn't need to be in
PlaybackInfo because it's never updated in EPII.
2. The sleepingForOffload value in EPII wasn't updated explicitly
(just via the return value of a method). It was also only
meant to be enabled while the player is actively playing, but
confusingly triggered from a path where the player may
theoretically be buffering as well.
3. The offload sleeping (=not scheduling doSomeWork) was interwoven
into the actual scheduling code making it slightly hard to follow.
This can be improved slightly by keeping the offload sleeping
decision and the scheduling separate.
PiperOrigin-RevId: 457427293
This will be useful for downgrading to a lower resolution during
a slow preview and for processing slide-shows once sequential
multi-asset editing is supported.
PiperOrigin-RevId: 457017255
videoEncoderFormatUnsupported_completesWithError() has recently
been flaky on API 31 emulators on presubmit because a different
exception than the expected exception is thrown.
This disables it on those emulators to reduce testing noise
until the underlying problem is investigated and resolved.
PiperOrigin-RevId: 456765512
This change is just renaming. There is no functional change intended.
The FrameProcessor interface will be created in a follow-up.
PiperOrigin-RevId: 456741628
After this change GlEffects can use any GlTextureProcessor not just
SingleFrameGlTextureProcessor.
MediaPipeProcessor now implements GlTextureProcessor directly which
allows it to reuse MediaPipe's output texture for its output texture
and avoids an extra copy shader step.
PiperOrigin-RevId: 456530718
After this change, FrameProcessorChain chains any GlTextureProcessors
instead of only SingleFrameGlTextureProcessors.
The GlTextureProcessors are chained in a bidirectional manner using
ChainingGlTextureProcessorListener to feed input and output related
events forward and release events backwards.
PiperOrigin-RevId: 456478414
The offload sleeping stops as soon as a new DO_SOME_WORK message
is handled (because this indicates an expected change in rendering
and we want to stop sleeping until we know it's safe to do so).
Every exit path from doSomeWork needs to clear other pending
DO_SOME_WORK messages as these requests have already been handled by
the current method invocation. This currently doesn't happen from the
offload sleeping return path and a previously queued DO_SOME_WORK
message can immediately wake up the rendering loop again.
Fix this by moving the message removal to the beginning of the
doSomeWork method (as it prevents forgetting it in one of the
exit paths later).
PiperOrigin-RevId: 456259715
In follow-ups the FrameProcessorChain will set an instance of this
listener for each GlTextureProcessor to chain it with its previous
and next GlTextureProcesssor.
PiperOrigin-RevId: 455628942
1. The offloadSchedulingEnabled value doesn't need to be in
PlaybackInfo because it's never updated in EPII.
2. The sleepingForOffload value in EPII wasn't updated explicitly
(just via the return value of a method). It was also only
meant to be enabled while the player is actively playing, but
confusingly triggered from a path where the player may
theoretically be buffering as well.
3. The offload sleeping (=not scheduling doSomeWork) was interwoven
into the actual scheduling code making it slightly hard to follow.
This can be improved slightly by keeping the offload sleeping
decision and the scheduling separate.
PiperOrigin-RevId: 457427293
(cherry picked from commit aedde2de39)
The offload sleeping stops as soon as a new DO_SOME_WORK message
is handled (because this indicates an expected change in rendering
and we want to stop sleeping until we know it's safe to do so).
Every exit path from doSomeWork needs to clear other pending
DO_SOME_WORK messages as these requests have already been handled by
the current method invocation. This currently doesn't happen from the
offload sleeping return path and a previously queued DO_SOME_WORK
message can immediately wake up the rendering loop again.
Fix this by moving the message removal to the beginning of the
doSomeWork method (as it prevents forgetting it in one of the
exit paths later).
PiperOrigin-RevId: 456259715
(cherry picked from commit 251389d744)
Fixing lint errors in the string.xml files makes no sense because these are
overridden with the next automated string import. Adding a lint-baseline.xml
instead for the ui module.
See https://issuetracker.google.com/208178382
#minor-release
PiperOrigin-RevId: 455354304
The wrapper
* catches exceptions for each task and notifies the
listener (this will be used more in follow-ups when processFrame
is split into lots of listeners and callbacks),
* removes finished tasks from the queue and signals any exceptions
that occurred to the listener each time a new task is executed.
PiperOrigin-RevId: 455345184
This change adds a SurfaceProvider interface which is necessary to
allow for texture processors whose output size becomes available
asynchronously in follow-ups.
VTSP's implementation of this interface wraps the encoder and provides
its input surface together with the output frame width, height, and
orientation as used for encoder configuration.
The FrameProcessorChain converts the output frames to the provided
orientation and resolution using a ScaleToFitTransformation and
Presentation replacing EncoderCompatibilityTransformation.
PiperOrigin-RevId: 455112598
The default constructor is only allowed to be called on
API < 32 and the test should use the defined UNSET constant
to be API independent.
#minor-release
PiperOrigin-RevId: 454568893
The native GL errors are in an arbitrary order according to
https://www.khronos.org/registry/OpenGL-Refpages/es2.0/xhtml/glGetError.xml
This means any of them could contain the most useful message
so it's better to use all for the GlException's message rather than
picking the last message and logging all others.
PiperOrigin-RevId: 454130460
`codecDrainAction` is set to `DRAIN_ACTION_NONE` in 3 places in
`MediaCodecRenderer`:
* The constructor (so there's no prior state to worry about)
* `updateDrmSessionV23()`: Where `mediaCrypto` is reconfigured based
on `sourceDrmSession` and `codecDrmSession` is also updated to
`sourceDrmSession`.
* `resetCodecStateForFlush()`: Where (before this change) the action
is unconditionally set back to `DRAIN_ACTION_NONE` and so any
required updated implied by
`DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION` is not done.
This change ensures that `flushOrReleaseCodec()` handles
`DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION` before calling .
This probably also resolves Issue: google/ExoPlayer#10274
#minor-release
PiperOrigin-RevId: 454114428
Based on
https://developer.android.com/reference/android/media/MediaCodec#using-an-output-surface,
frame dropping behaviour depends on the target SDK version.
After this change transformer will only use
MediaFormat#KEY_ALLOW_FRAME_DROP if both the target and system SDK
version are at least 29 and default to its pre 29 behaviour where each
decoder output frame must be processed before a new one is rendered
to prevent frame dropping otherwise.
Also remove deprecated Transformer.Builder constructor without a
context and the context setter.
PiperOrigin-RevId: 453971097
Transformer always enabled glAssertionsEnabled, so there should
be no functional change.
ExoPlayer previously disabled glAssertionsEnabled, so GlUtil logged
GlExceptions instead of throwing them. The GlExceptions are now
caught and logged by the callers so that there should also be no
functional change overall.
This change also replaces EGLSurfaceTexture#GlException with
GlUtil#GlException.
PiperOrigin-RevId: 453963741
The track selector will select multi-channel formats when those can be
spatialized, otherwise the selector will prefer stereo/mono audio
tracks. When the device supports audio spatialization (Android 12L+),
the DefaultTrackSelector will monitor for changes in the platform
Spatializer and trigger a new track selection upon a
Spatializer change event.
Devices with a `television` UI mode are excluded from audio channel
count constraints.
#minor-release
PiperOrigin-RevId: 453957269
This change fixes a bug where the player is frozen with HLS chunkless
preparation because the audio stream wrappers are not marked as master
timestamp sources before preparation.
#minor-release
PiperOrigin-RevId: 453941815
Some Player implementations have no playlist capability but can still
set a MediaItem for playback. Examples are a MediaController connected
to a legacy MediaSession, ExoPlayer up to 2.12 or MediaPlayer.
To indicate this capability, we need an allowed command in addition
to COMMAND_CHANGE_MEDIA_ITEMS that just allows to set a single item
that replaces everything that is currently played.
#minor-release
PiperOrigin-RevId: 453879626
This removes the prior restriction of needing to remember not to crop and set aspect ratio in the same Presentation.Builder, and makes each class a bit more targeted.
This is partially made feasible by the past work to merge consecutive
MatrixTransformations into a single MatrixTransformationFrameProcessor, which
ensures that there's no loss in quality between successive MatrixTransformations.
PiperOrigin-RevId: 453660582
With HLS chunkless preparation, audio formats may have no value
for channel count. In this case, the DefaultAudioSink will either query
the platform for a supported channel count (API 29+) or assume a max
channel count based on the encoding spec in order to decide whether the
audio format can be played with audio passthrough.
Issue: google/ExoPlayer#10204
#minor-release
PiperOrigin-RevId: 453644548
SingleFrameGlTextureProcessor is now an abstract class containing a
default implementation of the more flexible GlTextureProcessor interface
while still exposing the same simple abstract methods for single frame
processing it previously did.
FrameProcessorChain and GlEffect will be changed to use
GlTextureProcessor in follow-ups.
PiperOrigin-RevId: 453633000
The API 32 SDK has incorrect versioning metadata for Spatializer. It
reports the whole class has only been present since API 33 (which is
surely impossible given it's present in the API 32 SDK):
https://issuetracker.google.com/234009300
The metadata seems to be correct in the API 33 SDK, so this baseline
will no longer be needed when we bump to `compileSdkVersion = 33`.
Fixing lint errors in the string.xml files makes no sense because these are
overridden with the next automated string import. Adding a lint-baseline.xml
instead for the ui module.
See https://issuetracker.google.com/208178382
#minor-release
PiperOrigin-RevId: 455354304
(cherry picked from commit 495480a66a)
The default constructor is only allowed to be called on
API < 32 and the test should use the defined UNSET constant
to be API independent.
#minor-release
PiperOrigin-RevId: 454568893
(cherry picked from commit 0ed53215c2)
`codecDrainAction` is set to `DRAIN_ACTION_NONE` in 3 places in
`MediaCodecRenderer`:
* The constructor (so there's no prior state to worry about)
* `updateDrmSessionV23()`: Where `mediaCrypto` is reconfigured based
on `sourceDrmSession` and `codecDrmSession` is also updated to
`sourceDrmSession`.
* `resetCodecStateForFlush()`: Where (before this change) the action
is unconditionally set back to `DRAIN_ACTION_NONE` and so any
required updated implied by
`DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION` is not done.
This change ensures that `flushOrReleaseCodec()` handles
`DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION` before calling .
This probably also resolves Issue: google/ExoPlayer#10274
#minor-release
PiperOrigin-RevId: 454114428
(cherry picked from commit c736a72c75)
The track selector will select multi-channel formats when those can be
spatialized, otherwise the selector will prefer stereo/mono audio
tracks. When the device supports audio spatialization (Android 12L+),
the DefaultTrackSelector will monitor for changes in the platform
Spatializer and trigger a new track selection upon a
Spatializer change event.
Devices with a `television` UI mode are excluded from audio channel
count constraints.
#minor-release
PiperOrigin-RevId: 453957269
(cherry picked from commit 9f3c595e02)
This change fixes a bug where the player is frozen with HLS chunkless
preparation because the audio stream wrappers are not marked as master
timestamp sources before preparation.
#minor-release
PiperOrigin-RevId: 453941815
(cherry picked from commit 917484100d)
Some Player implementations have no playlist capability but can still
set a MediaItem for playback. Examples are a MediaController connected
to a legacy MediaSession, ExoPlayer up to 2.12 or MediaPlayer.
To indicate this capability, we need an allowed command in addition
to COMMAND_CHANGE_MEDIA_ITEMS that just allows to set a single item
that replaces everything that is currently played.
#minor-release
PiperOrigin-RevId: 453879626
(cherry picked from commit 2056152824)
With HLS chunkless preparation, audio formats may have no value
for channel count. In this case, the DefaultAudioSink will either query
the platform for a supported channel count (API 29+) or assume a max
channel count based on the encoding spec in order to decide whether the
audio format can be played with audio passthrough.
Issue: google/ExoPlayer#10204
#minor-release
PiperOrigin-RevId: 453644548
(cherry picked from commit b3b57bc93d)
The current setup with distinct, private `keyForField` implementations,
leaves open the (theoretical) possibility of a clash in the `Bundle`
keys used by the superclass and subclass. This change brings
consistency with our only other extensible `Bundleable` type
(`PlaybackException`).
#minor-release
PiperOrigin-RevId: 453385875
(cherry picked from commit a056f08a3d)
We generally nest the `Builder` for `Foo` inside `Foo`. In this case,
there's already a `DefaultTrackSelector.Parameters.Builder` type visible
to a developer, it just happens to be the 'common'
`TrackSelectorParameters.Builder`, so using it is a bit weird. For
example this code snippet doesn't compile because
`DefaultTrackSelector.Parameters.Builder#build()` returns
`TrackSelectionParameters`. This CL fixes that problem and the code
snippet now compiles.
```java
DefaultTrackSelector.Parameters params =
new DefaultTrackSelector.Parameters.Builder(context).build()
```
#minor-release
PiperOrigin-RevId: 453215702
(cherry picked from commit 881622385d)
This is done by removing the `@FieldNumber` IntDef completely. It's not
really adding much value anyway, because it's `open` so there's no real
enforcement to prevent passing 'incorrect' values.
#minor-release
PiperOrigin-RevId: 452108972
(cherry picked from commit 97210b5aa0)
The current setup with distinct, private `keyForField` implementations,
leaves open the (theoretical) possibility of a clash in the `Bundle`
keys used by the superclass and subclass. This change brings
consistency with our only other extensible `Bundleable` type
(`PlaybackException`).
#minor-release
PiperOrigin-RevId: 453385875
We generally nest the `Builder` for `Foo` inside `Foo`. In this case,
there's already a `DefaultTrackSelector.Parameters.Builder` type visible
to a developer, it just happens to be the 'common'
`TrackSelectorParameters.Builder`, so using it is a bit weird. For
example this code snippet doesn't compile because
`DefaultTrackSelector.Parameters.Builder#build()` returns
`TrackSelectionParameters`. This CL fixes that problem and the code
snippet now compiles.
```java
DefaultTrackSelector.Parameters params =
new DefaultTrackSelector.Parameters.Builder(context).build()
```
#minor-release
PiperOrigin-RevId: 453215702
Implementations of this interface will be able to drop or add frames,
change timestamps, accept multiple input frames before producing
output, and process frames on their own background thread.
A default implementation of this interface will be added to SingleFrameGlTextureProcessor in a follow-up.
PiperOrigin-RevId: 453159835
This is done by removing the `@FieldNumber` IntDef completely. It's not
really adding much value anyway, because it's `open` so there's no real
enforcement to prevent passing 'incorrect' values.
#minor-release
PiperOrigin-RevId: 452108972
This internal listener avoids wrapping the TransformationExceptions
in PlaybackExceptions that are handled via the Player.Listener and
is also used for FrameProcessingExceptions which already avoided
the PlaybackException layer previously.
This listener will also be useful in follow-ups for encoder-related
TransformationExceptions that are thrown in the SurfaceProvider that
will be called on the GL thread.
PiperOrigin-RevId: 452074575
This reinstates the permissive behaviour removed by
fe7e5b8181
Test file created by opening bear.opus in a hex editor and naively
duplicating the two header packets, starting at (and including) the
first `OggS` in the file and ending just before the third `OggS`.
#minor-release
Issue: google/ExoPlayer#10038
PiperOrigin-RevId: 452015662
(cherry picked from commit b6b282672c)
This reinstates the permissive behaviour removed by
fe7e5b8181
Test file created by opening bear.opus in a hex editor and naively
duplicating the two header packets, starting at (and including) the
first `OggS` in the file and ending just before the third `OggS`.
#minor-release
Issue: google/ExoPlayer#10038
PiperOrigin-RevId: 452015662
Once the more advanced GlTextureProcessor interface exists,
it will be possible to change the output size of a GlTextureProcessor
between frames. To keep the re-configuration based on the frame sizes
minimal, things indepedent of the frame size, such as the GlProgram,
can be initialized in the constructor.
PiperOrigin-RevId: 451997584
This value only existed to allow setting media URLs from external sources
(e.g. in a MediaController) so that a player can start playing this item.
Now that we have MediaItem.RequestMetadata.mediaUrl we can remove this value
from MediaMetadata because it's request metadata, not media metadata.
PiperOrigin-RevId: 451857413
These fields can be used to transport additional request properties
when the requester doesn't know the details needed for the actual
playback (i.e. the LocalConfiguration).
PiperOrigin-RevId: 451857093
ExoPlayer applies a large time offset to buffers so that, if the input has negative timestamps, generally buffers seen by the decoders should have positive timestamps. Modify how the offset is handled in `Transformer` so that decoders and encoders generally see positive timestamps, by leaving the offset on samples when reading them in the base renderer (remove the code that removed the offset), and then removing the offset when muxing. Also update the frame processor chain and slow motion flattening code to retain the existing behavior after this change (these both need original media presentation timestamps)
Tested via existing end-to-end tests and manually verified that the overlay frame processor shows the expected original media timestamps.
Aside: we don't need the same logic as ExoPlayer to track stream offsets across the decoder yet, because we don't try to handle stream changes during playback in single asset editing. (There is an edge case of multi-period DASH that may not work but I doubt anyone will use that as input to `Transformer` before we change the code to handle multi-asset properly.) In future we should try to refactor interaction with the decoder to use the same code for Transformer and ExoPlayer.
PiperOrigin-RevId: 451846055
Most devices won't support 8k decoding, so they'll skip this test entirely.
As the video is quite short, this test shouldn't be any longer than the nearby,
long-running 4k60 test.
PiperOrigin-RevId: 451423368
Unconditionally sleep for offload, if the audio buffer is full.
Previously ExoPlayer would not sleep if the expected wake-up was
in 2s. This was to prevent underrun if the wake-up was delayed.
Experiments have shown that the wakup audio buffer is far more
than 2s (around 1min). Additionally,
the metric was incorrect because it measured both,
AudioTrack + DSP.
Finally, this metric was erroneous after a gapless transition,
when the head position would reset to 0 and thus the computed
delay until next wakeup was too large.
PiperOrigin-RevId: 451383701
SSIM calculation requires the input and output dimensions to be identical.
For devices that can't encode the input dimensions, skip SSIM calculations and
log the cause. Only apply this on tests where the encoder may not support the
input file dimensions.
PiperOrigin-RevId: 451364904
Decode-only video frames (needed when the frame at / first frame after the
clipping start is not a key frame) need to be decoded but not passed to
the frame processor chain or encoder.
The clipping start offset needs to be removed from the frame timestamps
in the passthrough and video pipelines.
There are no changes needed for this in the audio pipeline, as it doesn't
use the input timestamps -- it uses its own timestamps derived from the
buffer sizes instead.
Also add demo option to try this out.
#minor-release
PiperOrigin-RevId: 451353609
Also update names of implementations to match design doc.
In follow-ups, SingleFrameGlTextureProcessor will become
an abstract implementation of a new GlTextureProcessor
interface.
Texture processor makes sense as it processes OpenGL textures.
The term frame processor will be used for something else in
follow-ups.
PiperOrigin-RevId: 451142085
* Rename (via deprecation)
`MediaItem.DrmConfiguration.Builder#forceSessionsForAudioAndVideoTracks`
to `setForceSessionsForAudioAndVideoTracks`. This is more consistent
with existing 'force' method names both in this class and in
`TrackSelectionParameters.Builder`.
* Add missing `@Nullable` annotation to the parameter for
`MediaItem.SubtitleConfiguration.Builder#setMimeType`. This annotation
is already present on the `MediaItem.SubtitleConfiguration#mimeType`
field that this setter corresponds to.
PiperOrigin-RevId: 450941336
There is a problem with the ImageReader formats used by the
SSIM helper that only occurs for Nexus 5 API 21, so as a workaround
we can skip the SSIM calculation on Nexus 5 API 21.
This skips just the SSIM calculation (by setting the value to
1.0 instead and logging). The tests still run when SSIM is skipped
so that we can detect other failures.
PiperOrigin-RevId: 450903183
This value is only needed by subclasses of `Rating`, all of which are
in this package (the `Rating` constructor is already package-private to
ensure this).
PiperOrigin-RevId: 450886872
*** Original commit ***
Rollback of 57182ac7bd
*** Original commit ***
Remove `@Nullable` from `MediaSource.Factory` setters
The null-behaviour of these methods creates a minimization footgun,
because **any** call to these setters will prevent R8 from removing
the default implementation (even if it's never used by the app) - this
is because R8 can't tell the default imple...
***
PiperOrigin-RevId: 450453325
AudioTrack.setOffloadEndOfStream should be called after a track
has been buffered. Additionally, the AudioTrack must be playing.
It has been observed that for very short media (<1s), the AudioTrack
might not have started immediately after the read that buffered
the audio.
In such a situation, calling AudioTrack.setOffloadEndOfStream throws
and playback fails.
Avoid this failure by checking that the AudioTrack is playing before
calling setOffloadEndOfStream.
This means that very short gapless media will not be gapless, this was
deemed acceptable given that such very short media should be very rare
in offload.
PiperOrigin-RevId: 450431146
*** Original commit ***
Remove `@Nullable` from `MediaSource.Factory` setters
The null-behaviour of these methods creates a minimization footgun,
because **any** call to these setters will prevent R8 from removing
the default implementation (even if it's never used by the app) - this
is because R8 can't tell the default implementation is only used if the
parameter is `null`.
******
PiperOrigin-RevId: 450410833
The null-behaviour of these methods creates a minimization footgun,
because **any** call to these setters will prevent R8 from removing
the default implementation (even if it's never used by the app) - this
is because R8 can't tell the default implementation is only used if the
parameter is `null`.
Follow-up to 57182ac7bd
PiperOrigin-RevId: 450395941
The null-behaviour of these methods creates a minimization footgun,
because **any** call to these setters will prevent R8 from removing
the default implementation (even if it's never used by the app) - this
is because R8 can't tell the default implementation is only used if the
parameter is `null`.
PiperOrigin-RevId: 450386627
This detection relies on an unsupported workaround and may trigger
permission warnings in tools analyzing permission usage although
no permission is needed or requested by app code.
Given the majority of 5G-NSA playbacks are on API 31+ by now,
we can remove this path to avoid the permission confusion and the
unsupported detection workaround.
PiperOrigin-RevId: 450382586
The debug surface view's output surface can become invalid during a transformation due to the parent activity pausing, for example. This can currently cause a crash when backing out of the `TransformerActivity` in the demo because the surface can be destroyed before the transformer has fully canceled.
Also clarify naming of the outputSurface and inline the private method that created `EGLSurface`s (it was shorter after removing the debug preview).
PiperOrigin-RevId: 449963440
This causes a bug where the forwarded selections are no longer
assumed equal and the child MediaPeriods will think they need
to reset streams even though the selection stayed the same.
Issue: Issue: google/ExoPlayer#10248
PiperOrigin-RevId: 449454038
We need to pass timestamp for the list of cues so we are defining a new class CueGroup which will store both cues and timestamp.
PiperOrigin-RevId: 449212054
The old reference was just for prototyping HDR. The new reference is for planned work to use the correct formats for input and output for HDR editing in GL.
PiperOrigin-RevId: 449211792
The MMWAVE constant was deprecated in favour of a new constant
with a better name. Thus, we need to check for both constants now.
PiperOrigin-RevId: 449018959
This is an individual language (ISO 639-3) part of the Arabic
macrolanguage ("ar" in ISO 639-1). Add this mapping to our
existing list similar to other individual to macrolanguage
mappings we have already.
Issue: Issue: google/ExoPlayer#10255
PiperOrigin-RevId: 448911950
When using a MatrixTransformationFrameProcessor per transformation
matrix, each frame processor's shader applies the matrix to the
vertices and clips the result to the NDC range when drawing the
output frame.
This change combines consecutive MatrixTransformations into a single
MatrixTransformationFrameProcessor by multiplying the individual
matrices while updating and clipping the visible polygon after
each matrix and mapping the resulting visible polygon back to the
input space so that its vertices and the combined transformation
matrix can be used in the shader.
PiperOrigin-RevId: 448521068
Network type detection on these API levels couldn't be tested
yet because of a missing Robolectric feature. This was fixed by
the recent Robolectric upgrade and the restrictions can be removed.
This also requires to replicate the platform hack we rely on on
these API levels.
PiperOrigin-RevId: 448240431
Some RTP foramts are statically assigned, so they don't have the rtpmap
attribute. Create the missing rtpmap attribute in this case.
PiperOrigin-RevId: 448239724
This listener replaces
FrameProcessorChain#getAndRethrowBackgroundExceptions.
The listener uses a new exception type FrameProcessingException
separate from TransformationException as the frame processing
components will be made reusable outside of transformer soon.
PiperOrigin-RevId: 447455746
This constant is used for https://docs.gl/es2/glVertexAttribPointer
which takes the number of components per generic vertex attribute
(meaning the size of the individual coordinate vectors here) not the
number of attributes (the number of vertices that the old constant
name referred to).
PiperOrigin-RevId: 447427241
Before this change, we list the formats for which we don't
want transcoding. This change disables transcoding altogether.
This was tested by checking that transcoding takes place on a
short camera recording only when the added flag is not present
(and AndroidManifest does not declare support for HEVC).
PiperOrigin-RevId: 446986580
We won't try to provide/rethrow helpful error messages that are already
provided by GL, as this sort of task would expand into writing a GL verifier.
A Gl verifier is unnecessarily complex for minimal payoff, especially as Apps
expected to read GL error messages would mostly be those writing custom
GlFrameProcessors, who should be already be familiar with reading GL error
messages anyways.
PiperOrigin-RevId: 446950837
ScaleToFitFrameProcessor, PresentationFrameProcessor,
and EncoderCompatibilityFrameProcessor now each implement
MatrixTransformation instead of wrapping
MatrixTransformationFrameProcessor.
PiperOrigin-RevId: 446480286
The TrackSelector is released when the player is released. The
TrackSelector can be reused if TrackSelector.init() is called again.
PiperOrigin-RevId: 446439717
This change splits AdvancedFrameProcessor into 4 files:
- MatrixTransformationFrameProcessor for the GlFrameProcessor
implementation
- MatrixTransformation and GlMatrixTransformation for the GlEffect
specification
- MatrixUtils for the static matrix helpers
PiperOrigin-RevId: 446236384
Some APIs from Android 12L were used either via reflection or
constants values were hard-coded. We can now use these APIs directly
since we upgraded the compile SDK version to 32.
PiperOrigin-RevId: 446167543
Also add warning for what can happen if this is ignored.
Previously the return value was overridden by setOutputSurface, now
setOutputSurface only overrides the values using internally.
PiperOrigin-RevId: 445377036
Developers are expected to (eventually) only use methods on
PlayerView (and not PlayerControlView) to interact with the UI
controller.
PiperOrigin-RevId: 445361488
This is consistent with the IntDef name, and frees up the CONTENT_TYPE_
prefix for the @ContentType values (which are currently just TYPE_*,
and therefore ambiguous with lots of other 'type' values in C).
PiperOrigin-RevId: 445356476
This method was introduced in 754eb1527a
as a replacement for Util.inferContentType(String) but it incorrectly
didn't return TYPE_SS when passed "ism" or "isml".
PiperOrigin-RevId: 445217167
We add an entire class like we do for parsing other codec initialization formats; it's currently not doing any parsing though (... initialization data is really simple for AV1 though: just the entire contents of the box).
For testing, we add the sample file, having been re-encoded with ffmpeg (and we also happen to have another av1 file, too).
PiperOrigin-RevId: 444890282
This fixes some small niggles:
1. `inferContentType(String)` is documented to take a path, but in the
tests we're passing full URIs.
2. A `String` parameter is usually a path, but also a MIME type or an
extension. In the new methods, the meaning of a `String` parameter
is always clear from the name of the method.
3. `inferContentType(String)` is always passed an extension in
'production' code (which has to be manually prefixed with a dot).
4. `inferContentType(Uri, @Nullable String)` always ignores the Uri if
the String is non-null. IMO this logic is clearer to a reader if it's
just in-lined at the call-site.
These methods are used from the demo apps, so will be part of the stable
API.
PiperOrigin-RevId: 444826053
To ensure frame processor operations operate on square pixels,
make the frame taller or wider for non-square input pixels.
In addition to automated tests, this was tested by changing the
inputFormat.pixelWidthHeightRatio in the TransformerVideoRenderer.
PiperOrigin-RevId: 444553517
This allows the actual bitmap to be saved, even if output
dimensions are different than expected. Otherwise, differing
output dimensions would throw an exception, preventing the bitmap
from being saved.
PiperOrigin-RevId: 444512210
The last frame processor could use a different drawing command
than a four vertex triangle strip. So we need call its drawing
method again instead of assuming the shape.
PiperOrigin-RevId: 444217274
Avoid no-op FrameProcessors in VideoTranscodingSamplePipeline, to avoid
creating resources for and executing GL for a no-op vertex+fragment shader.
EncoderCompatibilityFrameProcessor will still always be exercised.
PiperOrigin-RevId: 443675833
This test should run on all devices from API 21 (the media uses Baseline
profile level 3.0 H.264) to give us coverage of the full pipeline (forcing
re-encoding) and SSIM calculation on all devices.
PiperOrigin-RevId: 443650002
Split rotationDegrees changes to EncoderCompatibilityFrameProcessor, a new
FrameProcessor.
This removes automatic rotationDegrees adjustments from Presentation, which
allows Presentation to be used for changes before the end of a
FrameProcessorChain pipeline.
PiperOrigin-RevId: 443387226
The issue with decoding getting stuck is still reproducible on `samsung/beyond1lteeea/beyond1:12/SP1A.210812.016/G973FXXUEGULB:user/release-keys`.
PiperOrigin-RevId: 443368546
Some devices under report their resolution support, like 2144 for 2160 in
H265, 1072 for 1080 in H264. This CL only takes care of these two cases,
- reporting 1920x1080 is supported when the device reports 1920x1072, and
- reporting 3840x2160 is supported when the device reports 3840x2144
PiperOrigin-RevId: 443095042
When downlading an adaptive asset, if an ExoPlaybackException happens
during track selection, the player raises an
UnsupportedOperationException which is not handled gracefully and can
crash the app main thread.
This change catches the error and forwards it to
DownloadHelper.Callback.onPrepareError() as an IOException.
PiperOrigin-RevId: 443015332
This is safer because it will prevent any future problems with
creating a new FrameProcessorChain before the previous one has
completed its async release.
From [eglDestroyContext documentation](https://www.khronos.org/registry/EGL/sdk/docs/man/html/eglDestroyContext.xhtml):
"If the EGL rendering context is not current to any thread,
eglDestroyContext destroys it immediately."
The context isn't current to any thread here because GlUtil calls
eglMakeCurrent with EGL_NO_CONTEXT before calling eglDestroyContext.
So everthing should be released once the FrameProcessorChain's
release task terminates.
PiperOrigin-RevId: 442807484
* Group what's now many related test PNGs by moving them to their own directory.
* Move bitmap references to files where they're used, as each bitmap is only
used once each, except the original bitmap.
PiperOrigin-RevId: 441485489
Scaling and rotation using ScaleToFitFrameProcessor may change the
the resolution and aspect ratio, so defaulting to the same as the
input is no longer accurate.
PiperOrigin-RevId: 441463349
What a minimal implementation should include is now explained in the
interface javadoc while the method name reflects what the method does.
PiperOrigin-RevId: 441432059
The stream with the mp3 track is added because
- We only encode to AAC
- We only encode when the source track is not AAC
Now that we have a way to force encoding, we no longer need the mp3 track.
The test asset is kept for later parameterized testing.
PiperOrigin-RevId: 440876080
The MIME type was set to H265 to force transcoding. Now that we have an encoder
factory that forces encoding, switching back to H264 ensures the quality test
is conducted on more devices (those don't support H265 can be tested now).
However, H265 should be part of the quality test after we have proper mechanism
to skip test based on device capability.
PiperOrigin-RevId: 440132471
The DefaultMediaNotificationProvider checks if a command is available
before putting the respective action in the notification.
PiperOrigin-RevId: 440114422
We don't currently have enough understanding of the correlation between a
specific SSIM score and video quality. Dropping to .90 to make most tests pass.
Especially when there's no discernible difference from the videos with .9 and
.95 SSIM.
PiperOrigin-RevId: 440047551
PeriodicDimmingFrameProcessor is an example of how a custom fragment
shader can be used to apply color changes that change over time.
PiperOrigin-RevId: 439840609
The matrix provider allows the transformation matrix to be updated
for each frame based on the timestamp.
The following example effects using this were added to the demo:
* a zoom-in transition for the start of the video,
* cropping a rotating rectangular frame portion,
* rotating the frame around the y-axis in 3D.
PiperOrigin-RevId: 439791592
We add an entire class like we do for parsing other codec initialization formats; it's currently not doing any parsing though (... initialization data is really simple for AV1 though: just the entire contents of the box).
For testing, we add the sample file, having been re-encoded with ffmpeg (and we also happen to have another av1 file, too).
PiperOrigin-RevId: 439453823
This allows apps to use AdvancedFrameProcessor to apply transformations
in 3D space. This functionality is not used in transformer otherwise.
PiperOrigin-RevId: 439313406
The encoder surface is no longer needed for the OpenGL setup and frame
processor initialization, as a placeholder surface is used instead. So
all of the setup can now be done in the factory method.
PiperOrigin-RevId: 438844450
App code should get all of this information from TrackGroupInfo,
and should only need TrackGroup as a key to use for overrides.
PiperOrigin-RevId: 438840925
This allows the MuxerWrapper to keep using trackTypeToTimeUs for
calculating the video duration but slightly changes the meaning of
its interleaving constraints.
PiperOrigin-RevId: 438780686
This provides better compatibility with MediaExtractor, which does read these fields; we also need them for being able to mux file contents into another mp4 file.
Also, there is a minor refactor included so that we have an actual type for esds box contents instead of a pair.
PiperOrigin-RevId: 438673825
The placeholder surface is either EGL_NO_SURFACE or a 1x1 pbuffer
depending on whether the device supports EGL_KHR_surfaceless_context.
PiperOrigin-RevId: 438541846
Since the output textures and surfaces are managed by the
FrameProcessorChain, clearing them there makes sense.
This is also less error-prone as it might not be obvious to
someone implementing a GlFrameProcessor that they need to
glClear. (Clearing twice won't cause any problems.)
PiperOrigin-RevId: 438532247
This requires an additional nanos to micros conversion because
the SurfaceTexture uses nanos. But as the timestamps from the
MediaCodec decoder (propagated in DefaultCodec#releaseOutputBuffer) are
in microseconds no precision is lost here.
Also add test that checks output video duration.
PiperOrigin-RevId: 438010490
MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE was copied from a test
class, but BitmapTestUtil isn't a test. So the javadoc needs
rewording to reflect that.
PiperOrigin-RevId: 438001833
The problem is not the IntDef array, it's the fact the lint tool
is unable to correctly infer the annotations on the lambda parameters
without them being explicitly annotated. It seems explicitly annotating
is better than suppressing all IntDef warnings in the whole method.
PiperOrigin-RevId: 437969271
From Android T onwards `MediaCodec` supports requesting tone-mapping down to
SDR. Add an option to request this behavior and document that it isn't
supported before T. Also add an option in the demo app to try it out.
Tested manually on a prerelease build.
PiperOrigin-RevId: 437765325
The thread name is used to verify the thread in both createOpenGlObjectsAndInitializeFrameProcessors() and processFrame().
Also remove glThread field that was only used for this verification.
PiperOrigin-RevId: 437730804
Since the output size can be overridden, the viewport should be
ouputWidth/Height and NOT the ExternalCopyFrameProcessor's output size
which matches the input size.
PiperOrigin-RevId: 437256635
OutlineColour should be treated as the background color if BorderStyle=3. Since currently BorderStyle is ignored, we can always treat OutlineColor as the background color.
* The AdvancedFrameProcessor calls use() in updateProgramAndDraw().
* The AdvancedFrameProcessor has the same input and output dimensions.
PiperOrigin-RevId: 437231350
