Allow the VideoFrameProcessor to output multiple textures at a time, so that
lifetime of textures is up to the consumer calling VFP.releaseOutputFrame.
The FinalShaderProgramWrapper also has a new maxCapacity limit added, to ensure
the a reasonable amount of textures is used and avoid using up memory.
PiperOrigin-RevId: 532094256
(cherry picked from commit 07ec1eaa48)
The periodic updates are only meant to happen while we are in the
same period or ad. This was already guaranteed except for two cases:
1. The Player in a session has updated its state without yet calling
its listeners
2. The session scheduled a PlayerInfo update that hasn't been sent yet
... and in both cases, the following happened:
- The change updated the mediaItemIndex to an index that didn't exist
in a previous Timeline known to the Controller
- One of the period position updates happened to be sent at exactly
this time
This problem can be avoided by only scheduling the update if we are
still in the same period/ad and haven't scheduled a normal PlayerInfo
update already.
Since new MediaControllers may still connect to old sessons with this
bug, we need an equivalent change on the controller side to ignore such
buggy updates.
PiperOrigin-RevId: 532089328
(cherry picked from commit 96dd0ae583)
When exporting compositions with multiple images in a row, transformation could
get stuck if a shader was ready to accept input when end-of-stream was already
signaled and queued from upstream. Fix accounting for the downstream capacity.
Manually tested on concatenations with several images and several videos in a
row, by adding logging and verifying the capacity updates as expected across
edited media item transitions.
PiperOrigin-RevId: 532088793
(cherry picked from commit 6850391e45)
In the past, the SequenceAssetLoader was released in TransformerInternal
when the export ended.
fc539da061 was made to release the SequenceAssetLoader earlier, when
loading ended. This was causing player release timeouts because the last
AssetLoader in the sequence was released before the SamplePipelines (see
0b40bc37ab for more information).
The code that was releasing the SequenceAssetLoader was first commented
out because we didn't have an understanding of what was happening.
This change removes the early SequenceAssetLoader release all together.
It doesn't have any effect as this code was already commented out.
PiperOrigin-RevId: 532065673
(cherry picked from commit 6dfb387117)
The output end-of-stream notification from the last shader could theoretically
arrive before the latch for detecting it is created, which might cause waiting
on the latch indefinitely. Create the latch before signaling end of stream so
that it's guaranteed to be set before the end-of-stream signal arrives.
PiperOrigin-RevId: 532056472
(cherry picked from commit 857e6ebee8)
Use a non-deprecated constructor that takes a `DatabaseProvider`
instead for better performance.
#minor-release
PiperOrigin-RevId: 532046598
(cherry picked from commit 0a86790be2)
*** Original commit ***
Rollback of 221a56da38
*** Original commit ***
Rollback of 749d77b1d9
*** Original commit ***
PiperOrigin-RevId: 531530885
(cherry picked from commit 9366b4e50a)
When a `MediaButtonReceiver` is found in the manifest, the library
can implement the contract of SystemUI to signal that the app wants
a playback resumption notification to be displayed.
And, vice versa, if no `MediaButtonReceiver` is in the manifest, the
library will signal to not show the notification after the app has been
terminated.
#minor-release
PiperOrigin-RevId: 531516023
(cherry picked from commit 9bf6b7ea20)
*** Original commit ***
Rollback of 65d5132f76
*** Original commit ***
Create InAppMuxer in transformer
To use the InAppMuxer, the client needs to pass InAppMuxer Factory.
***
***
PiperOrigin-RevId: 531470081
(cherry picked from commit 867355fdc5)
Until the linked bug is fixed, relax constraints to allow this one device to
pass, to suppress failures and avoid triage toil.
PiperOrigin-RevId: 531259233
An app must only have a single service in the manifest that is either
exposing `MediaLibraryService.SERVICE_INTERFACE` or
`MediaSessionService.SERVICE_INTERFACE`. Hence the component name found by
querying the package manager for a service never returns a different result
when queried again.
The static `MediaSessionImpl.serviceComponentName` can hence be removed
and the package manager queried again when a second session instance
is created.
#minor-release
PiperOrigin-RevId: 531210456
This change adds end-to-end Robolectric playback tests which handle
the scenario the player is playing audio via passthrough and
AudioTrack raises the ERROR_DEAD_OBJECT error upon which the player
attempts to recover by switching to another audio format.
PiperOrigin-RevId: 531180183
The texture input tests in TransformerEndToEndTest were not passing on
Pixel 7. Implemented a fix and fixed other minor threading issues I
spotted while looking at the code.
PiperOrigin-RevId: 531141659
*** Original commit ***
Create InAppMuxer in transformer
To use the InAppMuxer, the client needs to pass InAppMuxer Factory.
***
PiperOrigin-RevId: 531056436
Tone-mapping an HDR video with MediaCodec on sm-s908u1 was timing out.
The reason for that is that the decoder was dropping frames, and the
ExternalTextureManager was therefore never propagating the end-of-stream
signal.
There was already a workaround for a similar issue but restricted to
sm-f936b. Removed the model check as the bug is probably present on more
devices.
PiperOrigin-RevId: 530639437
The session activity is already sent to the controller with the
`ConnectionState` when it connects. This change adds the ability to update the
activity.
This allows an app to change the intent that is used to open an activity
for the notification. An app is likely to want to change the session activity
just before the session is released. This allows to use a different activity or
more importantly the back stack of the activity for while the app is running
and when used for the playback resumption notification.
PiperOrigin-RevId: 530627102
Exceptions thrown when switching AssetLoader were not reported through
the error listener. Therefore, the resources were not released and the
export was not ending.
PiperOrigin-RevId: 530626300
If tone-mapping was requested for a device supporting HDR encoding,
isToneMapped was false in VideoSamplePipeline.EncoderWrapper. This was
causing the encoder to expect HDR.
Also did some renamings to improve readability
PiperOrigin-RevId: 530584010
similar error to 9baa6f6be5L `registerInputStream` sets the textureManager, so reordering makes sure the textureManager is set before you set the frame info. This is important for texture input, where the frame info provides the width and height.
PiperOrigin-RevId: 530579584
Adding items to an empty playlist is slightly different from adding
items to a non-empty playlist, because the former usually requires to
handle a change in the current item, position and playback state,
while the latter is not expected to affect the current item, position
or state.
The current ExoPlayer and SimpleBasePlayer code doesn't account for
this difference, leading to inconsistent behavior between
setMediaItem(s) and addMediaItem(s) when called on an empty playlist.
PiperOrigin-RevId: 530549928
For DefaultVideoFrameProcessorVideoFrameRenderingTest: `registerInputStream` sets the textureManager, so reordering makes sure the textureManager is set before you set the frame info. This is important for texture input, where the frame info provides the width and height.
PiperOrigin-RevId: 529753404
`VERSION_INT` is quite long with several sections, and it's easy to make
a mistake when updating it - this should help since it checks it against
`VERSION`, which is more easily human readable/writable.
PiperOrigin-RevId: 529747023
Calling LibraryResult.toBundle() could have caused a CastClassException.
This was because when unbundled with UNKNOWN_TYPE_CREATOR.fromBundle(Bundle),
the valueType was set to VALUE_TYPE_ITEM_LIST for all types and the MediaItem
was attempted to be casted to a list.
PiperOrigin-RevId: 529717688
Also change some type parameter names in `MediaSession.BuilderBase`
because `C` now clashes with the import of `androidx.media3.common.C`.
#minor-release
PiperOrigin-RevId: 529665698
The media button has API support with
`Callback.getPlaybackResumption()` that apps need to override to provide
a playlist to resume playback with.
Issue: androidx/media#167
PiperOrigin-RevId: 529495845
* Implement RendererCapabilities.Listener in DefaultTrackSelector.
* Add new methods TrackSelector.invalidateForRendererCapabilitiesChange and TrackSelector.InvalidateListener.onRendererCapabilitiesChanged.
* Add new field allowInvalidateSelectionsOnRendererCapabilitiesChange to DefaultTrackSelector.Parameter to allow opt-in of the renderer capabilities detection feature.
* Add logics of triggering track reselection when renderer capabilities change.
PiperOrigin-RevId: 529067433
When we created androidTests, in the past, they always had a URI pointing to a resource, therefore we always had a URI scheme. With texture input, this will not longer be the case (EditedMediaItems's may have URI.EMPTY, which have a null scheme) so we need to check for this so tests don't falsely fail.
PiperOrigin-RevId: 528848411
The existing logic to drop (actually fail on) 0 sized samples seems no op if
if 2 out of 10 samples are of size 0.
Checked same scenario with MediaMuxer where
1. If input file has 300 samples.
2. Make every 5th sample as an empty byte buffer.
3. Output file is generated without error.
4. Output file has 240 samples.
5. Exoplayer is able to play output file (blurry).
The new change is in line with MediaMuxer behaviour.
PiperOrigin-RevId: 528798046
Systems accepting URIs should treat schemes as case-insensitive
([RFC 3986 Section 3.1](https://www.rfc-editor.org/rfc/rfc3986#section-3.1)):
> An implementation should accept uppercase letters as equivalent to
> lowercase in scheme names (e.g., allow "HTTP" as well as "http") for
> the sake of robustness
PiperOrigin-RevId: 528735287
This means that comments like `//copybara:media3-only` are now detected
(no space between `//` and `copybara`) which will ensure that lines like
this are correctly transformed for the media3 and exoplayer2 GitHub
repos:
aa4e008014/library/effect/build.gradle (L33)
PiperOrigin-RevId: 527919649
Also make FinalShaderProgramWrapper always receive internal texture.
This means it does not sample from a input texture, and its input color is
always linear, hence the input type does not matter.
PiperOrigin-RevId: 527869045
*** Original commit ***
Effect: glFlush instead of glFinish on tex output
This is much faster (~2-3x) than glFlush. While there's a risk that GL commands
queued to the GL server may not be complete by the time non-GL commands access
the texture, this should be unlikely as we only access the texture from GL.
If we see stability issues in the future, we can reconsider and move this back
to glFinish (or GL synchronization mechanisms like fences, which are more
complex)
***
PiperOrigin-RevId: 527848094
renderOutputFrame actually renders frames to an output surface. We'll soon have
a releaseOutputFrame method, that would release resources associated with an
output time, so rename this to disambiguate the two methods.
Also rename onOutputFrameAvailable to onOutputFrameAvailableForRendering, to
make it clear this is not available for "release"
This change should be a renaming-only change and have no functional differences.
PiperOrigin-RevId: 527844947
This is much faster (~2-3x) than glFlush. While there's a risk that GL commands
queued to the GL server may not be complete by the time non-GL commands access
the texture, this should be unlikely as we only access the texture from GL.
PiperOrigin-RevId: 527641520
References to the service are kept from MediaSessionStub
and from a long-delayed Handler messages in ConnectionTimeoutHandler.
Remove strong references from these places by making the timeout
handler static and ensuring ConnectedControllersManager only keeps
a weak reference to the service (as it's part of MediaSessionStub).
Issue: androidx/media#346
PiperOrigin-RevId: 527543396
Add `VideoFrameProcessor.registerInputStream()` to signal a new type of input.
And `InputHandler.signalEndOfCurrentInputStream()` to signal to `InputHandler`
partial input stream completion.
Fully processed means after FinalShaderProgramWrapper releases the last frame.
PiperOrigin-RevId: 527356646
In ASwB, all Transformer tests can be run by right-clicking on the project and
clicking "Run all tests". We cannot, however, select only some specific test
files within a project (ex. all non-analysis tests) to run tests on.
Add @Ignore to analysis tests, which are not intended to be run anyways
when determining whether Transformer is working on some device. These
tests also don't have proper skipping logic when a device doesn't support a
format, so they can't be run effectively on low-end devices anyways.
This eases manual testing, for example when debugging whether tests all pass
on a device.
When analysis tests are desired to be run, it should be easy to comment out
the @Ignore.
PiperOrigin-RevId: 527289600
This method doesn't really serve a purpose that isn't handled
elsewhere. The return value is also not forwarded to anyone.
PiperOrigin-RevId: 527283166
To reliably reject the System UI playback resumption notification on
all API levels (specifically API 30), the backward compatibility layer
needs to return `null` for the library root.
This is not possible in the Media3 implementation. This change allows
an app to return a `LibraryResult.ofError(RESULT_ERROR_NOT_SUPPORTED)`
that then is translated to return null by the backwards compatibility
layer.
Issue: androidx/media#355
Issue: androidx/media#167
Issue: androidx/media#27
See https://developer.android.com/guide/topics/media/media-controls#mediabrowserservice_implementation
PiperOrigin-RevId: 527276529
This is a helper method that can used to obtain information about
the controller that is currently calling a Player method.
PiperOrigin-RevId: 527268994
Allow the VideoFrameProcessor to output to a texture without an output surface.
Tested by updating texture output tests to no longer output to a surface.
PiperOrigin-RevId: 527244605
The only dependencies that are not updated are the ones that need
to be kept in sync with other system (like Android source tree) or
would require a Kotlin dependency in common or exoplayer modules.
As a side effect, some demo apps now need a Kotlin config and some
additional modules require desugaring/multidex logic. To simplify
the setup, the desugaring and multidex steps are added to the common
config.
PiperOrigin-RevId: 527243950
This was only used in EncoderWrapper, so move createSupportedTransformationRequest
from VideoSamplePipeline to EncoderWrapper to reduce its scope.
PiperOrigin-RevId: 527226578
The production code changes are in transformer, but the tests in effect have also been updated to confirm the is no color regression `inputColorInfo.colorTransfer=C.COLOR_TRANSFER_SRGB`
PiperOrigin-RevId: 526950435
Previously, I assumed that SDR contents must be BT709, and HDR contents must be
BT2020. Turns out BT2020 is just wide-gamut, and SDR contents / transfers may be
represented in BT2020 color spaces.
Relax the check, so that we don't throw when valid BT2020 SMPTE 170M contents
are input into effects.
PiperOrigin-RevId: 526668347
This implicitly fixes a bug by removing the buggy implementation in H265Reader in favour of a working one. This change also adds tests to confirm the parsing bug is fixed.
`prepare()` now logs a warning if it's called before `setPlayer()`
because it's not possible to tell if it's being called on the wrong
thread (since 3480a27994).
This change finds all the places one is called immediately after the
other and flips the order to be more correct.
Issue: androidx/media#350
#minor-release
PiperOrigin-RevId: 526582294
This simplifies the addition of new fields in the future.
Also do some misc clean up for the volume limit values:
- Add some documentation to mention assumed defaults
- Add the IntRange annotations to match the ones we have in Player
already
- Mention the limits in the relevant Player methods
- Avoid bundling default values
- Improve range checks for masking in MediaController
PiperOrigin-RevId: 526029619
This is a bug currently, where commands are created once but never
updated again if the actions in MediaSessionCompat are changed.
PiperOrigin-RevId: 525999084
If the limited number of input buffers causes reading of all samples except the last one conveying end of stream, then the last frame will not be rendered.
PiperOrigin-RevId: 525974445
Whenever a frame is queued to the shader, it's guaranteed that there will be a
subsequent `onInputFrameProcessed` callback, so we can pass on the
end-of-stream signal there.
PiperOrigin-RevId: 525850141
Creates a way for apps to provide their EGLContext to DefaultVideoFrameProcessor, so that we can attach their context to the one we create. See [the EGL docs for more information about how contexts are shared in GL](https://registry.khronos.org/EGL/sdk/docs/man/html/eglCreateContext.xhtml)
PiperOrigin-RevId: 525708652
Removes glObjectsProvider from `VideoFrameProcessor` and `Effects`. Apps will set the glObjectsProvider on the DefaultVideoFrameProcessor.Factory.Builder when providing a custom DefaultVideoFrameProcessor.Factory, rather than in `Effects`.
PiperOrigin-RevId: 525169059
A new texture manager will be created for input by texture ID so this texture manager won't be the only one to handle internal (i.e. non-external) textures.
PiperOrigin-RevId: 525116460
`findDecoder/EncoderForFormat` seem to be more restrictive than querying
`MediaCodecList` as done for playback, and it's not clear that the cases where
no codec is found correspond to actual cases that would fail given that it
seems this returns an empty string for many cases in production.
Switch to using `MediaCodecUtil` and `EncoderUtil` for querying codecs instead.
PiperOrigin-RevId: 525078321
This was just for testing and finishing this class is not useful
as it can't wrap an existing MediaPlayer without owning it to
track its state.
PiperOrigin-RevId: 524851654
The actual csd data contains NAL units so it should always start with
a NAL unit start code (00 00 00 01).
This issue was not caught before because in the code there was no
validation to check whether its a valid NAL unit.
PiperOrigin-RevId: 524849867
The sessions may have different application threads for their players,
and the service with its notification provider runs on the main thread.
To ensure everything runs on the correct thread, this change labels
methods where needed and fixes thread access in some places.
Issue: androidx/media#318
PiperOrigin-RevId: 524849598
This check was a leftover from when the metadata was generated from
the MediaItem only. Since we moved to the actual MediaMetadata fields,
the check is completely unnecessary and prevents accessing metadata
when the GET_CURRENT_MEDIA_ITEM command is not available.
#minor-release
PiperOrigin-RevId: 524837587
*** Original commit ***
Fix double initialisation of createIsoLanguageReplacementMap
In the absence of any lock `createIsoLanguageReplacementMap`
method was getting called twice due to race condition.
Used Suppliers.memoize() which is by default thread safe.
***
PiperOrigin-RevId: 524794650
Reusing the loader wrappers allows us to use the current RTSP connection
without having to set up a new RTSP connection. Consequently, the Extractors,
RTP readers are also preserved.
PiperOrigin-RevId: 524663012
In `ExternalTextureManager` in seemingly rare cases end of stream is signaled
at the point where a frame is currently pending processing. In that case the
video end of stream signal was lost. If the muxer timeout was enabled this
case would result in throwing an exception, but otherwise the operation would
get stuck
Add code to signal end of stream in `onInputFrameProcessed` as well, so that we
signal end of stream when the pending frame is handled.
Tested by running
`TransformerEndToEndTest.loopingTranscodedVideo_producesExpectedResult` several
times.
PiperOrigin-RevId: 524361069
MediaMuxer does not support out-of-order video frames API 25. Use
a test file with frames in order in
loopingTranscodedAudio_producesExpectedResult. This is necessary for
tests that transmux video samples.
PiperOrigin-RevId: 524309318
A passthrough effect allows for testing having an intermediate
effect injected, which uses different OpenGL shaders from having no
effects.
PiperOrigin-RevId: 524276991
In the absence of any lock `createIsoLanguageReplacementMap`
method was getting called twice due to race condition.
Used Suppliers.memoize() which is by default thread safe.
PiperOrigin-RevId: 524007754
