Flushing resets all the texture processors within the `FrameProcessor`. This
includes:
- At the back, the FinalMatrixTextureProcessorWrapper, and its MatrixTextureProcessor
- At the front, the ExternalTextureManager
- All the texture processors in between
- All the ChainingGlTextureProcessorListeners in between texture processors
- All the internal states in the aforementioned components
The flush process follows the order, from `GlEffectsFrameProcessor.flush()`
1. Flush the `FrameProcessingTaskExecutor`, so that after it returns, all tasks queued before calling `flush()` completes
2. Post to `FrameProcessingTaskExecutor`, to flush the `FinalMatrixTextureProcessorWrapper`
3. Flushing the `FinalMatrixTextureProcessorWrapper` will propagate flushing through, via the `ChainingGlTextureProcessorListener`
Startblock:
has LGTM from christosts
and then
add reviewer andrewlewis
PiperOrigin-RevId: 506296469
On some devices/builds it seems that the `SurfaceView` constructor tries to create a handler. Move instantiation to the main thread when getting the debug surface view so that handler creation succeeds.
PiperOrigin-RevId: 505745387
Also add a Builder to EditedMediaItem to avoid having a constructor with
many optional parameters, or a chain of constructors.
PiperOrigin-RevId: 504588544
Also remove usages of TransformationRequest convenience methods
(setScale, setRotationDegrees and setResolution).
Some usages of setResolution can't be removed yet because they are used
for fallback.
PiperOrigin-RevId: 502415748
Starting with API 33 the POST_NOTIFICATION permission needs to be
requested at runtime or the notification is not shown.
Note that with an app with targetSdkVersion < 33
but on a device with API 33 the notification permission is automatically
requested when the app starts for the first time. If the user does not
grant the permission, requesting the permission at runtime result in
an empty array of grant results.
Issue: google/ExoPlayer#10884
PiperOrigin-RevId: 501320632
- Do not fail silently if selectedEffects is null.
- Do not add an empty overlay effect to the Transformer to avoid
transcoding video all the time.
PiperOrigin-RevId: 499168759
We might as well keep this enabled by default, rather than having to
manually toggle it on to investigate RTL issues like Issue: androidx/media#227.
PiperOrigin-RevId: 497159744
Note that we simply use GlEffectsFrameProcessor in-app / GL tone-mapping, so PQ->SDR tone-mapping isn't yet implemented.
Tested manually using the demo on Pixel 7, to confirm that device and in-app tone
mapping behave similarly.
PiperOrigin-RevId: 496700231
To do that, rename PROGRESS_STATE_NO_TRANSFORMATION to
PROGRESS_STATE_NOT_STARTED and update Javadoc of ProgressState to not be
Transformer specific.
PiperOrigin-RevId: 496653460
The folder type has a mix of information about the item. It shows
whether the item is browsable (type != FOLDER_TYPE_NONE) and
which Bluetooth folder type to set for legacy session information.
It's a lot clearer to split this into a boolean isBrowsable and
use the existing mediaType to map back to the bluetooth folder type
where required.
folderType is not marked as deprecated yet as this would be an API
change, which will be done later.
PiperOrigin-RevId: 493544589
From this CL on, FrameProcessor listeners will be invoked from an Executor that
is passed in when creating the FrameProcessor.
GlTextureProcessor needs to invoke the ErrorListener on the said Executor too.
PiperOrigin-RevId: 493018583
When app is deployed with device's screen being off, MainActivity's onStart is called swiftly by its onStop. The onStop method cancels the browserFuture task which in turn "completes" the task. Upon task "completion", pushRoot() runs and then throws error as it calls get() a cancelled task.
PiperOrigin-RevId: 492416445
This change makes adding ad events in live streams more robust by allowing ad
groups to grow in number of ads if more ad events are received than initially
announced by the SDK.
With the IMA prefetch feature, an AdPod can grow in size in certain conditions
like from initially 2 ads to 4 ads being part of the ad group. With this change,
if an additional ad event arrives while the ad group is still being played,
the ad group is expanded. If the event arrives late and the ad group is already
completed, a new group is created for the remaining ads.
This also covers the case where we join the live stream while an ad is being
played and we missed at least one LOADED event from the SDK. Ads of the group
before the first LOADED event are ignored in such a case.
PiperOrigin-RevId: 484214760
Most demo videos aren't very long, and the default demo video is only 10 seconds.
Shorten the maximum trim duration to 10 seconds, to demonstrate transformer functionality more easily, and allow this to be used more easily when trimming short sections of a longer video (ex. to make test clips)
PiperOrigin-RevId: 480602037
Player controls are somewhat distracting when showing the difference between the
input and output video, as they obscure and darken the video players.
PiperOrigin-RevId: 480597804
Before, slider values were read as `floor()`'ed `longValue()`s, so that trimming to
intervals less than one second would be interpreted as a request for a zero-
duration trim.
Also, rename `radiusRange` references here to `trimRange`, since this is not a
radius range.
PiperOrigin-RevId: 480401556
PlayableFolderActivity currently uses two seperate commands
to set the playlist and then seek to the requested item.
This is slightly inefficient and also causes analytics
listeners to believe the first item in the playlist was
briefly active.
#minor-release
Issue: androidx/media#180
PiperOrigin-RevId: 480031682
Replacing remaining usage of MoreExecutors.directExecutor. This allows the service to be switched to run in another process and the app still works the same as if it is running in the same process.
Issue: androidx/media#100
PiperOrigin-RevId: 471547177
This change adds a new method onReadyToAcceptInputFrame to
GlTextureProcesssor.InputListener and changes maybeQueueInputFrame
to queueInputFrame, removing the boolean return value.
This avoids the re-trying in ChainingGlTextureProcessorListener
by allowing it to only feed frames from the producing to the consuming
GlTextureProcessor when there is capacity.
MediaPipeProcessor still needs re-trying when processing isn't 1:1.
PiperOrigin-RevId: 466626369
Without this permission, files in `/sdcard` or other directories cannot be read by the demo, as they're not in the transformer demo's scoped storage container.
For more information, see https://developer.android.com/training/data-storage/use-cases
Tested by uninstalling and re-installing the demo app, granting permission by
starting a transformation, and launching an intent using a local file in `/sdcard`
to start another transformation. Without this CL, this threw an error, and with
this CL it succeeded.
PiperOrigin-RevId: 466399023
This simplifies ChainingGlTextureProcessor as it now only connects a
consuming and a producing GlTextureProcessor rather than a previous,
current, and next GlTextureProcessor.
Also use default no-op implementations of the listeners in
SingleFrameGlTextureProcessor and MediaPipeProcessor to avoid
null-checks.
PiperOrigin-RevId: 466301642
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.
PiperOrigin-RevId: 464060047
Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.
PiperOrigin-RevId: 463206474
This is needed for applying effects to a playlist.
The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.
Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.
PiperOrigin-RevId: 462714966
This allows us to use BT.2020 RGB linear for intermediate shaders, which also
allows us to re-enable PeriodicVignetteProcessor, which should work properly in
linear color-spaces.
Manually tested by adding a GlEffectsWrapper, and confirming that HLG HDR editing still looks correct.
PiperOrigin-RevId: 462265821
* Introduced `useHdr` for `GlEffect#toGlTextureProcessor`, so
`TextureProcessor` implementations can decide how to handle HDR.
* Creating FP16 color textures for HDR input.
Tested via manual testing, adding a no-op GlEffectWrapper to the transformation to
force use of intermediate textures, adding a linear ramp to the fragment shader,
and trying to ascertain that there's a real reduction in posterization when
switching from 4-bit to 8-bit unsigned bytes, and again from 8-bit unsigned bytes
to 16-bit floating point.
PiperOrigin-RevId: 461613117
The old URL doesn't correctly signal the HDR10 color info in the container.
The new URL signals ST2084 (PQ) transfer function and BT.2020 color space as expected.
PiperOrigin-RevId: 461560107
This allows the service to be switched to run in another process and the app still works the same as if it is running in the same process.
Issue: androidx/media#100
PiperOrigin-RevId: 458460005
The GlEffectsFrameProcessor that will be part of the effects module
uses the DebugViewProvider. So it does not make sense for it
to be an inner interface of Transformer.
PiperOrigin-RevId: 458014932
After this change GlEffects can use any GlTextureProcessor not just
SingleFrameGlTextureProcessor.
MediaPipeProcessor now implements GlTextureProcessor directly which
allows it to reuse MediaPipe's output texture for its output texture
and avoids an extra copy shader step.
PiperOrigin-RevId: 456530718
After this change, FrameProcessorChain chains any GlTextureProcessors
instead of only SingleFrameGlTextureProcessors.
The GlTextureProcessors are chained in a bidirectional manner using
ChainingGlTextureProcessorListener to feed input and output related
events forward and release events backwards.
PiperOrigin-RevId: 456478414
This is useful for testing Transformer in the same way as it is used
in tests and to see only the real transformation time.
PiperOrigin-RevId: 456058466
Based on
https://developer.android.com/reference/android/media/MediaCodec#using-an-output-surface,
frame dropping behaviour depends on the target SDK version.
After this change transformer will only use
MediaFormat#KEY_ALLOW_FRAME_DROP if both the target and system SDK
version are at least 29 and default to its pre 29 behaviour where each
decoder output frame must be processed before a new one is rendered
to prevent frame dropping otherwise.
Also remove deprecated Transformer.Builder constructor without a
context and the context setter.
PiperOrigin-RevId: 453971097
Transformer always enabled glAssertionsEnabled, so there should
be no functional change.
ExoPlayer previously disabled glAssertionsEnabled, so GlUtil logged
GlExceptions instead of throwing them. The GlExceptions are now
caught and logged by the callers so that there should also be no
functional change overall.
This change also replaces EGLSurfaceTexture#GlException with
GlUtil#GlException.
PiperOrigin-RevId: 453963741
SingleFrameGlTextureProcessor is now an abstract class containing a
default implementation of the more flexible GlTextureProcessor interface
while still exposing the same simple abstract methods for single frame
processing it previously did.
FrameProcessorChain and GlEffect will be changed to use
GlTextureProcessor in follow-ups.
PiperOrigin-RevId: 453633000
These legacy callbacks are currently forwarded to onSetMediaUri which
will be removed in the future.
Also make sure to only call player.prepare/play after the items have
been set.
The calls to onAddQueueItem are also forwarded to onAddMediaItems to
actually allow a session to resolve these items to playable media, which
wasn't possible so far.
PiperOrigin-RevId: 453625204
Once the more advanced GlTextureProcessor interface exists,
it will be possible to change the output size of a GlTextureProcessor
between frames. To keep the re-configuration based on the frame sizes
minimal, things indepedent of the frame size, such as the GlProgram,
can be initialized in the constructor.
PiperOrigin-RevId: 451997584
The MediaItemFiller is not flexible enough for most realworld usages
because:
- it doesn't allow asynchronous resolution of MediaItems (e.g. to
look up URIs from a database)
- it doesn't allow to batch updates for multiple items or do more
advanced customizations (e.g. expanding a mediaId representing
a playlist to multiple items).
Both issues can be solved by passing in a list of items and
returning a ListenableFuture. The callback itself can also move
into MediaSession.Callback for consistency with the other
callbacks.
PiperOrigin-RevId: 451857319
Decode-only video frames (needed when the frame at / first frame after the
clipping start is not a key frame) need to be decoded but not passed to
the frame processor chain or encoder.
The clipping start offset needs to be removed from the frame timestamps
in the passthrough and video pipelines.
There are no changes needed for this in the audio pipeline, as it doesn't
use the input timestamps -- it uses its own timestamps derived from the
buffer sizes instead.
Also add demo option to try this out.
#minor-release
PiperOrigin-RevId: 451353609
Also update names of implementations to match design doc.
In follow-ups, SingleFrameGlTextureProcessor will become
an abstract implementation of a new GlTextureProcessor
interface.
Texture processor makes sense as it processes OpenGL textures.
The term frame processor will be used for something else in
follow-ups.
PiperOrigin-RevId: 451142085
* Rename (via deprecation)
`MediaItem.DrmConfiguration.Builder#forceSessionsForAudioAndVideoTracks`
to `setForceSessionsForAudioAndVideoTracks`. This is more consistent
with existing 'force' method names both in this class and in
`TrackSelectionParameters.Builder`.
* Add missing `@Nullable` annotation to the parameter for
`MediaItem.SubtitleConfiguration.Builder#setMimeType`. This annotation
is already present on the `MediaItem.SubtitleConfiguration#mimeType`
field that this setter corresponds to.
PiperOrigin-RevId: 450941336
Put cloud storage samples at the top to avoid having a sample at the top of the list where we don't control the server.
Also update labels not to mention progressive container type, as it's irrelevant for Transformer, which always transmuxes even if it doesn't transcode.
#ame-bug-week
PiperOrigin-RevId: 450403784
