Deprecate `setInputDefaultBufferSize` and `setRequireRegisteringAllInputFrames`
as the new input stream type replaces these (as far as we know they are always
used together).
This is in preparation for supporting asset loaders signaling that they require
these features, specifically for recording from a surface.
PiperOrigin-RevId: 648686087
Previously, if renderFramesAutomatically = false, DefaultVideoFrameProcessor
may call onInputStreamProcessedListener after all frames have been
onOutputFrameAvailableForRendering, but before they had all been rendered and
freed.
Delay onInputStreamProcessedListener being called and subsequent DVFP
reconfiguration of effects, until all frames are rendered.
Tested using exoplayer setVideoEffects demo with playlist
PiperOrigin-RevId: 642963100
useHdr is unused option and doesn't make sense is the dynamic range of the overlay and video must match
Also reorders and adds javadoc in line with coding conventions
PiperOrigin-RevId: 642555396
Avoid scaling the lanczos windown function when scaling up.
When scaling up (output > input), we shouldn't scale the window
function or we risk not sampling any of the input pixels.
PiperOrigin-RevId: 641838149
Add SeparableConvolution.configure(inputSize) to allow effect configuration
depending on input dimensions.
Add LanczosResample.scaleToFit method to scale input images to fit inside
given dimensions.
PiperOrigin-RevId: 640498008
fragment_shader_separable_convolution_es2.glsl had optimizations that assumed
all convolution coefficients are positive. Support negative coefficients,
and add tests.
PiperOrigin-RevId: 640104741
Switch from 4-channel RGBA_16F lookup texture to 1-channel R_16F.
Do not use a bitmap when creating the lookup table texture.
Instead, fill the texture directly.
Do not manually convert 32-bit float to 16-bit. Instead, let OpenGL
libraries do this for us.
PiperOrigin-RevId: 639717235
Add DefaultVideosFrameProcessor experimental flag that controls
whether input Bitmaps are sampled once for a repeating sequence of
output frames with the same contents, or once for each output frame.
PiperOrigin-RevId: 637921350
Only sample from input bitmap when the input image has changed.
Introduce GainmapShaderProgram.newImmutableBitmap API that signals
input bitmap changes to GainmapShaderProgram (DefaultShaderProgram).
PiperOrigin-RevId: 637920207
Before this CL, externalShaderProgramInputCapacity was not reset when
the external shader program was reset (which occurs when the
InputSwitcher switches to an input with a different ColorInfo). This is
due to a regression introduced in bef3d518d2.
PiperOrigin-RevId: 637869215
PQ and HLG have different luminance ranges (max 10k nits and max 1k nits resp). In GL, colors work in a normalised 0 to 1 scale, so for PQ content, 1=10k nits and and for HLG content, 1=1k nits.
This cl scales and normalises PQ content appropriately so that all HDR content works in the HLG luminance range. This fixes two things
1. Conversions between HLG and PQ are "fixed" (before the output colors looked too bright or too dark depending on which way you are converting)
2. color-altering effects will be able to work consistently across HLG and PQ content
1 is tested in this cl. 2 will be tested when ultra HDR overlays are implemented, both cases have been manually tested to ensure the output looks correct on a screen.
PiperOrigin-RevId: 636851701
Fragment shaders in OpenGL ES shader language aren't guaranteed
to support highp, required to correctly represent pixel coordinates
inside large images (e.g. 1920x1080).
This change moves coordinate mirroring for images out of fragment shader.
Fixes http://Issue: androidx/media#1331
PiperOrigin-RevId: 635732208
If MediaCodec allocates passes an image buffer with a cropped region,
SurfaceTexture.getTransformMatrix will cut off 2 pixels from each dimensions.
The resulting videos will appear a little stretched.
This patch inspects the SurfaceTexture transform matrix, and guesses what the
unscaled transform matrix should be.
Behind experimentalAdjustSurfaceTextureTransformationMatrix flag
PiperOrigin-RevId: 635721267
Gainmaps don't currently have an equals override, only reference equality is checked by Objects.equals(gainmap1, gainmap2). Create an equals method for gainmaps with the fields we care about to ensure we don't incur false positives in our equality checks.
PiperOrigin-RevId: 635510451
This annotation is only needed on public classes.
This change also removes the `/* package */` comment from some `public`
classes.
PiperOrigin-RevId: 633864544
After this CL, DVFP waits for flushing until all frames registered previously
arrives.
Previously, ETM records the difference between the number of registered frames,
and the number of frames arrivd on the SurfaceTexture, when flushing. (Note
that ETM is flushed the last in the chain, as flushing is done backwards from
FinalShaderProgramWrapper). ETM then waits until the number of frames arrive
after flush.
The normal flow is, MediaCodecVideoRenderer (MCVR) registers a new decoded
frame, in `processOutputBuffer()` to DVFP, MCVR call `codec.releaseOutputBuffer()`
to have MediaCodec render the frame, and then the frame arrives in DVFP's ETM.
However there might be a discrepancy. When registering the frame, ETM records
the frame on the calling thread, ~instantly. Later when the rendered frame
arrive, ETM records a frame is available on the task executor thread (or
commonly known as the GL thread). More specifically, when a frame arrives
in `onFrameAvailableListener`, ETM posts all subsequent processing to
the task executor. When seeking, the task executor is flushed as the first
step. It might be a frame that has already arrived on ETM, and the processing
of such frame has already been queued into the task executor; only to be
flushed as a result of flushing the task executor. If this happens, the frame
is considered to be never have arrived. This causes a freeze on the app,
because ETM'll wait until this frame arrives to declare flushing has completed.
PiperOrigin-RevId: 631524332
The second stage of the changes remove the conversion to linear colors in the SDR effects pipeline by default.
also resolves Issue: androidx/media#1050
PiperOrigin-RevId: 630108296
Part of a two stage change to remove the conversion to linear colors in the SDR effects pipeline by default. Changes the boolean to an intdef, introducing a third option that gets all sdr input into the same colorspace.
This is a planned API breaking change, but this change should not change the behavior of the pipeline.
PiperOrigin-RevId: 629013747
also makes the setter more flexible by ignoring the value of the setter when the output is hdr rather than throwing (since all HDR content must be have a linear color space)
PiperOrigin-RevId: 627388436
We currently check certain properties about inputColorInfo twice: once in checkColors in the DVFP and once when creating the samplers in the defaultshaderprogam. The inputcolorinfo shouldn't change between these two components so removing the checks in the shader program to have one source of truth for inputColorInfo checks, which will make the code simpler when it comes to adding to color-related features (like the ones related to removing de-gamma-ing)
PiperOrigin-RevId: 626400960
Makes OverlayFrameAnchor works as described in the OverlaySettings documentation. Currently the code does the opposite e.g setting the anchor to (+1,-1) makes the code anchor to the top left rather than the bottom right.
PiperOrigin-RevId: 621585558
Otherwise, apps overriding BitmapOverlay.getVertexTransformation may not
realize this is being done, and may ask why bitmaps are flipped vertically from
what they expect
Reference: https://github.com/androidx/media/issues/1128
PiperOrigin-RevId: 621544348
nullness checks were removed a while ago from editing, so no need for them now.
Bit neater to have less code to parse too.
Also, removes RequiresNonNull
PiperOrigin-RevId: 616887502
This also makes it more consistent with newer testId tests that don't have nullness, so tests don't look inconsistent, and was pretty easy to do :P
PiperOrigin-RevId: 616856888
Confirmed that the HLG extension displays properly on Pixel 7 Pro, API 34 and Samsung S24, API 34. On these devices, this fixes the washed out HLG preview issue on API 34, where the PQ ext had a washed out look, and where the HLG ext is supported (this bug didn't occur on API 33, only 34).
More info on manual tests done to sanity-check:
* Test cases: Transformer debug SurfaceView and ExoPlayer.setVideoEffects
* Test inputs: HLG and PQ
* Test devices: Pixel 7 Pro (API 33 & 34), Samsung Galaxy S24 (API 34)
* Added debugging: Logging colorInfo used in GlUtil.createEglSurface
No regressions were seen. HLG extension is used more in API 34, and behavior stays the same on API 33. Only human-visible change without logging is that HLG content looks better on API 34, for Samsung S24 and Pixel 7 Pro.
PiperOrigin-RevId: 616131192
This new test is for `ExoPlayer.setVideoEffects()`. It plays the
one-second-long video, applies an overlay that prints the video frame timestamp
onto the frame, captures the output frame and compares the captured output
frame with golden.
PiperOrigin-RevId: 610781590
VideoFrameProcessor requires every input frame to be registered before they are
rendered to its input Surface. This CL adds the option to register the input
frame only once. This is useful for camera inputs where not all input frames
come with changing FrameInfo.
PiperOrigin-RevId: 606294894
The public APIs of these modules reference symbols in some of their
dependencies, so these should be API dependencies, not implementation:
> An API dependency is one that contains at least one type that is
> exposed in the library binary interface, often referred to as its ABI
> (Application Binary Interface).
https://docs.gradle.org/current/userguide/java_library_plugin.html#sec:java_library_recognizing_dependencies
Transformer also uses symbols from `lib-common`, but these are already
an API dep of `lib-exoplayer` so no need to duplicate that here.
PiperOrigin-RevId: 605660621
Previously, we missed the BT2020->BT709 color-space conversion.
A user-visible impact of this is that red and green channels used to be
undersaturated, but now are more correctly saturated.
PiperOrigin-RevId: 605411926
removing the useHdr parameter from queueInputBitmap() it suggests we support changing between HDR and SDR within a stream, which we don't support. instead, identifying whether to use HDR from the shaderprogram which is informed by the inputColorInfo when the stream is registered.
PiperOrigin-RevId: 603681736
Transformer export and ExoPlayer previewing both read inputColorInfo from
registerInputStream now, instead of maintaining a consistent input color
throughout multiple streams in a sequence.
Therefore, we can remove inputColor-related arguments and methods now.
PiperOrigin-RevId: 603423509
If nullness should result in an exception, we should throw as soon as possible,
so that stack traces are easier to follow.
Did a quick scan of media3.effect+transformer and this covers all uses of
peek that immediately throw there.
PiperOrigin-RevId: 603393366
VideoFrameProcessor treats BT601 and BT709 as roughly equivalent now, so we
shouldn't be making checks that assume BT709 <-> requires tone-mapping.
Also, the color transfer is a better determinant for tone-mapping than color range, so use just the transfer to determine if tone-mapping is required.
PiperOrigin-RevId: 603083100
This happens when using `ExoPlayer.setVideoEffects()`.
This CL also fixes the first frame not rendered problem, originally solved in
7e65cce967, but rolled back in 5056dfaa2b because the solution introduces
the flash that is observed in b/292111083.
Before media3 1.1 release, the output size of `VideoFrameProcessor` is not
reported to the app. This was changed later after introducing
`CompositingVideoSinkProvider`, where the video size after processing **is**
reported to the app. After this CL, the size is again, not reported.
PiperOrigin-RevId: 602345087
Many usages are needed to support other deprecations and some
can be replaced by the recommended direct alternative.
Also replace links to deprecated/redirected dev site
PiperOrigin-RevId: 601795998
