This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.
PiperOrigin-RevId: 464060047
(cherry picked from commit 06d41c2775)
Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.
PiperOrigin-RevId: 463206474
(cherry picked from commit b994f8bfa0)
This is needed for applying effects to a playlist.
The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.
Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.
PiperOrigin-RevId: 462714966
(cherry picked from commit 46ab06b816)
- Update profile selection logic to pick an HDR-compatible profile when doing HDR editing on H.264/AVC videos.
- Handle doing the capabilities check for all MIME types that support HDR (not just H.265/HEVC).
- Fix a bug where we would pass an HDR input color format to the encoder when using tone-mapping.
- Tweak how `EncoderWrapper` works so decisions at made at construction time.
Manually tested cases:
- Transformation of an SDR video.
- Transformation of an HDR video to AVC (which triggers fallback/tone-mapping on a device that doesn't support HDR editing for AVC).
- Transformation of an HDR video with HDR editing.
PiperOrigin-RevId: 461572973
(cherry picked from commit 0db07c6791)
- Added setter to disable this feature.
- Added accompanying tests.
- Plan to run tests on the same set of settings on H265.
PiperOrigin-RevId: 460238673
(cherry picked from commit 6e12600099)
The FinalMatrixTransformationProcessorWrapper ensures that the
surface is only replaced when it is not being rendered to and vice
versa.
PiperOrigin-RevId: 458007639
(cherry picked from commit e5527a8add)
This change is just renaming. There is no functional change intended.
The FrameProcessor interface will be created in a follow-up.
PiperOrigin-RevId: 456741628
(cherry picked from commit 216fefd669)
After this change, FrameProcessorChain chains any GlTextureProcessors
instead of only SingleFrameGlTextureProcessors.
The GlTextureProcessors are chained in a bidirectional manner using
ChainingGlTextureProcessorListener to feed input and output related
events forward and release events backwards.
PiperOrigin-RevId: 456478414
(cherry picked from commit 3a96691654)
In follow-ups the FrameProcessorChain will set an instance of this
listener for each GlTextureProcessor to chain it with its previous
and next GlTextureProcesssor.
PiperOrigin-RevId: 455628942
(cherry picked from commit c92e18ec58)
This change adds a SurfaceProvider interface which is necessary to
allow for texture processors whose output size becomes available
asynchronously in follow-ups.
VTSP's implementation of this interface wraps the encoder and provides
its input surface together with the output frame width, height, and
orientation as used for encoder configuration.
The FrameProcessorChain converts the output frames to the provided
orientation and resolution using a ScaleToFitTransformation and
Presentation replacing EncoderCompatibilityTransformation.
PiperOrigin-RevId: 455112598
(cherry picked from commit d20f684986)
Based on
https://developer.android.com/reference/android/media/MediaCodec#using-an-output-surface,
frame dropping behaviour depends on the target SDK version.
After this change transformer will only use
MediaFormat#KEY_ALLOW_FRAME_DROP if both the target and system SDK
version are at least 29 and default to its pre 29 behaviour where each
decoder output frame must be processed before a new one is rendered
to prevent frame dropping otherwise.
Also remove deprecated Transformer.Builder constructor without a
context and the context setter.
PiperOrigin-RevId: 453971097
(cherry picked from commit 3f718b0d10)
This removes the prior restriction of needing to remember not to crop and set aspect ratio in the same Presentation.Builder, and makes each class a bit more targeted.
This is partially made feasible by the past work to merge consecutive
MatrixTransformations into a single MatrixTransformationFrameProcessor, which
ensures that there's no loss in quality between successive MatrixTransformations.
PiperOrigin-RevId: 453660582
(cherry picked from commit b33dc5e57b)
Also update names of implementations to match design doc.
In follow-ups, SingleFrameGlTextureProcessor will become
an abstract implementation of a new GlTextureProcessor
interface.
Texture processor makes sense as it processes OpenGL textures.
The term frame processor will be used for something else in
follow-ups.
PiperOrigin-RevId: 451142085
When using a MatrixTransformationFrameProcessor per transformation
matrix, each frame processor's shader applies the matrix to the
vertices and clips the result to the NDC range when drawing the
output frame.
This change combines consecutive MatrixTransformations into a single
MatrixTransformationFrameProcessor by multiplying the individual
matrices while updating and clipping the visible polygon after
each matrix and mapping the resulting visible polygon back to the
input space so that its vertices and the combined transformation
matrix can be used in the shader.
PiperOrigin-RevId: 448521068
ScaleToFitFrameProcessor, PresentationFrameProcessor,
and EncoderCompatibilityFrameProcessor now each implement
MatrixTransformation instead of wrapping
MatrixTransformationFrameProcessor.
PiperOrigin-RevId: 446480286
This change splits AdvancedFrameProcessor into 4 files:
- MatrixTransformationFrameProcessor for the GlFrameProcessor
implementation
- MatrixTransformation and GlMatrixTransformation for the GlEffect
specification
- MatrixUtils for the static matrix helpers
PiperOrigin-RevId: 446236384
Split rotationDegrees changes to EncoderCompatibilityFrameProcessor, a new
FrameProcessor.
This removes automatic rotationDegrees adjustments from Presentation, which
allows Presentation to be used for changes before the end of a
FrameProcessorChain pipeline.
PiperOrigin-RevId: 443387226
This allows apps to use AdvancedFrameProcessor to apply transformations
in 3D space. This functionality is not used in transformer otherwise.
PiperOrigin-RevId: 439313406
The encoder surface is no longer needed for the OpenGL setup and frame
processor initialization, as a placeholder surface is used instead. So
all of the setup can now be done in the factory method.
PiperOrigin-RevId: 438844450
Configuring the frame sizes between frame processors is now the
FrameProcessorChain's rather than the caller's responsibility.
The caller can getOutputSize() and override it for encoder fallback
in configure().
PiperOrigin-RevId: 437048436
The factory method is replaced by a public constructor and
configure() method which configures the input/output surfaces
and handles the OpenGL setup.
This is a prerequisite for removing the responsibility of the
caller to configureSizes() before creating the chain in a follow-up.
PiperOrigin-RevId: 437028882
This allows us to bypass many device-specific issues, that only occur when
decoding directly to an encoder surface, without OpenGL. This also allows us
to maintain fewer code branches, which require additional testing to verify
correctness.
PiperOrigin-RevId: 437003138
Allow apps to modify how frames are presented, via modifying resolution.
A follow-up CL will provide aspect ratio, cropping, etc.
PiperOrigin-RevId: 436963312
The FrameProcessorChain manages a List<GlFrameProcessor>.
FrameProcessorChainDataProcessingTest now tests chaining ScaleToFit-
and AdvancedFrameProcessors.
PiperOrigin-RevId: 436468037
ExternalCopyFrameProcessor's output dimensions match the input
size not the output size. So the intermediate texture size
should match the input size.
Also rename configureOutputDimensions to configureOutputSize.
PiperOrigin-RevId: 435058789
Use android.util.Size, whose naming is much easier to understand than Pair<Integer, Integer>, in both FrameProcessor and EncoderUtil.
PiperOrigin-RevId: 434813986
* Move auto-adjustments for transformation matrices from the
VideoTranscodingSamplePipeline constructor to the new
ScaleToFitFrameProcessor.
* Add GlFrameProcessor#getOutputDimensions() to allow for GlFrameProcessors with
different input and output dimensions. This is a prerequisite for
Presentation.
* Tested with unit tests (and manually just in case).
* A follow up CL will implement change the FrameProcessor input to be scale and
rotate values as requested by the user. This was kept out of this CL to
reduce CL review size. Presentation will also be implemented in a follow up
CL.
PiperOrigin-RevId: 434774854
With the new version, we try the following before fixing resolution:
- Fix size alignment
- Try 3/4 the width and height
- Try 2/3 the width and height
- Try 1/2 the width and height
Also: align the resolution ends in 1 or 9 to 0.
PiperOrigin-RevId: 433206358
Previously, we've used getSupportedHeights/Widths() to find the supported
resolution. However, the height/width can be over-reported when using these
APIs. For example, getSupportedWidths and getSupportedHeights can both return
3840, but the supported height when using 3840 as width is only 2160.
PiperOrigin-RevId: 432926192
