AssetLoader declares the tracks with a setTrackCount() method. Setting
the track count on the MuxerWrapper is easier than calling
registerTrack() as many times as the number of tracks.
PiperOrigin-RevId: 496933501
AssetLoader declares the tracks with a setTrackCount() method. Setting
the track count on the FallbackListener is easier than calling
registerTrack() as many times as the number of tracks.
PiperOrigin-RevId: 496919969
The TransformationRequest passed to FallbackListener (and
createSupportedTransformationRequest) can have null for values that
are inferred from source. Within fallback, this can be height, width,
video mime type and audio mime type (HDR mode is not linked to source).
requestedFormat has these values populated from source, and
supportedFormat then finds the closest supported values to the
requested.
If any of the values in supportedFormat do not match the
requestedFormat, then this method would build upon the
TransformationRequest and update ALL possible fallback fields. This is
a problem because the fallback listener compares the original request
to the fallback one and notifies about all the fields that have
changed.
This CL changes this so that only the values that are not the same as
requested are changed in the supported request that is given to the
fallback listener.
PiperOrigin-RevId: 496908492
When trying to run the test on Android Studio, error "incompatible
types: Buffer cannot be converted to ByteBuffer" is logged. This is
because ByteBuffer.flip() returns a Buffer (and not a ByteBuffer).
Annotation @CovariantReturnType on ByteBuffer.flip() should resolve this
automatically but it doesn't seem supported at the moment.
PiperOrigin-RevId: 496894723
Note that we simply use GlEffectsFrameProcessor in-app / GL tone-mapping, so PQ->SDR tone-mapping isn't yet implemented.
Tested manually using the demo on Pixel 7, to confirm that device and in-app tone
mapping behave similarly.
PiperOrigin-RevId: 496700231
Adds the AudioMixerAlgorithm interface which allows for specialized
implementations of audio mixing that also efficiently convert between
source and mixing formats.
Initial implementation has two algorithms:
1. Float -> float (with channel mixing)
2. S16 -> float (with channel mixing)
PiperOrigin-RevId: 496686805
To do that, rename PROGRESS_STATE_NO_TRANSFORMATION to
PROGRESS_STATE_NOT_STARTED and update Javadoc of ProgressState to not be
Transformer specific.
PiperOrigin-RevId: 496653460
This should allow us to focus on HDR failures instead of network buffering
failures when debugging HDR issues.
These files are each used on several files, so it should be more worth the
test binary impact to move these files to local first.
Locally, tests did take less time after this diff
PiperOrigin-RevId: 496398130
The MediaSourceFactory won't be used by the other AssetLoaders
In order to do that, ExoPlayerAssetLoader has been made public, and the
DefaultAssetLoaderFactory has become a wrapper around
ExoPlayerAssetLoader.
PiperOrigin-RevId: 496386853
Otherwise, the decoders are not captured. It works at the moment for the
video decoder because decoding is still done on the sample pipeline but
it will moved to the AssetLoader soon.
PiperOrigin-RevId: 495275575
`AudioProcessor`s expect direct buffers. This shouldn't make any functional difference in our code, but a custom audio processor might try to access the buffer from JNI in which case a direct byte buffer is more efficient.
PiperOrigin-RevId: 495241669
`SilentAudioGenerator` could output a fractional audio frame, and this could cause downstream components to throw because of trying to read a complete audio frame but only seeing a partial one.
Calculate the output buffer size based on the frame size (which is a no-op for stereo 16-bit audio) and calculate a total number of frames to output then multiple by the frame size.
PiperOrigin-RevId: 494992941
The duration in TransformerInternal.ComponentListener is set on the
Transformer internal thread, and is read on the playback thread. Making
this field volatile ensures that the playback thread reads the updated
value.
PiperOrigin-RevId: 493908385
Rename ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED to
ERROR_CODE_ENCODING_FORMAT_UNSUPPORTED.
This makes the error code more consistent with ERROR_CODE_DECODING_FAILED on the
decoding side. Also, the error code is in the "Encoding errors (4xxx)" section,
so muxer errors probably should be in the "Muxer errors (7xxx)" section instead.
Additionally, no muxer errors currently seem to use
ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED, so this should be a safe change.
PiperOrigin-RevId: 493322880
This is necessary to move video slow motion flattening to the
AssetLoader because this step can change the duration. As we use the
duration before flattening to calculate the progress, we must also use
the position before flattening.
PiperOrigin-RevId: 493291990
Modify FrameProcessor and MatrixTextureProcessor interfaces to support
different input and output color transfers. Does not implement conversion between
color ranges (ex. HDR and SDR), but should allow for conversion between color
transfers of the same color range (ex. HLG and PQ).
This supports in-app tone mapping, where we need a single FrameProcessor to
input HDR color transfers (ex. HLG/PQ) and output SDR (ex. gamma2.2). This also
supports previewing, where we need a single FrameProcessor to be able to input HLG
and output PQ.
Manually tested by confirming colors still look right on SDR and HDR videos
with a rotation and color affect applied.
PiperOrigin-RevId: 493108678
From this CL on, FrameProcessor listeners will be invoked from an Executor that
is passed in when creating the FrameProcessor.
GlTextureProcessor needs to invoke the ErrorListener on the said Executor too.
PiperOrigin-RevId: 493018583
This is necessary to move video decoding to the AssetLoader. Otherwise,
if the decoder max pending frame count is reached, the AssetLoader will
stop queuing frames to the pipeline, and process data will not be called
anymore.
PiperOrigin-RevId: 492392621
EncoderUtil using a static list to cache encoders means it's not possible to
use a different set of encoders for different tests when running all
robolectric tests together.
PiperOrigin-RevId: 491959350
Bypass Z Fold 4 HDR10 tone-mapping bug by limiting the max frame count to 12.
This passed with a value of 14, and failed with a value of 15, but I figured I'd use 12 just to be safe.
PiperOrigin-RevId: 491684058
Main change:
- Removed `Codec.EncoderFactory.createForVideoEncoding`'s argument of a list
of allowed MIME types
- Moved the check for whether a video MIME type is supported to VTSP
PiperOrigin-RevId: 491611799
This thread just starts the player and handles the player callbacks for
now. Sample pipelines are still run on the playback thread.
PiperOrigin-RevId: 491299671
This reduces the priority to best effort (from the default that seems to be best effort), and allows us to run SSIM even on 8k24fps video. Without this CL,
start()'ing a second codec may result in a MediaCodec.CodecException.
Tested to confirm that transformation8k24():
* fails deterministically without this CL, or with KEY_PRIORITY set to 0.
* succeeds deterministically after this CL (~18s on Samsung Z Fold 4)
PiperOrigin-RevId: 490570416
We cannot check this in code, due to DEVICE_INITIAL_SDK_INT being a @SystemApi, and
reflection being a bit risky/unstable.
PiperOrigin-RevId: 490537916
The bug has since been fixed.
The values still could change, as the API is labelled as @UnstableApi, so it's
probably fine to leave the <p> tag mostly as is.
PiperOrigin-RevId: 490509205
This allows us to release both codecs used in SSIM when one fails to
configure() or start().
Tested and confirmed that on Samsung Galaxy Z Flip 4, running all
TransformationTest.java tests, tests after transform8k24() fails to start the
2nd codec:
* Before this CL, all fail.
* After this CL, all pass.
PiperOrigin-RevId: 490461560
This exception is a bit shorter and more clear (and is more clear that this is a
test issue, as opposed to the prior issue that was thrown as an ExoPlayer
DataSourceException, which may seem like a legitimate Transformer failure)
PiperOrigin-RevId: 490252772
Following naming conventions throughout AndroidTestUtil, REMOTE files should have REMOTE instead of ASSET. Update the URI and FORMAT names accordingly.
PiperOrigin-RevId: 490237673
AssetLoader will have multiple implementations and be customizable. We
want to remove the responsibility of computing the progress from this
class and centralize the logic in TransformerInternal.
PiperOrigin-RevId: 488608890
Not setting the color info results in a missing "colr" box in the produced
container, under file/moov/trak/mdia/minf/stbl/stsd/hvc1. This means extractors
will not be able to find out the transcoded file is HDR.
In `Transformer`, this means it can't transcode this transcoded file, because
it currently relies on the container bearing HDR info to construct the
transcoding sample pipeline.
PiperOrigin-RevId: 487276712
In startTransformation method we were throwing UnsupportedEncodingException (IOException) when mediaItem with unsupported arguments is passed.
Changed this to IllegalArgumentException which seems more logical here.
PiperOrigin-RevId: 487259296
This logic is currently in the player renderers. With multi-asset, the
renderers will go into the AssetLoader, which shouldn't be responsible
for muxing.
PiperOrigin-RevId: 486860502
Problem: We are initialising muxer as soon as we start the transformation. Now the startTransformation() method can be called from main thread, but muxer creation is an I/O operation and should be not be done on main thread.
Solution: Added lazy initialisation of muxer object. The actual transformation happens on background thread so the muxer will be initialised lazily from background thread only.
Another way was to provide an initialize() method on MuxerWrapper which will explicitly initialise muxer object but with this approach the caller need to call the initialise method before calling anything else. With current implementation the renderers are calling MuxerWrapper methods on various callbacks (Not sequentially) and also we are sharing same muxer with multiple renderers so It might become confusing for the caller on when to call the initialise() method. Also there are few methods on MuxerWrapper which dont really need muxer object. So in short it might make MuxerWrapper APIs more confusing.
Validation: Verified the transformation from demo app.
PiperOrigin-RevId: 486735787
This should be necessary to ensure decoders see fewer errors.
Setting this resulted in removing native_dequeueOutputBuffer errors on OMX.MTK decoders for in-app tone mapping prototyping.
PiperOrigin-RevId: 486715941
Although it can be useful to check the output format, it's not required or needed.
For some AudioProcessor implementations, it is stated/obvious that
the output format will match the input, in which case there is no
a need to check the return value.
#cleanup
PiperOrigin-RevId: 483403679
The reason for making the Muxer public is that we want to add an option
to disable or configure the timer that will throw when the muxer doesn't
receive any data for a given period of time.
PiperOrigin-RevId: 482199360
- The naming DefaultMuxer is more consistent with the rest of
Transformer codebase (e.g. DefaultEncoderFactory).
- By hiding the implementation details of DefaultMuxer, the transition
to in-app Muxer will be seamless for apps using DefaultMuxer.
- The current plan is that DefaultMuxer will become the in-app muxer.
PiperOrigin-RevId: 481838790
Assert that tone mapping is applied when an HDR edit cannot be HDR, but is successfully tone mapped. Meanwhile, assert that fallback, which is applied after codec configuration (which throws the "Tone-mapping requested but not supported by the decoder" error) is not applied when that error is called.
PiperOrigin-RevId: 478762951
Rename test files to avoid substrings that can be implied by the directory name,
like "Transformation" and "Test"
No functional changes. Renaming-only.
PiperOrigin-RevId: 477724724
* Transform the intermediate color space to linear SDR by applying the SMPTE 170M EOTF and OETF.
* Use linear colors for the color filter pixel tests and update all golden bitmaps.
PiperOrigin-RevId: 476124592
*** Original commit ***
Handle int instead of byte in SSIM.
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.
This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.
Tested manually and SSIM values match for the same video
before and after this change.
***
PiperOrigin-RevId: 473259446
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.
This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.
Tested manually and SSIM values match for the same video
before and after this change.
PiperOrigin-RevId: 473231779
Test that HDR editing succeeds on devices supporting HDR editing, tone maps on
devices supporting tone mapping, and throws exceptions on all other devices.
Also, only restrict HDR editing and tone mapping support to API 31+ only when
transcoding, not for all transformations.
PiperOrigin-RevId: 472958965
shouldPassthrough's internal checks seem to be check whether we should *not*
pass through, which seemed a bit like a confusing double-negative to me.
shouldTranscode is slightly more clear, by instead returning true when we do
want to transcode.
No functional changes intended.
PiperOrigin-RevId: 471753771
3b0d2c1586 made `shouldPassthrough` always return false for `enableHdrVideoEditing`:
>We force using `FrameEditor` (no passthrough) to avoid the need to select another edit operation, and use the new shaders. The `EGLContext` and `EGLSurface` also need to be set up differently for this path.
However, this was introduced before the `videoNeedsEncoding` setting was introduced in 3f615040c0. That setting should apply to HDR videos as much as SDR videos.
PiperOrigin-RevId: 471569853
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.
More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.
PiperOrigin-RevId: 468473072
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.
PiperOrigin-RevId: 466641277
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).
PiperOrigin-RevId: 466425738
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.
Therefore, it's more specific to mention just that the transfer is HDR.
PiperOrigin-RevId: 466316960
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.
No functional change intended in this CL.
PiperOrigin-RevId: 466070764
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.
PiperOrigin-RevId: 465584814
`requestCalculateSsim` more clearly represents the intention of the caller.
Also rephrase the javadoc to simplify it and make it more precise.
PiperOrigin-RevId: 465575578
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.
When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.
Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.
PiperOrigin-RevId: 464135080
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.
PiperOrigin-RevId: 464060047
