Commit graph

457 commits

Author SHA1 Message Date
kimvde
868375ead1 Add muxer timer to detect when generating an output sample is too slow
This allows to throw when the Transformer is stuck or is too slow.

PiperOrigin-RevId: 484179037
(cherry picked from commit 16cb5cbc1f)
2022-10-27 07:58:56 +00:00
samrobinson
35900f94fa Move AudioProcessor to common.
PiperOrigin-RevId: 483699606
(cherry picked from commit fc34542864)
2022-10-25 16:40:38 +00:00
samrobinson
6dd0ace992 Add CanIgnoreReturnValue to AudioProcessor#configure + implementations
Although it can be useful to check the output format, it's not required or needed.

For some AudioProcessor implementations, it is stated/obvious that
the output format will match the input, in which case there is no
a need to check the return value.

#cleanup

PiperOrigin-RevId: 483403679
(cherry picked from commit 8723e74b1f)
2022-10-24 16:27:18 +00:00
kimvde
f10021ff7f Remove muxerFactory dependency on playerListener
This listener will need to be passed to the MuxerWrapper to throw when
the Transformer is stuck.

PiperOrigin-RevId: 482433552
(cherry picked from commit e6d5595545)
2022-10-20 09:36:28 +00:00
kimvde
b0ad5c2681 Make Muxer public
The reason for making the Muxer public is that we want to add an option
to disable or configure the timer that will throw when the muxer doesn't
receive any data for a given period of time.

PiperOrigin-RevId: 482199360
(cherry picked from commit cf14d0687d)
2022-10-19 14:31:06 +00:00
kimvde
b5af908e53 Remove deprecated setOutputMimeType
This is to prepare Muxer to become public

PiperOrigin-RevId: 481893842
(cherry picked from commit 9221b1251c)
2022-10-18 12:26:36 +00:00
samrobinson
130edf8f26 Fix parameter comment block.
#cleanup

PiperOrigin-RevId: 481882181
(cherry picked from commit 7ac7e643ac)
2022-10-18 11:13:44 +00:00
kimvde
34dd66d674 Add DefaultMuxer forwarding to FrameworkMuxer
- The naming DefaultMuxer is more consistent with the rest of
Transformer codebase (e.g. DefaultEncoderFactory).
- By hiding the implementation details of DefaultMuxer, the transition
to in-app Muxer will be seamless for apps using DefaultMuxer.
- The current plan is that DefaultMuxer will become the in-app muxer.

PiperOrigin-RevId: 481838790
(cherry picked from commit 3399f4ecdf)
2022-10-18 07:23:57 +00:00
kimvde
7a488a192b Remove supportsSampleMimeType from Muxer.Factory
- This method is redundant with getSupportedSampleMimeTypes().
- This is to prepare the Muxer class to become public.

PiperOrigin-RevId: 480840902
(cherry picked from commit 2786db9e73)
2022-10-13 09:52:26 +00:00
huangdarwin
355d81c6bb HDR: Disable tone mapping on unsupported pixel build ID.
Also, update tests to allow AnyOf error codes, and no longer check exception messages, which caused quite a bit of churn.

PiperOrigin-RevId: 479570861
(cherry picked from commit 8548e3519e)
2022-10-07 13:57:06 +00:00
huangdarwin
4449ba9c99 HDR: Check isToneMappingApplied in HDR editing test.
Assert that tone mapping is applied when an HDR edit cannot be HDR, but is successfully tone mapped. Meanwhile, assert that fallback, which is applied after codec configuration (which throws the "Tone-mapping requested but not supported by the decoder" error) is not applied when that error is called.

PiperOrigin-RevId: 478762951
(cherry picked from commit 253486d4af)
2022-10-04 13:01:27 +00:00
huangdarwin
186f516b65 HDR: Add tests with HLG10 input.
PiperOrigin-RevId: 478019046
(cherry picked from commit 6e5a4c05b5)
2022-09-30 17:02:37 +00:00
huangdarwin
3fd8e47245 Test: Rename to avoid redundant substrings.
Rename test files to avoid substrings that can be implied by the directory name,
like "Transformation" and "Test"

No functional changes. Renaming-only.

PiperOrigin-RevId: 477724724
(cherry picked from commit 62b95a7190)
2022-09-29 14:54:25 +00:00
huangdarwin
55a7316158 HDR: Add period to end of error message.
For tone mapping error messages.

PiperOrigin-RevId: 477447799
(cherry picked from commit 1711d7f4c2)
2022-09-28 14:30:48 +00:00
huangdarwin
875766ddfe HDR: Add tests for tone mapping.
Also, add checks for output file color.

PiperOrigin-RevId: 477439139
(cherry picked from commit aec4fe7387)
2022-09-28 13:42:36 +00:00
huangdarwin
ab37184917 HDR: Assert output C.ColorTransfer for tests.
To confirm that tone mapping did or did not happen.

PiperOrigin-RevId: 476354606
(cherry picked from commit 87fd51a39b)
2022-09-23 13:33:28 +00:00
leonwind
63f9df5b1a Add OETF and EOTF ES2 fragment shaders for non-HDR frames.
* Transform the intermediate color space to linear SDR by applying the SMPTE 170M EOTF and OETF.
* Use linear colors for the color filter pixel tests and update all golden bitmaps.

PiperOrigin-RevId: 476124592
(cherry picked from commit 3433758c3b)
2022-09-22 16:45:28 +00:00
samrobinson
7ce758f40b Add file size to TransformationResult at the end of a Transformation.
Tested:
  * Manually using both path and file descriptor.
PiperOrigin-RevId: 475860978
(cherry picked from commit 9ec4e1340a)
2022-09-21 17:18:42 +00:00
samrobinson
c7cd3b9c91 Add support for skipping windows in MssimCalculator.
By skipping every other row and column, SSIM calculation time reduces by 10-30%.

PiperOrigin-RevId: 474286702
(cherry picked from commit 314b312af1)
2022-09-14 14:35:12 +00:00
samrobinson
eeba63ab3e Add MssimCalculatorTest to verify SSIM calculations.
As part of this change, MssimCalculator is moved from androidTest/ to main/

PiperOrigin-RevId: 473771344
(cherry picked from commit 8ce42f0670)
2022-09-12 16:45:36 +00:00
samrobinson
8085fe22f5 Rollback of 3d5ddf0c42
*** Original commit ***

Handle int instead of byte in SSIM.

The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

***

PiperOrigin-RevId: 473259446
(cherry picked from commit d188babd2d)
2022-09-09 15:33:08 +00:00
samrobinson
88584c311d Handle int instead of byte in SSIM.
The value of pixels are converted to integers at the point of use,
move this logic to the initialisation step.

This is a prerequisite step for testing SSIM calculation, which
will lead on to some SSIM improvements being verifiable.

Tested manually and SSIM values match for the same video
before and after this change.

PiperOrigin-RevId: 473231779
(cherry picked from commit 3d5ddf0c42)
2022-09-09 13:07:14 +00:00
huangdarwin
60737f66a6 Decoder: Reduce max pending frame count limit down to 5.
PiperOrigin-RevId: 473212543
(cherry picked from commit 4133bb6070)
2022-09-09 10:56:03 +00:00
huangdarwin
0a13578989 HDR: Add fallback MH tests.
Test that HDR editing succeeds on devices supporting HDR editing, tone maps on
devices supporting tone mapping, and throws exceptions on all other devices.

Also, only restrict HDR editing and tone mapping support to API 31+ only when
transcoding, not for all transformations.

PiperOrigin-RevId: 472958965
(cherry picked from commit 0d8fd3d4fc)
2022-09-08 12:03:40 +00:00
claincly
1ded03ce13 Remove conditional check for no-longer flaky API version
PiperOrigin-RevId: 472949850
(cherry picked from commit 8b7638bed3)
2022-09-08 11:02:51 +00:00
claincly
17cc7b59b8 Remove redundant constant definition
PiperOrigin-RevId: 472522846
(cherry picked from commit c401fb9771)
2022-09-06 19:06:17 +00:00
claincly
18188ebe01 Include CamcorderProfile resolution in encoder capability test
PiperOrigin-RevId: 472459423
(cherry picked from commit 3f6a59f0f7)
2022-09-06 15:05:46 +00:00
huangdarwin
c4fa1978b8 Minor javadoc and scoping cleanup.
No functional changes.

PiperOrigin-RevId: 472245797
(cherry picked from commit fa1f09fc51)
2022-09-05 11:06:21 +00:00
huangdarwin
eac4829406 Rename shouldPassthrough to shouldTranscode
shouldPassthrough's internal checks seem to be check whether we should *not*
pass through, which seemed a bit like a confusing double-negative to me.

shouldTranscode is slightly more clear, by instead returning true when we do
want to transcode.

No functional changes intended.

PiperOrigin-RevId: 471753771
(cherry picked from commit 7085c2fa6e)
2022-09-02 09:49:26 +00:00
Googler
15f8e361b6 Skip transcoding if HDR video does not need encoding.
3b0d2c1586 made `shouldPassthrough` always return false for `enableHdrVideoEditing`:

>We force using `FrameEditor` (no passthrough) to avoid the need to select another edit operation, and use the new shaders. The `EGLContext` and `EGLSurface` also need to be set up differently for this path.

However, this was introduced before the `videoNeedsEncoding` setting was introduced in 3f615040c0. That setting should apply to HDR videos as much as SDR videos.

PiperOrigin-RevId: 471569853
(cherry picked from commit bc88f8be54)
2022-09-01 17:37:30 +00:00
ibaker
30e55ad772 Remove media3-only line from exoplayer2 build.gradle file
#minor-release

PiperOrigin-RevId: 470999044
(cherry picked from commit 932f0d2248)
2022-08-30 15:27:12 +00:00
andrewlewis
19fad593e5 Update color info mismatch test
This should now expect transformation to succeed.

PiperOrigin-RevId: 470950411
(cherry picked from commit daf1e5e2eb)
2022-08-30 11:02:46 +00:00
andrewlewis
f3adc5a5d2 Log instead of throwing for transfer mismatch
PiperOrigin-RevId: 470354448
(cherry picked from commit dbe6677551)
2022-08-26 23:08:53 +00:00
Googler
373ac4c7bd Change onOutputFrameAvailable timestamp from nanos to micros.
Upstream timestamps from the decoder are also in microseconds,
so using microseconds here is consistent with that.

PiperOrigin-RevId: 468659099
(cherry picked from commit 0b1c540ff9)
2022-08-19 09:47:28 +00:00
Googler
166838ae70 Allow frame release to be controlled outside FrameProcessor.
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.

More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.

PiperOrigin-RevId: 468473072
(cherry picked from commit a5d7fdcab5)
2022-08-18 16:10:48 +00:00
huangdarwin
da10c96229 HDR: Clarify tone mapping fallback.
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.

PiperOrigin-RevId: 466641277
(cherry picked from commit c5a5364673)
2022-08-10 11:25:56 +00:00
huangdarwin
f83753d4e0 HDR: Update experimental_setEnableHdrEditing javadoc.
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).

PiperOrigin-RevId: 466425738
(cherry picked from commit 05fcda510a)
2022-08-09 18:56:08 +00:00
Googler
ccef3950ca Remove times(1) from effect and transformer tests as it is the default.
PiperOrigin-RevId: 466324322
(cherry picked from commit 1bdc252d2c)
2022-08-09 12:08:39 +00:00
huangdarwin
00749d9d26 HDR: Rename ColorInfo#isHdr to isTransferHdr.
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.

Therefore, it's more specific to mention just that the transfer is HDR.

PiperOrigin-RevId: 466316960
(cherry picked from commit 35161c7489)
2022-08-09 11:28:31 +00:00
huangdarwin
37274c91de HDR: Input ColorInfo to the FrameProcessor.
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.

No functional change intended in this CL.

PiperOrigin-RevId: 466070764
(cherry picked from commit 536d42c865)
2022-08-08 16:55:22 +00:00
andrewlewis
5385cc0422 Remove unused log tag constant
PiperOrigin-RevId: 466025324
(cherry picked from commit 2315a39f52)
2022-08-08 13:20:34 +00:00
andrewlewis
08bbb79c21 Throw if tone-mapping is requested but not enabled
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.

PiperOrigin-RevId: 465584814
(cherry picked from commit 2bd1a93d27)
2022-08-05 16:49:26 +00:00
andrewlewis
99202c5568 Clarify SSIM request method name
`requestCalculateSsim` more clearly represents the intention of the caller.

Also rephrase the javadoc to simplify it and make it more precise.

PiperOrigin-RevId: 465575578
(cherry picked from commit 2eb59fa1af)
2022-08-05 16:02:07 +00:00
andrewlewis
5dd796e14d Fix frame processor reference
PiperOrigin-RevId: 465562260
(cherry picked from commit 16a325217a)
2022-08-05 14:49:23 +00:00
huangdarwin
7eab760b18 HDR: Add comment to clarify lack of support HDR10 for AVC
PiperOrigin-RevId: 465360186
(cherry picked from commit 348662283a)
2022-08-04 18:18:18 +00:00
andrewlewis
f8aaed1ac7 Avoid encoder default color info when tone-mapping
The encoder color info [defaults](https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/foundation/ColorUtils.cpp;l=377;drc=891c19ccfa4953b5e5f7b87118e007b994d8074c) to a value that depends on the input resolution, but when tone-mapping is enabled we should always get BT.709 color space. Hard-code this constant for now to avoid behavior depending on the resolution. A future change should use the decoder output media format to populate the color info.

PiperOrigin-RevId: 465070378
(cherry picked from commit 3c4ee51e10)
2022-08-03 16:06:45 +00:00
hschlueter
8b9203a3ea Move effects functionality out of transformer to effects module.
PiperOrigin-RevId: 465038852
(cherry picked from commit 22725ddfa8)
2022-08-03 13:19:15 +00:00
leonwind
2c4e946f71 Implement RGBA color matrices processor
* Transform frame colors using a defined RGBA Matrix to apply filters.

PiperOrigin-RevId: 464523581
(cherry picked from commit d15f2ed388)
2022-08-01 13:47:21 +00:00
leonwind
1ef01adc74 Migrate Contrast asset to emulator generated one.
PiperOrigin-RevId: 464519030
(cherry picked from commit ae1915ea1a)
2022-08-01 13:14:49 +00:00
huangdarwin
c34e41950b HDR: Have @C.ColorTransfer check consider NO_VALUE as SDR.
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.

When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.

Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.

PiperOrigin-RevId: 464135080
(cherry picked from commit 687a50e9b3)
2022-07-29 19:33:44 +00:00