Commit graph

26 commits

Author SHA1 Message Date
huangdarwin
e7ac678ba4 HDR: Add device checks and mh for GL tone mapping PixelTests.
Add checks to GL tone-mapping pixel tests, to ensure the device's decoder, API
version, and OpenGL implementation support GL tone-mapping before attempting it.

These tests should be run on mobile harness, to detect per-device failures, and
so are moved to transforemr/mh. Per b/263395272, these tests should ultimately
be in an effect/mh directory.

PiperOrigin-RevId: 505749974
2023-02-01 14:59:13 +00:00
huangdarwin
5a0ccdc2ed Test: Rename pixel tests for golden.
producesExpectedOutput -> matchesGoldenFile

PiperOrigin-RevId: 504863604
2023-02-01 13:46:43 +00:00
huangdarwin
237a98e7fb Effect: Move FrameProcessorTestRunner to test-utils.
This is necessary in order to move HDR to SDR tone-mapping tests to transformer/mh.

PiperOrigin-RevId: 504858508
2023-02-01 13:42:30 +00:00
huangdarwin
a2521bff07 Effect: Decouple GlEffectsFrameProcessorTestRunner from GLEffects.
This is necessary in order to move HDR to SDR tone-mapping tests to transformer/mh,
and to move the test runner to androidx.media3.test-utils, which should not have to
include androidx.media3.effects.

PiperOrigin-RevId: 504607929
2023-01-25 20:33:24 +00:00
huangdarwin
5d04d38d35 Test: Rename FrameProcessor Pixel Tests.
Rename tests using the action state expectation pattern.

PiperOrigin-RevId: 504546583
2023-01-25 20:21:27 +00:00
kimvde
a4f9f9487b Add the possility to shift frame timestamps in SampleConsumer
This is needed for constrained multi-asset to shift the timestamps of
the media items that are not the first in the sequence.

PiperOrigin-RevId: 502409923
2023-01-17 03:30:32 +00:00
huangdarwin
e28d434c9e Effect: Automatically save bitmaps in pixel test.
Also, omit the "actual" label from output files, as this boilerplate isn't necessary
(it doesn't disambiguate between any other saved filename like "expected").

PiperOrigin-RevId: 502378188
2023-01-17 03:22:36 +00:00
huangdarwin
27d44e86fe Effect: Create GlEffectsFrameProcessorTestRunner.
Makes GlEffectsFrameProcessorPixelTest slightly more modular in preparation for
copying this into transformer/mh.

Refactoring change. No functional change intended.

PiperOrigin-RevId: 501902223
2023-01-17 03:14:24 +00:00
claincly
42aecce353 Add a frame cache
Frame cache compensates for the fluctuation in frame processing times.

Imagine a frame takes 10ms to process, and the interval between two frames is
33ms. The third frame took 40ms to process.

If we don't have frame cache:
- Process frame 1, ready after 10ms, starts playback, now t=0 ms
- Start processing frame 2, ready at t=10ms,
- Release frame 2 at t=33ms
- We start processing the third frame at t=33ms
- The third frame is due presentation at t=66ms
- But frame 3 is available at t=73ms, late

If we have a frame cache of say 3 frams,
- Process frame 1, ready after 10ms, starts playback, now t=0 ms
- Start processing frame 2, ready at t=10ms
- Start processing frame 3, ready at t=50ms
- Release frame 2 at t=33ms
- Start frame 4, ready at t=60ms
- Frame 3 is due presentation at t=66ms
- Frame 3 isn't late

PiperOrigin-RevId: 501869948
2023-01-17 03:06:36 +00:00
huangdarwin
e773031927 Effect: Move BitmapTestUtil to common test util class.
Move BitmapTestUtil from media3.effect to media3.test.utils.

This allows this class to be accessed from both transformer and effect tests.

Refactoring change. No functional change intended.

PiperOrigin-RevId: 501837130
2023-01-17 02:54:54 +00:00
huangdarwin
4ade37c48d HDR: Implement PQ to SDR tone-mapping.
Tested manually on the Pixel 7 and Samsung S10.

PiperOrigin-RevId: 501626354
2023-01-17 02:46:59 +00:00
andrewlewis
bec46b66eb Reduce flakiness of GL release test
Based on experimentation it seems that buffers can occasionally (roughly 1% of test runs) be dropped when rendering off-screen from EGL on the emulator. Specifically, in this test, sometimes after rendering three buffers with distinct timestamps only the first and third buffers' timestamps are handled in the `ImageReader`'s image available callback causing the assertion checking all frames rendered to fail. This behavior seems to be independent of the nanosecond presentation time attached to the buffers (as expected for off-screen rendering).

Introducing a pause of 1 second between rendering each frame reduces the flake rate to around 1/2000. This increases the run time of some of the tests, so this change also removes the 5 second `FRAME_PROCESSING_WAIT_MS` (it seems to be unnecessary when rendering off-screen) and instead uses a latch to wait until the frame processor has handled 'end of stream'.

PiperOrigin-RevId: 499440591
2023-01-04 19:39:12 +00:00
tofunmi
967e44c066 Add TextureOverlay.configure() to configure overlay before frame processing
This change allows an overlay to be sized with respect to the video dimensions.

PiperOrigin-RevId: 497337734
2023-01-04 19:15:18 +00:00
tofunmi
4d1283f43e Add support for customizing overlay anchor point.
PiperOrigin-RevId: 496956035
2023-01-04 18:33:09 +00:00
tofunmi
ce766d7622 Create dynamically created shaders for multiple overlays.
PiperOrigin-RevId: 496379904
2022-12-21 16:55:44 +00:00
claincly
4a3d693c3e Replace Pair with Size in effects.
PiperOrigin-RevId: 495646341
2022-12-21 15:57:19 +00:00
huangdarwin
b836212b39 HDR: Implement GlEffectsFrameProcessor HLG to SDR tone-mapping.
Adds COLOR_TRANSFER_GAMMA_2_2, to match behavior in other HDR to SDR tone mapping
implementations.

PiperOrigin-RevId: 495371736
2022-12-15 16:59:52 +00:00
tofunmi
05c84a6795 Add support for changing translucency of overlays through alpha values
Implements milestone 1.3 of the [overlays implementation plan](https://docs.google.com/document/d/1EcP2GN8k8N74hHZyD0KTqm9oQo5-W1dZMqIVyqVGtlo/edit#bookmark=id.76uzcie1dg9d)

PiperOrigin-RevId: 493290147
2022-12-12 13:19:58 +00:00
tofunmi
369f9bae43 Implement text and drawable overlays
Implements milestone 1.2 of the [overlays implementation plan](https://docs.google.com/document/d/1EcP2GN8k8N74hHZyD0KTqm9oQo5-W1dZMqIVyqVGtlo/edit#bookmark=id.76uzcie1dg9d)

PiperOrigin-RevId: 493282868
2022-12-12 13:15:47 +00:00
huangdarwin
cf61e219ef HDR: Add effect interface support for different in/out color transfers
Modify FrameProcessor and MatrixTextureProcessor interfaces to support
different input and output color transfers. Does not implement conversion between
color ranges (ex. HDR and SDR), but should allow for conversion between color
transfers of the same color range (ex. HLG and PQ).

This supports in-app tone mapping, where we need a single FrameProcessor to
input HDR color transfers (ex. HLG/PQ) and output SDR (ex. gamma2.2). This also
supports previewing, where we need a single FrameProcessor to be able to input HLG
and output PQ.

Manually tested by confirming colors still look right on SDR and HDR videos
with a rotation and color affect applied.

PiperOrigin-RevId: 493108678
2022-12-12 12:54:38 +00:00
claincly
88a7d67f4a Change argument order to match API council suggestion
By putting Listener as the last argument.

PiperOrigin-RevId: 493100906
2022-12-12 12:50:21 +00:00
claincly
656b346e6c Allow specifying an Executor in FrameProcessor.Factory.create()
From this CL on, FrameProcessor listeners will be invoked from an Executor that
is passed in when creating the FrameProcessor.

GlTextureProcessor needs to invoke the ErrorListener on the said Executor too.

PiperOrigin-RevId: 493018583
2022-12-12 12:41:47 +00:00
tofunmi
3964176421 Add the processor, GLEffect, texture and bitmap overlays
Implements milestone 1.1 of the [overlays implementation plan](https://docs.google.com/document/d/1EcP2GN8k8N74hHZyD0KTqm9oQo5-W1dZMqIVyqVGtlo/edit#bookmark=id.76uzcie1dg9d)

PiperOrigin-RevId: 491696361
2022-12-12 10:45:31 +00:00
samrobinson
52f3ee81d4 Handle buffers in DefaultAudioSink with AudioProcessingPipeline.
PiperOrigin-RevId: 488412695
2022-11-16 12:10:21 +00:00
huangdarwin
9e1adee4cb HDR: Support RGBA_1010102 in GlUtil as a parameter.
Make it easier to support use of RGBA_101012 rather than RGBA_8888 for EGL
contexts, displays, and surfaces.

This tangentially supports adding HDR tests, by slightly simplifying the color
selection logic we'd have to add in HDR tests.

PiperOrigin-RevId: 482219428
2022-10-24 10:35:28 +00:00
bachinger
28c4be2f6b Map lib-effect from androidx to exoplayer2
PiperOrigin-RevId: 482179761
2022-10-20 03:15:38 +00:00