Commit graph

839 commits

Author SHA1 Message Date
claincly
0cf3ab31bf Factor out video decoding and fix two minor issues
1. Not treating 0 as valid buffer index
2. Not handling the case the last frame is a comparison frame

PiperOrigin-RevId: 539607482
(cherry picked from commit 4b1ac2f172)
2023-06-12 11:05:08 +00:00
sheenachhabra
ebfb9cf402 Add support for passing custom metadata via transformer
Changes included:
1. Enable MP4 extractor to read all types of metadata.
2. Allow passing String and Float metadata via Transformer.

Reference to QuickTime spec: https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/Metadata/Metadata.html#//apple_ref/doc/uid/TP40000939-CH1-SW21

PiperOrigin-RevId: 538783982
(cherry picked from commit 53c174f047)
2023-06-12 09:15:53 +00:00
huangdarwin
6dcfe44b89 Test: Add no-op effect test for GL tone mapping.
To ensure no regressions for the potentially confusing pipeline of:
* HDR electrical -> SDR linear EOTF+OOTF, and
* SDR linear -> SDR electrical OETF

PiperOrigin-RevId: 538741079
(cherry picked from commit 0c924fcb40)
2023-06-12 09:15:53 +00:00
tofunmi
5afc256eba DefaultVideoFrameProcessorMultipleTextureOutputPixelTest fixes & cleanup
PiperOrigin-RevId: 538495675
(cherry picked from commit ce203ccfed)
2023-06-12 09:15:52 +00:00
huangdarwin
797ee99baf Test: Move duplicated GL tone mapping logic into helper methods.
PiperOrigin-RevId: 538491957
(cherry picked from commit 9dad207603)
2023-06-12 09:15:52 +00:00
kimvde
71c2d7ba92 Remove unnecessary thread in AssetLoader tests
Also queue textures from a different thread in TextureAssetLoader, to
have a behaviour closer to reality.

PiperOrigin-RevId: 538473089
(cherry picked from commit 3ba8f6dd8f)
2023-06-12 09:15:52 +00:00
huangdarwin
295eb06145 Effect: Add multiple texture output test.
Confirms that multiple textures can be output, and that timestamps and pixels
are as expected.

PiperOrigin-RevId: 538459296
(cherry picked from commit adf53b4d50)
2023-06-12 09:15:52 +00:00
andrewlewis
752a49c8d0 Log additional information on test runner timeout
PiperOrigin-RevId: 538437142
(cherry picked from commit 828a05e0fa)
2023-06-12 09:15:52 +00:00
sheenachhabra
f1d285ed6f Add support for passing creation time via InAppMuxer
PiperOrigin-RevId: 538175466
(cherry picked from commit 7e14811e25)
2023-06-12 09:15:51 +00:00
claincly
70a25b01bf Add utility to create ScheduledExecutorService
PiperOrigin-RevId: 538129792
(cherry picked from commit 08e7158be5)
2023-06-12 09:15:51 +00:00
tofunmi
a8dbea74ce Revert AndroidTestUtil.canDecode to use EncoderUtil.findCodecForFormat
208eefc0fd introduced using `DefaultDecoderFactory.getDecoderInfo(format) != null` caused certain tests not to be skipped when they were expected to be, creating more mh failures.

PiperOrigin-RevId: 537820370
(cherry picked from commit 2af5752785)
2023-06-12 09:13:53 +00:00
andrewlewis
3ef7bc291c Tidy color info checking tests
ExoPlayer extractors (backing `MetadataRetriever`) now parse the color format
from the bitstream so using `MetadataRetriever` should be an equivalent but
more lightweight way to verify the color info.

Also remove try/catch blocks in test code calling into these methods, and add
skipping based on decoder capabilities in the cases where it was missing.

PiperOrigin-RevId: 537789483
(cherry picked from commit 74478f2478)
2023-06-12 09:13:53 +00:00
huangdarwin
4bb00d5e2e Test: Add special effects for forcing transcode.
Previously, we would apply a general effect to signal wanting to transcode.

PiperOrigin-RevId: 537034455
(cherry picked from commit c52130a212)
2023-06-12 09:13:52 +00:00
kimvde
55a2e7ff9e Add TransformationRequest toString method
PiperOrigin-RevId: 536727079
(cherry picked from commit 108000834b)
2023-06-12 09:13:52 +00:00
claincly
ba84de02f6 Fix codec's MIME type is not used
In some cases the codec selected for decoding has a different MIME type than
the media. In thoses cases Transformer continued to use the media's MIME type
and that caused codec configuration failures.

Removed `EncoderUtil.findCodecForFormat()` as we stopped using the method it
uses for finding a codec. Plus, the method is only used in the test.

See also `MediaCodecUtil.getALternativeCodecMimeType()`.

PiperOrigin-RevId: 536683663
(cherry picked from commit 208eefc0fd)
2023-06-12 09:13:52 +00:00
sheenachhabra
6e74b1770c Add only supported MdtaMetadataEntry
PiperOrigin-RevId: 536351494
(cherry picked from commit 99d2cf6713)
2023-06-12 09:13:51 +00:00
tofunmi
75ad65f14c Image transcoding: Add support for bmp image format.
With this change we will now support loading bitmaps from all the formats documented [here](https://developer.android.com/guide/topics/media/media-formats#image-formats) except for gifs (because they are animated). Java doc is added to express this.

PiperOrigin-RevId: 535610152
(cherry picked from commit 94d29f35fc)
2023-06-12 09:13:51 +00:00
kimvde
2ed784f156 Transmux video if rotation is only effect applied
PiperOrigin-RevId: 535554628
(cherry picked from commit f4d1a6c453)
2023-06-12 09:13:51 +00:00
tofunmi
7047f44564 Move OnInputFrameProcessedListener into it's own file
PiperOrigin-RevId: 535183521
(cherry picked from commit a19e07c4d2)
2023-05-26 13:34:52 +00:00
sheenachhabra
c38f094b32 Add support for adding capture FPS via transformer
PiperOrigin-RevId: 534814892
(cherry picked from commit a944ffecb9)
2023-05-26 13:25:09 +00:00
sheenachhabra
fb952145fc Add support for adding XMP data via transformer
PiperOrigin-RevId: 534801202
(cherry picked from commit 71facd825e)
2023-05-26 13:25:09 +00:00
huangdarwin
9e19c60907 Codec: Reduce limit for max decoder pending output frames.
Tentative/experimental value to reduce codec timeouts. We will reconsider using a larger limit after seeing whether this really does reduce error rate.

PiperOrigin-RevId: 534491615
(cherry picked from commit 66554b9b68)
2023-05-26 13:25:09 +00:00
sheenachhabra
d3f88a434b Add support for updating Metadata entries via InAppMuxer
Mp4Muxer already supports writing Mp4LocationData so added that
as supported Metadata entry.
Support for more Metadata entries will be added in upcoming CLs.

PiperOrigin-RevId: 534473866
(cherry picked from commit 7c477589e5)
2023-05-26 13:25:09 +00:00
sheenachhabra
cebdb17134 Write metadata to Mp4Muxer in the release() method
Earlier metadata was written multiple times as it came.
With new changes, all the distinct metadata entries will
get collected and will be written at once in the end.

PiperOrigin-RevId: 534088401
(cherry picked from commit a9e3f5def4)
2023-05-26 13:25:08 +00:00
huangdarwin
9d04b11d49 Effect: Add GlTextureInfo release() and accessor methods.
This allows us to disallow access after release.

PiperOrigin-RevId: 534046475
(cherry picked from commit a6897aedaa)
2023-05-26 13:25:08 +00:00
andrewlewis
8a9503c01a Use Ascii for conversion to lower case
PiperOrigin-RevId: 534016337
(cherry picked from commit b0418f1a2a)
2023-05-26 13:25:07 +00:00
huangdarwin
3ac01d43dd Effect: Use callback to release texture
This allows us to avoid needing a reference to the VideoFrameProcessor, which
can be especially difficult if an App only has a reference to the
VideoFrameProcessor.Factory it passes into Transformer/ExoPlayer.

PiperOrigin-RevId: 533205983
(cherry picked from commit 25fa2df2de)
2023-05-25 10:44:07 +00:00
tofunmi
431b985a04 HDR texture asset loading
PiperOrigin-RevId: 532846248
(cherry picked from commit 5fe10d7652)
2023-05-18 16:07:52 +00:00
sheenachhabra
b97ec5edfc Add default constructor for InAppMuxer.Factory
PiperOrigin-RevId: 532838813
(cherry picked from commit 410840c9e1)
2023-05-18 16:07:52 +00:00
huangdarwin
55c9d10022 HDR: Blocklist Galaxy Z Fold 4 for HLG tone mapping.
This device failed on HdrEditingTest's exportAndTranscode_hlg10File_whenHdrEditingUnsupported_toneMapsOrThrows
before this CL, and succeeds on that test after this CL.

PiperOrigin-RevId: 532796897
(cherry picked from commit 83190a0fe9)
2023-05-18 16:07:52 +00:00
sheenachhabra
7adca46e4b Move Mp4LocationData from extractor module to container module
This class will be shared between extractor and muxer module.

PiperOrigin-RevId: 532784415
(cherry picked from commit 7b62b33127)
2023-05-18 16:07:52 +00:00
sheenachhabra
f5ec1bb6f9 Move NAL unit related methods from common to container module
PiperOrigin-RevId: 532750099
(cherry picked from commit e0d6f67dd9)
2023-05-18 16:07:51 +00:00
tofunmi
43f8d080df Update mixed input test to include images of different aspect ratios
Makes this test a little more thorough.

PiperOrigin-RevId: 532386515
(cherry picked from commit 88642587ac)
2023-05-17 16:59:55 +00:00
huangdarwin
04a00c7ac4 Test: Suppress SSIM timeouts on galaxy fold/flip.
This is a failure only in SSIM, so it seems unlikely we'll prioritize this over
other work or bugs soon. Suppress test failures to reduce triage burden.

PiperOrigin-RevId: 532200729
(cherry picked from commit 62afbe87bb)
2023-05-17 16:59:55 +00:00
huangdarwin
a1fbb12ade Effect: Multiple Texture output
Allow the VideoFrameProcessor to output multiple textures at a time, so that
lifetime of textures is up to the consumer calling VFP.releaseOutputFrame.

The FinalShaderProgramWrapper also has a new maxCapacity limit added, to ensure
the a reasonable amount of textures is used and avoid using up memory.

PiperOrigin-RevId: 532094256
(cherry picked from commit 07ec1eaa48)
2023-05-17 16:59:55 +00:00
kimvde
5e926cdce5 Refactor SequenceAssetLoader release
In the past, the SequenceAssetLoader was released in TransformerInternal
when the export ended.

fc539da061 was made to release the SequenceAssetLoader earlier, when
loading ended. This was causing player release timeouts because the last
AssetLoader in the sequence was released before the SamplePipelines (see
0b40bc37ab for more information).

The code that was releasing the SequenceAssetLoader was first commented
out because we didn't have an understanding of what was happening.

This change removes the early SequenceAssetLoader release all together.
It doesn't have any effect as this code was already commented out.

PiperOrigin-RevId: 532065673
(cherry picked from commit 6dfb387117)
2023-05-17 16:59:55 +00:00
huangdarwin
2ab9c0c556 Transformer: Add latest input format to DebugTraceUtil
This can provide more information to help debug muxer errors

PiperOrigin-RevId: 531521974
(cherry picked from commit 2736b11845)
2023-05-17 16:59:54 +00:00
sheenachhabra
2478b48997 Rollback of 925aa34e13
*** Original commit ***

Rollback of 65d5132f76

*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

***

PiperOrigin-RevId: 531470081
(cherry picked from commit 867355fdc5)
2023-05-17 16:59:54 +00:00
huangdarwin
5313763eb3 Test: Filter test more precisely, with model instead of manufacturer
PiperOrigin-RevId: 531233480
2023-05-12 10:10:15 +01:00
huangdarwin
674885cfa5 Test: Suppress failing 8k24 SSIM bug.
PiperOrigin-RevId: 531193242
2023-05-12 10:07:55 +01:00
kimvde
203450d244 Small fixes in TextureAssetLoader
The texture input tests in TransformerEndToEndTest were not passing on
Pixel 7. Implemented a fix and fixed other minor threading issues I
spotted while looking at the code.

PiperOrigin-RevId: 531141659
2023-05-11 10:07:43 +00:00
kimvde
e8072ca2c7 Move VideoFrameProcessorFactory setter to Transformer
PiperOrigin-RevId: 531123743
2023-05-11 10:06:21 +00:00
huangdarwin
82ede47398 media3: Capitalize "MIME" for MIME types.
Find and replace "mime" to "MIME", where appropriate, throughout media3.

PiperOrigin-RevId: 531122121
2023-05-11 10:05:01 +00:00
Googler
925aa34e13 Rollback of 65d5132f76
*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

PiperOrigin-RevId: 531056436
2023-05-11 10:03:35 +00:00
tofunmi
7fea435b7e Transformer: Support Texture asset loading
PiperOrigin-RevId: 530888319
2023-05-11 10:01:00 +00:00
Googler
ecaf219d74 Adding mime type checking to Transformer to identify images.
PiperOrigin-RevId: 530711523
2023-05-11 09:58:15 +00:00
sheenachhabra
65d5132f76 Create InAppMuxer in transformer
To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

PiperOrigin-RevId: 530684007
2023-05-11 09:56:46 +00:00
kimvde
dc4c6daf0b Fix HDR tone mapping stuck on SM device
Tone-mapping an HDR video with MediaCodec on sm-s908u1 was timing out.
The reason for that is that the decoder was dropping frames, and the
ExternalTextureManager was therefore never propagating the end-of-stream
signal.

There was already a workaround for a similar issue but restricted to
sm-f936b. Removed the model check as the bug is probably present on more
devices.

PiperOrigin-RevId: 530639437
2023-05-11 09:53:25 +00:00
kimvde
1105f194ca Avoid swallowing exceptions when switching AssetLoader
Exceptions thrown when switching AssetLoader were not reported through
the error listener. Therefore, the resources were not released and the
export was not ending.

PiperOrigin-RevId: 530626300
2023-05-11 09:50:43 +00:00
tofunmi
7875131e2a Effect: Support input via texture ID
PiperOrigin-RevId: 530624195
2023-05-11 09:49:26 +00:00