Commit graph

970 commits

Author SHA1 Message Date
sheenachhabra
7c477589e5 Add support for updating Metadata entries via InAppMuxer
Mp4Muxer already supports writing Mp4LocationData so added that
as supported Metadata entry.
Support for more Metadata entries will be added in upcoming CLs.

PiperOrigin-RevId: 534473866
2023-05-24 16:02:48 +01:00
sheenachhabra
a9e3f5def4 Write metadata to Mp4Muxer in the release() method
Earlier metadata was written multiple times as it came.
With new changes, all the distinct metadata entries will
get collected and will be written at once in the end.

PiperOrigin-RevId: 534088401
2023-05-24 15:43:46 +01:00
huangdarwin
a6897aedaa Effect: Add GlTextureInfo release() and accessor methods.
This allows us to disallow access after release.

PiperOrigin-RevId: 534046475
2023-05-24 15:42:47 +01:00
andrewlewis
b0418f1a2a Use Ascii for conversion to lower case
PiperOrigin-RevId: 534016337
2023-05-24 15:34:40 +01:00
ibaker
d5f9cf4f19 Add singleVariant publishing config to all published gradle files
Docs:
* https://developer.android.com/build/publish-library/configure-pub-variants#single-pub-var
* https://developer.android.com/reference/tools/gradle-api/8.0/com/android/build/api/dsl/PublishingOptions

This resolves the following warning from Android Gradle Plugin 7.2.2 and
helps unblock the upgrade to AGP 8.0.1:

> Software Components will not be created automatically for Maven
> publishing from Android Gradle Plugin 8.0. To opt-in to the future
> behavior, set the Gradle property
> `android.disableAutomaticComponentCreation=true` in the
> `gradle.properties` file or use the new publishing DSL.
>
> Affected Modules: `lib-cast`, `lib-common`, `lib-container`,
> `lib-database`, `lib-datasource`, `lib-datasource-cronet`,
> `lib-datasource-okhttp`, `lib-datasource-rtmp`, `lib-decoder`,
> `lib-effect`, `lib-exoplayer`, `lib-exoplayer-all (legacy)`,
> `lib-exoplayer-dash`, `lib-exoplayer-hls`, `lib-exoplayer-ima`,
> `lib-exoplayer-rtsp`, `lib-exoplayer-smoothstreaming`,
> `lib-exoplayer-workmanager`, `lib-extractor`, `lib-media2 (legacy)`,
> `lib-mediasession (legacy)`, `lib-muxer`, `lib-session`,
> `lib-transformer`, `lib-ui`, `lib-ui-leanback`, `test-utils`,
> `test-utils-robolectric`

Issue: androidx/media#409
PiperOrigin-RevId: 533464133
2023-05-19 17:22:21 +01:00
ibaker
b3db85133a Add android.namespace to all build.gradle files
This is a pre-requisite for the Android Studio upgrade assistant to
upgrade from AGP 7.2.2 to 8.0.1, otherwise it fails and complains
this is missing.

Issue: androidx/media#409
PiperOrigin-RevId: 533463246
2023-05-19 17:19:15 +01:00
ibaker
0f8ddd5d1a Suffix package in test AndroidManifest.xml files with .test
This is blocking the upgrade to AGP 8.0.1. The Android Studio Upgrade
Assistant refuses to run with the error:
> Modules have the same package for their `main` and `androidTest`
> artifacts
>
> The package specifications in `AndroidManifest.xml` files define the
> same package for the `main` and `androidTest` artifacts, in the
> following modules: `effect`, `muxer`, `transformer`,
`test_exoplayer_abr`, `test_exoplayer_performance`, and `container`.

This change also adds the `.test` suffix to `test/AndroidManifest.xml`
files where it's missing, because although it's not mentioned in the
Upgrade Assistant error it does cause problems later.

```
> Incorrect package="androidx.media3.database" found in source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
  Setting the namespace via the package attribute in the source AndroidManifest.xml is no longer supported.
  Recommendation: remove package="androidx.media3.database" from the source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
```

Issue: androidx/media#409
PiperOrigin-RevId: 533460180
2023-05-19 17:18:22 +01:00
huangdarwin
25fa2df2de Effect: Use callback to release texture
This allows us to avoid needing a reference to the VideoFrameProcessor, which
can be especially difficult if an App only has a reference to the
VideoFrameProcessor.Factory it passes into Transformer/ExoPlayer.

PiperOrigin-RevId: 533205983
2023-05-19 17:10:53 +01:00
tofunmi
5fe10d7652 HDR texture asset loading
PiperOrigin-RevId: 532846248
2023-05-18 12:21:38 +00:00
sheenachhabra
410840c9e1 Add default constructor for InAppMuxer.Factory
PiperOrigin-RevId: 532838813
2023-05-18 12:18:43 +00:00
huangdarwin
83190a0fe9 HDR: Blocklist Galaxy Z Fold 4 for HLG tone mapping.
This device failed on HdrEditingTest's exportAndTranscode_hlg10File_whenHdrEditingUnsupported_toneMapsOrThrows
before this CL, and succeeds on that test after this CL.

PiperOrigin-RevId: 532796897
2023-05-18 12:15:47 +00:00
sheenachhabra
7b62b33127 Move Mp4LocationData from extractor module to container module
This class will be shared between extractor and muxer module.

PiperOrigin-RevId: 532784415
2023-05-17 16:37:57 +01:00
sheenachhabra
e0d6f67dd9 Move NAL unit related methods from common to container module
PiperOrigin-RevId: 532750099
2023-05-17 16:32:46 +01:00
tofunmi
88642587ac Update mixed input test to include images of different aspect ratios
Makes this test a little more thorough.

PiperOrigin-RevId: 532386515
2023-05-17 08:35:36 +01:00
huangdarwin
62afbe87bb Test: Suppress SSIM timeouts on galaxy fold/flip.
This is a failure only in SSIM, so it seems unlikely we'll prioritize this over
other work or bugs soon. Suppress test failures to reduce triage burden.

PiperOrigin-RevId: 532200729
2023-05-16 09:59:36 +01:00
huangdarwin
07ec1eaa48 Effect: Multiple Texture output
Allow the VideoFrameProcessor to output multiple textures at a time, so that
lifetime of textures is up to the consumer calling VFP.releaseOutputFrame.

The FinalShaderProgramWrapper also has a new maxCapacity limit added, to ensure
the a reasonable amount of textures is used and avoid using up memory.

PiperOrigin-RevId: 532094256
2023-05-16 09:49:52 +01:00
kimvde
6dfb387117 Refactor SequenceAssetLoader release
In the past, the SequenceAssetLoader was released in TransformerInternal
when the export ended.

fc539da061 was made to release the SequenceAssetLoader earlier, when
loading ended. This was causing player release timeouts because the last
AssetLoader in the sequence was released before the SamplePipelines (see
0b40bc37ab for more information).

The code that was releasing the SequenceAssetLoader was first commented
out because we didn't have an understanding of what was happening.

This change removes the early SequenceAssetLoader release all together.
It doesn't have any effect as this code was already commented out.

PiperOrigin-RevId: 532065673
2023-05-16 09:43:36 +01:00
huangdarwin
2736b11845 Transformer: Add latest input format to DebugTraceUtil
This can provide more information to help debug muxer errors

PiperOrigin-RevId: 531521974
2023-05-15 10:39:57 +01:00
sheenachhabra
867355fdc5 Rollback of 925aa34e13
*** Original commit ***

Rollback of 65d5132f76

*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

***

PiperOrigin-RevId: 531470081
2023-05-12 13:20:31 +00:00
huangdarwin
5313763eb3 Test: Filter test more precisely, with model instead of manufacturer
PiperOrigin-RevId: 531233480
2023-05-12 10:10:15 +01:00
huangdarwin
674885cfa5 Test: Suppress failing 8k24 SSIM bug.
PiperOrigin-RevId: 531193242
2023-05-12 10:07:55 +01:00
kimvde
203450d244 Small fixes in TextureAssetLoader
The texture input tests in TransformerEndToEndTest were not passing on
Pixel 7. Implemented a fix and fixed other minor threading issues I
spotted while looking at the code.

PiperOrigin-RevId: 531141659
2023-05-11 10:07:43 +00:00
kimvde
e8072ca2c7 Move VideoFrameProcessorFactory setter to Transformer
PiperOrigin-RevId: 531123743
2023-05-11 10:06:21 +00:00
huangdarwin
82ede47398 media3: Capitalize "MIME" for MIME types.
Find and replace "mime" to "MIME", where appropriate, throughout media3.

PiperOrigin-RevId: 531122121
2023-05-11 10:05:01 +00:00
Googler
925aa34e13 Rollback of 65d5132f76
*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

PiperOrigin-RevId: 531056436
2023-05-11 10:03:35 +00:00
tofunmi
7fea435b7e Transformer: Support Texture asset loading
PiperOrigin-RevId: 530888319
2023-05-11 10:01:00 +00:00
Googler
ecaf219d74 Adding mime type checking to Transformer to identify images.
PiperOrigin-RevId: 530711523
2023-05-11 09:58:15 +00:00
sheenachhabra
65d5132f76 Create InAppMuxer in transformer
To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

PiperOrigin-RevId: 530684007
2023-05-11 09:56:46 +00:00
kimvde
dc4c6daf0b Fix HDR tone mapping stuck on SM device
Tone-mapping an HDR video with MediaCodec on sm-s908u1 was timing out.
The reason for that is that the decoder was dropping frames, and the
ExternalTextureManager was therefore never propagating the end-of-stream
signal.

There was already a workaround for a similar issue but restricted to
sm-f936b. Removed the model check as the bug is probably present on more
devices.

PiperOrigin-RevId: 530639437
2023-05-11 09:53:25 +00:00
kimvde
1105f194ca Avoid swallowing exceptions when switching AssetLoader
Exceptions thrown when switching AssetLoader were not reported through
the error listener. Therefore, the resources were not released and the
export was not ending.

PiperOrigin-RevId: 530626300
2023-05-11 09:50:43 +00:00
tofunmi
7875131e2a Effect: Support input via texture ID
PiperOrigin-RevId: 530624195
2023-05-11 09:49:26 +00:00
huangdarwin
2db2de5993 Transformer: Add codec support for Dolby Vision HDR video
Allow use of H265/H264 codecs for Dolby Vision video.

Also, reflow ExoPlayer code to use this new utility class

PiperOrigin-RevId: 530619388
2023-05-11 09:47:52 +00:00
kimvde
129a6e0cc6 Fix encoder expecting HDR while tone-mapping requested
If tone-mapping was requested for a device supporting HDR encoding,
isToneMapped was false in VideoSamplePipeline.EncoderWrapper. This was
causing the encoder to expect HDR.

Also did some renamings to improve readability

PiperOrigin-RevId: 530584010
2023-05-11 09:45:09 +00:00
tofunmi
b1ebd0e2a6 Change order of operations in videoSamplePipeline
similar error to 9baa6f6be5L `registerInputStream` sets the textureManager, so reordering makes sure the textureManager is set before you set the frame info. This is important for texture input, where the frame info provides the width and height.

PiperOrigin-RevId: 530579584
2023-05-11 09:43:40 +00:00
tofunmi
477994461d Image transcoding: Add support for heic/heif image formats
PiperOrigin-RevId: 530578549
2023-05-11 09:42:12 +00:00
andrewlewis
f0ff57882a Remove stray stuck debug logging line
PiperOrigin-RevId: 530314222
2023-05-11 09:39:17 +00:00
kimvde
30957df635 Remove B-frame setting from DefaultEncoderFactory
This setting is buggy on some devices (for example sm-n960f)

PiperOrigin-RevId: 529712770
2023-05-05 16:54:27 +00:00
claincly
83b5f824f8 Add a tracing utility for debugging video related muxer timeout
PiperOrigin-RevId: 529632886
2023-05-05 16:45:38 +00:00
claincly
213740c456 Enable switching input types via InputMultiplexer.
PiperOrigin-RevId: 529624205
2023-05-05 16:43:58 +00:00
tofunmi
3011a29940 Update InternetConnection check to skip check when uri scheme is null
When we created androidTests, in the past, they always had a URI pointing to a resource, therefore we always had a URI scheme. With texture input, this will not longer be the case (EditedMediaItems's may have URI.EMPTY, which have a null scheme) so we need to check for this so tests don't falsely fail.

PiperOrigin-RevId: 528848411
2023-05-03 17:12:29 +01:00
huangdarwin
24343f55af effect: Rename VFP frame release to render.
renderOutputFrame actually renders frames to an output surface. We'll soon have
a releaseOutputFrame method, that would release resources associated with an
output time, so rename this to disambiguate the two methods.

Also rename onOutputFrameAvailable to onOutputFrameAvailableForRendering, to
make it clear this is not available for "release"

This change should be a renaming-only change and have no functional differences.

PiperOrigin-RevId: 527844947
2023-05-03 17:00:48 +01:00
tofunmi
0902fad55d Effect: split ending frame Processing and getting the output bitmap in tests
PiperOrigin-RevId: 527554408
2023-05-03 16:54:53 +01:00
claincly
12cac0d69f Add a listener once one MediaItem is fully processed
Add `VideoFrameProcessor.registerInputStream()` to signal a new type of input.

And `InputHandler.signalEndOfCurrentInputStream()` to signal to `InputHandler`
partial input stream completion.

Fully processed means after FinalShaderProgramWrapper releases the last frame.

PiperOrigin-RevId: 527356646
2023-04-27 12:28:40 +01:00
huangdarwin
3db4205ac6 Test: Add @Ignore to Analysis tests
In ASwB, all Transformer tests can be run by right-clicking on the project and
clicking "Run all tests". We cannot, however, select only some specific test
files within a project (ex. all non-analysis tests) to run tests on.

Add @Ignore to analysis tests, which are not intended to be run anyways
when determining whether Transformer is working on some device. These
tests also don't have proper skipping logic when a device doesn't support a
format, so they can't be run effectively on low-end devices anyways.

This eases manual testing, for example when debugging whether tests all pass
on a device.

When analysis tests are desired to be run, it should be easy to comment out
the @Ignore.

PiperOrigin-RevId: 527289600
2023-04-27 12:25:42 +01:00
huangdarwin
97272c139c Effects: Output to texture without surface in VFP.
Allow the VideoFrameProcessor to output to a texture without an output surface.

Tested by updating texture output tests to no longer output to a surface.

PiperOrigin-RevId: 527244605
2023-04-26 15:51:06 +01:00
tonihei
889f435a49 Update dependencies to latest versions
The only dependencies that are not updated are the ones that need
to be kept in sync with other system (like Android source tree) or
would require a Kotlin dependency in common or exoplayer modules.

As a side effect, some demo apps now need a Kotlin config and some
additional modules require desugaring/multidex logic. To simplify
the setup, the desugaring and multidex steps are added to the common
config.

PiperOrigin-RevId: 527243950
2023-04-26 15:49:57 +01:00
huangdarwin
d8cf4449a4 Transformer: Reduce scope of createSupportedTransformationRequest
This was only used in EncoderWrapper, so move createSupportedTransformationRequest
from VideoSamplePipeline to EncoderWrapper to reduce its scope.

PiperOrigin-RevId: 527226578
2023-04-26 15:48:50 +01:00
tofunmi
8612d2820d Update image transcoding pipeline to signal input COLOR_TRANSFER_SRGB
The production code changes are in transformer, but the tests in effect have also been updated to confirm the is no color regression `inputColorInfo.colorTransfer=C.COLOR_TRANSFER_SRGB`

PiperOrigin-RevId: 526950435
2023-04-26 15:46:21 +01:00
tofunmi
42c9c28daf Update VideoSamplePipeline to take account of decoder colors
PiperOrigin-RevId: 526940261
2023-04-26 15:44:52 +01:00
huangdarwin
324115f6cf Effects: Allow BT2020 colors with SDR transfers
Previously, I assumed that SDR contents must be BT709, and HDR contents must be
BT2020. Turns out BT2020 is just wide-gamut, and SDR contents / transfers may be
represented in BT2020 color spaces.

Relax the check, so that we don't throw when valid BT2020 SMPTE 170M contents
are input into effects.

PiperOrigin-RevId: 526668347
2023-04-26 15:41:33 +01:00
sheenachhabra
b11dd106ae Replace MediaFormat with Format class in muxer module
PiperOrigin-RevId: 526655859
2023-04-26 15:40:40 +01:00
tofunmi
f1d9a0a819 Add nullness-compat-qual dependency for android tests
PiperOrigin-RevId: 526652129
2023-04-26 15:38:22 +01:00
ibaker
2d968a5d97 Rollback of 8d17faea33
*** Original commit ***

Rollback of d7983f9485

*** Original commit ***

Bump Guava version to 31.1

***

***

PiperOrigin-RevId: 526601244
2023-04-26 15:36:00 +01:00
claincly
fcec5a29c1 Fix not ignoring invalid color info.
PiperOrigin-RevId: 526595567
2023-04-26 15:34:52 +01:00
huangdarwin
de32536e20 HDR: Remove exportUnexpectedColorInfo test.
PiperOrigin-RevId: 526086973
2023-04-24 11:39:09 +01:00
tofunmi
0f8fa232e7 Change isInputTextureExternal boolean parameter to inputType intDef
PiperOrigin-RevId: 525690361
2023-04-24 11:21:41 +01:00
huangdarwin
3bc0172188 Remove extra param from createForVideoFrameProcessingException
This arg was always the same value, so it can be simplified.

PiperOrigin-RevId: 525677735
2023-04-24 11:20:32 +01:00
tofunmi
316a2ce0ee Remove deprecated constructor in DefaultVideoFrameProcessor.Factory
Remove deprecated factory constructor and replace all call instances of it.

PiperOrigin-RevId: 525476062
2023-04-24 11:19:49 +01:00
tofunmi
9c3d30791a Move glObjectsProvider to DefaultVideoFrameProcessor.Factory.Builder
Removes glObjectsProvider from `VideoFrameProcessor` and `Effects`. Apps will set the glObjectsProvider on the DefaultVideoFrameProcessor.Factory.Builder when providing a custom DefaultVideoFrameProcessor.Factory, rather than in `Effects`.

PiperOrigin-RevId: 525169059
2023-04-18 18:12:14 +01:00
andrewlewis
990f7ef5b5 Select codecs without util methods
`findDecoder/EncoderForFormat` seem to be more restrictive than querying
`MediaCodecList` as done for playback, and it's not clear that the cases where
no codec is found correspond to actual cases that would fail given that it
seems this returns an empty string for many cases in production.

Switch to using `MediaCodecUtil` and `EncoderUtil` for querying codecs instead.

PiperOrigin-RevId: 525078321
2023-04-18 18:08:18 +01:00
kimvde
23b1cab56c Fix video timestamp 0 set to large value in some cases
PiperOrigin-RevId: 525064269
2023-04-18 18:07:19 +01:00
Googler
ffa969f97c Expose MssimCaluclator for use in Duetdev tests
Child: <unknown commit>

PiperOrigin-RevId: 524876651
2023-04-18 18:06:01 +01:00
huangdarwin
0e85491d4e Transformer: Update TODOs to use new bug link
PiperOrigin-RevId: 524349341
2023-04-17 17:12:28 +01:00
kimvde
9c5fca8e65 Fix muxer failing in test
MediaMuxer does not support out-of-order video frames API 25. Use
a test file with frames in order in
loopingTranscodedAudio_producesExpectedResult. This is necessary for
tests that transmux video samples.

PiperOrigin-RevId: 524309318
2023-04-17 17:10:03 +01:00
huangdarwin
9835e58e4c HDR: Add comment describing HDR_MODE_KEEP_HDR behavior.
PiperOrigin-RevId: 524301315
2023-04-17 17:07:53 +01:00
huangdarwin
e0314f3041 HDR: Add HDR passthrough effect pixel tests.
A passthrough effect allows for testing having an intermediate
effect injected, which uses different OpenGL shaders from having no
effects.

PiperOrigin-RevId: 524276991
2023-04-17 17:06:05 +01:00
claincly
cb28960c07 Set encoded level in DefaultDecoderFactory.
PiperOrigin-RevId: 524248369
2023-04-17 17:05:03 +01:00
andrewlewis
affc237055 Channel mix to 16-bit int not float
Previously `ChannelMixingAudioProcessor` output float because it was
implemented using the audio mixer's float mixing support.

Move the implementation over to just using the `ChannelMixingMatrix` and make
it publicly visible in the common module so it can be used by apps for both
playback and export.

Also resolve a TODO that no longer had a bug attached by implementing support
for putting multiple mixing matrices to handle different input audio channel
counts, and fix some nits in the test code.

Tested via unit tests and manually configuring a `ChannelMixingAudioProcessor`
in the transformer demo app and playing an audio stream that identifies
channels, and verifying that they are remapped as expected.

PiperOrigin-RevId: 523653901
2023-04-12 16:58:43 +01:00
huangdarwin
acdb7ee921 HDR: Add HDR pixel tests.
Implement HDR input support for texture output, and add HDR pixel tests.

PiperOrigin-RevId: 523417701
2023-04-12 16:51:51 +01:00
andrewlewis
d43fe3470f Fix audio encode timestamp off by one
Simplify the audio encoder input timestamp calculation. The new calculation
avoids drifting by tracking the total number of bytes encoded rather than
tracking the timestamp and remainder separately, and also makes the timestamps
match the decoder output buffer timestamps.

Also switch one of the export tests that was passing through AMR samples over
to using WAVE audio. The problem with using AMR is that the compressed samples
are not necessarily an integer number of audio frames and the shadow decoder
would pass them from input to output, so the audio encoder was receiving
non-integer numbers of audio frames.

Tested by logging the timestamps at the decoder output and encoder input with
forcing transcoding audio, and verifying that after this change the audio
timestamps are no longer off by one.

PiperOrigin-RevId: 523409869
2023-04-12 16:50:25 +01:00
andrewlewis
0b40bc37ab Release sample pipelines before asset loaders
The video asset loader renders decoder output to a surface texture, and if the
video sample pipeline is in the process of updating the surface texture image
at the moment when the asset loader video decoder is released this seems to
cause `MediaCodec.release` to get stuck.

Swap the release order so that we stop updating the texture before trying to
release the codec.

PiperOrigin-RevId: 523401619
2023-04-12 16:49:30 +01:00
andrewlewis
ba2c32738f Improve error logging
Log at debug level immediately when MediaCodec throws. This logging will be
output closer to the time when the error actually happened so should make it
easier to identify the order of components failing.

Downgrade logging of errors after export ends to warning level, as output may
still be fine if there was a problem after exporting completed (though it's
still worth logging a warning as the device may not be in a good state).

PiperOrigin-RevId: 523370457
2023-04-12 16:48:33 +01:00
huangdarwin
0d30edae75 Test: Add TextureOutputListener for texture output tests
Before this CL, SurfaceTexture.onFrameAvailable was used to tell whether a frame
was available in the VideoFrameProcessor's output texture. This was incorrect, as
it would rely on having the texture be written to before the
SurfaceTexture.onFrameAvailableListener is invoked, leading to null-pointer-
exceptions on timeouts.

Instead of using DefaultVideoFrameProcessor different interfaces to set that we
want to output to a texture, and get that output texture, use one interface that
sets a listener, and renders to a texture iff that listener is set. As this
listener is executed on the GL thread, this also allows us to no longer need to
expand visibility for the GL task executor and tasks.

PiperOrigin-RevId: 522362101
2023-04-11 10:34:38 +01:00
samrobinson
c3f25651bd Use AudioFormat in SilentAudioGenerator.
PiperOrigin-RevId: 521790733
2023-04-05 15:47:24 +01:00
sheenachhabra
28aa5e847b Move NAL unit utility classes into common
Need to use NAL unit util methods in muxer module.

PiperOrigin-RevId: 521772831
2023-04-05 15:45:35 +01:00
huangdarwin
167677ec84 HDR: Fix exportUnexpectedColorInfo.
PiperOrigin-RevId: 521427239
2023-04-05 15:41:37 +01:00
kimvde
0bfe43866a Add test for clipped media items concatenation
This was broken and has been fixed in <unknown commit>.

PiperOrigin-RevId: 521380415
2023-04-05 15:40:09 +01:00
kimvde
3a5c4277a7 Fix and simplify timestamps handling
Handling of the stream offset and start position was unnecessarily
complex and even incorrect. It was going to be an issue for
concatenation of video and image input.

The stream offset is the offset added before decoding/encoding to
make sure it doesn’t fail in case of negative timestamps (which do
rarely occur).
The start position is equal to the stream offset, plus the clipping
start time if the media is clipped.

Before this change:
- Samples were offset by the start position before decoding, and this
offset was removed before muxing.
- The startPosition of the first MediaItem in a sequence was used for
all the MediaItems in this sequence (which is incorrect).
- The stream offset was removed before applying the GL effects and
added back before encoding so that it was not visible to the OpenGL
processing.

After this change:
- The start position is subtracted in the AssetLoader, so that the
downstream components don’t have to deal with the stream offsets and
start positions.
- Decoded samples with negative timestamps are not passed to the
SamplePipelines. The MediaMuxer doesn’t handle negative timestamps
well. If a stream is 10 secondes long and starts at timestamp -2
seconds, the output will only contain the samples corresponding to the
first 8 (10 - 2) seconds. It won’t contain the last 2 seconds of the
stream. It seems acceptable to remove the first 2 seconds instead.

PiperOrigin-RevId: 520916464
2023-04-05 15:38:27 +01:00
kimvde
9cdf60d791 Fix emulator tests flakiness
Releasing the player once a sequence has ended seems to make our
emulator tests flaky. Comment out until we find the cause. The player
will still be released from TransformerInternal, when the export ends.

PiperOrigin-RevId: 520886181
2023-04-05 15:36:52 +01:00
kimvde
104cfc322c Clarify and fix SequenceAssetLoader threading
PiperOrigin-RevId: 520663415
2023-03-30 17:28:46 +00:00
rohks
766e7d7d35 Add media3 guide entries in README files
PiperOrigin-RevId: 520650881
2023-03-30 17:28:03 +00:00
rohks
6a928805d4 Fix javadoc links in media README files
Also fixed the javadoc link in devsite and removed javadoc links from decoder extensions as it is not published yet on developer.android.com.

#minor-release

PiperOrigin-RevId: 520636868
2023-03-30 17:26:40 +00:00
kimvde
1dbb19a44c Rollback of 7e6236cfdb
*** Original commit ***

Rollback of fc539da061

*** Original commit ***

Release SequenceAssetLoader when ended

Before, the SequenceAssetLoader was released at the end of the export.
Release resources earlier if possible.

***

***

PiperOrigin-RevId: 520036433
2023-03-30 17:18:30 +00:00
kimvde
24a164827e Fix handling of non-empty decoder end-of-stream output buffer
PiperOrigin-RevId: 520027236
2023-03-30 17:17:48 +00:00
kimvde
7e6236cfdb Rollback of fc539da061
*** Original commit ***

Release SequenceAssetLoader when ended

Before, the SequenceAssetLoader was released at the end of the export.
Release resources earlier if possible.

***

PiperOrigin-RevId: 519993003
2023-03-30 17:17:06 +00:00
kimvde
49d5955ce1 Add error code for muxer timeout
PiperOrigin-RevId: 519965132
2023-03-30 17:15:41 +00:00
kimvde
fc539da061 Release SequenceAssetLoader when ended
Before, the SequenceAssetLoader was released at the end of the export.
Release resources earlier if possible.

PiperOrigin-RevId: 519952572
2023-03-30 17:14:58 +00:00
huangdarwin
a0838771d3 HDR: Implement DefaultVideoFrameProcessor texture output for tests.
Previously, we always used ImageReader to read from the output of DefaultVideoFrameProcessor, for pixel tests. This has a limitation of not being
able to read HDR contents, so that we couldn't support HDR pixel tests.

Reading from a texture allows us to use glReadPixels to read from
DefaultVideoFrameProcessor, and build upon this to implement HDR pixel tests. We do
still want tests for surface output though, because real use-cases only will output
to Surfaces.

Also, add some tests for outputting to textures, since this test infrastructure is
a bit complex.

PiperOrigin-RevId: 519786535
2023-03-30 17:14:15 +00:00
kimvde
710b462b5f Update documentation of supported Compositions
Account for features implemented recently.

PiperOrigin-RevId: 519113898
2023-03-30 17:08:46 +00:00
kimvde
b118565730 Add an API entry point for looping a sequence
Also
- Add unit tests
- Fix bug discovered by unit tests

PiperOrigin-RevId: 519092249
2023-03-30 17:07:27 +00:00
kimvde
1f6b000219 Handle looping sequences in TransformerInternal
PiperOrigin-RevId: 518895943
2023-03-30 17:06:01 +00:00
sheenachhabra
69cece1d82 Make getMp4LocationData method inline
PiperOrigin-RevId: 518827223
2023-03-30 17:04:46 +00:00
tofunmi
8df4fd769c Log export errors that occur after export completes
PiperOrigin-RevId: 518281002
2023-03-22 10:35:05 +00:00
tofunmi
3a6ee77daf skip transformer tests if no internet connection
The failure reason (no network) will be logged in the test output, but the test will not be marked as failing, rather skipped.

PiperOrigin-RevId: 518262203
2023-03-21 14:06:11 +00:00
huangdarwin
fca9197e1c HDR: Only check for HDR encoders on API 33+.
FEATURE_HdrEditing is only available on and above API 33, so it doesn't make
sense to check for support under API 33.

PiperOrigin-RevId: 518036855
2023-03-21 14:20:01 +00:00
kimvde
aed5d62602 Remove TODOs
PiperOrigin-RevId: 517444208
2023-03-21 14:17:04 +00:00
kimvde
d41d025803 Add support for looping sequences in SequenceAssetLoader
PiperOrigin-RevId: 517418886
2023-03-21 14:16:19 +00:00
huangdarwin
e54c7a7c79 Test: Trim 4k60 test file to 300ms.
This was trimmed using the Transformer Demo, from ~4 seconds to 300ms

Otherwise, this test file will be quite large, and folks who clone the repo will
have to clone this content as well. Trimming the file also allows the test to
take less time, while still testing the core purpose of the file, to confirm that 4k extraction/decode/GL/encode/muxing works.

PiperOrigin-RevId: 517408463
2023-03-21 14:15:35 +00:00
samrobinson
78373991b8 Fix nits in FloatAudioMixingAlgorithmTest.
* Reorder methods.
* Arrange, act, assert split.

PiperOrigin-RevId: 517158351
2023-03-21 14:14:47 +00:00
samrobinson
9e66998ea4 Add test cases for mixing to mono.
PiperOrigin-RevId: 517127021
2023-03-16 15:36:09 +00:00
huangdarwin
a6479c3d10 Test: Move 4k60 test file to local from remote.
This test file used to be remote, which led to various errors in loading the file or timing out during the muxer or overall transformer.

export4k60 test timing on Pixel 7:

|Condition                 |elapsedTimeMs|AS Test "Duration" (s)|
|--------------------------|-------------|----------------------|
|local file (this CL)      |4253         |14                    |
|Remote file, googleguest  |5510         |17                    |
|Remote file, 4G           |11423        |25                    |

PiperOrigin-RevId: 516815462
2023-03-16 15:31:56 +00:00
kimvde
1ad68ffb17 Add a way in the SampleConsumer to indicate that it couldn't consume
- To support looping EditedMediaItemSequences, we need a way to tell the
AssetLoader that a sample couldn't be consumed and that it should retry
later. This is necessary in case we don't know yet whether the looping
sequence should load more samples because the other sequences haven't
made sufficient progress yet.
- The decision on whether to consume a sample is based on its timestamp
so it needs to be available.

PiperOrigin-RevId: 516546026
2023-03-16 15:29:26 +00:00
samrobinson
eabf952048 Move RepeatedTranscodeTest to mh/analysis.
This test is not run on emulator, triage or postsubmit, and does not
provide value to the library in it's current state. It could be
deleted, however it still may have use for analysis of exported files
on MH.

PiperOrigin-RevId: 516491160
2023-03-14 15:54:56 +00:00
sheenachhabra
7b8c562d7b Remove setLocation() method from Muxer interface
PiperOrigin-RevId: 516314175
2023-03-14 07:56:52 +00:00
huangdarwin
f0d91b30d5 Effect: Implement TimestampWrapper.
To allow applying an effect only on a range of timestamps.

PiperOrigin-RevId: 515615662
2023-03-14 07:52:12 +00:00
samrobinson
a3e93f74cb Remove PCM encoding from ExportResult.
PCM encoding is a property of decoded audio, however Transformer does
not output decoded audio.

PiperOrigin-RevId: 515580886
2023-03-14 07:51:26 +00:00
kimvde
1865e38108 Add test for audio and video from different sources
PiperOrigin-RevId: 515379858
2023-03-14 07:50:18 +00:00
sheenachhabra
1c9c384aaa Add support for writing MP4 location data using media muxer
PiperOrigin-RevId: 515297752
2023-03-14 07:48:20 +00:00
kimvde
6d623bfad7 Split TransformerEndToEndTest
Split test/ TransformerEndToEndTest into SingleMediaItemEndToEndTest and
SingleSequenceEndToEndTest to reduce the file size and split the tests
by category.

PiperOrigin-RevId: 515039502
2023-03-14 07:42:55 +00:00
kimvde
bf60302cf2 Make sure errors thrown in the PlayerListener are not swallowed
PiperOrigin-RevId: 515037497
2023-03-14 07:42:10 +00:00
kimvde
d152175e27 Remove VideoSamplePipeline unused param
PiperOrigin-RevId: 515002313
2023-03-14 07:38:10 +00:00
samrobinson
320d9a7ed2 Verify the audio properties being used within Transformer.
Add TransformerAudioEndToEndTest to emulated tests.

PiperOrigin-RevId: 514991434
2023-03-14 07:37:26 +00:00
kimvde
67d15ec575 Fix threading issueis in SequenceAssetLoaderListener
Callbacks onTrackCount and onTrackAdded can be called simultaneously
from different threads.

Before this fix, it was possible for the MuxerWrapper and
FallbackListener track count to never be set, or to be set
with incorrect values.

PiperOrigin-RevId: 514779719
2023-03-14 07:35:25 +00:00
tofunmi
e478d81b52 Update effect to take in and use a GlObjectsProvider
PiperOrigin-RevId: 514744747
2023-03-14 07:33:25 +00:00
samrobinson
f07e6b63ea Fix AudioMixer javadoc.
PiperOrigin-RevId: 514699994
2023-03-07 15:50:33 +00:00
samrobinson
3264cb8271 Pass Metadata to Muxer when adding a track.
PiperOrigin-RevId: 514575400
2023-03-07 11:56:53 +00:00
kimvde
5115df1147 Rename CompositeAssetLoader to SequenceAssetLoader
This clarifies that a SequenceAssetLoader loads data corresponding to an
EditedMediaItemSequence.

PiperOrigin-RevId: 514442681
2023-03-07 11:55:22 +00:00
tofunmi
fe38901028 ImageAssetLoaderTest minor fixes
PiperOrigin-RevId: 514436359
2023-03-07 11:54:36 +00:00
tofunmi
f4b88cd8a2 Add ImageAssetLoader tests.
PiperOrigin-RevId: 514431184
2023-03-07 11:53:36 +00:00
kimvde
0f8b67b875 Make onOutputFormat nullable
- This is to make sure we know about all the tracks before initializing
the SamplePipelines. This allows to set the muxer and the fallback
listener track count before the SamplePipelines are built.
- As a result, the test files had to be updated because the order in
which the tracks are written has changed.
- The ImageAssetLoader also had to be updated to call onOutputFormat
repeatedly until it returns a non-null SampleConsumer.
- Also fix the trackCount sent to the muxer and fallback listener. The
correct track count can be computed now that we know about all the
tracks before building the SamplePipelines.

PiperOrigin-RevId: 514426123
2023-03-07 11:52:52 +00:00
tofunmi
efaf4e3f33 Remove effect/SimpleBitmapLoader & replace with DataSourceBitmapLoader
PiperOrigin-RevId: 513824487
2023-03-07 11:46:58 +00:00
kimvde
71fb4f9a5c Make sure that tracks are transcoded/transmuxed consistently
All audio tracks should either all be transcoded or all be transmuxed.
Same for video tracks.

To achieve this, simplify the behaviour of transmuxAudio/Video.

PiperOrigin-RevId: 513809287
2023-03-07 11:46:07 +00:00
samrobinson
07ba24ab1a Provide TransformerInternal with the AssetLoader output format.
TI can use this Format as part of creating the SamplePipelines.

PiperOrigin-RevId: 513777622
2023-03-07 11:45:16 +00:00
andrewlewis
82119bbbc2 Fix typo
PiperOrigin-RevId: 513529059
2023-03-02 17:20:30 +00:00
samrobinson
ae4471debf Add getNormalizedTrackType, mapping image to video track type.
PiperOrigin-RevId: 513514142
2023-03-02 17:18:39 +00:00
samrobinson
65afd40622 Only init SpeedChangingAudioProcessor if Format.Metadata not null.
If the Metadata passed to SegmentSpeedProvider is null, then the
SegmentSpeedProvider will always return 1f from getSpeed.

Initializing a SpeedChangingAudioProcessor requires a SpeedProvider.
Once configured,this audioProcessor is always active, so buffers are
passed through it. Because getSpeed is always 1, the processor performs
a no-op, but still has to do a buffer copy for each buffer.

By not initializing the audio processor when metadata is null, this
copy can be skipped and the audio pipeline is more performant.

Note: This change does not affect the multiple media-item case, which
is not supported with speed changes, as per Transformer API
documentation.
PiperOrigin-RevId: 513261811
2023-03-02 13:21:41 +00:00
ibaker
c07cf3dc41 Remove @see tags with <a> tags
These are not supported by Dackka

#minor-release

PiperOrigin-RevId: 513176533
2023-03-01 17:25:48 +00:00
andrewlewis
cccc582595 Don't log removed encoders
We shouldn't have this logging unless we really need it to debug
a specific problem, as it can be noisy (even at debug level).

PiperOrigin-RevId: 512904412
2023-03-01 17:25:03 +00:00
kimvde
d2d5174f09 Handle multiple sequences in a Composition
This lays the groundwork for full multi-asset, and more particularly for
adding looping background audio.

PiperOrigin-RevId: 512887888
2023-03-01 17:22:10 +00:00
claincly
c7b4ec4d65 Allow video format change.
Uses the first mediaItem's format as the output format.

If there is `Presentation` supplied in the `Composition.effects`, add it as the
last effect of the first EditedMediaItem.

PiperOrigin-RevId: 512082659
2023-02-27 18:45:32 +00:00
kimvde
0afe5923d8 Clarify some threading requirements.
PiperOrigin-RevId: 512079471
2023-02-27 18:44:26 +00:00
kimvde
0134d7b048 Update experimentalSetForceAudioTrack Javadoc to throw
This is consistent with the code behaviour.

PiperOrigin-RevId: 512073465
2023-02-27 18:43:38 +00:00
kimvde
dd8b64787a Small fixes in CompositeAssetLoader
PiperOrigin-RevId: 512038052
2023-02-27 18:39:57 +00:00
samrobinson
4b09a29082 Fix broken test due to file not found.
PiperOrigin-RevId: 512012099
2023-02-27 18:37:42 +00:00
kimvde
c2fd7339e1 Throw when the video track disappears during export
Throw when the output has a video track but the current MediaItem in
the sequence doesn't have any video.

PiperOrigin-RevId: 512004463
2023-02-27 18:36:51 +00:00
samrobinson
a6932f9bdd Propagate PCM encoding passed to AudioSamplePipeline.
PiperOrigin-RevId: 511810757
2023-02-27 18:33:48 +00:00
huangdarwin
cc1ca860b1 Test: Add format codec info and update HDR10 file to 720p.
Add format codec info, which can make test skipping checks more similar to the
actual Transformer decoder checks.

Also for the test file, the actual format was 720p, but somehow the file name and
media metadata indicated 1080p. This format mismatch led to some decoding errors,
so fix the format (and associated errors). This also allows us to remove the
exception catch in ForceInterpretHdrVideoAsSdrTest, which was included due to
errors from the incorrect format.

PiperOrigin-RevId: 511809507
2023-02-27 18:32:55 +00:00
kimvde
ecf168b359 Clarify slow mo flattening Javadoc
PiperOrigin-RevId: 511778088
2023-02-27 18:31:01 +00:00
kimvde
dd5dd76595 Improve shouldTranscode() logic
PiperOrigin-RevId: 511767953
2023-02-27 18:29:20 +00:00
kimvde
5c54a7dffb Add unit tests for audio track dis(appearing) during export
PiperOrigin-RevId: 511764841
2023-02-27 18:28:14 +00:00
kimvde
01e35ad13b Document that slow mo flattening is only supported for single-asset
PiperOrigin-RevId: 511757712
2023-02-27 18:27:17 +00:00
kimvde
31c44c7061 Split transmux into transmuxAudio/Video
- Split the transmux setting into transmuxAudio and transmuxVideo. This
  is more flexible for apps and will also be useful for unit testing
  (particularly as we can't test video transcoding on Robolectric at the
  moment).
- Move these settings to Composition. It makes sense for these settings
  to be next to forceAudioTrack. Apps may also want to set these
  settings based on the current Composition's MediaItems.
- Add a Composition.Builder because Composition now contains a few
  optional fields.

PiperOrigin-RevId: 511708618
2023-02-27 18:26:30 +00:00
samrobinson
2baa70206c Fix disallowed tag.
If ChannelMixingMatrix is public, this causes code to not build.

PiperOrigin-RevId: 511500694
2023-02-27 18:25:27 +00:00
kimvde
bede06546d Restrict SampleConsumer and OnMediaItemChanged threading
This is more future-proof because it is likely to simplify the upcoming
changes to the sample pipelines.

PiperOrigin-RevId: 511492014
2023-02-27 18:24:22 +00:00
samrobinson
f4d470ac4c Only log list of removed encoders if not empty.
Logcat had the following lines, with no other information.
```
DefaultEncoderFactory: Encoders removed for resolution:
DefaultEncoderFactory: Encoders removed for bitrate:
DefaultEncoderFactory: Encoders removed for bitrate mode:
```
PiperOrigin-RevId: 511470231
2023-02-27 18:23:29 +00:00
kimvde
9771629498 Make sure that VideoSamplePipeline is picked for image input
PiperOrigin-RevId: 511449658
2023-02-27 18:22:28 +00:00
kimvde
2365bbf3db Document that the output file is not deleted in case of error/cancel
PiperOrigin-RevId: 511189130
2023-02-27 18:19:56 +00:00
huangdarwin
465301da7c MediaFormatUtil: Reduce Color API 29 restriction to API 24.
Implement getMediaFormatInteger, a helper method simulating mediaformat.getInteger(name, defaultValue).

This reduces the API 29 restriction from MediaFormatUtil.getColorInfo to API 24, in
particular removing the method-based restriction to a constant-based restriction,
so that we can reduce usage of the API 29 class.

This also allows us to slightly simplify prior use-cases where we'd check
containsKey and getInteger to have a default value.

PiperOrigin-RevId: 511184301
2023-02-27 18:18:53 +00:00
kimvde
faad46318b Fix clipping in AudioSamplePipeline
When clipping a MediaItem with start time > 0, the audio was ending
before the video. This is because:
- Audio timestamps are computed based on the sample sizes, with a start
  time set to streamOffsetUs (i.e. the streamStartPositionUs is not
  taken into account).
- The SamplePipeline was subtracting streamStartPositionUs from the
  timestamps before sending the samples to the muxer.
- As a result, the audio timestamps were shifted by
  streamStartPositionUs, while they should be shifter by streamOffsetUs.

PiperOrigin-RevId: 511175923
2023-02-27 18:18:00 +00:00
samrobinson
37bb432869 Expand and refactor TransformerInternal#shouldTranscode.
* Account for `AssetLoader` output types.
* Consider cases that are not audio/video specific.
* Use `Format#sampleMimeType` for track specific conditions to check.
* Untangle `SamplePipeline` initilization from `AssetLoader` state.

PiperOrigin-RevId: 511020865
2023-02-27 18:14:47 +00:00
kimvde
86dca78845 Support added/removed audio track between MediaItems
- Add silent audio when the output contains an audio track but the
  current MediaItem doesn't have any audio.
- Add an audio track when generateSilentAudio is set to true.

PiperOrigin-RevId: 511005887
2023-02-27 18:13:58 +00:00
huangdarwin
518fe3771a HDR: Fix ForceInterpretHdrVideoAsSdrTest, and expand to all APIs.
Previously, this was limited to API 29. Expand this to all API versions.

Also, update the test to:
(1) skip based on SDR format input instead of HDR format input
(2) Check the exception message in order to disambiguate between the decoder tone
mapping error, and general video format support error.

PiperOrigin-RevId: 511002218
2023-02-27 18:12:06 +00:00
kimvde
ac12b96f11 Fix stuck muxer
Unstuck the muxer if the next timestamp in the track with the minimum
timestamp is larger than this minimum timestamp plus
MAX_TRACK_WRITE_AHEAD_US.

PiperOrigin-RevId: 510977088
2023-02-27 18:09:20 +00:00
huangdarwin
d1df8e21d2 HDR: Remove comment explaining force sdr behavior with assetloaders.
Before, if the upstream AssetLoader provides HDR to the VideoSamplePipeline when
HDR_MODE_EXPERIMENTAL_FORCE_INTERPRET_HDR_AS_SDR is requested, the
VideoSamplePipeline would attempt to tell the AssetLoader to output SDR, which
could be accomplished via MediaCodec tone-mapping in the AssetLoader.

However, this makes an assumption of the AssetLoader implementation, and
AssetLoaders may not all implement support for decoder tone-mapping. Remove javadoc
attempting to explain how AssetLoaders (ex. custom ones) could behave.

PiperOrigin-RevId: 510956820
2023-02-27 18:05:16 +00:00
samrobinson
4a7326a286 Pass AudioFormat to ASP#computeNextEncoderInputBufferTimeUs.
This method uses sampleRate, channelCount and pcmEncoding, so passing
AudioFormat is easier.

This will lead into a future change that builds the
encoderInputAudioFormat from encoder.getConfigurationFormat()

PiperOrigin-RevId: 510956177
2023-02-27 18:04:26 +00:00
huangdarwin
84acfe7867 Effect: Rename to ScaleAndRotateTransformation
Rename ScaleToFitTransformation to ScaleAndRotateTransformation.

This better represents the operations that can be accomplished using this
effect. The name was originally named ScaleToFit* because it's not obvious how
to scale to fit using OpenGL, and this effect handled the scaling to fit in a way that no other MatrixTransformations did.

However, it's hard to discover how to rotate when skimming names of effects, so
it's probably more useful to convey that this effect rotates, than that it
scales to fit.

PiperOrigin-RevId: 510480078
2023-02-27 18:02:45 +00:00
huangdarwin
f295dd4211 HDR: Add colorinfo to exception logs.
Format.toString unfortunately doesn't log colorInfo, and as Format holds a very
large set of values, it's unclear that it should. ColorInfo is useful for codec
exceptions though, so log this in ExportException.createForCodec.

PiperOrigin-RevId: 510475520
2023-02-27 18:01:53 +00:00
andrewlewis
b89b13f16b Align test codec caps check with real implementation
PiperOrigin-RevId: 510437262
2023-02-27 17:57:30 +00:00
kimvde
5806414fba Move generateSilentAudio to Composition
Also rename to forceAudioTrack

PiperOrigin-RevId: 510394620
2023-02-27 17:55:27 +00:00
huangdarwin
da8b4db278 HDR: Catch test util decoder support error.
Otherwise, a lack of HDR decoding support will result in the tests checking output files for HDR output, like HdrEditingTest.transform_noRequestedTranscode_hdr10File_transformsOrThrows, failing.

PiperOrigin-RevId: 510213020
2023-02-17 11:53:53 +00:00
samrobinson
e3484d632f Remove space in exception message.
This space was added by mistake in a prior CL of mine.

PiperOrigin-RevId: 510157363
2023-02-17 11:53:03 +00:00
huangdarwin
77ea31e01c HDR: Add comment explaining force-SDR and AssetLoader interaction.
PiperOrigin-RevId: 510155318
2023-02-17 11:52:13 +00:00
huangdarwin
fb8bbce5f1 HDR: Put force HDR as SDR into AssetLoader
This allows us to fix usage of HDR_MODE_EXPERIMENTAL_FORCE_INTERPRET_HDR_AS_SDR.

Before, this was checked in the VideoSamplePipeline, which no longer decides on the decoder configuration input format.

PiperOrigin-RevId: 510142097
2023-02-17 11:51:23 +00:00
kimvde
221c5afb1b Rename occurrences of transform in lib-transformer
PiperOrigin-RevId: 510118760
2023-02-17 11:50:20 +00:00
andrewlewis
5d8a83f924 Remove Pixel watch check
It's unlikely anyone will try to use Transformer on watches.

PiperOrigin-RevId: 510115645
2023-02-17 11:49:29 +00:00
andrewlewis
2ed5819a19 Check capabilities for more tests
Some of the test methods in `TransformationTest` don't check
capabilities. Add a check just scoped to decoding (not checking
encoding, but the default encoder factory may fall back).

PiperOrigin-RevId: 510088562
2023-02-17 11:45:46 +00:00
kimvde
afb52627b6 Rename TransformationTest to ExportTest
PiperOrigin-RevId: 510087543
2023-02-17 11:44:57 +00:00
kimvde
c2fdab847f Rename TransformationException to ExportException
PiperOrigin-RevId: 510062674
2023-02-17 11:42:50 +00:00
huangdarwin
cf768329e6 Effect: Rename FrameProcessor
Rename FrameProcessor to VideoFrameProcessor, and GlEffectsFrameProcessor to
DefaultVideoFrameProcessor.

Most changes are semi-mechanical, semi-manual find-replace, preserving case:
* "FrameProc" -> "VideoFrameProc" (ex. FrameProcessor -> VideoFrameProcessor, and
   FrameProcessingException -> VideoFrameProcessingException)
* "GlEffectsVideoFrameProc" -> "DefaultVideoFrameProc"

PiperOrigin-RevId: 509887384
2023-02-17 11:41:06 +00:00
andrewlewis
41a03dd8a6 Fix some minor nits
PiperOrigin-RevId: 509879029
2023-02-17 11:40:13 +00:00
tofunmi
7e33bdfc94 Change FrameProcessor.create() inputTrackType parameter to a boolean
PiperOrigin-RevId: 509808913
2023-02-15 18:17:52 +00:00
andrewlewis
f03a6ba0f0 Skip tests if muxing is unsupported
PiperOrigin-RevId: 509802784
2023-02-15 18:17:00 +00:00
samrobinson
3608c2e831 Use MediaFormatUtil for creating MediaFormat from Format.
PiperOrigin-RevId: 509785247
2023-02-15 12:14:10 +00:00
samrobinson
79a3464014 Clarify the mime type used in EncoderUtilTest.
PiperOrigin-RevId: 509750806
2023-02-15 12:12:33 +00:00
samrobinson
ee8832ff21 Align ASP and VSP on findSupportedMimeTypeForEncoder/Muxer logic.
* Moved the logic to SamplePipeline.
* Pass the requested values via Format.
* Moved exception throwing inside the methods.
* Build up the mimeTypesToCheck as a set - removing possible duplicate
checks.
* Simplified logic that calls the findSupportedMimeType method.
* Improved javadoc.

PiperOrigin-RevId: 509594062
2023-02-15 12:11:42 +00:00
samrobinson
ccc349abfd Preserve aspect ratio when scaling resolution is insufficient.
PiperOrigin-RevId: 509546771
2023-02-15 12:10:48 +00:00
claincly
3aca3ad170 Use ColorInfo.Builder in transformer and common.
Because the ColorInfo constructor is deprecated.

PiperOrigin-RevId: 509468663
2023-02-14 10:28:33 +00:00
samrobinson
c94adf9041 Generalize getSupportedEncodersForHdr to return List<MediaCodecInfo>.
This now matches EncoderUtil.getSupportedEncoders return type.

PiperOrigin-RevId: 509222078
2023-02-13 15:33:02 +00:00
huangdarwin
fdf636bc7e Test: Move FileUtil to mh from mh.analysis.
FileUtil is only ever used in mh, and never in mh.analysis

PiperOrigin-RevId: 509206721
2023-02-13 14:12:46 +00:00
huangdarwin
cf089b34f0 HDR: Remove HDR-specific error codes.
It may be confusing for 3P apps, for us to have separate error codes for (1) if
encoding HDR is not supported at all by the device, and (2) if encoding the
format, which happens to be HDR, is not supported by the device. Instead, we can
communicate this in the error message.

PiperOrigin-RevId: 509188666
2023-02-13 12:22:55 +00:00
kimvde
fedb74907e Handle when some MediaItems require transcoding but others don't
- For single-asset, the behavior stays the same. Transcode if and only
  if it's necessary,
- For constrained multi-asset, always transcode, except if the setter to
  transmux is set. This is to avoid failing if a MediaItem that doesn't
  require transcoding is followed by a MediaItem that does require
  transcoding.

PiperOrigin-RevId: 508097798
2023-02-14 14:22:52 +00:00
kimvde
16db2bd0a1 Add an API entry point to pass a Composition
PiperOrigin-RevId: 508031337
2023-02-08 14:11:13 +00:00
huangdarwin
0c0c972ea0 Effect: Create GlEffect.isNoOp to simplify Transformer.
This will also allow us to skip transcodes in the future for other effects, like
cropping, color effects, etc.

PiperOrigin-RevId: 507765618
2023-02-08 14:07:41 +00:00
kimvde
c434cc0c9f Rename TransformationResult to ExportResult
Also replace some usages of deprecated Transformer listeners with the new
ones.

PiperOrigin-RevId: 507743860
2023-02-08 14:06:51 +00:00
tofunmi
9224a36add Update CompositeAssetLoader to override queueInputBitmap
PiperOrigin-RevId: 507515587
2023-02-08 14:03:17 +00:00
kimvde
0eb11e269c Rename startTransformation to start
Also update some Javadoc in Transformer

PiperOrigin-RevId: 507395956
2023-02-08 13:59:41 +00:00
tofunmi
4e3c6c6167 Update sample pipelines and frame processors to handle image input.
PiperOrigin-RevId: 506965394
2023-02-08 13:58:50 +00:00
huangdarwin
c45859dde1 HDR: Move codec errors from VideoSamplePipeline
It doesn't actually make sense for them to be placed in the Transformer, because the error's root causes are actually only in codecs. Also, a few codec errors were
repeated, so deduplicate these instances

PiperOrigin-RevId: 506937695
2023-02-08 13:57:57 +00:00
tofunmi
f67569d2dd Update FrameProcessor.create() to accept an input track type.
Based on [this conversation thread](https://chat.google.com/room/AAAA--f88ao/76Rem_cRCK8), I've opted to update the existing FrameProcessor.create() rather than deprecate it, as it is unlikely to be in use by apps outside google3.

PiperOrigin-RevId: 506920930
2023-02-08 13:57:02 +00:00
huangdarwin
c4faae6b03 Demo: Request READ_MEDIA_VIDEO on API 33+
More info in: https://developer.android.com/about/versions/13/behavior-changes-13#granular-media-permissions

PiperOrigin-RevId: 506908079
2023-02-08 13:56:06 +00:00
kimvde
9cb2b6e196 Update Composition structure to take List instead of ImmutableList
This is more flexible for apps and more consitent with the Player API.

PiperOrigin-RevId: 506901645
2023-02-08 13:55:14 +00:00
samrobinson
e2adb567bf Expand createNoSupportedMimeTypeException for HDR ColorInfo case.
PiperOrigin-RevId: 506900817
2023-02-08 13:54:22 +00:00
kimvde
9fdc64785a Adapt TransformationResult for multi-asset
PiperOrigin-RevId: 506898392
2023-02-08 13:53:27 +00:00
samrobinson
2754529ce7 Refactor TransformationException.createForCodec method overloads.
* Overload added `(cause, errorCode, isVideo, isDecoder, details)`,
where `details` is a string of values to be added to the error message
of the `TransformationException`.
* Overload with `MediaFormat` and `mediaCodecName` moved to
`DefaultCodec`, because all usages of that overload were from
`DefaultCodec`, and this allows a simplified API because of internally
stored values.
* `mediaCodecName` removed from overload that takes a `Format`.
* Reordered `createForCodec` parameters.

PiperOrigin-RevId: 506895268
2023-02-08 13:52:32 +00:00
kimvde
dce83d70c5 Rename Transformer callbacks
PiperOrigin-RevId: 506890459
2023-02-08 13:51:40 +00:00
samrobinson
0b7679ed88 Implement a ChannelMixingAudioProcessor.
PiperOrigin-RevId: 506886903
2023-02-08 13:50:45 +00:00
claincly
3b7ebecf16 Prefer hardware encoders in EncoderSelector.
In encoding small and odd-numbered resolutions, like `316x61` ([this image](https://upload.wikimedia.org/wikipedia/commons/6/65/100winners.png)), the current fallback logic prefers a software encoder to hardware ones. The assumption was, the encoder factory applies the encoder size alignment and changes the resolution to `316x60` for SW encoders and `316x64` for HW ones. SW encoders is selected because the supported resolution 60 is closer to requested 61, than the hardware supported 64.

This change changes the default encoder selection process to only expose hardware encoders if there is any.

PiperOrigin-RevId: 506879983
2023-02-08 13:49:52 +00:00
kimvde
4094aef9a5 Rename inputFormat to firstInputFormat in multiple places
With multi-asset, the sample pipelines can process more than one
MediaItem. The renaming makes it clear that the format passed to the
SamplePipeline constructors is the one corresponding to the first
MediaItem. Indeed, the first format is the one used to configure the
SamplePipelines.

PiperOrigin-RevId: 506879260
2023-02-08 13:48:59 +00:00
samrobinson
ed5af63227 Turn off ExoPlayer metrics reporting when used by Transformer.
This avoids affecting ExoPlayer metrics with non-typical usage.

PiperOrigin-RevId: 506878231
2023-02-08 13:48:06 +00:00
tofunmi
bffa6a9653 Update duration & frame rate javadoc in EditedMediaItem
PiperOrigin-RevId: 506874799
2023-02-08 13:47:10 +00:00