Commit graph

307 commits

Author SHA1 Message Date
samrobinson
81b9739bda Consolidate dump files for item effect vs composition effect.
PiperOrigin-RevId: 623161140
2024-04-09 07:33:38 -07:00
samrobinson
24b86e73cb Group together PCM audio samples by fixed size in CapturingMuxer dumps.
Reduces flakiness of tests that assert on PCM audio. Tests now have to
clearly choose how they want the capturing muxer to handle pcm audio.

Note that the only dump files that have changed are those that deal
with PCM audio (.wav, sowt, twos, silence). Because of the continuous
nature of PCM, timestamps are not part of the dump.

PiperOrigin-RevId: 623155302
2024-04-09 07:04:49 -07:00
kimvde
04a3889998 Reset maxPositionOfRemovedSources in DefaultAudioMixer
If the mixer is reset without resetting maxPositionOfRemovedSources and
then reused, the value of maxPositionOfRemovedSources can be outdated,
leading to an incorrect number of bytes being output by the mixer.

PiperOrigin-RevId: 619832502
2024-03-28 01:22:25 -07:00
tofunmi
65e94480f4 Transformer: add single-asset ultraHDR image to HLG HDR vid transcoding
PiperOrigin-RevId: 619138202
2024-03-26 04:06:41 -07:00
sheenachhabra
2eaece5ed9 Move setMaxDelayBetweenSamplesMs on Transformer builder
This property is transformer specific and does not belong to
muxer interface.
This is to eventually replace muxer interface in Transformer module with
muxer interface in Muxer module.

PiperOrigin-RevId: 618895836
2024-03-25 11:02:15 -07:00
sheenachhabra
16aac07bce Add support for processing 3 byte NAL start code in Mp4Muxer
Issue: androidx/media#725

Ideally the test to transmux a `ts` file (having 3 bytes NAL start code)
should go into `Mp4MuxerEndToEndTest`but `Mp4MuxerEndToEndTest` uses `MediaExtractor` and `MediaExtractor` is returning samples with `4 byte NAL start code` which will not exercise the newly added code path.
Hence the test is added in `TransformerWithInAppMuxerEndToEndTest` which
internally uses `media3 extractor` and feeds samples with `3 bytes NAL start code`
only.

PiperOrigin-RevId: 617985866
2024-03-21 15:49:12 -07:00
kimvde
6fc4f0263f Add methods required for seek to AudioGraph
Seeking will consist of the following steps:
- Block the AudioGraph input data.
- Flush the AudioGraph.
- Seek the ExoPlayers.
- Unblock the AudioGraph input data.

PiperOrigin-RevId: 617868124
2024-03-21 09:31:51 -07:00
Googler
415d779c62 Explicitly check that AudioGraph input is PCM 16
AudioGraph implicitly assumes that input will be PCM 16 bit, and not float.
Make the check explicit.

PiperOrigin-RevId: 616149131
2024-03-15 09:14:04 -07:00
kimvde
e40ce150bf Implement AudioGraph.flush()
When flushing the AudioGraph, the AudioMixer will be preserved but the
sources will be recreated. This is probably a bit less efficient but
makes the logic simpler. Indeed, if we had to keep the sources alive,
we would need to add a way to reconfigure them with a new timestamp for
seeking. We would also need to change the way sources are ended because
they are currently removed when they are ended. Also, it is acceptable
to have a small delay when seeking, which means that performance is less
critical than for playback.

PiperOrigin-RevId: 615775501
2024-03-14 07:49:07 -07:00
kimvde
3248b7962a Add clipFloatOutput parameter to AudioMixer
PiperOrigin-RevId: 615720340
2024-03-14 03:44:37 -07:00
huangdarwin
ca6031deab Update old method name.
getSupportedEncoderNamesForHdrEditing became getSupportedEncodersForHdrEditing at some point, but this comment wasn't updated before...

PiperOrigin-RevId: 615712707
2024-03-14 03:11:03 -07:00
sheenachhabra
668a172683 Rename MuxerWrapper.release() method
PiperOrigin-RevId: 615399319
2024-03-13 06:54:19 -07:00
kimvde
f02dc8e528 Implement AudioGraphInput.flush
PiperOrigin-RevId: 613563855
2024-03-07 06:33:02 -08:00
Googler
e175a772db Move AudioGraph effects to configure method
Move old configure behavior to private configureMixer.
registerInput() is now required after reset().

PiperOrigin-RevId: 612795418
2024-03-05 04:46:36 -08:00
tofunmi
bcfad4b3b4 Move hdrMode from defaultAssetLoaderFactory constructors to create()
Plumbing hdrMode through the default asset loader factory via the constructor is problematic because it breaks API boundaries. It means there is another way to set hdrMode outside of Composition.java and TransformationRequest.java, which is error prone and cause problems if someone an app starts customizing the assetloaderfactory. It also means custom asset loaders can't receive this information without hacking around.

The introduction of the composition-level settings class makes this approach easily extensible for other settings applied on the composition level but use in an individual asset level basis (e.g. ultraHDR support).

PiperOrigin-RevId: 611466920
2024-02-29 07:41:18 -08:00
Googler
6f28eeff31 Pick max H.264 level when trim optimizing
Instead of always starting with the transcoded H.264 level, take the maximum
from transcoded and transmuxed levels

PiperOrigin-RevId: 610759438
2024-02-27 08:27:38 -08:00
Googler
c3aec4f19d Comments for AudioGraph constructor
Add /*effects=*/ at callsites of AudioGraph constructor.

PiperOrigin-RevId: 610741692
2024-02-27 07:12:53 -08:00
Googler
0e0e1c4f1a Tests for composition audio effects
Add tests for the changes to AudioGraph that support composition audio effects

PiperOrigin-RevId: 610691768
2024-02-27 03:04:20 -08:00
Googler
477ace1be9 Composition audio effects
Implement composition-level audio effects in AudioGraph.

PiperOrigin-RevId: 610689632
2024-02-27 02:54:08 -08:00
tofunmi
5a892509f7 remove media/bitmap/input_images
move the images into the respective places in the file extension directory so they file the pre-existing structure

PiperOrigin-RevId: 609744673
2024-02-23 09:07:03 -08:00
sheenachhabra
284d17cb13 Add release reason argument in the MuxerWrapper.release() method
Also called `muxer.release()` in other tests where it is
appropriate.

PiperOrigin-RevId: 609736956
2024-02-23 08:34:04 -08:00
sheenachhabra
fd8f45b38e Add MdtaMetadataEntry constructor with default locale indicator
PiperOrigin-RevId: 609427529
2024-02-22 10:55:50 -08:00
Googler
dca7bae416 Allow mismatching H.264/AVC level for trimming or resuming
MediaCodec docs already allude to potentially mismatching H.264 level
between container and bitstream. Relax the initialization data check to
reflect this.

PiperOrigin-RevId: 608942322
2024-02-21 04:20:50 -08:00
sheenachhabra
a82d7e7098 Improve retrieveTrackFormat() java doc and remove catch block
PiperOrigin-RevId: 608939192
2024-02-21 04:11:39 -08:00
Googler
e43f96687c Fix failure to write negative PTS sample
Fixes an issue caused by no support for negative audio PTS and edit lists
in FrameworkMuxer, Android versions before 11

PiperOrigin-RevId: 607690507
2024-02-16 08:03:38 -08:00
sheenachhabra
69c555be0a Move retrieveTrackFormat() method to TestUtil
TestUtil class is more appropriate for the given method.
In the next CL, the only method in FileUtil.java will be moved back
into transformer library and the FileUtil class will be removed.

PiperOrigin-RevId: 605648034
2024-02-09 09:18:14 -08:00
sheenachhabra
8a758c2ed7 Replace setModificationTime API with setTimestampData in Mp4Muxer
The new API will take both `creation time` and `modification time`.

Till now, Mp4Muxer wrote `modification time` in both
`creation time` and `modification time` field, which was
incorrect.

PiperOrigin-RevId: 605590623
2024-02-09 04:15:53 -08:00
tofunmi
2cb9d9b383 Export: Add methods to determine the conversion process of a track
PiperOrigin-RevId: 604754827
2024-02-06 13:58:14 -08:00
sheenachhabra
20053dcdc6 Test output file's metadata using MetadataRetriever
Earlier implementation compared the whole file against a golden
data. The new implementation compares only the metadata being tested.
This will avoid updating the golden data when any unrelated change
(unrelated to scenario being tested) is made.

Added a separate test to compare the whole output file against a golden data.

PiperOrigin-RevId: 604692985
2024-02-06 10:36:47 -08:00
samrobinson
c768e60bf2 Clearer testing of isEnded state of AudioGraphInput.
PiperOrigin-RevId: 604680924
2024-02-06 10:00:24 -08:00
tofunmi
4da576f05f Support transmux when both no op effects and regular rotations are set
PiperOrigin-RevId: 604319412
2024-02-05 08:05:32 -08:00
samrobinson
f8352580cb Add AudioGraphInput tests around getOutput behaviour.
PiperOrigin-RevId: 604297126
2024-02-05 06:18:46 -08:00
tofunmi
49c6d25106 Move setting the muxerWrapper rotation out of shouldTranscodeVideo()
Before supporting transmuxing when both no op effects and regular rotations are set, move setting the muxerWrapper rotation out of shouldTranscodeVideo() to ensure the muxerWrapper rotation is only set at the appropriate times.

This cl also ensures the state between the muxerWrapper and the list of video effects is consistent by clearing the list of videoEffects in trim optimization. If trim optimisation is being applied, then EditedMediItem.effects.videoEffects only contains no-op effects or regular rotations that get be applied in the muxer wrapper. Therefore, we should clear the list of video effects to ensure that no effect gets applied twice.

PiperOrigin-RevId: 604292052
2024-02-05 05:54:11 -08:00
samrobinson
029071a342 Improve checks around AudioGraphInput configuring with requested format
Also adds extra AudioGraphInputTest cases.

PiperOrigin-RevId: 603690041
2024-02-02 08:13:54 -08:00
samrobinson
7c8a31e2b1 Remove redundant boilerplate in AudioGraphTest & AudioGraphInputTest.
* getPcmFormat Util method already sets the mime type as AUDIO_RAW.
* static final AudioFormat consts improve test case readability.

PiperOrigin-RevId: 602740007
2024-01-30 08:53:10 -08:00
Googler
994e2fc317 Rollback of f9eb8626eb
PiperOrigin-RevId: 601912101
2024-01-26 17:55:30 -08:00
Googler
9e48fcb8fb Rollback of 2e6982be2c
PiperOrigin-RevId: 601911260
2024-01-26 17:50:24 -08:00
huangdarwin
b1c1c2a6dd Transformer: Plumb hdrMode into AssetLoaders instead of forceInterpret.
This allows us to also plumb whether we're doing MediaCodec tone-mapping,
which will be used in a follow-up CL in the ExoAssetLoaderVideoRenderer.

PiperOrigin-RevId: 601774435
2024-01-26 08:36:21 -08:00
tofunmi
2e6982be2c limit transmuxing to regular rotations only
PiperOrigin-RevId: 601710663
2024-01-26 02:46:50 -08:00
tofunmi
f9eb8626eb Support transmuxing when no op effects and regular rotations are set
PiperOrigin-RevId: 601419245
2024-01-25 05:34:06 -08:00
tofunmi
b94c7d08c1 MuxerWrapper rotation degree fix
Allow setAdditionalRotationDegrees to be called with same rotation after tracks added. This is needed for processes that mux files partially trim optimization so they don't error out after hitting the check state

Manually tested to ensure trim optimization succeeds, automated test added here as well

PiperOrigin-RevId: 601081778
2024-01-24 04:44:28 -08:00
tofunmi
5488d33da8 Transformer: add api to cutomize image loading
PiperOrigin-RevId: 598793134
2024-01-16 03:56:08 -08:00
Googler
460501fcd1 Log ExportResult information when running transformer demo app
Log elapsed time, in addition to displaying it on screen.
Reuse logging logic between tests and demo.

PiperOrigin-RevId: 597186692
2024-01-10 02:24:31 -08:00
ibaker
62f6c64a91 Rename test.mp3 test asset to test-cbr-info-header.mp3
This file is CBR encoded with LAME, so it has an `Info` header (the CBR
equivalent to `Xing`).

A follow-up change will use this file in `Mp3ExtractorTest`.

Issue: androidx/media#878
PiperOrigin-RevId: 595938327
2024-01-05 02:31:20 -08:00
sheenachhabra
27ae6d974e Allow creating fragmented MP4 file via InAppMuxer
PiperOrigin-RevId: 594431665
2023-12-29 04:15:01 -08:00
sheenachhabra
1609928242 Throw exception when B-frames are written to Mp4Muxer
Mp4Muxer does not support out of order B-frames. Currently it
silently writes out of order B-frames, producing an invalid file (with
negative sample durations).

Although `Mp4Extractor` is somehow able to process this invalid file and
`Exoplayer` is able to play it but that is unexpected.

The `sample.mp4` test file contains B frames. Other test files does not
contain `H264 video + AAC audio` format hence created a new test file by
running `sample.mp4` via `Transformer` after applying some effects.

PiperOrigin-RevId: 594016144
2023-12-27 08:32:36 -08:00
tofunmi
e3056dacac Rename Mp4MetadataInfo to mp4Info
PiperOrigin-RevId: 593143940
2023-12-22 11:10:30 -08:00
tofunmi
7e12b9e15f Add rough progress updates in trim optimization
PiperOrigin-RevId: 593116025
2023-12-22 07:37:23 -08:00
tofunmi
1632f37d70 Transformer: Add api to drop audio samples before the first video frame
fix for Issue: androidx/media#829

Manual Testing: Viewed the transformer output file of previously problematic case in Exoplayer, Chrome, VLC, Quicktime and Safari and all showed the issue not to occur anymore. The newly produced output file can be found at https://github.com/androidx/media/assets/42352357/fdf105c1-9550-422f-b088-7900f655ac78

PiperOrigin-RevId: 593104752
2023-12-22 06:18:07 -08:00
tofunmi
3081ceb179 Remove the matchInitializationData option from TransformerInternal
Essentially a manual revert of ba0724ca78

PiperOrigin-RevId: 592869049
2023-12-21 08:38:58 -08:00