Commit graph

266 commits

Author SHA1 Message Date
tofunmi
5488d33da8 Transformer: add api to cutomize image loading
PiperOrigin-RevId: 598793134
2024-01-16 03:56:08 -08:00
Googler
460501fcd1 Log ExportResult information when running transformer demo app
Log elapsed time, in addition to displaying it on screen.
Reuse logging logic between tests and demo.

PiperOrigin-RevId: 597186692
2024-01-10 02:24:31 -08:00
ibaker
62f6c64a91 Rename test.mp3 test asset to test-cbr-info-header.mp3
This file is CBR encoded with LAME, so it has an `Info` header (the CBR
equivalent to `Xing`).

A follow-up change will use this file in `Mp3ExtractorTest`.

Issue: androidx/media#878
PiperOrigin-RevId: 595938327
2024-01-05 02:31:20 -08:00
sheenachhabra
27ae6d974e Allow creating fragmented MP4 file via InAppMuxer
PiperOrigin-RevId: 594431665
2023-12-29 04:15:01 -08:00
sheenachhabra
1609928242 Throw exception when B-frames are written to Mp4Muxer
Mp4Muxer does not support out of order B-frames. Currently it
silently writes out of order B-frames, producing an invalid file (with
negative sample durations).

Although `Mp4Extractor` is somehow able to process this invalid file and
`Exoplayer` is able to play it but that is unexpected.

The `sample.mp4` test file contains B frames. Other test files does not
contain `H264 video + AAC audio` format hence created a new test file by
running `sample.mp4` via `Transformer` after applying some effects.

PiperOrigin-RevId: 594016144
2023-12-27 08:32:36 -08:00
tofunmi
e3056dacac Rename Mp4MetadataInfo to mp4Info
PiperOrigin-RevId: 593143940
2023-12-22 11:10:30 -08:00
tofunmi
7e12b9e15f Add rough progress updates in trim optimization
PiperOrigin-RevId: 593116025
2023-12-22 07:37:23 -08:00
tofunmi
1632f37d70 Transformer: Add api to drop audio samples before the first video frame
fix for Issue: androidx/media#829

Manual Testing: Viewed the transformer output file of previously problematic case in Exoplayer, Chrome, VLC, Quicktime and Safari and all showed the issue not to occur anymore. The newly produced output file can be found at https://github.com/androidx/media/assets/42352357/fdf105c1-9550-422f-b088-7900f655ac78

PiperOrigin-RevId: 593104752
2023-12-22 06:18:07 -08:00
tofunmi
3081ceb179 Remove the matchInitializationData option from TransformerInternal
Essentially a manual revert of ba0724ca78

PiperOrigin-RevId: 592869049
2023-12-21 08:38:58 -08:00
tofunmi
087c07e596 Rename OptimizationResult options
Rename options to be more accurate of what they capture

PiperOrigin-RevId: 592155563
2023-12-19 01:50:40 -08:00
tofunmi
2fa5430417 Update OptimizationResult to specify different reasons for abandonment
PiperOrigin-RevId: 590530694
2023-12-13 03:22:55 -08:00
tofunmi
c5c8e988e8 Abandon trim optimization when transcoding effects are set
PiperOrigin-RevId: 588839072
2023-12-07 10:39:38 -08:00
samrobinson
ab798659d9 Request specific AudioFormat from AudioGraphInputs on creation.
Also adds an alternate way to configure the AudioGraph.

Apps should no longer need to ensure that inputs have the same sample
rate.

PiperOrigin-RevId: 588747431
2023-12-07 04:42:03 -08:00
tofunmi
be5b29d92d Update ExportResult.OptimizationResult options
Added a new ABANDONED option so one can tell the difference between when the optimization has been requested but not applied vs not requested at all. also changed the ordering do better represent the hierarchy of failure modes

PiperOrigin-RevId: 588720513
2023-12-07 02:52:16 -08:00
tofunmi
cd346ca14d Transformer: Add support for transmuxing audio in trim optimization
PiperOrigin-RevId: 588711597
2023-12-07 02:18:21 -08:00
samrobinson
757585ce6d Release MuxerWrapper in every test (in an @After block).
PiperOrigin-RevId: 588110566
2023-12-05 10:13:27 -08:00
tofunmi
b0e2fd78af Use getter for presentationDurationUs
Delays failures to when the field is used in preview code, so that exports in the demo app can continue succeed

PiperOrigin-RevId: 588022569
2023-12-05 04:43:57 -08:00
samrobinson
fe272d3c49 Follow practices/guidance in Muxer tests around:
* @Before and @After usage. [1].
* static fields before non-static. [2].

@Before method should typically be paired with an @After method,
focused on ensuring that the component is released regardless of what
the test does.

In tests, inlining final class variables is preferrable [1]. In general
things like the file path should be part of the test (the @Rule means
we don't need before/after) if only used once.

Statically importing values and using them directly is preferable to
having a variable declared as non-final that's effectively final,
because from a readability perspective someone can see (the caps) that
the value is final static and immutable, so doesn't have to check if
it's changed/reassigned.

PiperOrigin-RevId: 586697887
2023-11-30 09:29:18 -08:00
andrewlewis
e84a13fb54 Restrict operating rate workaround to SM8550
PiperOrigin-RevId: 585613041
2023-11-27 04:42:42 -08:00
andrewlewis
ad40db4489 Avoid value close to overflow for KEY_OPERATING_RATE
Using `Integer.MAX_VALUE` risks causing arithmetic overflow in the codec
implementation.

Issue: androidx/media#810

#minor-release

PiperOrigin-RevId: 585104621
2023-11-24 08:57:59 -08:00
tofunmi
2d77e4d22c Implement trim optimization in Transformer
PiperOrigin-RevId: 584622392
2023-11-22 07:28:34 -08:00
andrewlewis
7b9aa87344 Allow allocating more buffers when transmuxing
When transmuxing, the `EncodedSampleExporter` maintains a queue of input
buffers that get filled with encoded data by the asset loader. The number of
buffers was limited to avoid using more and more memory if producer (asset
loader) gets far ahead of the consumer (exporter).

Previously this limit was fixed at 10 buffers, but increasing the number of
buffers can make some transmux operations much faster. Allow allocating between
a min and max number of buffers, and also set a target allocation size beyond
which new buffers can't be allocated. This allows audio formats which require
many small buffers to be processed more quickly, while preventing allocating
too much memory for hypothetical very high bitrate formats.

'Remove video' edits on local videos in particular get much faster, because
audio buffers are very short and there are lots of them. With a sample 10
minute video, a 'remove video' edit took 2 seconds (36 seconds before this
change).  With a sample 1 minute removing video took 0.25 seconds after this
change (2.5 seconds before).

The speed improvement is smaller for other types of edits that retain the video
track. Transmuxing a 10 minute video retaining the video track took 26 seconds
(40 seconds before).

PiperOrigin-RevId: 583390284
2023-11-17 08:11:25 -08:00
tofunmi
4311cf1d6c Mp4MetadataInfo: add format and time-based I-frame timestamp extraction
PiperOrigin-RevId: 580132463
2023-11-07 04:31:25 -08:00
tofunmi
ba0724ca78 Pass initializationData to EncoderFactory when requested
PiperOrigin-RevId: 580098432
2023-11-07 02:04:39 -08:00
tofunmi
3253f1b5cd Add microsecond precision to MediaItem.ClippingConfiguration
PiperOrigin-RevId: 578881990
2023-11-02 09:40:11 -07:00
sheenachhabra
1e2815cade Simplify Mp4ExtractorWrapper implementation
Changes includes:
1. Create static factory method and hide constructor.
2. Move all the fetching logic to init() method.

PiperOrigin-RevId: 576544902
2023-10-25 09:02:02 -07:00
claincly
e8adbd9075 Require duration in EditedMediaItem in composition previewing
Adds an additional field for actual presentation duration of the
EditedMediaItem

PiperOrigin-RevId: 576090574
2023-10-24 04:09:14 -07:00
sheenachhabra
dc865e6e98 Return TIME_UNSET for last sync sample if there is no video track
PiperOrigin-RevId: 565395263
2023-09-14 09:44:42 -07:00
sheenachhabra
b042943102 Add different modes in MuxerWrapper
For pause and resume feature we need to use same `MuxerWrapper`
for `remuxing processed video` and then to `process remaining video`.
In order to use same `MuxerWrapper` across `different Exports`
we need to preserve its state.

PiperOrigin-RevId: 564728396
2023-09-12 08:28:35 -07:00
samrobinson
866d62dd34 Improve CompositionExportTest assertions by using dump files.
PiperOrigin-RevId: 563708666
2023-09-08 04:09:21 -07:00
samrobinson
763dddfbd4 Dump with C.TrackType as key, rather than muxer track index.
Modifying dumping to not required "released" to be called.

Track index is an arbitrary value based on the order of addTrack calls.
Samples are dumped by track (rather than as soon as they are written),
so it's preferable to use a value that provides more context.

By using the track type as a key, dump files will be more deterministic
and will have more similarities when branched.

PiperOrigin-RevId: 563700982
2023-09-08 03:25:13 -07:00
samrobinson
cff2816da4 Add silence generation parameterized test case.
PiperOrigin-RevId: 563098931
2023-09-06 07:34:32 -07:00
samrobinson
92814b84a8 Add alternate MP4 asset with PCM audio.
Audio is Mono 44.1kHz.

Created using:
`ffmpeg -i <input> -c:v copy -c:a pcm_s16be <output>`

PiperOrigin-RevId: 563079553
2023-09-06 05:58:27 -07:00
samrobinson
ef45c0fe5d Add parameterized dump tests for single item exports.
PiperOrigin-RevId: 563075771
2023-09-06 05:35:51 -07:00
sheenachhabra
657856dfe5 Fix/Update TODOs in Mp4Muxer tests
PiperOrigin-RevId: 562728777
2023-09-05 04:04:51 -07:00
samrobinson
6ecd3e9c2d Dump DumpableSamples list in track order.
Switch to SparseArray<List> rather than a Map<Integer, List>.

Track indices now determine sample list dumping, reducing flakiness.

PiperOrigin-RevId: 561658586
2023-08-31 08:15:41 -07:00
tofunmi
667103f2bd Fix SequenceAssetLoader signalling End Of Video Input twice
PiperOrigin-RevId: 560170216
2023-08-25 12:46:50 -07:00
sheenachhabra
1f013afea8 Add Mp4ExtractorWrapper into Transformer module
For pause and resume feature we need to find the timestamp of
the last sync sample in an MP4 file. The Mp4ExtractorWrapper
provides an easy to use API on top of Mp4Extractor.

PiperOrigin-RevId: 560113112
2023-08-25 09:29:28 -07:00
tofunmi
404a259295 Remove queueInputBitmap by framerate and duration interfaces
PiperOrigin-RevId: 559384011
2023-08-24 09:25:49 +01:00
samrobinson
ff39726368 Remove redundant "aac" tag for dump file.
PiperOrigin-RevId: 559147235
2023-08-24 09:17:32 +01:00
samrobinson
2309fc5edb Add AudioGraph test verifying silence outputs correct amount of bytes.
PiperOrigin-RevId: 559072505
2023-08-22 15:47:54 +01:00
samrobinson
b7782b5119 Inline ProgressHolder and ArgumentCaptor test class variables.
PiperOrigin-RevId: 559069221
2023-08-22 15:46:13 +01:00
samrobinson
7346029f72 Use TemporaryFolder @Rule for automated tear down of temp files.
Reduce @Before specific logic, following java dev guide.

PiperOrigin-RevId: 558825609
2023-08-22 15:42:07 +01:00
samrobinson
ae7667783c Split dump file directories based on input file name.
Remove old unused dump files.

PiperOrigin-RevId: 558820926
2023-08-22 15:39:28 +01:00
samrobinson
352916b182 Add parameterized tests for generating silence.
Covers all permutations&combinations of:
* AV asset.
* AV asset with effects.
* AV asset with silence.
* AV asset with silence and effects.

Note that the video is not relevant (therefore transmuxed), but is
needed for silence generation.

PiperOrigin-RevId: 558734593
2023-08-22 15:26:19 +01:00
samrobinson
54797d4cc5 Use @Rule and other util to simplify ParameterizedAudioExportTest.
PiperOrigin-RevId: 558168355
2023-08-22 15:19:15 +01:00
samrobinson
99ac6feeee Add varargs convenience constructors to Sequence & Composition.Builder
Improves API ease-of-use & readability and reduces verbose boilerplate.

PiperOrigin-RevId: 558163557
2023-08-22 15:17:55 +01:00
samrobinson
2db6f0aee7 Ensure audio components check incoming data is valid.
Default PCM encoding is only set for decoders outputting raw.

Tests migrated to abide by tighter restrictions.

PiperOrigin-RevId: 558129452
2023-08-18 15:33:44 +01:00
samrobinson
74fa0ed6bf Move FakeAssetLoader to inner class of MediaItemExportTest.
MediaItemExportTest is the only class using this FakeAssetLoader, so it
should be moved out of TestUtil.

PiperOrigin-RevId: 557546717
2023-08-16 22:03:14 +01:00
samrobinson
27bc2e6844 Add new effects helper util for unit tests.
PiperOrigin-RevId: 557515564
2023-08-16 22:01:52 +01:00