Commit graph

231 commits

Author SHA1 Message Date
samrobinson
6ecd3e9c2d Dump DumpableSamples list in track order.
Switch to SparseArray<List> rather than a Map<Integer, List>.

Track indices now determine sample list dumping, reducing flakiness.

PiperOrigin-RevId: 561658586
2023-08-31 08:15:41 -07:00
tofunmi
667103f2bd Fix SequenceAssetLoader signalling End Of Video Input twice
PiperOrigin-RevId: 560170216
2023-08-25 12:46:50 -07:00
sheenachhabra
1f013afea8 Add Mp4ExtractorWrapper into Transformer module
For pause and resume feature we need to find the timestamp of
the last sync sample in an MP4 file. The Mp4ExtractorWrapper
provides an easy to use API on top of Mp4Extractor.

PiperOrigin-RevId: 560113112
2023-08-25 09:29:28 -07:00
tofunmi
404a259295 Remove queueInputBitmap by framerate and duration interfaces
PiperOrigin-RevId: 559384011
2023-08-24 09:25:49 +01:00
samrobinson
ff39726368 Remove redundant "aac" tag for dump file.
PiperOrigin-RevId: 559147235
2023-08-24 09:17:32 +01:00
samrobinson
2309fc5edb Add AudioGraph test verifying silence outputs correct amount of bytes.
PiperOrigin-RevId: 559072505
2023-08-22 15:47:54 +01:00
samrobinson
b7782b5119 Inline ProgressHolder and ArgumentCaptor test class variables.
PiperOrigin-RevId: 559069221
2023-08-22 15:46:13 +01:00
samrobinson
7346029f72 Use TemporaryFolder @Rule for automated tear down of temp files.
Reduce @Before specific logic, following java dev guide.

PiperOrigin-RevId: 558825609
2023-08-22 15:42:07 +01:00
samrobinson
ae7667783c Split dump file directories based on input file name.
Remove old unused dump files.

PiperOrigin-RevId: 558820926
2023-08-22 15:39:28 +01:00
samrobinson
352916b182 Add parameterized tests for generating silence.
Covers all permutations&combinations of:
* AV asset.
* AV asset with effects.
* AV asset with silence.
* AV asset with silence and effects.

Note that the video is not relevant (therefore transmuxed), but is
needed for silence generation.

PiperOrigin-RevId: 558734593
2023-08-22 15:26:19 +01:00
samrobinson
54797d4cc5 Use @Rule and other util to simplify ParameterizedAudioExportTest.
PiperOrigin-RevId: 558168355
2023-08-22 15:19:15 +01:00
samrobinson
99ac6feeee Add varargs convenience constructors to Sequence & Composition.Builder
Improves API ease-of-use & readability and reduces verbose boilerplate.

PiperOrigin-RevId: 558163557
2023-08-22 15:17:55 +01:00
samrobinson
2db6f0aee7 Ensure audio components check incoming data is valid.
Default PCM encoding is only set for decoders outputting raw.

Tests migrated to abide by tighter restrictions.

PiperOrigin-RevId: 558129452
2023-08-18 15:33:44 +01:00
samrobinson
74fa0ed6bf Move FakeAssetLoader to inner class of MediaItemExportTest.
MediaItemExportTest is the only class using this FakeAssetLoader, so it
should be moved out of TestUtil.

PiperOrigin-RevId: 557546717
2023-08-16 22:03:14 +01:00
samrobinson
27bc2e6844 Add new effects helper util for unit tests.
PiperOrigin-RevId: 557515564
2023-08-16 22:01:52 +01:00
jbibik
14e5b5b865 Mark CompositionPlayerTest as ClosedSource and copybara exclusion
#minor-release

PiperOrigin-RevId: 557416683
2023-08-16 14:33:02 +01:00
samrobinson
9670c74044 Add parameterized test for audio export.
Verifies export completes successfully for every permutation.

PiperOrigin-RevId: 557157527
2023-08-16 14:28:35 +01:00
samrobinson
ab8c5a6b63 Clarify & reduce the ShadowMediaCodec configurations added by default.
PiperOrigin-RevId: 556875714
2023-08-14 20:35:06 +01:00
samrobinson
c740b58efa Add API for injecting AudioMixer.Factory.
Rename AudioMixerImpl to DefaultAudioMixer.

Removes the AudioMixerImpl specific getOutputAudioFormat, as the caller
defines and sets this.

PiperOrigin-RevId: 555887722
2023-08-11 20:26:21 +00:00
samrobinson
6045190103 Rename TestMuxer to CapturingMuxer to match other dumpable components.
See CapturingAudioSink, CapturingRenderersFactory etc.

PiperOrigin-RevId: 555437158
2023-08-10 12:32:31 +00:00
samrobinson
79725fd332 Use TestMuxer.Factory as the holder of the Dumpable test muxer.
PiperOrigin-RevId: 555222322
2023-08-10 12:28:18 +00:00
kimvde
2a645ac2b6 Add CompositionPlayer listener tests
PiperOrigin-RevId: 555139635
2023-08-10 12:20:03 +00:00
kimvde
f43146718c Add CompositionPlayer unit test
Test the audio side only because the video side won't be testable with
unit tests once we add the OpenGL components.

PiperOrigin-RevId: 554508703
2023-08-10 11:59:42 +00:00
samrobinson
42425334ae Mark always passing test as @Ignore.
Added a TODO inline linking to the created bug to address this.

PiperOrigin-RevId: 554422990
2023-08-07 11:35:34 +00:00
samrobinson
9e975b25d1 Define TestMuxer Factory and Holder as inner classes.
PiperOrigin-RevId: 552469061
2023-08-01 14:29:38 +01:00
samrobinson
ccdde8806e Account for endTimeUs when outputting with no sources.
When endTimeUs is set, restrict output to that point.

PiperOrigin-RevId: 551555392
2023-08-01 14:15:34 +01:00
samrobinson
54093a152e Integrate AudioMixer for audio export.
Adds support for compositions with multiple audio sequences.

PiperOrigin-RevId: 550880626
2023-08-01 14:03:22 +01:00
samrobinson
357c458028 Ensure EOS is queued after processing generated silence with effects.
When generating silence for AudioProcessingPipeline, audio never
queued EOS downstream.

Linked to this, when silence followed an item with audio, the silence
was added to SilentAudioGenerator before the mediaItem reconfiguration
occurred. If the silence had effects, the APP would be flushed after
silence queued EOS, resetting APP.isEnded back to false, so AudioGraph
never ended.

Regression tests reproduce failure without fix, but pass with it.

PiperOrigin-RevId: 550853714
2023-08-01 14:02:09 +01:00
huangdarwin
d7b5ab9f23 Add Thread.currentThread().interrupt() for InterruptedExceptions.
PiperOrigin-RevId: 549975878
2023-08-01 13:49:48 +01:00
samrobinson
035934c6d4 Combine audio mixing logic across Transformer.
AudioMixingUtil#mix handles input & output in float or Int16 PCM. Given
Float and Int16 use different sample ratnes, this util handles
conversion between the two, based on the encoding being mixed to.

Migrate AudioMixer to use the util, removing AudioMixingAlgorithm
interface and implementation. ChannelMixingAudioProcessor will be
migrated after additional performance checks.

PiperOrigin-RevId: 548994584
2023-07-20 10:22:58 +01:00
samrobinson
847cc9b841 Use asset with encoded video & raw audio for Robolectric test.
Test requires file to have video track for forcing silence.

PiperOrigin-RevId: 547839076
2023-07-14 10:19:29 +01:00
claincly
50c6efe95d Rename SamplePipeline to SampleExporter
PiperOrigin-RevId: 545974776
2023-07-13 15:06:10 +01:00
kimvde
81c83e4213 Remove HDR modes from TransformationRequest
PiperOrigin-RevId: 545352306
2023-07-05 09:16:52 +00:00
samrobinson
f60f79bb10 Handle media item (Effects/Format) changes in AudioSamplePipeline.
On a MediaItem change, the input Format (and Effects to apply) may be
different. Therefore the AudioProcessingPipeline must be reconfigured
to determine what processing is active, and what the AudioFormat of the
data output is. In the event that it is different, additional
AudioProcessor instances must be used to ensure the encoder will still
be able to accept the audio buffers.

PiperOrigin-RevId: 544338451
2023-06-29 23:14:10 +00:00
tofunmi
8cecb93570 Texture Input: Add GLSyncToken to the OnInputFrameProcessedListener
PiperOrigin-RevId: 543867944
2023-06-29 23:02:20 +00:00
kimvde
b358d075a1 Replace usages of deprecated TransformationRequest setter
PiperOrigin-RevId: 543765376
2023-06-29 22:59:06 +00:00
samrobinson
0d67733d28 Update media in silence concatentation test to match silent format.
Goal of tests (SequenceExportTest) that use this media is for the
silence and the media to match exactly with audio format, however
`sample_with_increasing_timestamps.mp4` had a different sample rate.

testvid_1022ms.mp4: channel count = 2, sample rate = 44100.

PiperOrigin-RevId: 543458948
2023-06-29 22:52:32 +00:00
sheenachhabra
8735382590 Move SSIM related files to test_util
PiperOrigin-RevId: 543444043
2023-06-29 22:51:20 +00:00
samrobinson
b46b6a8278 Use stereo audio in silence -> audio SequenceExportTest.
With the upcoming "handle format changes" CL, stereo -> mono audio
would add an AudioProcessor. Robolectric decodes output encoded data,
which crashes some AudioProcessors because the number of frames may not
be an integer.

PiperOrigin-RevId: 542568875
2023-06-23 16:40:57 +00:00
kimvde
69cef452a0 Make SpeedChangingAudioProcessor public
PiperOrigin-RevId: 542533743
2023-06-22 15:46:42 +00:00
samrobinson
6d648f8bdb Add dump tests for concatenating 2 audio items.
Audio only tests are now using RAW audio where possible, which is
passed through the Robolectric decoders/encoders, and can be handled by
the AudioProcessor instances accurately.

PiperOrigin-RevId: 541648853
2023-06-20 13:57:33 +01:00
sheenachhabra
c0e8513b7a Make dump files deterministic
Issue: When running the Transformer related test cases, the tests are flaky
because the order in which audio and video samples are interleaved seems to
differ in few instances.

Root cause: When running a transformation the sample producer (Asset loader)
and sample consumer (Sample pipeline) both runs on different thread and
theoretically there is no reason for behaviour to be deterministic because
the number of samples produced/written depends on how fast individual thread
works. So it is indeed surprising that test somehow worked deterministically in
majority of instances (may be something to do with Robolectric environment).

Solution: Since we don't expect the order of sample interleaving to be deterministic, make the dumping logic deterministic where all the video
samples will be collected and then dumped together (similarly for audio). This would mean we won't be able to see the interleaving so for that we need to
add separate test case verifying the interleaving logic only.

Pending: Test case for interleaving.
PiperOrigin-RevId: 540930871
2023-06-19 16:28:07 +01:00
samrobinson
1236d37acb Improve SequenceExportTest test and dump file naming.
No-op change to highlight when video and/or audio are transmuxed and
reorder the methods.

PiperOrigin-RevId: 540567375
2023-06-19 16:14:38 +01:00
tonihei
51fb72b00d Replace deprecated NullableType from checkerframework with our own one
The existing NullableType has been deprecated 5 years ago and causes
crashes in Kotlin apps because Kotlin doesn't recognize this annotation
as a nullable type annotation.

While we can't align on a single @Nullable annotation yet, we can at
least replace this one by JSR305's @Nonnull(MAYBE) as it fulfils all
requirements, including full Kotlin compatiblity. To avoid the
cumbersome name, we can redefine it as our own @NullableType
annotation. (We can't use @Nullable to avoid name clashes with the main
@Nullable annotation from AndroidX)

Issue: google/ExoPlayer#6792
PiperOrigin-RevId: 540497469
2023-06-19 16:08:20 +01:00
sheenachhabra
d0eda433ea Replace CreationTime class with Mp4TimestampData class
PiperOrigin-RevId: 540257484
2023-06-14 20:42:36 +01:00
sheenachhabra
53c174f047 Add support for passing custom metadata via transformer
Changes included:
1. Enable MP4 extractor to read all types of metadata.
2. Allow passing String and Float metadata via Transformer.

Reference to QuickTime spec: https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/Metadata/Metadata.html#//apple_ref/doc/uid/TP40000939-CH1-SW21

PiperOrigin-RevId: 538783982
2023-06-09 13:51:15 +00:00
kimvde
3ba8f6dd8f Remove unnecessary thread in AssetLoader tests
Also queue textures from a different thread in TextureAssetLoader, to
have a behaviour closer to reality.

PiperOrigin-RevId: 538473089
2023-06-08 12:41:30 +00:00
sheenachhabra
7e14811e25 Add support for passing creation time via InAppMuxer
PiperOrigin-RevId: 538175466
2023-06-06 18:12:51 +00:00
kimvde
f4d1a6c453 Transmux video if rotation is only effect applied
PiperOrigin-RevId: 535554628
2023-05-26 15:14:52 +00:00
tofunmi
a19e07c4d2 Move OnInputFrameProcessedListener into it's own file
PiperOrigin-RevId: 535183521
2023-05-25 15:09:36 +00:00