Commit graph

332 commits

Author SHA1 Message Date
kimvde
048d71e392 Change muxer video duration unit to microseconds
PiperOrigin-RevId: 644402109
2024-06-18 08:56:26 -07:00
samrobinson
06e95ad2fb Reduce flakiness of getProgress tests around percentage values.
There is no requirement for the first progress value to be 0, if
progress is made instantly.

PiperOrigin-RevId: 642245457
2024-06-11 06:14:31 -07:00
samrobinson
6bb3af527a Add support for setting and getting volume for composition preview.
PiperOrigin-RevId: 641822822
2024-06-10 01:46:10 -07:00
tofunmi
cd5a638e43 MuxerWrapper: Fix spelling mistake in method name
PiperOrigin-RevId: 640838741
2024-06-06 03:32:19 -07:00
samrobinson
67c19f87a7 Update FILE_UNKNOWN_DURATION to a longer fmp4 (~15s),
Removes the flakiness of
MediaItemExportTest.getProgress_unknownDuration_returnsConsistentStates
by using a longer input asset, such that ExoPlayer does not determine
the duration of the media.

PiperOrigin-RevId: 640502470
2024-06-05 06:19:11 -07:00
samrobinson
2c916dc306 Make progress tests stricter.
These changes are possible because getProgress is no longer a blocking
operation on transformer.

* Tests call getProgress after every looper message executed.
* Use longer media assets for getProgress tests to give more progress
  intervals.
* Remove conditional assertions.

PiperOrigin-RevId: 639734368
2024-06-03 04:42:36 -07:00
dancho
d3fa33250b Fix a race condition in AudioGraphInput
AudioGraphInput.onMediaItemChanged is called on input thread. Pending
media item changes are processed on processing thread, inside calls to
getOutput().
This change allows multiple pending media item changes to be enqueued,
and processed in sequence.

PiperOrigin-RevId: 638995291
2024-05-31 04:53:13 -07:00
samrobinson
21eb482baf Add an experimental analyzer mode to Transformer.
PiperOrigin-RevId: 637926059
2024-05-28 09:31:40 -07:00
dancho
a74076f691 Trigger silence generation when end of stream is encountered
This change avoids a muxer deadlock when:
1. Sequence of items
2. First item has audio track that is shorter than video
3. Audio finishes, and muxer refuses to write more than 500ms of video
   consecutively.

SequenceAssetLoader fails to progress to the second item. A muxer
deadlock is possible when the audio of the first item finishes,
audio end-of-stream is not propagated through AudioGraph, and muxer blocks
video, preventing SequenceAssetLoader to move to the next item in sequence.

By triggering silence generation early as soon as audio EOS is
encountered, we ensure SequenceAssetLoader can progress to the next item.

PiperOrigin-RevId: 636179966
2024-05-22 08:44:28 -07:00
dancho
b9ec24a269 Fix AV sync for sequences with audio track shorter than video
For each item, AudioGraphInput now pads the input audio track with silence
to the duration given in onMediaItemChanged.

Possibly resolves Issue: androidx/media#921 .

PiperOrigin-RevId: 634753721
2024-05-17 06:54:39 -07:00
ibaker
0e5a5e0294 Publish CompositionPlayer for playing compositions
This class is not ready for production app usage yet, so it is still
marked `@RestrictTo(LIBRARY_GROUP)` for now. Apps can experiment with it
in a non-prod context by suppressing the associated lint error.

* Issue: androidx/media#1014
* Issue: androidx/media#1185
* Issue: androidx/media#816

PiperOrigin-RevId: 633921353
2024-05-15 05:55:08 -07:00
Googler
55b9c391e8 Box: Implement ctts box
Add ctts box implementation to handle muxing B-frame videos.
Add method convertPresentationTimestampsToCompositionOffset to
provide sample offsets. Return empty ctts box in case of video
does not contain B-frame. Add ctts box to MoovStructure to handle
muxing the video containing B-frames.

PiperOrigin-RevId: 633537106
2024-05-14 04:32:04 -07:00
sheenachhabra
3a3145521b Replace Transformer.Muxer interface with Muxer.Muxer
PiperOrigin-RevId: 633193701
2024-05-13 06:54:19 -07:00
samrobinson
9942255894 Allow any pcm encoding raw input pre-effects in AudioGraphInput.
AudioGraphInput now accepts a range of inputs, as long as the effects
provided modify the audio to be int 16.

As part of this, add the workaround to DefaultCodec to ensure pcm
encoding is correct, and remove parameterized tests that are not valid.

PiperOrigin-RevId: 631404152
2024-05-07 06:39:09 -07:00
sheenachhabra
24305c043e Remove forCancellation flag from Transformer/Muxer.release() method
PiperOrigin-RevId: 630337930
2024-05-03 03:18:24 -07:00
sheenachhabra
bc77cef403 Rollback of 08abc964ab
PiperOrigin-RevId: 629382753
2024-04-30 05:41:45 -07:00
ibaker
5666678d8a Use Dumper.addTime in more places
This ensures that `C.TIME_UNSET` is more clear in dump files. Some of
these call-sites will **never** pass `C.TIME_UNSET`, but it seems
clearest to always use `addTime` and maybe it will ensure when these
sites are copied in future, `addTime` will be used in the new location
too.

PiperOrigin-RevId: 628363183
2024-04-26 04:20:56 -07:00
Googler
08abc964ab Box: Implement ctts box
Add ctts box implementation to handle muxing B-frame videos.
Add method convertPresentationTimestampsToCompositionOffset to
provide sample offsets. Return empty ctts box in case of video
does not contain B-frame. Add ctts box to MoovStructure to handle
muxing the video containing B-frames.

PiperOrigin-RevId: 628346257
2024-04-26 02:57:53 -07:00
kimvde
c87940eb27 AudioGraph refactoring: configure mixer and add sources in one place
PiperOrigin-RevId: 628076773
2024-04-25 08:02:00 -07:00
kimvde
abd0ff7352 Remove configure methods
Pass the parameters to the constructor instead

PiperOrigin-RevId: 627420999
2024-04-23 09:59:56 -07:00
sheenachhabra
03a041c452 Pass BufferInfo in writeSampleData() method in Transformer/Muxer.java
This is to eventually replace Transformer/Muxer.java with Muxer/Muxer.java

PiperOrigin-RevId: 627043808
2024-04-22 08:23:34 -07:00
sheenachhabra
86ef571644 Replace trackIndex with TrackToken in Transformer
This is to eventually replace Transformer/Muxer.java with Muxer/Muxer.java

PiperOrigin-RevId: 627027254
2024-04-22 07:20:38 -07:00
sheenachhabra
4aa2fb883f Add RawAssetLoader and deprecate TextureAssetLoader
All the changes are based on existing implementations
and their related tests.

No new functionality is being added.

PiperOrigin-RevId: 625756836
2024-04-17 11:48:43 -07:00
sheenachhabra
794900bfb8 Update addMetadata method in Transformer/Muxer.java
This is to align the interface with Muxer/Muxer.java and
then finally replace it.

PiperOrigin-RevId: 625283725
2024-04-16 04:40:13 -07:00
kimvde
fa0fb38ca6 Handle clip start position equal to end position in Transformer
After this CL, Transformer will throw if the clipping start and end
positions are the same because MediaMuxer doesn't support writing a
file with no samples. This should work once we default to the in-app
muxer.

Issue: androidx/media#1242
PiperOrigin-RevId: 624861950
2024-04-15 00:53:38 -07:00
samrobinson
81b9739bda Consolidate dump files for item effect vs composition effect.
PiperOrigin-RevId: 623161140
2024-04-09 07:33:38 -07:00
samrobinson
24b86e73cb Group together PCM audio samples by fixed size in CapturingMuxer dumps.
Reduces flakiness of tests that assert on PCM audio. Tests now have to
clearly choose how they want the capturing muxer to handle pcm audio.

Note that the only dump files that have changed are those that deal
with PCM audio (.wav, sowt, twos, silence). Because of the continuous
nature of PCM, timestamps are not part of the dump.

PiperOrigin-RevId: 623155302
2024-04-09 07:04:49 -07:00
kimvde
04a3889998 Reset maxPositionOfRemovedSources in DefaultAudioMixer
If the mixer is reset without resetting maxPositionOfRemovedSources and
then reused, the value of maxPositionOfRemovedSources can be outdated,
leading to an incorrect number of bytes being output by the mixer.

PiperOrigin-RevId: 619832502
2024-03-28 01:22:25 -07:00
tofunmi
65e94480f4 Transformer: add single-asset ultraHDR image to HLG HDR vid transcoding
PiperOrigin-RevId: 619138202
2024-03-26 04:06:41 -07:00
sheenachhabra
2eaece5ed9 Move setMaxDelayBetweenSamplesMs on Transformer builder
This property is transformer specific and does not belong to
muxer interface.
This is to eventually replace muxer interface in Transformer module with
muxer interface in Muxer module.

PiperOrigin-RevId: 618895836
2024-03-25 11:02:15 -07:00
sheenachhabra
16aac07bce Add support for processing 3 byte NAL start code in Mp4Muxer
Issue: androidx/media#725

Ideally the test to transmux a `ts` file (having 3 bytes NAL start code)
should go into `Mp4MuxerEndToEndTest`but `Mp4MuxerEndToEndTest` uses `MediaExtractor` and `MediaExtractor` is returning samples with `4 byte NAL start code` which will not exercise the newly added code path.
Hence the test is added in `TransformerWithInAppMuxerEndToEndTest` which
internally uses `media3 extractor` and feeds samples with `3 bytes NAL start code`
only.

PiperOrigin-RevId: 617985866
2024-03-21 15:49:12 -07:00
kimvde
6fc4f0263f Add methods required for seek to AudioGraph
Seeking will consist of the following steps:
- Block the AudioGraph input data.
- Flush the AudioGraph.
- Seek the ExoPlayers.
- Unblock the AudioGraph input data.

PiperOrigin-RevId: 617868124
2024-03-21 09:31:51 -07:00
Googler
415d779c62 Explicitly check that AudioGraph input is PCM 16
AudioGraph implicitly assumes that input will be PCM 16 bit, and not float.
Make the check explicit.

PiperOrigin-RevId: 616149131
2024-03-15 09:14:04 -07:00
kimvde
e40ce150bf Implement AudioGraph.flush()
When flushing the AudioGraph, the AudioMixer will be preserved but the
sources will be recreated. This is probably a bit less efficient but
makes the logic simpler. Indeed, if we had to keep the sources alive,
we would need to add a way to reconfigure them with a new timestamp for
seeking. We would also need to change the way sources are ended because
they are currently removed when they are ended. Also, it is acceptable
to have a small delay when seeking, which means that performance is less
critical than for playback.

PiperOrigin-RevId: 615775501
2024-03-14 07:49:07 -07:00
kimvde
3248b7962a Add clipFloatOutput parameter to AudioMixer
PiperOrigin-RevId: 615720340
2024-03-14 03:44:37 -07:00
huangdarwin
ca6031deab Update old method name.
getSupportedEncoderNamesForHdrEditing became getSupportedEncodersForHdrEditing at some point, but this comment wasn't updated before...

PiperOrigin-RevId: 615712707
2024-03-14 03:11:03 -07:00
sheenachhabra
668a172683 Rename MuxerWrapper.release() method
PiperOrigin-RevId: 615399319
2024-03-13 06:54:19 -07:00
kimvde
f02dc8e528 Implement AudioGraphInput.flush
PiperOrigin-RevId: 613563855
2024-03-07 06:33:02 -08:00
Googler
e175a772db Move AudioGraph effects to configure method
Move old configure behavior to private configureMixer.
registerInput() is now required after reset().

PiperOrigin-RevId: 612795418
2024-03-05 04:46:36 -08:00
tofunmi
bcfad4b3b4 Move hdrMode from defaultAssetLoaderFactory constructors to create()
Plumbing hdrMode through the default asset loader factory via the constructor is problematic because it breaks API boundaries. It means there is another way to set hdrMode outside of Composition.java and TransformationRequest.java, which is error prone and cause problems if someone an app starts customizing the assetloaderfactory. It also means custom asset loaders can't receive this information without hacking around.

The introduction of the composition-level settings class makes this approach easily extensible for other settings applied on the composition level but use in an individual asset level basis (e.g. ultraHDR support).

PiperOrigin-RevId: 611466920
2024-02-29 07:41:18 -08:00
Googler
6f28eeff31 Pick max H.264 level when trim optimizing
Instead of always starting with the transcoded H.264 level, take the maximum
from transcoded and transmuxed levels

PiperOrigin-RevId: 610759438
2024-02-27 08:27:38 -08:00
Googler
c3aec4f19d Comments for AudioGraph constructor
Add /*effects=*/ at callsites of AudioGraph constructor.

PiperOrigin-RevId: 610741692
2024-02-27 07:12:53 -08:00
Googler
0e0e1c4f1a Tests for composition audio effects
Add tests for the changes to AudioGraph that support composition audio effects

PiperOrigin-RevId: 610691768
2024-02-27 03:04:20 -08:00
Googler
477ace1be9 Composition audio effects
Implement composition-level audio effects in AudioGraph.

PiperOrigin-RevId: 610689632
2024-02-27 02:54:08 -08:00
tofunmi
5a892509f7 remove media/bitmap/input_images
move the images into the respective places in the file extension directory so they file the pre-existing structure

PiperOrigin-RevId: 609744673
2024-02-23 09:07:03 -08:00
sheenachhabra
284d17cb13 Add release reason argument in the MuxerWrapper.release() method
Also called `muxer.release()` in other tests where it is
appropriate.

PiperOrigin-RevId: 609736956
2024-02-23 08:34:04 -08:00
sheenachhabra
fd8f45b38e Add MdtaMetadataEntry constructor with default locale indicator
PiperOrigin-RevId: 609427529
2024-02-22 10:55:50 -08:00
Googler
dca7bae416 Allow mismatching H.264/AVC level for trimming or resuming
MediaCodec docs already allude to potentially mismatching H.264 level
between container and bitstream. Relax the initialization data check to
reflect this.

PiperOrigin-RevId: 608942322
2024-02-21 04:20:50 -08:00
sheenachhabra
a82d7e7098 Improve retrieveTrackFormat() java doc and remove catch block
PiperOrigin-RevId: 608939192
2024-02-21 04:11:39 -08:00
Googler
e43f96687c Fix failure to write negative PTS sample
Fixes an issue caused by no support for negative audio PTS and edit lists
in FrameworkMuxer, Android versions before 11

PiperOrigin-RevId: 607690507
2024-02-16 08:03:38 -08:00