Commit graph

633 commits

Author SHA1 Message Date
kimvde
9adb3aaf41 Transformer: add an entry point to disable automatic rotation
PiperOrigin-RevId: 686067527
2024-10-15 05:40:41 -07:00
sheenachhabra
407bd49ed5 Skip audio encoding bitrate setting test on API <= 23
The encoder output format on API 23 does not seem to contain bitrate,
hence the test fails.

PiperOrigin-RevId: 686047480
2024-10-15 04:15:38 -07:00
Googler
0100f1d902 Boxes: Add edit list box.
PiperOrigin-RevId: 685974308
2024-10-14 23:26:08 -07:00
sheenachhabra
a0ccd46653 Skip AACObjectHE encoding on faulty devices
These devices claim to have the AACObjectHE profile but the profile never gets applied.

PiperOrigin-RevId: 684786157
2024-10-11 04:18:52 -07:00
ibaker
2c46cea088 Use RoundingMode.DOWN in Util.scaleLargeTimestamp and friends
The implementation of these methods was updated from direct java integer
arithmetic in 885ddb167e.
In this change, `RoundingMode.FLOOR` was used to try and maintain
compatibility with java integer division. This was incorrect, because
java integer division uses `DOWN` (i.e. towards zero), rather than
`FLOOR` (i.e. towards negative infinity) semantics.

This change fixes the compatibility.

The dump file changes in this CL relate to tests that exercise edit
list behaviour. This involves manipulating negative timestamps, which
explains why they are impacted by this change.

PiperOrigin-RevId: 684013175
2024-10-09 06:24:26 -07:00
kimvde
52f08d46c2 Add motion photo support to Transformer
PiperOrigin-RevId: 683540867
2024-10-08 02:57:52 -07:00
ivanbuper
af922fbcb0 Fix truncation error accumulation on Sonic's time stretching algorithm
This CL also fixes EOS handling to account for not-yet-copied samples in
`remainingInputToCopyFrameCount`, which would throw off the final output
sample count calculation.

For testing, we allow a tolerance of 0.000017% drift between expected
and actual number of output samples. The value was obtained from running
100 iterations of `timeStretching_returnsExpectedNumberOfSamples()` and
calculating the average delta percentage between expected and actual
number of output samples. Roughly, this means a tolerance of 40 samples
on a 90 min mono stream @48KHz.

PiperOrigin-RevId: 683133461
2024-10-07 04:59:54 -07:00
dancho
7b08bedf2c Do not force EOS when decoder has produced all frames
The workaround in ExternalTextureManager.forceSignalEndOfStream
was being applied even when the decoder did output all frames, and only
the GL pipeline was slow.

This change ensures that workaround is not applied when the decoder
has already produced all output frames.

PiperOrigin-RevId: 680471587
2024-09-30 01:50:10 -07:00
claincly
e0e9f5b057 Increase the maxImage on ImageReader, hoping less flaky tests
PiperOrigin-RevId: 679528425
2024-09-27 04:03:15 -07:00
dancho
b9aed0a937 Have AndroidTestUtil.canEncode mirror VideoSampleExporter
Some devices can encode portrait 720x1080 but not landscape
1080x720. But VideoSampleExporter always prefers encoding
landscape. Have `assumeFormatsSupported` mirror sample exporter
logic more closely

PiperOrigin-RevId: 679495210
2024-09-27 02:01:05 -07:00
sheenachhabra
b6192f7a39 Add AudioEncoderSettings to Transformer
It allows clients to specify the audio encoding profile and bitrate.
This is similar to VideoEncoderSettings.

PiperOrigin-RevId: 679131963
2024-09-26 06:59:53 -07:00
Googler
020ce7765c Reduce rounding error and stts table entries.
To avoid rounding errors, set the `Rounding mode` of the `uvFromVu` and `vuFromUs` results to `HALF_UP`. This `Rounding mode` rounds numbers towards the "nearest neighbor" unless both neighbors are equidistant, in which case round up.

PiperOrigin-RevId: 679003943
2024-09-25 23:19:20 -07:00
claincly
0ea63e3fa6 Disable video only on the sequence that has items that disable video
This means we need a custom track selector for each SequencePlayer.

PiperOrigin-RevId: 678779216
2024-09-25 11:28:26 -07:00
kimvde
2dc32360d6 Remove usages of deprecated DefaultDecoderFactory constructor
The constructor was deprecated but usages were not migrated.

PiperOrigin-RevId: 678683678
2024-09-25 07:01:14 -07:00
kimvde
eaec7a4a61 Set MediaItem's image duration for image export
This field will become mandatory

PiperOrigin-RevId: 678656879
2024-09-25 05:23:27 -07:00
kimvde
f83d2b1392 Stop calling setDurationUs when image duration already set
The EditedMediaItem's duration is set to the MediaItem's image duration
by default.

PiperOrigin-RevId: 678597324
2024-09-25 02:02:30 -07:00
samrobinson
076eea283e Adding API and internal export support for audio gaps in sequences.
PiperOrigin-RevId: 678235289
2024-09-24 07:14:59 -07:00
kimvde
d8dc513431 Apply video composition effects in preview
PiperOrigin-RevId: 678207818
2024-09-24 05:46:33 -07:00
kimvde
c19d910f6b Fix EffectPlaybackPixelTest
The tests using createTimestampOverlay() were passing even if the effect
was removed, because the overlay was too small.

PiperOrigin-RevId: 678169395
2024-09-24 03:24:49 -07:00
samrobinson
b4436c523c Migrate Media3 EditedMediaItemSequence usages to the Builder.
Removed the unnecessary wrapping of items in an ImmutableList.

PiperOrigin-RevId: 677796662
2024-09-23 08:06:55 -07:00
ivanbuper
3d3ec85c12 Setup basic testing for Sonic and assert expected sample count drift
This CL adds `SonicTest` and `RandomParameterizedSonicTest` as
initial basic unit testing for `Sonic.java`. The tested scenarios
do not necessarily verify a correct implementation of Sonic, but rather
hope to catch any behaviour change from the current implementation.

The change includes a small fix for a lossy simplification and also
checks whether the output sample count matches the expected drift from
the truncation accumulation error present in Sonic's resampler. This is
important as pre-work for fixing issues with unexpected durations within
`SonicAudioProcessor` and `SpeedChangingAudioProcessor` that cause AV
sync issues for speed changing effects.

This is a partial roll forward of e88d6fe459, which was rolled back in
873d485056.

PiperOrigin-RevId: 677756854
2024-09-23 05:53:58 -07:00
kimvde
be4d31ba87 Prevent ExoPlayer dropping frames in EffectPlaybackPixelTest
Some tests check all the output frames and fail if there are frames
missing.

PiperOrigin-RevId: 677676143
2024-09-23 01:12:40 -07:00
kimvde
980f24d906 Avoid ImageReader frame drops in EffectPlaybackPixelTest
If this doesn't work, we could make this test resilient to frame drops
by only comparing the frames that weren't dropped.

PiperOrigin-RevId: 676294944
2024-09-19 00:18:46 -07:00
kimvde
2951a2599c Apply all video Composition effects to single sequence exports
PiperOrigin-RevId: 675933601
2024-09-18 04:18:55 -07:00
ivanbuper
873d485056 Rollback of e88d6fe459
PiperOrigin-RevId: 675525508
2024-09-17 05:25:07 -07:00
sheenachhabra
47d45a82ca Change the default value of lastSampleDurationBehavior
to
LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS

This CL also combines LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER
and LAST_SAMPLE_DURATION_BEHAVIOR_DUPLICATE_PREVIOUS.

The reason for combining the two enums is that, when the option
to use END_OF_STREAM_BUFFER is selected and if the EOS buffer is
not provided then the muxer anyways fallbacks to duplicate
duration behavior.

The last sample with 0 durations seems less useful so
change the default behavior to non-zero duration.
This will also match the behavior with MediaMuxer.

PiperOrigin-RevId: 675189932
2024-09-16 10:02:10 -07:00
ivanbuper
e88d6fe459 Fix truncation error acumulation for Sonic's resampling algorithm
Sonic would accumulate truncation errors on float to int conversions
that caused the final output sample count to drift noticeably, by
hundreds of samples on streams of a few minutes of length. The fix now
keeps track of the truncation error and compensates for it.

Other small fixes include eliminating lossy operations (e.g. int
division) and using doubles instead of floats for resampling where
helpful.

This CL also introduces `SonicParameterizedTest`, which helps test
resampling on an arbitrary number of randomly generated parameters,
with random sample data. `SonicParameterizedTest` uses `BigDecimal`s
for calculating sample count values, as to avoid precision issues with
large sample counts.

PiperOrigin-RevId: 673852768
2024-09-12 08:14:25 -07:00
rohks
f133e8d1f2 Fix preroll sample handling for non-keyframe media start positions
When processing edit lists in MP4 files, the media start position may be a non-keyframe. To ensure proper playback, the decoder must preroll to the preceding keyframe, as these preroll samples are essential for decoding but are not rendered.

Issue: google/ExoPlayer#1659

#cherrypick

PiperOrigin-RevId: 673457615
2024-09-11 11:02:39 -07:00
kimvde
8271a5f920 Rename CompositingVideoSinkProvider and PreviewAudioPipeline
The components are mirror components for video and audio so they should
have a matching name

PiperOrigin-RevId: 673357081
2024-09-11 05:49:35 -07:00
dancho
1c61fbadf7 Downscale bitmaps during decoding in Transformer
Limit input image size in Transformer to be less than 4096x4096.
For very large images, this can reduce memory usage substantially,
and stays away from `GL_MAX_TEXTURE_SIZE` - often 4096

PiperOrigin-RevId: 670555939
2024-09-03 07:53:13 -07:00
kimvde
af61c03e09 Move ExoPlayerEffectPlaybackSeekTest out of performance directory
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on all physical devices.

PiperOrigin-RevId: 670505992
2024-09-03 04:54:12 -07:00
kimvde
af2f9cb37f Increase ExoplayerEffectPlaybackSeekTest time out
This test will be moved so that it is run on emulators, which are
generally slower to run.

PiperOrigin-RevId: 670495551
2024-09-03 04:13:24 -07:00
sheenachhabra
95f69b649d Use InAppMuxer in tests for API < 25
MediaMuxer doesn't support B-frame before API 25.

The fix is added only in those test which appeared in triage failure.
It can be added to other tests as they are discovered.

PiperOrigin-RevId: 670480966
2024-09-03 03:18:01 -07:00
dancho
207684ca66 Prototype frame extraction based on analyzer mode
Make ByteBufferGlEffect public.
Build a speed test, and end to end test that verify
frames can be copied to CPU-accessible ByteBuffer

PiperOrigin-RevId: 670213343
2024-09-02 07:34:33 -07:00
kimvde
748e4e5230 Fix hanging ExoPlayerPlaybackSeekTest
The test uses a video renderer to count the number of frames and could
time out if some frames were dropped.

PiperOrigin-RevId: 670181693
2024-09-02 05:21:01 -07:00
kimvde
3587afc9d7 Move CompositionPlayerSeekTest out of performance directory
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on more than 1 physical device.

PiperOrigin-RevId: 668859017
2024-08-29 03:14:12 -07:00
kimvde
8367e420ad Import correct nullable annotation
PiperOrigin-RevId: 668521379
2024-08-28 10:54:29 -07:00
claincly
36d61000fd Reorder the assetions in tests
So it's more clear what's actually broke in the error log.

PiperOrigin-RevId: 667964963
2024-08-27 06:43:58 -07:00
kimvde
4ba9f59492 Improve logic to wait for a seek in CompositionPlayerSeekTest
The logic was assuming that the shader program was only flushed when
seeking. This is true if a single renderer is used but, with multiple
renderers, the shader program can be flushed at the transition.

This change is necessary to make the test pass for prewarming because 2
video renderers will be used in that case.

PiperOrigin-RevId: 666215967
2024-08-22 00:15:30 -07:00
kimvde
059ad62377 Add timeout to CountDownLatch awaits
This is to avoid tests hanging forever

PiperOrigin-RevId: 665826507
2024-08-21 05:02:18 -07:00
kimvde
9b69690856 Move some tests from CompositionPlaybackTest out of performance
directory

PiperOrigin-RevId: 665202382
2024-08-20 00:13:21 -07:00
claincly
677f8ad9f4 Fix edge case: no frame between trim position and the next sync sample
PiperOrigin-RevId: 664841893
2024-08-19 09:32:32 -07:00
kimvde
cd0a7e1143 Document why playback tests are in the performance package
PiperOrigin-RevId: 664792272
2024-08-19 07:14:32 -07:00
claincly
92635342fb Move the test of removeAudio to robolectric
The audio mixing is not deterministic on real device testing because of
threading, but runs deterministically on robolectric tests

PiperOrigin-RevId: 662925912
2024-08-14 08:23:22 -07:00
dancho
879771ded2 Add an experimental flag for renderer dynamic scheduling
Use ExoPlayer dynamic scheduling to reduce the render() interval for
older API devices where `DefaultCodec.getMaxPendingFrameCount()` is set
to 1 in order to prevent frame drops.

Controlled via API on DefaultDecoderFactory.

Add TransformerForegroundSpeedTest that mimics transcoding while the app
is in foreground.

PiperOrigin-RevId: 662925764
2024-08-14 08:20:21 -07:00
claincly
62da288caf Support removing video in previewing
Video playback will be disabled if *any* `EditedMediaItem` removes video. This
is consistent with Transformer.

PiperOrigin-RevId: 662093484
2024-08-12 08:28:16 -07:00
claincly
46eeabb877 Support setRemoveAudio in CompositionPlayer
PiperOrigin-RevId: 662063725
2024-08-12 06:34:44 -07:00
dancho
931b0e25f1 Add a DefaultDecoderFactory option to configure operating rate
This has the largest impact during operations with no encoder, such as
frame extraction. Add a matching performance test.

PiperOrigin-RevId: 661220044
2024-08-09 05:03:02 -07:00
claincly
5165d7df68 Rollback of ffc45820b9
PiperOrigin-RevId: 659520675
2024-08-05 05:31:46 -07:00
dancho
a79b80fcee Support setting H.264 level
Try to follow recommendations in
https://developer.android.com/media/optimize/sharing
more closely

For maximum compatibility the H.264 level should be less than or equal to 4.1

PiperOrigin-RevId: 658755139
2024-08-02 05:06:53 -07:00