Commit graph

609 commits

Author SHA1 Message Date
ivanbuper
873d485056 Rollback of e88d6fe459
PiperOrigin-RevId: 675525508
2024-09-17 05:25:07 -07:00
sheenachhabra
47d45a82ca Change the default value of lastSampleDurationBehavior
to
LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS

This CL also combines LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER
and LAST_SAMPLE_DURATION_BEHAVIOR_DUPLICATE_PREVIOUS.

The reason for combining the two enums is that, when the option
to use END_OF_STREAM_BUFFER is selected and if the EOS buffer is
not provided then the muxer anyways fallbacks to duplicate
duration behavior.

The last sample with 0 durations seems less useful so
change the default behavior to non-zero duration.
This will also match the behavior with MediaMuxer.

PiperOrigin-RevId: 675189932
2024-09-16 10:02:10 -07:00
ivanbuper
e88d6fe459 Fix truncation error acumulation for Sonic's resampling algorithm
Sonic would accumulate truncation errors on float to int conversions
that caused the final output sample count to drift noticeably, by
hundreds of samples on streams of a few minutes of length. The fix now
keeps track of the truncation error and compensates for it.

Other small fixes include eliminating lossy operations (e.g. int
division) and using doubles instead of floats for resampling where
helpful.

This CL also introduces `SonicParameterizedTest`, which helps test
resampling on an arbitrary number of randomly generated parameters,
with random sample data. `SonicParameterizedTest` uses `BigDecimal`s
for calculating sample count values, as to avoid precision issues with
large sample counts.

PiperOrigin-RevId: 673852768
2024-09-12 08:14:25 -07:00
rohks
f133e8d1f2 Fix preroll sample handling for non-keyframe media start positions
When processing edit lists in MP4 files, the media start position may be a non-keyframe. To ensure proper playback, the decoder must preroll to the preceding keyframe, as these preroll samples are essential for decoding but are not rendered.

Issue: google/ExoPlayer#1659

#cherrypick

PiperOrigin-RevId: 673457615
2024-09-11 11:02:39 -07:00
kimvde
8271a5f920 Rename CompositingVideoSinkProvider and PreviewAudioPipeline
The components are mirror components for video and audio so they should
have a matching name

PiperOrigin-RevId: 673357081
2024-09-11 05:49:35 -07:00
dancho
1c61fbadf7 Downscale bitmaps during decoding in Transformer
Limit input image size in Transformer to be less than 4096x4096.
For very large images, this can reduce memory usage substantially,
and stays away from `GL_MAX_TEXTURE_SIZE` - often 4096

PiperOrigin-RevId: 670555939
2024-09-03 07:53:13 -07:00
kimvde
af61c03e09 Move ExoPlayerEffectPlaybackSeekTest out of performance directory
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on all physical devices.

PiperOrigin-RevId: 670505992
2024-09-03 04:54:12 -07:00
kimvde
af2f9cb37f Increase ExoplayerEffectPlaybackSeekTest time out
This test will be moved so that it is run on emulators, which are
generally slower to run.

PiperOrigin-RevId: 670495551
2024-09-03 04:13:24 -07:00
sheenachhabra
95f69b649d Use InAppMuxer in tests for API < 25
MediaMuxer doesn't support B-frame before API 25.

The fix is added only in those test which appeared in triage failure.
It can be added to other tests as they are discovered.

PiperOrigin-RevId: 670480966
2024-09-03 03:18:01 -07:00
dancho
207684ca66 Prototype frame extraction based on analyzer mode
Make ByteBufferGlEffect public.
Build a speed test, and end to end test that verify
frames can be copied to CPU-accessible ByteBuffer

PiperOrigin-RevId: 670213343
2024-09-02 07:34:33 -07:00
kimvde
748e4e5230 Fix hanging ExoPlayerPlaybackSeekTest
The test uses a video renderer to count the number of frames and could
time out if some frames were dropped.

PiperOrigin-RevId: 670181693
2024-09-02 05:21:01 -07:00
kimvde
3587afc9d7 Move CompositionPlayerSeekTest out of performance directory
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on more than 1 physical device.

PiperOrigin-RevId: 668859017
2024-08-29 03:14:12 -07:00
kimvde
8367e420ad Import correct nullable annotation
PiperOrigin-RevId: 668521379
2024-08-28 10:54:29 -07:00
claincly
36d61000fd Reorder the assetions in tests
So it's more clear what's actually broke in the error log.

PiperOrigin-RevId: 667964963
2024-08-27 06:43:58 -07:00
kimvde
4ba9f59492 Improve logic to wait for a seek in CompositionPlayerSeekTest
The logic was assuming that the shader program was only flushed when
seeking. This is true if a single renderer is used but, with multiple
renderers, the shader program can be flushed at the transition.

This change is necessary to make the test pass for prewarming because 2
video renderers will be used in that case.

PiperOrigin-RevId: 666215967
2024-08-22 00:15:30 -07:00
kimvde
059ad62377 Add timeout to CountDownLatch awaits
This is to avoid tests hanging forever

PiperOrigin-RevId: 665826507
2024-08-21 05:02:18 -07:00
kimvde
9b69690856 Move some tests from CompositionPlaybackTest out of performance
directory

PiperOrigin-RevId: 665202382
2024-08-20 00:13:21 -07:00
claincly
677f8ad9f4 Fix edge case: no frame between trim position and the next sync sample
PiperOrigin-RevId: 664841893
2024-08-19 09:32:32 -07:00
kimvde
cd0a7e1143 Document why playback tests are in the performance package
PiperOrigin-RevId: 664792272
2024-08-19 07:14:32 -07:00
claincly
92635342fb Move the test of removeAudio to robolectric
The audio mixing is not deterministic on real device testing because of
threading, but runs deterministically on robolectric tests

PiperOrigin-RevId: 662925912
2024-08-14 08:23:22 -07:00
dancho
879771ded2 Add an experimental flag for renderer dynamic scheduling
Use ExoPlayer dynamic scheduling to reduce the render() interval for
older API devices where `DefaultCodec.getMaxPendingFrameCount()` is set
to 1 in order to prevent frame drops.

Controlled via API on DefaultDecoderFactory.

Add TransformerForegroundSpeedTest that mimics transcoding while the app
is in foreground.

PiperOrigin-RevId: 662925764
2024-08-14 08:20:21 -07:00
claincly
62da288caf Support removing video in previewing
Video playback will be disabled if *any* `EditedMediaItem` removes video. This
is consistent with Transformer.

PiperOrigin-RevId: 662093484
2024-08-12 08:28:16 -07:00
claincly
46eeabb877 Support setRemoveAudio in CompositionPlayer
PiperOrigin-RevId: 662063725
2024-08-12 06:34:44 -07:00
dancho
931b0e25f1 Add a DefaultDecoderFactory option to configure operating rate
This has the largest impact during operations with no encoder, such as
frame extraction. Add a matching performance test.

PiperOrigin-RevId: 661220044
2024-08-09 05:03:02 -07:00
claincly
5165d7df68 Rollback of ffc45820b9
PiperOrigin-RevId: 659520675
2024-08-05 05:31:46 -07:00
dancho
a79b80fcee Support setting H.264 level
Try to follow recommendations in
https://developer.android.com/media/optimize/sharing
more closely

For maximum compatibility the H.264 level should be less than or equal to 4.1

PiperOrigin-RevId: 658755139
2024-08-02 05:06:53 -07:00
dancho
a98a37aa75 Initial Frame Extractor Transformer Factory
Package-private until API is more useable.

Similar to frame analyzer mode: uses ImageReader instead of an encoder,
and no muxer.

PiperOrigin-RevId: 658446675
2024-08-01 10:21:10 -07:00
dancho
ddc86686b7 Experimental flag to limit the number of frames in encoder
Transformer.experimentalSetMaxFramesInEncoder controls max number
of frames in encoder.

VideoFrameProcessor now allows delayed releasing of frames to Surface,
while still using the original presentation time.

VideoSampleExporter can now configure video graphs to not render frames
automatically to output surface.

VideoSampleExporter.VideoGraphWrapper tracks how many frames are ready
to be rendered to Surface, and how many frames are already in-use by encoder.

PiperOrigin-RevId: 658429969
2024-08-01 09:33:16 -07:00
kimvde
ffc45820b9 Move playback tests outside of performance directory
PiperOrigin-RevId: 658055853
2024-07-31 10:29:39 -07:00
dancho
6e678e511b Enable experimentalRepeatInputBitmapWithoutResampling
Speed up image to video export by default.

PiperOrigin-RevId: 657958037
2024-07-31 04:40:29 -07:00
kimvde
ca5a26a409 Add frame count tests for preview
This is to ensure prewarming doesn't introduce any regression

PiperOrigin-RevId: 657559693
2024-07-30 06:04:41 -07:00
tofunmi
685ea1e616 create and use SpeedProviderMediaPeriod in CompositionPlayer
PiperOrigin-RevId: 655945332
2024-07-25 07:21:31 -07:00
kimvde
3211f38ebc Add tests for image seeking in CompositionPlayer
This is to make sure prewarming won't introduce any regression when it
will be implemented.

PiperOrigin-RevId: 655879558
2024-07-25 02:27:45 -07:00
dancho
7103f21da9 Use WORKING_COLOR_SPACE_DEFAULT in multi-sequence compositions
Build upon Transformer.videoFrameProcessorFactory in MultipleInputVideoGraph
Check that Transformer.videoFrameProcessorFactory is DefaultVideoFrameProcessor
for multi-input video

Fixes https://github.com/androidx/media/issues/1509

PiperOrigin-RevId: 655232381
2024-07-23 11:14:12 -07:00
dancho
9a42d03466 Enable experimentalAdjustSurfaceTextureTransformationMatrix
PiperOrigin-RevId: 655231134
2024-07-23 11:11:14 -07:00
dancho
f68cf30791 Change ExportTest assertion to allow mismatching level
Due to differences in MediaCodec behavior between minor Android
versions, this test was flaky.

PiperOrigin-RevId: 652841512
2024-07-16 07:55:34 -07:00
sheenachhabra
d747f38f59 Skip TransformerPauseResumeTest on vivo 1901
The process crashes unexpectedly on vivo devices.
When test is run individually it completes successfully.

PiperOrigin-RevId: 652496852
2024-07-15 08:56:37 -07:00
Googler
1bb8d5f956 Update internal reference.
PiperOrigin-RevId: 652448337
2024-07-15 05:34:32 -07:00
andrewlewis
4b7cc80593 Handle muxing with timestamps offset from zero in wrapper
Sources (for example media projection) can populate the `Surface` from
`SurfaceAssetLoader` with timestamps that don't start from zero. But
`MuxerWrapper` assumes the latest sample timestamp can be used as the duration
when it calculates average bitrate and notifies its listener.

This can cause a crash because the calculated average bitrate can be zero if
the denominator duration is large enough.

Use the max minus first sample timestamp across tracks instead to get the
duration.

Side note: the large timestamps from the surface texture when using media
projection arrive unchanged (apart from conversion from ns to us) in effect
implementations and in the muxer wrapper (and are passed to the underlying
muxer). The outputs of media3 muxer and the framework muxer are similar.

PiperOrigin-RevId: 652422674
2024-07-15 03:30:26 -07:00
dancho
09239a8a55 Add tests for releasing DefaultVideoFrameProcessor output surface
Add parametrized tests for DefaultVideoFrameProcessor that ensure
output EGL surface can be released, and deallocated.

PiperOrigin-RevId: 651712624
2024-07-12 03:07:20 -07:00
claincly
1ba2d98fce Remove extra comment line
PiperOrigin-RevId: 651367551
2024-07-11 05:06:17 -07:00
claincly
81f5a5f5f3 Set a longer muxer timeout on emulators
This is because the force EOS workaround for videos is longer than the original
muxer timeout. This way the apps depending on Transformer won't have to manually
set the muxer timeout on emulators.

PiperOrigin-RevId: 651355755
2024-07-11 04:12:59 -07:00
andrewlewis
74c06dc2f4 Add SurfaceAssetLoader
This supports queueing input to Transformer via a `Surface`.

PiperOrigin-RevId: 650318396
2024-07-08 11:33:05 -07:00
claincly
d0a29400ea Fix test failuer on real-device
In the previous CL, muxer timeout is set to the surface timeout, which is
incorrect.

PiperOrigin-RevId: 650203505
2024-07-08 04:40:22 -07:00
claincly
b4722ef1ea Make timeout longer for emulators
Some test is flaky because frame processing is quite slow and triggered this
time out.

PiperOrigin-RevId: 650191068
2024-07-08 03:47:15 -07:00
claincly
ccdc0ffc27 Use us (microsecond) API.
PiperOrigin-RevId: 649058648
2024-07-03 07:00:18 -07:00
claincly
ce8ab84b7c Add test that covers clipping all videos in a sequence
PiperOrigin-RevId: 649048322
2024-07-03 06:13:08 -07:00
claincly
b9d101f090 Use a resolution that should be encode-able on all devices
The image in the test has a resolution of 1x1 which some device will reject.

PiperOrigin-RevId: 649039791
2024-07-03 05:32:56 -07:00
dancho
91bf3d1da1 Use software decoder in VideoDecodingWrapper
Hardware decoder on some devices fails to write 1920x1080 YUV_420_888 buffers into an ImageReader. This change allows us to remove skipCalculateSsim device workaround in ExportTest.java.

VideoDecodingWrapper now uses media3.MediaExtractorCompat: necessary for parsing of MediaFormat#KEY_CODECS_STRING and decoder capabilities check

PiperOrigin-RevId: 648726721
2024-07-02 08:29:36 -07:00
dancho
9939d77d14 Change test to output 16x width and height
PiperOrigin-RevId: 647716870
2024-06-28 10:03:48 -07:00