This CL adds `SonicTest` and `RandomParameterizedSonicTest` as
initial basic unit testing for `Sonic.java`. The tested scenarios
do not necessarily verify a correct implementation of Sonic, but rather
hope to catch any behaviour change from the current implementation.
The change includes a small fix for a lossy simplification and also
checks whether the output sample count matches the expected drift from
the truncation accumulation error present in Sonic's resampler. This is
important as pre-work for fixing issues with unexpected durations within
`SonicAudioProcessor` and `SpeedChangingAudioProcessor` that cause AV
sync issues for speed changing effects.
This is a partial roll forward of e88d6fe459, which was rolled back in
873d485056.
PiperOrigin-RevId: 677756854
Last buffer was not flipped, so was writing the garbage data between
limit and capacity, rather than the actual data between position and
limit.
As a result, all PCM audio dump files need updating.
PiperOrigin-RevId: 676452990
This is no longer needed now our `compileSdk` implies a new-enough AGP
which does this out-lining automatically via R8. See also
https://issuetracker.google.com/345472586#comment7
There's no plan to remove the `ApiXXX` classes, but no new ones need
to be added.
PiperOrigin-RevId: 675940634
This is to allow not setting the MediaFormat OPERATING_RATE and PRIORITY
altogether. The current behvaiour, if left the value `UNSET`, it'll apply the
our optimizations, but apps might want to disable this optimization.
PiperOrigin-RevId: 675923909
After the change in a879bc2154, the Sequence won't have repeated
EditedMediaItems. Thus if the sequence is looping, the last EditedMediaItems
in the Sequence object might not corresponds to the last item in the "logical"
sequence.
PiperOrigin-RevId: 675912197
The muxer might not have accepted the first sample, if it
is waiting for audio track.
This bug causes issue when
1. VideoSampleExporter gives first sample (timestamp = 0) to the muxer.
2. Muxer does not write it because its waiting for audio track.
3. The video pipleline has processed all the sample and they are ready
to be consumed.
4. VideoSampleExporter fetches the next available sample from encoder (which is still with timestamp = 0) but it changes its timestamp to last timestamp because VideoSampleExporter thinks it has muxed the sample at timestamp zero, but in reality it hasn't. This is because the flag `hasMuxedTimestampZero` is set when queueing the input, rather than actually muxing the input.
This scenario can happen when video is processed much faster than
the audio.
PiperOrigin-RevId: 675565603
to
LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS
This CL also combines LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER
and LAST_SAMPLE_DURATION_BEHAVIOR_DUPLICATE_PREVIOUS.
The reason for combining the two enums is that, when the option
to use END_OF_STREAM_BUFFER is selected and if the EOS buffer is
not provided then the muxer anyways fallbacks to duplicate
duration behavior.
The last sample with 0 durations seems less useful so
change the default behavior to non-zero duration.
This will also match the behavior with MediaMuxer.
PiperOrigin-RevId: 675189932
The tests were flaky because CompositionPlayer registers audio sequences'
formats to the audio pipeline in first-come-first-serve order. With the change
in bc8d82355f, the audio format is deterministic.
The test is turned off in 060356ea00
PiperOrigin-RevId: 674261801
Sonic would accumulate truncation errors on float to int conversions
that caused the final output sample count to drift noticeably, by
hundreds of samples on streams of a few minutes of length. The fix now
keeps track of the truncation error and compensates for it.
Other small fixes include eliminating lossy operations (e.g. int
division) and using doubles instead of floats for resampling where
helpful.
This CL also introduces `SonicParameterizedTest`, which helps test
resampling on an arbitrary number of randomly generated parameters,
with random sample data. `SonicParameterizedTest` uses `BigDecimal`s
for calculating sample count values, as to avoid precision issues with
large sample counts.
PiperOrigin-RevId: 673852768
When processing edit lists in MP4 files, the media start position may be a non-keyframe. To ensure proper playback, the decoder must preroll to the preceding keyframe, as these preroll samples are essential for decoding but are not rendered.
Issue: google/ExoPlayer#1659
#cherrypick
PiperOrigin-RevId: 673457615
In case of negative PTS workaround, instead of disallowing all cases where samples are not in presentation order, we now validate that the first sample's presentation time is the smallest. This adjustment allows for correctly applying an offset to ensure all samples have a presentation time >= 0.
#cherrypick
PiperOrigin-RevId: 673434793
Limit input image size in Transformer to be less than 4096x4096.
For very large images, this can reduce memory usage substantially,
and stays away from `GL_MAX_TEXTURE_SIZE` - often 4096
PiperOrigin-RevId: 670555939
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on all physical devices.
PiperOrigin-RevId: 670505992
MediaMuxer doesn't support B-frame before API 25.
The fix is added only in those test which appeared in triage failure.
It can be added to other tests as they are discovered.
PiperOrigin-RevId: 670480966
Make ByteBufferGlEffect public.
Build a speed test, and end to end test that verify
frames can be copied to CPU-accessible ByteBuffer
PiperOrigin-RevId: 670213343
Disable assertions and make sure playback passes.
The flake is caused by having different sequences starting with MediaItems of
different audio format, and it's undefined behaviour as to which one CompositionPlayer chooses to use.
PiperOrigin-RevId: 670195113
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on more than 1 physical device.
PiperOrigin-RevId: 668859017
The logic was assuming that the shader program was only flushed when
seeking. This is true if a single renderer is used but, with multiple
renderers, the shader program can be flushed at the transition.
This change is necessary to make the test pass for prewarming because 2
video renderers will be used in that case.
PiperOrigin-RevId: 666215967
The audio mixing is not deterministic on real device testing because of
threading, but runs deterministically on robolectric tests
PiperOrigin-RevId: 662925912
Use ExoPlayer dynamic scheduling to reduce the render() interval for
older API devices where `DefaultCodec.getMaxPendingFrameCount()` is set
to 1 in order to prevent frame drops.
Controlled via API on DefaultDecoderFactory.
Add TransformerForegroundSpeedTest that mimics transcoding while the app
is in foreground.
PiperOrigin-RevId: 662925764
App users can choose arbitrary data that might not be
anticipated by developers. Transformer shouldn't `checkState` based on
media data or file type -- report an error for unsupported data instead.
Public API change `ImageAssetLoader` needs to parse MIME type and now accepts
`Context` as parameter.
PiperOrigin-RevId: 660762459
Package-private until API is more useable.
Similar to frame analyzer mode: uses ImageReader instead of an encoder,
and no muxer.
PiperOrigin-RevId: 658446675
Transformer.experimentalSetMaxFramesInEncoder controls max number
of frames in encoder.
VideoFrameProcessor now allows delayed releasing of frames to Surface,
while still using the original presentation time.
VideoSampleExporter can now configure video graphs to not render frames
automatically to output surface.
VideoSampleExporter.VideoGraphWrapper tracks how many frames are ready
to be rendered to Surface, and how many frames are already in-use by encoder.
PiperOrigin-RevId: 658429969
Add parameterized test for codecs supported by InAppMuxer.
Split TransformerWithInAppMuxerEndToEndParameterizedTest tests
between parameterized and non parameterized
This will be helpful for adding more tests which need not to
be parameterized.
PiperOrigin-RevId: 658353532
This is following a renaming of registerInputFrame to handleInputFrame.
- queueInputBitmap is renamed to handleInputBitmap for consistency with
handleInputFrame.
- registerInputStream is renamed to onInputStreamChanged for consistency
with media3 method names.
PiperOrigin-RevId: 655529699
Build upon Transformer.videoFrameProcessorFactory in MultipleInputVideoGraph
Check that Transformer.videoFrameProcessorFactory is DefaultVideoFrameProcessor
for multi-input video
Fixes https://github.com/androidx/media/issues/1509
PiperOrigin-RevId: 655232381
This test is flaky at p4head, because CompositionPlayer has no logic to
ensure a specific input is used to configure the audio graph. Depending
on which sequence registers input first, it changes the output format
of the media.
By setting effects on the 2nd sequence, both inputs are the same format
and this flakiness is avoided in the test.
PiperOrigin-RevId: 653582441
-----
Context:
* Each sequence is wrapped as a single MediaSource, each being played
by an underlying ExoPlayer.
* Repeat mode is typically implemented in Players as a seek to the next
item in the playlist.
-----
This CL:
Repeat mode is triggered by listening for when the main input player
sees a play when ready change due to the end of the media item.
There is a slight delay at the end of the playback, before it repeats.
Setting repeat mode on the underlying players addresses this, but means
that the players will seek without waiting for the CompositionPlayer,
and as such previewAudioPipeline does not get the correct signals
around blocking/flushing.
PreviewAudioPipeline - The seek position can validly be C.TIME_UNSET,
however preview pipeline did not handle this case.
CompositionPlayer getContentPosition is given (through a lambda) as a
supplier to the State object, which means any comparisons between
previous/new state for this value does not work. In SimpleBasePlayer,
there is logic to use the positionDiscontinuityPositionUs for the
position change (see getPositionInfo called from
updateStateAndInformListeners), however this logic is not considered in
getMediaItemTransitionReason, so a condition needed to be added for
this case.
-----
Tests:
* Dump files clearly show the position and data is repeated.
* Assertions on the reasons for transitions or position
discontinuities.
PiperOrigin-RevId: 653210278
Sources (for example media projection) can populate the `Surface` from
`SurfaceAssetLoader` with timestamps that don't start from zero. But
`MuxerWrapper` assumes the latest sample timestamp can be used as the duration
when it calculates average bitrate and notifies its listener.
This can cause a crash because the calculated average bitrate can be zero if
the denominator duration is large enough.
Use the max minus first sample timestamp across tracks instead to get the
duration.
Side note: the large timestamps from the surface texture when using media
projection arrive unchanged (apart from conversion from ns to us) in effect
implementations and in the muxer wrapper (and are passed to the underlying
muxer). The outputs of media3 muxer and the framework muxer are similar.
PiperOrigin-RevId: 652422674
This should have no influence on app behavior and other policies
and just allows code to depend on new API 35 platform symbols.
PiperOrigin-RevId: 652414026
Also promote all H.265 constants to be public in `NalUnitUtil` with
`H265_` prefixes, for consistency.
A lot of these names are used in h.265 too (and `NalUnitUtil` handles
both), but with different values, so this rename aims to avoid
accidentally using an h.264 value in an h.265 context.
PiperOrigin-RevId: 651774188
