This is no longer needed now our `compileSdk` implies a new-enough AGP
which does this out-lining automatically via R8. See also
https://issuetracker.google.com/345472586#comment7
There's no plan to remove the `ApiXXX` classes, but no new ones need
to be added.
PiperOrigin-RevId: 675940634
This is to allow not setting the MediaFormat OPERATING_RATE and PRIORITY
altogether. The current behvaiour, if left the value `UNSET`, it'll apply the
our optimizations, but apps might want to disable this optimization.
PiperOrigin-RevId: 675923909
After the change in a879bc2154, the Sequence won't have repeated
EditedMediaItems. Thus if the sequence is looping, the last EditedMediaItems
in the Sequence object might not corresponds to the last item in the "logical"
sequence.
PiperOrigin-RevId: 675912197
The muxer might not have accepted the first sample, if it
is waiting for audio track.
This bug causes issue when
1. VideoSampleExporter gives first sample (timestamp = 0) to the muxer.
2. Muxer does not write it because its waiting for audio track.
3. The video pipleline has processed all the sample and they are ready
to be consumed.
4. VideoSampleExporter fetches the next available sample from encoder (which is still with timestamp = 0) but it changes its timestamp to last timestamp because VideoSampleExporter thinks it has muxed the sample at timestamp zero, but in reality it hasn't. This is because the flag `hasMuxedTimestampZero` is set when queueing the input, rather than actually muxing the input.
This scenario can happen when video is processed much faster than
the audio.
PiperOrigin-RevId: 675565603
to
LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS
This CL also combines LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER
and LAST_SAMPLE_DURATION_BEHAVIOR_DUPLICATE_PREVIOUS.
The reason for combining the two enums is that, when the option
to use END_OF_STREAM_BUFFER is selected and if the EOS buffer is
not provided then the muxer anyways fallbacks to duplicate
duration behavior.
The last sample with 0 durations seems less useful so
change the default behavior to non-zero duration.
This will also match the behavior with MediaMuxer.
PiperOrigin-RevId: 675189932
The tests were flaky because CompositionPlayer registers audio sequences'
formats to the audio pipeline in first-come-first-serve order. With the change
in bc8d82355f, the audio format is deterministic.
The test is turned off in 060356ea00
PiperOrigin-RevId: 674261801
Sonic would accumulate truncation errors on float to int conversions
that caused the final output sample count to drift noticeably, by
hundreds of samples on streams of a few minutes of length. The fix now
keeps track of the truncation error and compensates for it.
Other small fixes include eliminating lossy operations (e.g. int
division) and using doubles instead of floats for resampling where
helpful.
This CL also introduces `SonicParameterizedTest`, which helps test
resampling on an arbitrary number of randomly generated parameters,
with random sample data. `SonicParameterizedTest` uses `BigDecimal`s
for calculating sample count values, as to avoid precision issues with
large sample counts.
PiperOrigin-RevId: 673852768
When processing edit lists in MP4 files, the media start position may be a non-keyframe. To ensure proper playback, the decoder must preroll to the preceding keyframe, as these preroll samples are essential for decoding but are not rendered.
Issue: google/ExoPlayer#1659
#cherrypick
PiperOrigin-RevId: 673457615
In case of negative PTS workaround, instead of disallowing all cases where samples are not in presentation order, we now validate that the first sample's presentation time is the smallest. This adjustment allows for correctly applying an offset to ensure all samples have a presentation time >= 0.
#cherrypick
PiperOrigin-RevId: 673434793
Limit input image size in Transformer to be less than 4096x4096.
For very large images, this can reduce memory usage substantially,
and stays away from `GL_MAX_TEXTURE_SIZE` - often 4096
PiperOrigin-RevId: 670555939
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on all physical devices.
PiperOrigin-RevId: 670505992
MediaMuxer doesn't support B-frame before API 25.
The fix is added only in those test which appeared in triage failure.
It can be added to other tests as they are discovered.
PiperOrigin-RevId: 670480966
Make ByteBufferGlEffect public.
Build a speed test, and end to end test that verify
frames can be copied to CPU-accessible ByteBuffer
PiperOrigin-RevId: 670213343
Disable assertions and make sure playback passes.
The flake is caused by having different sequences starting with MediaItems of
different audio format, and it's undefined behaviour as to which one CompositionPlayer chooses to use.
PiperOrigin-RevId: 670195113
These tests do not test performance. Moving them out of this directory
ensures they are run on emulators and on more than 1 physical device.
PiperOrigin-RevId: 668859017
The logic was assuming that the shader program was only flushed when
seeking. This is true if a single renderer is used but, with multiple
renderers, the shader program can be flushed at the transition.
This change is necessary to make the test pass for prewarming because 2
video renderers will be used in that case.
PiperOrigin-RevId: 666215967
The audio mixing is not deterministic on real device testing because of
threading, but runs deterministically on robolectric tests
PiperOrigin-RevId: 662925912
Use ExoPlayer dynamic scheduling to reduce the render() interval for
older API devices where `DefaultCodec.getMaxPendingFrameCount()` is set
to 1 in order to prevent frame drops.
Controlled via API on DefaultDecoderFactory.
Add TransformerForegroundSpeedTest that mimics transcoding while the app
is in foreground.
PiperOrigin-RevId: 662925764
App users can choose arbitrary data that might not be
anticipated by developers. Transformer shouldn't `checkState` based on
media data or file type -- report an error for unsupported data instead.
Public API change `ImageAssetLoader` needs to parse MIME type and now accepts
`Context` as parameter.
PiperOrigin-RevId: 660762459
Package-private until API is more useable.
Similar to frame analyzer mode: uses ImageReader instead of an encoder,
and no muxer.
PiperOrigin-RevId: 658446675
Transformer.experimentalSetMaxFramesInEncoder controls max number
of frames in encoder.
VideoFrameProcessor now allows delayed releasing of frames to Surface,
while still using the original presentation time.
VideoSampleExporter can now configure video graphs to not render frames
automatically to output surface.
VideoSampleExporter.VideoGraphWrapper tracks how many frames are ready
to be rendered to Surface, and how many frames are already in-use by encoder.
PiperOrigin-RevId: 658429969
Add parameterized test for codecs supported by InAppMuxer.
Split TransformerWithInAppMuxerEndToEndParameterizedTest tests
between parameterized and non parameterized
This will be helpful for adding more tests which need not to
be parameterized.
PiperOrigin-RevId: 658353532
This is following a renaming of registerInputFrame to handleInputFrame.
- queueInputBitmap is renamed to handleInputBitmap for consistency with
handleInputFrame.
- registerInputStream is renamed to onInputStreamChanged for consistency
with media3 method names.
PiperOrigin-RevId: 655529699
Build upon Transformer.videoFrameProcessorFactory in MultipleInputVideoGraph
Check that Transformer.videoFrameProcessorFactory is DefaultVideoFrameProcessor
for multi-input video
Fixes https://github.com/androidx/media/issues/1509
PiperOrigin-RevId: 655232381
This test is flaky at p4head, because CompositionPlayer has no logic to
ensure a specific input is used to configure the audio graph. Depending
on which sequence registers input first, it changes the output format
of the media.
By setting effects on the 2nd sequence, both inputs are the same format
and this flakiness is avoided in the test.
PiperOrigin-RevId: 653582441
-----
Context:
* Each sequence is wrapped as a single MediaSource, each being played
by an underlying ExoPlayer.
* Repeat mode is typically implemented in Players as a seek to the next
item in the playlist.
-----
This CL:
Repeat mode is triggered by listening for when the main input player
sees a play when ready change due to the end of the media item.
There is a slight delay at the end of the playback, before it repeats.
Setting repeat mode on the underlying players addresses this, but means
that the players will seek without waiting for the CompositionPlayer,
and as such previewAudioPipeline does not get the correct signals
around blocking/flushing.
PreviewAudioPipeline - The seek position can validly be C.TIME_UNSET,
however preview pipeline did not handle this case.
CompositionPlayer getContentPosition is given (through a lambda) as a
supplier to the State object, which means any comparisons between
previous/new state for this value does not work. In SimpleBasePlayer,
there is logic to use the positionDiscontinuityPositionUs for the
position change (see getPositionInfo called from
updateStateAndInformListeners), however this logic is not considered in
getMediaItemTransitionReason, so a condition needed to be added for
this case.
-----
Tests:
* Dump files clearly show the position and data is repeated.
* Assertions on the reasons for transitions or position
discontinuities.
PiperOrigin-RevId: 653210278
Sources (for example media projection) can populate the `Surface` from
`SurfaceAssetLoader` with timestamps that don't start from zero. But
`MuxerWrapper` assumes the latest sample timestamp can be used as the duration
when it calculates average bitrate and notifies its listener.
This can cause a crash because the calculated average bitrate can be zero if
the denominator duration is large enough.
Use the max minus first sample timestamp across tracks instead to get the
duration.
Side note: the large timestamps from the surface texture when using media
projection arrive unchanged (apart from conversion from ns to us) in effect
implementations and in the muxer wrapper (and are passed to the underlying
muxer). The outputs of media3 muxer and the framework muxer are similar.
PiperOrigin-RevId: 652422674
This should have no influence on app behavior and other policies
and just allows code to depend on new API 35 platform symbols.
PiperOrigin-RevId: 652414026
Also promote all H.265 constants to be public in `NalUnitUtil` with
`H265_` prefixes, for consistency.
A lot of these names are used in h.265 too (and `NalUnitUtil` handles
both), but with different values, so this rename aims to avoid
accidentally using an h.264 value in an h.265 context.
PiperOrigin-RevId: 651774188
This is because the force EOS workaround for videos is longer than the original
muxer timeout. This way the apps depending on Transformer won't have to manually
set the muxer timeout on emulators.
PiperOrigin-RevId: 651355755
VideoSink.registerInputFrame is now called for every input frame (not
only the ones that should be rendered to the input surface) because it's
the VideoSink that decides whether it wants the frame to be rendered.
PiperOrigin-RevId: 651049851
MCVR crashed because MCVR registers a new input stream to VideoSink on every
`onOutputFormatChanged()`, assuming that `onOutputFormatChanged()` is only
invoked on media item transition. However, it can be called multiple times for
one media item.
PiperOrigin-RevId: 649050576
Hardware decoder on some devices fails to write 1920x1080 YUV_420_888 buffers into an ImageReader. This change allows us to remove skipCalculateSsim device workaround in ExportTest.java.
VideoDecodingWrapper now uses media3.MediaExtractorCompat: necessary for parsing of MediaFormat#KEY_CODECS_STRING and decoder capabilities check
PiperOrigin-RevId: 648726721
The Javadoc was indicating that the duration should always be set,
but it doesn't need to be set in most cases for export.
PiperOrigin-RevId: 644685827
Includes adjusting how effects are defined, to make it clearer in a
test that a given ItemConfig has effects associated with it.
PiperOrigin-RevId: 644684308
These calls have no effect because the VideoFrameReleaseControl of
CompositionPlayer is created with allowedJoiningTimeMs set to 0.
PiperOrigin-RevId: 644274524
This is an internal refactor with no logic changed.
There is a duplication in information and a lack of consistency around
the use of test assets in transformer androidTest. This change
refactors each asset to be defined as its own AssetInfo, with the other
relevant parts stored alongside the uri.
Once this is in place, we can consider other useful functionality, such
as having boolean flags for what tracks an asset has, helper methods
for whether an asset is local or remote, and more. This will reduce the
manual overhead necessary to use more assets in tests, and in
particular leads towards easily using new & existing assets in
parameterized tests.
PiperOrigin-RevId: 644040595
This CL is a pure refactor of the existing tests, iterative changes
will be done in follow-ups.
The only logic change is having all MP4 assets have the presentaton of
height=360. The old code had some with height 480.
PiperOrigin-RevId: 643020773
- Video release should check for buffer timestamp (which is renderer-offsetted), rather than the frame timestamp
- ImageRenderer should report ended after all of it's outputs are released, rather than when finished consuming its input.
Add tests for timestamp handling
PiperOrigin-RevId: 642587290
In order to do that, make the VideoSink nullable in MCVR.
We want to avoid calling VideoFrameReleaseControl.setClock directly
from MCVR when the sink is enabled. The goal is to handle all the
communication with the release control from the sink/sink provider.
PiperOrigin-RevId: 642542063
Rational for keeping field package private:
Since the overall export already very complex, by looking at the output file
its hard to know if the desired processing happened or not. Since we now
support sequences with "n" media items, its even more important to know if all
the media items were processed or not.
Although the field exposes implementation details, it seems ok as we get benefit of detailed testing.
PiperOrigin-RevId: 642337888
Before this change, the timestamps output from composition playback is offset
with the renderer offset. After this change, the offset is removed and the
timestamp behaviour converges with Transformer, that is, the timestamps of
video/images frames will follow that of the composition. For example, with a
composition of two 10-s items, clipping the first with 2s at the start, the
timestamp of the first frame in the second item, will be 8s.
PiperOrigin-RevId: 641121358
Ensures valid progress state is returned. Should not return NOT_STARTED
once transformer.start has been called, until export ends.
PiperOrigin-RevId: 640533805
Usages removed in this CL are:
- onProcessedStreamChange, which was already called from the VideoSink
(via VideoFrameRenderControl)
- setOutputSurface, which was also already called from the VideoSink
- setFrameRate, which this CL now sets in the VideoSink
PiperOrigin-RevId: 640530903
Removes the flakiness of
MediaItemExportTest.getProgress_unknownDuration_returnsConsistentStates
by using a longer input asset, such that ExoPlayer does not determine
the duration of the media.
PiperOrigin-RevId: 640502470
Add SeparableConvolution.configure(inputSize) to allow effect configuration
depending on input dimensions.
Add LanczosResample.scaleToFit method to scale input images to fit inside
given dimensions.
PiperOrigin-RevId: 640498008
These changes are possible because getProgress is no longer a blocking
operation on transformer.
* Tests call getProgress after every looper message executed.
* Use longer media assets for getProgress tests to give more progress
intervals.
* Remove conditional assertions.
PiperOrigin-RevId: 639734368
