- Video release should check for buffer timestamp (which is renderer-offsetted), rather than the frame timestamp
- ImageRenderer should report ended after all of it's outputs are released, rather than when finished consuming its input.
Add tests for timestamp handling
PiperOrigin-RevId: 642587290
In order to do that, make the VideoSink nullable in MCVR.
We want to avoid calling VideoFrameReleaseControl.setClock directly
from MCVR when the sink is enabled. The goal is to handle all the
communication with the release control from the sink/sink provider.
PiperOrigin-RevId: 642542063
Rational for keeping field package private:
Since the overall export already very complex, by looking at the output file
its hard to know if the desired processing happened or not. Since we now
support sequences with "n" media items, its even more important to know if all
the media items were processed or not.
Although the field exposes implementation details, it seems ok as we get benefit of detailed testing.
PiperOrigin-RevId: 642337888
Before this change, the timestamps output from composition playback is offset
with the renderer offset. After this change, the offset is removed and the
timestamp behaviour converges with Transformer, that is, the timestamps of
video/images frames will follow that of the composition. For example, with a
composition of two 10-s items, clipping the first with 2s at the start, the
timestamp of the first frame in the second item, will be 8s.
PiperOrigin-RevId: 641121358
Ensures valid progress state is returned. Should not return NOT_STARTED
once transformer.start has been called, until export ends.
PiperOrigin-RevId: 640533805
Usages removed in this CL are:
- onProcessedStreamChange, which was already called from the VideoSink
(via VideoFrameRenderControl)
- setOutputSurface, which was also already called from the VideoSink
- setFrameRate, which this CL now sets in the VideoSink
PiperOrigin-RevId: 640530903
AudioGraphInput.onMediaItemChanged is called on input thread. Pending
media item changes are processed on processing thread, inside calls to
getOutput().
This change allows multiple pending media item changes to be enqueued,
and processed in sequence.
PiperOrigin-RevId: 638995291
Add a wait in DefaultCodec.signalEndOfInputStream when no
video encoder output has been seen. This avoids a thread synchronization problem
between writing frames to video surface, and signaling end of stream,
which was hit for video input of only one frame on some devices.
PiperOrigin-RevId: 637844690
This change avoids a muxer deadlock when:
1. Sequence of items
2. First item has audio track that is shorter than video
3. Audio finishes, and muxer refuses to write more than 500ms of video
consecutively.
SequenceAssetLoader fails to progress to the second item. A muxer
deadlock is possible when the audio of the first item finishes,
audio end-of-stream is not propagated through AudioGraph, and muxer blocks
video, preventing SequenceAssetLoader to move to the next item in sequence.
By triggering silence generation early as soon as audio EOS is
encountered, we ensure SequenceAssetLoader can progress to the next item.
PiperOrigin-RevId: 636179966
This was previously only set on images because it was not ignored on
other media types. This parameter was made no-op for non-images in
7b2a1b4443.
PiperOrigin-RevId: 636078142
If MediaCodec allocates passes an image buffer with a cropped region,
SurfaceTexture.getTransformMatrix will cut off 2 pixels from each dimensions.
The resulting videos will appear a little stretched.
This patch inspects the SurfaceTexture transform matrix, and guesses what the
unscaled transform matrix should be.
Behind experimentalAdjustSurfaceTextureTransformationMatrix flag
PiperOrigin-RevId: 635721267
For each item, AudioGraphInput now pads the input audio track with silence
to the duration given in onMediaItemChanged.
Possibly resolves Issue: androidx/media#921 .
PiperOrigin-RevId: 634753721
ExoPlayer sometimes calls AudioSink.playToEndOfStream before configuring
the sink. Before this CL, the composition player was failing if this
happened.
PiperOrigin-RevId: 634306592
This class is not ready for production app usage yet, so it is still
marked `@RestrictTo(LIBRARY_GROUP)` for now. Apps can experiment with it
in a non-prod context by suppressing the associated lint error.
* Issue: androidx/media#1014
* Issue: androidx/media#1185
* Issue: androidx/media#816
PiperOrigin-RevId: 633921353
This annotation is only needed on public classes.
This change also removes the `/* package */` comment from some `public`
classes.
PiperOrigin-RevId: 633864544
This treats heic as a separate mimetype to heif (even though heic files are a subset of heif files). This is in line with other platform classes like android.content.ContentResolver
https://developer.android.com/media/platform/supported-formats#image-formats was updated to include avif support or API level 34, so added this MimeType as well and updated our associated util.
solves Issue: androidx/media#1373
PiperOrigin-RevId: 633616214
Also add documentation that suggests to use them in
PriorityTaskManager and adjust codec priorities in
Transformer's DefaultDe/EncoderFactory accordingly.
PiperOrigin-RevId: 633272667
allows apps to use the logic statically before running transformer.
Also ensure the mime type reported is the mime type outputted
PiperOrigin-RevId: 631811763
The Javadoc is stating that the image duration is ignored for non-image
input but this is incorrect (for example, it affects seeking
performance). The Javadoc will be updated in another CL.
PiperOrigin-RevId: 631730980
AudioGraphInput now accepts a range of inputs, as long as the effects
provided modify the audio to be int 16.
As part of this, add the workaround to DefaultCodec to ensure pcm
encoding is correct, and remove parameterized tests that are not valid.
PiperOrigin-RevId: 631404152
The second stage of the changes remove the conversion to linear colors in the SDR effects pipeline by default.
also resolves Issue: androidx/media#1050
PiperOrigin-RevId: 630108296
Before this CL, if all the video (resp. audio) samples were fed before
the audio (resp. video) output format was reported, the
SequenceAssetLoader was incorrectly reporting a MediaItem change with a
null format.
PiperOrigin-RevId: 625978268
Transformer's input shouldn't be constrained to the number of playable audio channels on the current device because the media may be edited (to mix channels for example) or encoded for playback on another device (a server for example).
PiperOrigin-RevId: 625604243
The raw audio decoder's output audio format is stereo when the number of input
channels is (for example) 10 channels. Add a temporary workaround that uses the
input channel count for raw audio. This code should be removed in future when
we bypass the decoder entirely for raw audio.
Tested manually on a WAVE file with 18 audio channels.
PiperOrigin-RevId: 625307624
Before this CL, currentInputBufferBeingOutput was set to null without
adding the buffer to the queue of available buffers, which was making
this buffer unusable. After multiple seeks, playback was stuck because
the AudioGraphInput had no input buffer left.
PiperOrigin-RevId: 624943271
After this CL, Transformer will throw if the clipping start and end
positions are the same because MediaMuxer doesn't support writing a
file with no samples. This should work once we default to the in-app
muxer.
Issue: androidx/media#1242
PiperOrigin-RevId: 624861950
the output format conatins key information about the output of the assetloader being hdr, so we must signal the output format, not the input format to the sample exporter
fixes mh ultraHdr test failures.
Also discovered images created are very device specfic so got rid of the pixel tests, we have pixel tests in the effects library that cover the same case.
PiperOrigin-RevId: 619899249
If the mixer is reset without resetting maxPositionOfRemovedSources and
then reused, the value of maxPositionOfRemovedSources can be outdated,
leading to an incorrect number of bytes being output by the mixer.
PiperOrigin-RevId: 619832502
This CL aims to separate Fragmented MP4 related logic in a separate public class.
Earlier all the logic was in a single class `Mp4Muxer`.
PiperOrigin-RevId: 619206661
This property is transformer specific and does not belong to
muxer interface.
This is to eventually replace muxer interface in Transformer module with
muxer interface in Muxer module.
PiperOrigin-RevId: 618895836
The immediate plan is to replace `Transformer Muxer interface`
with this one.
It was not straight forward to move/change `Transformer Muxer interface`
hence this intermediate step.
The follow up CL will also remove fragmented MP4 related code from
`Mp4Muxer` and move it into `FragmentedMp4Muxer implements Muxer`.
PiperOrigin-RevId: 618789265
Seeking will consist of the following steps:
- Block the AudioGraph input data.
- Flush the AudioGraph.
- Seek the ExoPlayers.
- Unblock the AudioGraph input data.
PiperOrigin-RevId: 617868124
When flushing the AudioGraph, the AudioMixer will be preserved but the
sources will be recreated. This is probably a bit less efficient but
makes the logic simpler. Indeed, if we had to keep the sources alive,
we would need to add a way to reconfigure them with a new timestamp for
seeking. We would also need to change the way sources are ended because
they are currently removed when they are ended. Also, it is acceptable
to have a small delay when seeking, which means that performance is less
critical than for playback.
PiperOrigin-RevId: 615775501
Pass firstAssetLoaderOutputFormat to videoSampleExporter for non-video use cases, so that the downstream components like the videoFrameProcessor can be set up with the right output color. Surface creation is still in the VSP so can't do this for all use cases currently.
also moves getDecoderOutputColor() to TransformerUtil, since it is used in multiple places and doesn't make sense for once to have reference to the other.
PiperOrigin-RevId: 613113958
