JaCoCo introduces private synthetic methods (even on interfaces) which
have to be skipped when checking that a 'forwarding' implementation does
forward everything. Instead we can use the existing `getPublicMethods()`
method which implicitly skips these (since they're private).
PiperOrigin-RevId: 519665752
When MediaCodecRenderer is given an empty sample stream, it puts
its output format change tracking in a bad state where we never
process future stream changes because we are waiting for a sample
that doesn't exist.
We can fix this by:
- Looping the pending output stream changes to see if we processed
more than one change at once (this fixes the tracking for empty
sample streams that are not the first in the queue).
- Checking if none of the previous streams queued any samples in
onStreamChanged to handle this in the same way as the case
where we already output all samples (this fixes the problem when
the empty sample stream comes first in the queue).
- Also calling onProcessedStreamChange for the case above, which
was missing previously.
#minor-release
PiperOrigin-RevId: 519226637
This wasn't added so far because releasing is always allowed from a
MediaController (as it just releases the connection, not the session
player). But Player instances can be created for other purposes and
the receiver of a Player instance should not always be allowed to
call release if it doesn't own the player resource.
PiperOrigin-RevId: 519121122
Since the composite matrix is ultimately rewritten to, we need to ensure it's cleared (to identity) before update.
Test plan: use an effect as a time based approach and see that the effect no longer clips
PiperOrigin-RevId: 518886623
If RTSP Setup Request with UDP receives HTTP Error Status 461 UnsupportedTransport, then client will retry with TCP.
Issue: google/ExoPlayer#11069
PiperOrigin-RevId: 518807829
The failure reason (no network) will be logged in the test output, but the test will not be marked as failing, rather skipped.
PiperOrigin-RevId: 518262203
Reduce nesting by using an early return. Also, rename
`outputSizeOrRotationChanged` to `outputChanged`, because this also applies to when the output surface changes.
Also, update local variable initialization, add some javadoc, and remove unneeded
local variable
PiperOrigin-RevId: 518241708
The content timeline field may be updated when the live timeline
is refreshed in the looper event preceding the runnable that is
posted to the player thread. Hence a new timeline may contain a
new period uid that is not present in the ad playback state map.
Using a final reference makes sure period and ad playback state
match when asserted.
PiperOrigin-RevId: 518228165
This change makes sure that live ad periods that are played are
skip when attempted to be added to the queue. To make this work
the existing filter logic had to be take into account the content
resume offset that live periods use.
PiperOrigin-RevId: 518068138
The FakeExtractorOutput dumps data in more readable format
so using that where ever possible.
The MdtaMetadataEntry which contains key and value dumped only "key".
Added fix to dump "value" as well.
PiperOrigin-RevId: 517968996
This was trimmed using the Transformer Demo, from ~4 seconds to 300ms
Otherwise, this test file will be quite large, and folks who clone the repo will
have to clone this content as well. Trimming the file also allows the test to
take less time, while still testing the core purpose of the file, to confirm that 4k extraction/decode/GL/encode/muxing works.
PiperOrigin-RevId: 517408463
d288891c77 added clearing depth buffers to GLUtil, so there is no need to have allow apps to have a custom clearOutputFrame.
Also removes default implementations in GLObjectsProvider know that these methods have been implemented.
PiperOrigin-RevId: 517156304
* Add `release` method to Renderer and AudioSink interfaces.
* Call the `release` method for renderers when the ExoPlayer is going to be released.
PiperOrigin-RevId: 517135677
[GL documentation for glClear](https://registry.khronos.org/OpenGL-Refpages/gl4/html/glClear.xhtml) says "If a buffer is not present, then a glClear directed at that buffer has no effect." so it's okay to clear the depth buffer even if there isn't one set.
Also manually tested to have no impact when contrast effect and dizzy crop effect form transformer demo was added to image/video input.
PiperOrigin-RevId: 516879598
This test file used to be remote, which led to various errors in loading the file or timing out during the muxer or overall transformer.
export4k60 test timing on Pixel 7:
|Condition |elapsedTimeMs|AS Test "Duration" (s)|
|--------------------------|-------------|----------------------|
|local file (this CL) |4253 |14 |
|Remote file, googleguest |5510 |17 |
|Remote file, 4G |11423 |25 |
PiperOrigin-RevId: 516815462
With the Mp4Muxer, while writing a large file if the something unexpected happens and muxer is not closed properly then it still outputs a valid MP4 file with partial data.
PiperOrigin-RevId: 516572804
While playback thread is 'asleep' during audio offload playback, the playbackInfo.positionUs is not being constantly updated. During this time, the returned value from getCurrentPosition should return an estimate based on the most recent value and playback speed.
PiperOrigin-RevId: 516550509
- To support looping EditedMediaItemSequences, we need a way to tell the
AssetLoader that a sample couldn't be consumed and that it should retry
later. This is necessary in case we don't know yet whether the looping
sequence should load more samples because the other sequences haven't
made sufficient progress yet.
- The decision on whether to consume a sample is based on its timestamp
so it needs to be available.
PiperOrigin-RevId: 516546026
Instead of using a long frameDurationUs with Math.floor, use a double
frameDurationUs with Math.round,
Before, playing an image with 30 fps over 1 second would result in the final
timestamp not being aligned to the expected 1 second timestamp. Over long
periods of time, this can lead to significant timestamp drift. Additionally,
for TimestampWrapper and constrained multi-asset, where TimestampWrapper
begins the 2nd effect on the 2nd asset's startTimeUs, this floor() behavior
can lead to the first few frames of the 2nd asset using the first asset's
effects, due to timestamps being mismatched.
PiperOrigin-RevId: 516529240
This test is not run on emulator, triage or postsubmit, and does not
provide value to the library in it's current state. It could be
deleted, however it still may have use for analysis of exported files
on MH.
PiperOrigin-RevId: 516491160
* Add AudioDeviceCallbackApi23 class which extends the AudioDeviceCallback.
* Add registerAudioDeviceCallback and unregisterAudioDeviceCallback methods for Api23.
* Modify the logics of AudioCapabilitiesReceiver constructor, register and unregister methods.
PiperOrigin-RevId: 516183997
This change does basically three things in preparation of a follow
up CL that will insert ad breaks into multi-period live windows.
- Make sure that only `AdPlaybackState`s of multi-period streams
are passed to `ImaUtil.splitAdPlaybackStateForPeriods()` from
the media source.
- Enhance the splitting algorithm in ImaUtil to support live windows.
Roughly, this includes calculating the window start position in the
UNIX epoch context, handling the end period with an unknown duration
and taking the `adResumePosition` of an ad group into account that
affects the ad group start positions (`adGroup.timeUs`).
- Append a placeholder postroll ad group to the ad playback state of
each period in a live stream to allow immediate transition to an
ad that's inserted by an ad event from the SDK.
PiperOrigin-RevId: 516167958
The value returned by `player.getContentPosition()` is calculated
in the timeline based on the position of the first period in the
window. In a single period live stream this position is advanced
when the live window advances on timeline refresh.
This calculation has produced slightly varying values below 1000
us which are likely caused by us/ms truncations for public API
values that we use in the IMASSAIMediaSource.
However, `AdGroup.timeUs` is the (recorded) content position at the
moment when the first ad of the an ad group has been inserted. While
playing an ad, we can always use this value instead of
`getContentPosition()` to not require recalculation.
#minor-release
PiperOrigin-RevId: 515093177
This change makes sure that the `AdPlaybackState` of any period can
contain an empty postroll placeholder.
The placeholder postroll should be represented in the `MediaPeriodId`
of a content period as `nextAdGroupIndex`, but should be ignored when
building the list of `MediaPeriodInfo` in the `MediaPeriodQueue`. This
is required to allow to add an ad group to ad playback state of the
content period that is currently being played, instantly insert an ad
period into the media period queue and immediately transition playback
to the new period.
This change makes sure and tests that
- a live server side inserted postroll placeholder can be inserted to
a `AdPlaybackState` in well-defined and tested way (helper method)
- a postroll placeholder is NOT ignored when
`AdPlaybackState.getAdGroupIndexAfterPositionUs` is called (this
is required when evaluating the `nextAdGroupIndex`).
- a postroll placeholder is ignored when
`AdPlaybackState.getAdGroupIndexForPositionUs` is called (this is
required to not attempt to play the ad and is analogous to ignore the
post roll placeholder in a single period timeline).
- `MediaPeriod.getFollowingMediaPeriodInfo()` does not include a
`MediaPeriodInfo` for the placeholder postroll when building the
queue.
PiperOrigin-RevId: 515079136
Split test/ TransformerEndToEndTest into SingleMediaItemEndToEndTest and
SingleSequenceEndToEndTest to reduce the file size and split the tests
by category.
PiperOrigin-RevId: 515039502
By making this method accept context, we can use DefaultDataSource.Factory in the DatasourceBitmapLoader to support a wider range on URI schemes in Bitmap Overlays. (and implement a local file picker for images for custom bitmap overlays in the demo transformer app)
PiperOrigin-RevId: 515013460
Callbacks onTrackCount and onTrackAdded can be called simultaneously
from different threads.
Before this fix, it was possible for the MuxerWrapper and
FallbackListener track count to never be set, or to be set
with incorrect values.
PiperOrigin-RevId: 514779719
We currently rely on the raw playback head position to check if any
data is pending in the AudioTrack (e.g. to know if the renderer is
still ready). We can use the value returned from getCurrentPositionUs
instead to align the "isReady" logic with the playback position logic.
This has the side effect that getPlaybackHeadPosition position is called
less often when the position is obtained via getTimestamp.
PiperOrigin-RevId: 514747613
Custom commands from controller to session are blocked if they are
not listed in the available session command list. This isn't well
documented in the Javadoc currently.
#minor-release
PiperOrigin-RevId: 514733568
Segment timelines are technically allowed to be empty, but not all places
add the necessary checks.
Issue: google/ExoPlayer#11014
PiperOrigin-RevId: 514722205
To create this file TextureInfo has been moved to common and renamed to GLTextureInfo.
We'll look to expand the interface in future to cover more of the methods around GL object maintenance in future as required.
PiperOrigin-RevId: 514445397
- This is to make sure we know about all the tracks before initializing
the SamplePipelines. This allows to set the muxer and the fallback
listener track count before the SamplePipelines are built.
- As a result, the test files had to be updated because the order in
which the tracks are written has changed.
- The ImageAssetLoader also had to be updated to call onOutputFormat
repeatedly until it returns a non-null SampleConsumer.
- Also fix the trackCount sent to the muxer and fallback listener. The
correct track count can be computed now that we know about all the
tracks before building the SamplePipelines.
PiperOrigin-RevId: 514426123
Before, we used to never call glDeleteFramebuffers, which could
in theory lead to leaks in the number of frame buffers
available and make releasing the GL context more expensive.
PiperOrigin-RevId: 514387847
The `@CallSuper` annotation should help catch cases where subclasses are
calling `delegate.addListener` instead of `super.addListener` but it
will also (unintentionally) prevent subclasses from either completely
no-opping the listener registration, or implementing it themselves in a
very custom way. I think that's probably OK, since these cases are
probably unusual, and they should be able to suppress the warning/error.
Issue: androidx/media#258
#minor-release
PiperOrigin-RevId: 513848402
All audio tracks should either all be transcoded or all be transmuxed.
Same for video tracks.
To achieve this, simplify the behaviour of transmuxAudio/Video.
PiperOrigin-RevId: 513809287
Renamed MuxerEndToEndTest.java to Mp4MuxerEndToEndTest.java to align it with class under test.
Removed muxed prefix from dump file name because Mp4 implicitely means muxed only.
PiperOrigin-RevId: 513574681
Used an actual captured image with set color profile for test to minimise the chance of the test flaking. Also renamed the media/bitmap/overlay folder to media/bitmap/input_images for clarity.
PiperOrigin-RevId: 513273353
If the Metadata passed to SegmentSpeedProvider is null, then the
SegmentSpeedProvider will always return 1f from getSpeed.
Initializing a SpeedChangingAudioProcessor requires a SpeedProvider.
Once configured,this audioProcessor is always active, so buffers are
passed through it. Because getSpeed is always 1, the processor performs
a no-op, but still has to do a buffer copy for each buffer.
By not initializing the audio processor when metadata is null, this
copy can be skipped and the audio pipeline is more performant.
Note: This change does not affect the multiple media-item case, which
is not supported with speed changes, as per Transformer API
documentation.
PiperOrigin-RevId: 513261811
Changes include:
1. Move the test fine into muxer module.
2. Use dump file infra for test cases.
3. Add one additional test for adding float metadata.
4. Few improvements in the code.
In next CL will remove Mp4 term from the file name as we are not using this term in test file names.
PiperOrigin-RevId: 513222506
We shouldn't have this logging unless we really need it to debug
a specific problem, as it can be noisy (even at debug level).
PiperOrigin-RevId: 512904412
Once the value returned from AudioTimestampPoller advances, we
only need getPlaybackHeadPosition to sample sync params and
verify the returned timestamp. Both of these happen less often
and we can avoid calling getPlaybackHeadPosition if we don't
actually need it.
PiperOrigin-RevId: 512882170
Based on 1000 test runs an emulator, with the current timeout releasing
fails (even with no custom effects) about one percent of the time.
Releasing normally completes in about 30 ms but occasionally
`eglTerminate` took up to 200 ms (and even releasing an effect
took up to 80 ms in one case).
With the new timeout of 500 ms, we still catch stuck effects reasonably
quickly but the number of flaky test failures should be less than one in
ten thousand.
PiperOrigin-RevId: 512690715
This timeline will be used in unit test cases of follow-up
CLs. It basically can be used to emulate the timeline created by a
multi-period live media source when the real time advances.
PiperOrigin-RevId: 512665552
Playback parameter signalling can be quite complex because
(a) the renderer clock often has a delay before it realizes
that it doesn't support a previously set speed and
(b) the speed set on media clock sometimes intentionally
differs from the one surfaced to the user, e.g. during
live speed adjustment or when overriding ad playback
speed to 1.0f.
This change fixes two problems related to this signalling:
1. When resetting the media clock speed at a period transition,
we don't currently tell the renderers that this happened.
2. When a delayed speed change update from the media clock is
pending and the renderer for this media clock is disabled
before the change can be handled, the pending update becomes
stale but it still applied later and overrides any other valid
speed set in the meantime.
Both edge cases are also covered by extended or new player tests.
Issue: google/ExoPlayer#10882
#minor-release
PiperOrigin-RevId: 512658918
Following test cases are added:
1. Mux H264 video
2. Mux H265 video
3. Mux HDR video
Each test case performs following actions:
1. Extract track and samples from input Mp4 using MediaExtractor.
2. Feed those samples into Mp4 muxer.
3. Use extractor to extract the samples from muxed file and create dump file.
PiperOrigin-RevId: 512589069
MediaCodecRenderer currently has two independent paths to trigger
events at stream changes:
1. Detection of the last output buffer of the old stream to trigger
onProcessedStreamChange and setting the new output stream offset.
2. Detection of the first input buffer of the new stream to trigger
onOutputFormatChanged.
Both events are identical for most media. However, there are two
problematic cases:
A. (1) happens after (2). This may happen if the declared media
duration is shorter than the actual last sample timestamp.
B. (2) is too late and there are output samples between (1) and (2).
This can happen if the new media outputs samples with a timestamp
less than the first input timestamp.
This can be made more robust by:
- Keeping a separate formatQueue for each stream to avoid case A.
- Force outputting the first format after a stream change to
avoid case B.
Issue: google/ExoPlayer#8594
#minor-release
PiperOrigin-RevId: 512586838
Some devices were reported to have wrong PerformancePoint sets
that cause 60 fps to be marked as unsupported even though they
are supported.
Issue: google/ExoPlayer#10898
#minor-release
PiperOrigin-RevId: 512580395
The output info for a new stream is marked pending until the last
sample of the previous stream has been processed. However, this fails
if the previous stream has already been fully processed. We need to
detect this case explicitly to avoid signalling the output change one
sample too late.
#minor-release
PiperOrigin-RevId: 512572854
This test became flaky after ab7e84fb34 because some of the
unrealistic frame times ended up on the same release time.
Using realistic numbers avoids the flakiness.
PiperOrigin-RevId: 512566469
Uses the first mediaItem's format as the output format.
If there is `Presentation` supplied in the `Composition.effects`, add it as the
last effect of the first EditedMediaItem.
PiperOrigin-RevId: 512082659
Also remove @WorkerThread annotations, as static checks associated with
this annotation aren't useful in this part of the codebase because
almost no methods are called on the main thread.
This change should be a no-op.
PiperOrigin-RevId: 512060367
Currently if releasing a shader program throws we don't release the GL
context, which could leak resources, and any errors are silently dropped
as we suppress notifications during releasing.
Improve resource cleanup and debuggability of errors from custom effects
by continuing to release shaders on failure (for runtime and
`VideoFrameProcessingException`s) and always clean up the GL context.
Note: this doesn't help with the case where releasing a custom shader
blocks for a long time, causing releasing the frame processor to
time out.
PiperOrigin-RevId: 512042501
Protected system broadcasts should not specify the export flag.
Marking them as NOT_EXPORTED breaks sticky broadcasts in some
cases.
Issue: google/ExoPlayer#10970
#minor-release
PiperOrigin-RevId: 512020154
Add format codec info, which can make test skipping checks more similar to the
actual Transformer decoder checks.
Also for the test file, the actual format was 720p, but somehow the file name and
media metadata indicated 1080p. This format mismatch led to some decoding errors,
so fix the format (and associated errors). This also allows us to remove the
exception catch in ForceInterpretHdrVideoAsSdrTest, which was included due to
errors from the incorrect format.
PiperOrigin-RevId: 511809507
The current logic uses manual array operations to keep track of pending
changes. Modernize this code by using an ArrayDeque and a data class.
This also allows to extend the output stream information in the future.
This also fixes a bug where a position reset accidentally assigns a pending
stream offset instead of keeping the current one.
#minor-release
PiperOrigin-RevId: 511787571
In most cases, this means updating Subject subclasses and their assertThat methods to accept null actual values. (They should always accept null so that assertions like "assertThat(foo).isNull()" succeed instead of throwing NullPointerException.)
Occasionally, it involves other changes, like writing `isGreaterThan(1L)` instead of `isGreaterThan(1)` to resolve an ambiguity it Kotlin overload resolution.
PiperOrigin-RevId: 511776581
- Split the transmux setting into transmuxAudio and transmuxVideo. This
is more flexible for apps and will also be useful for unit testing
(particularly as we can't test video transcoding on Robolectric at the
moment).
- Move these settings to Composition. It makes sense for these settings
to be next to forceAudioTrack. Apps may also want to set these
settings based on the current Composition's MediaItems.
- Add a Composition.Builder because Composition now contains a few
optional fields.
PiperOrigin-RevId: 511708618
Logcat had the following lines, with no other information.
```
DefaultEncoderFactory: Encoders removed for resolution:
DefaultEncoderFactory: Encoders removed for bitrate:
DefaultEncoderFactory: Encoders removed for bitrate mode:
```
PiperOrigin-RevId: 511470231
Also, allow isNoOp to default to false without the TODO, so that implementations
of isNoOp must opt-in to implementing the override in order to be considered for
skipping the effect (ex. for transcoding in Transformer).
PiperOrigin-RevId: 511223540
Implement getMediaFormatInteger, a helper method simulating mediaformat.getInteger(name, defaultValue).
This reduces the API 29 restriction from MediaFormatUtil.getColorInfo to API 24, in
particular removing the method-based restriction to a constant-based restriction,
so that we can reduce usage of the API 29 class.
This also allows us to slightly simplify prior use-cases where we'd check
containsKey and getInteger to have a default value.
PiperOrigin-RevId: 511184301
When clipping a MediaItem with start time > 0, the audio was ending
before the video. This is because:
- Audio timestamps are computed based on the sample sizes, with a start
time set to streamOffsetUs (i.e. the streamStartPositionUs is not
taken into account).
- The SamplePipeline was subtracting streamStartPositionUs from the
timestamps before sending the samples to the muxer.
- As a result, the audio timestamps were shifted by
streamStartPositionUs, while they should be shifter by streamOffsetUs.
PiperOrigin-RevId: 511175923
