This makes sense until we need to support seeking in the live window.
Issue: #676
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112402026
Also apply the workaround for the secure variant of OMX.SEC.avc.dec.
Note that it's not necessary to do the same for the RK decoder in
the method below, since that workaround is targeted at SDK_INT<=17
and secure decoders only came along in 18.
Issue: #603
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112395376
After this change, AC-3 uses about 20 KB and DTS uses 49 KB.
For comparison, 'normal' PCM playbacks use by default (depending on the device
and input format) about 45 KB. For passthrough, the following buffer sizes were
used before this change:
- Nexus Player AC-3: 23 KB
- Nexus Player DTS: 25 KB
- NVIDIA Shield AC-3: 15 KB
- NVIDIA Shield DTS: 16 KB (caused underruns in some DTS-HD playbacks)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112254836
Using the provided methods by the previous refactors, it is now possible to use all of the WebVTT features already available.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112243172
The TTML 1 spec. defines an exact RGBA color tuple as #rrggbbaa
See https://www.w3.org/TR/ttml1/#style-value-color
Android's internal representation is ARGB.
The correct parsing therefore requires a bit of extra byte shuffling ...
I'm not really sure how best to document this in TrackRenderer;
it's a bit of a weird feature. For now, I've gone with the vague
approach.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112150939
See the documentation of buildTracks for the gory details.
Issue: #151
Issue: #676
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112149293
This CL exposes the cue settings parser in order to allow its usage from the MP4Webvtt extractor. Also fixes a few mistakes from the previous related CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112145806
* AudioTagPayloadReader was strangely parsing an audioSpecificConfig
itself, using the parsed values to build a new audioSpecificConfig,
then passing the newly constructed instance to be parsed by
CodecSpecificDataUtil. Unfortunately the translation was lossy ;).
* Treat Duration=0 as an unknown duration.
Issue #1137
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112143569
- I think \r and \n are handled the wrong way around?
- We only expect to encounter a BOM sequence at the start of a
file, but it feels fine to automatically discard it in all
cases for simplicity. A BOM sequence doesn't mean anything in
UTF-8. See https://en.wikipedia.org/wiki/Byte_order_mark. Note
that I think the advice not to remove it on that page relates
only to the case where the file is being edited + saved.
Issue #1136
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=112143407
There are multiple issues with HlsSampleSource's use of
LoadControl that become apparent when you attempt to use
the same LoadControl for loads by another source.
* In the "limbo" state HlsSampleSource doesn't start any
new loads, but doesn't update the LoadControl to tell
it that it doesn't want to load anything either. This
can prevent another source from starting the loads that
it needs to make to complete preparation, causing
playback to become stuck.
* The LoadControl isn't updated properly when the EOS is
reached. This can cause playback to become stuck near
the end of the media.
* If HlsSampleSource is released from being in the "limbo"
state, it doesn't unregister itself with the control.
Issue: #151
Issue: #676
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111942009
Given we need to do this in HlsPlaylistParser in the normal
case (i.e. not MEDIA_TAG), we may as well just be consistent
and do it everywhere.
Issue: #151
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111941335
Currently only supports a single offset to the full media timeline
(indicated by a duration of 0). This is most often used to fix the
non-zero starting presentation timestamp introduced when B-frames
are present.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111816916
This is equivalent to DashTrackSelector and SmoothStreamingTrackSelector.
This is a step toward allowing HlsChunkSource to expose multiple tracks,
which is a requirement for supporting WebVtt.
This change also enables WebVtt extractor instantiation.
Issue: #151
Issue: #676
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111698833
This allows the same adjusters to be used by multiple
HlsChunkSource instances. This is necessary because
WebVTT chunks will be loaded by a second chunk source
to the one loading audio/video. In both cases the same
timestamp adjustments will need to be applied.
Each source may transition from one discontinuity sequence
to the next at a slightly different time, so it's necessary
to maintain a separate adjuster for each sequence.
An adjuster can only be initialized correctly using audio/video
and not WebVTT, because the start time in a WebVTT file in
HLS doesn't necessarily correspond to the chunk start time,
which means the timestamp offset calculated by the adjuster
could end up being incorrect. Hence sources providing WebVTT
chunks will set isMasterSource to false. Lovely, right :(?
Issue: #151
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111693126
This gives DefaultSmoothStreamingTrackSelector feature parity
with DefaultDashTrackSelector. Note that the code duplication
across these classes will go away eventually, when we rework
track selection as described in Github Issue #1121.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111688784
This is a no-op change for clarity only. maxWidth/maxHeight
don't mean anything for AAC/MP3, so it makes sense to pass
MediaFormat.NO_VALUE in all cases.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111619832
Moved the behaviors related to Cue's to the WebvttCueParser class.
This way, the parsing methods will be more easily accessible to
other classes, such as the MP4Webvtt parser. This class also has
some methods that require state to avoid repetitive avoidable
allocations. The method visibility is subject to changes in
further CLs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111616824
Targeting to all API level 19 devices using the OMX.SEC.avc.dec
decoder is probably the right thing to do. It shouldn't have
negative implications if we apply the workaround on devices that
don't really need it, except to slow down seeking slightly due
to decoder re-allocation. Given we're talking about JB, I think
the priority should be to "make sure it works".
Issue: #951
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111514406
We normally expect each frame to come in its own PES packet,
but it seems that this is not always the case. This change
uses the frame rate in the stream to increment the frame
timestamp in the case of multiple frames contained within
a single PES.
Note that since we don't expect 100s of frames in a single
PES, or anything close to that really, the rounding errors
that may accumulate due to use of a frame duration should be
fine.
Issue: #1112
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111499052
This is in preparation for removing BufferingInput,
and using peeking instead.
Also add tests for peeking with allowEndOfInput and
resetPeekPosition.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111318236
This CL adds a class responsible for managing the lifecycle
of a single Choreographer to be shared among all
VideoFrameReleaseTimeHelper instances.
Issue: #1066
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=110839824
This CL is prepares the ground for refactoring the Webvtt parser,
so as to use the common parsing algorithms in both parsers. In order
to do this, the Webvtt Parser will be refactored. As a side note, many
more test cases will be added once the new subtitle features are
implemented. Some useful test cases have also been left for a following
CL, to allow an easy code review.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=110466914
Seems (marginally) nicer than making one up :). I didn't
realize there was a method that didn't require a framerate
to be passed!
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=110456874
This allows implementation and injection of custom MediaCodecSelector
instances. By injecting a custom selector, it's possible for applications
to exert more control over which decoder(s) they instantiate. For example,
applications can force use of a software decoder.
GitHub Issue #938
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=110369810
Fix behavior of getTimeUs when seeking after the last entry in the table of
contents. Round correctly in getPosition, clipping to the stream duration based
on the input length (if known), falling back to the stream size from the header.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=109993852
Multiple identical TTML fontStyles or fontWeights or textDecorations
on the content for a specific moment were rendered as if there's only
one decoration (span).
That's because SpannableStringBuilder.setSpan(span, start, end, flag)
found an earlier set span (the static allocated span's reference
is the same each time) and only refreshed this first span's start and
end values instead of adding a new span at its (new) different range.
This patch removes the static data members;
this makes the newly allocated span objects distinguishable.
A correct implementation is favoured over worries about memory
consumption.
- Use allowDefaults to fix crash if params are passed without
the speed being explicitly set.
- Allow null to be passed to clear previously set params.
- Clarify in doc that the passed params shouldn't be modified
after they're passed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=109591580
* Add additional Widevine samples.
* Improve error messaging in demo app around decoders.
* Display toasts for playback errors related to missing insecure
decoders, missing secure decoders, decoder instantiation failure
and decoder query failure.
* Remove checks from SampleChooserActivity, since the above largely
covers off this problem.
This is the main component required to enable WebVTT subtitles in HLS.
It passes through each WebVTT file as a sample, and derives the correct,
adjusted timestamp for each of them on the way through.
Not yet wired up because we need to properly share the same
PtsTimestampAdjuster everywhere, and also stop instantiating new instances
of the adjuster. The adjuster will also need to correctly handle
discontinuities, since we'll no longer be creating new instances of it.
Issue: #151
- parse webvtt cue
- remove all tags from string (supported or not)
- apply spans for b, i and u
- honor class names in tags to properly parse the cue but do not apply styles for them
- Propagate BehindLiveWindowException if we fall off the back
of an HLS live stream.
- Consolidate seekPositionUs and playbackPositionUs into a
single parameter.
Issue: #765
* Split findNextCueHeader and validateWebVttHeader into static methods.
This is a step toward WebVTT in HLS, where we'll need to re-use these
to peek at the top of the WebVTT file (they'll be moved into a util
class).
* Made parser robust against bad cue headers + added a test.
* Removed spurious looking assertion in WebvttSubtitle.
They don't seem particularly useful; they don't technically force
strict compliance, but rather just catch a few token things in
each case. Furthermore, for playback, probably the right thing to
do is to always turn strict mode off.
We were previously using the container format of the media being
played as the mimeType generating key requests, but this is not
always correct. As an example, where a manifest contains webm streams
but specifies initialization data using cenc:pssh elements in the
manifest, the media has a webm mimeType, but the DRM initialization
data has an mp4 mimeType.
It appears the spec calculation gives the h:w pixel ratio, where-as
we want w:h. It's pretty easy to convince oneself that this way round
is correct. Consider a video that's 100px by 100px, and setting
aspectRatioCode=3 to achieve this. The pixelWidthHeightRatio needs to
be 16/9 and not 9/16 :).
Issue: #965
Previously, when spectral band replication (SBR) or parametric
stereo (PS) was in use in an MPEG-4 stream, the channel configuration
chosen was likely incorrect. The channel configuration was *always*
incorrect for 7.1 audio (gave 7 channels instead of 8).
- NAME is optionial now in the Hls Manifest
- Use the id field in Format to store the NAME instead of
a field in Variant to mimic DASH's behaviour
(see the DASH Id PR, which is not merged yet at this time).
Added the property 'id' to the MediaFormat class
which serves as an identifier for the track.
DASH Representations will have the "id's" from their
Media Presentation Description mapped to the id property
in the MediaFormat class that will represent the track.
We needed this for an use case where we wanted to read the 'id'
value from the DASH representation and present it to the user
in order for the user to select the right track.
The sample data position is the sum of the data offset and the base data offset.
The base data offset is either specified in the stream, or defaults to the first
byte position in the moof box. (We only support one traf per moof currently, so
the offset does not need to be assigned for later track fragments.) The data
position can optionally be offset by a data position read from the trun.
The auxiliary information offset is calculated in the same way, but using an
offset read from the saio box.
Issue: #837
Issue: #861
Context:
- Currently, playback is significantly more juddery with it disabled,
particularly on AndroidTV.
- We should be able to do the "best" job of this internally, so injection
doesn't buy anything useful. If someone has a better implementation for
adjusting the frame release, they should improve the core library.
The only Samsung devices with names starting "d2" that we're aware of
are Galaxy S3 variants, and also one Samsung Galaxy Pocket Neo d2aio
SAMSUNG-SGH-I747Z. This change speculatively includes that device too because
its name is very similar to SAMSUNG-SGH-I747 which is known to be affected.
Issue: #548
Try re-sync'ing to the next level 1 element when invalid data is found. This
corrects the behavior for test case 4 in the mkv test suite.
Partially Fixes Issue #631
This change keeps the proportion offset * 256 as a floating point value rather
than rounding it before linear interpolation, which will increase precision
slightly when seeking in streams with XING headers.
In practice, this won't make much of a difference because precise seeking in VBR
MP3s with XING headers seems not to be possible without reading the entire file,
due to the fact that the (uneven) distribution of bits is represented by a fixed
number of table of contents entries.
- The old approach was technically incorrect, because the checks
were "capacity < sampleSize" and hence neglected the fact that
the buffer position may be greater than 0 (e.g. if the caller
wants to prefix the sample with some additional data).
- Also proactively throw an exception if the buffer is too small,
rather than wait for the failure when we actually do the write.
- Line parameter
- Added support for value and line alignment attributes.
- Support negative numbers when line is an absolute number (not a
percentage).
- Position parameter
- Added support for value and position alignment attributes
- Added support for WebVTT comment blocks
- Percentage values now accept decimal numbers (as webvtt spec states)
- Added new WebVTT tests for testing all new implemented features
- do not denormalize styles at parsing time but only put normalized style info
into TtmlNode tree. Resolve styles on demand when Cues are requested for a
given timeUs.
- create TtmlRenderUtil to have static render functions separate
- added unit test for TtmlRenderUtil
- adjusted testing strategy for unit test to check resolved style on Spannables after rendering
When switching format in HLS, we instantiate a new extractor, which
adjusts TS presentation timestamps so that they align properly with
the start of the first segment in the new format. Some HLS streams
appear to have slightly misalignment that causes a glitch when using
this approach.
It's better to re-use the same timestamp adjustment across formats,
and only reset it when seeking or when there's an actual discontinuity.
This is because the HLS spec guarantees PTS timestamp alignment across
different formats.
We'll also need something like PtsTimestampAdjuster to share between
separated audio and WebVTT tracks, which also contain PTS timestamps
that are aligned, and will need to share a common adjustment.
Issue: #692
- Don't allow "nothing has changed" optimization in the case
that only styling has changed (TextUtils.equals will return
true in this case, but we shouldn't optimize).
- Add functionality to suppress embedded styling; seems useful
to have.
- Added "this." for clarity.
When a live stream ends, what typically happens is that the manifest
is refreshed and the refreshed version is not marked as live/dynamic.
When this happens we:
1. Don't want the duration of the track to change.
2. Still want to consider the possibility that we may have fallen behind
the live window.
3. Don't want to allow futher manifest refreshes.
This change uses the right thing in the right place.
- Admit we don't know the mime type (using unknown mime types) rather
than passing the container mime type.
- Pass the correct mime type for opus, vp9 and vp8, and remove the incorrect
container checks in the corresponding extensions.
- With this change, you can select from the individual video formats in
the demo app, as well as the regular "auto" (adaptive) track.
- DashRendererBuilder no longer needs to create MultiTrackChunkSource
instances for the multiple tracks to be exposed.
I think such manifests are invalid, and I haven't seen any examples,
but given it's trivial to fill in the duration if the periods define
durations, it seems worth being robust.
- It's not possible to determine a period's duration just from the
corresponding Period element in a DASH manifest. It's necessary
to look at the start time of the next period (or the duration of the
manifest for the last period) to determine how long a period is. We
don't currently do this in the parser, and hence set duration incorrectly.
We also set period start times incorrectly because we don't set it to
equal to sum of the durations of prior periods in the case where it's
not explicitly defined.
- We're currently propagating these (incorrect) values all over the place
through data-structures that we build when parsing the Period element.
- This CL removes this redundancy, storing only the start time of each
period in Period elements, and not propagating it elsewhere. It's then
used when required in DashChunkSource.
The following sequence was problematic:
1. See start of a cluster having not output a seek map. Decide
to seek for the cues. Enter CUES_STATE_BUILDING state.
2. Error occurs before seek map is output.
3. ExtractorSampleSource isn't prepared yet, so restarts from the
start of the stream.
4. See start of the same cluster having not output a seek map.
This time cuesState is CUES_STATE_BUILDING, so we just carry
on. We then fill the buffer with sample data, despite the
source not being prepared, at which point we get stuck.
It's unclear to me why cuesState needed three states, so I've rm'd
the BUILDING state. Step (4) above will now do the same thing as
in step (1). If the failure repeats, we'll eventually fail, which
is WAI.
These MP3s are unseekable but allow calculating the VBR duration correctly.
Treat streams as live only if they are unseekable and lack a duration.
Issue: #713
- Remove unused method in DashChunkSource.
- Remove inputEncoding parameter for subtitle parsers. We're
ignoring it in all but one of the parsers, and for the one
that does use it, it'll only ever receive null, since that's
all we're passing.
- Make TextTrackRenderer advance to the next subtitle even if
the current one hasn't finished, in the case that they overlap.
This shouldn't ever really happen, but it seems best to trust
the start time of the new sample rather than the last event
time of the previous one.
- Video track is always marked as adaptive, the resolution is
stripped out (since it's otherwise just set to whatever the
resolution of the first selected variant is), and the max
dimensions are set.
Issue #514
This is needed for several use cases:
- ExtractorSampleSource with option to play both embedded and out-of-band
subtitles.
- HLS multi-audio and out-of-band-webvtt.
- Migrate demo app to use new APIs.
- Add multi-track support for ExtractorSampleSource case.
- Add multi-track support for SmoothStreaming use case.
The final step is to add support back for the DASH use case and
delete MultiTrackChunkSource. This is blocked on multi-period support
landing, in order to prevent a horrendous merge conflict. We also
need to update HLS to expose sensible track information.
Issue: #514
When ChunkSource implementations implement multi-track for DASH and SS,
format selection will move inside of ChunkSource. If we, for example, fail
to query the decoder to determine which tracks are playable, we need an
opportunity to fail (i.e. say we're not prepared, so that maybeThrowError
is called, from which we can throw).
This may go away in the future if we remove the distinct preparation step
and treat tracks/formats as things that can change dynamically, but for now
this is what we have.
Issue #514.
Fix reading the first slice flag, which before could cause a read out of bounds
if the NAL unit started at the end of the buffer.
Handle non-VCL NAL units by flushing a pending sample when starting to read one.
multi-track support upstream to the ChunkSource interface.
This change does not yet make use of the newly exposed APIs. This
will come in a subsequent CL.
Issue #514.
- Currently all subtitles we parse contain timestamps relative to the sample
timestamp, however we add the sample timestamp in inconsistent ways (sometimes
in the Subtitle, sometimes in the SubtitleParser). This change converges on
a single approach. It also paves the way for passing absolute offsets to use
instead, and being able to apply them in a consistent way in a single place
(PlayableSubtitle). This functionality will be required for ISO 14496-30 TTML
embedded subtitles.
Issue: #689
- Parse all attributes that may exist in either the AdaptationSet or
in the child Representation elements at both levels.
- Correctly infer TYPE_TEXT for Representation elements whose mimeType
is application/mp4 and whose codecs attribute indicates a known text
codec type.
Issue: #689
Remove MPEG TS stream filtering based on AudioCapabilities.
Pass AudioCapabilities to MediaCodecAudioTrackRenderer so it can choose between
passthrough/raw and decoding for AC-3 tracks.
- Generalize rendererEnabledFlags to be selected track indices through
ExoPlayerImpl/ExoPlayerImplInternal.
- Selecting an out-of-bound track index (e.g. -1) is equivalent to
disabling a renderer prior to the generalization.
- A prepared TrackRenderer that exposes 0 tracks is equivalent to a
TrackRenderer in the STATE_IGNORE state prior to the generalization.
Issue #514.
On NVIDIA Shield, recreated passthrough AudioTracks have incorrect playback head
positions, due to shared state with the previously destroyed passthrough
AudioTrack. Calling AudioTrack.flush() before AudioTrack.release() ensures this
state is cleared.
This is a safe change so I have not made it a device-specific workaround. The
combined time for flush and release is less than 10 ms (with flush normally
taking less than two ms).
Everything I've seen that uses ExoPlayer sets it to true, and
setting it to false is poorly supported / likely to result in
bad initial A/V sync after each seek.
Both FragmentedMp4Extractor and WebmExtractor now invoke seekMap() with
SeekMap.UNSEEKABLE if there isn't an index in the stream, so DASH playbacks
will end up printing this warning every 5 seconds or so. This is obviously
not desirable, so this CL just removes the warning! The alternative would
have been to print a warning if the value is anything other than UNSEEKABLE,
but it doesn't really seem worth it.
This is useful to allow playback of individual segments from a
DASH stream as regular fmp4 files. These segments don't typically
contain a segment index. For playback to start, we need to invoke
seekMap with the UNSEEKABLE index. We do this if we haven't seen
a segment index when we encounter an mdat box (if one were present,
it would have been located earlier than this point).
1. [Cleanup] Remove unused Track types, including TYPE_TIME_CODE.
2. Add subtitle track type, which is different to the existing text type.
3. Set duration on the media formats for text and subtitle tracks. This
was causing the player to report unknown media duration for mp4 files
containing such tracks.
4. Make TextTrackRenderer do the right thing when not started.
Issue: #635
- Make UnrecognizedInputFormatException public so the app can more easily handle
it when it is propagated to onPlayerError.
- Format the description using the simple class names for the extractors.
- Don't retry loading after it's thrown, but just throw immediately.
- ExtractorSampleSource takes an array of extractors to test for suitability.
- Extractors now implement a sniff() method that returns whether they can
extract samples in the input stream's format.
- Switch demo app samples to use format detection.
Issue: #438
1. Fix seeking in test2.mkv by handling non-default timescale
after duration.
2. Fix handling of missing cues in test6.mkv by allowing playback
to continue (but all seeks will reset to t=0).
Issue #631
