diff --git a/RELEASENOTES.md b/RELEASENOTES.md index 1704847087..b22039ccd6 100644 --- a/RELEASENOTES.md +++ b/RELEASENOTES.md @@ -4,7 +4,7 @@ * Core library: * Support preferred video role flags in track selection - ((#9402)[https://github.com/google/ExoPlayer/issues/9402]). + ([#9402](https://github.com/google/ExoPlayer/issues/9402)). * Prefer audio content preferences (for example, "default" audio track or track matching system Locale language) over technical track selection constraints (for example, preferred MIME type, or maximum channel @@ -13,24 +13,34 @@ can always be made distinguishable by setting an `id` in the `TrackGroup` constructor. This fixes a crash when resuming playback after backgrounding the app with an active track override - ((#9718)[https://github.com/google/ExoPlayer/issues/9718]). + ([#9718](https://github.com/google/ExoPlayer/issues/9718)). * Sleep and retry when creating a `MediaCodec` instance fails. This works around an issue that occurs on some devices when switching a surface from a secure codec to another codec - ((#8696)[https://github.com/google/ExoPlayer/issues/8696]). + ([#8696](https://github.com/google/ExoPlayer/issues/8696)). * Add `MediaCodecAdapter.getMetrics()` to allow users obtain metrics data from `MediaCodec`. ([#9766](https://github.com/google/ExoPlayer/issues/9766)). * Amend logic in `AdaptiveTrackSelection` to allow a quality increase under sufficient network bandwidth even if playback is very close to the - live edge ((#9784)[https://github.com/google/ExoPlayer/issues/9784]). + live edge ([#9784](https://github.com/google/ExoPlayer/issues/9784)). * Fix Maven dependency resolution - ((#8353)[https://github.com/google/ExoPlayer/issues/8353]). + ([#8353](https://github.com/google/ExoPlayer/issues/8353)). * Fix decoder fallback logic for Dolby Atmos (E-AC3-JOC) and Dolby Vision to use a compatible base decoder (E-AC3 or H264/H265) if needed. * Disable automatic speed adjustment for live streams that neither have low-latency features nor a user request setting the speed - ((#9329)[https://github.com/google/ExoPlayer/issues/9329]). + ([#9329](https://github.com/google/ExoPlayer/issues/9329)). + * Update video track selection logic to take preferred MIME types and role + flags into account when selecting multiple video tracks for adaptation + ([#9519](https://github.com/google/ExoPlayer/issues/9519)). + * Update video and audio track selection logic to only choose formats for + adaptive selections that have the same level of decoder and hardware + support ([#9565](https://github.com/google/ExoPlayer/issues/9565)). + * Update video track selection logic to prefer more efficient codecs if + multiple codecs are supported by primary, hardware-accelerated decoders + ([#4835](https://github.com/google/ExoPlayer/issues/4835)). + * Rename `DecoderCounters#inputBufferCount` to `queuedInputBufferCount`. * Android 12 compatibility: * Upgrade the Cast extension to depend on `com.google.android.gms:play-services-cast-framework:20.1.0`. Earlier @@ -43,28 +53,40 @@ constructors. * Change `AudioCapabilities` APIs to require passing explicitly `AudioCapabilities.DEFAULT_AUDIO_CAPABILITIES` instead of `null`. + * Allow customization of the `AudioTrack` buffer size calculation by + injecting an `AudioTrackBufferSizeProvider` to `DefaultAudioSink`. + ([#8891](https://github.com/google/ExoPlayer/issues/8891)). * Extractors: * Fix inconsistency with spec in H.265 SPS nal units parsing - ((#9719)[https://github.com/google/ExoPlayer/issues/9719]). + ([#9719](https://github.com/google/ExoPlayer/issues/9719)). + * Parse Vorbis Comments (including `METADATA_BLOCK_PICTURE`) in Ogg Opus + and Vorbis files. * Text: * Add a `MediaItem.SubtitleConfiguration#id` field which is propagated to the `Format#id` field of the subtitle track created from the configuration - ((#9673)[https://github.com/google/ExoPlayer/issues/9673]). - * Rename `DecoderCounters#inputBufferCount` to `queuedInputBufferCount`. + ([#9673](https://github.com/google/ExoPlayer/issues/9673)). + * Add basic support for WebVTT subtitles in Matroska containers + ([#9886](https://github.com/google/ExoPlayer/issues/9886)). * DRM: * Remove `playbackLooper` from `DrmSessionManager.(pre)acquireSession`. When a `DrmSessionManager` is used by an app in a custom `MediaSource`, the `playbackLooper` needs to be passed to `DrmSessionManager.setPlayer` instead. -* IMA: +* Ad playback / IMA: * Add a method to `AdPlaybackState` to allow resetting an ad group so that it can be played again ([#9615](https://github.com/google/ExoPlayer/issues/9615)). + * Enforce playback speed of 1.0 during ad playback + ([#9018](https://github.com/google/ExoPlayer/issues/9018)). * DASH: * Support the `forced-subtitle` track role ([#9727](https://github.com/google/ExoPlayer/issues/9727)). * Stop interpreting the `main` track role as `C.SELECTION_FLAG_DEFAULT`. + * Fix bug when base URLs have been assigned the same service location and + priority in manifests that do not declare the dvb namespace. This + prevents the exclusion logic to exclude base URL when they actually + should be used as a fallback base URL. * HLS: * Use chunkless preparation by default to improve start up time. If your renditions contain muxed closed-caption tracks that are *not* declared @@ -81,23 +103,36 @@ * Fix the color of the numbers in `StyledPlayerView` rewind and fastforward buttons when using certain themes ([#9765](https://github.com/google/ExoPlayer/issues/9765)). + * Correctly translate playback speed strings + ([#9811](https://github.com/google/ExoPlayer/issues/9811)). * Transformer: * Increase required min API version to 21. * `TransformationException` is now used to describe errors that occur during a transformation. * Add `TransformationRequest` for specifying the transformation options. * Allow multiple listeners to be registered. + * Fix Transformer being stuck when the codec output is partially read. + * Fix potential NPE in `Transformer.getProgress` when releasing the muxer + throws. + * Add a demo app for applying transformations. * MediaSession extension: * Remove deprecated call to `onStop(/* reset= */ true)` and provide an opt-out flag for apps that don't want to clear the playlist on stop. * RTSP: * Provide a client API to override the `SocketFactory` used for any server connection ([#9606](https://github.com/google/ExoPlayer/pull/9606)). - * Prefers DIGEST authentication method over BASIC if both are present. + * Prefers DIGEST authentication method over BASIC if both are present ([#9800](https://github.com/google/ExoPlayer/issues/9800)). + * Handle when RTSP track timing is not available + ([#9775](https://github.com/google/ExoPlayer/issues/9775)). + * Ignores invalid RTP-Info header values + ([#9619](https://github.com/google/ExoPlayer/issues/9619)). * Cast extension * Fix bug that prevented `CastPlayer` from calling `onIsPlayingChanged` - correctly. + correctly ([#9792](https://github.com/google/ExoPlayer/issues/9792)). + * Support audio metadata including artwork with + `DefaultMediaItemConverter` + ([#9663](https://github.com/google/ExoPlayer/issues/9663)). * Remove deprecated symbols: * Remove `MediaSourceFactory#setDrmSessionManager`, `MediaSourceFactory#setDrmHttpDataSourceFactory`, and @@ -114,6 +149,8 @@ `MediaItem.LiveConfiguration.Builder#setTargetOffsetMs` to override the manifest, or `DashMediaSource#setFallbackTargetLiveOffsetMs` to provide a fallback value. + * Remove `(Simple)ExoPlayer.setThrowsWhenUsingWrongThread`. Opting out of + the thread enforcement is no longer possible. ### 2.16.1 (2021-11-18) diff --git a/constants.gradle b/constants.gradle index 853ff6ff20..997a0d3fd1 100644 --- a/constants.gradle +++ b/constants.gradle @@ -37,6 +37,7 @@ project.ext { androidxAnnotationVersion = '1.3.0' androidxAppCompatVersion = '1.3.1' androidxCollectionVersion = '1.1.0' + androidxConstraintLayoutVersion = '2.0.4' androidxCoreVersion = '1.7.0' androidxFuturesVersion = '1.1.0' androidxMediaVersion = '1.4.3' diff --git a/demos/cast/src/main/java/com/google/android/exoplayer2/castdemo/PlayerManager.java b/demos/cast/src/main/java/com/google/android/exoplayer2/castdemo/PlayerManager.java index 32fb65c2fa..8d78ca1112 100644 --- a/demos/cast/src/main/java/com/google/android/exoplayer2/castdemo/PlayerManager.java +++ b/demos/cast/src/main/java/com/google/android/exoplayer2/castdemo/PlayerManager.java @@ -223,10 +223,12 @@ import java.util.ArrayList; if (currentPlayer != localPlayer || tracksInfo == lastSeenTrackGroupInfo) { return; } - if (!tracksInfo.isTypeSupportedOrEmpty(C.TRACK_TYPE_VIDEO)) { + if (!tracksInfo.isTypeSupportedOrEmpty( + C.TRACK_TYPE_VIDEO, /* allowExceedsCapabilities= */ true)) { listener.onUnsupportedTrack(C.TRACK_TYPE_VIDEO); } - if (!tracksInfo.isTypeSupportedOrEmpty(C.TRACK_TYPE_AUDIO)) { + if (!tracksInfo.isTypeSupportedOrEmpty( + C.TRACK_TYPE_AUDIO, /* allowExceedsCapabilities= */ true)) { listener.onUnsupportedTrack(C.TRACK_TYPE_AUDIO); } lastSeenTrackGroupInfo = tracksInfo; diff --git a/demos/gl/src/main/assets/bitmap_overlay_video_processor_fragment.glsl b/demos/gl/src/main/assets/bitmap_overlay_video_processor_fragment.glsl index 17fec0601d..1c39979a50 100644 --- a/demos/gl/src/main/assets/bitmap_overlay_video_processor_fragment.glsl +++ b/demos/gl/src/main/assets/bitmap_overlay_video_processor_fragment.glsl @@ -15,19 +15,19 @@ #extension GL_OES_EGL_image_external : require precision mediump float; // External texture containing video decoder output. -uniform samplerExternalOES tex_sampler_0; +uniform samplerExternalOES uTexSampler0; // Texture containing the overlap bitmap. -uniform sampler2D tex_sampler_1; +uniform sampler2D uTexSampler1; // Horizontal scaling factor for the overlap bitmap. -uniform float scaleX; +uniform float uScaleX; // Vertical scaling factory for the overlap bitmap. -uniform float scaleY; -varying vec2 v_texcoord; +uniform float uScaleY; +varying vec2 vTexCoords; void main() { - vec4 videoColor = texture2D(tex_sampler_0, v_texcoord); - vec4 overlayColor = texture2D(tex_sampler_1, - vec2(v_texcoord.x * scaleX, - v_texcoord.y * scaleY)); + vec4 videoColor = texture2D(uTexSampler0, vTexCoords); + vec4 overlayColor = texture2D(uTexSampler1, + vec2(vTexCoords.x * uScaleX, + vTexCoords.y * uScaleY)); // Blend the video decoder output and the overlay bitmap. gl_FragColor = videoColor * (1.0 - overlayColor.a) + overlayColor * overlayColor.a; diff --git a/demos/gl/src/main/assets/bitmap_overlay_video_processor_vertex.glsl b/demos/gl/src/main/assets/bitmap_overlay_video_processor_vertex.glsl index 1cb01b8293..b10aa6880e 100644 --- a/demos/gl/src/main/assets/bitmap_overlay_video_processor_vertex.glsl +++ b/demos/gl/src/main/assets/bitmap_overlay_video_processor_vertex.glsl @@ -11,11 +11,11 @@ // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. -attribute vec4 a_position; -attribute vec4 a_texcoord; -uniform mat4 tex_transform; -varying vec2 v_texcoord; +attribute vec4 aFramePosition; +attribute vec4 aTexCoords; +uniform mat4 uTexTransform; +varying vec2 vTexCoords; void main() { - gl_Position = a_position; - v_texcoord = (tex_transform * a_texcoord).xy; + gl_Position = aFramePosition; + vTexCoords = (uTexTransform * aTexCoords).xy; } diff --git a/demos/gl/src/main/java/com/google/android/exoplayer2/gldemo/BitmapOverlayVideoProcessor.java b/demos/gl/src/main/java/com/google/android/exoplayer2/gldemo/BitmapOverlayVideoProcessor.java index 1294990ec5..8a5d135dee 100644 --- a/demos/gl/src/main/java/com/google/android/exoplayer2/gldemo/BitmapOverlayVideoProcessor.java +++ b/demos/gl/src/main/java/com/google/android/exoplayer2/gldemo/BitmapOverlayVideoProcessor.java @@ -86,9 +86,9 @@ import org.checkerframework.checker.nullness.qual.MonotonicNonNull; throw new IllegalStateException(e); } program.setBufferAttribute( - "a_position", GlUtil.getNormalizedCoordinateBounds(), GlUtil.RECTANGLE_VERTICES_COUNT); + "aFramePosition", GlUtil.getNormalizedCoordinateBounds(), GlUtil.RECTANGLE_VERTICES_COUNT); program.setBufferAttribute( - "a_texcoord", GlUtil.getTextureCoordinateBounds(), GlUtil.RECTANGLE_VERTICES_COUNT); + "aTexCoords", GlUtil.getTextureCoordinateBounds(), GlUtil.RECTANGLE_VERTICES_COUNT); GLES20.glGenTextures(1, textures, 0); GLES20.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]); GLES20.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_NEAREST); @@ -118,11 +118,11 @@ import org.checkerframework.checker.nullness.qual.MonotonicNonNull; // Run the shader program. GlUtil.Program program = checkNotNull(this.program); - program.setSamplerTexIdUniform("tex_sampler_0", frameTexture, /* unit= */ 0); - program.setSamplerTexIdUniform("tex_sampler_1", textures[0], /* unit= */ 1); - program.setFloatUniform("scaleX", bitmapScaleX); - program.setFloatUniform("scaleY", bitmapScaleY); - program.setFloatsUniform("tex_transform", transformMatrix); + program.setSamplerTexIdUniform("uTexSampler0", frameTexture, /* unit= */ 0); + program.setSamplerTexIdUniform("uTexSampler1", textures[0], /* unit= */ 1); + program.setFloatUniform("uScaleX", bitmapScaleX); + program.setFloatUniform("uScaleY", bitmapScaleY); + program.setFloatsUniform("uTexTransform", transformMatrix); program.bindAttributesAndUniforms(); GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT); GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, /* first= */ 0, /* count= */ 4); diff --git a/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java b/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java index d0aa4d58af..eaff9a185a 100644 --- a/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java +++ b/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java @@ -441,10 +441,12 @@ public class PlayerActivity extends AppCompatActivity if (tracksInfo == lastSeenTracksInfo) { return; } - if (!tracksInfo.isTypeSupportedOrEmpty(C.TRACK_TYPE_VIDEO)) { + if (!tracksInfo.isTypeSupportedOrEmpty( + C.TRACK_TYPE_VIDEO, /* allowExceedsCapabilities= */ true)) { showToast(R.string.error_unsupported_video); } - if (!tracksInfo.isTypeSupportedOrEmpty(C.TRACK_TYPE_AUDIO)) { + if (!tracksInfo.isTypeSupportedOrEmpty( + C.TRACK_TYPE_AUDIO, /* allowExceedsCapabilities= */ true)) { showToast(R.string.error_unsupported_audio); } lastSeenTracksInfo = tracksInfo; diff --git a/demos/surface/src/main/AndroidManifest.xml b/demos/surface/src/main/AndroidManifest.xml index fab5b2070c..9e9cbeed5c 100644 --- a/demos/surface/src/main/AndroidManifest.xml +++ b/demos/surface/src/main/AndroidManifest.xml @@ -22,12 +22,14 @@ + android:allowBackup="false" + android:icon="@mipmap/ic_launcher" + android:label="@string/application_name" + android:exported="true"> - + diff --git a/demos/transformer/README.md b/demos/transformer/README.md new file mode 100644 index 0000000000..fb2657001e --- /dev/null +++ b/demos/transformer/README.md @@ -0,0 +1,9 @@ +# Transformer demo + +This app demonstrates how to use the [Transformer][] API to modify videos, for +example by removing audio or video. + +See the [demos README](../README.md) for instructions on how to build and run +this demo. + +[Transformer]: https://exoplayer.dev/transforming-media.html diff --git a/demos/transformer/build.gradle b/demos/transformer/build.gradle new file mode 100644 index 0000000000..179d52dd0a --- /dev/null +++ b/demos/transformer/build.gradle @@ -0,0 +1,60 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +apply from: '../../constants.gradle' +apply plugin: 'com.android.application' + +android { + compileSdkVersion project.ext.compileSdkVersion + + compileOptions { + sourceCompatibility JavaVersion.VERSION_1_8 + targetCompatibility JavaVersion.VERSION_1_8 + } + + defaultConfig { + versionName project.ext.releaseVersion + versionCode project.ext.releaseVersionCode + minSdkVersion 21 + targetSdkVersion project.ext.appTargetSdkVersion + multiDexEnabled true + } + + buildTypes { + release { + shrinkResources true + minifyEnabled true + proguardFiles getDefaultProguardFile('proguard-android.txt') + signingConfig signingConfigs.debug + } + } + + lintOptions { + // This demo app isn't indexed and doesn't have translations. + disable 'GoogleAppIndexingWarning','MissingTranslation' + } +} + +dependencies { + compileOnly 'org.checkerframework:checker-qual:' + checkerframeworkVersion + implementation 'androidx.annotation:annotation:' + androidxAnnotationVersion + implementation 'androidx.appcompat:appcompat:' + androidxAppCompatVersion + implementation 'androidx.constraintlayout:constraintlayout:' + androidxConstraintLayoutVersion + implementation 'androidx.multidex:multidex:' + androidxMultidexVersion + implementation 'com.google.android.material:material:' + androidxMaterialVersion + implementation project(modulePrefix + 'library-core') + implementation project(modulePrefix + 'library-transformer') + implementation project(modulePrefix + 'library-ui') +} diff --git a/demos/transformer/src/main/AndroidManifest.xml b/demos/transformer/src/main/AndroidManifest.xml new file mode 100644 index 0000000000..f309d20851 --- /dev/null +++ b/demos/transformer/src/main/AndroidManifest.xml @@ -0,0 +1,61 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/ConfigurationActivity.java b/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/ConfigurationActivity.java new file mode 100644 index 0000000000..7b1cd08cba --- /dev/null +++ b/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/ConfigurationActivity.java @@ -0,0 +1,302 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformerdemo; + +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; + +import android.app.Activity; +import android.content.DialogInterface; +import android.content.Intent; +import android.net.Uri; +import android.os.Bundle; +import android.view.View; +import android.widget.ArrayAdapter; +import android.widget.Button; +import android.widget.CheckBox; +import android.widget.Spinner; +import android.widget.TextView; +import androidx.annotation.Nullable; +import androidx.appcompat.app.AlertDialog; +import androidx.appcompat.app.AppCompatActivity; +import com.google.android.exoplayer2.util.MimeTypes; +import java.util.Arrays; +import java.util.List; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.checkerframework.checker.nullness.qual.RequiresNonNull; + +/** + * An {@link Activity} that sets the configuration to use for transforming and playing media, using + * {@link TransformerActivity}. + */ +public final class ConfigurationActivity extends AppCompatActivity { + public static final String SHOULD_REMOVE_AUDIO = "should_remove_audio"; + public static final String SHOULD_REMOVE_VIDEO = "should_remove_video"; + public static final String SHOULD_FLATTEN_FOR_SLOW_MOTION = "should_flatten_for_slow_motion"; + public static final String AUDIO_MIME_TYPE = "audio_mime_type"; + public static final String VIDEO_MIME_TYPE = "video_mime_type"; + public static final String RESOLUTION_HEIGHT = "resolution_height"; + public static final String TRANSLATE_X = "translate_x"; + public static final String TRANSLATE_Y = "translate_y"; + public static final String SCALE_X = "scale_x"; + public static final String SCALE_Y = "scale_y"; + public static final String ROTATE_DEGREES = "rotate_degrees"; + private static final String[] INPUT_URIS = { + "https://html5demos.com/assets/dizzy.mp4", + "https://storage.googleapis.com/exoplayer-test-media-0/android-block-1080-hevc.mp4", + "https://storage.googleapis.com/exoplayer-test-media-0/BigBuckBunny_320x180.mp4", + "https://html5demos.com/assets/dizzy.webm", + }; + private static final String[] URI_DESCRIPTIONS = { // same order as INPUT_URIS + "MP4 with H264 video and AAC audio", + "MP4 with H265 video and AAC audio", + "Long MP4 with H264 video and AAC audio", + "WebM with VP8 video and Vorbis audio", + }; + private static final String SAME_AS_INPUT_OPTION = "same as input"; + + private @MonotonicNonNull Button chooseFileButton; + private @MonotonicNonNull CheckBox removeAudioCheckbox; + private @MonotonicNonNull CheckBox removeVideoCheckbox; + private @MonotonicNonNull CheckBox flattenForSlowMotionCheckbox; + private @MonotonicNonNull Spinner audioMimeSpinner; + private @MonotonicNonNull Spinner videoMimeSpinner; + private @MonotonicNonNull Spinner resolutionHeightSpinner; + private @MonotonicNonNull Spinner translateSpinner; + private @MonotonicNonNull Spinner scaleSpinner; + private @MonotonicNonNull Spinner rotateSpinner; + private @MonotonicNonNull TextView chosenFileTextView; + private int inputUriPosition; + + @Override + protected void onCreate(@Nullable Bundle savedInstanceState) { + super.onCreate(savedInstanceState); + setContentView(R.layout.configuration_activity); + + findViewById(R.id.transform_button).setOnClickListener(this::startTransformation); + + chooseFileButton = findViewById(R.id.choose_file_button); + chooseFileButton.setOnClickListener(this::chooseFile); + + chosenFileTextView = findViewById(R.id.chosen_file_text_view); + chosenFileTextView.setText(URI_DESCRIPTIONS[inputUriPosition]); + + removeAudioCheckbox = findViewById(R.id.remove_audio_checkbox); + removeAudioCheckbox.setOnClickListener(this::onRemoveAudio); + + removeVideoCheckbox = findViewById(R.id.remove_video_checkbox); + removeVideoCheckbox.setOnClickListener(this::onRemoveVideo); + + flattenForSlowMotionCheckbox = findViewById(R.id.flatten_for_slow_motion_checkbox); + + ArrayAdapter audioMimeAdapter = + new ArrayAdapter<>(/* context= */ this, R.layout.spinner_item); + audioMimeAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item); + audioMimeSpinner = findViewById(R.id.audio_mime_spinner); + audioMimeSpinner.setAdapter(audioMimeAdapter); + audioMimeAdapter.addAll( + SAME_AS_INPUT_OPTION, MimeTypes.AUDIO_AAC, MimeTypes.AUDIO_AMR_NB, MimeTypes.AUDIO_AMR_WB); + + ArrayAdapter videoMimeAdapter = + new ArrayAdapter<>(/* context= */ this, R.layout.spinner_item); + videoMimeAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item); + videoMimeSpinner = findViewById(R.id.video_mime_spinner); + videoMimeSpinner.setAdapter(videoMimeAdapter); + videoMimeAdapter.addAll( + SAME_AS_INPUT_OPTION, + MimeTypes.VIDEO_H263, + MimeTypes.VIDEO_H264, + MimeTypes.VIDEO_H265, + MimeTypes.VIDEO_MP4V); + + ArrayAdapter resolutionHeightAdapter = + new ArrayAdapter<>(/* context= */ this, R.layout.spinner_item); + resolutionHeightAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item); + resolutionHeightSpinner = findViewById(R.id.resolution_height_spinner); + resolutionHeightSpinner.setAdapter(resolutionHeightAdapter); + resolutionHeightAdapter.addAll( + SAME_AS_INPUT_OPTION, "144", "240", "360", "480", "720", "1080", "1440", "2160"); + + ArrayAdapter translateAdapter = + new ArrayAdapter<>(/* context= */ this, R.layout.spinner_item); + translateAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item); + translateSpinner = findViewById(R.id.translate_spinner); + translateSpinner.setAdapter(translateAdapter); + translateAdapter.addAll( + SAME_AS_INPUT_OPTION, "-.1, -.1", "0, 0", ".5, 0", "0, .5", "1, 1", "1.9, 0", "0, 1.9"); + + ArrayAdapter scaleAdapter = + new ArrayAdapter<>(/* context= */ this, R.layout.spinner_item); + scaleAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item); + scaleSpinner = findViewById(R.id.scale_spinner); + scaleSpinner.setAdapter(scaleAdapter); + scaleAdapter.addAll(SAME_AS_INPUT_OPTION, "-1, -1", "-1, 1", "1, 1", ".5, 1", ".5, .5", "2, 2"); + + ArrayAdapter rotateAdapter = + new ArrayAdapter<>(/* context= */ this, R.layout.spinner_item); + rotateAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item); + rotateSpinner = findViewById(R.id.rotate_spinner); + rotateSpinner.setAdapter(rotateAdapter); + rotateAdapter.addAll(SAME_AS_INPUT_OPTION, "0", "10", "45", "90", "180"); + } + + @Override + protected void onResume() { + super.onResume(); + @Nullable Uri intentUri = getIntent().getData(); + if (intentUri != null) { + checkNotNull(chooseFileButton).setEnabled(false); + checkNotNull(chosenFileTextView).setText(intentUri.toString()); + } + } + + @Override + protected void onNewIntent(Intent intent) { + super.onNewIntent(intent); + setIntent(intent); + } + + @RequiresNonNull({ + "removeAudioCheckbox", + "removeVideoCheckbox", + "flattenForSlowMotionCheckbox", + "audioMimeSpinner", + "videoMimeSpinner", + "resolutionHeightSpinner", + "translateSpinner", + "scaleSpinner", + "rotateSpinner" + }) + private void startTransformation(View view) { + Intent transformerIntent = new Intent(this, TransformerActivity.class); + Bundle bundle = new Bundle(); + bundle.putBoolean(SHOULD_REMOVE_AUDIO, removeAudioCheckbox.isChecked()); + bundle.putBoolean(SHOULD_REMOVE_VIDEO, removeVideoCheckbox.isChecked()); + bundle.putBoolean(SHOULD_FLATTEN_FOR_SLOW_MOTION, flattenForSlowMotionCheckbox.isChecked()); + String selectedAudioMimeType = String.valueOf(audioMimeSpinner.getSelectedItem()); + if (!SAME_AS_INPUT_OPTION.equals(selectedAudioMimeType)) { + bundle.putString(AUDIO_MIME_TYPE, selectedAudioMimeType); + } + String selectedVideoMimeType = String.valueOf(videoMimeSpinner.getSelectedItem()); + if (!SAME_AS_INPUT_OPTION.equals(selectedVideoMimeType)) { + bundle.putString(VIDEO_MIME_TYPE, selectedVideoMimeType); + } + String selectedResolutionHeight = String.valueOf(resolutionHeightSpinner.getSelectedItem()); + if (!SAME_AS_INPUT_OPTION.equals(selectedResolutionHeight)) { + bundle.putInt(RESOLUTION_HEIGHT, Integer.valueOf(selectedResolutionHeight)); + } + String selectedTranslate = String.valueOf(translateSpinner.getSelectedItem()); + if (!SAME_AS_INPUT_OPTION.equals(selectedTranslate)) { + List translateXY = Arrays.asList(selectedTranslate.split(", ")); + checkState(translateXY.size() == 2); + bundle.putFloat(TRANSLATE_X, Float.valueOf(translateXY.get(0))); + bundle.putFloat(TRANSLATE_Y, Float.valueOf(translateXY.get(1))); + } + String selectedScale = String.valueOf(scaleSpinner.getSelectedItem()); + if (!SAME_AS_INPUT_OPTION.equals(selectedScale)) { + List scaleXY = Arrays.asList(selectedScale.split(", ")); + checkState(scaleXY.size() == 2); + bundle.putFloat(SCALE_X, Float.valueOf(scaleXY.get(0))); + bundle.putFloat(SCALE_Y, Float.valueOf(scaleXY.get(1))); + } + String selectedRotate = String.valueOf(rotateSpinner.getSelectedItem()); + if (!SAME_AS_INPUT_OPTION.equals(selectedRotate)) { + bundle.putFloat(ROTATE_DEGREES, Float.valueOf(selectedRotate)); + } + transformerIntent.putExtras(bundle); + + @Nullable Uri intentUri = getIntent().getData(); + transformerIntent.setData( + intentUri != null ? intentUri : Uri.parse(INPUT_URIS[inputUriPosition])); + + startActivity(transformerIntent); + } + + private void chooseFile(View view) { + new AlertDialog.Builder(/* context= */ this) + .setTitle(R.string.choose_file_title) + .setSingleChoiceItems(URI_DESCRIPTIONS, inputUriPosition, this::selectFileInDialog) + .setPositiveButton(android.R.string.ok, /* listener= */ null) + .create() + .show(); + } + + @RequiresNonNull("chosenFileTextView") + private void selectFileInDialog(DialogInterface dialog, int which) { + inputUriPosition = which; + chosenFileTextView.setText(URI_DESCRIPTIONS[inputUriPosition]); + } + + @RequiresNonNull({ + "removeVideoCheckbox", + "audioMimeSpinner", + "videoMimeSpinner", + "resolutionHeightSpinner", + "translateSpinner", + "scaleSpinner", + "rotateSpinner" + }) + private void onRemoveAudio(View view) { + if (((CheckBox) view).isChecked()) { + removeVideoCheckbox.setChecked(false); + enableTrackSpecificOptions(/* isAudioEnabled= */ false, /* isVideoEnabled= */ true); + } else { + enableTrackSpecificOptions(/* isAudioEnabled= */ true, /* isVideoEnabled= */ true); + } + } + + @RequiresNonNull({ + "removeAudioCheckbox", + "audioMimeSpinner", + "videoMimeSpinner", + "resolutionHeightSpinner", + "translateSpinner", + "scaleSpinner", + "rotateSpinner" + }) + private void onRemoveVideo(View view) { + if (((CheckBox) view).isChecked()) { + removeAudioCheckbox.setChecked(false); + enableTrackSpecificOptions(/* isAudioEnabled= */ true, /* isVideoEnabled= */ false); + } else { + enableTrackSpecificOptions(/* isAudioEnabled= */ true, /* isVideoEnabled= */ true); + } + } + + @RequiresNonNull({ + "audioMimeSpinner", + "videoMimeSpinner", + "resolutionHeightSpinner", + "translateSpinner", + "scaleSpinner", + "rotateSpinner" + }) + private void enableTrackSpecificOptions(boolean isAudioEnabled, boolean isVideoEnabled) { + audioMimeSpinner.setEnabled(isAudioEnabled); + videoMimeSpinner.setEnabled(isVideoEnabled); + resolutionHeightSpinner.setEnabled(isVideoEnabled); + translateSpinner.setEnabled(isVideoEnabled); + scaleSpinner.setEnabled(isVideoEnabled); + rotateSpinner.setEnabled(isVideoEnabled); + + findViewById(R.id.audio_mime_text_view).setEnabled(isAudioEnabled); + findViewById(R.id.video_mime_text_view).setEnabled(isVideoEnabled); + findViewById(R.id.resolution_height_text_view).setEnabled(isVideoEnabled); + findViewById(R.id.translate).setEnabled(isVideoEnabled); + findViewById(R.id.scale).setEnabled(isVideoEnabled); + findViewById(R.id.rotate).setEnabled(isVideoEnabled); + } +} diff --git a/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/TransformerActivity.java b/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/TransformerActivity.java new file mode 100644 index 0000000000..888ddd3edc --- /dev/null +++ b/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/TransformerActivity.java @@ -0,0 +1,372 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformerdemo; + +import static android.Manifest.permission.READ_EXTERNAL_STORAGE; +import static android.Manifest.permission.WRITE_EXTERNAL_STORAGE; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; + +import android.app.Activity; +import android.content.Intent; +import android.content.pm.PackageManager; +import android.graphics.Matrix; +import android.net.Uri; +import android.os.Bundle; +import android.os.Handler; +import android.view.SurfaceHolder; +import android.view.SurfaceView; +import android.view.View; +import android.view.ViewGroup; +import android.widget.TextView; +import android.widget.Toast; +import androidx.annotation.Nullable; +import androidx.appcompat.app.AppCompatActivity; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.ExoPlayer; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.transformer.ProgressHolder; +import com.google.android.exoplayer2.transformer.TransformationException; +import com.google.android.exoplayer2.transformer.TransformationRequest; +import com.google.android.exoplayer2.transformer.Transformer; +import com.google.android.exoplayer2.ui.AspectRatioFrameLayout; +import com.google.android.exoplayer2.ui.StyledPlayerView; +import com.google.android.exoplayer2.util.DebugTextViewHelper; +import com.google.android.exoplayer2.util.Log; +import com.google.android.exoplayer2.util.Util; +import com.google.android.material.progressindicator.LinearProgressIndicator; +import com.google.common.base.Stopwatch; +import com.google.common.base.Ticker; +import java.io.File; +import java.io.IOException; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.checkerframework.checker.nullness.qual.RequiresNonNull; + +/** An {@link Activity} that transforms and plays media using {@link Transformer}. */ +public final class TransformerActivity extends AppCompatActivity { + private static final String TAG = "TransformerActivity"; + + private @MonotonicNonNull StyledPlayerView playerView; + private @MonotonicNonNull TextView debugTextView; + private @MonotonicNonNull TextView informationTextView; + private @MonotonicNonNull ViewGroup progressViewGroup; + private @MonotonicNonNull LinearProgressIndicator progressIndicator; + private @MonotonicNonNull Stopwatch transformationStopwatch; + private @MonotonicNonNull AspectRatioFrameLayout debugFrame; + + @Nullable private DebugTextViewHelper debugTextViewHelper; + @Nullable private ExoPlayer player; + @Nullable private Transformer transformer; + @Nullable private File externalCacheFile; + + @Override + protected void onCreate(@Nullable Bundle savedInstanceState) { + super.onCreate(savedInstanceState); + setContentView(R.layout.transformer_activity); + + playerView = findViewById(R.id.player_view); + debugTextView = findViewById(R.id.debug_text_view); + informationTextView = findViewById(R.id.information_text_view); + progressViewGroup = findViewById(R.id.progress_view_group); + progressIndicator = findViewById(R.id.progress_indicator); + debugFrame = findViewById(R.id.debug_aspect_ratio_frame_layout); + + transformationStopwatch = + Stopwatch.createUnstarted( + new Ticker() { + public long read() { + return android.os.SystemClock.elapsedRealtimeNanos(); + } + }); + } + + @Override + protected void onStart() { + super.onStart(); + + checkNotNull(progressIndicator); + checkNotNull(informationTextView); + checkNotNull(transformationStopwatch); + checkNotNull(playerView); + checkNotNull(debugTextView); + checkNotNull(progressViewGroup); + startTransformation(); + + playerView.onResume(); + } + + @Override + protected void onStop() { + super.onStop(); + + checkNotNull(transformationStopwatch).reset(); + + checkNotNull(transformer).cancel(); + transformer = null; + + checkNotNull(playerView).onPause(); + releasePlayer(); + + checkNotNull(externalCacheFile).delete(); + externalCacheFile = null; + } + + @RequiresNonNull({ + "playerView", + "debugTextView", + "informationTextView", + "progressIndicator", + "transformationStopwatch", + "progressViewGroup", + }) + private void startTransformation() { + requestTransformerPermissions(); + + Intent intent = getIntent(); + Uri uri = checkNotNull(intent.getData()); + try { + externalCacheFile = createExternalCacheFile("transformer-output.mp4"); + String filePath = externalCacheFile.getAbsolutePath(); + @Nullable Bundle bundle = intent.getExtras(); + Transformer transformer = createTransformer(bundle, filePath); + transformationStopwatch.start(); + transformer.startTransformation(MediaItem.fromUri(uri), filePath); + this.transformer = transformer; + } catch (IOException e) { + throw new IllegalStateException(e); + } + informationTextView.setText(R.string.transformation_started); + Handler mainHandler = new Handler(getMainLooper()); + ProgressHolder progressHolder = new ProgressHolder(); + mainHandler.post( + new Runnable() { + @Override + public void run() { + if (transformer != null + && transformer.getProgress(progressHolder) + != Transformer.PROGRESS_STATE_NO_TRANSFORMATION) { + progressIndicator.setProgress(progressHolder.progress); + informationTextView.setText( + getString( + R.string.transformation_timer, + transformationStopwatch.elapsed(TimeUnit.SECONDS))); + mainHandler.postDelayed(/* r= */ this, /* delayMillis= */ 500); + } + } + }); + } + + // Create a cache file, resetting it if it already exists. + private File createExternalCacheFile(String fileName) throws IOException { + File file = new File(getExternalCacheDir(), fileName); + if (file.exists() && !file.delete()) { + throw new IllegalStateException("Could not delete the previous transformer output file"); + } + if (!file.createNewFile()) { + throw new IllegalStateException("Could not create the transformer output file"); + } + return file; + } + + @RequiresNonNull({ + "playerView", + "debugTextView", + "informationTextView", + "transformationStopwatch", + "progressViewGroup", + }) + private Transformer createTransformer(@Nullable Bundle bundle, String filePath) { + Transformer.Builder transformerBuilder = new Transformer.Builder(/* context= */ this); + if (bundle != null) { + TransformationRequest.Builder requestBuilder = new TransformationRequest.Builder(); + requestBuilder.setFlattenForSlowMotion( + bundle.getBoolean(ConfigurationActivity.SHOULD_FLATTEN_FOR_SLOW_MOTION)); + @Nullable String audioMimeType = bundle.getString(ConfigurationActivity.AUDIO_MIME_TYPE); + if (audioMimeType != null) { + requestBuilder.setAudioMimeType(audioMimeType); + } + @Nullable String videoMimeType = bundle.getString(ConfigurationActivity.VIDEO_MIME_TYPE); + if (videoMimeType != null) { + requestBuilder.setVideoMimeType(videoMimeType); + } + int resolutionHeight = + bundle.getInt( + ConfigurationActivity.RESOLUTION_HEIGHT, /* defaultValue= */ C.LENGTH_UNSET); + if (resolutionHeight != C.LENGTH_UNSET) { + requestBuilder.setResolution(resolutionHeight); + } + Matrix transformationMatrix = getTransformationMatrix(bundle); + if (!transformationMatrix.isIdentity()) { + requestBuilder.setTransformationMatrix(transformationMatrix); + } + transformerBuilder + .setTransformationRequest(requestBuilder.build()) + .setRemoveAudio(bundle.getBoolean(ConfigurationActivity.SHOULD_REMOVE_AUDIO)) + .setRemoveVideo(bundle.getBoolean(ConfigurationActivity.SHOULD_REMOVE_VIDEO)); + } + return transformerBuilder + .addListener( + new Transformer.Listener() { + @Override + public void onTransformationCompleted(MediaItem mediaItem) { + TransformerActivity.this.onTransformationCompleted(filePath); + } + + @Override + public void onTransformationError( + MediaItem mediaItem, TransformationException exception) { + TransformerActivity.this.onTransformationError(exception); + } + }) + .setDebugViewProvider(new DemoDebugViewProvider()) + .build(); + } + + private static Matrix getTransformationMatrix(Bundle bundle) { + Matrix transformationMatrix = new Matrix(); + + float translateX = bundle.getFloat(ConfigurationActivity.TRANSLATE_X, /* defaultValue= */ 0); + float translateY = bundle.getFloat(ConfigurationActivity.TRANSLATE_Y, /* defaultValue= */ 0); + // TODO(b/213198690): Get resolution for aspect ratio and scale all translations' translateX + // by this aspect ratio. + transformationMatrix.postTranslate(translateX, translateY); + + float scaleX = bundle.getFloat(ConfigurationActivity.SCALE_X, /* defaultValue= */ 1); + float scaleY = bundle.getFloat(ConfigurationActivity.SCALE_Y, /* defaultValue= */ 1); + transformationMatrix.postScale(scaleX, scaleY); + + float rotateDegrees = + bundle.getFloat(ConfigurationActivity.ROTATE_DEGREES, /* defaultValue= */ 0); + transformationMatrix.postRotate(rotateDegrees); + + return transformationMatrix; + } + + @RequiresNonNull({ + "informationTextView", + "progressViewGroup", + "transformationStopwatch", + }) + private void onTransformationError(TransformationException exception) { + transformationStopwatch.stop(); + informationTextView.setText(R.string.transformation_error); + progressViewGroup.setVisibility(View.GONE); + Toast.makeText( + TransformerActivity.this, "Transformation error: " + exception, Toast.LENGTH_LONG) + .show(); + Log.e(TAG, "Transformation error", exception); + } + + @RequiresNonNull({ + "playerView", + "debugTextView", + "informationTextView", + "progressViewGroup", + "transformationStopwatch", + }) + private void onTransformationCompleted(String filePath) { + transformationStopwatch.stop(); + informationTextView.setText( + getString( + R.string.transformation_completed, transformationStopwatch.elapsed(TimeUnit.SECONDS))); + progressViewGroup.setVisibility(View.GONE); + playMediaItem(MediaItem.fromUri("file://" + filePath)); + Log.d(TAG, "Output file path: file://" + filePath); + } + + @RequiresNonNull({"playerView", "debugTextView"}) + private void playMediaItem(MediaItem mediaItem) { + playerView.setPlayer(null); + releasePlayer(); + + ExoPlayer player = new ExoPlayer.Builder(/* context= */ this).build(); + playerView.setPlayer(player); + player.setMediaItem(mediaItem); + player.play(); + player.prepare(); + this.player = player; + debugTextViewHelper = new DebugTextViewHelper(player, debugTextView); + debugTextViewHelper.start(); + } + + private void releasePlayer() { + if (debugTextViewHelper != null) { + debugTextViewHelper.stop(); + debugTextViewHelper = null; + } + if (player != null) { + player.release(); + player = null; + } + } + + private void requestTransformerPermissions() { + if (Util.SDK_INT < 23) { + return; + } + if (checkSelfPermission(READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED + || checkSelfPermission(WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) { + requestPermissions( + new String[] {READ_EXTERNAL_STORAGE, WRITE_EXTERNAL_STORAGE}, /* requestCode= */ 0); + } + } + + private final class DemoDebugViewProvider implements Transformer.DebugViewProvider { + + @Nullable + @Override + public SurfaceView getDebugPreviewSurfaceView(int width, int height) { + // Update the UI on the main thread and wait for the output surface to be available. + CountDownLatch surfaceCreatedCountDownLatch = new CountDownLatch(1); + SurfaceView surfaceView = new SurfaceView(/* context= */ TransformerActivity.this); + runOnUiThread( + () -> { + AspectRatioFrameLayout debugFrame = checkNotNull(TransformerActivity.this.debugFrame); + debugFrame.addView(surfaceView); + debugFrame.setAspectRatio((float) width / height); + surfaceView + .getHolder() + .addCallback( + new SurfaceHolder.Callback() { + @Override + public void surfaceCreated(SurfaceHolder surfaceHolder) { + surfaceCreatedCountDownLatch.countDown(); + } + + @Override + public void surfaceChanged( + SurfaceHolder surfaceHolder, int format, int width, int height) { + // Do nothing. + } + + @Override + public void surfaceDestroyed(SurfaceHolder surfaceHolder) { + // Do nothing. + } + }); + }); + try { + surfaceCreatedCountDownLatch.await(); + } catch (InterruptedException e) { + Log.w(TAG, "Interrupted waiting for debug surface."); + Thread.currentThread().interrupt(); + return null; + } + return surfaceView; + } + } +} diff --git a/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/package-info.java b/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/package-info.java new file mode 100644 index 0000000000..4996a4516d --- /dev/null +++ b/demos/transformer/src/main/java/com/google/android/exoplayer2/transformerdemo/package-info.java @@ -0,0 +1,19 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +@NonNullApi +package com.google.android.exoplayer2.transformerdemo; + +import com.google.android.exoplayer2.util.NonNullApi; diff --git a/demos/transformer/src/main/res/layout/configuration_activity.xml b/demos/transformer/src/main/res/layout/configuration_activity.xml new file mode 100644 index 0000000000..a9a9410a35 --- /dev/null +++ b/demos/transformer/src/main/res/layout/configuration_activity.xml @@ -0,0 +1,179 @@ + + + + + +