From c2fb2f15209319bbe67b42b773023ce3730fb00a Mon Sep 17 00:00:00 2001 From: dancho Date: Wed, 22 May 2024 05:35:50 -0700 Subject: [PATCH] Work around 1080p export failures on certain devices Fall back to using software decoder for 1920x1080 for certain devices. PiperOrigin-RevId: 636132298 --- .../mp4/long_1080p_videoonly_lowbitrate.mp4 | Bin 0 -> 446434 bytes .../media3/transformer/AndroidTestUtil.java | 11 +++ .../transformer/mh/TranscodeSpeedTest.java | 80 ++++++++++++++++++ .../transformer/DefaultDecoderFactory.java | 39 ++++++++- 4 files changed, 126 insertions(+), 4 deletions(-) create mode 100644 libraries/test_data/src/test/assets/media/mp4/long_1080p_videoonly_lowbitrate.mp4 create mode 100644 libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeSpeedTest.java diff --git a/libraries/test_data/src/test/assets/media/mp4/long_1080p_videoonly_lowbitrate.mp4 b/libraries/test_data/src/test/assets/media/mp4/long_1080p_videoonly_lowbitrate.mp4 new file mode 100644 index 0000000000000000000000000000000000000000..9ec8754c4eea4e42c159de1c3a056bae6ed3999f GIT binary patch literal 446434 zcmeI*UyNjFSqI>6S$07|7G#m(pB@BUPNsJXBX1Vf8L%P9~S4M#=FN_I7UFe0un0U1kH7R2>k{B^0CM0fLPj$aL-|M;) zaE8gy=RB3|>9_w>b)Bc*ug+H9UwvIlX)aBk-rd~a*)HW}W%<5+J3G3%zPi2p&}u2= zm6!I$<8trY|809?G-aC2^Ty!!r2?{B~G=;`(w&V1|_KKzjfA8J1Mi4V2kxiMaEzwn`* z-KV$4mnO~Gm6dZ3o?SV6uKncY$z=DDGiRQ9>Z#LLH#f#RTcazdclI7Tv-~NiFHg3& z+V9!foow!0*?**2A6*=+udOtD<4bGjnvLJs zZFP0E*&eT59$#zrpS;*sKill?KfSiP{P)7%#@g!XmG(;QKh5^$weiM<iK5<^4`w&=t6td)n>9c-rCySZ>xu|J-o3#Y45`N6Wi^-8{^T&CwH!l*UqlK zxBZEiM*EWsyZeuC?zZ0=H}J&ng`G>6_Q#X82hTQ>%X{rL_Seom)NJkSJU+VIestl5 zPrk3&-`ZSn@5~FISZS{89sQ8?&F#@-c}F&{OvZa#qxPHH&tKena&PqXh4r27-OLv$@+sx4+c(_t4(f=!y1sxcrTjX(^}f z8$a`zSAF~9w0!ZeK7U6pUwHD_U-KUU0t5&UAV7cs0RjXF5FkK+009C72oNA}I|Y7y zIhgw3!NYG{-1Xn5j_Y5#=R4&$f9tp&o|)hM+{$r1P4z6*^HeWteOOZ6qP4*&N~)Vw4^ll$^)%J9RL@hrsI~il{Chu4K8MMNf2@n-bC7)WkN!Cr*2j~B zX{u+bo~L?IYyG2t^iT9alh(x`)x%UzQ$0)dJk^U@JAXNU9S&rFg?!Z z$nP9Gr1^~bx%s*Ix%s*LTFm*UX?8tJ^*q%cKab8Mhsj6(=%2%;u93q*s)wnbrh1m@ zd8!w+z9adOHtX&FM?aVJyJ%89NcAw)(^St=Jx}$b)-U=0pGVv8yk@!m&R3tiBfIZ> z{tLJB)daWu*^b*MK!5-N0t5&UAV7cs0RjXF5FkK+009CgLcsPr2mhb(iSbq4j;Yj9 z(?OH!L8^zTo~C-1>UpXcwSLL*6@UKA?ZWiWaZF^G*2^^2$wz!dd_;WYxCd+>7i%Fs zKKTEi=#7)`_-Xqc=^+9H2oNAZfB*pk1PBlyK!5-N0t5&UAkYJW>G8HRw%<8+Sfm+v zNp+LzL8^zTo~C-1>UpXcwf6G)m?b73e|{=viTdYwJr2`)nWj4Vm_MJO`Lq0;{GI&W z^&U8Jf4<53z5ctqBme(q`<-V`#LFVwNdg225FkK+009C72oNAZfB*pk1PBlya4Q0~ z-!c9-{yS_}2LR&W_Ld^A|t5-UH%Cao%j|IpScD>S3y< zsh*{Jp6W%dU-JBG`<))Vq~cbd&$%uE0t5&UAV7cs0RjXF5FkK+009C72y{ch_B--B z@;mZ7@;irVzIvGEtN#2{^hWaW=gZj~rMn;1XqAN^zdn(IAa`+OnA0fYUJ-eZQ z^pE^b^gvS|V{hx9pE~J}KepfL{!1#lv9GJ6AV7cs0RjXF5FkK+009C72oNAZfB=D8 z5wQJ^_)f$Wrgem7`<)mo4y%d3{t|sMs3ykG#?KxvjcS;}$? zrsSi4zQ_Kt{Z0>FQgJKK=UkTn0RjXF5FkK+009C72oNAZfB*pk1iB$$`yKOV^JnwJ z?{R*3(zh>g|84)W%nrVEY~U9r+#k9r+#k zo#=ry-#!@DS+_qQ7mt(4=futb<-e|HfBgBTSnu<+#uv3dEU9i%JxKL1)zehZQaw-g zqSpLx!r%5g-G50%H}-XP6a)wmAV7cs0RjXF5FkK+009C72oNA}D+0FP5#NciwDDUh z%dIU2X~Z~8b@K7occN#KkN8WBWA)F`Ix_x`H8QV{A%}}vi!a1_Nj?Y3NB`&_@xkbU zCLT9zzte-4RNTt*IoBmXfB*pk1PBlyK!5-N0t5&UAV7csfo=%cen);MW<=9ETRxnn zdYY@wk+HuY!RudmsD$M!n} z2oNAZfB*pk1PBlyK!5-N0t5&UAV8o80=D0g-;v)rXwqDLkm_Nor>UN$dYs_j zU*uopUu^#pIUSFm$6E#s(m7(7>f~elyXcwZbC7&``{yfdztjDfRCHrsS4TmB009C7 z2oNAZfB*pk1PBlyK!5-N0=FVy`yKI}7_q1EnDL?Uq4-g(i+LKcFKTW3lvpFl=OFp$ zAN?b~7Cq3^#|_&z#p6b)CgQVU^4%UkZNJllmsH%!^EuZgK!5-N0t5&UAV7cs0RjXF z5FkK+0D*1@*nUTTCuWS(I!}||iFw+vnt1tt^vR%_`13E(CrvduNIvol(KE?Mej$3M zq@GDW`X_oO`5Y!6{iAY+~@DeMkFA%?oJ}IeBl8^XB^i1-x{YaQB zQqLqG@ts&x$;b9z(KGYfGo9c5%l13B`T0;cUXe!}--+4au$sv4M4t?*iTqCVNmEVaccM>9HIcsvlSMU=zlgOn zPbSI7pI?ifNj`_kNB{J&e@@*ue&#c;`S!)B?RTC%=}*1fe*y#u5FkK+009C72oNAZ zfB*pk1PBlya3cZR?}+cjm~dLhsNy>@8XQ&=@tx?CK{XNIi9TtniTF`H1gC&mfa>Y3zY`<(}F`{hQ@Bz&F#0RjXF5FkK+009C72oNAZ zfB*pk1PBoLehJuqM}8;fgwr|`li!Is;jo&>??j&rs)_tg^hr}q826GfyVTM}8-ICi%$kM9<7p&m^Dj=XY$s^Zk0}<8}xTAV7cs0RjXF5FkK+009C7 z2oNAZfWQx|fbDn0cVbL9ts^nFqwPRt3XbtWdi6LZ30HId(mJ{eRK`JL#K zrkcp_M4yyuBEJ(Ri)tdj6KiLlOp=fMPV`Ljk>81)nWdgdKHbmn*nX!oPrV!n0RjXF z5FkK+009C72oNAZfB*pk1PBngk$~-Y#CKv$IISZw@tqhG4y%dyPV~v3nuza2pET7( zd?)&(R1@)?Fj-U+@ts&Z^JJ2I#CM`+l8^XK^vo>vO!Db|e8=`X&)(>%jL#DwK!5-N z0t5&UAV7cs0RjXF5FkK+0D;a3*nUTTC+39HIun!Mi8qQ%&S| zqEAXSk>3fEMKzJ%iM2CNCdo&BCweCN$nQkY%u>%JpYG>(Y`@c)r(O<(009C72oNAZ zfB*pk1PBlyK!5-N0t5)$NWk_x;yW=WoYs+;_)d%oht))UC;DViO~iMiPnv2Xz7u^? zs)_hcm@KM^_)e^yc``{p;yckZ$wz!AdS;e-Ci!$fzGM3x+wTw{K!5-N0t5&UAV7cs z0RjXF5FkK+0DqQ%&S|qEAXSk>3fE zMKzJ%iM2CNCdo&BCweCN$nQkY%u>%JpYG>(Y`=4(S0ea40RjXF5FkK+009C72oNAZ zfB*pk1PBo5jDYQT#CKv$IISZw@tqhG4y%dyPV~v3nuza2pET7(d?)&(R1@)?Fj-U+ z@ts&Z^JJ2I#CM`+l8^XK^vo>vO!Db|e8=`X-G50%XP#U-5CQ}U5FkK+009C72oNAZ zfB*pk1PBngX#v~s$nV6Qa9U?#@;fmn999$go#>N6HId(mK543n{7&>qsV4F}VX~+u z@;kA1=E)@a$nQkYBp>;m=$TpSndH;`{EqE+Zu+T?D-s|;fB*pk1PBlyK!5-N0t5&U zAV7csfs-O&`yKI}7!ywGNKAYu#)QLaBEA!SGN>lvJJBaiH4)#5J}K2id?!p6)kJ(J z*3LYcBp>me=$Ygrz7st&OFff(x*y-M{f_N-2oNAZfB*pk1PBlyK!5-N0t5&UAV7dX z_XTXfBfk@K!fBm}$?wFRa9B;`ccM=Q)kJ%JpYF$ZY`@d}msH&JlQ~x;K!5-N0t5&UAV7cs0RjXF5FkK+0D;a3 z*nUTTC+39HIun!Mi8qQ%&S|qEAXSk>3fEMKzJ%iM2CNCdo&B zCweCN$nQkY%u>%JpYG>(Y`@c)r(O<(009C72oNAZfB*pk1PBlyK!5-N0t5)$NWk_x z;yW=WoYs+;_)d%oht))UC;DViO~iMiPnv2Xz7u^?s)_hcm@KM^_)e^yc``{p;yckZ z$wz!AdS;e-Ci!$fzGM3x+wTw{K!5-N0t5&UAV7cs0RjXF5FkK+0DqQ%&S|qEAXSk>3fEMKzJ%iM2CNCdo&BCweCN$nQkY z%u>%JpYG>(Y`=4(S0ea40RjXF5FkK+009C72oNAZfB*pk1PBo5jDYQT#CKv$IISZw z@tqhG4y%dyPV~v3nuza2pET7(d?)&(R1@)?Fj-U+@ts&Z^JJ2I#CM`+l8^XK^vo>v zO!Db|e8=`X-G50%XP#U-5CQ}U5FkK+009C72oNAZfB*pk1PBngX#v~s$nV6Qa9U?# z@;fmn999$go#>N6HId(mK543n{7&>qsV4F}VX~+u@;kA1=E)@a$nQkYBp>;m=$TpS zndH;`{EqE+Zu+T?D-s|;fB*pk1PBlyK!5-N0t5&UAV7csfs-O&`yKI}7!ywGNKAYu z#)QLaBEA!SGN>lvJJBaiH4)#5J}K2id?!p6)kJ(J*3LYcBp>me=$Ygrz7st&OFff( zx*y-M{f_N-2oNAZfB*pk1PBlyK!5-N0t5&UAV7dX_XTXfBfk@K!fBm}$?wFRa9B;` zccM=Q)kJ%JpYF$ZY`@d} zmsH&JlQ~x;K!5-N0t5&UAV7cs0RjXF5FkK+0D;a3*nUTTC+39HIun!Mi8qQ%&S|qEAXSk>3fEMKzJ%iM2CNCdo&BCweCN$nQkY%u>%JpYG>(Y`@c) zr(O<(009C72oNAZfB*pk1PBlyK!5-N0t5)$NWk_x;yW=WoYs+;_)d%oht))UC;DVi zO~iMiPnv2Xz7u^?s)_hcm@KM^_)e^yc``{p;yckZ$wz!AdS;e-Ci!$fzGM3x+wTw{ zK!5-N0t5&UAV7cs0RjXF5FkK+0Dq zQ%$;+-}#yO-Ot4_P9Hd~zx>zRJ3CiPDO=lDFK@K()8+1eE%$u(zH(Pt{#P&m>x1^w zFZy^;zw@Q4mzDCR?@abakGEg^)yd;WKlASNjmNjP{JAf_yy)k=)X#mzw3L@We00Tk zjyERzZS~gi*8U`J>aL^XS$=SITVLAU-q;+qA2r(>FFy7Ul=AU=%g6ux=$a2+-q_kZ zs@i|9Zf=ZUsF&|Q(SE%1=;+GE)_D0nUwX^-=9NorasSorqZ`e4`<)xdUwiAu-gx;v z%ggfXuYPiGt2wT}_;>r0i(75|#r?_t3%`)%_Z*I{uCKQ5mzR8-N1u4bv_0%c+j9A~ zyxjGMkG9``e&zJJ)zhmhXPd3fi`UMcf9Uv&NB^FB{`vO(YWrOuy6flX?|%Lt%bOl+ z-+!ZAd)Wj3{FzejE-zpH^B*4={$Yz<$GhI+ zj?ejA4|tsGLk~Eg>+?SJdEbxG^KySbL+ehectzDbUwGv`@RPH zaQx2a)_EWLyzj^8d~TiheGT;C_?^$K^FH)>-;dGx+&b_38tB9EJD*$UedzPPAEWcR zb>8PQ=zMOS_k9iY;rN};t@A$gdEbxG`P@41`x@xO@jIVe=Y8n&z8|CWxpm(6 zHPDCScRshy`_SioKSt+s>%8x4pby9Id~Tihq0jq%jLzrQdEeJSACBMo+&b?=pZEP3 zozJcFzOR8k9KZ9qb^d?){QmvP`cCVzyxi5kt+a0!pWK*?+V{V*JwAHhmgVi2H@5bc z#qzSfvAH+e-Q7C===k3UH@EgDZTYT0oa`)b@NFBTqbo0Ow0v}Q_0sv3)2ruCudb~8 E54 decoderInfos = ImmutableList.of(); checkNotNull(format.sampleMimeType); @@ -265,10 +271,21 @@ public final class DefaultDecoderFactory implements Codec.DecoderFactory { Log.e(TAG, "Error querying decoders", e); throw createExportException(format, /* reason= */ "Querying codecs failed"); } - if (decoderInfos.isEmpty()) { throw createExportException(format, /* reason= */ "No decoders for format"); } + if (devicePrefersSoftwareDecoder) { + List softwareDecoderInfos = new ArrayList<>(); + for (int i = 0; i < decoderInfos.size(); ++i) { + MediaCodecInfo mediaCodecInfo = decoderInfos.get(i); + if (!mediaCodecInfo.hardwareAccelerated) { + softwareDecoderInfos.add(mediaCodecInfo); + } + } + if (!softwareDecoderInfos.isEmpty()) { + decoderInfos = softwareDecoderInfos; + } + } List codecInitExceptions = new ArrayList<>(); DefaultCodec codec = @@ -354,6 +371,20 @@ public final class DefaultDecoderFactory implements Codec.DecoderFactory { return SDK_INT >= 29 && context.getApplicationInfo().targetSdkVersion >= 29; } + private static boolean devicePrefersSoftwareDecoder(Format format) { + // TODO: b/255953153 - Capture this corner case with refactored fallback API. + // Some devices fail to configure a 1080p hardware encoder when a 1080p hardware decoder + // was created. Fall back to using a software decoder (see b/283768701). + // During a 1080p -> 180p export, using the hardware decoder would be faster than software + // decoder (68 fps vs 45 fps). + // When transcoding 1080p to 1080p, software decoder + hardware encoder (33 fps) outperforms + // hardware decoder + software encoder (17 fps). + // Due to b/267740292 using hardware to software encoder fallback is risky. + return format.width * format.height >= 1920 * 1080 + && (Ascii.equalsIgnoreCase(Util.MODEL, "vivo 1906") + || Ascii.equalsIgnoreCase(Util.MODEL, "redmi 8")); + } + private static ExportException createExportException(Format format, String reason) { return ExportException.createForCodec( new IllegalArgumentException(reason),