On Jun 11, 2015, at 7:05 PM, Fabrice Florin
<fflorin(a)wikimedia.org> wrote:
Nicely done, Brion!
We’re very grateful for all the mobile multimedia work you’ve been doing on your ‘spare
time’ …
Much appreciated,
Fabrice
On Jun 11, 2015, at 7:34 AM, Brion Vibber
<bvibber(a)wikimedia.org <mailto:bvibber@wikimedia.org>> wrote:
I've been passing the last few days feverishly working on audio/video stuff, cause
it's been driving me nuts that it's not quite in working shape.
TL;DR: Major fixes in the works for Android, Safari (iOS and Mac), and IE/Edge (Windows).
Need testers and patch reviewers.
== ogv.js for Safari/IE/Edge ==
In recent versions of Safari, Internet Explorer, and Microsoft's upcoming Edge
browser, there's still no default Ogg or WebM support but JavaScript has gotten fast
enough to run an Ogg Theora/Vorbis decoder with CPU to spare for drawing and outputting
sound in real time.
The ogv.js decoder/player has been one of my fun projects for some time, and I think
I'm finally happy with my TimedMediaHandler/MwEmbedPlayer integration patch
<https://gerrit.wikimedia.org/r/#/c/165478/> for the desktop MediaWiki interface.
I'll want to update it to work with Video.js later, but I'd love to get this
version reviewed and deployed in the meantime.
Please head over to
https://ogvjs-testing.wmflabs.org/
<https://ogvjs-testing.wmflabs.org/> in Safari 6.1+ or IE 10+ (or 'Project
Spartan' on Windows 10 preview) and try it out! Particularly interested in cases where
it doesn't work or messes up.
== Non-JavaScript fallback for iOS ==
I've found that Safari on iOS supports QuickTime movies with Motion-JPEG video and
mu-law PCM audio <https://gerrit.wikimedia.org/r/#/c/217295/>. JPEG and PCM are, as
it happens, old and not so much patented. \o/
As such this should work as a fallback for basic audio and video on older iPhones and
iPads that can't run ogv.js well, or in web views in apps that use Apple's older
web embedding APIs where JavaScript is slow (for example, Chrome for iOS).
However these get really bad compression ratios, so to keep bandwidth down similar to the
360p Ogg and WebM versions I had to reduce quality and resolution significantly. Hold an
iPhone at arm's length and it's maybe ok, but zoom full-screen on your iPad and
you'll hate the giant blurry pixels!
This should also provide a working basic audio/video experience in our Wikipedia iOS app,
until such time as we integrate Ogg or WebM decoding natively into the app.
Note that it seems tricky to bulk-run new transcodes on old files with TimedMediaHandler.
I assume there's a convenient way to do it that I just haven't found in the
extension maint scripts...
== In progress: mobile video fixes ==
Audio has worked on Android for a while -- the .ogg files show up in native <audio>
elements and Just Work.
But video has been often broken, with TimedMediaHandler's "popup
transforms" reducing most video embeds into a thumbnail and a link to the original
file -- which might play if WebM (not if Ogg Theora) but it might also be a 1080p original
which you don't want to pull down on 3G! And neither audio nor video has worked on
iOS.
This patch <https://gerrit.wikimedia.org/r/#/c/217485/> adds a simple mobile target
for TMH, which fixes the popup transforms to look better and actually work by loading up
an embedded-size player with the appropriately playable transcodes (WebM, Ogg, and the
MJPEG last-ditch fallback).
ogv.js is used if available and necessary, for instance in iOS Safari when the CPU is
fast enough. (Known to work only on 64-bit models.)
== Future: codec.js and WebM and OGVKit ==
For the future, I'm also working on extending ogv.js to support WebM
<https://brionv.com/log/2015/06/07/im-in-ur-javascript-decoding-ur-webm/> for better
quality (especially in high-motion scenes) -- once that stabilizes I'll rename the
combined package codec.js. Performance of WebM is not yet good enough to deploy, and some
features like seeking are still missing, but breaking out the codec modules means I can
develop the codecs in parallel and keep the high-level player logic in common.
Browser infrastructure improvements like SIMD, threading, and more GPU access should
continue to make WebM decoding faster in the future as well.
I'd also like to finish up my OGVKit package <https://github.com/brion/OGVKit>
for iOS, so we can embed a basic audio/video player at full quality into the Wikipedia iOS
app. This needs some more cleanup work still.
Phew! Ok that's about it.
-- brion
_______________________________________________
Multimedia mailing list
Multimedia(a)lists.wikimedia.org <mailto:Multimedia@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/multimedia
_______________________________________________
Multimedia mailing list
Multimedia(a)lists.wikimedia.org