I'm landing a big update to the ogv.js Ogg player shim for Safari/IE/Edge;
ogv.js 1.1.3 fixes some seek bugs with the future video.js frontend code,
and has a bunch of performance & correctness updates.
On slower machines (like older iPhones, or old Windows machines running IE)
ogv.js now makes better use of dual-core CPUs to overlap decoding one frame
with drawing another, etc. It's also much more aggressive about keeping
audio and video synchronized, making short delays instead of letting it get
wildly out of sync followed by a weird catchup period.
With the seek & sync bugs mostly cleaned up, I'm also turning some of my
A/V attention to iOS, making updates to my OGVKit Objective-C library which
uses native builds of the Ogg and VPX libraries, able to handle higher
resolutions on slower phones than ogv.js can. I'll track down the current
iOS mobile app folks and see what the best way to work with them on this
is. :)
Back to TimedMediaHandler -- Derk-Jan has been doing some more updates on
the Video.JS-based new frontend code, which I'm reviewing through this week
and hope to have merged and cleaned up for further testing. In addition to
looking more modern and being more actively maintained, video.js is more
mobile-friendly -- deploying the new frontend when it's ready will get us
more consistent media playback on Android, and ogv.js-based playback on
iPhone and iPads in the mobile view.
I've also got a number of back-end fixes to finish up such as a new
maintenance script for re-running video transcodes. It currently needs just
one more option, which is a 'throttle' so we can background-run it
unattended without worrying it'll fill up the job queues.
In the closer future, 720p and 1080p Ogg transcodes should start rolling
out in the next few weeks for manual switching, and we'll see how it all
goes. :)
Farther out in the future -- I did testing last month on transcoding, and
some preliminary planning for how to do adaptive streaming (seamless
resolution switching as necessary). I've got a pretty good handle on how it
will work for native WebM playback, and a good theoretical model for how it
should be possible with ogv.js and Ogg playback by creating an analogue of
the native Media Source Extensions APIs, and encoding / dividing up the
packetized files _just so_.
Note that creating adaptive-switchable Ogg transcodes will require
customizing ffmpeg2theora and/or creating a separate similar tool specific
to our needs. Shouldn't require any new daemons/services though.
-- brion
I've gone ahead and self-merged a couple small tweaks to
TimedMediaHandler's .ogv handling, which can go out with the next release
if there's no rush to deploy them earlier:
*Enabled 720p and 1080p .ogv output*, using existing variable bitrate
configuration:
https://gerrit.wikimedia.org/r/291921
This will enable new generation of high-definition .ogv transcodes for
files whose sources are high-definition. The files will be smaller than
they would be at constant bitrate, running somewhere from 'about the size
of existing webm' to 'up to 40% larger than webm' depending on which
configuration the webms were generated against. :)
*Disabled forcing of audio output to max 44.1 kHz* for .ogv output due to
bugs in the resampling leading to a/v sync mismatch on long videos:
https://gerrit.wikimedia.org/r/289900
Once existing .ogv transcodes are redone, this will fix a/v sync drift on
the .ogv versions of longer videos with 48kHz audio, such as
https://commons.wikimedia.org/wiki/File:Knowledge_for_Everyone_(no_subtitle…
In the next week I'm going to try to finish up a couple more things:
*Update ogv.js to 1.1.2 release* (just released alpha.7, probalby final):
I've made major improvements to ogv.js on slower machines, with better
audio/video sync maintenance and more effective use of multiple CPU cores.
This should improve performance on old 32-bit iOS devices as well as older
Windows machines running IE.
*Maintenance script to re-run video transcodes in bulk:*
I'm replacing several half-working maintenance scripts in TimedMediaHandler
with a new one that actually works. Once I've added a 'throttle' mode I'll
commit it, and we can bulk re-run transcodes in production to update things
to current configurations.
*Re-tuning .webm and low-res .ogv transcodes for VBR:*
Transcode settings other than the HD .ogvs are currently specifying
constant bitrates, which are higher than they need to be for a lot of
material in order to provide enough headroom for high-complexity scenes. A
cleaner variable bitrate configuration will save space on most material,
while using more bits only when needed. (That will free up more space for
the HD .ogv transcodes.)
...And then try to finish up more of the video.js front-end with TheDJ and
anybody else who's interested. We're so close! And that'll get us video on
mobile as well... :D
-- brion
I spent some time between projects today exploring the idea of progressive
image decoding using the VP9 video codec -- sort of a mashup of progressive
JPEG and WebP.
Like a progressive JPEG, each resolution step (a separate frame of the
"video") encodes only the differences from the previous resolution step.
Like WebP, it's more space-efficient than the ancient JPEG codec.
This sort of technique might be useful for lazy-loading images in our
modern internet, where screen densities keep going up and network speeds
can vary by a factor of thousands. On a slow network the user sees
immediate feedback during load, and on a fast network they can reach full
resolution quickly, still in less bandwidth than a JPEG. And since JS would
have control of loading, we can halt cleanly if the image scrolls
offscreen, or pick a maximum resolution based on actual network speed.
Detail notes on my blog:
https://brionv.com/log/2016/06/14/exploring-vp9-as-a-progressive-still-imag…
Sample page if you just want to look at some decoded images at various
resolutions (loading not optimized for slow networks yet!):
https://media-streaming.wmflabs.org/pic9/
It looks plausible, and should be able to use native VP9 decoding in
Firefox, Chrome, and eventually MS Edge in some configurations with a
JavaScript fallback for Safari/etc. Currently my demo just plops all the
frames into a single .webm, but to avoid loading unneeded high-resolution
frames they should eventually be in separate files.
-- brion