Just an update on this weekend project, see the current demo in your
browser or watch a video of Theora video playing on an iPhone 5s!
* Got some fixes and testing from one of the old Cortado maintainers --
* Audio/video sync is still flaky, but everything pretty much decodes and
plays properly now.
* IE 10/11 work, using a Flash shim for audio.
* OS X Safari 6.1+ works, including native audio.
* iOS 7 Safari works, including native audio.
Audio-only files run great on iOS 7 devices. The 160p video transcodes we
experimentally enabled recently run *great* on a shiny 64-bit iPhone 5s,
but are still slightly too slow on older models.
The Flash audio shim for IE is a very simple ActionScript3 program which
accepts audio samples from the host page and outputs them -- no proprietary
or patented codecs are in use. It builds to a .swf with the open-source
Apache Flex SDK, so no proprietary software is needed to create or update
I'm also doing some preliminary research on a fully Flash version, using
the Crossbridge compiler for the C codec libraries. Assuming it performs
about as well as the JS does on modern browsers, this should give us a
fallback for old versions of IE to supplement or replace the Cortado Java
player... Before I go too far down that rabbit hole though I'd like to get
peoples' opinions on using Flash fallbacks to serve browsers with open
As long as the scripts are open source and we're building them with an open
source toolchain, and the entire purpose is to be a shim for missing
browser feature support, does anyone have an objection?
On Mon, Oct 7, 2013 at 9:01 AM, Brion Vibber <bvibber(a)wikimedia.org> wrote:
TL;DR SUMMARY: check out this short, silent, black
& white video:
-- anybody interested in a side
project on in-browser audio/video decoding fallback?
One of my pet peeves is that we don't have audio/video playback on many
systems, including default Windows and Mac desktops and non-Android mobile
devices, which don't ship with Theora or WebM video decoding.
The technically simplest way to handle this is to transcode videos into
H.264 (.mp4 files) which is well supported by the troublesome browsers.
Unfortunately there are concerns about the patent licensing, which has held
us up from deploying any H.264 output options though all the software is
ready to go...
While I still hope we'll get that resolved eventually, there is an
alternative -- client-side software decoding.
We have used the 'Cortado <http://www.theora.org/cortado/>' Java applet
to do fallback software decoding in the browser for a few years, but Java
applets are aggressively being deprecated on today's web:
* no Java applets at all on major mobile browsers
* Java usually requires a manual install on desktop
* Java applets disabled by default for security on major desktop browsers
years, and performance is getting well in line with what Java applets can
As an experiment, I've built Xiph's ogg, vorbis, and theora C libraries
written a wrapper that decodes
Theora video from an .ogv stream and
draws the frames into a <canvas> element:
* demo: https://brionv.com/misc/ogv.js/demo/
* code: https://github.com/brion/ogv.js
* blog & some details:
It's just a proof of concept -- the colorspace conversion is incomplete so
it's grayscale, there's no audio or proper framerate sync, and it doesn't
really stream data properly. But I'm pleased it works so far! (Currently it
breaks in IE, but I think I can fix that at least for 10/11, possibly for
9. Probably not for 6/7/8.)
Performance on iOS devices isn't great, but is better with lower
resolution files :) On desktop it's screaming fast for moderate
resolutions, and could probably supplement or replace Cortado with further
Is anyone interested in helping out or picking up the project to move it
towards proper playback? If not, it'll be one of my weekend "fun" projects
I occasionally tinker with off the clock. :)