[Foundation-l] Format Conversion

[Gregory Maxwell]

On Jan 21, 2008 1:48 PM, teun spaans <teun.spaans at gmail.com> wrote:
[snip]
> A pragmatic approach, where we convert stuff on upload into a
> free format is something i asked for in vain at commons. I am very glad to
> see resurface that idea here.

Uploading to our projects sucks. It's hard all around.

I'm working on an upload tool for video that has an improved ajaxy
upload form to better gather data, and can transcode uploads.   So far
I only support pulling video from things like  Youtube, Google Video,
Dailymotion, Guba, Stage6 and Metacafe, but I will eventually support
a couple of other upload mechanisms (http upload, etc). It will allow
anonymous uploading, with a submission queue so that named users can
approve uploads before they hit the site. I think thats better than
not allowing anonymous uploading.

Like the WikiMediaPlayer I expect that this will eventually be
re-invented as a mediawiki extension once the uses and needs are
clear.

> I also agree that java is widely installed, and offers a potential base, but
> had the disadvantage of being slow. Is there a solution possible in
> cooperation with the theora / ogg people? Some solution where FF users
> automatically a non javascript plugin fast and easy, while IE users fall
> backon slow java?

We already have that...

We support Ogg Theora and Vorbis playback though several ways:

1) If the browser supports the HTML5 <video/> tag.
2) If the browser has the VLC plugin
3) If the browser has the QT plugin and the XiphQT codecs installed
4) If the browser has Java
5) If the browser otherwise claims to natively support Application/Ogg
(mostly totem plugin on Linux boxes)

We will automatically detect these options and choose the 'best',
perhaps the software's idea of best differs from yours so you can
click the "more..." link and pick your preferred method.

Right now HTML5 <video/> tag support is too new to have adoption, but
there is support in Firefox's code repository and the support will be
included in an upcoming version of firefox.

Not that it doesn't have some quirks and bugs, but it's a lot better
than what many sites do.

Erik Moeller wrote:
> It's worth noting that video on the web was viable for millions of users long before YouTube
> -- the YouTube success story is not one of bandwidth, but of usability.

But where YouTube's usability was most different had little to do with
playback and a lot to do with accepting user contributions and,
especially initially, having little copyright enforcement and hiding
behind the shield of the DMCA.  Bully for them, but we can't take
their success to mean a whole lot.

 ... and, of course, being able to afford accepting random videos from
people and serve them out over the web, rather than running a P2P
service, is also story about *bandwidth*, but on the service side.

> before Flash players became widespread,

Video in flash predates YouTube by a good four years or so.

It was not widely used for that, because it wasn't especially
attractive... Yet another proprietary video plugin, requiring
expensive support software.  The other solutions QT, Real, MS WMV,
etc.. all were more widely adopted.

Flash has been widely installed, at least in entertainment (vs office)
environments for a long time.

I really think you have the cause and effect reversed here:  Broadband
made consumer internet video possible, but P2P was too difficult and
seedy for mass adoption, cheap hosting capacity made commercial video
hosting sites possible, and their adoption of Flash has made flash a
very popular format for video on the net.

This doesn't mean that flash wasn't the best option for YouTube at the
time, ... I'm sure they considered it carefully.  But had they gone
with another one of the primary competing solutions, it too would have
enjoyed a considerable increase in adoption for video use.

> playing video on
> the web was a constant hassle: one would struggle with Real Player,
> Quicktime, Windows Media Player, etc., and an additional number of
> specialized plugins, all of course proprietary.

While it's true that the high profile commercial video sites are flash
based ... the Internet is still filled with sites using a multitude of
video solutions:  WMV, Real, QT, MPEG, etc.  The notion that video is
all sorted out on the Internet is not true.

> Whether one believe's Adobe's numbers of 98%+ adoption of Flash in
> "mature markets" (as opposed to 84.6% for Java) [1],

Both of those numbers are probably a little inflated, but the relative
penetration sounds about right.  Any degree of flash penetration in
business environments is a very new thing, and that does have a lot of
influence on the traffic we see.

> My fear is that by locking ourselves into Ogg Theora only, we are
> replicating the pre-YouTube experience of video that may or may not
> work, may or may not require installation of additional plugins, etc.

Even flash "may or may not work, may or may not require installation
of additional plugins". So you're not solved there, it's just a
numbers game. True, the numbers are better for flash, but until flash
is built into W3C standard built into web browsers it will remain a
numbers game.

With the inclusion of Theora/Vorbis into Firefox we will have a
zero-additional-install solution in a mainstream browser.  Thats a
position that even flash does not enjoy.

To the extent that your concern has merit you are playing exactly into
my primary argument against parallel distribution:  If there is
something wrong with using only the free format, then the free format
needs our help getting more adoption before it really is a usefully
free format.

> One should not confuse
> Wikipedia's power as a text-based medium with a universal lever that
> we can use to get anything that we want.
> If Ogg Theora is the future,

But thats the whole point of the anti-parallel distribution argument:
It will not be the future if people have no need to install it over
the more popular options.

It isn't yet inevitable, but we are certainly in a position to
influence the outcome.  It's inevitable that once it does achieve
adoption that free formats will stick.

Here it seems that you're arguing that a top ten website doesn't
influence over web standards, and web clients... yet you think we
should be in the business of getting software startups to freely
license their software.

We do have influence on this subject, and I can support that with
factual evidence if we must debate it.  That doesn't mean we have a
magical lever, ... if we did this argument would be done and over with
by now.

> Without easy & immediate playback ability on the vast majority of
> systems, it seems unlikely to me that we'll ever grow into a

What qualifies as a vast majority? Your own numbers quote Java at
84.6%, and we already play on more than Java.  We should expect an
even further bump as the video tag support shows up in browsers like
Opera and Firefox.

If you don't think someplace in the high 80 percent range plus a
reasonable expectation of an upward trend isn't a vast majority,
you've possibly been listening to enwp decision making process too
much! :)

If we want to talk about access to multimedia... Even in some
developed nations, like the US, access to *broadband* is still a
bigger gating factor that our non-use of flash. Not to mention even
having a computer and internet access and being able to read.

> My worst case scenario is that in the belief of doing something good
> for the world -- trying to lead towards greater freedom in
> distribution and authoring of content -- we'll actually achieve the
> opposite: lead people to repositories and archives that are much less
> principled and whose operators would never even conceive having a
> debate like this one.

For reading can you suggest a serious alternative contender?   When I
punch "pendulum" into Google, YouTube isn't showing as the first
couple hits ... and what does show up from YouTube doesn't look like
something we'd host.

Which is a big point.  By our very mission we don't accept a lot of
things, ... We don't generally accept non-educational works, our users
don't look the other way from overt copyright violations, we require
authors to release under a free license.  Because of this we'll always
be smaller than more general commercial webhosts, regardless of the
formats.   There isn't any harm in not being #1 at *everything*.

When it comes to the reading side we're not losing our position to
anyone because of video support. True, if people are looking for
videos specifically they won't come to us first.

On the content submission side, the player side isn't the issue.
People creating and uploading videos are simply not going to have huge
overlap with the "I can't install any plugins, I don't use Firefox, I
don't have Java" audience.

The issue on the submission side is a pure ease of use issue. Our
upload processes have usability problems, especially for multimedia.
Virtually no one uploads videos to YouTube in FLV: Users upload in
some other format and YouTube transcodes, we could (and, eventually
will) do that too.

> That doesn't mean that I believe the case for parallel distribution is
> unassailable. I do believe, however, that mandating a Foundation rule
> against it would be premature.

We've long had an firmly practiced rule of not distributing
proprietary formats.  It's something that would have been a written
foundation rule, if we had written foundation rules way back when.
But we didn't write foundation rules for this sort of thing back then.

As Wikimedia grows its staff and becomes active and involved in more
things it's natural that more "understood rules" will need to become
"written rules".


    "It is important to undertsand that the issue of convenience is at
the heart of our fight for freedom. People continue to use proprietary
formats and proprietary software because they perceive it as being
more convenient. One of the most important things that we can do is to
illustrate that proprietary formats are actually inconvenient, so long
as some people choose to use formats that are free. And if we at
Wikipedia do our small part to get people to download and install a
proper free decoder (this will generally just be a codec in a player)
so that we make the free Ogg Vorbis format as convenient as the
proprietary format, we will have achieved something important. Jimbo
Wales 06:59, 17 Jul 2004 (UTC)"

Since the time Jimmy said that Ogg Vorbis has gone from something that
practically no one but free software geeks could play, to something
that a majority can use on our site without any install.   We are on
the final leg of the challenge, and the costs are lower than they ever
were before.   If we abandon that vision we will risk losing what we
have gained.

Success takes work,  I've demonstrated that I'm willing to work to
help fix the usability of multimedia on our projects.  Are you?