[Foundation-l] Freedom, standards, and file formats

[Gregory Maxwell]

On Sat, Sep 27, 2008 at 4:59 PM, Michael Snow <wikipedia at verizon.net> wrote:
> I mentioned earlier that I wanted to discuss open standards and file
> formats in advance of the next board meeting. I'd especially like to
[snip]
> near-universal. The company has also been moving to make it more open
> for people watching, distributing, and working on content in this
> environment.
[snip]

There are so many distinct classes of problem with flash that it can
be hard to have a productive discussion about it. For example, you can
assault flash for being ruinous for accessibility to the disabled,
you can criticize the problems of security created if you allow
uploading flash software,  flash lacking freely licensed authoring
tools, for it being browser crashing, being mostly used for toys and
trash, so widely blocked in many business environments, etc.

For the purpose of this discussion I'm going to talk about
flash-as-a-video-format. For that purpose the important problems are
almost entirely caused by the codecs used inside flash.  Adobe does
not own the patents those formats and couldn't freely license if they
wanted to.   Not only has this situation not gotten better recently,
but it has gotten worse as Adobe has incorporated a newer highly
non-free video format.

>It's close to free, but I understand there are still some

I nominate this for the most misguided statement of the year award.  :)

Flash video requires the some of the most encumbered and expensive
video formats available.  The rights holders of H.264 have created a
"your first hit is free" short time offer where web use is possible at
no-cost until 2010. While I do not know what it will cost post-2010
H.264, today the licensing for non-web use of H.264 involves
per-use/per-device licensing fees which are the most expensive of any
widely used codec that I am
aware of.

Microsoft's OOXML office format is closer to a free format than the
video used in flash. :)

It's important to get details like this correct in a public message
since mistaken statements like this harm the hard work done by many
people working to improve the situation.

>It would make it much easier to achieve wide distribution of
> free content, while still making sure that it's also available

This is a discussion that no one can rationally have without an
objective look of what the benefits would actually be.  It needs to be
backed up with measurements.

But no where in your message do I see you raise a serious question of
what advantage would be gained in return.  It seems to be just
accepted that there would be some great benefit ("It would make it
much easier to achieve wide distribution"), with only an airy an
unquantifiable "evangelism" lost  as though there were no pragmatic
benefit.

The existing system correctly plays back for an overwhelming majority
of users (I posted some stats on this two years ago or so), though my
ability to measure the success rate was lost when it was made a part
of the site-proper.  The 'much easier' assertion is often made, but
not backed up with facts.  While I think we can agree that there would
be some benefit to using Flash Video, the evidence I've seen so far
would indicate that the benefit would not be that great.

The primary usability problems on our site have been pointed out to be
things mostly or totally unrelated to format choice.   For example, we
have no software facility for converting arbitrary uploads from users
to whatever we'd like to distribute.  This is a format-agnostic
problem and it results directly from Wikimedia's historic
non-investment in this area of functionality.    The same inability to
transcode on the fly means that if a user uploads a 10mbit/sec
video,your DSL connected PC will sputter as we attempted to cram
10mbit/s into a 1mbit/sec pipe.

The typical bitrate used by youtube has historically been around
250kbit/sec for audio + video, which is why the quality is poor, but
which is also a major reason why it works fairly well.  When last I
checked the average bitrate of videos on commons was about 4mbit/sec.

So you're making a significant error if you compare the results
obtained by a company whos bread and butter is working video with the
results we've obtained while targeting a very different use case, and
with video largely as an after-thought.

So I think the benefits question is still far from closed.

>  Are
> there situations in which this compromise doesn't work out for some
> reason? Why? (And none of this has to be limited to the Flash video
> example, discussion of other formats and standards is welcome.)
>
> In dealing with the limited adoption of certain free formats, some
> people have advocated a more evangelistic approach, if you will.

There is a real practical loss associated with offering formats that
require payment for their use even in parallel to free formats, one
which extends beyond the licensing costs that Wikimedia itself will
likely have to pay.

Consider: the only reason we (or almost anyone else) would ever
consider using a slightly non-free format is because there is some
'costly' shortcoming in the free-format.   For free audio and video
formats the shortcoming is that they are not widely adopted.   For
media types where a free format is widely adopted: hypertext (HTML),
uncompressed audio (wav), still photographs (jpg), absolutely no one
suggests that we even consider using non-free formats, although
proprietary formats do exist in these areas.

Right now there does not really exist a free format for low bandwidth
video:   There is Flash/H.264 which requires payment to the rights
holders.  Alternatively there is Ogg/Theora which you pay for in
reduced compatibility.

The difference is that if we offer only the latter we will cause (and
demonstrably have caused) people to install the latter and encourage
vendors to support the latter which decreases that cost for everyone,
eventually contributing to the format becoming truely without cost.

I don't see value in evangelism for formats, and I don't expect you
to.  I see value in creating a world where success in our mission,
which ought to include people being able to exchange the knowledge we
have  recorded without paying format taxes, is possible without
compromises. Getting there requires something that looks a little like
evangelism, but the motivation is entirely different.

Offering "Widely-adopted-non-free-format *and*
not-widely-adopted-free-format" is a minor compromise for someone
who's motivation is promoting "not-widely-adopted free-format".  But
if the motivation is "make the free format widely adopted, so no one
is forced by the market to use the non-free format" then it doesn't
make much sense as a compromise because it completely loses the
ability to use Wikimedia as an argument for including free formats in
browsers, and completely loses the encouragement for regular users to
click-install and get support themselves.

But until the free formats have enough adoption that people feel no
reason to consider this using non-free for reasons of adoption, there
really is no such thing as a free format, only formats with different
kinds of costs, and Wikimedia needs to help change that if it wants
it's work to live up to the licenses the work is distributed under.

Once adoption is no longer a factor in the use of non-free formats,
I'll gladly write the flash-video support for Wikimedia myself if
Wikimedia still wants it. ... but it won't want it, because it will no
longer be a consideration just like non-free text formats are not a
consideration.  The fact that it is a consideration is all the
evidence you should need to see that format-freeness is still a
problem which is obstructing Wikimedia's mission.

If you want to fix this problem you could start by investing in the
solution: There are organizations like Xiph and events like FOMS
(http://www.foms-workshop.org/foms2009/) which would benefit greatly
from Wikimedia's support.   Every Wikimania seems to have attendance
by people in the proprietary video world, but folks working on free
tools seem to get little audience because they can't afford the
travel, and can't sponsor PR blitzes.