[Commons-l] Alternative approach for better video support

[Daniel Arnold]

Hi,

At first let me point out that I appreciate the hard work of people that want 
to improve our projects. Please let us acknowledge the work of Erik, even if 
you think like me that his proposal is _definitely_ not the way to go. It's 
all about AGF. ;-) So please let us brainstorm how to turn this into a good 
opportunity for all of us without sacrificing parts of our goals and 
independence.

Two different things were not made clear in Eriks proposal and consequently 
were mixed in follow up mails:
a) Allowed media formats for _uploading_ files.
b) Provided media formats for _downloading_ files.

Allowing Flash Video for _upload_ is not acceptable in our projects for the 
named reasons of patent problems and (at least partly) proprietary software.

Providing Flash Video additionally for _download_ does not conflict with the 
GFDL or whatever (as you provide along side this copy as well a free format 
copy of the same content). This is as well the reason why e.g. providing 
TomeRaider _downloads_ of Wikipedia is in agreement with our understanding of 
digital freedom.

As pointed out by various people providing Flash Video for download means 
transcoding. As all relevant video formats are lossy you cannot transcode 
Ogg-Theora to let's say Mpeg (or any other particular codec used by Flash 
Video) without quality loss.

So Flash Video is a nice to have _second class_ alternative - a mere 
convenience for our users. But who wants to get the best quality will always 
use the original OGG-Theora files.

Philosophy aside here an analysis of current technical problems:

1. Wikimedia Commons badly needs a download service for the whole media file 
repository.
2. The media repository of Comons is so overwhelmingly large that it would 
need a lot of bandwith and download time even with broad band internet 
access.
3. Audio and Video streaming requires a lot of bandwith but it doesn't require 
sophisticated CPU intensive database queries and special designed software 
(aka MediaWiki).
4. Many organisations especially universities (I could name you at least one 
concrete example) want to support us with technical infrastructure but it is 
very complicated to integrate these kind donations into our technical 
infrastructure. Thus we are wasting a lot of donation potential beside money.

Proposed solution:

1. We provide an Rsync download of our media files for selected third party 
mirror servers. Rsync is specifically designed for such a purpose and is 
exeptionally bandwith saving on mirroring files where parts are constantly 
changing.
2. These selected third party mirrors are collected in a Round-Robin-Queue. 
Everytime a user directly wants to download the original file (and not a 
thumbnail, preview or the image page in the wiki) in Wikipedia (or another 
Wikimedia project) he is being redirected to this very file on a mirror out 
of this queue and downloads it there as usual via HTTP (just the download 
link changes, he even won't realise in most cases that he has been redirected 
for this download).
3. In case mirrors don't want to host the whole database we could provide them 
some filters for mirroring, e.g. Ogg files only. Ogg mirroring would make the 
most benefit for our infrastructure and audio/video streaming, as full images 
are downloaded relatively seldom.

This would make all these things possible:
* Download of Wikimedia Commons media database via mirrors.
* Saves us quite some bandwith especially at audio and video (you need to 
download the real file for streaming).
* Easy drop in and drop out of single third party supporters.
* Not dependent from one exclusive partner. We maintain our independence and 
increase technical failure resitance by reducing the single point of failure 
problem.

In case you fear the problem of license issues (providing the copyright infos 
alongside the files) this can be solved without troubles, too. There is 
static.wikipedia.org Just transfer the static html image wiki pages of 
Commons as well to the single mirrors and we're perfectly done (and third 
party mirrors can create their own software interface around that thing if 
they want).

Cheers, Arnomane
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url : http://lists.wikimedia.org/pipermail/commons-l/attachments/20070723/8e32408e/attachment-0001.pgp