Now that we have proper support for video in MediaWiki, the upload limit of 20 MB is reached much more easily. Could this limit be increased?
Bryan
On 10/25/07, Bryan Tong Minh bryan.tongminh@gmail.com wrote:
Now that we have proper support for video in MediaWiki, the upload limit of 20 MB is reached much more easily. Could this limit be increased?
<record mode="broken"> enable URL-based uploads... </record>
Magnus
Magnus Manske wrote:
On 10/25/07, Bryan Tong Minh bryan.tongminh@gmail.com wrote:
Now that we have proper support for video in MediaWiki, the upload limit of 20 MB is reached much more easily. Could this limit be increased?
<record mode="broken"> enable URL-based uploads... </record>
The resource requirements on the server side are the same, it just means the user has to have a web server. There's no reason we would should have a different limit for uploading via HTTP download than for HTTP uploads.
The system administration question is: do we have the resources and architecture to support a dramatic growth in disk space and bandwidth usage driven by large video uploads?
My tentative answer would be that I don't think we're very far ahead of the growth curve on storage. I don't know how expensive it will be or whether we will be able to deploy new hardware fast enough. Our record for deployment of storage hardware has been pretty poor.
-- Tim Starling
On Fri, Oct 26, 2007 at 02:29:10AM +1000, Tim Starling wrote:
My tentative answer would be that I don't think we're very far ahead of the growth curve on storage. I don't know how expensive it will be or whether we will be able to deploy new hardware fast enough. Our record for deployment of storage hardware has been pretty poor.
Would it be productive to look into ATAoverEthernet SAN shelves? They're a bit pricey still, but they have the advantage that you don't need expensive controllers like FC in the attached boxen as well: you just Ethernet stuff together.
You'd want it on a separate LAN, for oblivious raisins, but the client drivers are in 2.6.11+, IIRC. The company that's spearheading this protocol has drive shelves up to 16 position, as I recall. Coraid is their name.
#insert <std_disclaimer.h>
Their 3U 15 drive shelf is $4k; they have a 24 slot which is probably 6K. And, of course, 500GB Seagate 7200.11 SATAs are about $100 each right now. I'm pretty sure the 1TBs are still ... well, let me look.
Yup, I was within 5%: $334 a piece at Newegg.
So, assuming the 24 slot -- and these cages do hardware raid in front of the AoE -- that's 12TB raw for $8400.
Or, of course, 24TB raw for slightly over $14k.
Or is the problem not capex but engineering and deployment?
Cheers, -- jra
On 25/10/2007, Tim Starling tstarling@wikimedia.org wrote:
Magnus Manske wrote:
On 10/25/07, Bryan Tong Minh bryan.tongminh@gmail.com wrote:
Now that we have proper support for video in MediaWiki, the upload limit of 20 MB is reached much more easily. Could this limit be increased?
<record mode="broken"> enable URL-based uploads... </record>
The resource requirements on the server side are the same, it just means the user has to have a web server. There's no reason we would should have a different limit for uploading via HTTP download than for HTTP uploads.
The system administration question is: do we have the resources and architecture to support a dramatic growth in disk space and bandwidth usage driven by large video uploads?
My tentative answer would be that I don't think we're very far ahead of the growth curve on storage. I don't know how expensive it will be or whether we will be able to deploy new hardware fast enough. Our record for deployment of storage hardware has been pretty poor.
Ignoring the problems of practically and resources, I think a higher limit is necessary to allow Commons to fulfill it's mission. Not only video, but with music (long classical movements that we couldn't reasonably break into parts) and images (high-resolution maps, astronomical images, &c.).
There is a great wealth of PD films that would enrich Commons but are too big to go up at the moment (early silent films: Edison, e.g.).
There are two issues here: (1) should Wikimedia allow larger file uploads?; and (2) can Wikimedia allow larger file uploads? My answer (1) is yes, and my answer to (2) is a tentative "probably". Whether it is something we can reasonably do now or not, I think it is something we should push for.
On Thu, Oct 25, 2007 at 10:02:58PM +0100, Oldak Quill wrote:
limit is necessary to allow Commons to fulfill it's mission. Not only video, but with music (long classical movements that we couldn't reasonably break into parts) and images (high-resolution maps, astronomical images, &c.).
We *are* remembering that performances of no-longer-copyrighted classical music *themselves* have copyright, to the performers or the owners of the masters, right? ;-)
Cheers, -- jra
On 28/10/2007, Jay R. Ashworth jra@baylink.com wrote:
On Thu, Oct 25, 2007 at 10:02:58PM +0100, Oldak Quill wrote:
limit is necessary to allow Commons to fulfill it's mission. Not only video, but with music (long classical movements that we couldn't reasonably break into parts) and images (high-resolution maps, astronomical images, &c.).
We *are* remembering that performances of no-longer-copyrighted classical music *themselves* have copyright, to the performers or the owners of the masters, right? ;-)
Yes, the vast majority of classical music is under copyright (and is copyrightable because the performance involves some sort of creativity). There is some classical music in the public domain or licensed under a copyleft license. Take a look at http://commons.wikimedia.org/wiki/Category:Classical_music for free/libre music on Commons at the moment.
2 of the first pieces to appear in that category approach 20 MB. Both are licensed with the EFF Open Audio License.
Alessandro Poglietti - Air and Variations.ogg Antonio Pasculli - Gran Concerto.ogg
wikitech-l@lists.wikimedia.org