On Sat, Oct 30, 2010 at 9:29 PM, Bryan Tong Minh bryan.tongminh@gmail.com wrote:
Hi all,
As most of you probably know, I wrote Flickr upload bot back in May 2007 because there was a lot demand for uploading free images from Flickr to Commons. And apparently people find it useful, since as of September 2010, over 80k images have been uploaded via this bot. In addition over 50k images have been uploaded via a similar bot by Magnus Manske.
Unfortunately as you may know, every other day those tools break (mine more than Magnus'). Both have an annoying authentication mechanism, which requires you to do extra stuff to be able to upload (either post a token to a file page, or use TUSC). Both problems would be solved if there was a MediaWiki extension to handle this task.
I eventually plan to write a MediaWiki extension that does such a thing and get it enabled on Commons. Therefore, I need to know what you like and dislike about those tools, so that I can take this feedback into account when writing this extension. Don't expect to see something in the short term though, but I hope that in the mid-long term we will have such an extension on Commons.
An excellent idea. I would like to add two suggestions, since they are within the scope of such a framework, and I would really like to see these done in a single, elegant extension.
First, there are other sites besides flickr that have license-compatible files we can use. Flickr may be the largest today, but there are many specialized ones like geograph.org.uk (pictures of places) and GIMP-SAVVY, and general ones like Picasa, Ipernity, and pictures owned by the Brazilian government. I am supporting search for those and more at [1] (>670K uses), but upload for most of them is currently manual (download, upload). It should be comparatively simple to write a more generic "transfer" parent class, which then would have derived classes for each of these sites; a simple method (e.g. for an image page on flickr, give me the URL of the most high-res file) might be sufficient for specialization.
Second, the special case of transfering from other MediaWiki sites. This includes all Wiki(m|p)edia projects, as I do in [2] with >400K uses (and [3], when it works), but also WikiTravel, and basically any other MediaWiki installation where a license can be determined. While it might seem to be easy to implement this, as we are more familiar with the site behaviour, there is no API for image metadata in MediaWiki, and transcoding the wikitext correctly, from all projects and languages, can be a real b***h, as countless more-or-less botched transfers from my bot show. Alternatives would be parsing the HTML (lossy), or putting more weight on the user to check for correctness.
Even if you do not chose to implement any of these transfer options initially, I believe you should code with these as further additions in mind. IMHO it would be a real shame to "waste" such an opportunity on flickr alone.
Cheers, Magnus
[1] http://toolserver.org/~magnus/fist.php [2] http://toolserver.org/~magnus/commonshelper.php [3] http://toolserver.org/~commonshelper2/?language=en&project=wikipedia&...