Re: [Commons-l] GSoC 2016 | Porting catimages to pywikibot-core

(Just replying on Commons-l with a non-tech observation. If more tech stuff arises I'll add it to Phabricator instead) This looks like a useful contained project, though a lot to be done in 12 weeks. :-) I was not familiar with catimages.py. It would be great if using the module for the preparation or housekeeping of large batch uploads were easy and not time consuming to try. As Commons grows we are seeing more donations over 10,000 images and have had a few with over 1m. Uploads of this size make manual categorization a huge hurdle, so automatic 'tagging' of image characteristics would be a useful way of breaking down such a large batch to highlight the more interesting outliers or mistakes, which can then be prioritized on a backlog for human review. For example, in my upload projects I have problems detecting: * incomplete uploads resulting from server failures. Checksum comparisons would mean re-downloading files, which would be unnecessarily bandwidth expensive, but local image analysis would highlight these. * uploads that are mostly blank pages in old scanned books. I have a simple detection process, but it would be neat to have a more common standard way of doing this. * distinguishing between scans with diagrams and line drawings/cartoons, printed old photographs, newsprint and text pages. It would be great if the testing routines you use during the project could tackle any of these and be written up as practical case studies. As well as the Phabricator write-up/tracking of the project, it would be useful to have an on-wiki Commons or Mediawiki user guide. Perhaps this can be sketched out as you go along during the project, giving an insight into what other users or amateur Python programmers might do to customize or make better use of the module? Having an more easy to find manual, might avoid others going off on their own tangents using various off the shelf image modules, when they could just plug in catimages with a smallish amount of configuration. P.S. If you would like to test the tool on some large collections with predictable formats, try looking through < https://commons.wikimedia.org/wiki/User:Fae/Project list >. The 1/2 million images in the book plates project would be an interesting sample set. Thanks, Fae On 18 May 2016 at 02:53, Abdeali Kothari &lt;abdealikothari(a)gmail.com&gt; wrote: to automate towards useful to find T129611[2]. -- faewik(a)gmail.com https://commons.wikimedia.org/wiki/User:Fae Personal and confidential, please do not circulate or re-quote.