[Labs-l] Render Tools now on Tool Labs

[Johannes Kroll]

Hi all,

the supporting tools which we are developing in the context of the
RENDER project are now available on Tool Labs.

 * The Article List Generator[0] is a tool that allows deep recursive
   searches in the Wikipedia category structure, using set operations
   to refine the search, and filtering the result for certain
   properties. The new version on Labs has two new filters: No Links
   finds articles which aren't linked from any other article (orphans).
   Pending Changes displays articles which have unsighted edits older
   than 12 hours. We also made some changes to the user interface.

 * The Article Monitor[1] displays statistics about any single
   Wikipedia article. The version on labs has some structural changes
   [2] (german text).

In related news, the interactive Limes map[3] is also running on Labs
since some time. We are hoping to integrate the map directly into
Wikipedia Limes articles if time and server resources permit. 

A new tool called CatCycle[4] can find cycles in Wikipedia categories,
that is, sub-categories containing one of their ancestor categories,
which is usually a sign of an error in the category structure. CatCycle
finds cycles quickly even several levels deep. It can also find
root nodes (categories without a parent category) and the shortest path
between nodes. It uses CatGraph (aka Graphserv/Graphcore) as a backend.
The find-cycles command in Graphcore is still a bit experimental. Please
tell me if you see something fishy!

Last not least, CatGraph[5] is now also running on wmflabs. CatGraph is
a custom graph database which is intended to be a backend for
developers which want to do fast recursive category-related search
queries in Wikipedia. It is running on a separate instance[6] in the
catgraph project. We imported a few more languages than we had on the
Toolserver. So far, everything is working fine.

The move from the Toolserver to Tool labs was not very hard. Apart from
different login procedures and things like that, the environment is not
so different. There was one problem with the maximum number of SQL
connections allowed per tool, which was much too low. This has been
changed after persistently bugging the admins for a while. :-) In the
Article List Generator, the progress bar doesn't display anything,
because there is a problem with chunked transfer encoding. This is a
problem with the version of the library used for WSGI though - flup -
and not a fault of Tool labs. We're working on it. The Change Detector
barfed because the time limit for an unused SQL connection is much
lower on Tool labs. We're working on that too.

Please give us feedback on the tools, Catgraph, etc. If you need help
porting a tool to Labs, you can talk to me. I can answer some questions
now that I moved some tools, and if I can't, I know who to ask. :) The
#wikimedia-labs channel on freenode is also a good place to get
support. 

Cheers,
Johannes


[0] http://tools.wmflabs.org/render/stools/alg
[1] http://tools.wmflabs.org/render/stools/articleMonitor
[2]
http://de.wikipedia.org/wiki/Wikipedia:Helferlein/Artikelmonitor#.C3.84nderungsprotokoll
[3] http://tools.wmflabs.org/render-tests/erdbeer/limes/
[4] http://tools.wmflabs.org/render-tests/catcycle-dev/catcycle.py
[5] http://tools.wmflabs.org/render-tests/catgraph/
[6] http://sylvester.wmflabs.org:8090/list-graphs

-- 
Johannes Kroll
Softwareentwickler

Wikimedia Deutschland e.V. | NEU: Obentrautstr. 72 | 10963 Berlin
Tel. (030) 219 158 26-0

http://wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das
Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.