I have the same problem as Danny. My problem with AWB (which is free in itself) is that it depends on the non-free library .NET and it works on Windows only. A Python command-line tool will be perfect.

2009/7/25 John Doe phoenixoverride@gmail.com:

awb does but wont work on the ts

On Sat, Jul 25, 2009 at 8:46 AM, Simon Walker stwalkerster@googlemail.com wrote:

...

Does AWB not do something along those lines?

2009/7/25 Danny B. Wikipedia.Danny.B@email.cz

...

Hello,

I'm looking for any kind of tool which would take the XML dump (most probably the pages-meta-current.xml.bz2, at least the pages-articles.xml.bz2) and would return the list of page titles (or alternatively/configurably page ids) of pages containing given string.

Does anybody have such (kind of) tool and is willing to share? Both command line or webpage interface are OK.

Thank you.

Danny B.

---

Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l

-- Regards,

Simon Walker User:Stwalkerster on all public Wikimedia Foundation wikis Administrator on the English Wikipedia Developer of Helpmebot, the ACC tool, and Nubio 2 FAQ repository

---

Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l

---

Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Hi,

Am 25.07.2009, 17:13 Uhr, schrieb River Tarnell river@loreley.flyingparchment.org.uk:

.NET is not, itself, non-free. Microsoft's implementation (the most common one) is, but Mono (http://mono-project.com/Main_Page) is not. perhaps the AWB developers could make whatever changes are needed to run it on a free implementation.

Mono works great, I'm using bots using the DotNetWikiBot framework on the toolserver.

For simple parsing of a pages-articles.xml file, you may test a script, I used some time ago - it is a very simple xml parser (for the pages-articles.xml structure) and calls a function called "test" with the article title and the text of the article. Its not the perfect solution but the solution implemented in five minutes ;)

function test($title, $text) { // do something here }

$filename = "enwiki-200XXXXX-pages-articles.xml"; $dataFile = fopen($filename, "r"); if ($dataFile) { $status = 0; while (!feof($dataFile)) { $buffer = fgets($dataFile, 4096); if (($status == 0) && (stripos($buffer, "<page>") !== false)) $status = 1; elseif (($status == 1) && (stripos($buffer, "<title>") !== false)) $title = strip_tags($buffer); elseif (($status == 1) && (stripos($buffer, "<revision>") !== false)) $status = 2; elseif (($status == 2) && (stripos($buffer, "<text ") !== false)) { $status = 3; $text = strip_tags($buffer); if (stripos($buffer, "</text>") !== false) { $status = 2; } } elseif (($status == 3) && (stripos($buffer, "</text>") === false)) $text .= strip_tags($buffer); elseif ($status == 3) { $text .= strip_tags($buffer); $status = 2; } elseif (($status == 2) && (stripos($buffer, "</revision>") !== false)) $status = 1; elseif (($status == 1) && (stripos($buffer, "</page>") !== false)) { test(trim($title), trim($text)); $title = ""; $text = ""; $status = 0; } } fclose($dataFile); } else { die("File not found: $filename"); }

Hi,

Am 25.07.2009, 18:20 Uhr, schrieb Osama KM osamak.wfm@gmail.com:

Well, C# and .NET in general are patented so they aren't free and they're bad choices.[1].

This post is outdated in my opinion. Microsoft reacted on it and is now applying the Microsoft Community Promise on the ECMA 334 and ECMA 335 specs of .NET [2]. And this is not only a "promise" but legally binding. There will be a mono version only having the core parts [3]. So I don't see problems using mono on the toolserver. But of course I understand if you don't want to use it ;).

[2] http://port25.technet.com/archive/2009/07/06/the-ecma-c-and-cli-standards.as... [3] http://tirania.org/blog/archive/2009/Jul-06.html

Sincerely, Christian Thiele