Nick Jenkins wrote:
(Also the SQL dump output needs to actually be tested before we dedicate a few gigs to it.)
I'm happy to be a guinea pig. Just give tell me where to get it from, and I'll leave it importing overnight and report back any errors/problems and whether it worked or not.
I haven't had a chance to upload some test conversions yet but I'd appreciate it if you'd give a quick test of the program: http://leuksman.com/misc/mwdumper-preview.zip
This .zip contains the compiled converter program in its present state, with a bundled executable for Linux/x86 and as CIL assemblies for any other OS.
The Linux binary does *not* require a Mono installation; I've tested it on Ubuntu Hoary and Fedora Core 1. (It does require glib2, which is probably installed by default on any modern Linux.)
// O/S is Debian Woody 3.0r6 GNU/Linux, with a backport of the 2.6.7 kernel, so it's semi-modern (i.e. not truly ancient, but definitely not cutting edge either - more kind of middle-aged, with a few grey hairs, and driving a station wagon or mini-van).
// Tried running initially - got error loading libgmodule-2.0.so.0 : ludo:/home/nickj/wikipedia/new-dumper# wget http://leuksman.com/misc/mwdumper-preview.zip ludo:/home/nickj/wikipedia/new-dumper# unzip mwdumper-preview.zip ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview# ls README cil doc linux-i386 ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview# cd linux-i386/ ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386# ls mwdumper ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386# ./mwdumper ./mwdumper: error while loading shared libraries: libgmodule-2.0.so.0: cannot open shared object file: No such file or directory ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386# ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386# strace ./mwdumper execve("./mwdumper", ["./mwdumper"], [/* 18 vars */]) = 0 uname({sys="Linux", node="ludo", ...}) = 0 brk(0) = 0x857c000 open("/etc/ld.so.preload", O_RDONLY) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=10970, ...}) = 0 old_mmap(NULL, 10970, PROT_READ, MAP_PRIVATE, 3, 0) = 0x40014000 close(3) = 0 open("/lib/libpthread.so.0", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`C\0\000"..., 1024) = 1024 fstat64(3, {st_mode=S_IFREG|0644, st_size=102172, ...}) = 0 old_mmap(NULL, 81316, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x40017000 mprotect(0x40024000, 28068, PROT_NONE) = 0 old_mmap(0x40024000, 28672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0xd000) = 0x40024000 close(3) = 0 open("/lib/libm.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\2007\0"..., 1024) = 1024 fstat64(3, {st_mode=S_IFREG|0644, st_size=130088, ...}) = 0 old_mmap(NULL, 132708, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x4002b000 mprotect(0x4004b000, 1636, PROT_NONE) = 0 old_mmap(0x4004b000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1f000) = 0x4004b000 close(3) = 0 open("/lib/i686/mmx/libgmodule-2.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i686/mmx", 0xbfffee94) = -1 ENOENT (No such file or directory) open("/lib/i686/libgmodule-2.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i686", 0xbfffee94) = -1 ENOENT (No such file or directory) open("/lib/mmx/libgmodule-2.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/mmx", 0xbfffee94) = -1 ENOENT (No such file or directory) open("/lib/libgmodule-2.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 open("/usr/lib/i686/mmx/libgmodule-2.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i686/mmx", 0xbfffee94) = -1 ENOENT (No such file or directory) open("/usr/lib/i686/libgmodule-2.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i686", 0xbfffee94) = -1 ENOENT (No such file or directory) open("/usr/lib/mmx/libgmodule-2.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/mmx", 0xbfffee94) = -1 ENOENT (No such file or directory) open("/usr/lib/libgmodule-2.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib", {st_mode=S_IFDIR|0755, st_size=16384, ...}) = 0 writev(2, [{"./mwdumper", 10}, {": ", 2}, {"error while loading shared libra"..., 36}, {": ", 2}, {"libgmodule-2.0.so.0", 19}, {": ", 2}, {"cannot open shared object file", 30}, {": ", 2}, {"No such file or directory", 25}, {"\n", 1}], 10./mwdumper: error while loading shared libraries: libgmodule-2.0.so.0: cannot open shared object file: No such file or directory ) = 129 _exit(127) = ? ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386#
// Based on your message about needing glib2, looked for and installed this: ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/cil# apt-get install libglib2.0-0 Reading Package Lists... Done Building Dependency Tree... Done The following NEW packages will be installed: libglib2.0-0 0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 288kB of archives. After unpacking 723kB will be used. Get:1 http://debian.ihug.com.au woody/main libglib2.0-0 2.0.1-2 [288kB] Fetched 288kB in 5s (51.0kB/s) Selecting previously deselected package libglib2.0-0. (Reading database ... 24745 files and directories currently installed.) Unpacking libglib2.0-0 (from .../libglib2.0-0_2.0.1-2_i386.deb) ... Setting up libglib2.0-0 (2.0.1-2) ... ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386#
// tried running again: ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386# ./mwdumper ./mwdumper: /lib/ld-linux.so.2: version `GLIBC_2.3' not found (required by ./mwdumper) ./mwdumper: /lib/libpthread.so.0: version `GLIBC_2.3.2' not found (required by ./mwdumper) ./mwdumper: /lib/libc.so.6: version `GLIBC_2.3.2' not found (required by ./mwdumper) ./mwdumper: /lib/libc.so.6: version `GLIBC_2.3' not found (required by ./mwdumper) ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386# dpkg -S /lib/ld-linux.so.2 libc6: /lib/ld-linux.so.2 ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386# dpkg -s libc6 Package: libc6 Status: install ok installed Priority: required Section: base Installed-Size: 12760 Maintainer: Ben Collins bcollins@debian.org Source: glibc Version: 2.2.5-11.8 Replaces: ldso (<= 1.9.11-9), timezone, timezones, gconv-modules, libtricks, libc6-bin, netkit-rpc, netbase (<< 4.0) Provides: glibc-2.2.5-11.8 Suggests: locales, glibc-doc Conflicts: strace (<< 4.0-0), libnss-db (<< 2.2-3), timezone, timezones, gconv-modules, libtricks, libc6-doc, libc5 (<< 5.4.33-7), libpthread0 (<< 0.7-10), libc6-bin, libwcsmbs, apt (<< 0.3.0), libglib1.2 (<< 1.2.1-2), libc6-i586, libc6-i686, libc6-v9, netkit-rpc Conffiles: /etc/default/devpts fc857c5ac5fb84d80720ed4d1c624f6e Description: GNU C Library: Shared libraries and Timezone data Contains the standard libraries that are used by nearly all programs on the system. This package includes shared versions of the standard C library and the standard math library, as well as many others. Timezone data is also included.
ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386#
// I.e. only have glibc 2.2.5, need >= 2.3.2. :-(
// Upgrading currently isn't really viable as the machine's reason for existence is to be a test box configured in the same way as a production server (i.e. should have the same versions of software), and currently the production machine works fine, so the current feeling is "don't fix what ain't broken".
If you're on Linux and the binary does or doesn't work, please let me know. (If it fails, please include output of 'ldd mwdumper'.)
ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386# ldd mwdumper ./mwdumper: /lib/ld-linux.so.2: version `GLIBC_2.3' not found (required by ./mwdumper) ./mwdumper: /lib/libpthread.so.0: version `GLIBC_2.3.2' not found (required by ./mwdumper) ./mwdumper: /lib/libc.so.6: version `GLIBC_2.3.2' not found (required by ./mwdumper) ./mwdumper: /lib/libc.so.6: version `GLIBC_2.3' not found (required by ./mwdumper) libpthread.so.0 => /lib/libpthread.so.0 (0x40017000) libm.so.6 => /lib/libm.so.6 (0x4002b000) libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0x4004d000) libdl.so.2 => /lib/libdl.so.2 (0x40051000) libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0x40054000) libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x40059000) libc.so.6 => /lib/libc.so.6 (0x400bc000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) ludo:/home/nickj/wikipedia/new-dumper/mwdumper-preview/linux-i386#
All the best, Nick.
Nick Jenkins wrote:
// O/S is Debian Woody 3.0r6 GNU/Linux, with a backport of the 2.6.7 kernel, so it's semi-modern (i.e. not truly ancient, but definitely not cutting edge either - more kind of middle-aged, with a few grey hairs, and driving a station wagon or mini-van).
URRRGGGGGG!
Debian Woody is my mortal enemy. I thought it was dead finally, but no...
I'll see what I can whip up.
-- brion vibber (brion @ pobox.com)
wikitech-l@lists.wikimedia.org