[Foundation-l] [Wikitech-l] mwdumper does not work

[David A. Desrosiers]

> Different OS.   I am running Linux with FedoraCore 5.   I think its 
> related to java incompatibility and/or mysql issues with Federal Core
> 5.  The php-mysql and a lot of other subsystems were changed in the
> Linux distros due to licensing issues with the MySQL code moving
> forward. 

I've been doing this on Debian since at least 2004, and recently
switched to BSD because I needed some features of the OS that Linux
doesn't have yet (no OS wars from the peanut gallery please, I've been
running Linux for > 12 years here ;)

My development machine for the Plucker Wikipedia work is my Ubuntu
Feisty laptop (a Thinkpad T42p with as much disk and RAM as it will
hold, if it matters). 

My BSD machine is an AMD64/4600+/4G/SATA, and it works nicely there as
well (500k pages/sec., according to mwdumper). 

> >It works great. You're doing something wrong if it isn't working. 

> That's a stretch to say, since we are dealing with different OS's.
> I  have purged out the endb database completely and I am re-running
> the entire mwdumper test with Rob's UPDATE suggestions again.  The
> previous database had data leftover from an importDump.php run.  After
> the test completes, I will post the results.  I am farily certain
> mwdumper has some incompatiblity problems with the stock Linux 
                                                     ^^^^^^^^^^^
> Distributions as it has been reported in a lot of places.
^^^^^^^^^^^^^^^
i.e. NOT Fedora Core Xx or Red Hat... It works on Debian Unstable and
Ubuntu Feisty (previously Edgy Eft, also worked there). 

Please cite the "lots of places" that you found reports of mwdumper
failing on stock distributions.. 

I'd check that your distribution of choice doesn't have some bugs filed
against its MySQL packages, or perhaps your Java options aren't ideal
for your environment. 

Incidentally, you can just redirect mwdumper's output to a .sql file and
use mysql directly to import that if you wish. Both work perfectly here,
albeit without any sort of feedback from the redirect method.. 

	mwdumper [options] > big_ass_file.sql 
	mysql mediawiki < big_ass_file.sql

Since I don't know the configuration you're using for MySQL, I can't say
whether or not you can optimize it. The defaults are most-assuredly
*NOT* optimal for a Wikipedia import, period. You need to make quite a
few changes to optimize it for bulk inserts ala mwdumper. 

> Since more people run Linux than BSD at present, it bears looking into
> and getting fixed and/or understood.  I am happy to devote time to
> getting to the bottom  of it for Rob and Brion.

That's an opinion, not fact. BSD runs on a lot more machines than you
realize, and not just desktops. Ahem, I digress. 

It works here on Linux and BSD, and has since at least 2004 when I
started this. I have irc logs going back that far that can validate this
over and over if you wish. 

If you want, I can install FC5 here in VMware and give it a try, but I
suspect as long as those packages are current, it will probably work. 


-- 
David A. Desrosiers
desrod at gnu-designs.com
Skype username: setuid
http://gnu-designs.com
”Always strive to silence the voices that oppose you."