There is a memory leak when we use spawned fetchTextPass.php, this only affects wikis with millions of respawns. Of course there is only one f those, wikidata. I restarted this yesterdya morning without spawning a separate process and right now it's about 2/3 of the way done for the May run. Once that completes successfully I'll get the rest of that run going and eventually backtrack and get the missing bits of the previous run as well.
Ariel
Στις 16-05-2013, ημέρα Πεμ, και ώρα 18:53 +0300, ο/η Ariel T. Glenn έγραψε:
I'm seeing some issues with the history phase of the wikidata dumps taking a huge amount of memory and causing the server they are on to swap. I've shot the jobs and left less worker running on the one host for now; I'll investigate in depth tomorrow.
Ariel
Xmldatadumps-l mailing list Xmldatadumps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l