A heads up to anyone who uses these, builds packages for them, etc: after a
bit of tlc they have been moved to their own repo in the 'master' branch:
clone from gerrit:
operations/dumps/import-tools.git
or browse at
https://phabricator.wikimedia.org/diffusion/ODIM/
Patches to gerrit, bug reports to phab as before, except that I'l be a bit
more attentive to them now that these scripts are being treated as
semi-real tools instead of toys.
Ariel
I needed to clean up a bunch of tech debt before redoing the page content
dump 'divvy up into small pieces and rerun if necessary' mechanism. I
cleaned up a bit too much and broke stub and article recombine dumps in the
process.
The fix has been deployed, I shot all the dump processes, marked the
relevant jobs (big and huge wikis only) as failed, and tossed the bad files.
The dumps should resume from the failed steps in about an hour.
Follow along at https://phabricator.wikimedia.org/T160507 for all the gory
details.
Ariel