Probably no one else does know (how to run these dumps).
So to clear up a little of the confusion/lack of information,
automating staged dumps is currently tracked in Phabricator here:
https://phabricator.wikimedia.org/T107750
Just got the configuration files puppetized for the scheduler, which
was started manually this time but won't need any other intervention.
I'll be keeping an eye on it anyways since this too is new code.
I've been playing whack-a-mole with bugs in the streaming stubs/logs
production, which is why the delay again last month in completing enwp
and why only one set of dumps went off. This run should be better,
cross fingers that nothing else crept in.
After one complete run finishes for the non en wikipedia dumps, we'll
be able to do a second run, probably leaving off revision history since
most folks don't need those more than once a month. <-- If this is
wrong, speak up now.
The order in which things are run has been redone a bit, hopefully to
get the parts of the dumps people need done sooner in the cycle.
I'm still reluctant to do partial dump copies over labs unless we can
guarantee a little more space over there to keep e.g. the last full
good dump along with whatever new files are being copied. I've opened
a Phabricator task for that here
https://phabricator.wikimedia.org/T108077 and added Coren so we can
figure out the space issue.
I've not wanted to weigh in earlier until I had things in place to kick
off the new run. My apologies if that just made folks more annoyed.
Ariel
Στις 04-08-2015, ημέρα Τρι, και ώρα 20:08 -0600, ο/η Bryan White
έγραψε:
Again, I was on the message you mentioned. I was the
person Ariel
was
responding too. I have been in touch with Ariel in the past, but he
not responded to any emails from me or others for almost three weeks
about the situation. Problems in Greece, on holiday, real-life,
...?
> > Last month, French, German, Japanese,
Russian, Spanish and
> > others were never completed.
I use over 40 languages, not just enwiki. The one's that didn't
finish are useless for the majority of people. It takes two weeks to
dump all languages except for enwiki and only one has started in 18
days. By "started", I mean re-started a pre-existing dump to
completion. No new dump has commenced in over a month. For many
languages (French, German, etc) it has been two months. Things are
dead.
Does anybody know how to get dumps working who's name isn't Ariel?
Bryan
On Tue, Aug 4, 2015 at 7:36 PM, Hydriz Scholz <admin(a)alphacorp.tk>
wrote:
Hello,
I apologize if I was not clear enough. The dumps are being run
manually because Ariel is currently testing the new framework. It
is thus to be expected if the dumps are not being produced
according to schedule. Also, I believe the enwiki dump is still
being produced, so I doubt that the process is "dead", it's just
slow.
Perhaps Ariel is able to provide more information on this issue, as
I am unable to comment further than this.
On 5 Aug 2015, at 08:38, Bryan White
<bgwhite(a)gmail.com> wrote:
I did mention that dumps are now run manually. I was on the
message
you mentioned. Only one dump has started in the last 18 days,
this
is in addition to larger languages never completing. Going
from two
dumps a month to no dumps a month is not a "delay", it is dead.
Bryan
On Tue, Aug 4, 2015 at 6:08 PM, Hydriz Scholz
<
admin(a)alphacorp.tk> wrote:
Hello,
Ariel noted last month that the dumps are currently being ran
manually. [1] Having delays in the dumps production should thus
be expected.
Hope this helps.
[1]:
https://lists.wikimedia.org/pipermail/xmldatadumps-l/2015-
July/001153.html
> On 5 Aug 2015, at 07:26, Bryan White <bgwhite(a)gmail.com>
> wrote:
>
> Three months ago, "they" changed the way dumps are done.
> "They"
> wanted stubs for every language done first and work on the
> rest of the
> dump later.
>
> Old way: Two dumps a month for all languages except enwiki.
> A new
> dump was started when one completed. Except for larger
> languages,
> dumps were done in under a day.
>
> New way: One dump a month for majority of languages. Dumps
> have to
> be started manually. All dumps take 2-4 weeks to complete.
>
> Last month, French, German, Japanese, Russian, Spanish and
> others were
> never completed.
>
> It's about to be August 5. No dumps have started this month.
> Only
> one dump has started in over two weeks.
>
>
> Does anybody know how to get dumps working again.
>
> Bryan
>
> _______________________________________________
> Labs-l mailing list
> Labs-l(a)lists.wikimedia.org
>
https://lists.wikimedia.org/mailman/listinfo/labs-l
_______________________________________________
Labs-l mailing list
Labs-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l
_______________________________________________
Labs-l mailing list
Labs-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l
_______________________________________________
Labs-l mailing list
Labs-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l
_______________________________________________
Xmldatadumps-l mailing list
Xmldatadumps-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l