Also if you are a dumps user or have thoughts about how you would redo
them from scratch, get your ideas in now. We're not waiting for the
Dev Summit to get the work started. See
https://phabricator.wikimedia.org/T114019 for details, especially the
document linked at the end of the task description under 'FOR CURRENT
DISCUSSION'.
We need: comments on the strawman proposed model in the document
mentioned above; proposals for code we can reuse for any of those
pieces, especially the job queue/management piece. (Celery? Something
else?) What should the object store be based on, if we have one? Is
Ceph a dead end? Will Swift be deadly slow or should we use it since we
have it in house already? Etc. All comments on the ticket please so we
have them all in one place.
Expect that most of the code written here at WMF will be in python
unless someone else volunteers to write some. Anyone interested in
doing some of the development can step right up too of course.
Please forward this on to other fora as appropriate.
Ariel