Also if you are a dumps user or have thoughts about how you would redo them from scratch, get your ideas in now. We're not waiting for the Dev Summit to get the work started. See https://phabricator.wikimedia.org/T114019 for details, especially the document linked at the end of the task description under 'FOR CURRENT DISCUSSION'.

We need: comments on the strawman proposed model in the document mentioned above; proposals for code we can reuse for any of those pieces, especially the job queue/management piece. (Celery? Something else?) What should the object store be based on, if we have one? Is Ceph a dead end? Will Swift be deadly slow or should we use it since we have it in house already? Etc. All comments on the ticket please so we have them all in one place.

Expect that most of the code written here at WMF will be in python unless someone else volunteers to write some. Anyone interested in doing some of the development can step right up too of course.

Please forward this on to other fora as appropriate.

Ariel