On Thu, Jan 3, 2013 at 2:57 PM, Rob Lanphier robla@wikimedia.org wrote:
The thing that isn't covered here is how it works today, which I'll try to quickly sum up. Basically, it's a single cron job, running on hume[1]. So, that means that when a change is made on wikidata.org, one has to wait for this job to get around to running before the item. It'd be good for someone from the Wikidata team to
*sigh* the dangers of sending email in haste (and being someone who frequently composes email non-linearly). What I meant to say was this:
When a change is made on wikidata.org with the intent of updating an arbitrary wiki (say, Hungarian Wikipedia), one has to wait for this single job to get around to running the update on whatever wikis are in line prior to Hungarian WP before it gets around to updating that wiki, which could be hundreds of wikis. That isn't *such* a big deal, because the alternative is to purge the page, which will also work.
Another problem is that this is running on a specific, named machine. This will likely get to be a big enough job that one machine won't be enough, and we'll need to scale this up.
It would be good for Daniel or someone else from the Wikidata team to chime in to verify I'm characterizing the problem correctly.
Rob