Hi Daniel,
On Fri, Apr 26, 2013 at 6:15 PM, Daniel Kinzler <daniel.kinzler@wikimedia.de
wrote:
On 26.04.2013 16:56, Denny Vrandečić wrote:
The third party propagation is not very high on our priority list. Not
because
it is not important, but because there are things that are even more
important -
like getting it to work for Wikipedia :) And this seems to be
stabilizing.
What we have, for now:
- We have the broadcast of all edits through IRC.
This interface is quite unreliable, the output can't be parsed in an unambiguous way, and may get truncated. I did implement notifications via XMPP several years ago, but it never went beyond a proof of concept. Have a look at the XMLRC extension if you are interested.
- One could poll recent changes, but with 200-450 edits per minute, this
might
get problematic.
Well, polling isn't really the problem, fetching all the content is. And you'd need to do that no matter how you get the information of what has changed.
- We do have the OAIRepository extension installed on Wikidata. Did
anyone try that?
In principle that is a decent update interface, but I'd recommend not to use OAI before we have implemented feature 47714 ("Support RDF and API serializations of entity data via OAI-MPH"). Right now, what you'd get from there would be our *internal* JSON representation, which is different from what the API returns, and may change at any time without notice.
What we do right now in DBpedia Live is that we have a local clone of Wikipedia that get's in sync using the OAIRepository extension. This is done to abuse our local copy as we please.
The local copy also publishes updates with OAI-PMH that we use to get the list of modified page ids. Once we get the page ids, we use the normal mediawiki api to fetch the actual page content. So, feature 47714 should not be a problem in our case since we don't need the data serialized directly from OAI-PMH
Cheers, Dimitris
-- daniel
-- Daniel Kinzler, Softwarearchitekt Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l