On Thu, Jul 10, 2014 at 3:50 PM, Daniel Kinzler <daniel.kinzler@wikimedia.de
wrote:
Am 09.07.2014 19:39, schrieb Dimitris Kontokostas:
On Wed, Jul 9, 2014 at 6:13 PM, Daniel Kinzler <
daniel.kinzler@wikimedia.de
mailto:daniel.kinzler@wikimedia.de> wrote:
Am 09.07.2014 08:14, schrieb Dimitris Kontokostas: > Maybe I am biased with DBpedia but by doing some experiments onEnglish
> Wikipedia we found that the ideal update with OAI-PMH time wasevery ~5
minutes. > OAI aggregates multiple revisions of a page to a single edit > so when we ask: "get me the items that changed the last 5 minutes"we skip the
> processing of many minor edits > It looks like we lose this option with PubSubHubbub right? I'm not quite positive on this point, but I think with PuSH, this isdone by the
hub. If the hub gets 20 notifications for the same resource in oneminute, it
will only grab and distribute the latest version, not all 20. But perhaps someone from the PuSH development team could confirmthis.
It 'd be great if the dev team can confirm this. Besides push notifications, is polling an option in PuSH? I briefed
through the
spec but couldn't find this.
Yes. You can just poll the interface that the hub uses to fetch new data.
Thanks for the info Daniel
I'm waiting for the dev to confirm the revision merging and one last question / use case from me.
Since you'll sync to an external server (in Google right?), did you set any requirements on the durability of the changesets? I mean, are the changes stored *for ever* or did you set any ttl? e.g. my application breaks for a week and I want to resume, or I download a one-month old dump and want to get in sync, etc
In OAI-PMH I could for instance set the date to 15/01/2001 and get all pages by modification date In PuSH this would require some sort of importing and is probably out of the question right? :)
Cheers, Dimitris
-- daniel
-- Daniel Kinzler Senior Software Developer
Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.