Hi!
I'd like to raise a topic of handling change notifications and
up-to-date-ness of Wiki pages data with relation to page props.
First, a little background about how I arrived at the issue. I am
maintaining Wikidata Query Service, which updates from Wikidata using
recent changes API and RDF export format for Wikidata pages. Recently,
we have implemented using certain page properties, such as link &
statement counts. This is when I discovered the issue: the page
properties are not updated when the page (Wikidata item) is edited, but
are updated later, as I understand by a job.
Now, this leads to a situation where when I have a recent changes entry,
and I look at the RDF export page - which contains page props derived
data now - I can not know if page props data is up-to-date or not.
Moreover, if the job - some unknown and undefined time later - updates
the page props, I get no notification since the modification is not
reflected in recent changes. This makes usage of information derived
from page props very hard - you never know if the data is stale or
whether the data in page props matches the data in the page.
The problem is described in more detail in
https://phabricator.wikimedia.org/T145712
I'd like to find a solution for it, but not sure how to proceed.
The data specific to this case can be easily generated from the data
already present in memory during the page update, but I assume there
were some reasons why it was deferred.
We could make some kind of notification when updating page props, though
that would probably seriously increase the number of notifications and
thus slow the updates. Also, in some cases, the second notification may
not be necessary since the page props were updated before I've processed
the first one, but I have no way of knowing it now.
Any advice on how to solve this issue?
--
Stas Malyshev
smalyshev(a)wikimedia.org