Hello, Postrgesql starts now to support also JSON, so we should try to find a way to bring Wikidata available for us and I would prefer to use furthermore SQL.
One way could be minutely diff-files, that's the way OpenStreetMap use. Alternatively we could use API for each updated article.
Every central service is better than let's fighting everyone with the problem alone.
I think the support of hierarchical informations was the key to use JSON instead of a key-value store. A point that I can understand.
Greetings Tim
Am 19.04.2013 00:43, schrieb Daniel Schwen:
if these JSON-data is stored where the normal wiki-text is, it is imposable
To my understanding it is.
for us to replicate it: Because we have no access to these wmf-servers, there
IMO that was a questionable design decision. JSON plaintext storage in SQL is shoehorning a do-it-yourself object store onto a classical RDBMS. Postgres at least has hstore. This may be even a genuine usecase for one of those hipster databases (noSQL like mongdb etc.). But who knows what points were taken into consideration when making this decision.
would be no way to separate Wikidata from the rest
I don't understand why separating plaintext storage between different projects would be an issue. Is it all lumped into one storage "namespace"? I'm sure nobody at Wikimedia would be the least bit motivated to make this data available to the toolserver, but maybe it will be usable in labs. Otherwise it would be quite a waste of a great opportunity.
and/or we have not enough disc-space.
If you can separate it out I seriously doubt that wikidata would require storage any where near as large in magnitude as the other wikimedia projects (at least in the mid-term)
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette