Hi Mohammed,

Understood regarding changes to the relational schema, and great to hear
that optimising the APIs is in the works!

I'm wondering if there is some documentation relating to the relational
schema used, and would it be possible to report changes to the schema
across the different versions?

Would such changes to the relational schema also affect those who use
the standard APIs for data upload and later wish to upgrade their
version of Wikibase over the existing data?

Best,
Aidan

On 2021-08-17 8:44, Mohammed Sadat Abdulai wrote:
> Hello everyone,
>
>
> The Wikibase development team is excited to see the emergence of
> community-created import tooling such as RaiseWikibase and
> wikibase-insert, particularly because Wikibase does not yet come
> equipped with its own import mechanism “out of the box”. To help better
> support the community, we would like to offer some advice to toolmakers
> and provide some insight into our planned explorations into making
> API-based importing better and faster.
>
>
> We anticipate an inherent issue with tools that directly inject
> information into the Wikibase database tables. Namely, that the schemas
> that such tools rely on are subject to change. Normal development
> processes across Wikimedia can (and likely will) endanger the long-term
> health and stability of these tools, an outcome we would like to avoid
> as much as possible. Specifically, we can’t guarantee that the layout or
> content of the tables that these tools write to will not change.
> Although Wikibase does not have its own public stable interface policy,
> we work with an eye on Wikidata's policy
> <https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy#Unstable_Interfaces>.
>
>
> We would like to offer the following advice to tool developers:
>
>
>  1.
>
>     Wherever possible, use the HTTP APIs.
>
>  2.
>
>     When using them, make sure to batch requests when possible.
>
>  3.
>
>     Understand that by reading or writing directly to DB tables,
>     Wikibase behavior may end up broken in subtle ways.
>
>  4.
>
>     Understand that tools which read or write from the DB on today’s
>     Wikibase may not still work on tomorrow’s Wikibase.
>
>
> The Wikibase development roadmap
> <https://www.wikidata.org/wiki/Wikidata:Development_plan#Wikibase_ecosystem>for
> the year is already tightly booked. However, Wikimedia Germany intends
> to dedicate some resources during our next 2021 prototyping effort
> towards exploring ways to optimize our API -- specifically, a solution
> approaching something like an "import mode" which would bypass
> unnecessary actions when inserting a body of previously vetted
> information. This might, for example, include ignoring checks on the
> uniqueness of labels or user permissions. In addition, we plan to
> dedicate time toward evaluating OpenRefine’s new Wikibase reconciliation
> functionality
> <https://docs.openrefine.org/next/manual/wikibase/overview>on behalf of
> the community.  We will keep you updated on our efforts related to these
> topics.
>
>
> If you are interested, we welcome you to watch the progress of these
> related Phabricator tickets as well:
>
>
>   *
>
>     https://phabricator.wikimedia.org/T287164
>     <https://phabricator.wikimedia.org/T287164>  -- Improve bulk import
>     via API
>
>   *
>
>     https://phabricator.wikimedia.org/T285987
>     <https://phabricator.wikimedia.org/T285987>-- Do not generate full
>     html parser output at the end of Wikibase edit requests
>
>
> Cheers,
>
>
> --
> Mohammed Sadat
> *Community Communications Manager for Wikidata/Wikibase*
>
> Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
> Phone: +49 (0)30 219 158 26-0
> https://wikimedia.de <https://wikimedia.de>
>
> Keep up to date! Current news and exciting stories about Wikimedia,
> Wikipedia and Free Knowledge in our newsletter (in German): Subscribe
> now <https://www.wikimedia.de/newsletter/>.
>
> Imagine a world in which every single human being can freely share in
> the sum of all knowledge. Help us to achieve our vision!
> https://spenden.wikimedia.de <https://spenden.wikimedia.de>
>
> Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
> unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt
> für Körperschaften I Berlin, Steuernummer 27/029/42207.
>
> _______________________________________________
> Wikibaseug mailing list -- wikibaseug@lists.wikimedia.org
> To unsubscribe send an email to wikibaseug-leave@lists.wikimedia.org
>
_______________________________________________
Wikibaseug mailing list -- wikibaseug@lists.wikimedia.org
To unsubscribe send an email to wikibaseug-leave@lists.wikimedia.org