Wikidata-tech January 2020

wikidata-tech@lists.wikimedia.org

3 participants
3 discussions

nomunofu, WDQS, and future of wikidata
by Amirouche Boubekki 25 Jan '20

25 Jan '20

Hello, I would like to know if you are interested by the proposal I made at: https://meta.wikimedia.org/wiki/Grants:Project/WDQS_On_FoundationDB Like I said previously on wikidata mailing list, I can address the following problems related to WDQS: In an ideal world, WDQS should: * scale in terms of data size * scale in terms of number of edits * have low update latency * expose a SPARQL endpoint for queries * allow anyone to run any queries on the public WDQS endpoint * provide great query performance * provide a high level of availability ref: https://lists.wikimedia.org/pipermail/wikidata/2019-June/013124.html The other proposal I made is about replacing both wikibase and blazegraph: https://meta.wikimedia.org/wiki/Grants:Project/Iamamz3/Prototype_A_Scalable… nomunofu is a working prototype I made to micro benchmark (again) GNU Guile: https://github.com/amirouche/nomunofu What do you think?

1 1

Wikibase installations: new term store enabled
by Léa Lacroix 23 Jan '20

23 Jan '20

Hello all, This email is important for you if you are running an instance of Wikibase. As part of deprecating wb_terms table[1] for the whole Wikibase codebase, the new term store is now wired to the Wikibase default setup and per the default it's going to read from the new term store. It means that next time you update your Wikibase (to master or 1.35 once it’s out), you should run update.php [2] so it creates the new tables and migrate your data to the new term store. This might take some time if you have a large installation. If you’re not running the script, you might see errors appear. The default value until the next mediawiki major release (1.35) is going to stay "read new, write both", meaning you will have both old and new data side by side but it reads from the new one. By the next major release after 1.35, we are going to fully drop wb_terms table from Wikibase. Please note that there’s an exception related to search: if you are not using ElasticSearch, the search still uses the old term store (wb_terms table). That’s why the default is to write to both term stores for now until the search works for the new term store properly. If you have any questions or issues, feel free to write me. Cheers, Léa [1]: https://phabricator.wikimedia.org/T208425 [2]: https://github.com/wikimedia/mediawiki/blob/master/maintenance/update.php -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

how to load a dump of wikidata into a local wikibase
by Miquel Àngel Farré 16 Jan '20

16 Jan '20

Hello, We are having issues launching a local copy of wikidata, when we use the 'importDump.php' tool, below the issues that we are facing. If somebody has an idea of how we could solve this, please let me know. We are also considering professional services to get fixes for this being released in case somebody is professionally consulting around wikibase. Thanks, Miquel Here the issues: if I try to load the full dump, the error I get is: root@4fc8cc9b76b3:/var/www/html/maintenance# php importDump.php --conf ../LocalSettings.php ../images/wikidatawiki-20191101-pages-articles-multistream.xml.bz2 Warning: XMLReader::read(): uploadsource://d0cd78c216b067ffdd60946c258db6a7:45: parser error : Extra content at the end of the document in /var/www/html/includes/import/WikiImporter.php on line 646 Warning: XMLReader::read(): </siteinfo> in /var/www/html/includes/import/WikiImporter.php on line 646 Warning: XMLReader::read(): ^ in /var/www/html/includes/import/WikiImporter.php on line 646 Done! You might want to run rebuildrecentchanges.php to regenerate RecentChanges, If I try to load a partial dump, the warnings that I get (which I think those mean nothing is loading) are: root@4fc8cc9b76b3:/var/www/html/maintenance# php importDump.php --conf ../LocalSettings.php ../images/wikidatawiki-20191020-pages-meta-current1.xml-p1p235321.bz2 Revision 1033865598 using content model wikibase-item cannot be stored on "Q15" on this wiki, since that model is not supported on that page. Revision 1034542603 using content model wikibase-item cannot be stored on "Q17" on this wiki, since that model is not supported on that page. Revision 1032554298 using content model wikibase-item cannot be stored on "Q18" on this wiki, since that model is not supported on that page. Revision 1032534215 using content model wikibase-item cannot be stored on "Q20" on this wiki, since that model is not supported on that page. Revision 1026713626 using content model wikibase-item cannot be stored on "Q21" on this wiki, since that model is not supported on that page. Revision 1023703278 using content model wikibase-item cannot be stored on "Q22" on this wiki, since that model is not supported on that page. Revision 1032815802 using content model wikibase-item cannot be stored on "Q25" on this wiki, since that model is not supported on that page. Revision 1032910600 using content model wikibase-item cannot be stored on "Q26" on this wiki, since that model is not supported on that page.

4 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Wikidata-tech January 2020