Wikidata-tech April 2019

wikidata-tech@lists.wikimedia.org

7 participants
11 discussions

JSON-LD now on beta
by Léa Lacroix 08 Jul '19

08 Jul '19

Hello all, We’re planning to add JSON-LD as a serialization format for Wikidata. This will allow for example an easier access to RDF data from Javascript. This is now deployed on https://wikidata.beta.wmflabs.org. Example: https://wikidata.beta.wmflabs.org/wiki/Special:EntityData/Q64.jsonld As with the other formats we already support (like turtle or rdf/xml), content negotiation is used if the format is not indicated by a suffix like .jsonld. The MIME type that can be used in the Accept header to request JSON-LD output is application/ld+json. If you’re interested in this feature, please test it, and let us know if you find any issues. The related ticket is this one <https://phabricator.wikimedia.org/T207168>. If everything goes as planned, we will enable it to wikidata.org on October 31st. Cheers, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

3 3

[Breaking change] Important for Wikidata tools maintainers: wb_terms table to be dropped at the end of May
by Léa Lacroix 05 Jul '19

05 Jul '19

Hello all, This is an important announcement for all the tool builders and maintainers who access Wikidata’s data by *querying directly Labs database replicas*. In May-June 2019, the Wikidata development team will drop the wb_terms table from the database in favor of a new optimized schema. Over years, this table has become too big, causing various issues. This change requires the tools using wb_terms to be updated. Developers and maintainers will need to *adapt their code* to the new schema before the migration starts and switch to the new code when the migration starts. The migration will start on *May 29th*. On May 15th, a test system will be available for you to test your code. The table being used by plenty of external tools, we are setting up a process to make sure that the change can be done together with the developers and maintainers, without causing issues and broken tools. Most of the documentation and updates will take place on Phabricator: - In this Phabricator task <https://phabricator.wikimedia.org/T221764>, you can find a description of the changes and the process, and you can ask for more details or for help in the comments. This is also where updates will be announced if necessary. - On the Tool Builders Migration board <https://phabricator.wikimedia.org/tag/wb_terms_-_tool_builders_migration> you will find all the details about the migration, how to update your tool <https://phabricator.wikimedia.org/T221765>, and you can add your own tasks. - If you need to discuss with the Wikidata developers or get more specific help, we set up two dedicated IRC meetings and a session at the Wikimedia hackathon. More information in this task <https://phabricator.wikimedia.org/T221764>. We are aware that this change will ask you to make some important changes in your code, and we are willing to help you as much as our resources allow us to. We hope that you will understand that this change is made to avoid bigger issues in the near future. Note that this change is not impacting Wikibase instances outside of Wikidata. A dedicated migration plan and announcement will follow. We strongly encourage you to not wait until last minute to make the changes in your code. If you have any question or issue, we will be happy to help. In order to keep the discussions in one place, please ask questions or raise issues directly in the Phabricator task and board. Thanks for your understanding, Cheers, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

3 4

wb_terms redesign
by data_querier 04 May '19

04 May '19

This is really a defective redesign. It reintroduced numeric IDs to be removed by T114902. See also T179928. We should reconsider reintroduce a new table to link unperfixed and perfixed entity ID. Also oppose any "partial migration for first XXX items" process in T221765: this makes queries much more complicated. Please first fill all data in the new schema, then discontinue the old table.

3 2

[Breaking change] Empty containers in JSON outputs will be serialized as empty object "{}"
by Léa Lacroix 30 Apr '19

30 Apr '19

Hello all, This change is relevant for everyone *using APIs, JSON output and dumps in their tools or gadgets*. Currently, when an object is empty (for example descriptions and aliases), it is rendered as an empty array [] in JSON. (Example <https://www.wikidata.org/wiki/Special:EntityData/Q24075199.json>) We want to serialized it as an empty object {} instead. This change will ease the deserialization process and bring more consistency in our code as some other places already use JSON objects instead of arrays. The impact of this change will be in JSON outputs (Special:EntityData) and JSON dumps, as well as the output of wbgetentities, wbgetclaims and editing APIs. If you’re maintaining tools that use Special:EntityData, you may want to check your code to make sure that it reflects this change, e.g. items with no labels, descriptions, aliases or sitelinks are properly deserialized by your tool. You can already *test your code* against our test system on beta.wikidata.org, for example on this item <https://wikidata.beta.wmflabs.org/wiki/Special:EntityData/Q521829.json>. According to our stable interface policy <https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy>, the change will be enabled four weeks after this announcement, on April 30th. In the meantime, if you have any issue or question, feel free to leave a comment in this ticket <https://phabricator.wikimedia.org/T138104>. Thanks for your attention, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 1

Wednesday: Technical Advice IRC Meeting
by Michael Schönitzer 30 Apr '19

30 Apr '19

Reminder: Technical Advice IRC meeting this week **Wednesday 3-4 pm UTC** on #wikimedia-tech. Question can be asked in English & Hebrew! The Technical Advice IRC Meeting is a weekly support event for volunteer developers. Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting Hope to see you there! Michi (for the Technical Advice IRC Meeting crew) -- Michael F. Schönitzer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 https://wikimedia.de Unsere Vision ist eine Welt, in der alle Menschen am Wissens der Menschheit teilhaben, es nutzen und mehren können. Helfen Sie uns dabei! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

WikidataCon application, program submission and scholarship processes are now open
by Léa Lacroix 17 Apr '19

17 Apr '19

Hello all, As you may know, the *WikidataCon 2019 <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019>* will take place on October 25th-26th 2019 in Berlin. The conference will host 250 people from the Wikidata community, but also the emerging Wikibase community, as well as partners, organizations and companies the organizations who may be interested in *using Wikidata and Wikibase*, or contributing in various ways to the evolution of Wikidata. The event will be focused on networking and strategic discussions around Wikidata and Wikibase, with a special focus on *languages and Wikidata*. Because the number of seats is limited and in order to make sure that the access to the conference is fair, we decided to set up a selection process. People who wish to attend the conference can now apply by filling in the application form <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019/Attend/Apply>. A committee equally composed of volunteers and staff will evaluate the applications against criteria that are already published onwiki <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019/Attend>. This application form aims to gather all of the useful information at once, that’s why it also contains the program submissions (limited to 3 per person) and the scholarship application if needed. You can find more information about the content of the form on this page <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019/Attend/Apply>. Part of the attendees will be also invited directly by the organizing team, based on the connections we want to build with organizations during this conference, and the input they can bring to Wikidata’s strategy. The deadline to fill out the application form is April 26th. No application can be considered after this date. Over the following month, the committee will evaluate the applications, and the applicants will receive an answer latest on June 12th. Here are a few tips to help you prepare your application: - Read the description of the conference <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019> and the page describing the process and the criteria <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019/Attend> carefully. - Plan 20 to 30 minutes to fill out the form. - Don’t do it at the last minute :) it will be less stressful for you, as well as for the committee. - Prepare all your program submissions, as they will be requested in the same form (no submission can be done later). On this page <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019/Program> you can check the formats and questions. - Read the Friendly Space Policy <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019/Attend/Policy> that will apply to the entire event. - Look at the visa information <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019/Attend/Visa> to check if you need a visa to Germany and what actions you need to take. You can find a lot of information regarding the conference on Wikidata:WikidataCon 2019 <https://www.wikidata.org/wiki/Wikidata:WikidataCon_2019>. If you have any question or issue, feel free to share it on the talk page onwiki or to reach the organizing team at info(a)wikidatacon.org Finally, feel free to share this message with your local community, networks, or projects you’re working in! We want to make sure that everyone who’s involved in Wikidata or Wikibase has a chance to apply. We're looking forward to receiving your applications! Thanks for your attention, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 1

[Breaking change] "wbcheckconstraints" API new result status
by Léa Lacroix 17 Apr '19

17 Apr '19

Hello all, This is an announcement for a breaking change to the output format of the WikibaseQualityConstraints constraint checking API, to go live on 6 May 2019 (most likely around 12:00 UTC). It affects all clients that use the *wbcheckconstraints* API action. We are adding a new status for constraints <https://lists.wikimedia.org/pipermail/wikidata/2019-April/012910.html>, in addition to regular constraints and mandatory constraints: suggestion constraints indicate possible improvements to a statement, but are not inherently problematic like other constraint violations. This implies a new status for constraint results as well: in addition to 'violation' for violations of mandatory constraints and 'warning' for violations of regular constraints, as well as several statuses that are not violations, there is now 'suggestion' for violations of suggestion constraints. The default value of the status API parameter is changed to include this status as well (from violation|warning|bad-parameters to violation|warning|suggestion|bad-parameters), and it can appear as the "status" of a result in the response. API consumers that are not interested in suggestion constraints can specify a non-default value for the status API parameter, e. g. the old default violation|warning|bad-parameters, to avoid getting responses including this status. Others should decide how to handle it, and update their code accordingly if necessary. According to our stable interface policy <https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy>, the change will be enabled 4 weeks after this announcement, on May 6th, and a test system will be set up latest on April 17th on test.wikidata.org. If you have any question or issue, let us know in the related ticket <https://phabricator.wikimedia.org/T204439>. Cheers, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 1

Public Domain Ontology for Portuguese
by Houcemeddine A. Turki 13 Apr '19

13 Apr '19

Dear all, I thank you for your efforts. As Dr. David Abian asked me if there are public domain WordNets, I found that there is a public domain ontology for Portuguese. It is available in http://ontopt.dei.uc.pt. Yours Sincerely, Houcemeddine Turki (he/him) Medical Student, Faculty of Medicine of Sfax, University of Sfax, Tunisia Undergraduate Researcher, UR12SP36 GLAM and Education Coordinator, Wikimedia TN User Group Member, WikiResearch Tunisia Member, Wiki Project Med Member, WikiIndaba Steering Committee Member, Wikimedia and Library User Group Steering Committee Co-Founder, WikiLingua Maghreb Founder, TunSci ____________________ +21629499418

1 0

Integrating WordNet in Wikidata Lexicographical Data
by Houcemeddine A. Turki 13 Apr '19

13 Apr '19

Dear all, I thank you for your efforts. I saw with a lot of interest the work that has been done to ameliorate the quality and the quality of the Lexicographical Data. However, I know that most of the added data already exists in WordNets. It seems that we are just reinventing the wheel. I ask why we cannot have an automated method to integrate WordNets in Wikidata. Concerning the copyright issues, they are solved by putting the WordNets as references of the added statements. Yours Sincerely, Houcemeddine Turki (he/him) Medical Student, Faculty of Medicine of Sfax, University of Sfax, Tunisia Undergraduate Researcher, UR12SP36 GLAM and Education Coordinator, Wikimedia TN User Group Member, WikiResearch Tunisia Member, Wiki Project Med Member, WikiIndaba Steering Committee Member, Wikimedia and Library User Group Steering Committee Co-Founder, WikiLingua Maghreb Founder, TunSci ____________________ +21629499418

1 0

Helping request to have an own copy of WikiBase
by Ke Xu 08 Apr '19

08 Apr '19

Hello developers at WikiData, This is Ke from Information Sciences Institute of USC. We find WikiData really easy to use, and it may be a great help to our research. Thus we want to keep a whole copy of your services locally. But here’re some problems. Hope you can address these for us. First, we need to upload our own data, so that we need the whole services from you. How can we download and implement the whole bundle including GUI, Triple Store, WikiBase and ES as well. Second, we found you provided an API (https://github.com/filbertkm/WikibaseImport) to load data from Wikidata.org<http://Wikidata.org>. But it seems the script queries one item once and upload it into our WikiBase. It’s really time consuming. So that do you provide any method to load the dumps and upload them into our WikiBase (not Triple Store)? Also, we want our database auto updated with yours. Thank you for reading this letter. Hope to hear from you. Best Regards, Ke Xu

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Wikidata-tech April 2019