Wikidata-tech

wikidata-tech@lists.wikimedia.org

2 participants
623 discussions

Re: [Call for Beta Testing] New "mul" term language code on Wikidata
by Mohammed Sadat Abdulai 22 Jul '24

22 Jul '24

*(This [SIGNIFICANT Change Announcement] is relevant for all Wikidata users working with Labels and Aliases.)* Hello, You may recall our previous announcement <https://lists.wikimedia.org/hyperkitty/list/wikidata@lists.wikimedia.org/me…> inviting you to test out the “default values for all languages” feature, introducing the language code "mul" for labels and aliases, on https://test.wikidata.org. I'm pleased to announce that we are now proceeding with a limited release on Wikidata. This feature will be available for testing starting July 29. Note of Caution and Tips for Testing - To try out the new feature during this limited release, you need to add the language code “mul” to your Babel boxes <https://www.wikidata.org/wiki/Wikidata:Userboxes#Babel>. A full release will follow on the week of August 12, displaying default values to everyone by default. - Please refrain from starting any Bot-runs for mul during the limited release. We will use this limited release to test the performance and verify that everything works as planned. Bot-runs for mul should only commence after the full feature is released. - Please also test the help page <https://www.wikidata.org/wiki/Help:Default_values_for_labels_and_aliases> explaining how to use the feature. It will be important to get this help page right, as the full release will initially contain an onboarding element that guides people to this help page. I would like to thank everyone who has participated in the previous testing phases or provided feedback thus far. We are eager to hear more from you during this limited release. Please do not hesitate to reach out with questions or concerns on the talk page of the Help page <https://www.wikidata.org/wiki/Help_talk:Default_values_for_labels_and_alias…>, or for technical details, in this Phabricator ticket (phab:T356169 <https://phabricator.wikimedia.org/T356169>) There has also been some discussion on the help page <https://www.wikidata.org/wiki/Help_talk:Default_values_for_labels_and_alias…> regarding the copy used for the language. We would like to reach a consensus before the full release. Join the discussion to add your thoughts and input on the thread. This week, a survey will be included in the Weekly Summary for those who prefer to provide anonymous feedback. Cheers! -Mohammed On Mon, Feb 12, 2024 at 9:00 AM Mohammed Sadat Abdulai < mohammed.abdulai(a)wikimedia.de> wrote: > (This announcement is relevant for all Wikidata users working with Labels > and Aliases.) > > > Hello, > > We're excited to invite you to participate in the community testing of the > “default values for labels and aliases” feature. Based on a long-standing > community request around language fallback (T285156 > <https://phabricator.wikimedia.org/T285156>), and based on your feedback > from our previous announcement > <https://lists.wikimedia.org/hyperkitty/list/wikidata@lists.wikimedia.org/me…>, > we are currently testing this feature on https://test.wikidata.org > <https://test.wikidata.org/wiki/Q42> with the release planned for Q1 2024. > > What is coming? > > 1. > > Default values for labels and aliases (mul): Items that repeat the > same label and aliases over and over are hard to maintain and can strain > our infrastructure, especially the Wikidata Query Service. As a solution, > we are introducing the option for default values in the termbox. The > default values are considered in the language fallback chain (language code > “mul”). > > > > 2. > > Visualisation of the language fallback chain: Previously, Wikidata’s > fallback chain was not visible in the user interface, making it challenging > to determine if additional information was needed. As a first step toward > improvement, we will now clearly show what content is already available via > language fallback in the placeholders for labels (including default values). > > > We need your feedback > > Please leave your feedback on Help talk:Default values for labels and > aliases > <https://www.wikidata.org/w/index.php?title=Help_talk:Default_values_for_lab…>. > Your feedback will play a crucial role in verifying if the current version > is ready for Wikidata. Additionally, you can find some test Items, known > limitations and open questions there for you to review. We will analyze > your feedback one week after today's announcement. > > The next steps > > After you have left your feedback, please make sure that the guidelines > on Help:Default values for labels and aliases > <https://www.wikidata.org/wiki/Help:Default_values_for_labels_and_aliases> > are ready for the launch. You can also already plan bot-runs for mul, but > please be sure to start with uncontroversial cases. If there are no > blockers, we will then prepare a limited release on Wikidata in the > coming weeks. The limited release will only be visible if you put mul in > your Babel boxes. We will use this limited release to test the performance > and verify that everything works as planned. Please don’t start any > bot-runs for mul during the limited release. A full release will follow a > few weeks later, displaying default values for everyone by default, and > featuring an onboarding element that will guide people to the help page. > Bot-runs for mul can commence at this stage. > > Do not hesitate to reach out with questions or concerns in the talk page > of the help page > <https://www.wikidata.org/wiki/Help_talk:Default_values_for_labels_and_alias…> > and for technical details in this Phabricator ticket (T356169 > <https://phabricator.wikimedia.org/T356169>). > > > Thank you for your support! > > Cheers, > -- > Mohammed Sadat Abdulai > *Community Communications Manager, Wikidata* > > Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin > Phone: +49 (0) 30 577 116 2466 > https://wikimedia.de > > Grab a spot in my calendar for a chat: calendly.com/masssly. > > A lot is happening around Wikidata - Keep up to date! > <https://www.wikidata.org/wiki/Wikidata:Status_updates> Current news and > exciting stories about Wikimedia, Wikipedia and Free Knowledge in our > newsletter (in German): Subscribe now > <https://www.wikimedia.de/newsletter/>. > > Imagine a world in which every single human being can freely share in the > sum of all knowledge. Help us to achieve our vision! > https://spenden.wikimedia.de > > Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V. > Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B. > Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, > Steuernummer 27/029/42207. Geschäftsführende Vorstände: Franziska Heine, > Dr. Christian Humborg >

1 0

Wikibase: Releasing Wikibase Suite Deploy 1.0.0, 2.0.0 and 3.0.0
by Mónica Pinedo 15 Jul '24

15 Jul '24

Hello, We would like to announce a major update to the Wikibase Suite product offering: Wikibase Suite Deploy <https://github.com/wmde/wikibase-release-pipeline/blob/main/deploy/README.md>. It’s a total overhaul of what used to be the Wikibase Suite “example” directory, transforming it into a complete deployment kit for Wikibase production environments. Highlights: 1. It significantly improves the separation of config, code, and data. Making configuration for deployment clearer, easier, and more robust across updates. Now, all the config lives in a single “config” directory; all data is located in docker volumes; and container images containing the code can be updated individually, without touching config or data. We also consolidated our docker compose files and restructured them for better legibility. 2. We did a total overhaul of the deployment documentation <https://github.com/wmde/wikibase-release-pipeline/tree/main/deploy> along with individual docs for each of our containers. 3. We included a Traefik <https://doc.traefik.io/traefik/> reverse proxy with SSL termination and ACME support; allowing for, among other benefits, QuickStatements Authentication and https certification out of the box. 4. Simplifying our images, we combined the “wikibase” and “wikibase-bundle” images into a single image called “wikibase”. Please note that this means “wikibase-bundle” is now deprecated; however, all of its original functionality remains intact in “wikibase”. 5. A new versioning scheme. Each Wikibase Suite Image is now separately versioned as is the Wikibase Suite Deploy product. The new versioning scheme follows the rules of Semantic Versioning <https://semver.org>, which you’re likely familiar with already. The releases of Wikibase Suite Deploy we’re making today carry a major version for each Mediawiki version we support: - Wikibase Suite Deploy 1.0.0 <https://github.com/wmde/wikibase-release-pipeline/tree/deploy-1/deploy> ships Wikibase Image 1.0.0 based on Mediawiki 1.39 - Wikibase Suite Deploy 2.0.0 <https://github.com/wmde/wikibase-release-pipeline/tree/deploy-2/deploy> ships Wikibase Image 2.0.0 based on Mediawiki 1.41 - Wikibase Suite Deploy 3.0.0 <https://github.com/wmde/wikibase-release-pipeline/tree/deploy-3/deploy> ships Wikibase Image 3.0.0 based on Mediawiki 1.42 - All other services such as WDQS and Quickstatements shipped with all Wikibase Suite Deploy versions start on image version 1.0.0 today. Subsequent releases will be shipped as minor or patch updates to Wikibase Suite Deploy and the Wikibase Suite Images respectively. The upcoming Mediawiki 1.43 will be released as a new Wikibase Suite Deploy major version. Read more about versioning and updating in the Wikibase Suite Deploy documentation <https://github.com/wmde/wikibase-release-pipeline/tree/main/deploy>. All Wikibase Suite Image Releases are available on the Wikibase Docker Hub repository <https://hub.docker.com/repositories/wikibase>, from now on tagged with their semantic version. Here are the MediaWiki release notes <https://www.mediawiki.org/wiki/Release_notes> for more info. I want to say a huge thank you to you in the community who have partnered with us along the way trying the software out in production, offering us feedback, answering our and each other’s questions, sharing your needs, and submitting tickets on phabricator. I also want to say a huge thank you to our user researcher and wonderful team of engineers who made realising these improvements possible. We’re excited to continue to work with you to improve Wikibase Suite. Let us know how migrating to deploy works out for you and let us know if you need any help by creating a ticket on Phabricator <https://phabricator.wikimedia.org/maniphest/task/edit/form/129/>. Warm Regards, The Wikibase Suite team Mónica Pinedo Bajo (she/her) Engineering Manager Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0https://wikimedia.de Keep up to date! Current news and exciting stories about Wikimedia, Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now <https://www.wikimedia.de/newsletter/> Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision!https://spenden.wikimedia.de Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

dashes - in JSON keys - suck
by Thad Guidry 20 Jun '24

20 Jun '24

"mainsnak": { "snaktype": "value", "property": "P31", "hash": "d679b35e7a9523b92903b52661a9e4c15d8d76d3", "datavalue": { "value": { "entity-type": "item", "numeric-id": 112826905, "id": "Q112826905" }, "type": "wikibase-entityid" }, "datatype": "wikibase-item" }, One of the constant pain points that I have heard and run across is this: Any chance Wikidata's JSON format might be a little more developer friendly > for tooling in general? Like why do they sometimes have "entity-type" and > "numeric-id" as keys with dashes, when a lot of the world and our parsing > libraries have sorta standardized around at least only camelCase or > snake_case in JSON. I mean, look at what they did with "snaktype" and > "entity-type" ! Can you find out, Thad ? So I said to them, sure. 😉 (granted I know all characters are allowed for object property names in JSON and someone likely in the team at the time said, "eh, doesn't matter, it's JSON, it eats anything".) Thad https://www.linkedin.com/in/thadguidry/ https://calendly.com/thadguidry/

1 0

[BREAKING Change Announcement] Upcoming Changes to Wikibase: new EntitySchema data type
by Lydia Pintscher 18 Jun '24

18 Jun '24

Hi everyone, This is a breaking change announcement relevant for some Wikibase API users. What is Changing? On 2 July 2024, we will enable the EntitySchema data type on Wikidata, and we expect the community to create the first properties with that data type and start using them in statements shortly afterwards. This means that there will be values with the data value type wikibase-entityid containing an EntitySchema ID (with "entity-type": "entity-schema" in the JSON value). However, EntitySchema is not yet a full-featured Wikibase entity type; it is our intention to eventually make it work like other entity types (Item, Property, etc.), but at the moment, using EntitySchema in e.g. the wbgetentities API or Special:EntityData does not work. Who is Affected? Code which assumes that IDs found in wikibase-entityid data values can be used with other Wikibase entity APIs may need adjusting. Other Wikibases instances besides Wikidata are not affected unless they also use the EntitySchema extension and set $wgEntitySchemaEnableDatatype = true. What You Need to Do If your code works with statements of arbitrary data types, and looks up entities referenced in values with the data value type wikibase-entityid, you probably want to check that the entity-type is not entity-schema before proceeding. (Note that, if your goal is to display the value, you can instead use the wbformatvalue API for any data type.) If you use the wbformatvalue API, you should make sure that you also specify the datatype or property parameter (depending on which information you have available; note that specifying both parameters at once is an error). Without this information, not all values can be formatted correctly; in particular, trying to format an EntitySchema value without specifying the datatype or property will result in an error (“An illegal set of parameters have been used”). (Specifying the datatype or property parameter for wbformatvalue has always been advisable, and necessary for some other data value types, but this is the first time it becomes relevant for wikibase-entityid data values.) As previously announced <https://lists.wikimedia.org/hyperkitty/list/wikidata@lists.wikimedia.org/th…>, the new data type is already enabled on Test Wikidata, so you can try out the behaviour there. (An example item on Test Wikidata with a statement linking to an EntitySchema is human <https://test.wikidata.org/wiki/Q497>.) If you have any questions or concerns about this change, please don’t hesitate to reach out to us in this ticket (T332157 <https://phabricator.wikimedia.org/T332157>). Cheers, Lydia -- Lydia Pintscher - https://lydiapintscher.de - WD:Q18016466 <https://www.wikidata.org/wiki/Q18016466> Portfolio Lead for Wikidata Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30-577 11 62-0 https://wikimedia.de Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. Geschäftsführende Vorstände: Franziska Heine, Dr. Christian Humborg

1 0

(DEC 2024) Huawei H13-811_V3.0 Dumps [Questions] Simple Strategy for Achievement
by xutehosaulle＠gmail.com 15 Apr '24

15 Apr '24

JustCerts offers three formats of actual and newest H13-811_V3.0 Exam Questions to help students customize their HCIA Cloud Service H13-811_V3.0 study experience. These real Huawei H13-811_V3.0 Dumps (Questions) are available with up to 90 days of free updates that remove post-purchase anxiety. Buy today and get free H13-811_V3.0 Dumps (Questions) updates.

1 0

Timestamps with calendarmodel other than Q1985727 and Q1985786
by Mitar 24 Mar '24

24 Mar '24

Hi! Recently, a timestamp with calendarmodel https://www.wikidata.org/wiki/Q12138 has been introduced into Wikidata: https://www.wikidata.org/w/index.php?title=Q105958428&oldid=2004936527 How is this possible? I thought that the only allowed values are Q1985727 and Q1985786? Mitar -- https://mitar.tnode.com/ https://twitter.com/mitar_m https://noc.social/@mitar

1 1

From JSON to RDF dumps
by daniel＠degu.cl 25 Feb '24

25 Feb '24

Hi all, I wonder if the Wikidata RDF dump (in https://dumps.wikimedia.org/wikidatawiki/entities/) is generated directly from the JSON dump. If this is the case, what tool used? Best Regards, Daniel

4 3

[Significant Change Announcement] Upcoming Changes to Wikibase API Modules: Addition of New Parameters
by Mohammed Sadat Abdulai 19 Feb '24

19 Feb '24

*(apologies for cross-posting)* Hello, This is a significant change announcement relevant for some Wikibase API users. What's Changing? Over the coming weeks, most Wikibase API modules will gain three new parameters: returnto, returntoquery, and returntoanchor. Additionally, these API modules may return tempusercreated and tempuserredirect fields in their responses. These parameters are being added to support user interaction flows related to IP Masking <https://meta.wikimedia.org/wiki/IP_Editing:_Privacy_Enhancement_and_Abuse_M…> . Who’s Affected? Only users who use the API to make anonymous edits are affected. If you don’t use the API to make edits, or if you only make authenticated edits, you can safely ignore these changes, and disregard the additional API parameters and response data. What You Need to Do If you use the API to make anonymous edits, the response may contain a tempusercreated string indicating the name of the temporary account that was created, and a tempuserredirect nullable string. If the tempuserredirect is not null, you should redirect the user to that URL to complete the setup for the temporary account. Afterwards, the user will automatically be returned to the current wiki; you can use the returnto, returntoquery, and/or returntoanchor parameters in your original API request to control the title, query, and anchor where the redirect returns to. If you have any questions or concerns about this change, please don’t hesitate to reach out to us in this ticket (T357024 <https://phabricator.wikimedia.org/T357024>). Cheers, -- Mohammed S. Abdulai *Community Communications Manager, Wikidata* Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0) 30 577 116 2466 https://wikimedia.de Grab a spot in my calendar for a chat: calendly.com/masssly. A lot is happening around Wikidata - Keep up to date! <https://www.wikidata.org/wiki/Wikidata:Status_updates> Current news and exciting stories about Wikimedia, Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now <https://www.wikimedia.de/newsletter/> . Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. Geschäftsführende Vorstände: Franziska Heine, Dr. Christian Humborg

1 0

Wikibase/Wikidata Database Technologies and Strategies
by Elton Figueiredo de Souza Soares 02 Feb '24

02 Feb '24

Dear Wikibase/Wikidata Community, We are trying to understand which database technologies and strategies Wikibase/Wikidata uses for storing, updating, and querying the data (knowledge) it manipulates. By looking at the documentation<https://wmde.github.io/wikidata-wikibase-architecture/assets/img/03-dataflo…> we understood that RDF is only used for the Wikidata Query Service, but we could not find out exactly how Wikibase/Wikidata stores the information that is translated to RDF during the data dump. More specifically, we understood that a MySQL (or is it MariaDB?) relational database is used as the key persistence component for most of Wikibase/Wikidata services and that the information that is maintained in this database is periodically exported to multiple formats, including RDF. In addition, looking at the relational database schema published in the documentation<https://www.mediawiki.org/wiki/Manual:Database_layout> we could not locate tables that are easily mappable to the Wikibase Data Model<https://www.mediawiki.org/wiki/Wikibase/DataModel>. Thus, we hypothesize that there is some software component (Wikibase Common Data Access?) that dynamically translates the data contained in those tables to Statements, Entities, etc. Is that hypothesis, correct? If yes, does this software component uses any intermediate storage mechanism for caching those Statements, Entities, ...? Or are those translations always performed at runtime on-the-fly (be it for querying, adding, or updating Statements, Entities, …)? Finally, we would like to understand more about how Wikidata REST API<https://www.wikidata.org/wiki/Wikidata:REST_API> is implemented: • In which database are the statements that added/retrieved through it stored? Is it being stored in the central MySQL database or in another database? • Does it have any support for pagination of statements? For example, if an item has many statements associated with a property, does the API assumes that both the underlying database and the network will support the retrieval of all those statements? • Are you currently considering implementing the support for more flexible querying of statements, or such requirement has been fully delegated to the Wikidata Query Service? If there is an updated documentation that could help us answer those questions, could you kindly point us to it? Otherwise, would you be able to share this information with us? Best Regards, Elton F. de S. Soares Advisory Software Engineer Rio de Janeiro, RJ, Brazil IBM Research E-mail: eltons(a)ibm.com<mailto:eltons@ibm.com>

2 2

Wikidata Database Technologies/Techniques
by Elton Figueiredo de Souza Soares 01 Feb '24

01 Feb '24

Dear Wikidata Community, We are trying to understand which database technologies and strategies Wikidata uses for storing, updating, and querying the data (knowledge) it manipulates. By looking at the documentation<https://wmde.github.io/wikidata-wikibase-architecture/assets/img/03-dataflo…> we understood that RDF is only used for the Wikidata Query Service, but we could not find out exactly how Wikidata stores the information that is translated to RDF during the data dump. More specifically, we understood that a MySQL (or is it MariaDB?) relational database is used as the key persistence component for most of Wikidata services and that the information that is maintained in this database is periodically exported to multiple formats, including RDF. In addition, looking at the relational database schema published in the documentation<https://www.mediawiki.org/wiki/Manual:Database_layout> we could not locate tables that are easily mappable to the Wikibase Data Model<https://www.mediawiki.org/wiki/Wikibase/DataModel>. Thus, we hypothesize that there is some software component (Wikibase Common Data Access?) that dynamically translates the data contained in those tables to Statements, Entities, etc. Is that hypothesis, correct? If yes, does this software component uses any intermediate storage mechanism for caching those Statements, Entities, ...? Or are those translations always performed at runtime on-the-fly (be it for querying, adding, or updating Statements, Entities, …)? Finally, we would like to understand more about how Wikidata REST API<https://www.wikidata.org/wiki/Wikidata:REST_API> is implemented: * In which database are the statements that added/retrieved through it stored? Is it being stored in the central MySQL database or in another database? * Does it have any support for pagination of statements? For example, if an item has many statements associated with a property, does the API assumes that both the underlying database and the network will support the retrieval of all those statements? * Are you currently considering implementing the support for more flexible querying of statements, or such requirement has been fully delegated to the Wikidata Query Service? If there is an updated documentation that could help us answer those questions, could you kindly point us to it? Otherwise, would you be able to share this information with us? Best Regards, Elton F. de S. Soares Advisory Software Engineer Rio de Janeiro, RJ, Brazil IBM Research E-mail: eltons(a)ibm.com<mailto:eltons@ibm.com>

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Wikidata-tech