Hello all,


This is an update regarding the progress and dates of migration of wb_terms table replacement solution in Wikidata production environment.


We have successfully put Wikidata production in the stage where property terms are stored in and written to both the old store (wb_terms table) and the new replacement store. Retrieving property terms is still being done using the old store.


The previously announced dates are no longer effective. No changes to tools are needed yet. Tools can continue to read from the old store (wb_terms table) for the moment. There will be a later announcement regarding the date when tools have to switch to reading property terms from the new store.


The next step will be to go to the stage of retrieving property terms from the new store, while we keep storing them in both stores. That step is blocked by a problem we discovered while testing that switch on beta cluster, and we are working on solving it at the moment (https://phabricator.wikimedia.org/T226008).


As for item terms, and in the light of the new information about switching master node to a better host (https://phabricator.wikimedia.org/T227062) that can actually host the migration of them till the end, we have as well decided to push out item terms migration on hold until after that failover is done and is stable.


The migration of all item terms will take weeks to finish, but it isn’t clear yet how long exactly. We will run it in several stages and there will be separate announcements regarding those stages to announce and inform about how to deal with it, in case it affects your work.


You can find more information regarding those dates and how to prepare for them in https://phabricator.wikimedia.org/T221765, and we have dedicated https://phabricator.wikimedia.org/project/view/4014 to receive and help with any questions from tool builders that need to update their tools accordingly.


(email & wiki) In order to keep all discussions in one place, we kindly ask you to react or ask your questions https://phabricator.wikimedia.org/T221764.


Thanks,


On Wed, 15 May 2019 at 15:06, Léa Lacroix <lea.lacroix@wikimedia.de> wrote:

Hello all,

This is an update regarding the dates of test environment and migration of wb_terms table replacement solution.

Due to various complications that the developers in the Wikidata team have been working on solving over the last few weeks, we unfortunately will have to push the dates for when a test environment for tools builders will be ready, which was supposed to be ready today, and the following dates for starting migration of wb_terms data into the new schema in production.

The new dates are:

  • 29th of May: Test environment for tool builders will be ready
  • 12th of June: Property Terms migration starts
  • 19th of June: Read property terms from new schema on Wikidata
  • 26th of June: Item terms migration begins
  • 3rd of July: Read item terms from one of the two schemas (as explained in this task)

You can find more information regarding those dates and how to prepare for them in this task, and we have dedicated a board to receive and help with any questions from tool builders that need to update their tools accordingly.

In order to keep all discussions in one place, we kindly ask you to react or ask your questions on Phabricator.

As a reminder, if you want to discuss with the developers, ask questions and get help in order to update your tools, you can join the IRC Mediawiki meeting, today at 15:00 UTC on #wikimedia-tech.

Thanks,

Léa


On Wed, 24 Apr 2019 at 16:30, Léa Lacroix <lea.lacroix@wikimedia.de> wrote:

Hello all,

This is an important announcement for all the tool builders and maintainers who access Wikidata’s data by querying directly Labs database replicas.

In May-June 2019, the Wikidata development team will drop the wb_terms table from the database in favor of a new optimized schema. Over years, this table has become too big, causing various issues.

This change requires the tools using wb_terms to be updated. Developers and maintainers will need to adapt their code to the new schema before the migration starts and switch to the new code when the migration starts.

The migration will start on May 29th. On May 15th, a test system will be available for you to test your code.

The table being used by plenty of external tools, we are setting up a process to make sure that the change can be done together with the developers and maintainers, without causing issues and broken tools. Most of the documentation and updates will take place on Phabricator:

We are aware that this change will ask you to make some important changes in your code, and we are willing to help you as much as our resources allow us to. We hope that you will understand that this change is made to avoid bigger issues in the near future.

Note that this change is not impacting Wikibase instances outside of Wikidata. A dedicated migration plan and announcement will follow.

We strongly encourage you to not wait until last minute to make the changes in your code. If you have any question or issue, we will be happy to help. In order to keep the discussions in one place, please ask questions or raise issues directly in the Phabricator task and board.

Thanks for your understanding,

Cheers,
--
Léa Lacroix
Project Manager Community Communication for Wikidata

Wikimedia Deutschland e.V.
Tempelhofer Ufer 23-24
10963 Berlin
www.wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.

Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.


--
Léa Lacroix
Project Manager Community Communication for Wikidata

Wikimedia Deutschland e.V.
Tempelhofer Ufer 23-24
10963 Berlin
www.wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.

Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
_______________________________________________
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech


-- 
Alaa Sarhan
Full Stack Developer

Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30 219 158 26-0
https://wikimedia.de

Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de

Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.