Hello all,
This is an update regarding the progress and dates of migration of wb_terms
table replacement solution in Wikidata production environment.
We have successfully put Wikidata production in the stage where property
terms are stored in and written to both the old store (wb_terms table) and
the new replacement store. Retrieving property terms is still being done
using the old store.
*The previously announced dates are no longer effective.* No changes to
tools are needed yet. Tools can continue to read from the old store
(wb_terms table) for the moment. There will be a later announcement
regarding the date when tools have to switch to reading property terms from
the new store.
The next step will be to go to the stage of retrieving property terms from
the new store, while we keep storing them in both stores. That step is
blocked by a problem we discovered while testing that switch on beta
cluster, and we are working on solving it at the moment (
https://phabricator.wikimedia.org/T226008).
As for item terms, and in the light of the new information about switching
master node to a better host (https://phabricator.wikimedia.org/T227062)
that can actually host the migration of them till the end, we have as well
decided to push out item terms migration on hold until after that failover
is done and is stable.
The migration of all item terms will take weeks to finish, but it isn’t
clear yet how long exactly. We will run it in several stages and there will
be separate announcements regarding those stages to announce and inform
about how to deal with it, in case it affects your work.
You can find more information regarding those dates and how to prepare for
them in https://phabricator.wikimedia.org/T221765
<https://phabricator.wikimedia.org/T221765>, and we have dedicated
https://phabricator.wikimedia.org/project/view/4014
<https://phabricator.wikimedia.org/project/view/4014/> to receive and help
with any questions from tool builders that need to update their tools
accordingly.
(email & wiki) In order to keep all discussions in one place, we kindly ask
you to react or ask your questions https://phabricator.wikimedia.org/T221764
.
Thanks,
On Wed, 15 May 2019 at 15:06, Léa Lacroix <lea.lacroix@wikimedia.de> wrote:
> Hello all,
>
> This is an update regarding the dates of test environment and migration of wb_terms
> table replacement solution
> <https://lists.wikimedia.org/pipermail/wikidata/2019-April/012987.html>.
>
> Due to various complications that the developers in the Wikidata team have
> been working on solving over the last few weeks, we unfortunately will have
> to push the dates for when a test environment for tools builders will be
> ready, which was supposed to be ready today, and the following dates for
> starting migration of wb_terms data into the new schema in production.
>
> The new dates are:
>
> - 29th of May: Test environment for tool builders will be ready
> - 12th of June: Property Terms migration starts
> - 19th of June: Read property terms from new schema on Wikidata
> - 26th of June: Item terms migration begins
> - 3rd of July: Read item terms from one of the two schemas (as
> explained in this task <https://phabricator.wikimedia.org/T221765>)
>
> You can find more information regarding those dates and how to prepare for
> them in this task <https://phabricator.wikimedia.org/T221765>, and we
> have dedicated a board
> <https://phabricator.wikimedia.org/project/view/4014> to receive and help
> with any questions from tool builders that need to update their tools
> accordingly.
>
> In order to keep all discussions in one place, we kindly ask you to react
> or ask your questions on Phabricator
> <https://phabricator.wikimedia.org/T221764>.
>
> As a reminder, if you want to discuss with the developers, ask questions
> and get help in order to update your tools, you can join the IRC Mediawiki
> meeting, today at 15:00 UTC on #wikimedia-tech.
>
> Thanks,
>
> Léa
>
> On Wed, 24 Apr 2019 at 16:30, Léa Lacroix <lea.lacroix@wikimedia.de>
> wrote:
>
>> Hello all,
>>
>> This is an important announcement for all the tool builders and
>> maintainers who access Wikidata’s data by *querying directly Labs
>> database replicas*.
>>
>> In May-June 2019, the Wikidata development team will drop the wb_terms
>> table from the database in favor of a new optimized schema. Over years,
>> this table has become too big, causing various issues.
>>
>> This change requires the tools using wb_terms to be updated. Developers
>> and maintainers will need to *adapt their code* to the new schema before
>> the migration starts and switch to the new code when the migration starts.
>>
>> The migration will start on *May 29th*. On May 15th, a test system will
>> be available for you to test your code.
>>
>> The table being used by plenty of external tools, we are setting up a
>> process to make sure that the change can be done together with the
>> developers and maintainers, without causing issues and broken tools. Most
>> of the documentation and updates will take place on Phabricator:
>>
>> - In this Phabricator task <https://phabricator.wikimedia.org/T221764>,
>> you can find a description of the changes and the process, and you can ask
>> for more details or for help in the comments. This is also where updates
>> will be announced if necessary.
>> - On the Tool Builders Migration board
>> <https://phabricator.wikimedia.org/tag/wb_terms_-_tool_builders_migration>
>> you will find all the details about the migration, how to update your
>> tool <https://phabricator.wikimedia.org/T221765>, and you can add
>> your own tasks.
>> - If you need to discuss with the Wikidata developers or get more
>> specific help, we set up two dedicated IRC meetings and a session at the
>> Wikimedia hackathon. More information in this task
>> <https://phabricator.wikimedia.org/T221764>.
>>
>> We are aware that this change will ask you to make some important changes
>> in your code, and we are willing to help you as much as our resources allow
>> us to. We hope that you will understand that this change is made to avoid
>> bigger issues in the near future.
>>
>> Note that this change is not impacting Wikibase instances outside of
>> Wikidata. A dedicated migration plan and announcement will follow.
>>
>> We strongly encourage you to not wait until last minute to make the
>> changes in your code. If you have any question or issue, we will be happy
>> to help. In order to keep the discussions in one place, please ask
>> questions or raise issues directly in the Phabricator task and board.
>>
>> Thanks for your understanding,
>> Cheers,
>> --
>> Léa Lacroix
>> Project Manager Community Communication for Wikidata
>>
>> Wikimedia Deutschland e.V.
>> Tempelhofer Ufer 23-24
>> 10963 Berlin
>> www.wikimedia.de
>>
>> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
>>
>> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
>> unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt
>> für Körperschaften I Berlin, Steuernummer 27/029/42207.
>>
>
>
> --
> Léa Lacroix
> Project Manager Community Communication for Wikidata
>
> Wikimedia Deutschland e.V.
> Tempelhofer Ufer 23-24
> 10963 Berlin
> www.wikimedia.de
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
>
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
> unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt
> für Körperschaften I Berlin, Steuernummer 27/029/42207.
> _______________________________________________
> Wikidata-tech mailing list
> Wikidata-tech@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
>
--
Alaa Sarhan
Full Stack Developer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30 219 158 26-0https://wikimedia.de
Imagine a world in which every single human being can freely share in
the sum of all knowledge. Help us to achieve our
vision!https://spenden.wikimedia.de
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e.
V. Eingetragen im Vereinsregister des Amtsgerichts
Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig
anerkannt durch das Finanzamt für Körperschaften I Berlin,
Steuernummer 27/029/42207.
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l