Hey folks,
we plan to drop the wb_entity_per_page table sometime soon[0], because it is just not required (as we will likely always have a programmatic mapping from entity id to page title) and it does not supported non -numeric entity ids as it is now. Due to this removing it is a blocker for the commons metadata.
Is anybody using that for their tools (on tool labs)? If so, please tell us so that we can give you instructions and a longer grace period to update your scripts.
Cheers,
Marius
Well, to answer your question if removing the one table connecting pages and items will break Wikidata-based tools:
Yes, just about all of them.
On Fri, Aug 7, 2015 at 6:56 PM hoo hoo@online.de wrote:
Hey folks,
we plan to drop the wb_entity_per_page table sometime soon[0], because it is just not required (as we will likely always have a programmatic mapping from entity id to page title) and it does not supported non -numeric entity ids as it is now. Due to this removing it is a blocker for the commons metadata.
Is anybody using that for their tools (on tool labs)? If so, please tell us so that we can give you instructions and a longer grace period to update your scripts.
Cheers,
Marius
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hi Marius,
hoo schreef op 7-8-2015 om 19:55:
Hey folks,
we plan to drop the wb_entity_per_page table sometime soon[0], because it is just not required (as we will likely always have a programmatic mapping from entity id to page title) and it does not supported non -numeric entity ids as it is now.
In the past I was alway told to use the wb_entity_per_page table instead of doing page_title=CONCAT('Q', <id>). The Wikibase code used to contain warnings not to make this assumption. I don't know, they might still be there.
Due to this removing it is a blocker for the commons metadata.
That's unfortunate.
Is anybody using that for their tools (on tool labs)? If so, please tell us so that we can give you instructions and a longer grace period to update your scripts.
Of the 117 Wikidata related sql queries that seem to be in my homedir, 48 of them use this table. Basically any Wikidata related tool that uses the sql database will break. What do you propose? That we start messing around with CONCAT()s in our SQL queries? Besides the hours of wasted volunteer time, that's probably a lot slower.
Maarten
Hi Maarten
Am 09.08.2015 um 17:28 schrieb Maarten Dammers:
Hi Marius,
hoo schreef op 7-8-2015 om 19:55:
Hey folks,
we plan to drop the wb_entity_per_page table sometime soon[0], because it is just not required (as we will likely always have a programmatic mapping from entity id to page title) and it does not supported non -numeric entity ids as it is now.
In the past I was alway told to use the wb_entity_per_page table instead of doing page_title=CONCAT('Q', <id>). The Wikibase code used to contain warnings not to make this assumption. I don't know, they might still be there.
Due to this removing it is a blocker for the commons metadata.
That's unfortunate.
Is anybody using that for their tools (on tool labs)? If so, please tell us so that we can give you instructions and a longer grace period to update your scripts.
Of the 117 Wikidata related sql queries that seem to be in my homedir, 48 of them use this table. Basically any Wikidata related tool that uses the sql database will break. What do you propose? That we start messing around with CONCAT()s in our SQL queries? Besides the hours of wasted volunteer time, that's probably a lot slower.
The idea is basically to not deal with numeric ids anymore but only use the fully prefixed Q### format. This is because on commons, the assumption that an identifier consists of a prefix and some digits no longer applies. Tools and code shouldn't rely on that assumption and therefore we try to get rid of numeric ids being stored in the database and use fully prefixed ids instead. Then one can make the assumption that id and page title are always the same and there is no need for an additional table.
Best regards Bene
So, am I to understand that wb_terms, wb_items_per_site, and probably others will also switch from numeric IDs to strings?
On Sun, Aug 9, 2015 at 5:34 PM Bene* benestar.wikimedia@gmail.com wrote:
Hi Maarten
Am 09.08.2015 um 17:28 schrieb Maarten Dammers:
Hi Marius,
hoo schreef op 7-8-2015 om 19:55:
Hey folks,
we plan to drop the wb_entity_per_page table sometime soon[0], because it is just not required (as we will likely always have a programmatic mapping from entity id to page title) and it does not supported non -numeric entity ids as it is now.
In the past I was alway told to use the wb_entity_per_page table instead of doing page_title=CONCAT('Q', <id>). The Wikibase code used to contain warnings not to make this assumption. I don't know, they might still be there.
Due to this removing it is a blocker for the commons metadata.
That's unfortunate.
Is anybody using that for their tools (on tool labs)? If so, please tell us so that we can give you instructions and a longer grace period to update your scripts.
Of the 117 Wikidata related sql queries that seem to be in my homedir, 48 of them use this table. Basically any Wikidata related tool that uses the sql database will break. What do you propose? That we start messing around with CONCAT()s in our SQL queries? Besides the hours of wasted volunteer time, that's probably a lot slower.
The idea is basically to not deal with numeric ids anymore but only use the fully prefixed Q### format. This is because on commons, the assumption that an identifier consists of a prefix and some digits no longer applies. Tools and code shouldn't rely on that assumption and therefore we try to get rid of numeric ids being stored in the database and use fully prefixed ids instead. Then one can make the assumption that id and page title are always the same and there is no need for an additional table.
Best regards Bene
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
On Fri, Aug 7, 2015 at 7:55 PM, hoo hoo@online.de wrote:
Hey folks,
we plan to drop the wb_entity_per_page table sometime soon[0], because it is just not required (as we will likely always have a programmatic mapping from entity id to page title) and it does not supported non -numeric entity ids as it is now. Due to this removing it is a blocker for the commons metadata.
Is anybody using that for their tools (on tool labs)? If so, please tell us so that we can give you instructions and a longer grace period to update your scripts.
Cheers,
Marius
Hi everyone,
This is an important message to all the people running external tools.
About 2 years ago we said we want to get rid of the "wb_entity_per_page" table. This is still the case and is becoming more pressing now with the introduction of support for lexicographical data.
On July 12th, we are going to stop updating the "wb_entity_per_page" table from the Wikibase database and stop its replication on ToolLabs. At a later point we will remove it completely.
"wb_entity_per_page" was a secondary database table, mapping Wikibase entity IDs (e.g. "Q42") to MediaWiki page IDs (e.g. 138, which can be seen at https://www.wikidata.org/wiki/Q42?action=info). "wb_entity_per_page" stored entity IDs as numbers, while page titles are always full entity IDs.
This mapping existed because Wikibase was designed with the possibility to have entity pages where the ID does not match the title. This idea was never used, and finally removed in 2015 (documented in https://phabricator.wikimedia.org/T95685). We decided to get rid of the table because it contains outdated information that could mislead users, it costs resources and could conflict in the future with our new entity types for lexicographical data.
Please check if you are maintaining any code that accesses the "wb_entity_per_page" table, and replace it with lookups to MediaWiki's "page" and "redirect" tables.
We will drop the replica of the table on ToolLabs for test.wikidata.org on June 28th. We will do the same for wikidata.org on July 12th.
If you have any question or issue, feel free to add a comment on the ticket (http://phabricator.wikimedia.org/T95685) or to ping me. Thanks for your understanding
Cheers Lydia