On 08/06/12 17:32, Jeroen De Dauw wrote:

Hey,

I am looking into generalizing how interwiki and interlanguage links are stored. The motivation for this is because what's currently there does not allow holding the information needed to have Wikidata work. Before I continue work on this (ie figuring out how to best modify core code to not break existing features and tools) I'd like some feedback on the schema changes I'm thinking of making.

They are described here: https://www.mediawiki.org/wiki/User:Jeroen_De_Dauw/Wikibase_sites_and_site_l...

Some important changes to interwiki/sites compared to the current setup:

• Allows having "interlanguage links" that are not "interwiki links"
• Allows automatic synchronizing of site definitions between between all

clients (ie all Wikipedias).

• Site definitions can exist that are not used as "interlanguage link" and

not used as "interwiki link"

• Allows distinguishing between local and global identifiers
• Maps between global and local identifiers
• Holds information on the type of site. This way code can know that

bugzilla does not have a MW api.php :)

It knows that now, too. iw_api is empty. :)

The change to the langlinks table is needed to distinguish between links that are defined in local articles and links pulled from an external source so that the code rebuilding links for an article knows to ditch the local notes not in the article but keep the external ones.

Why do you want to keep interwikis in the article? I think you could directly drop langlinks table if interwikis are managed by wikidata.

This is a table with lots of rows. Anyone an estimate on how long it would take to add a boolean field with initial value true?

No, but as you start populating the column from the slaves, it doesn't create lag issues.

On 08.06.2012 17:32, Jeroen De Dauw wrote:

They are described here: https://www.mediawiki.org/wiki/User:Jeroen_De_Dauw/Wikibase_sites_and_site_l...

I like your proposal in principle, but I'm a bit worried about just dropping a well established table on the production cluster.

When migrating from the interwiki table to the sites table, there should be a way to keep the interwiki table for B/C, and also keep it updated and in sync with the new sites table.

The simplest way would be, I think, to have a maintenance script that can be run to replace the interwiki table with a view on the sites table. This way, we would maintain the old interface and just point it to the new data. This way, obscure maintenance scripts, shell hack, toolserver tools etc will continue working as before.

This will only work on databases that supports views, but mysql > 5.0 does, and this level of B/C is probably only relevant to the wikimedia cluster (and maybe wikia). So we don't care too much if it (the b/c view) doesn't work with SQLite, etc.

-- daniel