Hey,
Daniel, thanks for your input.
TL;DR at the bottom :)
The issue I was trying to deal with was storage. Currently we 100% assume
that the interwiki list is a table and there will only ever be one of them.
Yes, we are not changing this. Having a more flexible system might or might not be something we'd want in MediaWiki. We do not need it in Wikidata though. The changes we're making here do not seem to affect this issue at all, so you can just as well implement it later on.
In practice we don't want one interwiki map. In projects like Wikimedia
we actually usually want two or three.
.. And sometimes we also want a wiki-local interwiki list because some
communities want to add links to sites that other wikis don't.
This we are actually tacking, although in a different fashion then you propose. Rather then having many different lists of sites to maintain, we have split sites from their configuration. The list of sites is global and shared by all clients. Their configuration however is local. So if wiki a wants to use site x as interwikilink with prefix foobar, wiki b wants to use it with prefix baz and wiki c does not want to use it as interwikilink at all, this is perfectly possible. This split and associated generalization our changes bring add a lot of flexibility compared to the current system and remove bad assumptions currently baked in.
Also anything in this area really needs to think of our lack of user
interface. If we rewrite this then we absolutely must include a UI to view and edit this in core.
Again, this is not something we're touching at all, or want to touch, as we don't need it. Personally I think I'd be great to have such facilities, and it makes sense to add these after the backend has been fixed. I'd be happy to work with you on this (or leave it entirely up to you) once we got the relevant rewrite work done.
By rewriting it we ditch every hack trying to make it easy to control the
interwiki list and only make the problem worse.
Our change will not drop any existing functionality. I will make sure there are tools/facilities at least as good (and probably better) then the current ones.
I would like to understand what Wikidata needs out of interwiki/sites and
what it's going to do with the data
We need this for our "equivalent links", which consist out of a global site id and a page. Right now we do not have consistent global ids, in fact we don't have global ids. We just have local ids that happen to be similar everywhere (while one might not want this, but is pretty much forced to right now), which must be language codes in order to be "languagelinks" or (better named) "equivalent links". Also, right now, all languagelinks are interwikilinks (wtf) - we want to be able to have "equivalent links" without then also being interwiki links!
I'd also like to know if Wikidata plans to add any interface that will
add/remove sites
The backend will have an interface to do this, but we're not planning on any API modules or UIs. The backend will be written keeping in mind people will want those though, so it ought to be easy to add them later on.
So to wrap up: I don't think there is any conflict between what we want to do (if you disagree, please provide some pointers). You can make your changes later on, and will have a much more solid base to work on then now.
Cheers
-- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. --