On Mon, Jul 22, 2013 at 6:25 AM, Denny Vrandečić < denny.vrandecic@wikimedia.de> wrote:
I assume Ryan didn't mean to single out the Wikidata development team. Other teams have done this as well -- the Translate extension depends on ULS, CodeEditor depends on WikiEditor, Semantic Result Formats depends on Semantic MediaWiki etc. I assume Ryan just choose Wikibase because it exemplifies the symptoms so well. Ryan, please correct me if my assumptions are wrong.
Indeed, this is a general problem, but Wikibase does this quite a bit more than other extensions have in the past. It's been a growing concern of mine as an admin for quite a while now.
The reminder of this mail has two parts, first it tries to explain the motivation of why the Wikidata dev team did things the way we did, and second, it asks this list to please resolve the underlying actual issue.
First:
The original message by Ryan lists the number of dependencies for Wikibase. He lists, e.g. Scribunto as an extension. The question is, how to avoid this dependency? Should we move Scribunto to core? Should we turn Scribunto and Wikibase into one extension?
The same for Babel and ULS, listed as optional extensions.
Another extension mentioned is Diff. Diff provides generic functionality that can be used by different extensions. It seems that in this case the suggestion that Ryan makes is to move it to core. So shall we move functionality that more than one extension depend on generally to core?
One of the reasons for DataValues, Ask and some others being in their own extension is that they already are or are planned very soonish to be reused in SMW. Since it is suggested that generic functionality should be moved to core, would that include functionality shared by Wikibase and SMW? Or how else would we manage that shared code?
This is the hardest scenario of the group. It's likely best that they are in separate extensions. This is still a complication, but is more an issue with MediaWiki's lack of extension management than it is with the modularity of the extensions themselves.
Another reason for having separate components like the WikibaseModel or Diff is that they are not MediaWiki extensions, but pure PHP libraries. Any PHP script can reuse them. Since the WikibaseModel is not trivial, this should help with the writing of bots and scripts dealing with Wikidata data. How should we handle such components? Should they be moved to Wikibase and we require every script to depend on the whole of Wikibase, and thus MediaWiki?
For pure PHP libraries, they could be distributed like pure PHP libraries usually are. They can be packaged for multiple distros and be available via apt/yum/composer (or pear). Having them as MediaWiki extensions is somewhat awkward.
If you add everything needed by Wikibase into a single extension, how do you ensure that no unnecessary dependencies creep in? Is there a code-analyzer that can run as part of Jenkins that check that the architecture is not being violated, and that parts of the code to not introduce dependencies on other parts where they should not? Separating such components allows us to check this part of the architecture during CI, which is indeed extremely helpful.
How does splitting extensions apart make it easier to check this during CI? Is there some automated way to do this when they are split, but not when they are together?
I would indeed be very much interested in better solutions for these questions than we currently have.
As Ryan said in his thread-opening email, "For legitimate library-like extensions I have no constructive alternative, but there must be some sane alternative to this." A lot of the issues would be resolved if we had this constructive alternative. The solution will likely also help to deal with the other dependencies. I hope it is understandable that I do not consider the time of the Wikidata development well spent to replace our architecture with something else, before we have agreed on what this something else should be.
Yes, that's surely a good position to take :).
Second:
I would be interested in answers to the above questions. But maybe we really should concentrate on getting the actual question resolved, which has been discussed on this list several times without consensus, those that would allows us to answer the above questions trivially: how should we deal with modularity, extensions, components, etc. in MediaWiki? I hope the answer is not "throw everything in core or into monolithical extensions which do not have dependencies among each other", but let's see what the discussion will bring. Once we have this answer, we can implement the results. Until then I am not sure whether I found it productive to single the Wikidata team out in the way we are doing things.
My concern was increasing complexity for admins with no effort towards decreasing it. Composer may indeed be an answer to some of the issues. From an admin's perspective using composer shouldn't be much harder than what they are currently expected to do and it would simplify the extension process.
- Ryan