In response to recent comments in our code review tool about whether some extensions should be merged into core MediaWiki, or not. I would like to try and initiate a productive conversation about this topic in hopes that we can collaboratively define a set of guidelines for evaluating when to move features out of core and into an extension or out of an extension and into core.
<unintended bias>
Arguments I have made/observed *against* merging things into core include:
1. Fewer developers have commit access to core, which pushes people off into branches who would have otherwise been able to contribute directly to trunk, inhibiting entry-level contribution. 2. Extensions encourage modularity and are easier to learn and work on because they are smaller sets of code organized in discreet bundles. 3. We should be looking to make core less inclusive, not more. The line between Wikipedia and MediaWiki is already blurry enough as it is.
Arguments I have made/observed *for* merging things into core include:
1. MediaWiki should be awesome out-of-the-box, so extensions that would be good for virtually everyone seem silly to bury deep within the poorly organized depths of the extensions folder. 2. When an extension is unable to do what it needs to do because it's dependent on a limited set of hooks, none of which quite do what it needs. 3. Because someone said so.
</unintended bias>
<obvious bias>
I will respond to these three pro-integration points; mostly because I am generally biased against integration and would like to state why. I realize that there are probably additional pro-integration points that are far less biased than the three I've listed, but I am basing these on arguments I've actually seen presented.
1. This is a very valid and important goal, but am unconvinced and merging extensions into core is the only way to achieve it. We can, for instance, take advantage the new installer that demon is working on which has the ability to automate the installation of extensions at setup-time. 2. This seems like a call for better APIs/a more robust set of hooks. Integration for this sort of reason is more likely to introduce cruft and noise than improve the software in any way. 3. Noting that "so-and-so said I should integrate this into core" is not going to magically absolve anyone of having to stand behind their decision to proceed with such action and support it with logic and reason.
</obvious bias>
If we are to develop guidelines for when to push things in/pull things out of core, it's going to be important that we reach some general consensus on the merits of these points. This mailing list is not historically known for its efficacy in consensus building, but I still feel like this conversation is going to be better off here than in a series of disjointed code review comments.
- Trevor
On 22 September 2010 06:34, Trevor Parscal tparscal@wikimedia.org wrote:
In response to recent comments in our code review tool about whether some extensions should be merged into core MediaWiki, or not. I would like to try and initiate a productive conversation about this topic in hopes that we can collaboratively define a set of guidelines for evaluating when to move features out of core and into an extension or out of an extension and into core.
I'm generally against adding random extensions to the core, and there are many things I would like to rip of the core code if possible.
I don't think we can come up with any clear rules. I'd imagine the rule would just be: * if it's useful to most users (=installations) of mediawiki, add it to the core With the "most users" getting a slightly less important if the feature is small but useful.
I think that for example the new edit toolbar or one-button search are clearly such usability improvements, that only enhance existing core features and thus should be in the core, replacing the poor original ones where possible.
Maybe also code quality and stability plays a role, since making an extension is kind of like making a branch - you can develop mostly freely and nobody cares. On the other hand the extension code might also need extensive rewriting if merged to the core.
We have a quite free access policy even to the trunk code, so I do not see it as a limitation for including code in the core. On the other hand putting code in the trunk seems to be the only sure way to get your code reviewed even a little and deployed into WMF (it may still take a year).
In short: I agree keeping the core slick and clean, but I think the case is clear for (at least part of) the usability enhancements.
-Niklas
On 22/09/10 05:34, Trevor Parscal wrote: <argument #2 against merging>
- Extensions encourage modularity and are easier to learn and work on because they are smaller sets of code organized in discreet bundles.
That is probably the most important point. It ease code review, any new comer can easily understand code.
Arguments I have made/observed *for* merging things into core include:
- MediaWiki should be awesome out-of-the-box, so extensions that would be good for virtually everyone seem silly to bury deep within the poorly organized depths of the extensions folder.
What about shipping the Mediawiki release with some of the best extensions ? You do not need to actually merge them with trunk, just chip mediawiki + some extensions preconfigured. Hooks in the installer would let user tweak the configuration. I had a little script which, given a revision number would : - installed trunk and stable branch - copy my favorite extensions in the extension folders - append to local settings my extension configuration files
<snip>
If we are to develop guidelines for when to push things in/pull things out of core, ...
Would you move as extensions : squid support, math, languages, external stores, external auth, API, chinese support, Tidy, Antivirus ... They are all awesome features but not necessarily needed in core.
On Tue, Sep 21, 2010 at 8:34 PM, Trevor Parscal tparscal@wikimedia.org wrote:
In response to recent comments in our code review tool about whether some extensions should be merged into core MediaWiki, or not. I would like to try and initiate a productive conversation about this topic in hopes that we can collaboratively define a set of guidelines for evaluating when to move features out of core and into an extension or out of an extension and into core.
<unintended bias>
Arguments I have made/observed *against* merging things into core include:
1. Fewer developers have commit access to core, which pushes people off into branches who would have otherwise been able to contribute directly to trunk, inhibiting entry-level contribution.
People can get access to core. This is a poor argument for not integrating things into core. If this is a problem, then it is a problem with us, and our processes, and should be fixed.
2. Extensions encourage modularity and are easier to learn and work on because they are smaller sets of code organized in discreet bundles. 3. We should be looking to make core less inclusive, not more. The line between Wikipedia and MediaWiki is already blurry enough as it is.
I think this should be the case, but there are certain sets of functionality that people expect to be installed in a default installation. A usable editing interface is definitely one of those things.
Arguments I have made/observed *for* merging things into core include:
1. MediaWiki should be awesome out-of-the-box, so extensions that would be good for virtually everyone seem silly to bury deep within the poorly organized depths of the extensions folder.
I totally agree with this to an extent. I don't think we should be including everything and the kitchen sink, but things that make MediaWiki simply feel broken when they are missing should be integrated. ParserFunctions is an amazing example of this. MediaWiki simply doesn't work without ParserFunctions. You can tell me it does, and that people can live without it, but I refuse to believe that. We get support issues extremely frequently that end in "install ParserFunctions".
I'd also like to mention that something I'm seeing in even greater frequency now is "how do I get Vector to work like it does on Wikipedia?" or "How do I make the sidebar collapse", or "why doesn't my search bar work like it does on Wikipedia"?
Why did we expend all of this effort on usability if we are going to make it a pain to use if you aren't Wikimedia? That is very anti-usability.
2. When an extension is unable to do what it needs to do because it's dependent on a limited set of hooks, none of which quite do what it needs.
This goes back to people needing core access to add hooks. In reality, they can send patches, or ask a dev with core access to do so. I've had a number of hooks added this way.
3. Because someone said so.
Agreed. This is never a reasonable argument.
</unintended bias>
<obvious bias>
I will respond to these three pro-integration points; mostly because I am generally biased against integration and would like to state why. I realize that there are probably additional pro-integration points that are far less biased than the three I've listed, but I am basing these on arguments I've actually seen presented.
1. This is a very valid and important goal, but am unconvinced and merging extensions into core is the only way to achieve it. We can, for instance, take advantage the new installer that demon is working on which has the ability to automate the installation of extensions at setup-time.
One problem I have every time I upgrade MediaWiki is extension compatibility. Extension management in MediaWiki is simply broken. Until we have a reasonable way to handle extension compatibility and some reasonable means for people to upgrade extensions, this is a painful course of action.
2. This seems like a call for better APIs/a more robust set of hooks. Integration for this sort of reason is more likely to introduce cruft and noise than improve the software in any way.
This would help.
3. Noting that "so-and-so said I should integrate this into core" is not going to magically absolve anyone of having to stand behind their decision to proceed with such action and support it with logic and reason.
Agreed. We need to be picky about what goes into core.
</obvious bias>
If we are to develop guidelines for when to push things in/pull things out of core, it's going to be important that we reach some general consensus on the merits of these points. This mailing list is not historically known for its efficacy in consensus building, but I still feel like this conversation is going to be better off here than in a series of disjointed code review comments.
Well, to restate my above opinions: I think in general we don't integrate enough into core. We create way too many extensions, even for things that are essentially required to properly run MediaWiki. I totally understand the want to not bloat our core software, and we should strive to keep it lean and fast, but we also need to take into consideration our third party users. Requiring users to install and manage a million extensions to get basic functionality is asking a lot.
Respectfully,
Ryan Lane
It seems like there are two dimensions to this problem.
X: Should it be part of stock MediaWiki or not Y: Is it better to write features as discreet modules or spread them out the way core is currently done
The problem is when you move across X, you are forced to cross Y; If we were to start moving extensions into core, it would mean breaking up otherwise well bundled units of code into lots of tiny parts and spreading them out. But why must this be the case?
Modular feature development being unique to extensions points out a significant flaw in the design of MediaWiki core. There's no reason we can't convert existing core features to discreet components, much like how extensions are written, while leaving them in stock MediaWiki. This sort of design would also make light work of adding extensions to core.
- Trevor
On 9/22/10 12:11 AM, Ryan Lane wrote:
On Tue, Sep 21, 2010 at 8:34 PM, Trevor Parscaltparscal@wikimedia.org wrote:
In response to recent comments in our code review tool about whether some extensions should be merged into core MediaWiki, or not. I would like to try and initiate a productive conversation about this topic in hopes that we can collaboratively define a set of guidelines for evaluating when to move features out of core and into an extension or out of an extension and into core.
<unintended bias>
Arguments I have made/observed *against* merging things into core include:
- Fewer developers have commit access to core, which pushes people off into branches who would have otherwise been able to contribute directly to trunk, inhibiting entry-level contribution.
People can get access to core. This is a poor argument for not integrating things into core. If this is a problem, then it is a problem with us, and our processes, and should be fixed.
- Extensions encourage modularity and are easier to learn and work on because they are smaller sets of code organized in discreet bundles.
- We should be looking to make core less inclusive, not more. The line between Wikipedia and MediaWiki is already blurry enough as it is.
I think this should be the case, but there are certain sets of functionality that people expect to be installed in a default installation. A usable editing interface is definitely one of those things.
Arguments I have made/observed *for* merging things into core include:
- MediaWiki should be awesome out-of-the-box, so extensions that would be good for virtually everyone seem silly to bury deep within the poorly organized depths of the extensions folder.
I totally agree with this to an extent. I don't think we should be including everything and the kitchen sink, but things that make MediaWiki simply feel broken when they are missing should be integrated. ParserFunctions is an amazing example of this. MediaWiki simply doesn't work without ParserFunctions. You can tell me it does, and that people can live without it, but I refuse to believe that. We get support issues extremely frequently that end in "install ParserFunctions".
I'd also like to mention that something I'm seeing in even greater frequency now is "how do I get Vector to work like it does on Wikipedia?" or "How do I make the sidebar collapse", or "why doesn't my search bar work like it does on Wikipedia"?
Why did we expend all of this effort on usability if we are going to make it a pain to use if you aren't Wikimedia? That is very anti-usability.
- When an extension is unable to do what it needs to do because it's dependent on a limited set of hooks, none of which quite do what it needs.
This goes back to people needing core access to add hooks. In reality, they can send patches, or ask a dev with core access to do so. I've had a number of hooks added this way.
- Because someone said so.
Agreed. This is never a reasonable argument.
</unintended bias>
<obvious bias>
I will respond to these three pro-integration points; mostly because I am generally biased against integration and would like to state why. I realize that there are probably additional pro-integration points that are far less biased than the three I've listed, but I am basing these on arguments I've actually seen presented.
- This is a very valid and important goal, but am unconvinced and merging extensions into core is the only way to achieve it. We can, for instance, take advantage the new installer that demon is working on which has the ability to automate the installation of extensions at setup-time.
One problem I have every time I upgrade MediaWiki is extension compatibility. Extension management in MediaWiki is simply broken. Until we have a reasonable way to handle extension compatibility and some reasonable means for people to upgrade extensions, this is a painful course of action.
- This seems like a call for better APIs/a more robust set of hooks. Integration for this sort of reason is more likely to introduce cruft and noise than improve the software in any way.
This would help.
- Noting that "so-and-so said I should integrate this into core" is not going to magically absolve anyone of having to stand behind their decision to proceed with such action and support it with logic and reason.
Agreed. We need to be picky about what goes into core.
</obvious bias>
If we are to develop guidelines for when to push things in/pull things out of core, it's going to be important that we reach some general consensus on the merits of these points. This mailing list is not historically known for its efficacy in consensus building, but I still feel like this conversation is going to be better off here than in a series of disjointed code review comments.
Well, to restate my above opinions: I think in general we don't integrate enough into core. We create way too many extensions, even for things that are essentially required to properly run MediaWiki. I totally understand the want to not bloat our core software, and we should strive to keep it lean and fast, but we also need to take into consideration our third party users. Requiring users to install and manage a million extensions to get basic functionality is asking a lot.
Respectfully,
Ryan Lane
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
2010/9/22 Trevor Parscal tparscal@wikimedia.org:
Modular feature development being unique to extensions points out a significant flaw in the design of MediaWiki core. There's no reason we can't convert existing core features to discreet components, much like how extensions are written, while leaving them in stock MediaWiki. This sort of design would also make light work of adding extensions to core.
Making MediaWiki more modular won't magically make it possible (or even desirable) to write any old feature as an extension. Some things will still have to go in core and some things we'll simply /want/ to put in core because making them extensions would be ridiculous for some reason. It's true that more modularity will lower the difference in awkwardness between doing certain things as extensions vs. in core (and is generally desirable), but I also don't see how it would make (much) lighter work of adding extensions to core: most extensions should either be easy to add to core (Vector and WikiEditor for instance, any special page extension) or cause some bumps because they were written in an awkward way to begin with, probably because it'd have been better to put them in core.
Roan Kattouw (Catrope)
Roan Kattouw roan.kattouw@gmail.com wrote:
2010/9/22 Trevor Parscal tparscal@wikimedia.org:
Modular feature development being unique to extensions points out a significant flaw in the design of MediaWiki core. There's no reason we can't convert existing core features to discreet components, much like how extensions are written, while leaving them in stock MediaWiki. This sort of design would also make light work of adding extensions to core.
Making MediaWiki more modular won't magically make it possible (or even desirable) to write any old feature as an extension. Some things will still have to go in core and some things we'll simply /want/ to put in core because making them extensions would be ridiculous for some reason.
I'd rather have MediaWiki build on some classes & object instances with a clear responsibilities, Inversion of Control and possibility to test each _object_ separately without causing interference to other components. Discussion what's core/extension is to me secondary. Maybe at some point "hooks" as we know them will not be needed, we could be able to use interfaces as provided by the programming language instead of hooks, possibly (yes, I'm dreaming) tied together by something like PicoContainer.
Even still, those interfaces *will* change and it would be good if developers could refactor the code in the core and extensions at the same time.
I don't why we really duplicate so much functions even within core (API vs. standard special pages for example). But that's probably the issue to be solved in phase4 :)
So before asking how much to add into core, maybe we should first clean up some, and then possibly add. Or sometimes adding something (like a proper multi-wiki configuration management $wgConf++) may clean up and simplify some things inside core.
//Marcin
On Fri, Sep 24, 2010 at 3:30 AM, Marcin Cieslak saper@saper.info wrote:
So before asking how much to add into core, maybe we should first clean up some, and then possibly add. Or sometimes adding something (like a proper multi-wiki configuration management $wgConf++) may clean up and simplify some things inside core.
$wgConf sucks ;-)
But proper configuration management is on the horizon.
-Chad
On Wed, Sep 22, 2010 at 6:33 PM, Trevor Parscal tparscal@wikimedia.org wrote:
Modular feature development being unique to extensions points out a significant flaw in the design of MediaWiki core. There's no reason we can't convert existing core features to discreet components, much like how extensions are written, while leaving them in stock MediaWiki. This sort of design would also make light work of adding extensions to core.
I think that I would support allowing core features to use hooks instead of writing the code in all over the place. All you would have to do is initialise $wgHooks somewhere before LocalSettings.php is run.
Giving structural proximity to functionally-related code would make things much less confusing for new developers (and simpler for those with experience). It would allow us to declutter our core code (things like user permission checking come to mind), and it would make light work of merging extensions into core.
Trevor Parscal wrote:
In response to recent comments in our code review tool about whether some extensions should be merged into core MediaWiki, or not. I would like to try and initiate a productive conversation about this topic in hopes that we can collaboratively define a set of guidelines for evaluating when to move features out of core and into an extension or out of an extension and into core.
I know I mentioned this in a CodeReview comment, but I'll mention it again here: http://mediawiki.org/wiki/List_of_extensions_to_be_merged_to_the_core
I think developing a set of guidelines for this issue might be nice, though I'm not sure there's really all that much disagreement about what should and shouldn't be in core currently.
MZMcBride
On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal tparscal@wikimedia.org wrote:
when to move features out of core and into an extension or out of an extension and into core.
I don't think anyone's commented on the former (everyone's been talking about pushing in, not pulling out). IMO, the conditions for splitting something into an extension
A) Not a lot of people use it anyway (hard to gauge) B) It probably shouldn't have been in core in the first place (eg: AskSQL)
So far, the only successful case I can think of offhand for splitting an extension out was AskSQL, but it's a perfect example of what should happen.
2. When an extension is unable to do what it needs to do because it's dependent on a limited set of hooks, none of which quite do what it needs.
I think this also makes a good case for keeping something in core and not removing it. If we need to add 6 new hooks to keep it working, then it might be best to keep it in core (single purpose hooks are lame)
1. This is a very valid and important goal, but am unconvinced and merging extensions into core is the only way to achieve it. We can, for instance, take advantage the new installer that demon is working on which has the ability to automate the installation of extensions at setup-time.
Quick note on the installer. It only enables extensions that already reside in your extensions folder. Since we don't distribute any with the default package, this might not be terribly useful. More awesome is Jeroen's GSoC work on extension management. Something to look at post-1.17 branch though.
3. Because someone said so.
If enough people say so ;-)
-Chad
On 22 September 2010 13:52, Chad innocentkiller@gmail.com wrote:
Quick note on the installer. It only enables extensions that already reside in your extensions folder. Since we don't distribute any with the default package, this might not be terribly useful. More awesome is Jeroen's GSoC work on extension management. Something to look at post-1.17 branch though.
Note from the perspective of a tarball user: yes please. Though code management may be much more sensible with things organised into the extensions folder, having a basic set of extensions included by default in the tarball would be most helpful to the sysadmin whose user just asked for a new wiki.
(I'm thinking by analogy with operating systems - a "base" set of extensions which are a good working set to start with, but which you can strip right down to "core" if you really want to.)
- d.
Chad wrote:
On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal wrote:
when to move features out of core and into an extension or out of an extension and into core.
I don't think anyone's commented on the former (everyone's been talking about pushing in, not pulling out). IMO, the conditions for splitting something into an extension
A) Not a lot of people use it anyway (hard to gauge) B) It probably shouldn't have been in core in the first place (eg: AskSQL)
So far, the only successful case I can think of offhand for splitting an extension out was AskSQL, but it's a perfect example of what should happen.
DumpHTML was also split from core. texvc should have been moved out of core, but since that would change things set there from the mwbeginning, nobody did it yet.
- This is a very valid and important goal, but am unconvinced and merging extensions into core is the only way to achieve it. We can, for instance, take advantage the new installer that demon is working on which has the ability to automate the installation of extensions at setup-time.
Quick note on the installer. It only enables extensions that already reside in your extensions folder. Since we don't distribute any with the default package, this might not be terribly useful. More awesome is Jeroen's GSoC work on extension management. Something to look at post-1.17 branch though.
I think the point is to start shipping mediawiki with common extensions there.
Perhaps the new installer could contain that as an option during the inital setup. Like a two or three-column thing with a bunch of checkboxes.
Language: English [/] Default theme (X) Vector (_) Monobook (_) Foobar Common Extension: [X] ParserFunctions [X] SpecialInterwiki [X] Cite.php [X] CharInsert [X] CategoryTree
etc. you get the idea
Op 29 sep 2010, om 01:15 heeft Platonides het volgende geschreven:
Chad wrote:
On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal wrote:
when to move features out of core and into an extension or out of an extension and into core.
I don't think anyone's commented on the former (everyone's been talking about pushing in, not pulling out). IMO, the conditions for splitting something into an extension
A) Not a lot of people use it anyway (hard to gauge) B) It probably shouldn't have been in core in the first place (eg: AskSQL)
So far, the only successful case I can think of offhand for splitting an extension out was AskSQL, but it's a perfect example of what should happen.
DumpHTML was also split from core. texvc should have been moved out of core, but since that would change things set there from the mwbeginning, nobody did it yet.
- This is a very valid and important goal, but am unconvinced and merging extensions into core is the only way to achieve it. We can, for instance, take advantage the new installer that demon
is working on which has the ability to automate the installation of extensions at setup-time.
Quick note on the installer. It only enables extensions that already reside in your extensions folder. Since we don't distribute any with the default package, this might not be terribly useful. More awesome is Jeroen's GSoC work on extension management. Something to look at post-1.17 branch though.
I think the point is to start shipping mediawiki with common extensions there.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
That seems like a good idea. I would love to see this in action.
-X!
Sent from my iPod
On Sep 28, 2010, at 8:11 PM, Krinkle krinklemail@gmail.com wrote:
Perhaps the new installer could contain that as an option during the inital setup. Like a two or three-column thing with a bunch of checkboxes.
Language: English [/] Default theme (X) Vector (_) Monobook (_) Foobar Common Extension: [X] ParserFunctions [X] SpecialInterwiki [X] Cite.php [X] CharInsert [X] CategoryTree
etc. you get the idea
Op 29 sep 2010, om 01:15 heeft Platonides het volgende geschreven:
Chad wrote:
On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal wrote:
when to move features out of core and into an extension or out of an extension and into core.
I don't think anyone's commented on the former (everyone's been talking about pushing in, not pulling out). IMO, the conditions for splitting something into an extension
A) Not a lot of people use it anyway (hard to gauge) B) It probably shouldn't have been in core in the first place (eg: AskSQL)
So far, the only successful case I can think of offhand for splitting an extension out was AskSQL, but it's a perfect example of what should happen.
DumpHTML was also split from core. texvc should have been moved out of core, but since that would change things set there from the mwbeginning, nobody did it yet.
- This is a very valid and important goal, but am unconvinced and merging extensions into core is the only way to achieve it. We can, for instance, take advantage the new installer that demon
is working on which has the ability to automate the installation of extensions at setup-time.
Quick note on the installer. It only enables extensions that already reside in your extensions folder. Since we don't distribute any with the default package, this might not be terribly useful. More awesome is Jeroen's GSoC work on extension management. Something to look at post-1.17 branch though.
I think the point is to start shipping mediawiki with common extensions there.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Op 29 sep 2010, om 01:15 heeft Platonides het volgende geschreven:
I think the point is to start shipping mediawiki with common extensions there.
Yeah, that sounds good ! Perhaps the new installer could contain that as an option during the inital setup. Like a two or three-column thing with a bunch of checkboxes.
Language: English [/] Default theme (X) Vector (_) Monobook (_) Foobar Common Extension: [X] ParserFunctions [X] SpecialInterwiki [X] Cite.php [X] CharInsert [X] CategoryTree
etc. you get the idea
-- Krinkle
I think the point is to start shipping mediawiki with common extensions there.
This is great if we are going to version the extensions with the version of MediaWiki shipped, and backport changes to those extensions to ensure they get security and bug fixes like core does. Alternatively, we could provide a way for extensions to be easily upgraded to compatible versions, similar to wordpress and other sane applications.
If we don't plan on doing one of two things, then we are forcing our users into the crapshoot that is finding the correct extension version, or possibly running insecure or buggy extensions.
Respectfully,
Ryan Lane
On Wed, Sep 29, 2010 at 3:01 AM, Ryan Lane rlane32@gmail.com wrote:
If we don't plan on doing one of two things, then we are forcing our users into the crapshoot that is finding the correct extension version, or possibly running insecure or buggy extensions.
Like we are doing today. So yes, indeed something that we should fix.
On Tue, Sep 21, 2010 at 11:34 PM, Trevor Parscal tparscal@wikimedia.org wrote:
In response to recent comments in our code review tool about whether some extensions should be merged into core MediaWiki, or not. I would like to try and initiate a productive conversation about this topic in hopes that we can collaboratively define a set of guidelines for evaluating when to move features out of core and into an extension or out of an extension and into core.
Partly this is a matter of preference. I personally write things almost entirely for core -- I've never written an extension, so my only extension commits are to things that other people wrote as extensions. Other people prefer to write extensions instead of core features. As long as the code gets written and works either way, I don't think it's usually a huge deal. A lot of stuff will work fine either in core or as an extension -- this doesn't have to be made a religious issue or one where we have detailed official policies.
What I think is clear is that anything that the large majority of MediaWikis would benefit from should be enabled by default, which given our current setup means it has to be in core. I haven't looked closely, but I believe this includes a significant part of the usability stuff (although not all). MediaWiki should work as well as possible for as many users as possible out of the box, and whatever that implies for how the code is structured is what we should do. I cannot conceive of any reason why most of the features of the Vector extension should not be in core, for instance -- why would you *not* want them? Why would you ever want to disable them? ("Admins should have choice" is not a good reason by itself. Cf.: http://weblogs.mozillazine.org/bz/archives/020683.html)
Other than things you want to enable by default, here are some reasons to put some features in core rather than extensions:
* Core features are much easier for site admins to enable than extensions. It would be great if this were fixed, but currently, it's not, so it can't be dismissed. You can dismiss this reason (basically your #1) as soon as it's become irrelevant in practice, but not before. * For smaller features, it will take a lot more code to write them as extensions than as part of core. You can often add a useful core feature in less than ten lines of code -- extensions are much heavier-weight. This is unavoidable, because extensions have to have a whole bunch of metadata to specify where their code executes and so on; with core code you can just put it in the right place. * It's easier to understand how core code works in the absence of extensions, because the code is explicitly written out where it occurs. When extensions are involved, you have a wfRunHooks() line in core, which basically means "absolutely anything could happen at this point". When you run into a hook when reading code, it doesn't only complicate understanding how the code will run in practice, it also makes it much harder to change. If you want to refactor, you have to write a lot of painstaking and ugly logic to preserve the semantics of the hooks, or else just give up and get rid of them.
On Wed, Sep 22, 2010 at 4:33 AM, Trevor Parscal tparscal@wikimedia.org wrote:
Modular feature development being unique to extensions points out a significant flaw in the design of MediaWiki core. There's no reason we can't convert existing core features to discreet components, much like how extensions are written, while leaving them in stock MediaWiki.
This can be accomplished by regular old object-oriented programming. Keep all the related code in one class, and make as much possible private so that classes can only call a limited number of well-defined, well-documented, easily-understood entry points. I don't think extensions are a good model for this -- to the contrary, they're antithetical to modularity, since any extension can drastically change how any part of the program behaves (given enough hooks).
On Wed, Sep 22, 2010 at 10:17 AM, Andrew Garrett agarrett@wikimedia.org wrote:
I think that I would support allowing core features to use hooks instead of writing the code in all over the place. All you would have to do is initialise $wgHooks somewhere before LocalSettings.php is run.
Giving structural proximity to functionally-related code would make things much less confusing for new developers (and simpler for those with experience). It would allow us to declutter our core code (things like user permission checking come to mind), and it would make light work of merging extensions into core.
This is reminiscent of COMEFROM. It makes it really hard to understand what's going on if you have the code scattered everywhere. Modularity is great, but it needs to happen along clear boundaries to keep the code comprehensible -- not "anything can happen at this point".
On 9/22/10 9:49 AM, Aryeh Gregor wrote:
Partly this is a matter of preference. I personally write things almost entirely for core -- I've never written an extension, so my only extension commits are to things that other people wrote as extensions. Other people prefer to write extensions instead of core features. As long as the code gets written and works either way, I don't think it's usually a huge deal. A lot of stuff will work fine either in core or as an extension -- this doesn't have to be made a religious issue or one where we have detailed official policies.
I certainly never mentioned anything about making this a religious issue (metaphor understood) but I think being able to point to a simple guideline on the wiki would be an improvement on the current situation.
This can be accomplished by regular old object-oriented programming. Keep all the related code in one class, and make as much possible private so that classes can only call a limited number of well-defined, well-documented, easily-understood entry points. I don't think extensions are a good model for this -- to the contrary, they're antithetical to modularity, since any extension can drastically change how any part of the program behaves (given enough hooks).
And what about javascript and css files? What about messages? Generally speaking lots of the MediaWiki PHP code is modular, but it's the rest of the stuff that goes into making a feature work that is not. Extensions bundle this all together, and that proximity makes maintenance and review easier.
- Trevor
I suppose I have a foot in each camp here.
Ryan Lane wrote:
ParserFunctions is an amazing example of this. MediaWiki simply doesn't work without ParserFunctions. You can tell me it does, and that people can live without it, but I refuse to believe that. We get support issues extremely frequently that end in "install ParserFunctions".
I wrote ParserFunctions under protest, and I hoped at the time that it would not be essential or widely used. That's why it was an extension. Maybe the time has come to merge it in.
I also wrote the ImageMap extension, which some might argue should be integrated.
Aryeh Gregor wrote:
Partly this is a matter of preference. I personally write things almost entirely for core -- I've never written an extension, so my only extension commits are to things that other people wrote as extensions.
I agree that there is an element of personal preference. Brion was certainly in favour of putting lots of functionality in extensions, and, like Trevor, used the argument that at some point in the future, extensions will be easier to install and/or bundled with core releases. Brion wrote the Poem extension, for instance, which is easily as trivial as Trevor's ArticleEmblems extension.
My view is somewhat more moderate. I think that:
* Trivial, uncontroversial things like EditWarning should be in the core. * MediaWiki should be usable without extensions, some basic set of functionality is required. * Complex functionality, large code size, complex installation and external dependencies are all arguments for splitting a feature out to an extension.
Trevor Parscal wrote:
In response to recent comments in our code review tool about whether some extensions should be merged into core MediaWiki, or not. I would like to try and initiate a productive conversation about this topic in hopes that we can collaboratively define a set of guidelines for evaluating when to move features out of core and into an extension or out of an extension and into core.
I think there is a need for judgement, and a definitive set of guidelines will be difficult to produce.
- Noting that "so-and-so said I should integrate this into core" is not going to magically absolve anyone of having to stand behind their decision to proceed with such action and support it with logic and reason.
Yes, however, community consensus should be respected. The logic and reason you've been expressing in this thread does not appear to be particularly more convincing than anyone else's.
Aryeh Gregor wrote:
- It's easier to understand how core code works in the absence of
extensions, because the code is explicitly written out where it occurs. When extensions are involved, you have a wfRunHooks() line in core, which basically means "absolutely anything could happen at this point". When you run into a hook when reading code, it doesn't only complicate understanding how the code will run in practice, it also makes it much harder to change. If you want to refactor, you have to write a lot of painstaking and ugly logic to preserve the semantics of the hooks, or else just give up and get rid of them.
I agree with this. The point of modularity is to separate concerns, and a hook-based model is opposed to this. A well-designed modular architecture has a simple, well-documented interface between the modules. Some of our extensions are a long way from this.
Nowadays I prefer adding object-oriented extension interfaces, which work by having the extension subclass a core base class. See for instance MediaHandler, FileRepo, LBFactory and PoolCounter.
Trevor Parscal wrote:
And what about javascript and css files? What about messages? Generally speaking lots of the MediaWiki PHP code is modular, but it's the rest of the stuff that goes into making a feature work that is not. Extensions bundle this all together, and that proximity makes maintenance and review easier.
There are security problems with having web-accessible files like JavaScript and CSS in the same directory as non-web-accessible files like PHP includes and command-line scripts. Perhaps the resource loader will fix this, and we'll be able to organise such files by module in the future.
There is a significant performance advantage to splitting up messages by language. However it is possible to have module-specific message files in the core -- we have one now for the new installer.
-- Tim Starling
Tim Starling wrote:
Ryan Lane wrote:
ParserFunctions is an amazing example of this. MediaWiki simply doesn't work without ParserFunctions. You can tell me it does, and that people can live without it, but I refuse to believe that. We get support issues extremely frequently that end in "install ParserFunctions".
I wrote ParserFunctions under protest, and I hoped at the time that it would not be essential or widely used. That's why it was an extension. Maybe the time has come to merge it in.
To me, this has consistently seemed like a "perfect is the enemy of done" issue. I understand your reluctance to merge ParserFunctions into core, but there really doesn't appear to be any viable alternative on the horizon. For years now, we've added extra hassle to nearly every MediaWiki installation under the pretense that one day, there might be something better than ParserFunctions.
Similar arguments apply to StringFunctions, which are intentionally disabled on Wikimedia wikis, even though people have created far worse hacks and workarounds using functions like {{padleft:}}.
If there's a realistic goal and timeline for an better template syntax system (JavaScript, Lua, whatever), I think it would be fine to continue putting this off. But if it really is a pipe dream, I don't think it's sensible to continue shipping MediaWiki without at least ParserFunctions.
I also wrote the ImageMap extension, which some might argue should be integrated.
I'm not sure who might be arguing for ImageMap to be in core. The largest use-case of ImageMap I ever saw was as a hack to override the link destination of images, which was finally properly integrated into image syntax (by you, as it turns out). I don't think most wiki installations would find any value in this extension.
MZMcBride
wikitech-l@lists.wikimedia.org