Hi everyone,
The current MediaWiki Architecture Committee[1] has its roots in a 2013 Amsterdam Hackthon session[2], where we
On Thu, Jan 15, 2015 at 5:40 PM, Rob Lanphier robla@wikimedia.org wrote:
Hi everyone,
The current MediaWiki Architecture Committee[1] has its roots in a 2013 Amsterdam Hackthon session[2], where we
Heh....how's that for a teaser. Well, now I'm committed to finishing this email :-)
Stay tuned....
Rob
(Alright...let's try this again!)
Hi everyone,
The current MediaWiki Architecture Committee[1] has its roots in a 2013 Amsterdam Hackathon session[2], where we had a pair of sessions to try to establish our architectural guidelines[3]. It was there that we agreed that it would be good to revive our then moribund process for reviewing RFCs[4]. Since no one there really knew whose job it was to review these things, I believe I said "how about we start with everyone with 'architect' in their title at WMF?", which was met with uncomfortable shrugging that I interpreted as "consensus!", and no one corrected me. Thus Brion Vibber, Mark Bergsma, and Tim Starling became the founding members of the Arch Committee.
Subsequent to that meeting, I pretended to proceed as though a decision was made. However, over the past year and half since then, there's been much more uncomfortable shrugging. Even Brion, Mark, and Tim have not seemed entirely comfortable with the idea. It was widely acknowledged that the group was heavily biased toward the lower parts of our server software stack. The committee agreed to add Roan Kattouw and Daniel Kinzler to the group as a means of providing a wider perspective, with the added bonus of adding at least one person who isn't a WMF employee.
So, here we are today. I believe no one would dispute the credentials of every member of the group. Brion, Tim, and Mark have an extremely long history with the project, being employees #1, #2, and #3 of the WMF respectively, and all having contributed massively to the success of Wikipedia and to MediaWiki as general purpose wiki software. In most open source projects, one of them would probably be BFDL[5]. Roan and Daniel are more "recent", but only in relative terms, and also have very significant contributions to their name. All have the widespread respect of pretty much everyone in the MediaWiki developer community.
Additionally, I hear quite a bit of relief that the previously moribund RFC process is doing much better now. Things are moving, and if you know how to work the process and aren't proposing anything too wild, you can get an RFC approved pretty quickly. The committee has made a lap through the entire backlog of RFCs.
Still, the uncomfortable shrugging continues. The group is broader, but still lacks the breadth, particularly in front end and in the development of newer services such as Parsoid and RESTBase. This aspect is pretty obviously something that can be fixed. Another problem is that the scope of the group isn't clear to everyone. Is this group responsible for leading, or merely responsible for reviewing big ideas from others to ensure continuity and sanity? How big does an idea need to be before an RFC needs to be written (as opposed to just dropping a patch in Gerrit)? Defining the scope of the group is also a fixable problem.
However, I don't sense much of a desire to fix things. The dominant meme that I hear is that we should go back to the day before uncomfortable shrugging led to a committee becoming BFDL. What I fear, though, is that we will develop a system lacking in conceptual integrity[6], as individual warring fiefdoms emerge. It's quite simple to argue this is already happening.
So, where does that leave us? Do we need a BFDL? If so, who should pick? Should it be someone in the project? Should the WMF hire someone to lead this? If not, do we keep the committee? Do we just let this be consensus based?
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
In general, I feel a sense of urgency that seems lacking in the status quo. We've made progress over the past couple of years, but it doesn't feel like our progress is entirely up to the task. We have a collection of *many* instances of individual or small team excellence that are sadly the mere sum of their parts. My intuition is that we lose out on multiplicative effects as we fail to engage the wider group in our activities, and as we lack engineering-level orchestration. Team-level pride in fantastic work drifts into project-level despair, as many excellent engineers fail to grasp how to make big changes outside of their limited domains.
Perhaps I'm being too hyperbolic. Perhaps the answer is "embrace the chaos; it's the Wiki Way(tm)" I don't buy it, but I'm probably one of the easier people to convince of this. I think if this is the way it's gonna be, someone needs to make the case how this is actually working now. Step up.
Perhaps I'm also suffering from living inside the WMF echo chamber for too long. It could be that the general pessimism about the direction of MediaWiki (or lack thereof) is not shared out here. Perhaps people who get most of their news from this mailing list are perfectly happy with the status quo, and appreciate the balance we've struck with our weekly meetings and a committee whose membership is not entirely static.
To be clear here: this is not an announcement about actually disbanding the committee. Even though I had a hand in creating it, it'd be dumb for me to unilaterally pressure the committee to disband. I may nudge and cajole (like I'm doing here), but I'm first and foremost looking to figure out what the consensus is, and then follow through on it.
Discuss.
Rob
[1] https://www.mediawiki.org/wiki/Architecture_committee [2] https://www.mediawiki.org/wiki/Architecture_meetings/Amsterdam_Hackathon_201... [3] https://www.mediawiki.org/wiki/Architecture_guidelines [4] https://www.mediawiki.org/wiki/Requests_for_comment [5] https://en.wikipedia.org/wiki/Benevolent_dictator_for_life [6] http://c2.com/cgi/wiki?ConceptualIntegrity
----- Original Message -----
From: "Rob Lanphier" robla@wikimedia.org
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
<shrug/>
:-)
Seriously: Oh ghod, please, no. I'm not a real big fan of Joel Spolsky, as some people are, but I do in general agree with his assertion that throwing everything out and starting from scratch is nearly always an unconscionable approach to anything, especially something as sizable as MediaWiki.
Cheers, -- jra
On Thu Jan 15 2015 at 8:17:25 PM Jay Ashworth jra@baylink.com wrote:
----- Original Message -----
From: "Rob Lanphier" robla@wikimedia.org
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
<shrug/>
:-)
Seriously: Oh ghod, please, no. I'm not a real big fan of Joel Spolsky, as some people are, but I do in general agree with his assertion that throwing everything out and starting from scratch is nearly always an unconscionable approach to anything, especially something as sizable as MediaWiki.
Agreed. Although it means we'll never be able to use 2.0 because "2.0" from "1.x" has total rewrite implications even if it's not true.
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
-Chad
On Thu, Jan 15, 2015 at 9:26 PM, Chad innocentkiller@gmail.com wrote:
On Thu Jan 15 2015 at 8:17:25 PM Jay Ashworth jra@baylink.com wrote:
----- Original Message -----
From: "Rob Lanphier" robla@wikimedia.org
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
<shrug/>
:-)
Seriously: Oh ghod, please, no. I'm not a real big fan of Joel Spolsky, as some people are, but I do in general agree with his assertion that throwing everything out and starting from scratch is nearly always an unconscionable approach to anything, especially something as sizable as MediaWiki.
Agreed. Although it means we'll never be able to use 2.0 because "2.0" from "1.x" has total rewrite implications even if it's not true.
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
The current versioning scheme certainly doesn't follow "semver" [0]. The deprecation page [1] (which is itself deprecated?) suggests that the major version should probably be bumped at least every two releases and I imagine that we have been changing APIs often enough that really every 1.x release is a major version release, so +1 from me.
[0]: http://semver.org/ [1]: https://www.mediawiki.org/wiki/Deprecation
Bryan
Le 16/01/2015 05:26, Chad a écrit :
Agreed. Although it means we'll never be able to use 2.0 because "2.0" from "1.x" has total rewrite implications even if it's not true.
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
That would be on par with the semver. But since we are more or less continuously releasing, I would even go to use year/month as a version. Ie: 2015.01.
We are out of topic though.
----- Original Message -----
From: "Chad" innocentkiller@gmail.com
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
Oh dear ghod, no. I already want to massacre the entire release management staff at Mozilla for being this foolish.
Just because we shouldn't *plan* for a 2.0 does not mean there never *will* be one.
Cheers, -- jra
On 01/15/2015 08:26 PM, Chad wrote:
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
+1, let's do this. It would allow us to follow semver and still retain our current version number history instead of waiting for a magical 2.0.
-- Legoktm
+1 for something like this. Its not a huge problem not to do semver but it'd be simpler to explain if we did.
On Sun, Jan 25, 2015 at 10:27 AM, Legoktm legoktm.wikipedia@gmail.com wrote:
On 01/15/2015 08:26 PM, Chad wrote:
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
+1, let's do this. It would allow us to follow semver and still retain our current version number history instead of waiting for a magical 2.0.
-- Legoktm
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Sun, Jan 25, 2015 at 1:27 PM, Legoktm legoktm.wikipedia@gmail.com wrote:
On 01/15/2015 08:26 PM, Chad wrote:
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
-1 from me, for what little that's worth...
It would allow us to follow semver and still retain our current version number history instead of waiting for a magical 2.0.
This logic is the opposite of semver. Semver says you only bump the major version number when you make a breaking change. Since breaking changes are Bad Things, therefore bumping the major version number is also a Bad Thing. It is something that you should strive to *avoid* having to do.
Under semver, a version number of the form 1.<large integer> is a *badge of honor*. It means that you have successfully executed many releases *without* needing to make a breaking change. One should display that initial 1. proudly; one should not consider it to be superfluous.
zw
On 01/25/2015 06:04 PM, Zack Weinberg wrote:
On Sun, Jan 25, 2015 at 1:27 PM, Legoktm legoktm.wikipedia@gmail.com wrote:
On 01/15/2015 08:26 PM, Chad wrote:
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
-1 from me, for what little that's worth...
It would allow us to follow semver and still retain our current version number history instead of waiting for a magical 2.0.
This logic is the opposite of semver. Semver says you only bump the major version number when you make a breaking change. Since breaking changes are Bad Things, therefore bumping the major version number is also a Bad Thing. It is something that you should strive to *avoid* having to do.
Except that every 1.x release *does* contain breaking changes. If we followed semver we would be bumping the major version, but we don't.
-- Legoktm
On 2015-01-25 6:04 PM, Zack Weinberg wrote:
On Sun, Jan 25, 2015 at 1:27 PM, Legoktm legoktm.wikipedia@gmail.com wrote:
On 01/15/2015 08:26 PM, Chad wrote:
I've been saying for over a year now we should just drop the 1. from the 1.x.y release versions. So the next release would be 25.0, 26.0, etc etc.
-1 from me, for what little that's worth...
It would allow us to follow semver and still retain our current version number history instead of waiting for a magical 2.0.
This logic is the opposite of semver. Semver says you only bump the major version number when you make a breaking change. Since breaking changes are Bad Things, therefore bumping the major version number is also a Bad Thing. It is something that you should strive to *avoid* having to do.
Under semver, a version number of the form 1.<large integer> is a *badge of honor*. It means that you have successfully executed many releases *without* needing to make a breaking change. One should display that initial 1. proudly; one should not consider it to be superfluous.
zw
Whether fortunate or not 25.0, 26.0, etc... in reality is much closer to semver than you think. We passed semver 1.x years ago.
Our releases are made over periods of 6 months or sometimes a whole year. With this period nearly every one of our releases includes at least one breaking change somewhere in the code. We even have a dedicated breaking change section in the release notes.
Semver is a nice ideal. And for most libraries it works well, since they have defined APIs and are explicitly intended to consumed by other software and versions are directly used to control problems.
But MediaWiki is an application, and a large one at that. It's consumed in a completely different way and if you were actually versioning it there are actually multiple surfaces you would want to version which are almost isolated from the actual release version number.
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
On Thu Jan 15 2015 at 8:05:18 PM Rob Lanphier robla@wikimedia.org wrote:
So, where does that leave us? Do we need a BFDL? If so, who should pick? Should it be someone in the project? Should the WMF hire someone to lead this? If not, do we keep the committee? Do we just let this be consensus based?
The thing about a BDFL is they don't tend to be appointed or picked, they just naturally emerge from the developer community. In that respect, Brion and Tim are MW's BDFLs regardless of what committee they may or may not be a member of. We could hire someone to facilitate the process perhaps, but it would take a very long time for them to be in a position to really help arbitrate any disputes that may arise. And I would be hard pressed to call them a BDFL as they just haven't been around for over a decade.
I'm a huge fan of consensus. And even though we've invested the current committee with the power to decide, they've basically let the process run by consensus as it should be. So maybe we need less of a BDFL committee and more of a group who help facilitate RfCs? Such a group could be very fluid (people joining/leaving as they have interest and time) and would probably end up with more people from various areas of expertise.
-Chad
On Thu, Jan 15, 2015 at 9:32 PM, Chad innocentkiller@gmail.com wrote:
On Thu Jan 15 2015 at 8:05:18 PM Rob Lanphier robla@wikimedia.org wrote:
So, where does that leave us? Do we need a BFDL? If so, who should pick? Should it be someone in the project? Should the WMF hire someone to lead this? If not, do we keep the committee? Do we just let this be consensus based?
The thing about a BDFL is they don't tend to be appointed or picked, they just naturally emerge from the developer community. In that respect, Brion and Tim are MW's BDFLs regardless of what committee they may or may not be a member of. We could hire someone to facilitate the process perhaps, but it would take a very long time for them to be in a position to really help arbitrate any disputes that may arise. And I would be hard pressed to call them a BDFL as they just haven't been around for over a decade.
I'm a huge fan of consensus. And even though we've invested the current committee with the power to decide, they've basically let the process run by consensus as it should be. So maybe we need less of a BDFL committee and more of a group who help facilitate RfCs? Such a group could be very fluid (people joining/leaving as they have interest and time) and would probably end up with more people from various areas of expertise.
I like the idea of a rotating committee that helps examine the designs proposed and determines when consensus has been reached. It would probably also be useful to have other support staff who can help with the more mechanical parts of the process like polling RFC authors to see if they are ready for a review meeting, handling calendaring, posting meeting minutes and following up on the status of action items generated from the meetings.
I'd honestly like to see a lot more RFCs discussed. I see RFCs as the closest thing we have to a formal design review for a software feature. I know that up front design has fallen out of favor with many software developers but personally I think that a few days or weeks of planning can save weeks or even months of implementation time for non-trivial projects.
Bryan
On Jan 15, 2015 8:33 PM, "Chad" innocentkiller@gmail.com wrote:
On Thu Jan 15 2015 at 8:05:18 PM Rob Lanphier robla@wikimedia.org wrote:
So, where does that leave us? Do we need a BFDL? If so, who should pick? Should it be someone in the project? Should the WMF hire someone to lead this? If not, do we keep the committee? Do we just let this be consensus based?
The thing about a BDFL is they don't tend to be appointed or picked, they just naturally emerge from the developer community. In that respect, Brion and Tim are MW's BDFLs regardless of what committee they may or may not be a member of.
Are they really? We have a large number of developers at WMF that have no obvious accountability to what either of them says. Neither of them has asserted that power in quite some time and frequently deny any desire to do so.
We could hire someone to facilitate the process perhaps, but it would take a very long time for them to be in a position to really help arbitrate any disputes that may arise. And I would be hard pressed to call them a BDFL as they just haven't been around for over a decade.
What if that person very quickly received the respect of the vast majority of the WMF staff developers, even if they were unpopular here on this list? Given our staff/not ratio, wouldn't our newly minted BDFL win?
I'm a huge fan of consensus. And even though we've invested the current committee with the power to decide, they've basically let the process run by consensus as it should be. So maybe we need less of a BDFL committee and more of a group who help facilitate RfCs? Such a group could be very fluid (people joining/leaving as they have interest and
time)
and would probably end up with more people from various areas of expertise.
The egalitarian aspects of this are appealing. However, that seems to leave no one really in charge of *making sure* we have forward progress. Instead, this is largely a defensive posture where people are left to figure out which way the winds are blowing, and make sure that whatever they propose is in accordance with that. No one would be specifically responsible for making sure that the RFC queue is full of high-quality, important proposals, and for the rejected proposals, no one would be accountable for coming up with alternative solutions to the problems that the important rejected RFCs were designed to address.
In projects with an effective BFDL, that person feels at least some implicit responsibility and urgency to address the problems identified by erstwhile developers who come up with a misguided solution to a real problem. By "address", I don't necessarily mean "solve", but I think there's a reasonable expectation that someone rejecting a proposal to say something to the effect of "we don't think we should solve problem A using your solution X, because we plan to tackle solution Y in a few months, which solves not only problem A, but also problems B, C, D, and E", or at least "solving problem A really isn't what MediaWiki is about. If that's really important to you, you should use some other software". If the second answer is their answer, then that person should have at least some responsibility to the forward progress of the software, and should be giving that answer because we have a clear direction we are headed in (rather than merely a comfy spot we're nestled in), and the proposed solution takes us off course or slows us down.
Rob
Hi!
I am very new here so I would probably be saying a lot of generic things which may or may not apply to the MW situation, if it is the latter please feel free to either ignore, or, much better, educate me on where I am wrong or what I am missing, but I hope it may be relevant.
So, where does that leave us? Do we need a BFDL? If so, who should pick? Should it be someone in the project? Should the WMF hire someone to lead this? If not, do we keep the committee? Do we just let this be consensus based?
I think BDFL thing hugely depends on the willingness and ability of the person(s) to do the BDFL job over the long term (the FL part :). I have no idea about MW position in this regard, but in general that is a risk. Which can be reduced by replacing a person with a process or organizational structure (which may be, initially or permanently, still including same person(s) but can change if needed without losing function). From this, having a some kind of a committee/group sounds like a good idea to me.
I'm a huge fan of consensus. And even though we've invested the current committee with the power to decide, they've basically let the process run by consensus as it should be. So maybe we need less of a BDFL
I agree on the consensus point. Extending that, I am a member of a project which has been all in on "governing by consensus" for years. The downside of it is that it starts to be chaotic unless there are means and processes of sampling the consensus, establishing the consensus and resolving the lack of consensus in a clear way that is agreeable and clear to the overwhelming majority of the participants. So the RFC process sounds good here, provided it leads to a definite result (which can be yes, no, redo from start and resubmit, etc. but it has to be clear what is going on and where it is going) and it is predictable and has defined responsibilities and procedures.
With the above, though, the architecture committees and the RFCs can serve two purposes, and it wasn't clear for me which one is the one we're looking for (or both?)
1. Technical leadership/stewardship - i.e. "implementing X a good idea because it would make MediaWiki 42% more awesome, yes or no?" I'd say most of everyday RFCs would go here.
2. Planning/prioritization/driving the project forward - i.e. "next year, we want to have X, Y and Z implemented, to lay groundwork for Foo and Bar, and also we dearly need to improve A and phase out B, because that is awful". Something like "Mediawiki 2.0" would be here, I think.
These roles are slightly different and the management interaction with these roles should IMO be different too. They may be still served by the same unit, or maybe different units with intersecting sets of people.
On Thu, Jan 15, 2015 at 8:04 PM, Rob Lanphier robla@wikimedia.org wrote:
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
Let's throw out that hypothetical, because it's too grotesque even as a conversation starter.
The model I do think we should consider is Python 3. Python 3 did not jettison the Python 2 codebase. The intent behind the major version change was to open up a parallel development track in which it was permissible to break backward-compatibility in the name of making a substantial contribution to the coherence, elegance and utility of the language.
Python 3 development started with PEP-3000[1], which Guido published in 2006, some fifteen years after the initial public release of Python. MediaWiki has been around for almost as long, and it has been subjected to rather extraordinary stressors during that time.
On Thu, Jan 15, 2015 at 9:31 PM, Ori Livneh ori@wikimedia.org wrote:
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
Let's throw out that hypothetical, because it's too grotesque even as a conversation starter.
And for the record, I agree with this. Full rewrites suck.
The model I do think we should consider is Python 3. Python 3 did not jettison the Python 2 codebase. The intent behind the major version change was to open up a parallel development track in which it was permissible to break backward-compatibility in the name of making a substantial contribution to the coherence, elegance and utility of the language.
This is a more interesting model - especially if done in parallel with radical experiments creating new workspaces for content. Imagine 1) a version of MediaWiki where a lot of stuff is ripped out ruthlessly, and new features are added more quickly, 2) which powers a site which exists for creating, say, article drafts that can be imported into Wikipedia. That's the kind of thing I get very excited about.
Erik
Ori Livneh ori@wikimedia.org writes:
The model I do think we should consider is Python 3. Python 3 did not jettison the Python 2 codebase. The intent behind the major version change was to open up a parallel development track in which it was permissible to break backward-compatibility in the name of making a substantial contribution to the coherence, elegance and utility of the language.
I like the idea, but this makes it sound like we have some commitment in the current co-debase to backwards compatibility.
Currently, though, just as Robla points out that there is no clear vision for the future, there is no clear mandate to support interfaces, or what we usually call "backwards compatibility".
So, yes, let's have a parallel MW 2.0 development track that will allow developers to try out new things. But let that be accompanied with a MW 1.0 track so that makes stability a priority.
Now, the question is: what will Wikipedia run: MW 2.0 or MW 1.0? And, if they focus on MW 2.0 (My sense is that is where the WMF devs will want to be), then how do those of us with more conservative clients keep MW 1.0 viable?
Mark.
On Jan 16, 2015 9:21 AM, "Mark A. Hershberger" mah@nichework.com wrote:
Ori Livneh ori@wikimedia.org writes:
The model I do think we should consider is Python 3. Python 3 did not jettison the Python 2 codebase. The intent behind the major version
change
was to open up a parallel development track in which it was permissible
to
break backward-compatibility in the name of making a substantial contribution to the coherence, elegance and utility of the language.
I like the idea, but this makes it sound like we have some commitment in the current co-debase to backwards compatibility.
Currently, though, just as Robla points out that there is no clear vision for the future, there is no clear mandate to support interfaces, or what we usually call "backwards compatibility".
So, yes, let's have a parallel MW 2.0 development track that will allow developers to try out new things. But let that be accompanied with a MW 1.0 track so that makes stability a priority.
Now, the question is: what will Wikipedia run: MW 2.0 or MW 1.0? And, if they focus on MW 2.0 (My sense is that is where the WMF devs will want to be), then how do those of us with more conservative clients keep MW 1.0 viable?
Mark.
-- Mark A. Hershberger NicheWork LLC 717-271-1084
This seems a solution in search of a problem. Does anyone actually have anything they want that is difficult to do currently and requires a mass compat break? Proposing to rewrite mediawiki because we can without even a notion of what we would want to do differently seems silly.
--bawolff
Brian Wolff bawolff@gmail.com writes:
Does anyone actually have anything they want that is difficult to do currently and requires a mass compat break?
I haven't heard of any "mass compat break" items. But I have seen developers who aren't worried about compatibility or continuity of features since they only have to worry about the WMF cluster and issues of compatibility are handled there, or the features aren't used by the WMF.
Before I talk about the architecture committee, let me just say that I think the idea of MediaWiki 2.0 is off-topic and should be moved to a different thread.
/ontopic
At this point I’d like to contribute a little bit of my experience from working on the Architecture Committee at Chase Auto Finance. At JPMC our committee was perhaps the most orthodox definition of an architecture committee: our authority was enforced. Yes, we were still responsible to the company and the business, and we could not simply veto new features, but nonetheless it was considered an absolute requirement that before any dev team even began _thinking_ about how to implement their new project, they submit a proposal to the architecture committee detailing how the new feature / project would affect other projects, etc.
And I think this is a pretty important concept. As Tim has mentioned, right now we don’t have an architecture committee. What we have is a group of five people that sometimes have IRC meetings and discuss stuff. Sometimes a #agreed is slapped on top, for w/e that is worth. But going through the committee is not a requirement. If Erik or whoever your’e working for gives a green light, you can go right past it. Hell, even if I myself were to do some work on my own and submit a patch directly to Gerrit, it could go through without ever seeing the committee.
If we are really worried about the future of MediaWiki and its technical debt (which, in my opinion, we should be, considering it is growing exponentially by the day), then a formal organization needs to be established. This committee should report to the WMF as usual, but it should also have an authority of its own. New features cannot be implemented without the approval of this committee. Period. +2ing a new feature that was not approved could result in you losing your +2, etc.
However, maybe this isn’t the best thing to do? As other people mentioned, consensus is a really cool thing, and sometimes it’s just better for the community to decide as whole. But I can tell you right now it’s not going to work. And the reason is this: sometimes the community becomes divided. Sometimes we can’t come to a consensus, and other times we do but the consensus is in opposition with what the WMF wants. In all of these scenarios where we cannot make up our minds, what is going to happen (or rather, what has already been happening), is that somebody in the WMF, or maybe just somebody in general, is going to be WP:BOLD and make the decision themselves. Then somebody who maybe agrees with them is going to +2 it, and the rest of the community will silently shrug and move on.
There is a place for consensus and discussion, but other times you really just need a group of people whose job (or, if you’re not hired by the WMF, responsibility) it is to make software design decisions. And if the community as a whole ever disagrees with the committee’s decision, maybe we have a process for overriding the committee (or a process for impeaching the committee members). It’s similar to the English Wikipedia: why do they have an arbitration committee, whose members are elected, in a community that doesn’t like voting and prefers consensus?
Of course, there is some discussion to be had on the exact definition and process.
* The Big Question(tm): Do we want the committee in the first place? * Who will be on this committee? * What processes are there for people to be added or removed? * What is the process for submitting ideas to the committee? * Is the committee split into sub-committees, where each sub-committee is in charge of a different topic or area of expertise? * Are sub-committees broken up based on generic concepts, e.g., databases, UX, or are they broken up based on areas of MediaWiki, e.g., Parser, FileBackend? * Is the main committee required to approve everything, or can sub-committees approve on their own unless they feel the need to defer to the main committee? * Can people who are not on the committee itself be in sub-committees? * If so how are the memberships of subcommittees selected? * What is the committee’s authority and what are its responsibilities to the WMF? * How does the committee interact with other organizations, like the MediaWiki User Group, the on-site Wikipedia community, etc.? * When do aforementioned other organizations get a say in the process?
As mentioned, MediaWiki is not an open source community at the moment. It is an engineering organization that just happens to have open source contributors. And it is because of this that our technical debt has been increasing. (Like the age-old joke that we should rewrite MediaWiki in node.js, but look, for some reason we are actually using it in parts of our new products!)
Anyway, that’s really the end of my rant. In the end, I do think a committee is necessary for the future of MediaWiki as an open source software project, but it is something that should be created with a lot of thought and planning, rather than just throwing five people into a room and telling them to get to work.
-- Tyler Romeo 0x405D34A7C86B42DF
On Fri Jan 16 2015 at 8:15:45 AM Tyler Romeo tylerromeo@gmail.com wrote:
As mentioned, MediaWiki is not an open source community at the moment. It is an engineering organization that just happens to have open source contributors.
It's hard to have a community when we keep hiring everyone out of it.
-Chad
On January 16, 2015 at 11:59:14, Chad (innocentkiller@gmail.com) wrote: It's hard to have a community when we keep hiring everyone out of it. We should use this as an advertising point. “Come contribute to MediaWiki, there’s a pretty high percentage we’ll hire you.” :P
-- Tyler Romeo 0x405D34A7C86B42DF
Rob Lanphier wrote:
So, here we are today. I believe no one would dispute the credentials of every member of the group. Brion, Tim, and Mark have an extremely long history with the project, being employees #1, #2, and #3 of the WMF respectively, and all having contributed massively to the success of Wikipedia and to MediaWiki as general purpose wiki software. In most open source projects, one of them would probably be BFDL[5]. Roan and Daniel are more "recent", but only in relative terms, and also have very significant contributions to their name. All have the widespread respect of pretty much everyone in the MediaWiki developer community.
Yes, absolutely. My general thoughts:
* I really like and respect the five people on the committee, but from my perspective, it's only been one person actively involved most often.
** Perhaps the weekly meeting time is bad. ** Perhaps people on the committee can't commit to the necessary time and we need to find other trusted, senior developers.
* As the number of projects and size of Wikimedia development grows, it's even more important to have people looking at the big picture.
* The weekly IRC meetings are hugely valuable. Whether or not we continue to have an Architecture Committee, we should continue holding these weekly meetings.
** The scope of these weekly meetings should always be clearly defined, but it doesn't need to be limited to purely RFCs, per se. As this most recent week's meeting demonstrated, not every discussable idea must be shoehorned into the "Requests for comment/" pseudo-namespace (not that there's anything wrong with that). ** We need more people involved in discussions about and scrutinizing the components of big ideas (including people like Rob!).
I'm getting increasingly concerned that really smart people such as Brion and Roan and others are not looking at and poking holes in the larger ideas being proposed. (Or, more worryingly, bigger ideas aren't even being proposed and instead are skipping straight into implementation.) Architectural review is most definitely needed in some areas and a lack of upfront discussion and criticism and debate is going to result in a lot of future pain. Poor planning leads to poor execution. Doing everything twice (badly the first time, better the second) is really costly and we simply have too many ideas to be paying double for them.
Instead, we need to find the larger patterns in order to build the most modular and scalable solutions. This requires lots and lots of thinking.
Still, the uncomfortable shrugging continues. The group is broader, but still lacks the breadth, particularly in front end and in the development of newer services such as Parsoid and RESTBase. This aspect is pretty obviously something that can be fixed. Another problem is that the scope of the group isn't clear to everyone. Is this group responsible for leading, or merely responsible for reviewing big ideas from others to ensure continuity and sanity? How big does an idea need to be before an RFC needs to be written (as opposed to just dropping a patch in Gerrit)? Defining the scope of the group is also a fixable problem.
Thanks for expressing two problems that you see. For the first, we can always add people if certain areas are lacking. I agree that having someone a bit closer to the design/front-end side would be good, though I'm not sure how many trusted, senior people we have in that area. The scope of the group is to discuss concrete, cool ideas for Wikimedia development and try to move them forward. Some of these ideas are supported by the Wikimedia Foundation, others are supported by volunteer developers, but that's irrelevant as it's a forum for ideas alone.
Architectural guidance is hugely important in technical projects, so having a weekly forum to hash out ideas and look for potential security or performance or design or community concerns is a Very Good Thing, in my opinion. I don't think this is a hard sell.
However, I don't sense much of a desire to fix things. The dominant meme that I hear is that we should go back to the day before uncomfortable shrugging led to a committee becoming BFDL. What I fear, though, is that we will develop a system lacking in conceptual integrity[6], as individual warring fiefdoms emerge. It's quite simple to argue this is already happening.
Right, I think this is the third problem you've identified. We're definitely already seeing the formation of fiefdoms. This should be addressed. Requiring architectural sign-off for large projects is worth at least considering. We currently require security and performance sign-off prior to deployment to Wikimedia wikis. Requiring architectural sign-off (much earlier than deployment, obviously!) wouldn't be unprecedented.
Team-level pride in fantastic work drifts into project-level despair, as many excellent engineers fail to grasp how to make big changes outside of their limited domains.
Respectfully, this is garbage. Excellent engineers don't have a problem moving their ideas forward. Why? Because excellent engineers carefully study and interact with a project until they're comfortable with it and secure in the knowledge that they understand it well enough to improve, rather than hurt, the project. The people who struggle to move their ideas forward are the ones who don't understand wiki projects or wiki culture. It's really difficult to build tools and scripts and interfaces for projects that you don't use and that you don't understand.
Perhaps I'm also suffering from living inside the WMF echo chamber for too long. It could be that the general pessimism about the direction of MediaWiki (or lack thereof) is not shared out here.
What general pessimism? I don't see evidence to support such a claim.
MZMcBride
On 16/01/15 15:04, Rob Lanphier wrote:
Still, the uncomfortable shrugging continues. The group is broader, but still lacks the breadth, particularly in front end and in the development of newer services such as Parsoid and RESTBase.
It appears that we won't be able to keep the members we have, let alone broaden our membership. Mark has said that it's not worth his time, Brion hasn't attended a committee meeting since November, Daniel has given hints that his involvment might not continue, and Roan has been deeply skeptical from the outset. I think I am the only one who is committed to it, and that is out of a sense of duty rather than rational reflection.
The problem is that the work is mostly administrative and not empowered. Committee members are skeptical of many of the current ideas floating around at the moment, and have their own ideas about what things should be priorities, but have no expectation that those ideas will be considered for resourcing.
We review the technical details of design proposals, but I think most committee members do not find that to be engaging. We've all reviewed things before, and will presumably continue to do so regardless of whether we are on a committee. We could veto technical details as individuals, so what is the committee for?
I believe no one would dispute the credentials of every member of the group. Brion, Tim, and Mark have an extremely long history with the project, being employees #1, #2, and #3 of the WMF respectively, and all having contributed massively to the success of Wikipedia and to MediaWiki as general purpose wiki software. In most open source projects, one of them would probably be BFDL[5]. Roan and Daniel are more "recent", but only in relative terms, and also have very significant contributions to their name.
It's not a community open source project, it is an engineering organisation with a strict hierarchy. We don't have a BDFL, we have a VPE.
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
Sorry to labour the point, but the way to go about this at present is pretty straightforward, and it doesn't involve the architecture committee. You just convince the management (Damon, Erik, etc.) that it is a good thing to do, get yourself appointed head of the "MediaWiki 2.0" team, hire a bunch of people who agree with your outlook, get existing engineers transferred to your team. It's not even hypothetical, we've seen this pattern in practice.
-- Tim Starling
On Thu, Jan 15, 2015 at 10:57 PM, Tim Starling tstarling@wikimedia.org wrote:
Sorry to labour the point, but the way to go about this at present is pretty straightforward, and it doesn't involve the architecture committee. You just convince the management (Damon, Erik, etc.) that it is a good thing to do, get yourself appointed head of the "MediaWiki 2.0" team, hire a bunch of people who agree with your outlook, get existing engineers transferred to your team.
Yeah, there's a lot of truth to that (though also a lot of opportunities to circumvent organizational structure). WMF is a hierarchical org that operates internally through pretty conventional decision-making structures. On the flip side, the org has increasingly pushed to give individuals a very high degree of latitude of pushing and promoting projects, and leading them to conclusion (hence the project leads on the top priorities and such).
Within the organizational pattern we use, the way to complement Damon's and my roles is usually with a CTO who has deep technical experience & commitment and ongoing day-to-day involvement writing code, leading projects, and driving architectural change. That person may have some of the most senior engineers in the org reporting to them, and has a serious seat at the table in driving projects that satisfy highly technical concerns. I'm supportive of such a role (if properly defined and socialized), because in the model we're operating in, it seems one of the best ways to complement the team structure.
I'm also supportive of experimenting with less conventional models, like creating more official representation for an Architecture Committee in management decisions. Gravity is going to pull us more towards the conventional ways of doing things (it always does), so if you want to promote a different idea we need to start articulating and refining it.
Erik
On Thu, Jan 15, 2015 at 10:57 PM, Tim Starling tstarling@wikimedia.org wrote:
Committee members are skeptical of many of the current ideas floating around at the moment, and have their own ideas about what things should be priorities, but have no expectation that those ideas will be considered for resourcing.
Keeping these thoughts to yourselves doesn't help. Please speak up.
On Thu, Jan 15, 2015 at 10:57 PM, Tim Starling tstarling@wikimedia.org wrote:
On 16/01/15 15:04, Rob Lanphier wrote:
I believe no one would dispute the credentials
of every member of the group. Brion, Tim, and Mark have an extremely
long history with the project, being employees #1, #2, and #3 of the WMF respectively, and all having contributed massively to the success of Wikipedia and to MediaWiki as general purpose wiki software. In most open source projects, one of them would probably be BFDL[5]. Roan and Daniel are more "recent", but only in relative terms, and also have very significant contributions to their name.
It's not a community open source project, it is an engineering organisation with a strict hierarchy. We don't have a BDFL, we have a VPE.
Alrighty, Tim, I have a great deal of respect for you, and all of the good things I've said about you in this thread are objective facts, not me buttering you up. But this is an area where a lot of people are dying for you to step up, and have been for some time.
As someone who has been your manager for several years now, I have to openly giggle at the "strict hierarchy" characterization :-)
On the leadership front, let me throw out a hypothetical: should we
have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
Sorry to labour the point, but the way to go about this at present is pretty straightforward, and it doesn't involve the architecture committee. You just convince the management (Damon, Erik, etc.) that it is a good thing to do, get yourself appointed head of the "MediaWiki 2.0" team, hire a bunch of people who agree with your outlook, get existing engineers transferred to your team. It's not even hypothetical, we've seen this pattern in practice.
All it would take is for you to muse out loud on this list "hey, what is up with 'MediaWiki 2.0'?" (or whatever it is), and we could have a productive conversation about that. If the argument for its potentially misguided nature is made with respect and kindness, it will be welcomed.
I've clearly touched a nerve here. I knew I was risking that, and I'm sorry that I didn't find the exact way to raise the points I needed to raise without doing so. But I'm not sorry I started this conversation. There's a lot of folks here at the organization who have been agreeing to disagree for waaaaay too long, and we've gotta come clean here.
For you or anyone on my team (or for that matter, anyone) who wants help figuring out how to have a productive conversation on wikitech-l about important issues, I'm happy to help out.
Rob
On 2015-01-15 11:41 PM, Rob Lanphier wrote:
On the leadership front, let me throw out a hypothetical: should we
have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
Sorry to labour the point, but the way to go about this at present is pretty straightforward, and it doesn't involve the architecture committee. You just convince the management (Damon, Erik, etc.) that it is a good thing to do, get yourself appointed head of the "MediaWiki 2.0" team, hire a bunch of people who agree with your outlook, get existing engineers transferred to your team. It's not even hypothetical, we've seen this pattern in practice.
All it would take is for you to muse out loud on this list "hey, what is up with 'MediaWiki 2.0'?" (or whatever it is), and we could have a productive conversation about that. If the argument for its potentially misguided nature is made with respect and kindness, it will be welcomed.
Any such discussion should probably start with this: https://www.mediawiki.org/wiki/MediaWiki_2.0
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
On Thu, Jan 15, 2015 at 11:51 PM, Daniel Friesen <daniel@nadir-seen-fire.com
wrote:
Any such discussion should probably start with this: https://www.mediawiki.org/wiki/MediaWiki_2.0
The page is an outline of a plan. It sounds pretty good to me. Whatever happened to that initiative? Did it just fizzle out, or were there substantive disagreements to work out?
Rob, thanks for starting this discussion!
We definitely should be thinking about what kind of structures make sense; the arch committee was a good "spike solution" which I think we've determined doesn't quite work as it is, but the core idea of getting some active people to talk to each other regularly and go over stuff is a good impulse we should not discard!
I'm currently in the process of disentangling myself from other projects (we've got some awesome stuff coming up in the mobile apps!) to concentrate more on MediaWiki core, so ideas about modifying or adding to that structure are very welcome now.
In general, I feel a sense of urgency that seems lacking in the status
quo. We've made progress over the past couple of years, but it doesn't feel like our progress is entirely up to the task. We have a collection of *many* instances of individual or small team excellence that are sadly the mere sum of their parts. My intuition is that we lose out on multiplicative effects as we fail to engage the wider group in our activities, and as we lack engineering-level orchestration. Team-level pride in fantastic work drifts into project-level despair, as many excellent engineers fail to grasp how to make big changes outside of their limited domains.
^ This.
I think it's getting time to plan more aggressively for the future and work towards our goals with a shared vision. That's a tall order, of course, but I think we can make some ideas go around...
-- brion
On Thu, Jan 15, 2015 at 8:04 PM, Rob Lanphier robla@wikimedia.org wrote:
(Alright...let's try this again!)
Hi everyone,
The current MediaWiki Architecture Committee[1] has its roots in a 2013 Amsterdam Hackathon session[2], where we had a pair of sessions to try to establish our architectural guidelines[3]. It was there that we agreed that it would be good to revive our then moribund process for reviewing RFCs[4]. Since no one there really knew whose job it was to review these things, I believe I said "how about we start with everyone with 'architect' in their title at WMF?", which was met with uncomfortable shrugging that I interpreted as "consensus!", and no one corrected me. Thus Brion Vibber, Mark Bergsma, and Tim Starling became the founding members of the Arch Committee.
Subsequent to that meeting, I pretended to proceed as though a decision was made. However, over the past year and half since then, there's been much more uncomfortable shrugging. Even Brion, Mark, and Tim have not seemed entirely comfortable with the idea. It was widely acknowledged that the group was heavily biased toward the lower parts of our server software stack. The committee agreed to add Roan Kattouw and Daniel Kinzler to the group as a means of providing a wider perspective, with the added bonus of adding at least one person who isn't a WMF employee.
So, here we are today. I believe no one would dispute the credentials of every member of the group. Brion, Tim, and Mark have an extremely long history with the project, being employees #1, #2, and #3 of the WMF respectively, and all having contributed massively to the success of Wikipedia and to MediaWiki as general purpose wiki software. In most open source projects, one of them would probably be BFDL[5]. Roan and Daniel are more "recent", but only in relative terms, and also have very significant contributions to their name. All have the widespread respect of pretty much everyone in the MediaWiki developer community.
Additionally, I hear quite a bit of relief that the previously moribund RFC process is doing much better now. Things are moving, and if you know how to work the process and aren't proposing anything too wild, you can get an RFC approved pretty quickly. The committee has made a lap through the entire backlog of RFCs.
Still, the uncomfortable shrugging continues. The group is broader, but still lacks the breadth, particularly in front end and in the development of newer services such as Parsoid and RESTBase. This aspect is pretty obviously something that can be fixed. Another problem is that the scope of the group isn't clear to everyone. Is this group responsible for leading, or merely responsible for reviewing big ideas from others to ensure continuity and sanity? How big does an idea need to be before an RFC needs to be written (as opposed to just dropping a patch in Gerrit)? Defining the scope of the group is also a fixable problem.
However, I don't sense much of a desire to fix things. The dominant meme that I hear is that we should go back to the day before uncomfortable shrugging led to a committee becoming BFDL. What I fear, though, is that we will develop a system lacking in conceptual integrity[6], as individual warring fiefdoms emerge. It's quite simple to argue this is already happening.
So, where does that leave us? Do we need a BFDL? If so, who should pick? Should it be someone in the project? Should the WMF hire someone to lead this? If not, do we keep the committee? Do we just let this be consensus based?
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
In general, I feel a sense of urgency that seems lacking in the status quo. We've made progress over the past couple of years, but it doesn't feel like our progress is entirely up to the task. We have a collection of *many* instances of individual or small team excellence that are sadly the mere sum of their parts. My intuition is that we lose out on multiplicative effects as we fail to engage the wider group in our activities, and as we lack engineering-level orchestration. Team-level pride in fantastic work drifts into project-level despair, as many excellent engineers fail to grasp how to make big changes outside of their limited domains.
Perhaps I'm being too hyperbolic. Perhaps the answer is "embrace the chaos; it's the Wiki Way(tm)" I don't buy it, but I'm probably one of the easier people to convince of this. I think if this is the way it's gonna be, someone needs to make the case how this is actually working now. Step up.
Perhaps I'm also suffering from living inside the WMF echo chamber for too long. It could be that the general pessimism about the direction of MediaWiki (or lack thereof) is not shared out here. Perhaps people who get most of their news from this mailing list are perfectly happy with the status quo, and appreciate the balance we've struck with our weekly meetings and a committee whose membership is not entirely static.
To be clear here: this is not an announcement about actually disbanding the committee. Even though I had a hand in creating it, it'd be dumb for me to unilaterally pressure the committee to disband. I may nudge and cajole (like I'm doing here), but I'm first and foremost looking to figure out what the consensus is, and then follow through on it.
Discuss.
Rob
[1] https://www.mediawiki.org/wiki/Architecture_committee [2] https://www.mediawiki.org/wiki/Architecture_meetings/Amsterdam_Hackathon_201... [3] https://www.mediawiki.org/wiki/Architecture_guidelines [4] https://www.mediawiki.org/wiki/Requests_for_comment [5] https://en.wikipedia.org/wiki/Benevolent_dictator_for_life [6] http://c2.com/cgi/wiki?ConceptualIntegrity
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
It's really exciting to see people let go of unnecessary authority, dismantle bureaucracy and resist building empires. I applaud the restraint the committee is using here. I think it speaks volumes about who they are as leaders.
As Brion suggests, it's important to retain the "idea of getting some active people to talk to each other regularly and go over stuff". I look forward to supporting grass-roots efforts to achieve just that.
- Trevor
On Fri, Jan 16, 2015 at 10:01 AM, Brion Vibber bvibber@wikimedia.org wrote:
Rob, thanks for starting this discussion!
We definitely should be thinking about what kind of structures make sense; the arch committee was a good "spike solution" which I think we've determined doesn't quite work as it is, but the core idea of getting some active people to talk to each other regularly and go over stuff is a good impulse we should not discard!
I'm currently in the process of disentangling myself from other projects (we've got some awesome stuff coming up in the mobile apps!) to concentrate more on MediaWiki core, so ideas about modifying or adding to that structure are very welcome now.
In general, I feel a sense of urgency that seems lacking in the status
quo. We've made progress over the past couple of years, but it doesn't feel like our progress is entirely up to the task. We have a collection of *many* instances of individual or small team excellence that are sadly the mere sum of their parts. My intuition is that we lose out on multiplicative effects as we fail to engage the wider group in our activities, and as we lack engineering-level orchestration. Team-level pride in fantastic work drifts into project-level despair, as many excellent engineers fail to grasp how to make big changes outside of their limited domains.
^ This.
I think it's getting time to plan more aggressively for the future and work towards our goals with a shared vision. That's a tall order, of course, but I think we can make some ideas go around...
-- brion
On Thu, Jan 15, 2015 at 8:04 PM, Rob Lanphier robla@wikimedia.org wrote:
(Alright...let's try this again!)
Hi everyone,
The current MediaWiki Architecture Committee[1] has its roots in a 2013 Amsterdam Hackathon session[2], where we had a pair of sessions to try to establish our architectural guidelines[3]. It was there that we agreed that it would be good to revive our then moribund process for reviewing RFCs[4]. Since no one there really knew whose job it was to review these things, I believe I said "how about we start with everyone with 'architect' in their title at WMF?", which was met with uncomfortable shrugging that I interpreted as "consensus!", and no one corrected me. Thus Brion Vibber, Mark Bergsma, and Tim Starling became the founding members of the Arch Committee.
Subsequent to that meeting, I pretended to proceed as though a decision was made. However, over the past year and half since then, there's been much more uncomfortable shrugging. Even Brion, Mark, and Tim have not seemed entirely comfortable with the idea. It was widely acknowledged that the group was heavily biased toward the lower parts of our server software stack. The committee agreed to add Roan Kattouw and Daniel Kinzler to the group as a means of providing a wider perspective, with the added bonus of adding at least one person who isn't a WMF employee.
So, here we are today. I believe no one would dispute the credentials of every member of the group. Brion, Tim, and Mark have an extremely long history with the project, being employees #1, #2, and #3 of the WMF respectively, and all having contributed massively to the success of Wikipedia and to MediaWiki as general purpose wiki software. In most open source projects, one of them would probably be BFDL[5]. Roan and Daniel are more "recent", but only in relative terms, and also have very significant contributions to their name. All have the widespread respect of pretty much everyone in the MediaWiki developer community.
Additionally, I hear quite a bit of relief that the previously moribund RFC process is doing much better now. Things are moving, and if you know how to work the process and aren't proposing anything too wild, you can get an RFC approved pretty quickly. The committee has made a lap through the entire backlog of RFCs.
Still, the uncomfortable shrugging continues. The group is broader, but still lacks the breadth, particularly in front end and in the development of newer services such as Parsoid and RESTBase. This aspect is pretty obviously something that can be fixed. Another problem is that the scope of the group isn't clear to everyone. Is this group responsible for leading, or merely responsible for reviewing big ideas from others to ensure continuity and sanity? How big does an idea need to be before an RFC needs to be written (as opposed to just dropping a patch in Gerrit)? Defining the scope of the group is also a fixable problem.
However, I don't sense much of a desire to fix things. The dominant meme that I hear is that we should go back to the day before uncomfortable shrugging led to a committee becoming BFDL. What I fear, though, is that we will develop a system lacking in conceptual integrity[6], as individual warring fiefdoms emerge. It's quite simple to argue this is already happening.
So, where does that leave us? Do we need a BFDL? If so, who should pick? Should it be someone in the project? Should the WMF hire someone to lead this? If not, do we keep the committee? Do we just let this be consensus based?
On the leadership front, let me throw out a hypothetical: should we have MediaWiki 2.0, where we start with an empty repository and build up? If so, who makes that decision? If not, what is our alternative vision? Who is going to define it? Is what we have good enough?
In general, I feel a sense of urgency that seems lacking in the status quo. We've made progress over the past couple of years, but it doesn't feel like our progress is entirely up to the task. We have a collection of *many* instances of individual or small team excellence that are sadly the mere sum of their parts. My intuition is that we lose out on multiplicative effects as we fail to engage the wider group in our activities, and as we lack engineering-level orchestration. Team-level pride in fantastic work drifts into project-level despair, as many excellent engineers fail to grasp how to make big changes outside of their limited domains.
Perhaps I'm being too hyperbolic. Perhaps the answer is "embrace the chaos; it's the Wiki Way(tm)" I don't buy it, but I'm probably one of the easier people to convince of this. I think if this is the way it's gonna be, someone needs to make the case how this is actually working now. Step up.
Perhaps I'm also suffering from living inside the WMF echo chamber for too long. It could be that the general pessimism about the direction of MediaWiki (or lack thereof) is not shared out here. Perhaps people who get most of their news from this mailing list are perfectly happy with the status quo, and appreciate the balance we've struck with our weekly meetings and a committee whose membership is not entirely static.
To be clear here: this is not an announcement about actually disbanding the committee. Even though I had a hand in creating it, it'd be dumb for me to unilaterally pressure the committee to disband. I may nudge and cajole (like I'm doing here), but I'm first and foremost looking to figure out what the consensus is, and then follow through on it.
Discuss.
Rob
[1] https://www.mediawiki.org/wiki/Architecture_committee [2]
https://www.mediawiki.org/wiki/Architecture_meetings/Amsterdam_Hackathon_201...
[3] https://www.mediawiki.org/wiki/Architecture_guidelines [4] https://www.mediawiki.org/wiki/Requests_for_comment [5] https://en.wikipedia.org/wiki/Benevolent_dictator_for_life [6] http://c2.com/cgi/wiki?ConceptualIntegrity
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Jan 16, 2015 at 5:04 AM, Rob Lanphier robla@wikimedia.org wrote:
Do we need a BFDL?
On Fri, Jan 16, 2015 at 8:18 AM, Erik Moeller erik@wikimedia.org wrote:
Within the organizational pattern we use, the way to complement Damon's and my roles is usually with a CTO
On Fri, Jan 16, 2015 at 7:57 AM, Tim Starling tstarling@wikimedia.org wrote:
The problem is that the (Architecture Committee) work is mostly administrative and not empowered. Committee members are skeptical of many of the current ideas floating around at the moment, and have their own ideas about what things should be priorities, but have no expectation that those ideas will be considered for resourcing.
(...) It's not a community open source project, it is an engineering
organisation with a strict hierarchy. We don't have a BDFL, we have a VPE.
We need a healthy MediaWiki open source software project in the first place. Without this, no committee, dictator or CTO will save the project in the long run. And Wikimedia works for the long run.
Wikimedia is a movement and MediaWiki is an open source software project. While the Wikimedia Foundation plays a key role in the development of MediaWiki, the structure of the open source project must be driven by community merit. The MediaWiki platform roadmap should be discussed and agreed at a community level. Resourcing should be a consequence of technical agreement, not the cause. Project roles should be a consequence of project merit, not the cause. The WMF Engineering structure should support and complement the MediaWiki OSS community structure, not replace it.
The five people forming the Architecture Committee are in a privileged position, receiving the support from the MediaWiki developer community and the WMF Engineering management. I don't think dismantling this team is a good idea both in terms of open source project and WMF competence and health, no matter how problematic the current situation is. The rest of us should help moving the current situation forward, not backward.
Tim identifies lack of empowerment and top-down WMF management as key problems. Opinions might differ interpreting the current situation, but I guess all of us agree that an architecture committee can only work if they are empowered to be co-pilots of the MediaWiki platform roadmap.
However, it is true that the current situation is not sustainable. The ArchCom is not directly active in the WMF quarterly planning (some of its members take part, but they do it mainly because of their WMF individual roles). Meanwhile, most of the ArchCom work goes to administrative work and RfC discussions that are not among the top priorities of the project. A missed opportunity, certainly.
The first steps to fix this problem could be:
* Identification of key MediaWiki platform areas and their architects / maintainers. Currently "MediaWiki Core" has no less than 210 components identified at https://www.mediawiki.org/wiki/Developers/Maintainers . We should have just a handful of areas covering all these components, with owners identified and empowered to keep their area tidy and build the roadmap with the rest of architects.
* Prioritization of architecture topics, so we all agree at least on what is urgent/important. https://phabricator.wikimedia.org/tag/architecture/ and https://phabricator.wikimedia.org/tag/mw-rfcs/ are a step forward, but they still don't reflect the priorities needed.
If the current ArchCom grows to include the architects of MediaWiki areas, and if this wider team gets regularly involved in prioritization of architecture tasks, RfC discussions, and WMF platform planning... probably we will go in the direction we want this big software project to go. Only then I would start discussing the need of a BFDL or a CTO. Considering the appointment of one of these roles before addressing the core of the problem would put us more in a compromise, imho.
Quim Gil qgil@wikimedia.org writes:
We need a healthy MediaWiki open source software project in the first place. Without this, no committee, dictator or CTO will save the project in the long run.
Agreed.
While the Wikimedia Foundation plays a key role in the development of MediaWiki, the structure of the open source project must be driven by community merit. The MediaWiki platform roadmap should be discussed and agreed at a community level.
Thank you for saying this.
You also re-iterated Tim's point about ArchCom's powerlessness and pointed to some ways to address that.
I would also suggest that an effort be made to find community members who are not WMF employees to participate in the ArchCom and then to have their voices heard during in quarterly planning.
You also suggested that we "[identify] key MediaWiki platform areas and their architects / maintainers".
If we look at the whole MW ecosystem, I sure we'll find that some key platform areas have maintainers who are not WMF employees.
Mark.
I would also suggest that an effort be made to find community members who are not WMF employees to participate in the ArchCom and then to have their voices heard during in quarterly planning.
I dont know if this is practical. As Chad noted earlier, WMF hires the best and the brightest. Even if the entire arch comitte was hit by a bus, the people who i think would logically be next in line are still employed by wmf. Even if those people got hit by a bus, i still think their logical replacements would largely be wmf employees.
--bawolff
I think that’s kind of insulting to those of us who don’t work at the WMF. Just because they hire the “best and the brightest” does not mean there are not people out there who are just as intelligent, if not more, but do not or cannot work for the WMF for whatever reason. Restricting Archcom to WMF employees is just about the stupidest thing you could do for an open source software project. It defeats the entire purpose of MediaWiki being open-source.
-- Tyler Romeo 0x405D34A7C86B42DF
On January 22, 2015 at 06:31:29, Brian Wolff (bawolff@gmail.com) wrote:
I dont know if this is practical. As Chad noted earlier, WMF hires the best and the brightest. Even if the entire arch comitte was hit by a bus, the people who i think would logically be next in line are still employed by wmf. Even if those people got hit by a bus, i still think their logical replacements would largely be wmf employees.
Tyler Romeo tylerromeo@gmail.com writes:
Just because they hire the “best and the brightest” does not mean there are not people out there who are just as intelligent, if not more, but do not or cannot work for the WMF for whatever reason. Restricting Archcom to WMF employees is just about the stupidest thing you could do for an open source software project.
Exactly. It ensures that those of us using or supporting MediaWiki outside of the WMF have no voice in the direction of MW.
I am not qualified for the ArchCom, but I am familiar with several engineers working in government or running private wiki farms or their own businesses who I think would be qualified.
I think the WMF has hired the people it can hire, but that doesn't mean all ArchCom-level knowledge is found only in Foundation employees.
Thinking that only the WMF has high-level MW knowledge and ability is a myopic view that threatens the future of MediaWiki as software that is useful outside of the WMF.
Mark.
Mark A. Hershberger wrote:
I am not qualified for the ArchCom, but I am familiar with several engineers working in government or running private wiki farms or their own businesses who I think would be qualified.
Another qualification I'd look for is substantial Wikimedia community involvement. Architecting requires a very thorough understanding of who you're building for. (I recently raised this same issue on the design mailing list related to editing Wikipedia... I think it's very difficult to build great tools for a community that you're not a part of.)
Thankfully, we have lots of good candidates who are both brilliant and understand Wikimedia (Domas, Magnus, and Platonides come to mind off-hand).
The trickier part of having non-employees on a committee like this is that other people won't be getting paid to participate and may not have as much time to commit as a result. It's still a bit unclear to me how much work is involved on a weekly basis. If it's just an hour-long IRC session, that's a lot different than needing ten hours or more every week.
MZMcBride
For what is worth, nobody at the WMF is suggesting any kind of Architecture Committee discrimination or promotion based on affiliation. Can we please discuss the idea of nominating architects per area? Is there any good reason not to do it?
Yesterday I had a chance to speak with Tim, trying to get a specific action for the Engineering Community team in the middle of this dense and complex topic. So here is a simple one:
Create a high level overview of the MediaWiki architecture https://phabricator.wikimedia.org/T1287
Your feedback and help is welcome. There are two possible ways of approaching this:
1. experts defining a list of core areas and key extensions 2. non-experts doing the same and upsetting the experts, who step in
If 1 doesn't work, I'll advocate for 2, with a risk of starting with the list myself. :) Please don't let me.
On Thu, Jan 22, 2015 at 7:31 PM, MZMcBride z@mzmcbride.com wrote:
Mark A. Hershberger wrote:
I am not qualified for the ArchCom, but I am familiar with several engineers working in government or running private wiki farms or their own businesses who I think would be qualified.
Another qualification I'd look for is substantial Wikimedia community involvement. Architecting requires a very thorough understanding of who you're building for. (I recently raised this same issue on the design mailing list related to editing Wikipedia... I think it's very difficult to build great tools for a community that you're not a part of.)
Thankfully, we have lots of good candidates who are both brilliant and understand Wikimedia (Domas, Magnus, and Platonides come to mind off-hand).
The trickier part of having non-employees on a committee like this is that other people won't be getting paid to participate and may not have as much time to commit as a result. It's still a bit unclear to me how much work is involved on a weekly basis. If it's just an hour-long IRC session, that's a lot different than needing ten hours or more every week.
MZMcBride
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Quim Gil qgil@wikimedia.org writes:
Can we please discuss the idea of nominating architects per area? Is there any good reason not to do it?
There is no good reason not to do this. We should do it.
Mark.
[...] Perhaps people who get most of their news from this mailing list are [...]
Why does this matter? What other sources are there?
It seems you're saying there is a divide between the main development community (on this list) and some other unspecified group which communicates by some other unspecified means. If so, what you need is not a dictator or an architect or a process, it's a reunification.
Whoever has knowledge of such a divide can help with simple steps, in particular by publishing a "map" outlining this divide on a mediawiki.org page.
[...] the vast majority of the WMF staff developers, even if they were unpopular here on this list? Given our staff/not ratio [...]
See above.
[...] It could be that the general pessimism about the direction of MediaWiki (or lack thereof) is not shared out here. [...]
See above.
have their own ideas about what things should be priorities, but have no expectation that those ideas will be considered for resourcing.
The solution here seems rather straightforward, if not simple: ensure the architecture committee actually feels empowered. The WMF could promise to not invest resources on something that the committee officially declared a bad idea, for instance.
Nemo
wikitech-l@lists.wikimedia.org