Hi folks,
this is a heads-up that Tomasz, Rob, Alolita, CT, Brion, Roan[TBC], Howie and I will be meeting with ThoughtWorks ( http://en.wikipedia.org/wiki/ThoughtWorks ) Thursday and Friday. Rob and I also got a download from Tim ahead of time.
ThoughtWorks has done great work with the fundraising developers (Arthur, Katie, Kaldari) already in helping protect that team from distractions, prioritize its work, and ensure that roles/responsibilities are clearly understood vis-a-vis the fundraising folks in the community department. So, we're now exploring a deeper engagement with ThoughtWorks on problem-solving across the engineering organization.
Following initial conversations, the purpose of the meeting tomorrow is to do a deep-dive into a specific problem set: the code review, deployment and release management process. We'll be digging into questions like:
1) What are the best methods to ensure we keep up with the backlog while still maintaining a good clip of WMF priority development; 2) What's a realistic deployment and release cycle to shoot for (for trunk, extensions, branches); 3) How do we dissipate key skills more widely among both staff and volunteers (e.g. deployment, security reviews); 4) How/when can we split "big hairy projects" with integration issues into more manageable chunks; 5) What other high priority improvements do we need to prioritize (e.g. increased test coverage, improvements to the testing frameworks, het-deploy, staging environments, etc.)
We're intentionally keeping this first meeting at a manageable size to have a high face-to-face throughput and to explore where ThoughtWorks can best help us. But I'm very much intending to make our thinking public, and to form clear and visible groups around core problems we're tackling, just as we have been doing with all WMF engineering projects. So, I'll keep you posted, and if you have thoughts that you'd like to post ahead of time, please do it onlist or offlist :-)
Thanks, Erik
Erik Moeller wrote:
Following initial conversations, the purpose of the meeting tomorrow is to do a deep-dive into a specific problem set: the code review, deployment and release management process. We'll be digging into questions like:
- What are the best methods to ensure we keep up with the backlog
while still maintaining a good clip of WMF priority development; 2) What's a realistic deployment and release cycle to shoot for (for trunk, extensions, branches); 3) How do we dissipate key skills more widely among both staff and volunteers (e.g. deployment, security reviews); 4) How/when can we split "big hairy projects" with integration issues into more manageable chunks; 5) What other high priority improvements do we need to prioritize (e.g. increased test coverage, improvements to the testing frameworks, het-deploy, staging environments, etc.)
We're intentionally keeping this first meeting at a manageable size to have a high face-to-face throughput and to explore where ThoughtWorks can best help us. But I'm very much intending to make our thinking public, and to form clear and visible groups around core problems we're tackling, just as we have been doing with all WMF engineering projects. So, I'll keep you posted, and if you have thoughts that you'd like to post ahead of time, please do it onlist or offlist :-)
Thanks very much for the heads-up. Posting the minutes or any notes from the meeting on Meta-Wiki or mediawiki.org would be fantastic.
I might say that one more point to focus on specifically is to how to leverage volunteer development (this is hinted at in some of your five points). There are _a lot_ of people who are capable of coding in PHP and who are willing to donate their time and talents, but Wikimedia/MediaWiki code development has chased them off, generally through neglect (patches sitting, review sitting, etc.). If there are ways to specifically look at that, it would be an enormous benefit to Wikimedia/MediaWiki, I think.
Much like the wikis themselves, you can rely on people to do good work if there's a support structure in place. The only caution is that, of course, if patches and commits increase without additional review/deployment support, you just make the problem worse (backlogs and frustration grow). Still, it seems that a huge issue is that volunteer time and talent is being wasted, so I think that's a fairly high priority (or at least should be).
I'm really glad that these issues are being looked at. I know that a lot of people in the community (end-users and people related to the software development) have become frustrated with the speed of code review/development/time-to-new-features, so any step in a better direction is fantastic. :-)
MZMcBride
On Thu, Jul 7, 2011 at 12:40 PM, MZMcBride z@mzmcbride.com wrote:
Thanks very much for the heads-up. Posting the minutes or any notes from the meeting on Meta-Wiki or mediawiki.org would be fantastic.
We're taking notes on an etherpad for now; some process flowcharts are being done on a physical wall so once they're distilled onto the wiki w/ photos it should all be accessible together.
I might say that one more point to focus on specifically is to how to
leverage volunteer development (this is hinted at in some of your five points). There are _a lot_ of people who are capable of coding in PHP and who are willing to donate their time and talents, but Wikimedia/MediaWiki code development has chased them off, generally through neglect (patches sitting, review sitting, etc.). If there are ways to specifically look at that, it would be an enormous benefit to Wikimedia/MediaWiki, I think.
This is a big thing we want to solve -- current processes are very vague & laggy for a lot of stuff and only keep a fast deployment cycle for a few WMF-driven projects; this isn't because we're mean but because those are the only things that have internal drivers within WMF pushing them strongly enough; how to make sure other ongoing work gets the attention it needs too is very much on our minds.
-- brion
On Thu, Jul 7, 2011 at 12:40 PM, MZMcBride z@mzmcbride.com wrote:
I might say that one more point to focus on specifically is to how to leverage volunteer development (this is hinted at in some of your five points). There are _a lot_ of people who are capable of coding in PHP and who are willing to donate their time and talents, but Wikimedia/MediaWiki code development has chased them off, generally through neglect (patches sitting, review sitting, etc.). If there are ways to specifically look at that, it would be an enormous benefit to Wikimedia/MediaWiki, I think.
+1!
There's an enormous pool of volunteer developers out there who would gladly work for us, non-stop, if we can find a way to let them. For many things, our templating language can be lot harder to work with than PHP-- but despite its difficulty, look at how many useful advanced templates have been developed without us even having to ask for them.
Anyone who can make advanced templates can almost certainly handle PHP. The reason templates flourish while development flounders is "Openness"--- templating is essentially an open platform, WMF development is most certainly not an open platform.
Volunteer developers will do ridiculous amounts of work for us, innovating in ways we can't even imagine. Google's most popular program is it's "20% time" that allows them to spend one day a week working on whatever they want.
People want to innovate, just like people want to improve our projects' content. They will work for free-- but they have to know they'll be able to actually use their innovation themselves, and most have to know they can share it with others if it's popular. Most developers won't work for free only to have a third party decide whether it's sufficiently meritorious for its use to be allowed or not.
Right now, there's system in place to allow me to initiate, develop, implement, and share a feature without having to deal with a lot of read tape and permission-getting. If I want a Wikipedia that's a little different in some way, I have to implement on the client-side or I literally have to make my own fork of Wikipedia, that involved buying a domain name, setting up a host, raising money for it / paying for it, etc etc etc. A huge nightmare full of work that developers don't enjoy.
"Be Bold" hasn't been applied to the development or new projects yet. Right now, "Be Bold" is for an edits, not innovation. Right now, "Be Bold" is for new articles, not new projects.
We meed to figure out how to allow developer innovations instantly, automatically, in real time. But we also have to make sure those innovations don't affect the user experience for third-parties.
Once we get such a platform, development can take off. Until then, development will mostly be driven by third-party mediawiki project and paid staff-- both good to have, but orders of magnitude smaller than the size of the volunteer developer population that is going un-tapped.
Alec
wikitech-l@lists.wikimedia.org