Hi all,
The MediaWiki Core team is having the first of its fortnightly scoping
meetings. The intent of these meetings is to take a piece of work that the
MediaWiki Core team is considering undertaking and discussing it.
By the end of the meeting, the project should have:
* A description of the project and how it affects its end users
* A list of well defined stakeholders
* A proposed solution (roughly speaking; this will not be binding)
* An assessment of the difficulty of the project
* An assessment of the priority of the project
Some of these (e.g. difficulty assessment) are relative to other projects,
and only start to make sense once you've got a few projects on the list.
That's okay, we'll get to that stage eventually.
In the first meeting, we will be discussing the recent problems with the
job queue length. The length of the job queue has been increasing, and this
has a lot of effects on the end users of our wikis. What can we do about
it? Let's discuss.
The meeting will be on Tuesday 6th May at 4pm. I'll invite a few relevant
parties, but ultimately the meeting is open to everyone so if you want to
come then please do! If you're remote, ping me and I can invite you to the
hangout.
Thanks,
Dan
--
Dan Garry
Associate Product Manager for Platform
Wikimedia Foundation
Hi folks,
I'm putting this as an agenda item on our weekly meeting:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/April#Plat…
You can have everyone sit there and listen to me type this in on your
behalf, or you can get your edits in prior to our meeting. Your call.
:-)
Rob
---------- Forwarded message ----------
From: Guillaume Paumier <gpaumier(a)wikimedia.org>
Date: Mon, Apr 28, 2014 at 5:37 AM
Subject: [Engineering] April engineering report: please add your status updates
To: Development and Operations Engineers
<engineering(a)lists.wikimedia.org>, Lydia Pintscher
<lydia.pintscher(a)wikimedia.de>, Emmanuel Engelhart <kelson(a)kiwix.org>
Greetings,
April is almost over, meaning it's time to add our status updates so
we can put together the monthly engineering report:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/April
The Metrics meeting is Thursday, and I realize the month isn't
actually over until Wednesday, but anything you can do to add your
updates early is appreciated :)
Let me know if you need any help.
--
Guillaume Paumier
Technical Communications Manager — Wikimedia Foundation
_______________________________________________
Engineering mailing list
Engineering(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/engineering
Hi all (sorry for the duplicate Gabriel),
I was going to send this to wikitech, but I wanted to get an internal
opinion. We may have this documented already, but I haven't managed to find
it.
tl;dr: As I'm looking at authentication for whatever "SOA" plans we have,
the basic structure of how we design things will influence the best way to
do authn/z. If you have strong feelings one way or the other, speak up so
we can plan for it.
I wanted to get some clarity around what "SOA" means to us and our
community. Aaron asked the question during the SOA kickoff, something like
is everyone comfortable with making the endpoints public. I don't think
there was much discussion or consensus (or I may have missed it). When I
talked with Owen and Gabriel afterward, it seems like there were two
definitions of "SOA" that we're dealing with:
* Hidden services: e.g., a user gets an edit page, and does an HTTP POST
with action=edit, a session cookie to identify themselves, an anti-csrf
token. The wiki gets the request, then calls a service to check the edit
text for spam, gives the edit to a revision storage service that stores the
revision, maybe generates an event that a logging service picks up and
stores in the general and checkuser logs.
* Public services: A user gets an edit page from the wiki, then submits the
edit to the revision service directly. The revision service may call the
spam-detection service, or use a service for csrf tokens. The success /
failure is passed back to the user's browser, and the edit page lets the
user know the save happened, and probably shows the user the new version of
the page.
Parsoid, iirc, is moving towards the later definition. Wikia is mostly
looking at the first.
The answer I'm expecting is we need "both", and the distinction is probably
really only important when we decide how identification and authorization
work. But I want to make sure we make that decision consciously instead of
accidentally.
The first case has a more straightforward implement for authentication, and
I would argue is easier to secure. Http-only cookies are set and included
with each call to identify the user's session. These can't be stolen via
xss, and we can centrally invalidate all sessions. We have one place to
check if a nonce has been used (e.g. with OAuth), and have common code for
checking csrf tokens and logging.
Having a public service handle processing a user's edit would require the
service to check session invalidation and nonce use, and correctly handle
checking csrf tokens.
If we go the public route, we would need to decide if the various services
should run on the same domain (en.wikipedia.or/revision/whatever), same top
level domain (revision.wikipedia.org/whatever), or entirely different
domain (revision.wikimedia.org). The same top-level domain would only work
if all subdomains have the same idea of the user (at the WMF, that would
mean all users are CentralAuth/global users). A different domain means
either users have to go through a login process to get authentication
cookies on that domain (like we use for login.wikimedia.org), or the
authentication tokens have to be accessible to javascript (and vulnerable
to being stolen via xss attacks). So I have a strong preference for same
domain.
Does anyone have strong plans / ideas for the overall direction we want to
take here?
Hello,
One of the blockers for the effort to package HHVM for Debian is the
licensing of the JSON extension. This is the same problem that has
afflicted PHP (see <https://bugs.php.net/bug.php?id=63520>: "JSON extension
includes a problematic license statement") and that we have discussed in
the context of bug 47413 (<
https://bugzilla.wikimedia.org/show_bug.cgi?id=47431>: "JSON extension
dependency has a non-free component".
There is a drop-in replacement which is not encumbered with licensing
issues: Remi Collet's pecl-json-c, available at <
https://github.com/remicollet/pecl-json-c>. Paul Tarjan replied on the
packaging thread (see <
https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1208263.h…>),
saying that the HHVM team would be interested in swapping out the JSON
library and replacing it with pecl-json-c, if someone were to port it.
Since both extensions provide the same API, I imagine "porting" in this
context means tweaking the include paths and the build configuration.
Are any of you interested in taking this on?
Hi everyone,
I had intended to send this email prior to our weekly meeting this
week, but I ran out of time to send it. I believe I covered all of
the major points in our meeting, but I'll send what I had drafted
anyway just in case I missed something important.
As you all know, we've had a bit of a muddled planning process leading
up to our quarterly reviews, which has evolved to be something like
the following:
1. We have an ideas list page[1]
2. About a month out from the quarterly review, I ask everyone to put
what they would like to be working on in a spreadsheet broken down by
month[2]
3. We have discussions about what the "big" project should be
4. We refine that into a wiki page[3], consolidating and pruning the
spreadsheet
5. We boil that down into a slide deck, and present our plan at the
quarterly review.
One big problem with that process is the ideas list is a vague,
undifferentiated list of random items, some hugely important, some
trivial, and some too vague to assign priority to. Another big
problem is that it's unclear to others exactly how to influence our
plan. It's probably even unclear within the team.
Dan, Howie and I met earlier, and we'd like to propose something a bit
different. We'd like to achieve some important improvements:
* Have a well-groomed, well-vetted backlog of possible projects with
basic priority levels (e.g. high, medium, low) and reasonable
introductory descriptions for the high-level projects.
* Hold ourselves accountable to getting the non-emergency things we
work on on the list
* More methodical gathering of input from other parts of the organization
How we hope to accomplish this is to have a regular (probably
fortnightly) meeting to flesh out high priority ideas and triage low
priority ones. We'll make sure everything that is listed as high
priority meets a standard of completeness with respect to its
description and rationale. We'll more explicitly solicit input from
other parts of the organization on what the most important things are
from their perspective. We'll then consider only those things the team
agrees are high priority as work for the coming quarter, and use our
existing process to narrow things down to the final plan.
Dan has posted the proposed process to mediawiki.org, which you can read here:
https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Backlog/Proces…
The actual backlog would live here:
https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Backlog
Dan will lead this process, but the actual decision-making will be
team consensus driven by Tim and Dan. If there's a hopeless
stalemate, I'll step in with an arbitrary, ill-informed decision that
everyone will hate. Don't make me do it. :-)
Let's do some on-wiki tweaking/discussion during the week, and then
finalize something at our weekly meeting next week.
Make sense?
Rob
[1] https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Ideas_list
[2] MW Core Allocations sheet:
https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Agte_lJNpi-OdE…
[3] https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Quarterly_revi…
Hi everyone,
As of early next week, I will be in a 50/50 split between being Product
Manager for Mobile Apps and for Platform.
Exactly how this will work is yet to be determined. It may be that the
constraints end up being somewhat artificial in order to enforce the split.
For example, I may have half my working hours earmarked for each product
area, and during each period any questions I get about the other product
area always get the response "Ask me later". The Product team will be
figuring this out in the next few days.
I'll let you know more as the details become clear.
Thanks,
Dan
--
Dan Garry
Associate Product Manager for Platform
Wikimedia Foundation
Hi everyone,
As I'm sure you're aware, by virtue of my job's scope I sometimes find
areas of attention where a little bit of engineering work would result in a
reasonably sized improvement from a product and user experience standpoint.
Often these problems are ill-defined by the user reporting them, which
obscures the actual simplicity of the problem and deters engineers from
working on them.
I have been raising these issues from time to time in our weekly meetings.
I've thought about that, and decided that that meeting is long enough
already without me trying to cram more into it. I think a large part of the
lukewarm response I get trying to recruit people to do these things is due
to the forum I'm raising these issues in. So I've got an alternative
solution: product microtasks(tm) [1].
Product microtasks are problems that have been reported by a user that I
have assessed, prioritised, and defined a clear-cut engineering solution
for. If you're stuck on your current project and want something else to
work on to cleanse your programming palate, you can do one of these. The
intention for these is that they're little discrete chunks of work that are
easy for you to pick up and not have to worry about defining the solution,
just implementing it.
The list can be found at
https://www.mediawiki.org/wiki/Platform_product_microtasks, and could be
thought of as "annoying little bugs for more experienced developers".
Be warned, I may still bug people from time to time to attend to specific
issues. That is a sign of love and a mark of respect. Really. ;-)
Thanks,
Dan
[1]: Yeah, alright, you got me. I've not actually trademarked it. Don't
spoil my fun. :-(
--
Dan Garry
Associate Product Manager for Platform
Wikimedia Foundation
Hi everyone,
I'm basically done with the quarterly review slides, but I need help
filling in a few sections for the projects I've had limited to no
involvement in, particularly the stuff on HHVM, deployment development, and
RFC review.
Please keep to one slide! We should keep things short and sweet.
Here's the link, please take a look:
https://docs.google.com/a/wikimedia.org/presentation/d/1DCJ5QyfAYXZZYfFjeVe…
Dan
--
Dan Garry
Associate Product Manager for Platform
Wikimedia Foundation
Hi everyone,
Based on the discussions we've had, Rob and I have come up with the overall
plan for next quarter. It's available
here<https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Quarterly_revi…>.
In a word, it's "HHVM". Unless there are any objections raised before the
quarterly review next Tuesday, this will be ratified as our quarterly plan.
You only have a few days to object, so speak up!
Still, there are many projects that are not covered in the review. Please
go here<https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Quarterly_revi…>
and
fill in the names of the projects you'll be working on. It will only take
you a minute. If you're unsure how to do it, I've already filled mine in so
you can take a look at that.
Thanks,
Dan
--
Dan Garry
Associate Product Manager for Platform
Wikimedia Foundation
Hi everyone,
I have a working session with Dan tomorrow to do major work on the
wiki page here:
https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Quarterly_revi…
Given the relative lack of controversy around just going with HHVM,
the clear end-user benefit, and the lack of a well-articulated and
deeply-championed alternative, I think there's less urgency in getting
a consultative mail out to wikitech-l in advance of our quarterly
review. Nevertheless, I think it sets good precedent, so I'd like to
do it anyway. I'd like to have a solid articulation of all of the
work that we're doing in addition to the work we've considered and
decided not to do this time around, so that people are informed about
what we're not doing in addition to knowing what we are, and can make
a last minute case for something else.
I don't have a draft of said email started, but it's basically going
to be the highlights from the wiki page above. Dan and I will be
editing between 11am and 12pm PDT tomorrow, so that's the main time to
avoid doing a lot of work there. Any time before or after that is
fair game, so please take a look at it, and if there's areas that you
feel you can fill stuff in prior to our meeting, or elaborate/correct
after our meeting, please do.
Thanks
Rob