What is the naming convention for these data centers? I think it might be
interesting to name them for people who have contributed significantly to
Wikimedia or MediaWiki in one way or another, such as Pacha (
https://blog.wikimedia.org/2014/05/12/the-story-of-jim-pacha/), Wadewitz (
https://en.wikipedia.org/wiki/Adrianne_Wadewitz), and Manske (
https://en.wikipedia.org/wiki/Magnus_Manske).
Just a thought.
Pine
*This is an Encyclopedia* <https://www.wikipedia.org/>
*One gateway to the wide garden of knowledge, where lies The deep rock of
our past, in which we must delve The well of our future,The clear water we
must leave untainted for those who come after us,The fertile earth, in
which truth may grow in bright places, tended by many hands,And the broad
fall of sunshine, warming our first steps toward knowing how much we do not
know.*
*—Catherine Munro*
I saw a WMF tweet of a site outage (network?) around 9:30am Pacific time, by the time I could check now things seem ok on en
Any news on root cause?
George William Herbert
Sent from my iPhone
Hi, just a note to say that the Phabricator project for the Wikimedia
Hackathon 2015 (Lyon, 23-25 May) has been created.
https://phabricator.wikimedia.org/tag/wikimedia-hackathon-2015/
You are invited to ask questions, provide feedback, and get involved.
Important note: even if we are just bootstrapping the project there, the
volunteers at Wikimedia France have done a ton of work already (i.e. the
venue and main services are secured).
--
Quim Gil
Engineering Community Manager @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil
For the last year, I've been using schroot to run my local MediaWiki
test instance. After hearing some gripes about Vagrant at the Dev
Summit, I decided to share this idea by automating the setup procedure
and committing the scripts I use. You can read about it here:
https://www.mediawiki.org/wiki/MediaWiki-schroot
-- Tim Starling
I was really happy to hear Damon, at the MediaWiki Developer Summit,
ask us how long we take to code review and whether we had communicated
a timeframe in which we promised to do it to our community. He quite
rightly stressed that this was vital for the survival of our
community. I spoke to one of our more new developers during the summit
and he also confessed to me that the reason he was an active volunteer
in our extension was that he got feedback on his code pretty quickly.
I had a few ideas about how to measure this so in my spare time I have
generated this report based on data from Gerrit patchsets using a
hacked together python script [1] which I hope will be if nothing else
an interesting artifact to talk about and generate some discussion.
Introducing:
https://www.mediawiki.org/wiki/Extension_health
To help you understand what you are reading, let's take Echo as an example:
Project: mediawiki/extensions/Echo
524 patches analysed (23 open, 501 merged)
Average review time: 29 days
Oldest open patch: (bug 41987) Updating tables indexes' names. (766
days) - https://gerrit.wikimedia.org/r/40095
The average time for code to go from submitted to merged appears to be
29 days over a dataset of 524 patches, excluding all that were written
by the L10n bot. There is a patchset there that has been _open_ for
766 days - if you look at it it was uploaded on Dec 23, 2012 12:23 PM
is -1ed by me and needs a rebase.
There are many patches like this outside Echo. We should probably be
seeing those patchsets through to completion or abandoning them on the
basis that if it hasn't been merged in over 2 years it's probably not
important and shouldn't be clogging up people's review queues and
hiding other more important patchsets which need review.
The more troubling situation is when patches have been open for some
time and have not been reviewed at all or are awaiting for some
response... let's get better at this!
Help make this ecosystem better. I will rerun the script in a month
and see if we have improved. Note our average time to review across
all those projects seems to be 18 days. That's really not good. We can
do better. We will do better.
[1] https://github.com/jdlrobson/GerritCommandLine
tl/dr: The technology we started building against (Titan) is probably
dead. We're reopening the investigation for a backing technology.
Yesterday DataStax <http://www.datastax.com/> announced
<http://www.datastax.com/2015/02/datastax-acquires-aurelius-the-experts-behi…>
that they'd acquired
<http://www.datastax.com/2015/02/datastax-acquires-aurelius-the-experts-behi…>
ThinkAurelius <http://thinkaurelius.com/>, the company for whom almost all
the Titan developers work. The ZDNet article
<http://www.zdnet.com/article/datastax-snaps-up-aurelius-and-its-titan-team-…>
made it pretty clear that they are killing the project
> "We're not going to do an integration. The play here is we'll take
> everything that's been done on Titan as inspiration, and maybe some of the
> Titan project will make it into DSE Graph," DataStax engineering VP Martin
> Van Ryswyk said.
While its certainly possible that someone from the community will come out
of the woodwork and continue Titan its now lost almost all of its top
developers. It looks like there is some secret succession discussions
going on but I'm not holding out hope that anything will come of it. This
pretty much blows this project's schedule of having a hardware request by
the end of the month and a publicly released beta at the end of March.
Anyway, we're reopening the investigation to pick a new backend. We're
including more options than we had before as its become clear that open
source graph databases is a bit of a wild west space. But there are people
waiting on this. The developer summit made that clear. So we're not going
to do the month long dive into each choice like we did last time. I'm not
100% sure exactly what we'll do but I can assure you we'll be careful.
I know you might want to talk about other options - you may as well stuff
them on
https://www.mediawiki.org/wiki/Wikibase/Indexing#Other_possible_candidates
and we'll get to them. As always, you can check out our workboard
<https://phabricator.wikimedia.org/project/board/37/query/DwEBx9K4vaHo/> to
see what we're actually working on.
Titan is still in the running assuming it gets active maintainers.
OrientDB, which we evaluated last round, is still in there too. So too are
GraphX and Neo4j. And ArangoDB. And Magnus' WDQ. We'd get much more
involved in maintenance, I think. And writing a TinkerPop implementation
Elasticsearch. That's not a serious contender. It'd get geo support for
free but its really just a low bar to compare all the other options to.
Thanks,
Nik <https://phabricator.wikimedia.org/T88550>
Minutes and slides from last week's quarterly review of the
Foundation's Editing (formerly VisualEditor) team await perusal at
https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarter…
On Wed, Dec 19, 2012 at 6:49 PM, Erik Moeller <erik(a)wikimedia.org> wrote:
> Hi folks,
>
> to increase accountability and create more opportunities for course
> corrections and resourcing adjustments as necessary, Sue's asked me
> and Howie Fung to set up a quarterly project evaluation process,
> starting with our highest priority initiatives. These are, according
> to Sue's narrowing focus recommendations which were approved by the
> Board [1]:
>
> - Visual Editor
> - Mobile (mobile contributions + Wikipedia Zero)
> - Editor Engagement (also known as the E2 and E3 teams)
> - Funds Dissemination Committe and expanded grant-making capacity
>
> I'm proposing the following initial schedule:
>
> January:
> - Editor Engagement Experiments
>
> February:
> - Visual Editor
> - Mobile (Contribs + Zero)
>
> March:
> - Editor Engagement Features (Echo, Flow projects)
> - Funds Dissemination Committee
>
> We’ll try doing this on the same day or adjacent to the monthly
> metrics meetings [2], since the team(s) will give a presentation on
> their recent progress, which will help set some context that would
> otherwise need to be covered in the quarterly review itself. This will
> also create open opportunities for feedback and questions.
>
> My goal is to do this in a manner where even though the quarterly
> review meetings themselves are internal, the outcomes are captured as
> meeting minutes and shared publicly, which is why I'm starting this
> discussion on a public list as well. I've created a wiki page here
> which we can use to discuss the concept further:
>
> https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_r…
>
> The internal review will, at minimum, include:
>
> Sue Gardner
> myself
> Howie Fung
> Team members and relevant director(s)
> Designated minute-taker
>
> So for example, for Visual Editor, the review team would be the Visual
> Editor / Parsoid teams, Sue, me, Howie, Terry, and a minute-taker.
>
> I imagine the structure of the review roughly as follows, with a
> duration of about 2 1/2 hours divided into 25-30 minute blocks:
>
> - Brief team intro and recap of team's activities through the quarter,
> compared with goals
> - Drill into goals and targets: Did we achieve what we said we would?
> - Review of challenges, blockers and successes
> - Discussion of proposed changes (e.g. resourcing, targets) and other
> action items
> - Buffer time, debriefing
>
> Once again, the primary purpose of these reviews is to create improved
> structures for internal accountability, escalation points in cases
> where serious changes are necessary, and transparency to the world.
>
> In addition to these priority initiatives, my recommendation would be
> to conduct quarterly reviews for any activity that requires more than
> a set amount of resources (people/dollars). These additional reviews
> may however be conducted in a more lightweight manner and internally
> to the departments. We’re slowly getting into that habit in
> engineering.
>
> As we pilot this process, the format of the high priority reviews can
> help inform and support reviews across the organization.
>
> Feedback and questions are appreciated.
>
> All best,
> Erik
>
> [1] https://wikimediafoundation.org/wiki/Vote:Narrowing_Focus
> [2] https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings
> --
> Erik Möller
> VP of Engineering and Product Development, Wikimedia Foundation
>
> Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
>
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l(a)lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
--
Tilman Bayer
Senior Analyst
Wikimedia Foundation
IRC (Freenode): HaeB
On Wed, Feb 4, 2015 at 2:33 AM, Erik Moeller <erik(a)wikimedia.org> wrote:
> If not, then I think one thing to keep in mind is how to organize the
> transformation code in a manner that it doesn't just become a
> server-side hodgepodge still only useful to one consumer, to avoid
> some of the pitfalls Brian mentions.
I think the MobileFrontend extension has probably run into these pitfalls
already.
> Say you want to reformat infoboxes on the mobile web, but not do all the
> other stuff the mobile app does. Can you just get that specific
> transformation? Are some transformations dependent on others? Or say we
> want to make a change only for the output that gets fed into the PDF
> generator, but not for any other outputs. Can we do that?
>
Maybe what we really need is a way to register transformation classes (e.g.
something like $wgAPIModules). Then have ApiParse have a parameter to
select transformations and apply them to wikitext before and to HTML after
calling the parser. And we'd probably want to do the wikitext-before bit in
ApiExpandTemplates too, and add a new action that takes HTML and applies
only the HTML-after transforms to it.
Or we could go as far as giving ParserOptions (or the ParserEnvironment I
recently heard Tim propose) a list of transformations, to allow for
transformations at some of the points where we have parser hooks. Although
that would probably cause problems for Parsoid.
--
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
Hi all,
it has been since the Dev Summit discussions on SOA/Microservices[1] that I am pondering the outcomes and I am willing to post some afterthoughts to these lists. Having been one of the most vocal in raising concerns about microservices, and having had experience in an heavily service-oriented web platform before, I think I owe my fellow engineers some more lengthy explanations. Also, let me say that I am very happy with both the discussions we had in the Dev Summit and its outcomes - including the fact that the Ops and Services teams both share the desire to work strictly toghether on this.
I tried to write down some thoughts about this, and ended up with a way too long email. So I decided to put up a page on wikitech here:
https://wikitech.wikimedia.org/wiki/User:Giuseppe_Lavagetto/MicroServices
Apart from my blabbing, have three questions on our strategy: How, when, what? None of this is clear to me as of today, and I guess if anyone has a clear picture of where we want to be in 6-to-12 months with microservices. If someone has a clear plan, please speak up so that we can tackle the challenges ahead of us on a practical basis, and not just based on some grand principles :)
Cheers
Giuseppe
[1] I prefer the latter term, probably because SOA sounds bloated to me, and reminds me of enterprise software architectures that I don’t like.