[x-posted]
Hello,
The Wikimedia Language Engineering team will be hosting an IRC office
hour on Wednesday, December 11, 2013 between 17:00 - 18:00 UTC on
#wikimedia-office. (See below for timezone conversion and other details.)
We will be talking about some of our recent and upcoming projects and then
taking questions for the remaining time.
Questions and any other concerns can also be sent to me directly before the
event. See you there!
Thanks
Runa
=== Event Details ===
What: WMF Language Engineering Office hour
When: December 11, 2013 (Wednesday). 1700-1800 UTC
http://www.timeanddate.com/worldclock/fixedtime.html?iso=20131211T1700
Where: IRC Channel #wikimedia-office on FreeNode
--
Language Engineering - Outreach and QA Coordinator
Wikimedia Foundation
I am not very happy about this but we came to the case
where it might be useful to explicitly uninstall some
hook(s) for out unit tests.
You might want to checkout MediaWikiTestCase::uninstallHook
https://gerrit.wikimedia.org/r/#/c/99349/
I am not happy about blurring differences between unit
and integration testing, but breaking core with extensions
and vice versa is sometimes useful.
//Saper
More news:
It turns out the DDOS was inadvertent and caused by an ill-considered
addition to several projects rather than malice: a script hosted on tool
labs intended to tie Wikidata to search results was made to load on
every page view rather than just upon searching, causing every hit to a
page on the projects to cause the user's client to connect to tool labs.
Obviously, tool labs does not scale well to that many requests. :-)
It will probably take several hours for things to return to normal as we
hunt down and remove those loads and cached versions expire from
endusers' caches.
-- Marc
_______________________________________________
Labs-l mailing list
Labs-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l
Hello all,
This is a quick email to give you an update on the DevOps Sprint[0].
== What we focused on ==
As you can see on the project page, there were three main areas of
(attempted) focus: Monitoring, Cache improvements, and Deployment. That
may sound like a lot, and it was. It was too much, in fact. There were
also items under those categories that weren't as well defined as they
should have been (see below).
== What we did ==
* We (Tim and Brad) did fix a couple long standing annoying cache
improvement bugs ([1] and [2]).
* We (Ori) improved scap a bit
** IRC logging of commits
** Reporting time to graphite - This is useful in that it gives us a
real baseline to judge any future improvements against.
** Made it more atomic by having rsync defer updating of files until
they've all been transmitted to the server, thus minimizing the
amount of time code is in an inconsistent state
** Made rsync use --compress which helped in over scap time, but we
don't know exactly how much (the 'time reporting' thing above
happened after)
* We (Aaron, Antoine, Bryan, Ori) setup Logstash
** This is still ongoing, but it is close to being setup in production.
** There is also a labs instance[3] you can look at that uses log info
from the Beta Cluster.
* We (Ori, Bryan, Aaron) wrote an RFC for adding structured logging to
MediaWiki [4]
== Things we didn't get to ==
Much of the deployments category. This is mostly due to the above
mentioned issues (too much, too broad).
Specifically:
* Trebuchet (aka: git-deploy)
** Ryan put a lot of work into Trebuchet during this sprint which
brought it to a great position
** We (Platform) weren't able to devote the requisite time to it during
the high level of churn (which was good).
** We (Platform) hope to work on this in the coming months from the
deployer's experience end (ie: not the backend of transferring files
around, specifically): more on this later[5].
Best,
Greg and the DevOps Sprint team.
[0] https://www.mediawiki.org/wiki/DevOps_Sprint_2013
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=5382
[2] https://bugzilla.wikimedia.org/show_bug.cgi?id=27935
[3] http://logstash.instance-proxy.wmflabs.org/#/dashboard
user/password:
https://office.wikimedia.org/wiki/User:BDavis_%28WMF%29/logstash
[4] https://www.mediawiki.org/wiki/Requests_for_comment/Structured_logging
[5] Basically, there will be a documentation sprint to produce a
overview of the current dev and deploy process followed by a "where
we want to go" high level thing (text or flowchart, something). From
that we can create a pretty solid design doc for deployment 2.0.
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
I have just posted a new draft RFC to alter the wfErrorLog() family of
functions to make logged events carry more data and have a common
output format [0].
This RFC was started as a part of the current DevOps related work [1]
being done by the Platform Core team. Ori, Aaron and I would like to
get feedback from the community on this early stage proposal before
continuing to work towards a prototype implementation.
We feel that implementing a common and information rich log message
output standard will be a helpful step towards removing some of the
mystery from the process of monitoring an active wiki. The current
variety of log outputs slows down the process of creating log analysis
tools. The information provided by a given log message is also largely
in the hands of each developer and varies widely not only from
component to component but sometimes from commit to commit. The final
proposal will describe a common output standard that provides a means
to attach common data to all log messages while still allowing each
module to add unique data that is needed to diagnose problems at
runtime and at scale.
[0]: https://www.mediawiki.org/wiki/Requests_for_comment/Structured_logging
[1]: https://www.mediawiki.org/wiki/DevOps_Sprint_2013
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID
irc: bd808 v:415.839.6885 x6855
MediaWiki participates in a number of student competitions and programs as
an open source mentor (such as GSoC, Code-In, etc.). Today I ran into
another one: Facebook's Open Academy Program.
https://www.facebook.com/OpenAcademyProgram
I'm not sure how we would get involved in this program, but I'm sure people
would agree it might be a good thing to become a mentor organization and
have students contribute to MediaWiki as part of a college credit program.
Any thoughts?
*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
Again, I didn't do anything, it's all Magnus' merit. :) And on it.wiki,
Jalo's and Rotpunkt's.
Andrew Gray, 03/12/2013 13:29:
> Would it be worth moving the Reasonator results up to the top right,
> in a box? There's a lot of white space on the search page, because the
> result snippets are less than 500px wide...
As those with good memory may remember, that place used to be occupied
by Lucene's interwiki search results.
https://bugzilla.wikimedia.org/show_bug.cgi?id=44420
So, while usage of that space is not unprecedented, I'm not sure this
tool is consistent with that expectation (full? text search), which is
supposed to be coming back via CirrusSearch. Moving away from that space
would be possible any time if CirrusSearch claims it (I hope it will!),
but the results e.g. in your Richard II search are quite long and some
care would be needed or they'd get quite an untidy visual distraction.
If someone comes up with a good design mockup, that would probably help
Magnus speedy-code it as his habit. ;-)
Nemo