https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-06-01
= 2016-06-01 =
== Product ==
=== Reading ===
==== Reading Infrastructure ====
* AuthManager coming soon - it's on the beta cluster now! Please indicate
if you hae a serious blocker now at
https://phabricator.wikimedia.org/T135504
==== Web ====
* Starting Hovercards A/B test during this sprint (with 1% in HU.wiki)
* Rolling lazy loaded images / references to medium sized wikis
* Blocked (Ops/Analytics) - https://phabricator.wikimedia.org/T127883
==== iOS ====
==== Android ====
* Production release 2.2.146 rollout out this Tuesday.
https://lists.wikimedia.org/pipermail/mobile-l/2016-May/010243.html
(New features include: reading lists, edit here, Show redirect source in
search results, updates for AuthManager,
No longer download and widen high-quality images when on a metered
connection.)
* Working on new feed UI
==== Mobile Content Service ====
* Adding first couple of new feed endpoints this week (enwiki featured
article, most read articles, aggregated endpoint for previous and future
feed microservices)
=== Community Tech ===
* We're going to be testing the PageAssessments extension on Labs this week
(https://www.mediawiki.org/wiki/Extension:PageAssessments)
* Continuing work on new CopyPatrol Tool (
http://tools.wmflabs.org/plagiabot)
* More investigation on Category collation
* No blockers
=== Editing ===
==== Language ====
* Blocked: Parallel Corpora dumps:
https://phabricator.wikimedia.org/T127793 (Tech
Ops)
==== Multimedia ====
* Blocked: New repository request for FileAnnotations so we can experiment
and do design/code review on Mark's crappy prototype
* Updates
** ImageTweaks waiting on Thumbor and a few other things, no news there
AFAIK
** Gallery slideshow core work coming along, no huge updates from prtksxna
** UW fatals rate waaaay down since last year, thanks Sentry people for
helping us track it
==== Parsing ====
* Tidy replacement: With latest HTML5Depurate and visual diff test run,
~93.4% of test pages render with pixel perfect accuracy. Working through
remaining diff scenarios (a lot of the diffs are harmless minor pixel
shifts) to identify real issues and having wikitext be fixed or fix core
parser as required.
** Tim working on a HTML5 parser to fix some core parser issues around
doBlockLevels.
* Will attempt switching Parsoid to service-runner sometime next week (Arlo
& Services)
* Arlo close to getting html2html endpoint done (last remaining blocker to
enable Parsoid new HTML version with separate data-mw)
** VE & CX: Please start planning how to handle this new version.
* Kunal working on using the new Linker code for files/images.
==== Collaboration ====
* Blocked - None
* Blocking - No change
* Updates
** Working on Echo special page, for both JS and no-JS users
** Schema changes we requested are done, we now need to backfill and use
the new schema
== Technology ==
=== Services ===
* RESTBase
** enforcing rate limits as of today
*** pageview: 10 req/s
*** transforms: 5 req/s
* Change Propagation
** now handles summary and MCS updates as well as purges
** partially moved RestbaseUpdateJobs from job runners to it
*** still missing transclusions
* MathML enabled by default across all projects
** proper caching and purging still a TODO
*** RESTBase side for purging proposed in
https://github.com/wikimedia/restbase/pull/628
*** need Ops help for the Varnish side of things -
https://phabricator.wikimedia.org/T136205#2343850
=== Technical Operations ===
* '''Blocking''': none
* '''Blocked''': none
* Updates:
** libicu update successful
** Started preliminary work on migrating appservers to jessie
** Last day of Riccardo
** Getting graphite into better shape
=== Analytics ===
* Loaded 3 months of pageview data into Druid and querying it is very fast
* Cleaning up limn-flow-data, limn-edit-data, and limn-languge-data a
little bit, deploying today
* Working on processing data from the mediawiki databases and turning it
into an analytics-friendly schema (one example is slowly changing
dimensions such as page_title recorded as (page_id, page_title, valid_from,
valid_to))
* pageview requests that miss the cache are rate-limited to 10 req/s. More
req/s than that will throw 429
=== Security ===
* Two-factor usability surveying prep continues
* Reviews: Ex:DoubleWiki, Ex:TemplateStyles, Use of wheels for deployment
by Analytics
=== Discovery ===
* '''Blocking''': none
* '''Blocked''': none
* Tomasz last day at WMF was on Friday, Katie Horn leading Discovery now
* ElasticSearch 2.3 upgrade in progress:
https://phabricator.wikimedia.org/T133124
* TextCat A/B test for english wiki started
* Updated Portal language stats.
=== Fundraising Tech ===
* Fixed CentralNotice tests, now enforcing ES3
* More work to get off ActiveMQ, remove SPOF
* Testing new processor in Israel, Japan and Ukraine
** We quit messing with language fallback and just got the last few missing
messages translated to Ukrainian
* Ready to test new PayPal integration method
* Enhancing fraud & dos mitigation measures
=== Release Engineering ===
* '''Blocking''': None
* '''Blocked''': Nada
* '''Update:'''
** Auth Manager rolling out *soon*
There are currently 94 WMF wikis using UCA category collation rather than
the default "uppercase" collation. The Unicode Collation Algorithm (UCA) is
the official standard for how to sort Unicode characters, and generally
follows how a human would typically alphabetize strings. For example,
uppercase collation sorts Aztec, Ärsenik, Zoo, Aardvark as "Aardvark,
Aztec, Zoo, Ärsenik", but uca-default collation sorts them as "Aardvark,
Ärsenik, Aztec, Zoo". UCA collation also (optionally) supports natural
numeric sorting so that 100, 1, 99 sorts as "1, 99, 100" rather than "1,
100, 99". The WMF Community Tech team has recently posted proposals on
English Wikipedia and several Wiktionaries asking if these communities
would support switching to UCA collation. The proposal on English Wikipedia
has received unanimous support so far.[1] We thought that Wiktionaries
would be more skeptical of the change, but so far we have received only
positive responses.[2]
Since it seems that most wikis are receptive to switching to UCA, maybe we
should just make it the default rather than waiting on all the wikis to
request it separately. Of the large Wikipedias, French, Dutch, Polish,
Portuguese, and Russian are already using UCA, and German is in the process
of switching.[3] For non-Latin scripts, my understanding is that UCA will
be a big improvement, especially if we switch them to language-specific
implementations, like uca-ja, uca-zh, uca-ar, etc.
Three questions:
1. Does switching the default collation from "uppercase" to "uca-default"
sound like a good idea?
2. Should this be proposed on meta or is it too technical?
3. Are there any wikis that would need to opt out of this for some reason?
(I know there are issues with Kurdish,[4] but that's the only one I know
about.)
1.
https://en.wikipedia.org/wiki/Wikipedia_talk:Categorization#OK_to_switch_En…
2. https://phabricator.wikimedia.org/T128502
3. https://phabricator.wikimedia.org/T128806
4. https://phabricator.wikimedia.org/T48235
Question 1: Would anyone care if we kill the "loginCTA" campaign, which
tracks when people use the link at the bottom of Special:UserLogin to get
to the account creation page?
Question 2: Would anyone care if we remove the extension entirely from
Wikimedia wikis? Wikiapiary seems to show only one user outside of
Wikimedia.
Background: The extension needs a rewrite for AuthManager, and in
particular the "loginCTA" campaign will be a bit of a pain to keep working.
If someone is making use of the extension that's fine, but if not we may as
well not continue to spend development resources on it.
--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
Hi everyone,
We're planning to have our regular ArchCom-RFC IRC meeting in a couple
of hours. Phab event: <https://phabricator.wikimedia.org/E198>
Location: #wikimedia-office IRC channel
Meeting type: Problem definition
Time: 2016-06-01 Wednesday 21:00 UTC (2pm PDT, 23:00 CEST)
The experiment using Phab Conpherence rooms for public discussions
continues with Phab:Z425. I will optimistically say this has not
*yet* succeeded :-) [1]
The topic ArchCom agreed to focus on this week is Security. Here's a
few candidate topics for us to discuss in today's meeting:
* T135963: Add support for Content-Security-Policy (CSP) headers in MediaWiki
* T75953: RFC: MediaWiki HTTPS policy
* T123753: Establish retrospective reports for #security and
#performance incidents
Broadly speaking: the theme for today is "security is everyone's job".
Really, it is. If you have +2 rights, and you *don't* think security
is your job, please think about why you deserve to keep that right.
We all have a responsibility to step up our game in this area. Let's
use today's IRC meeting to figure out how we intend to step up.
Rob
[1] The thing that makes me optimistic about Conpherence is that
it's a persistent log of a linear conversation that integrates well
with the rest of Phabricator. The Z425 Conpherence:
<https://phabricator.wikimedia.org/Z425>
More thoughts later here:
<https://www.mediawiki.org/wiki/User:RobLa-WMF/Conpherence>, and the
talk page there is a good place for off-thread replies.
Hi all,
AuthManager [1] has been enabled on the beta cluster. If you see anything
strange, please file a bug and cc anomie (Brad) and me. If you are
maintaining authentication-related code, and have some time to test it,
that would be awesome. If you see serious breakage, please file blockers
for T135504 [2].
If everything goes well, this will result in very little visible change on
the web interface - some special page names are different (signup is on
Special:CreateAccount now), with the old ones redirecting; some interface
messages are different at login and account creation; pages which asked for
the password for extra security (such as email change) might send you
through the login form instead; there are some new special mages
(Special:ChangeCredentials, Special:RemoveCredentials).
For the API, this is a larger change: action=login becomes deprecated and
might break in certain situations (e.g. when the system wants you to change
your password), and action=createaccount requires different parameters.
(Bot passwords and OAuth are unaffected.) Hopefully all affected API
clients have been updated already; if not, please file blockers for T134945
[3], and update soon.
To avoid extra maintenance burden, we are aiming to convert everything to
AuthManager before 1.27 is fully released, which means we have to move
faster than we would normally prefer. We apologize for any inconvenience
this might cause.
regards,
Gergő
https://www.mediawiki.org/wiki/User:Tgr_(WMF)
[1] https://www.mediawiki.org/wiki/Manual:SessionManager_and_AuthManager
[2] https://phabricator.wikimedia.org/T135504 - Enable AuthManager in WMF
production
[3] https://phabricator.wikimedia.org/T134945 - Update API clients for
AuthManager
Hi everyone,
The important part of this email is this link:
https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey/Status_repor…
This is the second Community Wishlist Survey status report. In November and
December, active contributors to Wikimedia projects proposed, discussed and
voted on the features and fixes that they most want to see. The Wikimedia
Foundation Community Tech team has been tasked with working on
these. Additionally, Wikimedia Deutschland's Technical Wishes team has been
working on wishes from the German-speaking community. There's overlap
between the two wishlists, and the teams are collaborating on various
wishes, so this report includes progress made by both teams as well as
great work being done by volunteer developers and other WMF staff.
So far, we (in the broad sense) have added support for:
*) Migrating dead external links to archives (but there's more work to be
done!)
*) Pageview stats
*) Global notifications
*) A category watchlist
We're currently working on:
*) Improving the plagiarism detection bot
*) Improving the diff compare screen
*) Numerical sorting in categories
*) The possibility to add an expiry date to watchlist items
*) A revision slider to help editors navigate through diff pages
For more information on these projects as well as upcoming tasks, see the
full status report on Meta:
https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey/Status_repor…
We're looking forward to talking and working with you as we go along.
Thanks,
//Johan Jönsson
User:Johan (WMF)
--