https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-05-25
= 2016-05-25 =
== Technology ==
=== Analytics ===
- Trying to puppetize druid, harder than it looks
- Working on scaling on pageview API and cassandra, doing perf testing
on new nodes
- Trying to add throttling to pageview API
- Working on harvesting edit data from db/dumps into hadoop, WIP , main
goal this quarter
- Deployed visualization of Unique Devices:
https://vital-signs.wmflabs.org/#projects=ptwiki/metrics=UniqueDevices
-
=== Services ===
* RESTBase
** rate limiting in prod, log-only
*** Analytics, we need to discuss pageview limits
* Cassandra
** expanding from 2 to 3 instances per node in prod
** upgrade to 2.2.6 next
* Change prop
** handling updates for summary and mobile-sections* endpoints
** will move purging to it soon
* Math
** MathML by default on all wikibooks, early next week all projects
* heads up: Services team on Wikimania, then off-site at the end of June
=== Release Engineering ===
* '''Blocking''': ???
* '''Blocked''': none
* '''Updates''':
** wmf.3 is rolling forward this week
** rc.0 of 1.27 should be out this week
=== Technical Operations ===
* '''Blocking''':
** none
* '''Blocked''':
** none
* Updates:
** misc varnish cluster on route for being upgrade to varnish 4 again
** getting rid of tech debt on the database front (m1 cluster to be
reimaged)
** helping releng with scap3
** Getting finally a redundant link esams-eqiad
** libicu upgrade. See email from Giuseppe on wikitech-l
=== Security ===
* Two-factor authentication has been deployed to CentralAuth wikis with
permission enabled for staff
* Abbey and Daisy are assisting in usability surveying of two-factor
* Darian and Chris are working on knowledge transfer prior to Chris' last
day on Friday, May 27th
* Darian is working on onboarding documentation in anticipation of Security
Team hires in the near-ish future
* Security review schedule remains on track (
https://www.mediawiki.org/wiki/Wikimedia_Security_Team/Schedule ) and Brian
Wolff will be assisting
== Product ==
=== Reading ===
==== Web ====
* Getting ready for Hovercards A/B test (finialising bug fixes)
** Fundraising tech eng can have a look at JS variable for Popups
enabled/disabled
==== Android ====
* released Beta with Reading Lists
==== iOS ====
==== Mobile Content Service ====
* Working on feed endpoints
==== Reading Infrastructure ===
* AuthManager: please read email from Brad on wikitech-l
* https://gerrit.wikimedia.org/r/#/c/290269 (testing gem) pending review as
of 24-May-2016
=== Community Tech ===
* No blockers
* Numerical sorting in categories
** New indexes are in place thanks to JCrespo (
https://phabricator.wikimedia.org/T130692 )
** Will be proposing on Wikitech-l that we switch to "uca-default" as the
default page collation, rather than "uppercase" (
https://phabricator.wikimedia.org/T136113 )
* Launched MassViews interface for pageview stats (
http://tools.wmflabs.org/massviews/ )
* Working on new CopyPatrol tool for detecting plagiarism (
http://tools.wmflabs.org/plagiabot )
=== Editing ===
==== Parsing ====
(Subbu won't be there, updates only)
* Work ongoing to migrate Parsoid to use service-runner after a bunch of
fixes were pushed to service-runner - hoping to push this past the finish
line by next week before attempting a migration of Parsoid cluster to
jessie / node v4 (Follow along on
https://phabricator.wikimedia.org/T135176 and
blocking tasks). Conversation ongoing with services team to resolve details.
* Tidy replacement work proceeding well. After last round of fixes and
visual diff testing, ~88% of test pages render with pixel-perfect accuracy
and ~97% pages with < 1% pixel diffs with HTML5depurate. Additional CSS
fixes since then and new round of visual diff testing in progress. Tim
working to make some fixes to doBlockLevels in core parser to iron out some
kinks there which leads to different behavior in HTML5depurate compared to
Tidy (similar effects in Parsoid).
* Kunal's linker rewrite patch merged. Follow up work in progress to use
the new linker code.
* VE / CX: Please start thinking about how your code needs to change to use
split data-mw format. The data-mw split code is probably 2-3 weeks away in
terms of being ready for deployment, but you can start testing it with
Parsoid master which can provide you the split data-mw (ping arlolra on IRC
for details).
==== Language ====
* Blocked:
** Preference section "Internationalisation" balloons after clicking "More
language settings" https://phabricator.wikimedia.org/T133114(Frontend/Style
libs)
** Can't load In Progress or Published translation list
https://phabricator.wikimedia.org/T135743 (For: Collobration/Roan)
* Updates:
** Compact Language Links (as a non-beta feature) deployed in
Beta/testwikis; work on it continue along with ULS
=== Fundraising tech ===
* (force) merged security patches to fr branch, deployed
** got tests passing again Monday
* CentralNotice: api for a/b testing
* More work to get off ActiveMQ, remove SPOF (just bit us again last week)
* Prepping for fundraising in Israel, Japan and Ukraine
** Trying to mess with language fallbacks on payments cluster so we never
show Russian messages to those whose preferred language is Ukrainian
* Enhancing fraud & dos mitigation measures
=== Discovery ===
* '''Blocking''': none
* '''Blocked''': none
* Team offsite last week, got plans for next Q and year
* Upgrade to ElasticSearch 2.3 is coming on Thursday (
https://phabricator.wikimedia.org/T133124)
* Portal team added descriptive texts to project links on portal after A/B
test showed (small) positive impact
* Survey results for portal visitors:
https://commons.wikimedia.org/wiki/File:Wikipedia_Portal_Survey_-_May_2016.…
* Maps in Wikivoyage now support external layers:
https://en.wikivoyage.org/wiki/Wikivoyage:Travellers%27_pub#Maps_with_extra…
== Wikidata ==
* Blockers: none.
* Deleting files on Commons that are used in Wikidata statements was not
possible due to a bug. https://phabricator.wikimedia.org/T135485
* First prototype for structured data on Commons is close, finally.
https://phabricator.wikimedia.org/T125822
* QUnit tests timed out on Jenkins more often this week. Anybody knows why?
https://phabricator.wikimedia.org/T136303
So the RFC process page says I should email wikitech-l to propose an RFC, thus:
Content-Security-Policy (CSP) header is a header that disables certain
javascript features that are commonly used to exploit XSS attacks, in
order to mitigate the risks of XSS. I think we could massively benefit
from using this technology - XSS attacks probably being the most
common security issue in MediaWiki. The downside is that it would
break compatibility with older user scripts.
Please see the full text of my proposal at
https://www.mediawiki.org/wiki/Requests_for_comment/Content-Security-Policy
The associated phabricator ticket is: https://phabricator.wikimedia.org/T135963
I'd appreciate any comments anyone might have.
Thanks,
Brian
Please join for the following tech talk:
*Tech Talk**:* Integrating user behavior to design better products
*Presenter:* Pau Giner
*Date:* May 31, 2016
*Time: *19:00 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?msg=Tech+Talk%3A+Integ…>
*Length:* 30 minutes
Link to live YouTube stream <http://www.youtube.com/watch?v=mLeTABdcdR4>
*IRC channel for questions/discussion:* #wikimedia-office
*Summary: *The design process helps us to find solutions that respond to
the user needs. However, this process needs to rely on actual user behavior
to make sure we are addressing the right problems with the best possible
solutions.
Wikimedia projects are developed in the open and they reach millions of
users in very different contexts. This makes it challenging to integrate
the different observed behaviors, measured actions, opinions and other
forms of feedback.
After applying the design process on different Wikimedia projects, I want
to share some good practices and lessons learnt when integrating user
behavior to inform product decisions, and how you (in whichever role you
are playing) can help designers to better support this process.
*Feel free to forward this email to any other relevant wikimedia lists.*
On Sun, May 29, 2016 at 11:22 AM, Gergo Tisza <gtisza(a)wikimedia.org> wrote:
> On Sun, May 29, 2016 at 7:08 PM, Brion Vibber <bvibber(a)wikimedia.org>
> wrote:
> > We could change all that if there was a stable enough URL/linking
> > API... Can probably fake it for common cases and supported handlers by
> > generating a URL like we would for a local 404-handler repo; my main
> > concern is for future expansion where we add more fancy types on Commons
> > where the local site may not have an extension installed locally.
>
>
> Not sure if trying to use an unhandled file type via InstantCommons makes
> sense; the image parameters in the wikitext and the image parameters in the
> URL (or imageinfo API or hypothetical linking API) don't necessarily use
> the same syntax; what would transform that? (Granted, you get similar
> problems when parameter transformation is handled by the local extension
> and there is a version mismatch between the local and remote installation.)
>
Yep, it all comes back to T66214 <https://phabricator.wikimedia.org/T66214>
doesn't it. :D
> > (Iframe embedding as a fallback for unknown handlers sounds possible,
> > could expose
> > a frame URL in the imageinfo data.)
> >
>
> In general that would be a nice idea (see T27854
> <https://phabricator.wikimedia.org/T27854>). Specifically for embedding
> unknown file types in MediaWiki though, we would probably have to introduce
> some standard key-value syntax for image parameters first, instead of the
> current mess where it's parsed by an arbitrary set of regexes defined by
> the media handlers and generated in a similarly flexible way. And then you
> still have the problem of handling unknown media types in VisualEditor
> which would need some kind of parameter metadata at least.
>
Indeed. iframe embedding of video might be a good place to start for
unknown types; since we already have an iframe embedding system for video
in TimedMediaHandler, it should be easy to adapt it as a provider so that
sites can use videos from Commons without installing TimedMediaHandler
locally.
Key issues for options:
* separate the layout parameters from the file rendering parameters (eg
things like 'thumb' or 'frame' affect the layout, but the file
handler/renderer only cares about the resulting size)
* define standardish keys for file options already in use
* if extra file options are needed, have a standard option for extended
options!
* define that extra options will not be localizable or have magical
regexes; they should just be key-value pairs with fixed, predictable key
names
* define a standard way to pass file options to the iframe
Existing file options:
* 'width' and 'height' are implied through thumb/frame/upright/default
size, or explicitly via /\d+px/ or /\d+x\d+px' etc regexes defined in core.
* SvgHandler uses 'lang'; regex defined in core.
* PdfHandler and PagedTiffHandler use 'page'; regex defined in core.
* PagedTiffHandler uses 'lossy' switch to allow overriding the default
thumbnail type (JPG vs PNG); regex defined in extension
* TimedMediaHandler adds 'start', 'end', and 'thumbtime'; regex defined in
extension
There might be a couple more floating around out there, but that should
cover those already in use in Wikimedia content. For compatibility we'd
need to keep the core regexes, and might or might not want to move the
'lossy' and 'start/end/thumbtime' ones to core.
I can imagine some extended attributes I'd like for video:
* 'loop'
* 'mute'
* 'autoplay' (only allowed with 'loop' and 'mute', for GIF-like display)
for zoomable panoramic images (flat or spherical):
* 'zoom' - initial zoom level
* 'center' - coordinates of initial centering (what coord system? separate
x/y params or just one? should spherical use lat/lon? etc)
for viewing a 3d model:
* 'zoom' - just like panoramas
* 'distance' - viewer distance from origin point; this is different from
zoom!
* 'center' - origin point/center of view, extended to 3 dimensions (for
instance to concentrate on a portion of a larger model)
Let's imagine a file type that uses everything! An animated 3d model that's
localizable, and has multiple selectable "pages" of which we show one:
[[File:Cool file type.mytype|
<!-- layout attributes; keys may be localized -->
thumb
|alt=A bouncing ball changes colors while a calendar in background
counts off seasons in French.
<!-- width/height affects both layout and file; regex may be localized
-->
|640x480px
<!-- currently defined core file attributes; keys may be localized -->
|lang=fr
|page=3
<!-- currently defined extension file attributes; maybe localizable? -->
|start=25
|thumbtime=27.5
|end=35
|lossy=1
<!-- extended file attributes; requires key=val pair; not localizable
-->
|zoom=1.5
|center-x=0.5
|center-y=0.667
|center-z=0.25
|distance=0.75
|loop=1
|muted=1
|autoplay=1
<!-- caption -->
|Check out this an awesome [[3d model]] that's [[animation|animated]]!
]]
There is some ambiguity with accepting any unknown key followed by '='...
is [[File:Equation.mytype|E=mc^2]] using "E=mc^2" as a caption or passing a
parameter "E" with value "mc^2"?
We could add a 'caption=' magic option and use it explicitly:
[[File:Equation.mytype|caption=E=mc^2]]
Or just adding a stub <nowiki> ought to do it as a workaround, since it
should break the regex check for the generic key followed by "=":
[[File:Equation.mytype|<nowiki></nowiki>E=mc^2]]
For the media dialog in VE there's a few things we can do... For
locally-installed extensions, easy peasy! Extension registers the options
it needs along with localized labels and whatever necessaries for
validation/input. I'll have to take a look at how it works currently...
For remote files... probably we'd want to have an API method for the remote
site to expose a given file's option keys, localized labels, and some
metadata about the fields (just like a local extension might supply, but
without the ability to inject some JS or a custom widget).
Another possibility is for the remote site to expose a layout option editor
"preview" in its own iframe using a postMessage interface to bump data
values in/out of the dialog. This might be good for, say, specifying the
start/end/thumb times of a video, or the initial zoom/position of a
panorama, by scrubbing/dragging/zooming on the actual media instead of
manually typing in numbers. (Could use the exact same iframe interface for
local handlers to avoid having multiple code paths, or could remove the
intermediary frame for that case and have a common JS API for both methods.)
Back to implementation of the viewer: basically we'd take the final file
options in normalized key form (width & height and any other regex'd or
extended keys) and pass them as query string parameters to an iframe.
So when we look up info about the file itself, 'imageinfo' can return an
'iframe' key with, say,
https://commons.wikimedia.org/wiki/File:Curiosity%27s_Seven_Minutes_of_Terr…
we can take that iframe URL from the imageinfo and append our other
options, so an invocation like:
[[File:Curiosity's_Seven_Minutes_of_Terror.ogv|
640px|start=3:02|end=3:52|thumbtime=3:14]]
would yield something like
<iframe
src="...?embedplayer=yes&width=640&height=360&start=3:02&end=3:52&thumbtime=3:14"
width="640"
height="360"
frameborder="0"
allowFullScreen
></iframe>
Now this also brings up questions like did you want to play the video
inline vs show a thumbnail that brings up the video in a popup, which may
be something where we have to work out in general as well... this would
allow using a *standard* popup on the wiki side (say, MultimediaViewer?) to
load a remote media item.
If using an iframe to embed something with "plain" thumbnail behavior where
click means "link" or "zoom", we have to work out issues like passing click
events from the iframe to the parent (or else do some clickjacking ;)
-- brion
I am pleased to announce the launch of the third Inspire Campaign for
IdeaLab, focused on addressing harassment of Wikimedia project contributors:
<https://meta.wikimedia.org/wiki/Grants:IdeaLab/Inspire>
Harassment diminishes the experience of contributing and participation for
a substantial number of individuals, even those who simply witness it.
Current methods of dealing with harassment are considered unacceptable as
they often do not lead to productive outcomes.[1]
During the month-long campaign, you are invited to submit & review ideas on
how to better address harassment. Consider joining a team to help make an
idea happen. Ideas can be submitted in any language, and focus on
research, building tools or software, outreach efforts, or something
completely new. Grants are available from the Wikimedia Foundation to fund
projects that are eligible for financial support.[2] Ideas focused on
changes to community policies and guidelines are also welcome. Google
Hangout sessions are also scheduled in June if you’d like to discuss your
idea or have questions about WMF grants.[3]
Questions about the campaign can be directed at the Inspire talk page.[4]
An FAQ page about the campaign is also available.[5]
If you want to help make your projects safer for everyone to participate
in, I encourage you to participate in this Inspire Campaign. I believe we
can work together to address this difficult and important issue.
With thanks,
Chris "Jethro" Schilling
I JethroBT (WMF) <https://meta.wikimedia.org/wiki/User:I_JethroBT_(WMF)>
Community Organizer, Wikimedia Foundation
<https://wikimediafoundation.org/wiki/Home>
[1] <
https://commons.wikimedia.org/wiki/File:Harassment_Survey_2015_-_Results_Re…
>
[2] <https://meta.wikimedia.org/wiki/Grants:Start>
[3] <https://meta.wikimedia.org/wiki/Grants:IdeaLab/Events>
[4] <https://meta.wikimedia.org/wiki/Grants_talk:IdeaLab/Inspire>
[5] <https://meta.wikimedia.org/wiki/Grants:IdeaLab/Inspire/FAQ>
>From time to time mails like below pass by where I wish that voting should
be generalized on a technical level. So that any organization
participating in the wikiverse would be able to conduct votes and reuse
voting rights. Would this be something of broader value?
Best
Rupert
---------- Forwarded message ----------
From: "Sandister Tei" <sandistertei(a)gmail.com>
Date: May 31, 2016 16:33
Subject: Re: [Wikimedia-GH] Help us choose volunteers to serve you
To: "Sadat" <masssly(a)ymail.com>
Cc: "Planning Wikimedia Ghana Chapter" <wikimedia-gh(a)lists.wikimedia.org>
And if they vote with another account?
Regards,
Sandister Tei
www.sandistertei.com
Via mobile
On 31 May 2016 2:28 p.m., "Mohammed S. Abdulai" <masssly(a)ymail.com> wrote:
> You can just restrict multiple entries, I believe you're familiar with
> that process.
>
> Cheers
>
> -Masssly
>
> Sent from my Samsung Galaxy smartphone.
> -------- Original message --------
> From: Sandister Tei <sandistertei(a)gmail.com>
> Date: 05/31/2016 10:38 (GMT+00:00)
> To: Sadat <masssly(a)ymail.com>
> Cc: Planning Wikimedia Ghana Chapter <wikimedia-gh(a)lists.wikimedia.org>
> Subject: RE: [Wikimedia-GH] Help us choose volunteers to serve you
>
> You can suggest another means to check double voting.
>
> Regards,
> Sandister Tei
>
> www.sandistertei.com
>
> Via mobile
> On 31 May 2016 10:07 a.m., "Mohammed S. Abdulai" <masssly(a)ymail.com>
> wrote:
>
>> For best practices we shouldn't REQUIRE prospective respondents to enter
>> their names.
>>
>> I would like to see that relaxed.
>>
>> Thanks
>>
>> -Masssly
>>
>>
>> Sent from my Samsung Galaxy smartphone.
>> -------- Original message --------
>> From: Sandister Tei <sandistertei(a)gmail.com>
>> Date: 05/31/2016 08:00 (GMT+00:00)
>> To: Planning Wikimedia Ghana Chapter <wikimedia-gh(a)lists.wikimedia.org>
>> Subject: [Wikimedia-GH] Help us choose volunteers to serve you
>>
>> Hello all.
>>
>> Help us choose volunteers to serve you.
>> Please take this survey <http://goo.gl/forms/NQSLDL0CRGwwy3hi2>. It's
>> just two questions.
>>
>>
>> *Regards, *
>> *Sandister Tei *
>>
>> *www.sandistertei.com <http://www.sandistertei.com>*
>>
>
_______________________________________________
Wikimedia-GH mailing list
Wikimedia-GH(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimedia-gh
Hi all,
wrote an application for an IEG-grant on creating a testing environment for
Lua-scripts.[1] Perhaps it is interesting for you. It is mainly a tool for
on-wiki testing of scripts, and I'm not sure if it is that interesting for
use for off-wiki testing.
When I wrote the application I said it was about BDD-style testing, but it
would probably be better to describe it as testing with spec-statements.[2]
It is somewhat similar to the library "Busted",[3] but it use expect
instead of assert.
No changes to the Lua core is necessary to run this kinds of tests, it is
mostly to get proper localization of messages and a bit more stability and
reputability.
Likewise i sketched an alternate future grant application for ATDD-style
testing. This is testing with step-statements. The BDD-lib will only do
spec-type testing.
Add yourself to endorsement if you think this is a good idea! :)
[1]
https://meta.wikimedia.org/wiki/Grants:IEG/Lua_libs_for_behavior-driven_dev…
[2] https://en.wikipedia.org/wiki/Behavior-driven_development
[3] http://olivinelabs.com/busted/