Hello all,
Here is the Discovery weekly update for 2017-07-31. As always, feedback and
questions are welcome.
Programming note: We'll be spending our time and attention for the next
week at Wikimania (well a sizable number of us). As such, Discovery weekly
updates will return the week of 14 August.
== Highlights ==
* 2016/17 Q4 Metrics for the Discovery Department were presented on Aug 3,
2017
* The Audiences 2 (New Readers, Structured Data on Commons, Wikidata, &
Readers) Quarterly Check-in, July 2017, has been published that contains
the Discovery Department's update. [0]
== Discussions ==
=== Search ===
* Archive search on Special:Undelete enabled everywhere by Stas. [1]
* We began a search relevance test to see if we could get good feedback,
based on (initial) canned queries (Erik was the key person on this test to
get grading feedback from humans on search results) [2]
* David fixed a bug where reusing the ScriptService was hitting a circuit
breaker which prevented large featuresets to be compiled in one row [3].
This isn't complete deployed yet, due to needing a new plugin version to be
added into production first.
* Guillaume and Chris J helped with a big issue due to some elasticsearch
servers in eqiad overheating (applied thermal paste) [4]
* Trey completed another round of testing and analysis on the updated
Vietnamese language plugin; we're still not able to deploy it, but it has
improved. We may try again later after more updates. [5]
=== Analysis ===
* Mikhail and Chelsy worked diligently on getting metrics for the Discovery
Department's Q4 2016/2017 slide deck ready for delivery [6]
=== Maps ===
* Enabled mapframe for ptwikipedia, euwikipedia and uawikimedia projects
(thanks, Maxǃ) [7]
[0]
https://commons.wikimedia.org/wiki/File%3AAudiences_2_(New_Readers%2C_Struc…
[1] https://phabricator.wikimedia.org/T163235
[2] https://phabricator.wikimedia.org/T171740
[3] https://phabricator.wikimedia.org/T17157
[4] https://phabricator.wikimedia.org/T168816
[5] https://phabricator.wikimedia.org/T170423#3502328
[6] https://phabricator.wikimedia.org/T171528
[7] https://phabricator.wikimedia.org/T171805
---
The archive of all past updates can be found on MediaWiki.org:
https://www.mediawiki.org/wiki/Discovery/Status_updates
Interested in getting involved? See tasks marked as "Easy" or "Volunteer
needed" in Phabricator.
[1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R
[2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R
Yours,
Chris Koerner
Community Liaison
Wikimedia Foundation
Hi, I just proposed a session for the Wikimania Hackathon next week:
Creation of a Program Committee for Wikimedia developer events
https://phabricator.wikimedia.org/T160996
All the details are in the task. I am mentioning this here to promote the
session among those of you joining the hackathon in Montreal next week, but
most importantly to encourage feedback and discussion before the session
and by anyone anywhere. Preferably in the task itself, to avoid
fragmentation.
--
Quim Gil
Engineering Community Manager @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil
Hey, yesterday the patch implementing human-readable section IDs [0] was
merged (thanks, Tim!). The new feature has already been enabled on beta
cluster and you can try it yourselves, e.g. on [1] - some pages might still
have old HTML cached though and require a null edit to update.
What's next? We can probably flip it on testwiki after Wikimania, but
further deployments depend on Reading Web, Apps and Parsoid teams. We,
however, can already encourage editors to check it out in staging.
Unanswered question: do we really need to percent-encode the IDs? There is
extensive discussion of that in the aforementioned task, concluding that
percent-encoding is probably more "correct". However, not escaping it gives
much better browser compatibility (close to 100%). We can change this at
any time because no links will be broken due to the way browsers handle
percent-encoded fragments.
What's the impact for end users? When the commit above goes live, there
will be no user-visible changes, and almost no HTML changes at all (only
some interface IDs/classes generated from MediaWiki messages might slightly
change, but no links will be broken). When we migrate we will initially
enable new IDs as a fallback. After 1 month, the new IDs will become
default while old ones will be used as a fallback. This way, old links will
continue to work, and we have no plans to disable the fallbacks in the
foreseeable future.
What's the impact for developers? Sanitizer::escapeId() has been
deprecated, all code should migrate
to escapeIdForAttribute(), escapeIdForLink()
or escapeIdForExternalInterwiki(). Warning: unlike escapeId(), these
functions' output is not guaranteed to be HTML-safe so it must be escaped.
Our security guidelines say that everything should be HTML-safe anyway, so
even escapeId() should be properly escaped. The same deprecation happened
in JavaScript, mw.util.escapeId() is also deprecated.
----
[0] https://phabricator.wikimedia.org/T152540
[1]
https://ru.wikipedia.beta.wmflabs.org/wiki/%D0%92%D0%B8%D0%BA%D0%B8%D0%BF%D…
--
Best regards,
Max Semenik ([[User:MaxSem]])
I haven't tried this, but it sounds like translators and Language
Engineering folks might be interested.
Pine
---------- Forwarded message ----------
From: Alex Brollo <alex.brollo(a)gmail.com>
Date: Tue, Aug 1, 2017 at 12:10 AM
Subject: [Wikisource-l] An it.source gadget to manage diacritics
To: wikisource list <wikisource-l(a)lists.wikimedia.org>
Just to let it known, some it.source contributors are using a comfortable
gadget to manage diacritics - it can delete, replace or add a pretty large
list of diacritical marks to any character with a single click.
It uses .normalize() string method, so decomposing-recomposing (when
possible) unicode characters and allowing to manage diacritics alone
indipendently from base ascii character.
Perhaps is this gadget "rediscovering the wheel"....? Anyway, the code is
here: https://it.wikisource.org/wiki/MediaWiki:Gadget-pulsanti-diacritici.js
Alex brollo
_______________________________________________
Wikisource-l mailing list
Wikisource-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikisource-l
Due to today's train deployment more than doubling the error rate seen in
fatalmonitor[1], I have rolled-back group 1 wikis to wmf.11. I've submitted
an UBN task T172320 [2] detailing the error, which appears to be fallout
from issues in Wikibase first noticed two weeks ago.
I expect that this will be easily resolved once someone familiar with the
issue has had a chance to look into it and the train should be back on
track in time for tomorrow's deployment.
1. https://logstash.wikimedia.org/goto/5acde54fdb7b071b5723607c9f57eb9c
2. https://phabricator.wikimedia.org/T172320
Hello all,
Next week is Wikimania (yay!), for those of you who weren't aware.
Due to having good non-Wikimania coverage in both Release Engineering
and Operations we will, however, be doing the normal MediaWiki train
that week along with SWAT and (if needed) service deploys.
Please don't merge risky things thinking that you have an extra week
to make them less risky.
Best,
Greg
--
Greg Grossmeier
Release Team Manager
Hey,
For patrolling work, ORES usually has two levels of support:
- For basic support we usually provide a model that is called 'reverted'
and has less accuracy. It also risks perpetuating editor biases due to the
lack of differentiation between reasons that a change may have been
reverted.
- For advanced support, we require manual labeling of a large sample of
edits, but then we can provide two more models: 'damaging' and 'goodfaith'.
ORES review tool can only be enabled on wikis where advanced support is
available and most other tools prefer the 'damaging' over the basic
'reverted' model as well.
So, for performance and capacity reasons we think it makes sense to remove
'reverted' models from the ORES service when 'damaging' model is made
available. However, we also want to be careful about making sure this
change doesn't disrupt the work of tool developers that make use of the
ORES service.
If you do, please voice your concerns now. If there is no objection within
the next two weeks, we'll begin the process of removing the 'reverted'
model for wikis that have the 'damaging' model available.
Related phabricator card: https://phabricator.wikimedia.org/T171059
Best
--
Amir Sarabadani Tafreshi
Software Engineer (contractor)
-------------------------------------
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
http://wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
Hey everyone,
This is an email about Tech News. Partly to introduce or explain what
it is to those who might be unfamiliar with it, or have seen it but
don't know it well. Partly to ask if there's anything we could do
better. If you know all there’s to know, there’s a request for
feedback towards the bottom of this email.
ABOUT TECH NEWS
a) What is Tech News?
https://meta.wikimedia.org/wiki/Tech/News
Tech News is a newsletter for reaching out with technical updates to
the general Wikimedia editor communities, to make sure they can keep
track of what's happening. It's typically distributed in 15–20
languages, reaching roughly 90 community pages (Village Pumps etc) and
some 600 individual subscribers, in addition to those who read it on
Meta, see it included in the Signpost or get it in their email inbox.
b) How is Tech News written?
Simplification is key. Technical news for non-technical readers.
Should be easy to translate as well as be written with en-1 and en-2
readers in mind. A couple of sentence per item, then a link to a Phab
task, wiki page or email if they need more information. Too long and
we put an unreasonable burden on the translators.
c) I've done something technical. The communities should know. How do I add it?
* There's a "user-notice" tag in Phabricator. Add it to the task
together with a simple 1–3 sentence explanation of what this is and
how it affects editors. Don't worry about polish, we'll take care of
that.
* You can add it yourself!
https://meta.wikimedia.org/wiki/Tech/News/Next will take you to the
relevant issue. Remember to link to a relevant Phab task, wiki page or
email.
* You can always write on https://meta.wikimedia.org/wiki/Talk:Tech/News
d) When is it distributed?
Weekly, each Monday afternoon/evening UTC. The deadlines for additions
are several days prior to that to give the translators time to do
their work.
See https://meta.wikimedia.org/wiki/Tech/News/For_contributors#When_is_the_work…
for when to add things to have them included in the next newsletter.
e) What is Tech News not?
* A general Wikimedia newsletter. Everything in the Wikimedia world is
at most one step removed from being technical. This doesn't mean Tech
News is the best place. Typical items are new or upcoming features or
potential breaking changes.
* The way to reach the Wikimedia technical community. If you want to
reach Wikimedia developers, an email to wikitech-l is usually better
than an item in Tech News. Tech News is a way to keep Wikimedia
contributors up-to-date with technical changes.
* A way to talk about all the important things that happen in the
background. They're often awesome and we should talk more about them,
but if they don't affect how contributors interact with the sites,
then this is not the place.
* A place for updates about one wiki. If it's just relevant for
English or German Wikipedia, you should update English or German
Wikipedia, not the entire Wikimedia community. Exceptions to this rule
are Commons and Wikidata, because they're used by so many other
Wikimedia wikis.
More:
https://meta.wikimedia.org/wiki/Tech/News/For_contributors#What_is_typicall…
HOW CAN WE MAKE TECH NEWS BETTER?
Are there ways we could make Tech News better at spreading information
about technical updates that are relevant for Wikimedia contributors?
Something we do that's unnecessary? Things we're often missing? Things
we fail to explain? Anything we're particularly good at and should
keep doing? Tell us – on the list, on
https://meta.wikimedia.org/wiki/Talk:Tech/News, or privately (:
And, specifically – are there other places (wiki pages etc) where this
information should be available? Anything we could do to make this
easier for you?
//Johan Jönsson
--
https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-08-02
= *2017-08-02* =
contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering
== Callouts ==
*** Android App: We are investigating user reports of reading list pages
saved for offline not functioning correctly; trying to reproduce issues.
*** WikiData: Wikidiff2 patches about moved paragraphs need review:
https://phabricator.wikimedia.org/T146781
***Tech Ops: Still ongoing issues with pdfrender getting stuck
https://phabricator.wikimedia.org/T159922
*** Scoring Platform: Large file support in git: subscribe to this task if
you might have similar requirements!
https://phabricator.wikimedia.org/T171758
== Audiences ==
=== Readers ===
==== Web ====
Starting work on new page summary API
Desktop print styles
==== iOS native app ====
* Blocked by: none
* Blocking: none
* Updates: 5.6.0 (Reading themes, on this day) final bug fixes & polish,
submit to App Store on Friday 8/4
==== Android native app ====
* *Blocked by:* n/a
* Blocking: n/a
* Updates:
** Cookie release work is all in QA/design review –
https://phabricator.wikimedia.org/project/view/2763/
*** We are investigating user reports of reading list pages saved for
offline not functioning correctly; trying to reproduce issues. Want to
nail this down before the release.
** Offline compilations MVP client-side work is nearly complete –
https://phabricator.wikimedia.org/project/view/2833/
*** Work on file generation and storage is underway (
https://phabricator.wikimedia.org/T170843 )
** One engineer is newly hired, one more offer still to be made; PM search
is in progress
==== Reading Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** Extracting structured reference lists (JSON).
** Fixing pronunciation parsing.
** Gergo on vacation until Wikimania
==== Discovery (maps) ====
* Blocked by: none
* Blocking: none
* Deploying mapframe/maplink to four more wikis that the community has
requested
* Progressing with reloading test server
== Community tech ==
Updates:
* Human-readable sections merged, wikitech-l announcement today
* CodeMirror and LoginNotify hopefully deploying soon
* Started work on GlobalPreferences and ArticleCreationWorkflow
Not blocking and not blocked
=== Contributors ===
==== Global Collaboration ====
===== Language =====
* No blockers.
* Niklas looking at, T170591. See:
https://phabricator.wikimedia.org/T170591#3492148
* Update:
** cxserver's adaption changes merged; deploy pending.
** CX-VE work continue.
===== Collaboration =====
* Updates
** RCFilters
*** Filter duplicates when filtering for multiple tags
** RCFilters: Improve loading animation
*** Make 'related links' collapsible
*** Create a sticky preference for days/limit groups
*** Allow setting a new query as default
*** Ability to page through the results in the new UI
*** Some bug fixes
* Blocking
** Flow dumps - https://phabricator.wikimedia.org/T172025
==== Parsing ====
* Language Converter support code in Parsoid was deployed y'day -- the
second part of rendering a page in the desired language variant is the next
part in development
* Going to be publishing weekly linter count change stats on wiki --
https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy/Linter/Stats/July31 for
example
** Weekly plug :-) if you are an editor on a wiki, please help fix these
high-priority linter issues on your home wiki.
==== UI Standardization ====
* Updates:
** OOjs UI: v0.22.4 released, among changes:
*** (continued from last week): Further aligning OOjs UI with WikimediaUI
Base
**** WikimediaUI theme: Directly use the Less values from WikimediaUI Base
(James D. Forrester)
*** Accessibility FieldsetLayout: Use `<legend>` now that Chrome 55 bug is
less important (James D. Forrester)
*** Apex theme: Introduced focus states on all widgets as accessibility/UX
improvement
*** WikimediaUI theme: Work around a Firefox rendering bug for checkboxes
and radios (Bartosz Dziewoński)
== Wikidata ==
* Further improvements on the Query Service UI:
https://phabricator.wikimedia.org/T170279
* Preparing for Wikimania: cleaning up Lexeme demo data, widgets, and
design.
== German Technical Wishlist ==
* Blocked by:
** Waiting for C++ reviews of the wikidiff2 patches for changes in moved
paragraphs https://phabricator.wikimedia.org/T146781
*** the plan is to get at least +1s and then poke Tim
*** https://gerrit.wikimedia.org/r/#/c/356582
*** https://gerrit.wikimedia.org/r/#/c/319866
=== Search Platform ===
* Blocked by: none
* Blocking: none
* Updates:
** Continuing work on ML-assisted ranking
** Working on A/B test with interleaving search results (
https://phabricator.wikimedia.org/T150032 )
** Vietnamese analyzer will be re-evaluated after upstream bugfixes (
https://phabricator.wikimedia.org/T170423 )
** Archive search enabled everywhere (
https://phabricator.wikimedia.org/T163235 ), needs bugfix (
https://gerrit.wikimedia.org/r/#/c/369696/ )
** Wikidata prefix search with Elastic testing continues, some feedback
received & processing (test: http://elastic-wikidata.wmflabs.org/wb.html,
announcement:
https://lists.wikimedia.org/pipermail/wikidata/2017-July/010964.html )
** Trey published docs for tools used to analyze language analysers:
https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Analysis_Analysis_To…
=== Services ===
* Blockers: none
* Updates:
** Will truncate stored HTML soon to free up space for Cassandra 3 migration
*** No disruptions for VE
*** A bit elevated latencies for some time
** Testing new data model in the RESTBase-dev cluster. Please ignore alerts
from it
** Recommendation service API going public tomorrow
** Low team availability next week due to Debconf
=== Technical Operations ===
* Blocked by:
** Flow dumps speedup still blocked on revision content retrieval issue,
https://phabricator.wikimedia.org/T172025
** Still ongoing issues with pdfrender getting stuck
https://phabricator.wikimedia.org/T159922
* Blocking:
* Updates:
** Wikidata and dewiki read only for 1hour+
https://wikitech.wikimedia.org/wiki/Incident_documentation/20170728-s5_(Wik…
=== Fundraising Tech ===
* CentralNotice: working on clone campaign feature, fixed a couple little
bugs
* Working on generic script to clear up stranded payments
* Loading ever more mailing data into Civi from our 3rd party bulk mailer
* Continuing work on API update for our main credit card processor
* Also updating our audit parser for the same processor
=== Scoring Platform ===
* Blocked by:
**Note git-fat is still a long-term blocker, and some solutions are being
discussed.
* Blocking others:
**Mention that we're trying to resolve our blocker to using the new ORES
server cluster, by doing stress tests to estimate capacity. When we're
done, the celery workers will be moved off of SCB nodes
* Updates:
**Revscoring 2.0 is coming -- going break ORES "model_info" JSON structure
& add new functionality
***Will be deployed on labs before in prod, so code can be updated and
tested
***Announcement coming -- probably during Wikimania
***"Thresholds" is something Roan knows about
-
=== Security ===
* Reviews
** vue.js
** Kartography extension
** review of tables replicated to sanitarium