Hi,
I meant to send this Tuesday but I forgot.
MediaWiki 1.31 has been branched from master! You should now see a REL1_31
branch where appropriate items should be backported to.
Core was branched at 69257de17fc899c447c9f1229b6ed319bc05d316.
All extensions & skins were branched from their respective masters at about
the same time as core. I plan to cut rc.0 sometime next week.
PHP versions
The current plan of action is to leave master as compatible with 5.5 for
now. This is because Wikimedia production isn't ready quite yet. This is
being tracked at T172165[0]. We will be moving the REL1_31 branch to 7.0+
as the required minimum version. Once production is ready, we'll
forward-port this change to master. It should be a little inconvenient, but
not too terribly bad (and notably, makes life less stressful for our SREs).
In the meantime, please do NOT introduce changes to master that require
7.0+ for core, vendor, or WMF-deployed extensions & skins. Doing so will
make me sad :(
Otherwise, great job on 1.31.x everyone! I'm rather pleased with what I'm
seeing so far. Check out the workboard[1] for ways you can contribute to
getting it wrapped up (and as always, tag issues with that tag if they
should absolutely block release).
Have a fantastic weekend!
-Chad
[0] https://phabricator.wikimedia.org/T172165
[1] https://phabricator.wikimedia.org/project/view/3011/
[ Crossposting my wikitech-ambassadors post from y'day for those you
active on different wikis. ]
Hello everyone,
TL:DR;
------
As you are aware from previous postings on this list [1] [2] [3] [4] [5]
[6], we have been progressively replacing Tidy with RemexHtml on all
wikis on the wikimedia cluster. As of today, about 650 wikis have made
the switch that include a number of large wikis. We aim to complete this
switch over on the remaining 250 wikis by end of June 2018. Another 40
or so wikis will be switched on May 2nd.
There are a few large wikis (es, pt, uk, zh especially) that could use
more attention addressing Linter issues so that when we make the switch
end of June, some pages on these wiki don't render differently from how
they do now.
Details:
--------
I started investigating more closely where the remaining large wikis are
with respect to the linter issues (high priority categories on the
Special:LintErrors page) that are pertinent to these wikis. I am listing
below results from running sql queries on quarry.wmflabs.org for these
wikis. If you are a community member on any of these wikis, do try to
address these on your wiki.
15 other large wikis:
See https://quarry.wmflabs.org/query/26474 for counts of linter issues
for each of the 9 categories in the main namespace.
* es, pt, uk, zh wikis have total error counts over 10K and in some
cases, it is usually one category which needs attention.
* vi, ro, sr, sh, ar, tr, id are not too bad but don't seem to have seen
a lot of change which indicates that these wikis aren't looking at
linter issues.
* fr, hu, ja, pl wikis seem to be in good shape overall. There has been
a steady fixing of issues and I think all these will will be in fairly
decent shape for replacing Tidy by end of June.
https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy/FAQ#Simplified_instru…
has some summarized instructions for fixing issues in different categories
English Wikipedia:
See https://quarry.wmflabs.org/query/25665 for counts of linter issues
for each of the 9 categories in the main namespace.
English wp has been making slow and gradual progress. I think overall,
despite there still being ~8300 instances (not pages) that need fixing,
enwp is in pretty good shape for replacing Tidy by end of June.
Commons:
See https://quarry.wmflabs.org/query/25693 for counts of linter issues
for each of the 9 categories in the File (ns6), Gallery (ns0), and
Template (ns10) namespaces.
The vast majority of html5-misnesting errors on commons seem to come
from the use of the {{lang}} template which uses a <span> tag to wrap
content. However, it seems to be extremely common to pass content with
paragraphs into the {{lang}} template. Right now, this doesn't cause any
visible rendering issues and could be ignored temporarily, but we
strongly recommend fixing lang to use <div> or on pages which misuse
{{lang}} this way, replace use of {{lang}} by creating a new template
({{lang-block}} maybe?) that uses a <div> tag.
Some tips:
----------
1. On some wikis, fixing templates usually fixes the problem. Over the
last 6 months, I've personally spent many hours fixing 100s of templates
on 10s of different wikis and can personally attest to the efficacy of
that strategy.
2. A lot of the html5-misnesting errors seem to be from incorrectly
using a <span> tag to wrap content that has paragraphs, lists, tables.
In all these cases, changing them to <div> almost always fixes the problem.
If you need any assistance, please leave a message on
https://www.mediawiki.org/wiki/Help_talk:Extension:Linter. Between 8 am
- 4pm PST, you can also usually find us on IRC on #mediawiki-parsoid.
Thanks,
Subbu.
(on behalf of the Parsing team @ Wikimedia Foundation)
1.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2017-July/001625…
2.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2017-August/0016…
3.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-January/001…
4.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-February/00…
5.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-March/00180…
6.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-March/00182…
Hi everyone,
This is our last reminder for you to complete the Wikimedia Communities &
Contributors survey.
* To those of you who have taken the survey - thank you so much! We really
appreciate your responses. *
*This survey is closing in less than three days on Sunday 22 April 2018.*
*If you are volunteer developer, and have contributed code to any pieces of
MediaWiki, gadgets, or tools, please complete the survey. The opinions you
share will affect the work of the Wikimedia Foundation. *
*Follow this link to take the
survey: https://wikimedia.qualtrics.com/jfe/form/SV_5ABs6WwrDHzAeLr?aud=DEV
<https://wikimedia.qualtrics.com/jfe/form/SV_5ABs6WwrDHzAeLr?aud=DEV>If you
have already seen a similar message on Phabricator, Mediawiki.org,
Discourse, or other platforms for volunteer developers, please don't take
the survey twice. It is available in various languages and will take about
20 minutes to complete.You can find more information about
this survey on the project page
<https://meta.wikimedia.org/wiki/Community_Engagement_Insights/About_CE_> and
see how your feedback helps the Wikimedia Foundation support contributors
like you. This survey is hosted by a third-party service and governed
by this privacy statement
<https://wikimediafoundation.org/wiki/Community_Engagement_Insights_2018_Sur…>.
Please visit our frequently asked questions page
<https://meta.wikimedia.org/wiki/Community_Engagement_Insights/Frequently_as…>
to
find more information about this survey. Feel free to email me directly
with any questions you may have.Thank you!Edward Galvez from the Community
Engagement departmentWikimedia Foundation*
--
Edward Galvez
Evaluation Strategist, Surveys
Learning & Evaluation
Community Engagement
Wikimedia Foundation
https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-04-18#Android_native_app
= 2018-04-18 =
== Callouts ==
* Fundraising campaigns
https://meta.wikimedia.org/wiki/CentralNotice/Calendar - Netherlands
2018-04-03 through 2018-05-01
* Analytics: we added trash folder to hadoop to “aid” in case of accidental
deletions of data in cluster.
* Need help from RelEng with WDQS GUI build automation:
https://phabricator.wikimedia.org/T189811
* Research: In the next two/three weeks we'll be collecting citation usage
data via EventLogging: https://phabricator.wikimedia.org/T191086andhttps://meta.wikimedia.org/wiki/Schema:CitationUsage. cc Reading
Infrastructure, Adam Baso, Security, Legal, Analytics.
* Phabricator: Create tasks with a Deadline date/time which is displayed on
workboards: https://phabricator.wikimedia.org/maniphest/task/edit/form/37/
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
** Reading lists at 10% production, full rollout pending regression
tests and finalizing marketing materials.
==== Readers Web ====
* Blocked by:
* Blocking:
* Updates:
==== Readers Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
===== Maps =====
* Blocked by:
* Blocking:
* Updates:
==== Multimedia ====
* Blocked by:
* Blocking:
* Updates:
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
** GlobalPreferences, CodeMirror, TemplateWizard
==== Anti-Harassment Tools ====
* Blocked by: None
* Blocking: None
* Updates:
** Improving block notices on mobile
** Adding cookie blocking for anonymous users
==== Editing ====
* Blocked by:
* Blocking:
* Updates:
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
==== Collaboration ====
* Blocked by:
* Blocking:
* Updates:
==== Language ====
* Blocked by:
* Blocking:
* Updates:
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
==== UI Std ====
* Blocked by:
* Blocking:
* Updates:
** OOUI – v0.26.4 released:
https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v…
*** One hotfix, that has already made it to MW1.31 before, one keyboard
accessibility improvement
** Continued work on icon/size unification aftermath in different products
(Notifications, Flow, RevisionSlider)
** Style Guide technical and contents improvements ongoing with 2 open PRs
currently
** Transforming core/extensions to use OOUI, 6 extensions accomplished last
week, build on strong volunteer support
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
- Monitoring the launch of the page previews, currently working on the
aggregation job.
- Done with updates to jupyter notebooks for now
- Issues with pageviews from Opera Mini not geo locating correctly in
Africa are now resolved.
- Fnishing new geowiki infrastructure before we turn off the old jobs that
calculated this data (editors per country).
- Google Summer of Code, plenty Code Reviews for contributions to Wikistats
UI: http://stats.wikimedia.org/v2
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Updates:
Working on new recurring contribution API for main card processor
Debugging CN patch deploy to add CSP headers on banner preview
CiviCRM: Work on exports, fixing custom fields, new fraud-hunter's
report
Working on getting landing page and banner impression EventLogging data
into payments cluster DBs
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
=== Performance ===
* Blocked by: None
* Blocking: None
* Updates:
** Moving our apps/etc out of puppet, to scap deploy
** Blog post on Singapore DC should be up shortly, been working on that
with Traffic
** Lots of CRs
** working with RelEng/Platform on a few changes to CI and to MW Core
** mcrouter is now deployed in deployment-prep, next steps are to
configure, and working with SRE to figure out how to do cross-DC correctly.
** Trying out Digital Ocean for WebPageTest (got free credits from them)
=== Release Engineering ===
* Blocking
** Working on Scap 3.8.0 to unblock ores git-lfs deployments.
* Blocked
** n/a
* Updates
** Deadlines in phabricator are now visible on workboards.
=== Research ===
* Blocked by:
** None
* Blocking:
** None
* Updates:
** In the next two/three weeks we'll be collecting citation usage data via
EventLogging: https://phabricator.wikimedia.org/T191086 and
https://meta.wikimedia.org/wiki/Schema:CitationUsage. cc Reading
Infrastructure, Adam Baso, Security, Legal, Analytics.
=== Scoring Platform ===
* Blocked by:
* Blocking:
* Updates:
** Waiting on the scap deployment to unblock us from git-lfs--it works on
beta :) (Almost there! Thanks Releng!)*
** Virtualenv is now cached with deployments (so that we can quickly
rollback)
*** This is relevant for anyone who is deploying a python service with
virualenvs as they might want to follow our lead
** New, dynamic table with ORES support status.
*** https://tools.wmflabs.org/ores-support-checklist/
=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
- Need help with WDQS GUI build automation:
https://phabricator.wikimedia.org/T189811
- Selenium Wikibase tests still broken:
https://phabricator.wikimedia.org/T189762
- Job queue issues fix merged: https://phabricator.wikimedia.org/T191024https://phabricator.wikimedia.org/T190958
- Lexeme completion search merged:
https://phabricator.wikimedia.org/T190970 Continuing work on Forms
support
- Evaluating feature generation by `query_explorer` functionality of LTR
plugin: https://phabricator.wikimedia.org/T187148
- Working on query parsing refactoring:
https://phabricator.wikimedia.org/T185108
=== Security ===
* Blocked by:
* Blocking:
* Updates:
=== Services ===
* Blocked by:
** search with CirrusSearch job JSON serialization T191024 (Not any
more according to Stas :)
** language with Translate job JSON serialization T192111
* Blocking: none?
* Updates:
** There was an issue with wikidata recent changes job, resolved now
** EventBus (and kafka job queue) going to private wikis this week
** Cassandra 3.11.2 evaluation
=== Site Reliability Engineering ===
* Blocked by:
** None
* Blocking:
** None
* Updates:
** ICU57 migration completed on all wikis
** Work started on upgrading all mediawiki application servers to Debian
Stretch
** The image scalers have been decommissioned (thumbor provides fully that
fuctionality now)
** mcrouter is being tested and are going to post an architecture proposal
for review
** Discussions on establishing a timeline for deprecation of
non-forward-secret ciphers and TLSv1.0 started
** Proton discussions came to a result on hosting this on 2 ganeti VMs per
datacenter, will be resolved this week
** helm charts repo published https://releases.wikimedia.org/charts
== Wikidata ==
* Blocked by:
* Blocking:
* Updates:
**Send thanks to John and the security team for the quick response they
gave us for our security review for our coming release
**Lexeme will be released on May 23rd
**wb_terms fire fighting is stopping, strategy work starts now how to
get ourselves out of the situation properly and in a way we can scale
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
** MediaWiki CodeSniffer performance improvements already released as
version 18, thanks Lego!
** Backported a fix in a 10 year old File export/import util, please watch
for positive/negative consequences if this sounds familiar:
https://gerrit.wikimedia.org/r/426959
** Final 2 sprints working on MoveFilesToCommons:
https://phabricator.wikimedia.org/tag/move-files-to-commons/
** DeepCategory integration in AdvancedSearch:
https://phabricator.wikimedia.org/T170533
== SoS Meeting Bookkeeping ==
* Updates:
Hi!
With the recent final release of Page previews (Extension:Popups), we have
also published a series of technical blog posts about the frontend tooling
that we have used for developing this extension.
Master post: Extension:Popups (Page Previews) front-end tooling
<https://phabricator.wikimedia.org/phame/post/view/93/extension_popups_page_…>
Table of contents:
- Automatic JavaScript file bundling and library consumption
- Better minification for frontend sources
- Fast and isolated JS unit tests
- Conclusions
We hope this is interesting to read and hopefully useful to other teams or
projects working on products with complex front-end needs.
We're more than happy to chat about these topics, or help you with these
kind of workflows if you are interested.
Have a nice day!
---
Joaquin - Jhernandez - JHernandez (WMF)
Readers Web Team
Hi all,
Page previews[1] have now been promoted out of beta to the remaining two
Wikipedias - English and German, after a series of consultations on both
wikis [2][3][4]. The feature is now on by default for all logged-out users
and off by default for all logged-in users (can be enabled using user
preferences) on all Wikipedias. Further changes to configuration based on
our conversations with communities (such as turning the feature on for new
accounts) are coming up soon.
Page previews allows readers to read an excerpt of a linked article’s lead
section without leaving the page they’re currently on, by hovering their
mouse over the link. It has been activated by default for logged-out users
on all Wikipedias except German and English since August 2017, and numerous
further bug fixes and technical improvements have been implemented since. A
few weeks ago we published the results of our latest round of A/B testing
[5]. We found that when the feature is enabled, readers will open pages in
their browser slightly less often (a decrease of around 3–5% in regular
pageviews). But on the other hand, they interact with a lot more different
pages when one counts both the seen page previews and the regular pageviews
(an increase of around 20–22% in the number of distinct pages interacted
with via either method). In addition, the option to deactivate the feature
was used very rarely (disable rates were around 0.01%). This leads us to
believe that Page Previews is a welcomed feature that is helping readers
learn more during their visits.
More information about Page previews can be found on the project page on
MediaWIki.org [1]. The team also recently wrote a few blog posts. One about
the work that went into the design of the feature [6] and another giving an
overview of the functionality [7].
If you see any issues, please report them in Phabricator. [8]
Thanks,
Olga
[1] https://www.mediawiki.org/wiki/Page_Previews
[2]
https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)/Archive_16…
[3]
https://de.wikipedia.org/wiki/Wikipedia:Fragen_zur_Wikipedia/Archiv/2018/Wo…
[4]
https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)#Next_S…
[5] https://www.mediawiki.org/wiki/Page_Previews/2017-18_A/B_Tests
[6]
https://medium.com/freely-sharing-the-sum-of-all-knowledge/how-we-designed-…
[7] https://blog.wikimedia.org/2018/04/17/wikipedia-page-previews/
[8] https://phabricator.wikimedia.org/tag/page-previews/
--
Olga Vasileva // Product Manager // Reading Web Team
https://wikimediafoundation.org/