Hello everyone,
On behalf of the Parsing Team @ the WMF, I am announcing our plans to
replace Tidy with Remex on the next set of wikis.
On April 4th, we plan to turn off Tidy on all wikiquotes (except
frwikiquote) [1] and wikimedia wikis [2]. 23 wikis will have Tidy replaced.
On April 11th, we plan to turn off Tidy on all wikis with < 50 entries
in all high priority linter categories [3]. About 60 wikis will have
Tidy replaced.
Please follow T175706 [4] to monitor progress of Tidy replacement.
Currently about 600 wikis have had Tidy replaced and we have another 300
wikis to go. We plan to finish this transition from Tidy to RemexHtml by
end of June 2018.
If you have any questions or concerns, please leave a comment on the
associated phabricator tickets or leave a message on mw.org [5].
Thanks,
Subbu.
[1] https://phabricator.wikimedia.org/T188881
[2] https://phabricator.wikimedia.org/T190726
[3] https://phabricator.wikimedia.org/T190731
[4] https://phabricator.wikimedia.org/T175706
[5] https://www.mediawiki.org/wiki/Help_talk:Extension:Linter
TL;DR see <https://phabricator.wikimedia.org/T179461> for a proposed
naming change.
I proposed in Phabricator a wile ago [0] that we adopt the term
"Wikimedia developer account" across our wikis and other documentation
for the LDAP-backed user accounts that are created using
wikitech.wikimedia.org.
These same sign-on credentials are used for:
* wikitech.wikimedia.org
* Gerrit
* Phabricator (optionally)
* toolsadmin.wikimedia.org
* horizon.wikimedia.org
* ssh-based "shell access" to Toolforge and Cloud VPS servers
* a variety of web services providing access to operational and
analytics data related to the Wikimedia services
There are plans [1][2] in various stages of progress to change things
about how developer accounts are created and managed. Breaking
assumptions in our documentation about the back-end storage system
(LDAP) and the front-end management interface (wikitech) will help
make these changes less disruptive. It also helps remove another
lingering "labs" reference that the Cloud Services rebranding efforts
have been trying to address.
This change probably has almost no impact on existing technical
community members. It is really just targeted at making things a bit
less confusing for newcomers.
If you have thoughts or concerns about this proposal, please describe
them on the Phabricator task [0]. I'm proposing that on or about March
23rd (4 weeks from the date of this message) the comments on the task
will be evaluated for consensus and an approve/deny decision.
[0]: https://phabricator.wikimedia.org/T179461
[1]: https://phabricator.wikimedia.org/T161859
[2]: https://phabricator.wikimedia.org/T179463
Thanks,
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA
irc: bd808 v:415.839.6885 x6855
Hello everyone,
We are releasing the next version of the Parsoid deb and npm packages
(v0.9.0) later today. There is one significant change in this release
that might affect some VisualEditor installations. This version of
Parsoid wraps sections in <section> tags and bumps the HTML version to
1.6.1. However, VisualEditor installations older than Dec 12, 2017 are
not compatible with this section wrapped output.
In order to prevent silent failures, Parsoid will do a hard fail and
reject parse requests with a HTTP 406 when it receives a version string
smaller than 1.6.0 (which VE before Dec 12, 2017 would issue). VE will
then popup the following error: "Error loading data from server:
apierror-visualeditor-docserver-http: HTTP 406. Would you like to
retry?" Retries won't help in this scenario.
Knowing this, you have a couple of options:
* Not upgrade Parsoid. If you upgrade Parsoid without reading this
notice and later stumble on this (which we'll also add to the
Parsoid releases page), you have the option of downgrading your
Parsoid install by downloading an older deb package from
https://people.wikimedia.org/~ssastry/parsoid/debs/
* If you choose to upgrade Parsoid and your VE installation is from
before Dec 12, 2017 (which is probably most of you), the recommended
solution is to upgrade your VisualEditor installation as well.
If for some reason you really really need to upgrade Parsoid but cannot
upgrade VisualEditor, contact us on IRC in #mediawiki-parsoid to ask us
how you can do that.
Subbu.
(on behalf of Parsoid developers).
Hello everyone,
I'm excited to announce that we've released OOUI v0.26.0 today. It
will be in MediaWiki core from 1.31.0-wmf.26, which will be deployed
to Wikimedia production in the regular train, starting on Tuesday 27
March.
In this breaking release email I would like to begin with pointing out
two “normal” changes with visible user-interface amendments:
- The Design team at the Wikimedia Foundation has established new icon
guidelines over the course of recent months – resulting in a new set
which is part of this release. The set features improved universality,
consistency, neutrality, developer-friendliness and RTL language
support.
- We took the chance of integrating those icons by also unifying
VisualEditor interface on same base size as all other Vector/OOUI
interfaces already had in place. While this leads to a minimal
increase of space (f.e. 2px in height) for the toolbar items, it
reduces and optimizes the interface in all existing OOUI-based
Special:Pages and extensions.
As there are seven breaking changes in this release, at least
nominally, please carefully consider if they affect your code:
* icons: Remove 'alignCentre', renamed in v0.24.2
* icons: Remove 'arrowLast', deprecated since v0.25.0
* icons: Remove 'bellOn', deprecated in v0.25.0
* icons: Remove 'quotesAdd', deprecated in v0.24.4
* icons: Remove 'redirect', renamed in v0.24.4
* indicators: Remove 'next' and 'previous', deprecated in v0.25.0
All six changes above have been deprecated and announced in former
versions. With this release we've removed them completely.
* WikimediaUI theme: Unify available variants across icon packs
This could affect 'wikimedia' pack users with icons set to
'progressive'. Due to branding guideline reasons we need to remove
this color variation We are not aware of any such use case in
production, nonetheless mark it breaking.
Please update your icon pack references accordingly in case you're
using one of those icons or variants.
Additional details on 12 new features, 53 code-level and accessibility
changes, 18 styling and interaction design amendments, and all
improvements since v0.25.0 are in the full changelog[0]. If you have
any further queries or need help dealing with breaking changes, please
let me know.
As always, library documentation is available on mediawiki.org[1], and
there is comprehensive generated code-level documentation and
interactive demos hosted on doc.wikimedia.org[2].
[0] - https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md
[1] - https://www.mediawiki.org/wiki/OOUI
[2] - https://doc.wikimedia.org/oojs-ui/master/
Best,
Volker
--
Senior UX Engineer
Wikimedia Foundation
volker.e(a)wikimedia.org | @Volker_E
https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-03-21#Multimedia
= 2018-03-21 =
== Callouts ==
* Fundraising campaigns
https://meta.wikimedia.org/wiki/CentralNotice/Calendar - next up:
Netherlands 2018-04-03 through 2018-05-01
* WMDE needs a Wikidiff2 review: https://gerrit.wikimedia.org/r/404293
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
** Increased rollout of Reading List sync to 50% of Beta audience.
** Blocked on a few minor client-side issues, but should continue full
rollout by end of week.
==== Readers Web ====
* Blocked by:
* Blocking:
* Updates:
*Interrupt work for Hindi Wikipedia campaign (
https://phabricator.wikimedia.org/T190101)
*Discussion around DNT (https://phabricator.wikimedia.org/T187277)
*Improvements and some refactoring to Popups extension
*Quarterly goal dependency update:
** Increase learning by lowering the cost of exploration (Page previews)
*** Turning on virtual page view logging for all wikis (
https://phabricator.wikimedia.org/T189906)
*** Pushing for full-deploy in next 3 weeks. (
https://phabricator.wikimedia.org/T154635)
*** This goal was impacted by interrupt work.
** Continue improving the ways that users can download articles of interest
for later consumption
*** Proton work was delayed until next quarter due to external dependencies
in standing up a production endpoint
==== Readers Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** New /page/media endpoint is exposed via RESTBase, so apps can test it
and provide feedback.
** /page/metadata and /page/references to follow this week.
===== Maps =====
* Blocked by:
* Blocking:
* Updates:
==== Multimedia ====
* Blocked by: N/A
* Blocking: N/A
* Updates
** Search: Patch for MediaInfo caption indexing nearly complete, WMDE and
Cormac working diligently to get things merged, Search supporting (thanks,
both)
** File page prototyping: Progressing, slowly. Mark to meet with Thiemo
from WMDE to clear some things up.
Quarterly goal dependency update
** Prepare backend infrastructure for structured data search
*** SDC: Search PF/Multimedia/Wikidata
** Wikimedia Technology/Goals/2017-18 Q3#Segment 2: Search integration and
exposure
*** SDC: Research/Multimedia
** Prepare backend infrastructure for structured data search
*** SDC: Search PF/Multimedia/Wikidata
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
* GlobalPreferences going live today
* Work on TemplateWizard progressing well
==== Anti-Harassment Tools ====
* Blocked by: None
* Blocking: None
* Updates:
* Work continues on Blocking tools
* Bug fixes to Interaction Timeline
* Research on Harassment Reporting Tool
==== Editing ====
* Blocked by:
* Blocking:
* Updates:
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
** Tidy has been replaced on all wikiversities today. About 300 more to go
before the end June deadline.
** Whitespace trimming in headings, list items, table cells, headings and
captions (wikitext versions only) has been pushed to production. Found a
bug for which there is a fix in gerrit
** Arlo has been working on the heredocs syntax RFC (
https://phabricator.wikimedia.org/T114432 ) and has a WIP patch @
https://gerrit.wikimedia.org/r/#/c/418198/ if someone wants to test and
play with it on your local dev wikis.
*Quarterly goal dependency update:
** Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
*** Parsing:Mediawiki PF/Services
*** No new updates or blockers.
==== Collaboration ====
* Blocked by:
* Blocking:
* Updates:
** Flow uninstalled from wikis where it was unused; can still be requested
by communities, will be re-enabled with local consensus
** Almost done setting up maps test servers in labs
==== Language ====
* Blocked by: None
* Blocking:None
* Updates:
** CX2 work continue;
** Migration to recommend service from Production is planned; WIP.
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
==== UI Standardization ====
* Blocked by:
* Blocking:
* Updates:
** OOUI – v0.26.0 breaking change release
https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v…
*** 7 breaking changes, mostly already deprecated icons got removed
*** WikimediaUI theme's new icon set in place
https://phabricator.wikimedia.org/T177432 !
*** Unified VE in Vector/WikimediaUI interface base font-size to usage of
OOUI in core/extensions elsewhere https://phabricator.wikimedia.org/T97631
!
** Style Guide
*** Continued work on v1 goals
https://phabricator.wikimedia.org/tag/wikimediaui_style_guide/
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
** 504s in Pageview API RESTBase layer, problem with RESTBase running out
of TCP ports, applying some fixes now to prevent the problem
** notebook1003 is replacing notebook1001, so we need to migrate folks
using Jupyter, watch for communication on the lists
** responsive Wikistats work is ongoing
** porting geowiki to Hadoop is done, building a dashboard on top of it now
** more analytics processes migrated to Kafka Jumbo cluster (webrequest and
EventLogging are fully migrated now)
** improvements on automatic refining of EventLogging topics to Hadoop
* Quarterly goal dependency update:
**Improve, adjust, or create features geared at the needs identified in New
Editors research project.
*** New Editors Experience:Analytics
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Updates:
** Upgrading CiviCRM install to 5.0
** More work on main CC processor new API, including redesigning recurring
contribution charges to fit better with Civi
** Making sure all the unintentional recurring donor refunds happened as
they were supposed to
** More work on using EventLogging to get banner impression and fundraiser
landing page stats
** Looking into applying CSP to banner preview pages, but need to decide
how to make it fit with pending core patch:
https://gerrit.wikimedia.org/r/253969
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
** Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
*** Parsing:Mediawiki PF/Services
* Reduce product and technical debt to modernise our tools and
technologies, and to make future changes more effective and efficient
*** Parsing/Mediawiki PF
**1.1 It is possible to store structured data within wiki pages, in
particular on media file pages on Commons. We will enable the MediaWiki
storage layer to correctly store and process structured data elements
within wiki pages.
*** SDC: Mediawiki PF/Wikidata
=== Performance ===
* Blocked by: None
* Blocking: None
* Updates:
** Collecting a bunch of oversample data from countries in Asia/Oceania,
prep for Singapore to go live.
** Thumbor 100% done
** Expecting to spend a bit of time helping with reviews or ResourceLoader
changes related to Hindi wiki front page stuff
** auto_prepend_file being used to load profiler
** Re-did coal changes to use python-kafka, confluent_kafka doesn't work
** Deploying RUMSpeedIndex this week, essentially emulates Speed Index
calculation using the browser resource timing API
=== Release Engineering ===
* Blocking
**
* Blocked
**
* Updates
** Minor Gerrit upgrade planned for this week (2.14.6 -> 2.14.7)
** Incident analysis started last week of the last year’s worth of
incidents reports
** Scap 3.7.7 should be rolled out to production this week
*Quarterly goal dependency update:
** Continue improving the ways that users can download articles of interest
for later consumption
*** Reading Web: Tech Ops/RelEng (work is currently blocked on
https://phabricator.wikimedia.org/T187821 which is part of a larger epic
https://phabricator.wikimedia.org/T181084)
** Talked about in team meeting Monday
** is there a task?
=== Research ===
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
** Wikimedia Technology/Goals/2017-18 Q3#Segment 2: Search integration and
exposure
*** SDC: Research/Multimedia
=== Scoring Platform ===
* Blocked by:
** Releng is helping us with a git-lfs pilot deployment.
** Waiting for Scap 3.7.7 for a fast-rollback process.
* Blocking:
* Updates:
** The JADE extension is in beta.
*** https://www.mediawiki.org/wiki/Extension:JADE
*** This is for auditing ORES. It adds "Jade" and "Jade_talk" namespaces
to MediaWiki. We're using JSON content with schema validation. Prateek is
contributing some volunteer design time.
=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
** Prepare backend infrastructure for structured data search
*** SDC: Search PF/Multimedia/Wikidata
** Update:
*** Trey published new blog about supporting languages with multiple
writing systems:
https://blog.wikimedia.org/2018/03/12/supporting-languages-multiple-writing…
*** AB test for query explorer feature for LTR finished & analyzed:
https://phabricator.wikimedia.org/T189843 Results are looking pretty good,
seems to be improving things.
*** Wikidata search patches merged. The new search not enabled yet, but
improved search result format is.
*** Fixed performance issue with LTR cache:
https://phabricator.wikimedia.org/T188015
*** TermLookup implementation for ElasticSearch merged:
https://phabricator.wikimedia.org/T143706
*** Fixed deepcategory keyword to not return results if keyword has failed,
e.g. because there are too many categories:
https://phabricator.wikimedia.org/T189331
*** Working on query parsing refactoring:
https://phabricator.wikimedia.org/T185108
*** Working on Slovak analysis: https://phabricator.wikimedia.org/T178929
*** Started implementation work on Lexeme search:
https://www.wikidata.org/wiki/User:Smalyshev_(WMF)/Lexeme_search
=== Security ===
* Blocked by:
* Blocking:
* Updates:
=== Services ===
* Blocked by: none
* Blocking: none
* Updates:
** New REST endpoint to get lint errors for page/revision
***
https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_lint_title_re…
** New REST endpoint to get page media
***
https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_media_title_r…
** refreshLinks topic is now partitioned in kafka according to MySQL
shard, should decrease connection spikiness
=== Site Reliability Engineering ===
* Blocked by:
** None
* Blocking:
** None
* Updates:
** Firewalling of misc database hosts successful. Some delays in OTRS email
delivery experienced
** Consistent logical backups in codfw goal completed successfully
** mathoid deployed in kubernetes. Production traffic is partly served
(~50%) from the new kubernetes cluster in both eqiad+codfw
** eqsin turn-up planning procedure meetings starting.
** EtcdConfig in production goal almost completed
** Some varnish issues ongoing https://phabricator.wikimedia.org/T189892
*Quarterly goal dependency update:
** Continue improving the ways that users can download articles of interest
for later consumption
*** Reading Web: Tech Ops/RelEng
** Update: No updates, was postponed for early next quarter
** Audiences DesignStandardise our user interfaces to match user
expectation of quality from our products
*** Audiences Design: Ops
** Update: Done AFAIK.
== Wikidata ==
* Looking into current Lua usage to see where we can improve the Lua
functions we provide: https://phabricator.wikimedia.org/T189506
* Optimizing a heavily used database table (wb_terms):
https://phabricator.wikimedia.org/T188279
* Polishing a lot of things for lexicographical data first deployment
*Quarterly goal dependency update:
**1.1 It is possible to store structured data within wiki pages, in
particular on media file pages on Commons. We will enable the MediaWiki
storage layer to correctly store and process structured data elements
within wiki pages.
*** SDC: Mediawiki PF/Wikidata
** Prepare backend infrastructure for structured data search
*** SDC: Search PF/Multimedia/Wikidata
== German Technical Wishlist ==
* Blocked on external review for the most recent Wikidiff2 optimizations:
https://gerrit.wikimedia.org/r/404293
* Working towards an MVP of the FileImporter extension:
https://phabricator.wikimedia.org/tag/move-files-to-commons/
* Finishing touches on AdvancedSearch:
https://phabricator.wikimedia.org/tag/advanced-search/
== SoS Meeting Bookkeeping ==
* Updates:
Hi,
As part of auditing MediaWiki initialisation code[1], I noticed the state
of our OutputHandler logic. It is currently implemented through a series of
global functions in a dedicated includes file (separate from
GlobalFunctions.php) which are then ad-hoc loaded (as needed) in
WebStart.php.
I wouldn't consider these part of the public interface, because they were
only conditionally included, and not in any autoloader. Stable usage would
require a consumer to directly require the file from $IP/includes.
However, given that on many wikis they will appear to be consistently
available, it is not impossible someone somewhere might use it somehow.
I found zero callers apart from the primary call in WebStart.php (plus some
Api comments, and a false-positive match in a unit test).
Not usage in MediaWiki core or elsewhere in Wikimedia Git (per
CodeSearch[2] and GitHub[3]).
As such, per our policy [4] I propose to refactor these without leaving
back-compat aliases.
Feedback welcome at https://gerrit.wikimedia.org/r/420223
-- Krinkle
[1] https://phabricator.wikimedia.org/T189966
[2] https://codesearch.wmflabs.org/search/?q=wfOutputHandler
[3]
https://github.com/search?q=org:wikimedia+wfOutputHandler+-repo:wikimedia/m…
[4] https://www.mediawiki.org/wiki/Deprecation#Removal_without_deprecation
I'm refactoring MonoBook, starting with MonoBookTemplate. The current
change gets rid of the entire immediate print/html soup approach and
instead assembles a giant string and prints that in one statement at the
end. See: https://gerrit.wikimedia.org/r/#/c/420154/ and
https://gerrit.wikimedia.org/r/#/c/420161/
Pending reviews and assurances that I have indeed not totally broken
everything, we'll probably be merging this in the next week or so. If
anyone would like to point out particular reasons why this is a terrible
idea, please do so now.
Future plans include putting that ridiculous getPortlet into core
BaseTemplate, making a less dumb BaseTemplate::getFooter without
breaking anything already using it so MonoBook can lose its silly
replication thereof, organising all the files in MonoBook better
(putting them in resources, includes, etc according to standard skin
practices), and making MonoBook responsive.
There are also some problems that need addressing down the road: that
I'm not sure how safe it is for caching and the like to just go moving
images/css files around willy-nilly, that there are no 'standard' skin
practices as far as anyone can tell, and that some people seem to think
MonoBook is bad and not worth this. But MonoBook is not bad. It is a
delightful skin. We should preserve it, and not just in formaldehyde.
-I