Next Friday we'll be upgrading our OpenStack cluster. The upgrade
should not interrupt any existing tools or instances, but during the
upgrade it will be impossible to create, delete, or modify WMCS VMs.
I'll start the process at around 14:00 UTC (7AM PDT). The complete
upgrade may take much of the day; Horizon will be disabled for the
duration of this update. CI/Nodepool will function intermittently
during this time.
-Andrew
Hi,
As part of the goal to have MediaWiki 1.31 default to using RemexHtml
and replacing tidy, we want to remove MWTidy::checkErrors() and
$wgValidateAllHtml as part of MediaWiki 1.31, without a deprecation period.
MWTidy::checkErrors() basically ran the provided input through tidy, and
checked if it complained. This doesn't make sense using an HTML 5
parser, since they provide recovery conditions to fix parse errors.
If $wgValidateAllHtml was set, MediaWiki would output an error page
instead of the content if tidy reported errors in the output.
Technically, this also means removing TidyDriverBase::validate() (unused
besides MWTidy::checkErrors()), and
MediaWikiTestCase::assertValidHtmlDocument/assertValidHtmlSnippet() (now
unused; previously only used in Capiunto as a defensive check).
There is some more rationale and discussion on the task for this: [2].
The Gerrit patch is [3].
[1] https://phabricator.wikimedia.org/T185753
[2] https://phabricator.wikimedia.org/T191670
[3] https://gerrit.wikimedia.org/r/#/c/425093/
-- James & Legoktm
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Hi,
tl;dr: Read the "So what do you need to do?" section
We're currently in the process of upgrading the MediaWiki PHPUnit
version to 6. To avoid an immediate breaking change, we will be doing
this transition in two steps. First, we will start running tests with
both PHPUnit 4 and 6 (next week), using a compatibility layer to
support both. Then at some point during the 1.32 or 1.33 release
cycles, we'll drop the compatibility layer and just support 6 (and
maybe a later version too?).
There are two parts to the compatibility layer. First, an autoloader
hack aliases most of the old PHPUnit_Framework_Whatever classes to the
new namespaced versions, PHPUnit\Framework\Whatever. This allows tests
that use the old class names to continue to work, without any
modification.
The second part is a trait, PHPUnit4And6Compat, which provides
implementations of the getMock() and setExpectedException() functions
that were removed in PHPUnit 6. It also has a forward-port of the new
createMock() replacement. MediaWikiTestCase uses this trait by default
So what do you need to do?
First, run your extension tests locally against PHPUnit 6. Edit
MediaWiki core's composer.json's dependency on phpunit/phpunit to say
"^6.5", run composer update, and then run your extension's tests.
If you run into issues about getMock/setExpectedException, then you
probably need to add "use PHPUnit4And6Compat;" to your test class. See
[1] for an example.
Other problems could relate to some class names changing. If you get
stuck, please cc or ping me and I can take a look.
I'd like to start running voting tests against PHPUnit 4 and 6 next
week on Thursday (April 12). This should give us just enough time to
fix any resulting issues before the 1.31 branch.
[1] https://gerrit.wikimedia.org/r/#/c/424843/
Thanks,
- -- Kunal / Legoktm
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEE+h6fmkHn9DUCyl1jUvyOe+23/KIFAlrJlKcACgkQUvyOe+23
/KKmbQ//UalxjQ9xzsn7sceXUM6y8YFgQtJH08RRq+34tZMXJSEPdoif3xlJm53L
5HfVuFcF73wAtLElM1JKjB30fYOD+7yZ+joLzrgBJ+81bHDncCK+ldOUPrSSHvcZ
PuWMcjJ9QimFSRon/ar09GRIEaL1WbXDM+J4EOlViThuTAevoqO+lgeSimkPBkPD
K1mxtpES5DGLRiQhAX1OZtuOJBuKfX0bshX7RO833Z2ZnwCLjcTTyyUQClANP9yR
xsVQCqtUTvtbMnx3zH3FhWxbI6qQosEqdK+nS2kwXMKw72roKXe3r6yDJtqHNaTy
gz/lj3HJYynNjVcF9sVDmBL5pqrpG5NLDiz4AZJrViaRLsfqX+ibcKPQpoaWh9fh
gz6Y7DzFpB3gDXepJmy+SJCidksB6YMOs+BQMQjqKBSrVHvFuKIt3zHxZ13aUQ+L
lCyp7pfB83Q1NpKm8ss2lUVGDiEL+1yU45c3MixBjdx/I4fCdF5RfdEpHA/N1scG
G4uMiPf9TprV3vHk57sS7AAZembqOGj5lTCceKAgFVil0TlHzsYjgBcaQbU0qecI
bb9/uxPeEP7pJ4mEbgJGzCa9aQaVI6qQ+LORC84BD24l3k8M/6pFfkZ52BOz2lAy
VE/2rXRcmFPbTkWgh8J3yEIjwgCeDOKK/JDVyhdIeWwz65MOOMs=
=BmSB
-----END PGP SIGNATURE-----
Hi everyone,
This is a friendly reminder about the Wikimedia Communities and
Contributors Survey.
*We have only heard from 50 Wikimedia volunteer developers. The survey will
close Sunday 22 April 2018.*
If you are volunteer developer, and have contributed code to any pieces of
MediaWiki, gadgets, or tools, please complete the survey. The opinions you
share will affect the work of the Wikimedia Foundation.
*Follow this link to take the survey:* https://wikimedia.qualtrics.
com/jfe/form/SV_5ABs6WwrDHzAeLr?aud=DEV
If you have already seen a similar message on Phabricator, Mediawiki.org,
Discourse, or other platforms for volunteer developers, please don't take
the survey twice.
It is available in various languages and will take between 20 and 40
minutes to complete.
You can find more information about this survey on the project page
<https://meta.wikimedia.org/wiki/Community_Engagement_Insights/About_CE_> and
see how your feedback helps the Wikimedia Foundation support contributors
like you. This survey is hosted by a third-party service and governed by this
privacy statement
<https://wikimediafoundation.org/wiki/Community_Engagement_Insights_2018_Sur…>.
Please visit our frequently asked questions page
<https://meta.wikimedia.org/wiki/Community_Engagement_Insights/Frequently_as…>
to find more information about this survey.
Feel free to email me directly with any questions you may have.
Thank you!
Edward Galvez from the Community Engagement department
Wikimedia Foundation
Hello all,
I would like to announce the release of MediaWiki Language Extension
Bundle 2018.04. This bundle is The bundle is compatible with MediaWiki
1.29 and 1.30 or above and requires PHP 5.5.9 or above.
Next MLEB is expected to be released in 3 months. If there are major
changes or important bug fixes, we will do intermediate release.
Please give us your feedback at
[[Talk:MLEB|https://www.mediawiki.org/wiki/Talk:MLEB]].
* Download: https://translatewiki.net/mleb/MediaWikiLanguageExtensionBundle-2018.04.tar…
* sha256sum: e8b4da822b2dcc7906e444a65e8924d8505c028c8636cdfe8d38059dc9ef1d1c
Quick links:
* Installation instructions are at: https://www.mediawiki.org/wiki/MLEB
* Announcements of new releases will be posted to a mailing list:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-i18n
* Report bugs to: https://phabricator.wikimedia.org/
* Talk with us at: #mediawiki-i18n @ Freenode
Release notes for each extension are below.
-- Kartik Mistry
== Highlights and upgrade notes ==
== Babel, cldr, CleanChanges and LocalisationUpdate ==
=== Noteworthy changes ===
* Localization and maintenance updates.
== Translate ==
=== Noteworthy changes ===
* Fixed compatibility with MediaWiki 1.29 release.
* Added the 'translate' right to the 'editpage' grant. This will allow
OAuth apps to translate.
* Translation variables can now span multiple lines.
* AndroidXml file format now includes authors.
* Maintenance and performance updates.
== UniversalLanguageSelector ==
=== Noteworthy changes ===
* Fixed Ctrl-Clicking in Compact Language Links. ([[phab:T189582|T189582]])
* ULS trigger and settings are more keyboard accessible now.
([[phab:T52793|T52793]])
* Language search results are not grouped by region.
--
Kartik Mistry/કાર્તિક મિસ્ત્રી | IRC: kart_
{kartikm, 0x1f1f}.wordpress.com
Hi everyone,
We are very pleased to be making three big announcements:
1. The Platform Evolution (PE) cross-departmental program (CDP) is approved in the annual plan for next fiscal year.
This PE CDP was developed with input from staff across our organization in the Audiences and Technology Working Group. Together, these staff worked to identify some of our most pressing issues while opening healthy discussions between both departments and WMDE.
Along with the other Technology programs funded in the annual plan, funding of this program represents a renewed commitment by the Foundation to the long term health of the technology that is key to supporting our mission, staff and communities.
2. We are creating the Core Platform team, a new converged platform and services team to be the focus of the Platform Evolution CDP.
We are doing this in order to better support the PE CDP and begin the hard work of re-architecting our technology stack into a more sustainable and flexible platform, in support of the Wikimedia movement strategic direction. The MediaWiki Platform team alongside the Services Platform team hold some of the most senior technologists of our community. Their skill sets and experience are vitally important for the success of the CDP so they will be moving into the new Core Platform team. We want to thank Tim Starling and Marko Obrovac for their hard work, leadership and dedication which has brought us to this point. They and their teams are doing incredibly important work to sustain our software stack day in day out while also looking into the future and guiding the reengineering of our platform to support the mission for the years ahead.
3. Corey Floyd will be joining the Technology Department to lead the Core Platform team.
In addition to day to day management, Corey will operate in the program management capacity for the PE CDP. Corey was instrumental in formulating the PE CDP, assembling the program through extensive needs analysis, synthesis, and collaboration with Foundation team members and WMDE. He brings a proven track record in clarifying stakeholder needs and translating them into amazing products.
As many of you know, Corey started at the Foundation as an iOS engineer, was promoted to manage the iOS and Android native apps engineers who worked tirelessly to evolve the open source apps into award winners beloved by millions of our users, and has been operating in an
engineering product owner capacity for the Infrastructure team within Audiences-Readers. He is known for his work ethic, thought leadership, real world experience, and collegial spirit.
Audiences will be working to backfill the duties of Corey in Apps engineering management and Reading Infrastructure product ownership, and work is already underway to close these gap in conjunction with Corey’s cutover to Technology.
We’re happy to make the Platform Evolution CDP official. And please join me in welcoming Corey to Technology, where he’ll transition on July 1, 2018. We’ll be sharing more updates about the team and the PE CDP in the coming weeks.
We are incredibly excited!
Victoria and Toby
https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-04-11
= 2018-04-11 =
*Q4 FY1718*
== Callouts ==
* Fundraising campaigns
https://meta.wikimedia.org/wiki/CentralNotice/Calendar - Netherlands
2018-04-03 through 2018-05-01
* OpenStack upgrades on Friday 2018-04-13 -- Horizon and CI
(Zuul/Jenkins/Nodepool) will be disabled for a while beginning around 14:00
UTC
* Still need help from RelEng with WDQS GUI build automation:
https://phabricator.wikimedia.org/T189811
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
**
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
** Reading list sync rolled out to 10% of production (~800K users)
** On track to roll out to full production next week.
==== Readers Web ====
* Blocked by:
[performance] https://phabricator.wikimedia.org/T42812 - nice
collaboration going on here, to improve mw-collapsible behaviour on desktop
- [releng] https://phabricator.wikimedia.org/T190710 - getting browser
tests ported to Node for Minerva - Thanks Željko for being so responsive!
* Blocking:
* Updates:
** Updating clients to reflect new wikidata description changes (
https://phabricator.wikimedia.org/T190953)
**Deploying Page previews to German Wikipedia (right now!!)
*Quarterly goal dependency update:
**Outcome 1, Objective 4: Continue improving the ways that users can
download articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
==== Readers Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** CSS endpoint going to be deployed in RESTBase after MediaWiki train
completes
** Working on very simplistic browser extension to add a single page to a
reading list
** TemplateStyles was deployed on ruwiki
*Quarterly goal dependency update:
**Outcome 1, Objective 4: Continue improving the ways that users can
download articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
**Increase code sharing of client apps by coalescing and moving more logic
to the server.
***Reading Infra depends on Parsing, Services
===== Maps =====
* Blocked by: None
* Blocking: None
* Updates:
** Working on internationalization and enwiki rollout
==== Multimedia ====
* Updates
** Wikibase patches for file page prototype going through
** Search work continuing
** File captions - currently working through changing the name of the field
for MediaInfo - it's a bear
*Quarterly goal dependency update:
**Objective 3.1 Prepare for launch of the first Structured Data on Commons
feature (multilingual file captions)
***SDC depends on Multimedia,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
** Objective 2.1 Integrate structured file captions into search
*** SDC depends on Search Platform, Multimedia
**Segment 4, Outcome 2: Develop a better understanding of existing needs
for Structured Commons- T171252
***Research depends on Multimedia
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
** CodeMirror is getting ready to promotion from beta
** Test deployment of GlobalPreferences today
==== Anti-Harassment Tools ====
* Blocked by:
* Blocking:
* Updates:
Blocking Tool Improvments
==== Editing ====
* Blocked by:
* Blocking:
** Updates:
** Goal 3.1 Improve and consolidate our unified editing platform so that
it's great on all devices
***Editing depends on Parsing
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
** Tidy replaced with RemexHtml on 40 more wikis today (see
https://phabricator.wikimedia.org/T190731 ) Currently have about 650
running RemexHTML, another 250 yet to convert.
** Parsoid is exploring the use of Finite State Transducer formalism for
language variants ( see https://phabricator.wikimedia.org/T191925 && gerrit
- https://gerrit.wikimedia.org/r/#/c/423197/ )
** TTO and us had an IRC meeting last week about the Poem extension and TTO
summarized the decisions in the phabricator task ( see
https://phabricator.wikimedia.org/T54061#4110198 )
** No quarterly goal dependency updates at this time. Services, Editing,
Parsing teams will be syncing up this/next week to identify any blockers.
*Quarterly goal dependency update:
**Goal 3.6 Support work towards unifying MediaWiki's parser
implementations, in liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
**Increase code sharing of client apps by coalescing and moving more logic
to the server.
***Reading Infra depends on Parsing, Services
** Goal 3.1 Improve and consolidate our unified editing platform so that
it's great on all devices
***Editing depends on Parsing
Outcome 2: Objective 1: Revision storage scaling
- Services depends on SRE, Parsing
==== Collaboration ====
* Blocked by:
* Blocking:
* Updates:
** Hoping to dark-deploy maps localization this week; had some false starts
and reverted deploys
==== Language ====
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**Goal 3.1 Improve and consolidate our unified editing platform so that
it's great on all devices
***Language depends on Editing
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
**
==== UI Standardization ====
* Blocked by:
* Blocking:
* Updates:
** OOUI – v0.26.3 released:
https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v…
*** Apex theme now uses WikimediaUI icon, this was long overdue in regards
of maintainability cost and that there were under 10% Apex specific icons
left
*** 4 new icons for 'tableMoveColumn*' & 'tableMoveRow*'
*** CheckboxInput-/MenuOptionWidget/MenuToolGroup: Don't specify icon in
CSS, reduces CSS output (Bartosz Dziewoński)
** Work on size unification aftermath in different products (Notifications,
Flow, RevisionSlider)
** Style Guide technical and contents improvements ongoing with 5 open PRs
currently
*** PR with big impact: Using system fonts as sans-serif choice –
https://phabricator.wikimedia.org/T188799
** Transparency Report: Wrapping up accessiblity measurements (continuation
from last week)
**
== Technology ==
=== Analytics ===
* Blocked by: traffic in a bit of issues
* Blocking:
* Updates:
**
** Major change on hadoop cluster to spark2, we try to avoid non backwards
compatible changes
** but this actually non backwards compatible so it will be a
stop-the-world change for some hours TODAY:
https://phabricator.wikimedia.org/T159962
** Mystery of the week around geo-location of pageviews for opera mini,
almost there, talking to opera team this week:
https://phabricator.wikimedia.org/T187014
** Continue working on measuring popups on all sites (now feature is
enabled everywhere minus dewiki and enwiki).
** Completed first release of geowiki data (editors per country), available
now in superset internally:
https://superset.wikimedia.org/superset/dashboard/9/
** Issues with mirrormarker and newer versions of kafka producers not
solved, job topics blocked from jumbo cluster
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
** OpenStack upgrades on Friday -- Horizon and CI will be disabled for a
while beginning around 14:00 UTC
** Short blog post published on Q3 FY17/18 highlights for WMCS <
https://phabricator.wikimedia.org/phame/post/view/91/cloud_services_team_q3…
>
** Andrew fixed a privledge escalation security issue with Horizon's puppet
managment screens (T191433)
** Brooke is working on Wiki Replica view changes and needs some
instruction from the DBA team on how to depool servers for maintenance
there.
** Chase is working out Neutron gateway HA configuration in labtestn
** Madhu & Ariel migrated the last user-facing Dumps endpoints to the new
Cloud Services managed servers
=== Fundraising Tech ===
* CentralNotice: deploy CSP headers and violation alert on forced banner
previews
** Need to determine exact header to add
** Curious about the state of the larger CSP work
* Custom data field changes for our CiviCRM instance
* Custom reporting for CiviCRM to help find fraudsters faster
* More API update work for main CC processor, lately focused on the
recurring donation bits
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
** MW 1.31 release planning
*** Pencils down on April 16
*** considering additional bundled extensions (T178349 and subtasks)
*** Making Remex default tidy implementation w/ James_F (T185753)
*** Running phan for extensions that are proposed to be bundled -
https://gerrit.wikimedia.org/r/#/q/status:open+topic:phan
*** Run tests with PHPUnit 6 -
https://gerrit.wikimedia.org/r/#/q/topic:phpunit-6
*** Handle extension dependencies in the installer (for new bundled
extensions, still WIP) - https://gerrit.wikimedia.org/r/#/c/424967/
*** Also planning to do license reviews for new bundled extensions - e.g.
https://gerrit.wikimedia.org/r/#/c/424726/
*** Implement PlatformSettings.php RfC -
https://gerrit.wikimedia.org/r/#/c/423577/
** MCR:
*** ar_rev_id population patch merged!
*** maintenance/populateArchiveRevId.php run on all wikis!
*** Preparation schema change for ar_text dropping is still in progress
*** Working on MCR schema migration script
*** API updates for query modules are in Gerrit:
https://gerrit.wikimedia.org/r/#/c/413223/
*** Started working on the API "templated parameters" described in
https://phabricator.wikimedia.org/T174032#3990488. Not in Gerrit yet.
*** Code review
** Actor table:
*** Further deployment is waiting on the schema change (T188299)
*** 17 extension updates still need review:
https://gerrit.wikimedia.org/r/#/q/status:open+topic:bug/T167246-actor-migr…
** Comment table:
*** Follow up schema change is still in progress
https://gerrit.wikimedia.org/r/#/c/417039/
*** Toolforge view updating patch is merged and pending updates
*** The image_comment_temp merging patches are in Gerrit
*** All except one are blocked on the schema change
** Improving API test coverage:
*** 8 changes outstanding: improved test coverage for ApiMain,
ApiStructureTest, ApiBase, ApiParse, ApiMove, and ApiUserrights; two
improvements to ApiTestCase (getting rid of manual session mangling and
doLogin); and API usage errors in one more extension.
*** This is the last week for Aryeh
** Code review and bug fixes
** Quarterly goal dependency update:
**Goal 3.6 Support work towards unifying MediaWiki's parser
implementations, in liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
**** MWPT working on relevant planning for Platform Evolution CDP to
prepare for next FY and will coordinate with Parsing as this progresses
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**** MWPT meets weekly with SDC and WMDE
** Objective 1.1 Assist with deploying MultiContent Revisions on Commons
T174022 Implement multi-content revisions, T174023 Implement MCR storage
layer, T174045 DB schema migration for MCR, T174044 Deploy MCR storage
layer,T174043 Deploy Multi-Content Revisions
***SDC depends on MediaWiki Platform, WMDE
**** MWPT meets weekly with SDC and WMDE
=== Performance ===
* Blocked by: None
* Blocking: None
* Updates:
** Aaron/Timo are helping with train delay
**Moving our apps out of puppet, so that we don't need to bug SRE in order
to deploy changes
** Implemented WebP lossless compression support in thumbor on Vagrant for
testing, and it looks good. Gilles will be reachingout to folks on SRE and
Traffic to discuss some of the potential impacts of implementing in prod.
** bunches of code review
=== Release Engineering ===
* Blocked by:
* Blocking:
** scap and git-lfs work is blocking ores deployment. Mukunda is working on
it. Should be unblocked by the end of this week.
* Updates:
** Train is delayed, see task for updates:
https://phabricator.wikimedia.org/T183968
**Quarterly goal dependency update:
***Outcome 1, Objective 4: Continue improving the ways that users can
download articles of interest for later consumption
**** Reading Web depends on SRE, RelEng, Reading Infra
=== Research ===
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**Segment 4, Outcome 2: Develop a better understanding of existing needs
for Structured Commons- T171252
***Research depends on Multimedia
=== Scoring Platform ===
* Blocked by:
** Still blocked by git-lfs. This is urgent for us, Adam is happy to work
on it synchronously if this might be helpful for RelEng?
* Blocking:
* Updates:
=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
**Need help with WDQS GUI build automation:
https://phabricator.wikimedia.org/T189811
**Evaluating feature generation by `query_explorer` functionality of LTR
plugin: https://phabricator.wikimedia.org/T187148
**Working on some job queue issues:
https://phabricator.wikimedia.org/T191024
**Working on query parsing refactoring:
https://phabricator.wikimedia.org/T185108
**Working on Lexeme completion search:
https://phabricator.wikimedia.org/T190970
**Some new docs about running non-wikidata WDQS setups:
https://www.mediawiki.org/wiki/Wikidata_query_service/Implementation/Standa…
*Quarterly goal dependency update:
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**Objective 2.1 Integrate structured file captions into search
*** SDC depends on Search Platform, Multimedia
=== Security ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**Goal 3.6: Support work towards unifying MediaWiki's parser
implementations, in liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
=== Services ===
* Blocked by: Search on CirrusSearchElasticaWrite job serialization T191024
* Blocking: none
* Updates:
** Switched a big bulk of jobs (9) for test wikis to the kafka queue
** restbase-dev cluster work continues, so ignore any alert for it
*Quarterly goal dependency update:
**Increase code sharing of client apps by coalescing and moving more logic
to the server
***Reading Infra/Parsing, Services
**** working with Reading Infra on new CSS endpoints rollout
**Goal 3.6 Support work towards unifying MediaWiki's parser
implementations, in liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
Outcome 2: Objective 1: Revision storage scaling
***Services depends on SRE, Parsing
-
=== Site Reliability Engineering ===
* Blocked by:
** None
* Blocking:
** None
* Updates:
** ICU migration current ongoing https://phabricator.wikimedia.org/T189295
** mcrouter 0.37 released and packaged already for Debian stretch. Jessie
is a bit more difficult, WIP
** Various countries routed to eqsin (Singapore PoP) once more (Japan,
India, Pakistan, Australia is being experimented with still)
** image scalers no longer received thumb.php traffic
** ping offloading to a dedicated host is happening
https://phabricator.wikimedia.org/T190090. Keep in mind that pings to LVS
IPs from now on will not be replied to by the hosts they used to be replied
from
*Quarterly goal dependency update:
**Outcome 1, Objective 4: Continue improving the ways that users can
download articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
*** services/SRE will schedule a meeting to discuss this
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
Outcome 2: Objective 1: Revision storage scaling
- Services depends on SRE, Parsing
== Wikidata ==
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
** Objective 1.1 Assist with deploying MultiContent Revisions on Commons
T174022 Implement multi-content revisions, T174023 Implement MCR storage
layer, T174045 DB schema migration for MCR, T174044 Deploy MCR storage
layer,T174043 Deploy Multi-Content Revisions
***SDC depends on MediaWiki Platform, WMDE
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
** Raising FileImporter test coverage, currently at 48%:
https://phabricator.wikimedia.org/T190290
** Looking into MediaWiki CodeSniffer performance:
https://gerrit.wikimedia.org/r/#/projects/mediawiki/tools/codesniffer,dashb…
** Documenting FileImporter for performance review:
https://phabricator.wikimedia.org/T191283
** Currently deploying new Wikidiff2 to beta:
https://phabricator.wikimedia.org/T190717
** Blogging about the learnings on Wikidiff2:
https://phabricator.wikimedia.org/T190176
== SoS Meeting Bookkeeping ==
* Updates:
**please add links to Phab tickets in the goals update section
If it's gone, that's coincidence. Flagging this to look into, thanks for
the report. Please follow that ticket,
https://phabricator.wikimedia.org/T184258 for more info.
On Tue, Apr 10, 2018 at 5:35 PM, Derk-Jan Hartman <
d.j.hartman+wmf_ml(a)gmail.com> wrote:
> It seems that the pagecounts-ez sets disappeared from
> dumps.wikimedia.org starting this date. Is that a coincidence ?
> Is it https://phabricator.wikimedia.org/T189283 perhaps ?
>
> DJ
>
> On Thu, Mar 29, 2018 at 2:42 PM, Ariel Glenn WMF <ariel(a)wikimedia.org>
> wrote:
> > Here it comes:
> >
> > For the April 1st run and all following runs, the Wikidata dumps of
> > pages-meta-current.bz2 will be produced only as separate downloadable
> > files, no recombined single file will be produced.
> >
> > No other dump jobs will be impacted.
> >
> > A reminder that each of the single downloadable pieces has the siteinfo
> > header and the mediawiki footer so they may all be processed separately
> by
> > whatever tools you use to grab data out of the combined file. If your
> > workflow supports it, they may even be processed in parallel.
> >
> > I am still looking into what the best approach is for the pags-articles
> > dumps.
> >
> > Please forward wherever you deem appropriate. For further updates, don't
> > forget to check the Phab ticket! https://phabricator.wikimedia.
> org/T179059
> >
> > On Mon, Mar 19, 2018 at 2:00 PM, Ariel Glenn WMF <ariel(a)wikimedia.org>
> > wrote:
> >
> >> A reprieve! Code's not ready and I need to do some timing tests, so the
> >> March 20th run will do the standard recombining.
> >>
> >> For updates, don't forget to check the Phab ticket!
> >> https://phabricator.wikimedia.org/T179059
> >>
> >> On Mon, Mar 5, 2018 at 1:10 PM, Ariel Glenn WMF <ariel(a)wikimedia.org>
> >> wrote:
> >>
> >>> Please forward wherever you think appropriate.
> >>>
> >>> For some time we have provided multiple numbered pages-articles bz2
> file
> >>> for large wikis, as well as a single file with all of the contents
> combined
> >>> into one. This is consuming enough time for Wikidata that it is no
> longer
> >>> sustainable. For wikis where the sizes of these files to recombine is
> "too
> >>> large", we will skip this recombine step. This means that downloader
> >>> scripts relying on this file will need to check its existence, and if
> it's
> >>> not there, fall back to downloading the multiple numbered files.
> >>>
> >>> I expect to get this done and deployed by the March 20th dumps run.
> You
> >>> can follow along here: https://phabricator.wikimedia.org/T179059
> >>>
> >>> Thanks!
> >>>
> >>> Ariel
> >>>
> >>
> >>
> > _______________________________________________
> > Wikitech-l mailing list
> > Wikitech-l(a)lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>