Wikitech-l October 2018

wikitech-l@lists.wikimedia.org

71 participants
69 discussions

Readers Monthly update for September 2018
by Chris Koerner 11 Oct '18

11 Oct '18

Hello, This is the monthly update from the Readers team at the Wikimedia Foundation for September 2018. Feedback and questions are encouraged. ==Discussions== === Apps=== * Work is underway to bring Wikidata description editing to the iOS app. [0] === Web === * The Advanced mobile contributions project saw an updated navigation prototype go out to folks. We contacted ~30 people we met at Wikimania directly, shared in Tech News, posted to 15 language specific wikis where mobile usage was the highest, to the wikimedia-l, wikitech-l, and mobile-l mailing lists, and to the Wikipedia Weekly Facebook group. [1] [2] [3] === Reading Infrastructure === * Made small corrections to parentheses stripping in /page/summary endpoint: [4] [5] * Map cluster upgrade to Debian Stretch is in progress: [6] === New Readers === * New Reader updates can be found at Meta. [7] [0] https://phabricator.wikimedia.org/T193691 [1] https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions [2] https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions/Na… [3] https://phabricator.wikimedia.org/T197715 [4] https://phabricator.wikimedia.org/T198264 [5] https://phabricator.wikimedia.org/T202119 [6] https://phabricator.wikimedia.org/T198622 [7] https://meta.wikimedia.org/wiki/New_Readers/Updates --- Subscribe to receive theses updates as on-wiki notifications or opt-in email. https://www.mediawiki.org/wiki/Newsletter:Readers_Monthly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Reading/Status_updates Yours, Chris Koerner Community Relations Specialist Wikimedia Foundation

1 0

2018-09-26 Scrum of Scrums meeting notes
by Grace Gellerman 11 Oct '18

11 Oct '18

*https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-09-26 <https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-09-26>* *= 2018-09-26 =* == Callouts == Release Engineering: * *Deployments of MediaWiki with scap cause a spam of "web request took longer than 60 seconds and timed out"* ** https://phabricator.wikimedia.org/T204871 ** Scap is checking canary servers in dormant instead of active-dc* ** Switchover process needs an update. Long term the list of canaries should be in conftool. ** https://phabricator.wikimedia.org/T204907 * ComTech: ** We're starting to work on showing SVGs in page language - https://phabricator.wikimedia.org/T205040 == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: **Planning to be feature complete for 6.1 ( https://phabricator.wikimedia.org/tag/ios-app-v6.1-narwhal-on-a-bumper-car/) by the end of this week ** Apps offsite next week, ==== Android native app ==== * Blocked by: * Blocking: * Updates: **Working on productionising new navigation **Apps offsite next week. ==== Readers Web ==== * Blocked by: * Blocking: * Updates: ** Mobile website (MinervaNeue / MobileFrontend): *** SkinMinerva.php file logs "Undefined variable: returntoquery` error T205449 *** Invest in the MobileFrontend & MinervaNeue frontend architecture https://www.mediawiki.org/wiki/Reading/Web/Projects/Invest_in_the_MobileFro… **** Continue Webpack and test transition of mobile.startup T203817 **** Add more tests to mobile.startup files T203818 **** Simplify CategoryOverlay T191987 **** Add Sinon.JS configuration T204885 **** Update test file naming T197884 **** Bundle Hogan.js T205128 T205129 **** Enable client side error reporting T167699 *** Page issues https://www.mediawiki.org/wiki/Reading/Web/Projects/Mobile_Page_Issues **** Limited A/B test in progress T204609 **** ReadingDepth logging sometimes initializes before PageIssues T204144 **** Add new treatment opt-in query parameter override T204746 **** Don't cache config flag with HTML T205355 **** What % of pages feature issues? T201123 **** Increase sampling ratio for ReadingDepth T205176 *** Working with editing *** Add share icon to beta T181195 *** Lighten default theme color T204691 *** Maintenance and bug fixes T199066 T204584 T197105 T198018 T205321 *** Advanced mobile contributions https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions **** Generate list of most used special pages T198218 **** Design and product continuing special pages work in Minerva *** Data working on "better use of data" requirements ** PDF rendering (Proton) https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality *** Remaining work tracked in T186748 *** Working with Services on a Grafana dashboard T201158 T204055 *** Chromium-render doesn't handle browser connection abort well T181623 *** Miscellaneous maintenance and bug fixes ** Supporting Multimedia/Test hiring processes ==== Readers Infrastructure ==== * Blocked by: * Blocking: * Updates: ** PCS: Continue work on mobile-html endpoint. ** Maps *** maps1004 migrated to stretch successfully **** It will now receive OSM data load to start generating tiles T205462 *** Investigation of tilerator crash in eqiad has an workaround but still need permanent fix T204047 ==== Parsing ==== * Blocked by: None * Blocking: None * Updates: All code review requests from last week have been handled, thanks ** Will pass along: (Parsing) T205497 [Regression pre-wmf.23] REST API on Beta cluster returns content of different pages than requested (breaks VE) ==== Multimedia ==== * Blocked by: * Blocking: * Updates ** === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: ** We're starting to work on showing SVGs in page language - https://phabricator.wikimedia.org/T205040 *Ping Parsing and SRE* ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: ** ==== Editing ==== * Blocked by: * Blocking: ** Updates: ** Removed dedicated annotation operation types in the VE data model and instead model annotation transactions using replacements ** Improved mobile dialogs in the visual editor ** Completed move of MobileFrontend's VisualEditor styles back to MobileFrontend from Minerva ==== Growth ==== * Blocked by: * Blocking: * Updates: ** ==== Language ==== * Blocked by: * Blocking: * Updates: ** Preparing for ContentTranslation v2 continue. === Audiences Design === * Blocked by: * Blocking: * Updates: ** ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** Special:Preferences OOUI rollout ** Resolving remaining typography/imagery quests on Design Style Guide ** Accessibility work on wikimediafoundation.org website == Technology == === Analytics === * Blocked by: * Blocking: * Updates: - Analytics team off-site - Replacement of hadoop cluster node managers without issues - 2 new metrics for Wikistats2: pages to date (total article count) and top contributors, deploy before end of quarter - Final work on Wikistats2 graph annotations (tricky UI), also deploy before end of quarter - Modern Event Data Platform internal discussions, tickets and naming reworked for a clearer picture of what we will be working on === Cloud Services === * Blocked by: * Blocking: * Updates: ** === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Thanks to the people who are helping out with the MessageCache issue affecting CentralNotice (Ian Marlier, Jaime Crespo, Daniel Kinzler, Niklas Laxström et al) ** Got a patch out to support Amazon Alexa donations, just in time for their announcement ** Working on distinguishing endowment gifts throughout the whole donation pipeline ** Fixing a couple of issues in spreadsheet imports ** Interviewing for engineer position === Core Platform === * Blocked by: * Blocking: * Updates: ** Session management service design and implementation is being moved up to this quarter to get to unblock multi-DC *** Preparing for moving to production early Q3 ** MediaWIki tarball release script updated ** Work starting on Excimer - New profiler for PHP to replace XHProf https://phabricator.wikimedia.org/T205059 === Performance === * Blocked by: * Blocking: * Updates: ** === Release Engineering === * Blocked by: ** 1.32.0-wmf.23 deployment blockers *** (Readers Web) T205449 SkinMinerva.php file logs "Undefined variable: returntoquery` error **** Done *** (Core Platform) T205469 Fatal error from LinkRenderer on special pages ("Object of class HtmlArmor could not be converted") **** Done *** (Search Platform) T205473 Fatal error "Invalid operand type" from CirrusSearch LinksUpdate **** Done *** (Parsing) T205497 [Regression pre-wmf.23] REST API on Beta cluster returns content of different pages than requested (breaks VE) **** Done * Blocking: ** * Updates: ** Train Health: *** Scap is checking canary servers in dormant instead of active-dc - Switchover process needs an update. Long term the list of canaries should be in conftool - https://phabricator.wikimedia.org/T204907 ** Log Health: *** We are seing a lot of T204871 "web request took longer than 60 seconds and timed out" in `fatalmonitor` ** Code Health: *** Creating communication channels (Phabricator https://phabricator.wikimedia.org/tag/code-health-metrics/, IRC channel, mailing list) === Research === * Blocked by: None * Blocking: None * Updates: ** Data collection for CitationUsagePageLoad at 33.3% (was 10% previously) ** Retaining de-identified CitationUsage data beyond 90 days ** Recommendation API service to MySQL connection: https://phabricator.wikimedia.org/T205452 === Scoring Platform === * Blocked by: * Blocking: * Updates: ** PoolCounter is deployed everywhere, now the limit of parallel connections is being enforced ** Started to work on improving logs in ores, specially sending them to logstash ** Building javascript library in mediawiki to make requesting scores to ores easier: https://phabricator.wikimedia.org/T201691 === Search Platform === * Blocked by: * Blocking: * Updates: **Made sanetizer process to constantly reindex the wikis: https://phabricator.wikimedia.org/T203622 **Implemented support for Lexemes in RDF (not enabled yet, waiting for dumps): https://phabricator.wikimedia.org/T202459 **Finished evaluation of image quality as query-independent ranking signal: https://phabricator.wikimedia.org/T202339 ***Conclusion: positive, image quality is a good signal, will plan implementation ** Upgraded ElasticSearch plugins for 6.3: https://phabricator.wikimedia.org/T197960 ** Added ntriples RDF dump for Wikidata: https://phabricator.wikimedia.org/T144103 ** Reindexing Wikidata to enable several newly deployed features ** Reindexed Esperanto wiki after new analysis enabled: https://phabricator.wikimedia.org/T203005 ** Working on calculating examination probabilities for completion queries: https://phabricator.wikimedia.org/T205348 ** Working on improving Korean analyzers: https://phabricator.wikimedia.org/T178925 ** Working on running multiple Elastic instances on the same hardware: https://phabricator.wikimedia.org/T193654 ** Working on searching for statement values without additional keywords for Wikidata: https://phabricator.wikimedia.org/T163642 ** Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108 === Security === * Blocked by: * Blocking: * Updates: ** === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** Preparing for the switchback ** Finishing goals == Wikidata == * Blocked by: * Blocking: * Updates: ** == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** == Multi-Content Revisions == * Blocked by: * Blocking: * Updates: ** "read new" enabled on testwiki ** "read new" being enabled on mediawiki.org this week ** waiting for DC switch back to enable "read new" on Commons and other wikis ** reactive minor bug fixes == SoS Meeting Bookkeeping == * Updates: **

1 0

2018-10-03 Scrum of Scrums meeting notes
by Grace Gellerman 11 Oct '18

11 Oct '18

*https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-10-03 <https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-10-03>* *=2018-10-03=* == Callouts == * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar * Actively working to eliminate ongoing messages from the mediawiki-errors logging channel. Your team may be tagged on those, please take a look if/when you are. * SRE: Switchover/Switchback dates reminder.* It's next week!* Switchback: Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC * Release Engineering ** Next week: No train next week, DC switchover * Reading Infrastructure: We would like to ask the Security Team how is the security review of "JsonConfig and Kartographer interaction" (T163827) going and if they need any help with that (It's not blocking anything, but we would like to know the current status) * Research ** [Services]: Setting up a connection from the recommendation API service to MySQL: https://phabricator.wikimedia.org/T205452 * UI Standardization ** OOUI v0.29.0 released yesterday with 8 possible breaking changes, among those PHP 7 requirement and removal of CapsuleMultiselectWidget *** Please consider if these changes might affect your code https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v… * 5 Minute Demo - taking your sign-ups now! Next meeting on Oct 11 ** https://www.mediawiki.org/wiki/5_Minute_Demo#Recordings_and_Presenters * Parsoid Services (Core Platform) and Parsing team deployed code this week that implements HTML version (content) negotiation protocol alongwith a major version bump of Parsoid HTML from 1.8.0 to 2.0.0 ( https://www.mediawiki.org/wiki/Specs/HTML/2.0.0 ). As part of this, RESTBase + Parsoid will enforce the HTML version requested in the Accept header. If a version is requested that is different (^ semantics) than what is available in storage, RESTBase will ask Parsoid to get the appropriate version. If Parsoid cannot do that, it will return a HTTP 406. What does it mean for Parsoid clients? If you are requesting a really old version in your Accept header, you are very likely going to get a HTTP 406. If you request an older version (1.7 or 1.8), you will see an additional latency to your requests while Parsoid downgrades the HTML from the more up to date version. So, Parsoid clients should provide an accept header with the most up-to-date version of Parsoid HTML you can handle. Subsequently, you should update your code to handle the newer versions in a timely manner. A longer announcement and wiki docs will be coming out soon. == Audiences == === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: ** ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: ** ==== Editing ==== * Blocked by: * Blocking: ** Updates: ** ==== Growth ==== * Blocked by: * Blocking: * Updates: ** ==== Language ==== * Blocked by: None * Blocking: None * Updates: ** Content Translation v2 released! https://wikimediafoundation.org/2018/09/30/content-translation-version-two/ === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** ==== Android native app ==== * Blocked by: * Blocking: * Updates: ** ==== Readers Web ==== * Blocked by: * Blocking: * Updates: ** Mobile website (MinervaNeue / MobileFrontend): *** Invest in the MobileFrontend & MinervaNeue frontend architecture https://www.mediawiki.org/wiki/Reading/Web/Projects/Invest_in_the_MobileFro… **** Add unit tests for util and mfExtend T203818 **** Bundle widgets and models with Webpack; enable headless Page and View tests T204867 T204786 **** Make test set up explicit T204885 **** Lint unsupported JavaScript in build products T203976 **** Move MainMenu click tracking schema from MobileFrontend to Minerva T204584 **** Merge skin option modules into a single ResourceLoader module and move enabled logic to client T167713 *** Page issues https://www.mediawiki.org/wiki/Reading/Web/Projects/Mobile_Page_Issues **** A/B test in progress T204609 **** Lots of cross-wiki QA and A/B test discussions *** Maintenance and bug fixes T205449 T202756 *** Advanced mobile contributions https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions **** Design and product continuing special pages work in Minerva ** PDF rendering (Proton) https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality *** Remaining work tracked in T186748 *** Html2pdf route should return promise T201158 T204055 T181623 *** Consistently represent asynchronous code execution T204055 ** SEO: where should Schema property 'sameAs' live? T204070 *** The Performance team is going to be deploying a second set of sitemaps next week, they will also be testing the indexing on mobile in parallel. Mikhail has analyzed the data from the initial Italian sitemap launch and we haven't seen any conclusive evidence that they've helped referrals or increased pageviews. We will repeat the analysis with the other wikis. ** Management supporting Multimedia hiring processes ** Product working on 3-5 year planning ** Design exploring reference previews https://reference-previews.firebaseapp.com/ ==== Readers Infrastructure ==== * Blocked by: ** none * Blocking: ** none known * Updates: ** Maps: *** Loading OSM map into maps1004 T205462 *** Tilerator overloading proxy fixed https://wikitech.wikimedia.org/wiki/Incident_documentation/20181002-maps ** MCS: *** Now loading Parsoid HTML v2.0.0 ==== Multimedia ==== * Updates ** ==== Parsing ==== * Blocked by: * Blocking: * Updates: Services (Core Platform) and Parsing team deployed code this week that implements HTML version (content) negotiation protocol alongwith a major version bump of Parsoid HTML from 1.8.0 to 2.0.0 ( https://www.mediawiki.org/wiki/Specs/HTML/2.0.0 ). As part of this, RESTBase + Parsoid will enforce the HTML version requested in the Accept header. If a version is requested that is different (^ semantics) than what is available in storage, RESTBase will ask Parsoid to get the appropriate version. If Parsoid cannot do that, it will return a HTTP 406. What does it mean for Parsoid clients? If you are requesting a really old version in your Accept header, you are very likely going to get a HTTP 406. If you request an older version (1.7 or 1.8), you will see an additional latency to your requests while Parsoid downgrades the HTML from the more up to date version. So, Parsoid clients should provide an accept header with the most up-to-date version of Parsoid HTML you can handle. Subsequently, you should update your code to handle the newer versions in a timely manner. A longer announcement and wiki docs will be coming out soon. ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** OOUI v0.29.0 released yesterday with 8 possible breaking changes *** Please consider if these changes might affect your code https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v… **** PHP 7 is now required **** CapsuleMultiselectWidget is removed entirely (resulting in 8% decrease of widgets CSS after gzipping https://phabricator.wikimedia.org/T183299#4637565) ** Continuous work on wikimediafoundation.org website accessibility support == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** === Cloud Services === * Blocked by: * Blocking: * Updates: ** === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Big English mass emails have begun! Watching for errors and trying to keep donation infrastructure solid ** Debugging some CI issues with CiviCRM ** Following along with core and/or language teams' work on CentralNotice save slowness ** Support for Endowment Gifts via donations pipeline ** Fixing audit file downloads === MediaWiki Core Platform === * Blocked by: * Blocking: * Updates: ** Deployed content negotiation for Parsoid ( https://phabricator.wikimedia.org/T128040 ) *** result of a 2 1/2 year collaboration with Parsing *** allows clients to request a specific version of the content-type (how the content would look if produced by a specific version of the software) ** PHP7 profiler development work ** Tech Conf session design === Performance === * Blocked by: ** * Blocking: ** * Updates: ** Identified performance issue with saving banners due to RevisionStore, working with several teams on a fix ** Added some additional data points to NavTiming collection ** Chrome 69 shows a regression in firstPaint, reported upstream ** Investigating a memory leak in our front-end code: https://phabricator.wikimedia.org/T205127 ** Investigating memcache TKOs: https://phabricator.wikimedia.org/T203786 ** Actively working to eliminate ongoing messages from the mediawiki-errors logging channel. Your team may be tagged on those, please take a look if/when you are. === Release Engineering === * Blocked by: ** SRE/Services review of: [ https://gerrit.wikimedia.org/r/#/c/operations/software/service-checker/+/46… Support a literal body for POST requests in `fetch_url`] * Blocking: ** * Updates: ** Production Excellence: September 2018 https://phabricator.wikimedia.org/phame/post/view/119 ** Train Health: *** Last week: There were a few blockers, but they were resolved quickly. No delays in train. *** This week: no delays *** Next week: No train next week, DC switchover ** Log Health: *** ** Code Health: *** === Research === * Blocked by: ** [Services] Setting up a connection from the recommendation API service to MySQL: https://phabricator.wikimedia.org/T205452 * Blocking: None * Updates: ** Retain CitationUsage and CitationUsagePageLoad data after 90 days: https://phabricator.wikimedia.org/T205272 ** Resolve redirects for the Recomendation API: https://phabricator.wikimedia.org/T203041 === Scoring Platform === * Blocked by: None * Blocking: None * Updates: ** Working on improving logging and send more logs to logstash === Search Platform === * Blocked by: * Blocking: * Updates: ** Finished Wikidata reindex, now search also uses statement values ** Fixed bug where search was gluing together words separated by HTML tags: https://phabricator.wikimedia.org/T195389 ** Enabled Kafka poller on WDQS: https://phabricator.wikimedia.org/T189458 ** Fixed timeouts on RecentChanges API caused by bad indexes: https://phabricator.wikimedia.org/T202764 ** Working on calculating examination probabilities for completion queries: https://phabricator.wikimedia.org/T205348 ** Working on improving data and creating models from autocompletion click data: https://phabricator.wikimedia.org/T205111 ** Working on improving Korean analyzers: https://phabricator.wikimedia.org/T178925 ** Working on running multiple Elastic instances on the same hardware: https://phabricator.wikimedia.org/T193654 ** Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108 === Security === * Blocked by: * Blocking: * Updates: **https://phabricator.wikimedia.org/T200279 being worked on for this month === Services === * Blocked by: * Blocking: * Updates: ** === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** Switchback to eqiad happening next week, no train, only emergency deploys == Wikidata == * Blocked by: * Blocking: * Updates: ** == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** == Multi-Content Revisions == * Blocked by: * Blocking: * Updates: ** "read new" enabled on mediawiki.org on Monday ** "read new" will be enabled on commons on October 15 ** "read new" may be enabled on several small/medium wikis in the interim ** addressing gaps and regressions and beginning to address tech debt == SoS Meeting Bookkeeping == * Updates: **

1 0

2018-10-10 Scrum of Scrums meeting notes
by Grace Gellerman 11 Oct '18

11 Oct '18

https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-10-10 *=2018-10-10=* == Callouts == * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar * * SRE: ** Switchover week, no train. Up to now traffic and mediawiki switchovers have succeeded * Release Engineering ** Blocked by: need review from SRE/services [ https://gerrit.wikimedia.org/r/#/c/461457/ Support a literal body for POST requests in `fetch_url`] ** Train Health: no train due to DC switchover - T191071 1.32.0-wmf.25 deployment blockers ** Log Health: T204871 Deployments of MediaWiki with scap cause a spam of "web request took longer than 60 seconds and timed out" * Research ** *Services* please help with "Setting up a connection from the recommendation API service to MySQL": https://phabricator.wikimedia.org/T205452 * Parsoid == Audiences == === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: ** ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: ** ==== Editing ==== * Blocked by: Partially blocked by Performance on T176262. We want to preload code in the visual editor to improve loading times (T202720) and we need to understand the priority and timeline for T176262 to plan accordingly. * Blocking: ** Updates: ** ==== Growth ==== * Blocked by: * Blocking: * Updates: **We've released PageTriage/ORES integration. **Will release PageTriage/Copyvio integration in about a week. ***We are working on interventions for Understanding first day ( https://phabricator.wikimedia.org/T205754) and ==== Language ==== * Blocked by: None * Blocking: None * Updates: ** Work on polishing ContentTranslation v2 continue. Received positive feedback so far. === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: **Working on closing out 6.1 for beta ( https://phabricator.wikimedia.org/tag/ios-app-v6.1-narwhal-on-a-bumper-car/), changes to cookie handling in progress ==== Android native app ==== * Blocked by: * Blocking: * Updates: ** Offsite last week; focusing on releasing navigation updates. ==== Readers Web ==== * Blocked by: * Blocking: * Updates: ** Mobile website (MinervaNeue / MobileFrontend): *** Invest in the MobileFrontend & MinervaNeue frontend architecture https://www.mediawiki.org/wiki/Reading/Web/Projects/Invest_in_the_MobileFro… **** Transition Gruntfile tasks to NPM scripts T206069 **** Follow-ups from last week *** Page issues https://www.mediawiki.org/wiki/Reading/Web/Projects/Mobile_Page_Issues **** A/B test rollout increased on Farsi, Japanese, Russian, and English wikis T200792 *** Enable Dynamic Type in iOS 9+ browsers T204807 *** Maintenance and bug fixes T206274 T206174 T204835 T206337 T205582 *** Advanced mobile contributions https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions **** Design continuing special pages work in Minerva ** PDF rendering (Proton) https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality *** Remaining work tracked in T186748 *** Testing concurrency *** chromium-render naming is being updated to proton ** SEO: add Schema sameAs property pointing to Wikidata entries T198946 ** Management supporting Multimedia hiring processes ** Design continuing work on reference previews https://reference-previews.firebaseapp.com/. ** Product continuing 3-5 year planning. ==== Readers Infrastructure ==== * Blocked by: ** none * Blocking: ** none known * Updates: ** RI offsite next week, very limited availability ** Maps: *** Load OSM data into maps1004 continues this week, we expected to start generating tiles next week ==== Multimedia ==== * Updates ** Working on hiring, interviewing candidate this week ** All of the SDoC multi-lingual file captions work is merged - some small security fixes and we can deploy to beta (crosses fingers) ** lots of discussion/design on prototypes for adding 'depicts' to files containing artworks ==== Parsing ==== * Blocked by: * Blocking: * Updates: ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** OOUI v0.29.2 released with a hotfix on consistent document listeners from breaking change in v0.29.0 https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v… ** Design Style Guide “Resources” section inclusion – preparing patches ** wikimediafoundation.org priority accessibility issues successfully resolved https://phabricator.wikimedia.org/T201682 == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** Putting finishing touches on new metrics on wikistats, see for example “top editors” in italian wikipedia. We just added ability to split by bots/anonymous: https://stats.wikimedia.org/v2/#/it.wikipedia.org/contributing/top-editors (as always all metrics are available programmatically via APIs) ** Working with security in prioritizing items to work on as part of security’s review of analytics’ stack. ** Ingesting data from eventlogging for easy visualization into Druid, see, for example, NavigationTiming data collected by performance team, see DOMComplete time buckets: https://bit.ly/2C3zyAc ** Working on ingesting xml dumps to enable easier content processing on cluster, this would give us the ability , for example, to identify content that needs citations === Cloud Services === * Blocked by: * Blocking: * Updates: ** === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Little fixes and enhancements to main CC processor's new API (which is now getting almost all CC traffic) ** Scripts to compare data between old and new impression data pipelines ** Following progress on MessageCache work ** Still interviewing candidates for job === MediaWiki Core Platform === * Blocked by: * Blocking: * Updates: ** === Performance === * Blocked by: ** * Blocking: ** * Updates: ** Attended welovespeed conference ** Published https://phabricator.wikimedia.org/phame/post/view/122/best_friends_forever/ ** Tons of CRs for MW RDBMS and cache layer ( ** Helping with preparation of TechConf agenda ** Making progress on Fresnel (performance testing as part of the build pipeline) ** Security incident === Release Engineering === * Blocked by: ** need review from SRE/services [https://gerrit.wikimedia.org/r/#/c/461457/ Support a literal body for POST requests in `fetch_url`] * Blocking: ** * Updates: ** Hired Lars Wirzenius, senior engineer ** Interviewing on-going for our Developer Productivity position: https://boards.greenhouse.io/wikimedia/jobs/1225258?gh_src=f15731e11 ** Train Health: *** Last week: a few blockers, resolved in time, no problems - T191070 1.32.0-wmf.24 deployment blockers *** This week: No train this week due to DC switchover - T191071 1.32.0-wmf.25 deployment blockers *** Next week: the last 1.32 release, 1.33 starts the next week - T191072 1.32.0-wmf.26 deployment blockers ** Log Health: *** T204871 Deployments of MediaWiki with scap cause a spam of "web request took longer than 60 seconds and timed out" ** Code Health: *** === Research === * Blocked by: ** [Services]: Setting up a connection from the recommendation API service to MySQL: https://phabricator.wikimedia.org/T205452 * Blocking: None * Updates: ** Continued working on finding missing articles for the recommendation API: https://phabricator.wikimedia.org/T203041 ** Starting to collect data for the references event stream: https://phabricator.wikimedia.org/T199189 === Scoring Platform === * Blocked by: None * Blocking: None * Updates: ** JADE tables is waiting for TechCom review now ** Working to improve our job queue robustness has started, part of upgrading to celery 4 === Search Platform === * Blocked by: * Blocking: * Updates: ** Dropped support for caching interwiki queries that hadn’t worked properly since 2017: https://phabricator.wikimedia.org/T191961 ** Enable lexeme dumps for Wikidata: https://phabricator.wikimedia.org/T202830 ** Working on calculating examination probabilities for completion queries: https://phabricator.wikimedia.org/T205348 ** Working on improving data and creating models from autocompletion click data: https://phabricator.wikimedia.org/T205111 ** Working on improving Korean analyzers: https://phabricator.wikimedia.org/T178925 ** Working on running multiple Elastic instances on the same hardware: https://phabricator.wikimedia.org/T193654 ** Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108 === Security === * Blocked by: * Blocking: * Updates: https://phabricator.wikimedia.org/T200279 is awaiting remediation ** === Services === * Blocked by: * Blocking: * Updates: ** === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** Datacenter Switchback week. No train. Traffic + Mediawiki have been switched over successfully, tomorrow is swift+services == Wikidata == * Blocked by: * Blocking: * Updates: ** == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** == Multi-Content Revisions == * Blocked by: * Blocking: * Updates: == SoS Meeting Bookkeeping == * Updates: **

1 0

Debian & Ubuntu packages of MediaWiki 1.31 now available
by Kunal Mehta 10 Oct '18

10 Oct '18

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, Debian and Ubuntu packages of MediaWiki 1.31 (LTS) are now available. See <https://www.mediawiki.org/wiki/User:Legoktm/Packages> for more information on how to get them, and basic instructions for installing (hint: it's as simple as `sudo apt install mediawiki`). Using packaged versions of MediaWiki are a good fit for people who just want to have a basic wiki, and not really worry about sysadminning it. Thanks, - -- Legoktm -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEE+h6fmkHn9DUCyl1jUvyOe+23/KIFAlu+V9oACgkQUvyOe+23 /KITnA//b0dWfKHtwItzTbtlmeObpqii9i41XR4ZPJkYoUG80mj1LhQO8vh/4xDd dSiv33lqX3M++IpXS8KQ/ZB3mb0KcuuusKYtnKltqoDD9CJLVggI+fawt69wQzj6 zqYeJoMG8J8vH0CAYoYVbczcn2xm+yliZ7Arb0t4p+PXO4BaAl2EE6awm0X9BsjH 3X1Ij9tzTb9wVNLNq+kgPqgoIyW8WWkY7NkI5f8uuxvH41HYr4ONrXk7Y1eFr+nx 9uzjM6eXpuYZiCcp5pzBTB+Q5OPbGqvy5f2cv3IYlY4o6Tb+YSmC9SMnTVEdzUUf vdbVRRp6lF/tcaxwsGjwNpLdsIyZ6KhEKBxIJCs6/839WsVlYZrWMun26/LdWMGi VZm66ehrjLsexWKJOJUC8bXYiNsSUu0R/qrsROU0871UlpZEJOC6Oh2rre21kijc gOToTYVmZpEK7UMRgjAId9LRRE/hkDZr5ChiIi/cyE3ryZXIPTuHQeHN4zYKVlaN BDkkCZGcUYYh+0pnxRb0H5A4GWCybHPvVlC68nFHQg+aaWEcbAbRMXEJNtvfvP/2 Omq2EAvsnFkA89jbCM71PTX6i+t5vioobXwhThcoErqMOz914ed4AerMCuuXP4/z rRThFclx0Jwa/MWcBTFPdd1N38/eAQQHAXyk/Nvq4sXDzfk83gI= =jsxb -----END PGP SIGNATURE-----

1 0

Translations on hold until further notice
by Greg Grossmeier 10 Oct '18

10 Oct '18

tl;dr: Please refrain from adding new strings that require translations to MediaWiki and deployed extensions until further notice. Due to changes we are making in the way our translation systems process data we will be temporarily disabling the auto deployment of newly translated content until further notice. The goal of these changes will be to improve the evaluation of content submitted to our translation services. This means developers should not merge new translatable strings into MediaWiki Core nor Wikimedia deployed extensions until we re-enable the system. This is to prevent longer term issues where local wiki admins overwrite a string and thus prevent updates from new and improved translations. We apologize for any inconvenience this delay in service may cause and will alert you as soon as this maintenance is completed. Greg Grossmeier -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

13 19

Discovery Weekly Update for the week starting 2018-10-01
by Chris Koerner 09 Oct '18

09 Oct '18

Hello again. It's been a while. This is the weekly update from the Search Platform team for the week starting 2018-09-17 through 2018-10-01. As always, feedback and questions welcome. == Discussions == === Search === * Implemented indexing statement values as part of main data in Wikidata, so that statement values are now searchable without special syntax [0] * Reindexed wikidata which also enables qualifier indexing [1] * Mathew worked on resolving an elasticsearch shard size alert by doing an in place reindex [2] * There was a lot of work done to investigate a brief outage of CirrusSearch (mw exception spike for api.php) [3], but it's resolved enough for now. * Gehel and others worked on refactoring puppet to support multiple elasticsearch instances on same node [4] * Erik worked on an issue where the text content of wiki page in search index can merge words making them unfindable [5] * Stas updated the search engine of Wikidata to enable searching by author name string [6] * David and Erik worked together on evaluating adding an image quality score to media search result ranking [7] * Stas added X-Search-Id to WikidataCompletionSearchClicks events [8] * David added a way to configure timeouts of autocomplete queries [9] * Erik upgraded saneitizer to constantly re-index documents [10] * David investigated why interwiki cache hit/miss was no longer reported (since 2017) and decided to drop the support for caching interwiki queries [11] * Mathew and Gehel worked on raising the alert level on disk space for old elasticsearch servers [12] * Erik worked to correct issues where the Cirrus MLT cache had a 0% hit rate on switchover [13] === WDQS === * Added new NTriples RDF dump (which makes it easier to do per-line processing) [14] * Internal cluster switched to Kafka events as change source, public cluster next [15] == Did you know? == * Different languages can have a different number of sounds they use; the set of sounds used in a particular language is called its “phonemic inventory”. [16] The numbers of sounds can range from 11 to over 140! Having more sounds than letters, or different sounds than the usual sound associated with a letter, can be the source of unusual orthographies and/or transliteration schemes—including "q" formerly being used as a vowel in Natqgu (now Natügu), a language of the Solomon Islands. [0] https://phabricator.wikimedia.org/T163642 [1] https://phabricator.wikimedia.org/T193407 [2] https://phabricator.wikimedia.org/T204362 [3] https://phabricator.wikimedia.org/T204776 [4] https://phabricator.wikimedia.org/T198351 [5] https://phabricator.wikimedia.org/T195389 [6] https://phabricator.wikimedia.org/T179815 [7] https://phabricator.wikimedia.org/T202339 [8] https://phabricator.wikimedia.org/T205597 [9] https://phabricator.wikimedia.org/T204959 [10] https://phabricator.wikimedia.org/T203622 [11] https://phabricator.wikimedia.org/T191961 [12] https://phabricator.wikimedia.org/T204361 [13] https://phabricator.wikimedia.org/T204148 [14] https://phabricator.wikimedia.org/T144103 [15] https://phabricator.wikimedia.org/T189458 [16] https://en.wikipedia.org/wiki/Phonemic_inventory ---- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Relations Specialist Wikimedia Foundation

1 0

OOUI v0.29.2 released
by Volker E. 09 Oct '18

09 Oct '18

Hi everyone, last week we've released OOUI v0.29.0 and identified sequentially two issues on Beta-Cluster which we've addressed in v0.29.1 next day and v0.29.2 released yesterday. Bartosz Dziewoński took the chance and hardened the JS/PHP comparison test suite with former hotfix. Thanks to Pablo Grass of Wikimedia Deutschland and Petar Petkovic for help in identifying latter of those issues and Ed Sanders for instant fix! Key highlights of this release: - OOUI, similar to MediaWiki, now requires PHP 7 With this change we've also switched code to PHP 7 syntax in order to make code reusable between MediaWiki and OOUI (see below) - Deprecated CapsuleMultiselectWidget got dropped, which reduces widgets CSS by 8% percent after gzipping[0] Possible breaking changes: The following changes might cause breaking of your interface, please carefully consider if they affect your code. - Drop CapsuleMultiselectWidget, deprecated since v0.27.5. Use superior TagMultiselectWidget instead. - TextInputWidget: Drop support for `multiline: true`, deprecated in 0.22.2. Use MultilineTextInputWidget directly. - Consistently name document listeners[1] - Formally require PHP 7 -- Use PHP 5.6 variadic function syntax -- Use PHP 7 "\u{NNNN}" Unicode codepoint escapes[2] -- Use PHP 7 '??' operator instead of '?:' with 'isset()'[3] - Upgrade jQuery from 3.2.1 to 3.3.1 You can find details on additional new features & icons, code-level and accessibility changes, styling and interaction design amendments, and all improvements since v0.28.0 in the full changelog[4]. If you have any further queries or need help dealing with breaking changes, please let me know. As always, library documentation is available on mediawiki.org[5], there is comprehensive generated code-level documentation and interactive demos and tutorials hosted on doc.wikimedia.org[6]. OOUI version: 0.29.2 MediaWiki version: 1.32.0-wmf.26 Date of deployment to production: Regular train (due to DC switchover next week), starting Thursday 18 October Best, Volker [0] - https://phabricator.wikimedia.org/T183299#4637565 [1] - https://gerrit.wikimedia.org/r/#/c/oojs/ui/+/437960/ [2] - https://gerrit.wikimedia.org/r/#/c/oojs/ui/+/456500/ [3] - https://gerrit.wikimedia.org/r/#/c/oojs/ui/+/456499/ [4] - https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md [5] - https://www.mediawiki.org/wiki/OOUI [6] - https://doc.wikimedia.org/oojs-ui/master/ -- Senior User Experience Engineer Wikimedia Foundation volker.e(a)wikimedia.org | @Volker_E

1 0

Wednesday 3-4 pm UTC: Technical Advice IRC Meeting
by Birgit Müller 09 Oct '18

09 Oct '18

Hey all, another round of the Technical Advice IRC meeting is taking place on **Wednesday, Oct 10, 3-4 pm UTC** on #wikimedia-tech. This week's meeting will be hosted by @Amir1 and @CFisch_WMDE - Question are welcome in English, Persian & German :-) The Technical Advice IRC Meeting is a weekly support event for volunteer developers. Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting Birgit (for the Technical Advice IRC Meeting crew) -- Birgit Müller Community Communications Manager Software Development and Engineering Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

Datacenter switchover and switchback
by Alexandros Kosiaris 05 Oct '18

05 Oct '18

Hello everyone, This is to inform you that there will be a datacenter switchover and switchback on the next few weeks. The timeline's are Services: Tuesday, September 11th 2018 14:30 UTC Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC Traffic: Tuesday, September 11th 2018 19:00 UTC MediaWiki: Wednesday, September 12th 2018: 14:00 UTC Switchback: Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC For the duration of the switchover (1 month), deployers are kindly requested to refrain from large db schema changes and avoid deploying any kind of new feature that requires creation of tables. There will be a train freeze in the week of Sept 10th and Oct 8th. The net effect of the switchover and switchback for volunteers is expected to be some minutes of inability to save an edit. For readers, everything will be as usual. The tracking task for interested parties is https://phabricator.wikimedia.org/T199073 Regards, -- Alexandros Kosiaris <akosiaris(a)wikimedia.org>

7 10

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l October 2018