Wikitech-l September 2018

wikitech-l@lists.wikimedia.org

63 participants
71 discussions

Results from 2018 global Wikimedia survey are published!

by Edward Galvez

Hi everyone, I'm excited to share that our annual survey about Wikimedia communities is now published! This survey included 170 questions and reaches over 4,000 community members across four audiences: Contributors, Affiliate organizers, Program Organizers, and Volunteer Developers. This survey helps us hear from the experience of Wikimedians from across the movement so that teams are able to use community feedback in their planning and their work. This survey also helps us learn about long term changes in communities, such as community health or demographics. The report is available on meta: https://meta.wikimedia.org/wiki/Community_Engagement_Insights/2018_Report For this survey, we worked with 11 teams to develop the questions. Once the results were analyzed, we spent time with each team to help them understand their results. Most teams have already identified how they will use the results to help improve their work to support you. The report could be useful for your work in the Wikimedia movement as well! What are you learning from the data? Take some time to read the report and share your feedback on the talk pages. We have also published a blog that you can read.[1] We are hosting a livestream presentation[2] on September 20 at 1600 UTC. Hope to see you there! Feel free to email me directly with any questions. All the best, Edward [1] https://wikimediafoundation.org/2018/09/13/what-we-learned-surveying-4000-c… [2] https://www.youtube.com/watch?v=qGQtWFP9Cjc -- Edward Galvez Evaluation Strategist, Surveys Learning & Evaluation Community Engagement Wikimedia Foundation -- Edward Galvez Evaluation Strategist, Surveys Learning & Evaluation Community Engagement Wikimedia Foundation

4 years, 8 months

Enforcing ORES limit of parallel connections per IP

by Amir Sarabadani

Hello, ORES has policy that no more than four parallel connections per IP should be requesting the service. We weren't enforcing this policy and that caused several outages before. As of today if there is more than four connections per IP, the responses after fourth one will be throttled and will be slower, and after seventh connection, ORES just responds with immediate time out error. This might affect CloudVPS and Toolforge users who connect to ORES from there but there are several IPs coming from the cloud and looking at request statistics, this limit is bigger than the usage. But if you are having trouble using ORES in the cloud, let me know so I add those IPs to the whitelist. This limit only applies to external connections and it should not affect internal requests coming from mediawiki or the change propagation service. For more information, please read https://phabricator.wikimedia.org/T160692 and file a task against ORES project in phabricator in case anything is wrong. Happy scoring! Best -- Amir Sarabadani Software Engineer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

4 years, 8 months

2018-09-19 Scrum of Scrums meeting notes

by Grace Gellerman

https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-09-19 *=2018-09-1**9**=* == Callouts == * UI Standardization: Rolled-out OOUI Special:Preferences with user-experience improvements to Beta-Cluster yesterday, on normal train next week. ** Have a look at https://en.wikipedia.beta.wmflabs.org/wiki/Special:Preferences …and provide feedback for any issues – https://phabricator.wikimedia.org/T180538 * Performance: Prepped documentation on how to respond to performance alerts, aimed at engineering teams that are not Perf in order to speed up cycle time: https://wikitech.wikimedia.org/wiki/Performance/Regressions * Feedback requested on advanced mobile contributions prototype https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions/Na… [Readers Web] * Release Engineering: please help us to resolve train blockers, the train is not moving forward until they are resolved ** T204758 EchoForeignWikiRequest emits PHP error "Undefined index: csrftoken" ** T204797 [Regression 1.32.0-wmf.22] ParserOutput::getLanguageLinks returns invalid values (Undefined index from ApiParse and LinksUpdate) ** T204826 EchoForeignWikiRequest emits PHP error "Undefined index: query" == Audiences == === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: ** ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: ** Finishing up work on Partial Blocks ** Waiting for schema changes to be executed on production https://phabricator.wikimedia.org/T204006 ==== Editing ==== * Blocked by: * Blocking: ** Updates: ** Moved MobileFrontend's VisualEditor styles back to MobileFrontend **Fixed scrolling OOUI dialogs on iOS ==== Growth ==== * Blocked by: * Blocking: * Updates: ** ==== Language ==== * Blocked by: * Blocking: * Updates: ** Various fixes for Content Translation v2 to be deployed with this week's train. === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** Still working on 6.1 (wikidata description editing, echo notifications) ( https://phabricator.wikimedia.org/tag/ios-app-v6.1-narwhal-on-a-bumper-car/) ==== Android native app ==== * Blocked by: * Blocking: * Updates: ** Wrapping up navigation design updates/refinements. ==== Readers Web ==== * Blocked by: * Blocking: * Updates: ** Mobile website (MinervaNeue / MobileFrontend): *** Feedback requested on advanced mobile contributions prototype https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions/Na… *** Invest in the MobileFrontend & MinervaNeue frontend architecture https://www.mediawiki.org/wiki/Reading/Web/Projects/Invest_in_the_MobileFro… **** Transition View, utils, and other files to Webpack T203100 **** CategoryOverlay refactor T191987 **** Remove unused dev scripts from MobileFrontend T202743 *** Page issues https://www.mediawiki.org/wiki/Reading/Web/Projects/Mobile_Page_Issues **** Limited A/B test imminent T204609 **** ReadingDepth sometimes initialises before PageIssues T204144 **** Safari sendBeacon issues T204143 **** Add red link logging T204073 **** QA https://people.wikimedia.org/~jdrewniak/more_ambox/ https://people.wikimedia.org/~jdrewniak/page_issues_inventory/en.index.html **** New / old treatment A/B test can be seen here https://readers-web-master.wmflabs.org/wiki/Pharmacovigilance?mobileaction=… *** Fix Edit schema EventLogging T204584 T203615 *** Maintenance and bug fixes T197884 T199066 *** Design and product auditing special pages in Minerva ** PDF rendering (Proton) https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality *** Remaining work tracked in T186748 *** Working with Services on a Grafana dashboard T201158 *** Miscellaneous maintenance and bug fixes ** Supporting Readers hiring processes ==== Readers Infrastructure ==== * Blocked by: * Blocking: * Updates: ** MCS, PCS: Nothing particularly exciting. ** Maps: *** Prod upgrade to stretch in progress *** Beta testing of tilerator resource change events for purging map tiles is in progress **** Need change-propagation deployed for the next step ==== Multimedia ==== * Updates ** File page with multi-lingual captions in review ** MCR work has begun ** Artwork/editing wikidata design continue ** Looking into restore old revisions ** Refining structured data search prototype ** hiring! ==== Parsing ==== * Blocked by: * Blocking: * Updates: - Patches needing review from other teams - https://gerrit.wikimedia.org/r/460032 - Deprecate Preprocessor_Hash (Mediawiki Platform) - https://gerrit.wikimedia.org/r/460200 - Preprocessor dependency in wikibase (WMDE) - https://gerrit.wikimedia.org/r/458898 / https://gerrit.wikimedia.org/r/458899 - Remex on translatewiki (Language Team / Niklas) - https://gerrit.wikimedia.org/r/446766 - BCP47 in Babel extension (Language Team / Niklas) ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** UI Standardization: Rolled-out OOUI Special:Preferences with user-experience improvements to Beta-Cluster yesterday, on normal train next week. https://www.mediawiki.org/wiki/Improve_Special_Preferences_user_experience_… Have a look at https://en.wikipedia.beta.wmflabs.org/wiki/Special:Preferences …and provide feedback for any issues – https://phabricator.wikimedia.org/T180538 ** == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** === Cloud Services === * Blocked by: * Blocking: * Updates: ** === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Trying to rescue data scrambled by a payments-wiki bug last Thursday ** Fixing bugs in latest CiviCRM point release ** Supporting Amazon Alexa donations ** Still working on CentralNotice / translate issues === MediaWiki Core Platform === * Blocked by: * Blocking: * Updates: ** working on session themes for Wikimedia Technical Conference ** actor table and comment table deployment unblocked by DC switch, writing to new schema enabled on test wikis and mediawiki.org ** worked with Reedy to stop deploying symfony polyfills we don't need (gerrit:460464) - 3.5k less lines of code! === Performance === * Blocked by: * Blocking: * Updates: ** Starting perf testing on low-power Android Go phones ** Looking at moving our WebPageTest setup to bare metal - working with SRE to do optimization testing ** Analyzing sitemaps value ** Quieting some of our alerts, so that they don't bother SRE (only us) ** Prepped documentation on how to respond to performance alerts, aimed at engineering teams that are not Perf in order to speed up cycle time: https://wikitech.wikimedia.org/wiki/Performance/Regressions ** Helped investigated slowness of the recent changes API ** Major triage of mediawiki-database and wikimedia-prod-errors Phab dashboards === Release Engineering === * Blocked by: ** [WMCS] Increased quotas for vcpu and memory in integration project: https://phabricator.wikimedia.org/T204373 * Blocking: ** * Updates: ** Train Health: *** no train last week due to DC switchover, train continues this week *** Three blockers so far :( **** T204758 EchoForeignWikiRequest emits PHP error "Undefined index: csrftoken" **** T204797 [Regression 1.32.0-wmf.22] ParserOutput::getLanguageLinks returns invalid values (Undefined index from ApiParse and LinksUpdate) **** T204826 EchoForeignWikiRequest emits PHP error "Undefined index: query" ** Log Health: No complains. This time. ** Code Health: *** Code Health Metrics Working Group Kickoff last week *** Code Health Metrics Working Group meeting this week - further discuss/define the workgroup's scope and next steps === Research === * Blocked by: None * Blocking: None * Updates: - * Re-starting data collection for CitationUsage: https://phabricator.wikimedia.org/T191086 - * Exploring further improvements on the recommendation API: https://phabricator.wikimedia.org/T203041 - === Scoring Platform === * Blocked by: None * Blocking: None * Updates: ** PoolCounter for ORES is deployed ** Working on design of secondary tables for JADE and using MCR in JADE === Search Platform === * Blocked by: * Blocking: * Updates: ** === Security === * Blocked by: * Blocking: * Updates: ** === Services === * Blocked by: * Blocking: * Updates: ** === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** Goal planning in process ** Switchover went very well, switchback on week of Oct 8th ** Work on Apache traffic server continuing ** PHP7 work continuing, php 7.2 part stalled as no CI+mediawiki test acceptance yet ** Icinga upgrade to stretch ongoing (should be uneventful, but still FYI) == Wikidata == * Blocked by: * Blocking: * Updates: ** == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** == Multi-Content Revisions == * Blocked by: * Blocking: * Updates: ** software development for file caption and statement support in MCR is complete ** deployment is in progress *** switch to "read new" on testwiki this week followed by mediawiki.org after a few days of testing

4 years, 8 months

Security pre-release announcement: 1.27.5 / 1.29.3 / 1.30.1 / 1.31.1

by Sam Reed

Hi all, Tomorrow we will be issuing a security release to all supported branches of MediaWiki. The new releases will be: 1.31.1 1.30.1 1.29.3 1.27.5 This will resolve 4 issues in MediaWiki core, and also includes some previously committed to git minor security and hardening patches. Fixes will be available in these respective release branches, and also master. Tarballs will be available for the above mentioned point releases as well. 1.29 was due to be previously announced as end of life, and as such 1.29.3 will be the final security and maintenance release barring any unforeseen issues. This security release includes fixes for MediaWiki core. --- Sam Reed

4 years, 8 months

Discovery Weekly Update for the week starting 2018-09-10

by Chris Koerner

A belated greetings, This is the weekly update from the Search Platform team for the week starting '''2018-09-03''' and '''2018-09-10''' == Highlights== * New blog post: The anatomy of search: Variation under nature - "This is the second in a series of blog posts giving a basic overview of how full-text search engines work, with a bit of extra attention for some interesting language-specific issues. Our topic for today is how we normalize the words we found in our text during the previous step, tokenization." [0] == Discussions == === Search === * Trey finished up the Esperanto analysis, stemmer updates and reindexing of the wikis [1], [2], [3] * Erik tackled a lot of small bugs that will be in production the week of Sep 18th: redirect showing irrelevant info [4], strings starting with a # sign were redirecting to main page [5], error when using double quotes in the category or title [6], "close redirects" in error messages [7], UI issue where 'results 1 of 2' is displayed when only one result is found [8] * Mathew fixed an issue with alerting for when Elasticsearch has shards larger than the maximum size [9] * Erik worked on getting CirrusSearch to gracefully handle missing plugin sections in the ElasticSearch response [10] * Gehel and Erik did some Mjolnir query daemon monitor work [11], [12] * Trey fixed an issue where empty strings caused by ICU folding in ElasticSearch were getting indexed [13] * The work for using Kafka for communicating between the analytics cluster and elasticsearch has finished [14] === Wikidata Query Service === * Diskspace upgrade has been finished * Categories are now updated daily in WDQS from daily diffs [15] [0] https://wikimediafoundation.org/2018/09/13/anatomy-search-variation-under-n… [1] https://phabricator.wikimedia.org/T202173 [2] https://phabricator.wikimedia.org/T202662 [3] https://phabricator.wikimedia.org/T203005 [4] https://phabricator.wikimedia.org/T190010 [5] https://phabricator.wikimedia.org/T182452 [6] https://phabricator.wikimedia.org/T73123 [7] https://phabricator.wikimedia.org/T191485 [8] https://phabricator.wikimedia.org/T71382 [9] https://phabricator.wikimedia.org/T203546 [10] https://phabricator.wikimedia.org/T191493 [11] https://phabricator.wikimedia.org/T199732 [12] https://phabricator.wikimedia.org/T200740 [13] https://phabricator.wikimedia.org/T192502 [14] https://phabricator.wikimedia.org/T198490 [15] https://phabricator.wikimedia.org/T201217 ---- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Relations Specialist Wikimedia Foundation

4 years, 8 months

Wikimedia has been accepted for Google Code-in 2018

by Andre Klapper

As written in https://lists.wikimedia.org/pipermail/wikitech-l/2018-September/090799.html we applied. Today we got accepted as one of 27 organizations! Now we need many tasks and mentors. Please check https://www.mediawiki.org/wiki/Google_Code-in/Mentors if you have not yet and provide some smaller tasks that you'd mentor. Thanks! andre -- Andre Klapper | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

4 years, 8 months

Feedback wanted on mobile web contribution prototype

by Chris Koerner

Howdy, The Readers web team at the Wikimedia Foundation is working on improving the contribution features on the mobile website. [0] The team is working on showing pages that are currently unavailable on mobile. We are starting by designing improvements to the mobile navigation. We want your feedback on the latest navigation prototype. If you have a moment, please visit the project page and try out the demo on your mobile device. [1] Then help answer a few questions about your experience. Thank you! [0] https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions [1] https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions/Na… Yours, Chris Koerner Community Relations Specialist Wikimedia Foundation

4 years, 8 months

Wednesday: Technical Advice IRC Meeting

by Michael Schönitzer

Reminder: Technical Advice IRC meeting again **Wednesday 3-4 pm UTC** on #wikimedia-tech. Question can be asked in English & German. The Technical Advice IRC Meeting is a weekly support event for volunteer developers. Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting Hope to see you there! Michi (for the Technical Advice IRC Meeting crew) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

4 years, 8 months

Commons app update: v2.8.3

by Josephine Lim

Hi all, We've just released version 2.8.3 of the Commons Android app to all users. :) The new version is available for download on the Google Play Store[1] as well as F-Droid[2]. The update includes fixes for most of the high-priority bugs in the app, including failed uploads and overwrites. Additionally, we have several new features, notably: - When you upload an image to an item on the map of "Nearby places that need pictures", the image will be automatically added to that Wikidata item - Browse other images on Commons via the new "Explore" feature - View your achievements and upload statistics (by tapping on the trophy icon next to your username) We are aware that there are a few recent issues that need resolving, namely crashes for users on Android 8.1 and above, and issues with date templates. Those will be fixed ASAP in v2.9. Thank you all for your patience as we work on these! More details and screenshots can be found in my blog post[3]. Feedback, bug reports, and suggestions are always welcome on our GitHub page[4]. Best regards, Josephine / misaochan (project maintainer) [1]: https://play.google.com/store/apps/details?id=fr.free.nrw.commons [2]: https://f-droid.org/repository/browse/?fdid=fr.free.nrw.commons [3]: https://cookiesandcodeblog.wordpress.com/2018/08/10/commons-app-version-2-8… [4]: https://github.com/commons-app/apps-android-commons/issues

4 years, 8 months

57 VPS projects are candidates for deletion

by Andrew Bogott

There are currently 57 unclaimed projects on https://wikitech.wikimedia.org/wiki/Cloud_VPS_2018_Purge. I will start shutting down unclaimed projects at the beginning of next month, and those projects will be left behind in the future network migration[1] and, eventually, deleted. If you see anything in this list that you care about, please indicate that the project is in use on the Wikitech page. Here are the 57 projects that are currently unmarked: aicaptcha bots bstorm-test chicotestproject ci-staging collection-alt-renderer contributors deep-learning-services discourse-wam download etytree fastcci glampipe globaleducation hat-imagescalers hound iiifls1 ircd jupyter maps maps-team matrix mcr-dev mediahandler-tests mediawiki-docker multimedia mw-api-testing mw-extension-ids newsletter nonfreewiki openocr orig pagemigration piwik project-smtp rdfiodev reading-lists redirects reportcard sentry services services-testbed t136871 test-twemproxy tor traffic wcdo wdq-mm wikibrain wikidata-page-banner wikidata-primary-sources-tool wikidumpparse wikifactmine wikimetrics wikisource-tools wikistream wlmjudging [1] https://phabricator.wikimedia.org/phame/post/view/112/neutron_is_finally_co…

4 years, 8 months

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l September 2018