Wikitech-l January 2017

wikitech-l@lists.wikimedia.org

85 participants
80 discussions

[Wikimedia Labs][Announce] NFS (only labs projects) maintenance on 2017-01-18
by Madhumitha Viswanathan 19 Jan '17

19 Jan '17

Hello, Continuing the storage redundancy and reliability efforts for Labs ( https://phabricator.wikimedia.org/T126083), the final migration to the new NFS storage cluster for Labs projects with NFS enabled is upcoming. The migration is planned to happen 2017-01-18 starting 09:00 PST(16:00 UTC). This *does not* affect tools, maps or any other projects that don't have /home or /data/project mounted. The migration window is expected to be fairly short (<3 hours) - but could last up to 6 hours. During the migration, no new data will be written to NFS (/home or /data/project), but existing data will be accessible in Read-only mode for the most part. Post migration, any services or jobs that were running on top of NFS (/home or /data/project) will require manual restarts. Jobs running on top of /scratch or /public/dumps will be unaffected. I will keep the lists and #wikimedia-labs updated on progress during and after the migration. The list of labs projects that will be affected in this migration are: - catgraph - account-creation-assistance - contributors - wikidata-topicmaps - sugarcrm - wikidumpparse - video - openstack - testlabs - wikidata-dev - quarry - huggle - editor-engagement - utrs - wmt - cvn - fastcci - toolsbeta - project-proxy - dumps - bots - snuggle - math - wikisource-tools The tracking task on phabricator is here - https://phabricator.wikimedia.org/T154336. Let us know if you have any questions or concerns on the list or on #wikimedia-labs. -- Madhu Viswanathan Operations Engineer, Wikimedia Labs

1 6

ArchCom Status & Meeting Minutes, WD3
by Daniel Kinzler 19 Jan '17

19 Jan '17

Hi all! Here are the minutes from this week's ArchCom meeting. You can also find the minutes at <https://www.mediawiki.org/wiki/Architecture_committee/2017-01-18>. See also the ArchCom status page at <https://www.mediawiki.org/wiki/Architecture_committee/Status> and the RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. Here are the minutes, for your convenience: * 2 RFCs approved after final call: Deprecation Policy ([[Phab:T146965|T146965]]) and Content Model Storage ([[Phab:T105652|T105652]]). * Multi-Content-Revisions (MCR, [[Phab:T107595|T107595]]) discussed at DeveloperSummit. Sorted out details of the DB schema with Jaime. * Ongoing discussion regarding distribution methods for MediaWiki and suppor for 3rd party installs. ** Related discussion about whether new features can require services serparate from MediaWiki core. * Thumbnail API is mostly stalled for now [[Phab:T66214|T66214]]. * Oldimage RFC ([[Phab:T589|T589]]) to go on final call this week. ** Decided to keep media type and mime type fields for now. Thy are small, and now exposed via Cirrus Search; if we removed them from the table, they would need to love somewhere else. Also they are enums, and thus small. ** Tentative plan to fold image revision management into MCR at some point, but not until MCR is mature. Also, this schema change will make a later migration to MCR much easier. * ArchCom is considering changes to the RFC process. Discussion is ongoing. Key points: ** More focus on discussion in Phabricator ** More leight weight process for “small” RFCs ** ArchCom to focus more on review, less ond process ** IRC meeting should not be the primary tool for discussing and approving RFCs * Next week’s IRC discussion: [[Phab:T154738|T154738]] (Accessing page properties from wiki pages). Related: ** T71441: Feature request: add detection for disambiguation pages to Scribunto ** T131911: Allow retrieving/getting page image file name from wikitext using Scribunto/Lua or parser function or something ** T154346: Provide "wikitext" means of accessing arbitrary wiki page's default category sort key -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

3 4

2017-01-18 Scrum of Scrums meeting notes
by Grace Gellerman 18 Jan '17

18 Jan '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-01-18 = 2017-01-18 = == Product == === Reading === ==== Mobile Content Service (MCS) ==== * Board: https://phabricator.wikimedia.org/project/board/1323/query/open/ * Working on: ** Clicking on references is broken (Broken in the Android app; a Parsoid change caused this) https://phabricator.wikimedia.org/T155070 ** Improving regression testing ====Android==== * Last week: ** Dev summit and all hands ** Start on improving offline support quarterly goal ** New account creation broken in API 25 T155535 ** Investigate reverted edit notifications failure * Next week (https://phabricator.wikimedia.org/project/view/2352/ ): ** Work towards release of Wikidata description editing ** Continue on improved offline experience ==== iOS native app ==== * Last Week ** Dev summit, All hands, released 5.3.4 - https://phabricator.wikimedia.org/project/view/2396/ ** Continued work on 5.4 - https://phabricator.wikimedia.org/project/view/2326/ * This week ** Continue work on 5.4 - Places, Login enhancements https://phabricator.wikimedia.org/project/view/2326/ ==== Web ==== * Last week: ** Dev summit and all hands ** Current sprint: https://phabricator.wikimedia.org/tag/reading-web-sprint-89-%F0%9F%8E%82/ * Next sprint: https://phabricator.wikimedia.org/project/view/2426/ ** Deploy Wikidata description and Related Articles on enwiki mobile === Editing === ==== Collaboration ==== * Blocked - * Blocking - * Updates ** Main patch for RecentChanges page UI finished and merged (Adding new interface for review filters to RecentChanges). ** Add userExpLevel filter in the RCFilters UI ** Continuing related work to this, including backend. ** Some meetings at Dev Summit about ReviewStream (part of Edit Review Improvements) and WikiLabels admin interface ** Flow consistency fixes == Technology == === Analytics === Updates: * Continuing with the work to replace front end and backend of http://stats.wikimedia.org : Wikistats 2.0, a mjor development is that we will be able to import the edit history from labs rather than prod dbs, huge advantage as this data is redacted and already public.* * Working on launching RCStream replacement, review stream plus several other real time streams. Will start working with collaboration on augmented ORES recent changes stream. * Started work on processing user_agent field on eventlogging tables, it will be a json string like {"OS_Family": "Windows", "Browser_family":"Chrome"....} * Found issue with Heap on cluster, set up alarms, will be solved with update to cloudera's newest * Still waiting for hardware for pageview API, getting closer === Release Engineering === * '''Bocked''' ** None * '''Blocking''' ** None? * '''Updates''' ** Deployment is back to normal, train is on schedule: https://tools.wmflabs.org/versions/ ** Scap 3.5 release eminent, changelog will be sent, check on beta please === Services === * Blockers: none * Updates: ** Node 6 upgrade is coming: https://phabricator.wikimedia.org/T149331 *** RESTBase updated, great improvement in memory consumption *** Parsoid today *** SCB tomorrow *** MCS, CXServer, EventStreams - if you have something to deploy - do it TODAY! === Discovery === * No blockers * Continuing work on crosswiki searches * Special:Search refactoring merged in * Started work on improving wikidata search * Started preparing for upgrade to ElasticSearch 5 * Added app links to portal homepage == German Technical Wishlist == * Collecting what we learned at the Dev Summit * Decided to not render URLs in the Electron PDF export any more https://phabricator.wikimedia.org/T152393 * Can we help investigating why Kartographer does not work with flagged revs, to support German Wikipedia? https://phabricator.wikimedia.org/T151665 == Wikidata == * Slow week, key people are still in the US * Currently focusing on backend services for federation ("Wikidata on Commons") https://phabricator.wikimedia.org/T76007 == Fundraising Tech == * CentralNotice banner impressions in pivot: https://pivot.wikimedia.org/#banner-impressions-hourly * More work to prevent unintended duplicate donations * More CiviCRM customization moved to a real Civi extension * Easing import of employer matching gifts * Examining contractor fileshare change * Casey training on ops-y stuff with Jeff and preparing to announce a job opening

1 0

Google Code-in 2016 is over. Congratulations everybody!
by Andre Klapper 18 Jan '17

18 Jan '17

Google Code-in 2016 ended today. Our students resolved a total of 424 Wikimedia tasks. Thanks to our 46 mentors for being available, also on weekends & holidays. Thanks to everybody on IRC for your friendliness, patience, and help provided. Some latest students' achievements: * jQuery.suggestions used to add reason suggestions to block/delete/protect forms * Created a {{PAGELANGUAGE}} magic word * The Newsletter extension received more fixes and improvements * Integrated SwaggerUI with the service template * Special:PageLanguage allows a user to enter a reason/comment * More MediaWiki code removed that was marked for removal in a past release * More updated screencast videos on the Phabricator help pages * More split videos of Wikimedia's CREDIT showcases * https://meta.wikimedia.org/wiki/Research_on_open_source_team_communication_… The Grand Prize winners & finalists will be announced on January 30th. Do you have any ideas or feedback what to improve for next time? Add it to https://phabricator.wikimedia.org/T154620 ! Again congratulations everybody, and thanks for your hard work! See you around on IRC, mailing lists, Gerrit, and Phabricator! Cheers, andre -- Andre Klapper | Wikimedia Bugwrangler http://blogs.gnome.org/aklapper/

1 0

New 1.27/1.28 login mechanism
by Aran 18 Jan '17

18 Jan '17

Hello, I have a login system that extends the AbstractPrimaryAuthenticationProvider and uses an AuthenticationRequest that returns an empty fields array so that the login form is bypassed completely and login is determined by some other environmental parameters. But in 1.27 this does not bypass the login form. What is the proper way I should be making the login page determine login without a login form? Thanks, Aran

2 1

Final Call for Comments on the Content Model Storage RFC
by Daniel Kinzler 17 Jan '17

17 Jan '17

Hi all! This is a Final Call for Comments on the RFC on Content Model Storage [1][2]. If no new and serious objections are raised within a week, the Architecture Committee will approve this RFC and drive its implementation. The RFC on Content Model Storage was originally approved in 2015, but was then postponed in favor of another RFC, which proposes to create a separate content meta-data table [3] as part of the move towards multi-content revisions (MCR) [4]. However, MCR in turn got stuck on database performance concerns. So we now propose to go ahead with implementing the original RFC. The idea is to assign a number to every content model (and content format), and then use these numbers to refer to the models and formats in the database, instead of repeating the same string millions of times (which is my fault btw, sorry about that). Since the original RFC was already approved, and the situation does not seem to have changed since then, we see no need for another round of discussions. If nobody raises any new and serious objections within a week, this should be good to go. Cheers, Daniel [1] https://phabricator.wikimedia.org/T105652 [2] https://www.mediawiki.org/wiki/Requests_for_comment/Content_model_storage [3] https://phabricator.wikimedia.org/T142980 [4] https://www.mediawiki.org/wiki/Multi-Content_Revisions/Content_Meta-Data#Da… -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 1

Final Call for Comments on the new Deprecation Policy for PHP code
by Daniel Kinzler 17 Jan '17

17 Jan '17

Hi all! At the ArchCom office hour (aka RFC IRC meeting) on December 14, we discussed Legoktm's proposal for a deprecation policy for PHP code [1]. This would supersede the current guideline [2], allowing deprecated code to be removed after a couple of releases, even if it may still used by some extensions. At the IRC meeting, we decided to approve this policy for a final call [3][4]. This means that it will be adopted as official policy if no new pertinent issues are raised within a week. Cheers, Daniel [1] https://www.mediawiki.org/wiki/Requests_for_comment/Deprecation_policy [2] https://www.mediawiki.org/wiki/Deprecation [3] https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.20… [4] https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.20… -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 1

Divide XML dumps by page.page_namespace (and figure out what to do with the "pages-articles" dump)
by MZMcBride 17 Jan '17

17 Jan '17

Hi. Re: https://phabricator.wikimedia.org/T99483 This is a task proposing dividing XML dumps by the numeric page namespace ID (such as 2 for User pages). Please share your thoughts on the task. It's currently unclear whether implementing this task would result in getting rid of the "pages-articles" dump. We could keep generating it, but it costs a non-negligible amount of disk space to do so. If you regularly use the "pages-articles" XML dump format and have thoughts about keeping it as-is or changing it, please comment on the task. If someone could forward this e-mail to the xmldatadumps-l and wiki-research-l mailing lists, I would very much appreciate it. MZMcBride

1 0

PHP fatal error: Call to undefined method stdClass::get()
by Pine W 17 Jan '17

17 Jan '17

I just started getting a lot of these errors when trying to load pages on multiple Wikimedia sites. Pine

4 9

CI now uses NodeJs version 6
by Antoine Musso 17 Jan '17

17 Jan '17

Hello, The Wikimedia Jenkins system now uses NodeJS version 6 (was 4). That is part of a plan to upgrade the Wikimedia cluster to 6 which is tracked on: https://phabricator.wikimedia.org/T149331 If you see any issue with the *node-6-jessie Jenkins jobs, please fill them as sub tasks of T149331 and add in #contint Phabricator project. Thanks Paladox for the CI configuration change and everyone that has been involved in ensuring our most important softs are compatibles. -- Antoine "hashar" Musso

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l January 2017