Wikitech-l November 2017

wikitech-l@lists.wikimedia.org

60 participants
56 discussions

Database dumps
by Byrial Jensen 17 Apr '25

17 Apr '25

Until some weeks ago http://dumps.wikimedia.org/backup-index.html used to show 4 dumps in progress at the same time. That meant that new database dumps normally was available within about 3 weeks for all databases except for enwiki and maybe dewiki where the dump process due to size took longer time. However the 4 dumps processes at one time become 3 some weeks ago. And after massive failures at June 4, only one dump has been in progress at the same time. So at the current speed it will take several months to come thru all dumps. Is it possible to speed up the process again using several dump processes at the same time? Thank you, Byrial

3 2

User-Agent:
by Domas Mituzas 17 Apr '25

17 Apr '25

Hi! from now on specific per-bot/per-software/per-client User-Agent header is mandatory for contacting Wikimedia sites. Domas

19 61

Acquiring list of templates including external links
by Takashi OTA 17 Apr '25

17 Apr '25

Hoi, This is an inquiry from my friend in academia, researching about Wikipedia. He would like to know whether there's a way to acquire a list of templates including external links. Here are some examples including external links. https://ja.wikipedia.org/wiki/Template:JOI/doc https://ja.wikipedia.org/wiki/Template:Twitter/doc Such links are stored in externallinks.sql.gz, in an expanded form. When you want to check increase/decrease of linked domains in chronological order through edit history, you have to check pages-meta-history1.xml etc. In a such case, traditional links and links by templates are mixed, Therefore, the latter ones (links by templates) should be expanded to traditional link forms. Sorry if what I am saying does not make sense. Thanks in advance, --Takashi Ota [[U:Takot]]

13 24

EBNF grammar project status?
by Steve Bennett 01 Apr '25

01 Apr '25

What's the status of the project to create a grammar for Wikitext in EBNF? There are two pages: http://meta.wikimedia.org/wiki/Wikitext_Metasyntax http://www.mediawiki.org/wiki/Markup_spec Nothing seems to have happened since January this year. Also the comments on the latter page seem to indicate a lack of clear goal: is this just a fun project, is it to improve the existing parser, or is it to facilititate a new parser? It's obviously a lot of work, so it needs to be of clear benefit. Brion requested the grammar IIRC (and there's a comment to that effect at http://bugzilla.wikimedia.org/show_bug.cgi?id=7 ), so I'm wondering what became of it. Is there still a goal of replacing the parser? Or is there some alternative plan? Steve

26 217

Missing Section Headings
by Marc Riddell 13 Sep '24

13 Sep '24

Hello, I have been a WP editor since 2006. I hope you can help me. For some reason I no longer have Section Heading titles showing in the Articles. This is true of all Headings including the one that carries the Article subject's name. When there is a Table of Contents, it appears fine and, when I click on a particular Section, it goes to that Section, but all that is there is a straight line separating the Sections. There is also no button to edit a Section. If I edit the page and remove the "== ==" markers from the Section Titles, the Title then shows up, but not as a Section Heading. Also, I don't have any Date separators on my Want List. This started 2 days ago. Any thoughts? Thanks, Marc Riddell [[User:Michael David]]

10 11

How to make commits from mirrored Wikimedia repos show up on your GitHub profile
by Bartosz Dziewoński 27 Jun '24

27 Jun '24

I know it has been annoying a couple of people other than me, so now that I've learned how to make it work I'll share the knowledge here. tl;dr: Star the repositories. No, seriously. (And yes, you need to star each extension repo separately.) (Is there a place on mw.org to put this tidbit on?) ------- Forwarded message ------- From: "Brian Levine" <support(a)github.com> (GitHub Staff) To: matma.rex(a)gmail.com Cc: Subject: Re: Commits in mirrored repositories not showing up on my profile Date: Tue, 09 Jul 2013 06:47:19 +0200 Hi Bartosz In order to link your commits to your GitHub account, you need to have some association with the repository other than authoring the commit. Usually, having push access gives you that connection. In this case, you don't have push permission, so we don't link you to the commit. The easy solution here is for you to star the repository. If you star it - along with the other repositories that are giving you this problem - we'll see that you're connected to the repository and you'll get contribution credit for those commits. Cheers Brian -- Matma Rex

3 3

Research FAQ gets a facelift
by Dario Taraborelli 25 Jun '24

25 Jun '24

We just released a new version of Research:FAQ on Meta [1], significantly expanded and updated, to make our processes at WMF more transparent and to meet an explicit FDC request to clarify the role and responsibilities of individual teams involved in research across the organization. The previous version – written from the perspective of the (now inactive) Research:Committee, and mostly obsolete since the release of WMF's open access policy [2] – can still be found here [3]. Comments and bold edits to the new version of the document are welcome. For any question or concern, you can drop me a line or ping my username on-wiki. Thanks, Dario [1] https://meta.wikimedia.org/wiki/Research:FAQ [2] https://wikimediafoundation.org/wiki/Open_access_policy [3] https://meta.wikimedia.org/w/index.php?title=Research:FAQ&oldid=15176953 *Dario Taraborelli *Head of Research, Wikimedia Foundation wikimediafoundation.org • nitens.org • @readermeter <http://twitter.com/readermeter>

2 1

bluejeans
by Jeremy Baron 22 Feb '23

22 Feb '23

Hi, On Tue, Mar 1, 2016 at 3:36 PM, David Strine <dstrine(a)wikimedia.org> wrote: > We will be holding this brownbag in 25 minutes. The Bluejeans link has > changed: > > https://bluejeans.com/396234560 I'm not familiar with bluejeans and maybe have missed a transition because I wasn't paying enough attention. is this some kind of experiment? have all meetings transitioned to this service? anyway, my immediate question at the moment is how do you join without sharing your microphone and camera? am I correct thinking that this is an entirely proprietary stack that's neither gratis nor libre and has no on-premise (not cloud) hosting option? are we paying for this? -Jeremy

9 16

New DB_REPLICA constant; DB_SLAVE deprecated
by Aaron Schulz 04 Jul '20

04 Jul '20

As of 950cf6016c, the mediawiki/core repo was updated to use DB_REPLICA instead of DB_SLAVE, with the old constant left as an alias. This is part of a string of commits that cleaned up the mixed use of "replica" and "slave" by sticking to the former. Extensions have not been mass converted. Please use the new constant in any new code. The word "replica" is a bit more indicative of a broader range of DB setups*, is used by a range of large companies**, and is more neutral in connotations. Drupal and Django made similar updates (even replacing the word "master"): * https://www.drupal.org/node/2275877 * https://github.com/django/django/pull/2692/files & https://github.com/django/django/commit/beec05686ccc3bee8461f9a5a02c607a023… I don't plan on doing anything to DB_MASTER, since it seems fine by itself, like "master copy", "master tape" or "master key". This is analogous to a master RDBMs database. Even multi-master RDBMs systems tend to have a stronger consistency than classic RDBMs slave servers, and present themselves as one logical "master" or "authoritative" copy. Even in it's personified form, a "master" database can readily be thought of as analogous to "controller", "governer", "ruler", lead "officer", or such.** * clusters using two-phase commit, galera using certification-based replication, multi-master circular replication, ect... ** https://en.wikipedia.org/wiki/Master/slave_(technology)#Appropriateness_of_… *** http://www.merriam-webster.com/dictionary/master?utm_campaign=sd&utm_medium… -- -Aaron

4 3

Roadmap for CX?
by Strainu 06 Sep '18

06 Sep '18

Following the recent outage, we've had a new series of complaints about the lack of improvements in CX, especially related to server-side activities like saving/publishing pages. Now, I know the team is involved in a long-term effort to merge the editor with the VE, but is there an end in sight for that effort? Can I tell people who ask "look, 6 more months then we'll have a much better translation tool"? Is there a publicly available roadmap for this project and more generally, for CX? Thanks, Strainu

5 8

How to customize Preferences
by Sam Wilson 19 Dec '17

19 Dec '17

Hi all, (Reposting https://www.mediawiki.org/wiki/Topic:Tyvfh19mba4pway9 here to garner more input.) I'm working on Extension:GlobalPreferences and trying to figure out how best to do things with all preferences, after they've been defined (in order to show various extra Preferences-form bits and pieces that depend on knowing about all preferences). At the moment, we're using $wgExtensionFunctions and hacking the $wgHooks global to add a new callback at the end of $wgHooks['GetPreferences']. One idea is to add a new MediaWiki service called 'PreferencesFactory', that can be used to retrieve a new Preferences object. Extensions would then be able to use the MediaWikiServices hook to redefine the PreferencesFactory (with MediaWikiServices::redefineService()). Of course, only one extension would be able to do that (which maybe is a bit odd). Apart from being able to override the Preferences class, a service for this would also mean the Preferences class could be refactored (gradually?) to not be such a collection of static methods. The proposed patch is: https://gerrit.wikimedia.org/r/#/c/374451/ I'd love to hear anyone's ideas about this, including completely different and better ways to do things. :-) Another idea is to add a new hook, after GetPreferences. This wouldn't be as flexible as the PreferencesFactory idea, but is a lot simpler. Thanks, Sam.

1 1

You can now vote in the Community Wishlist Survey
by Johan Jönsson 10 Dec '17

10 Dec '17

Hey everyone, The voting phase of the 2017 Community Wishlist Survey has now started. Read the proposals and support the ones you want to support to make the wikis better: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey Click on the categories to find the proposals. The voting will close on December 10. That's the important part of this email. Feel free to follow the link above and starting voting right now. The longer version: The Community Wishlist Survey decides what the Wikimedia Foundation Community Tech team will work on over the next year. The team is responsible for addressing the top 10 wishes on the list, as well as some wishes from smaller groups and projects that are doing important work, but don't have the numbers to get their proposal into the top 10. The Wishlist is also used by volunteer developers and other teams, who want to find projects to work on that the community really wants. Come help set the agenda. If you want to see what the team has done in 2017, see the status report from last month: https://meta.wikimedia.org/wiki/2016_Community_Wishlist_Survey/Status_repor… What you can do now: *) Vote. This is the most important thing. *) Spread the word. We really want people to find this, of course, and we'll work on finding the best balance between spreading the news to everyone and not being annoying, but please do help to spread the information in your local community – Village Pump equivalents, IRC channels, social media groups and so on. *) Help translating the pages. We want the process to be as available as possible for everyone. It's not every available if it's only in English. *) If you want to get short updates through the notification system, you can sign up for the Community Tech Newsletter: https://www.mediawiki.org/wiki/Newsletter:The_Community_Tech_Newsletter //Johan Jönsson --

1 1

Can we drop revision hashes (rev_sha1)?
by Daniel Kinzler 07 Dec '17

07 Dec '17

Hi all! I'm working on the database schema for Multi-Content-Revisions (MCR) <https://www.mediawiki.org/wiki/Multi-Content_Revisions/Database_Schema> and I'd like to get rid of the rev_sha1 field: Maintaining revision hashes (the rev_sha1 field) is expensive, and becomes more expensive with MCR. With multiple content objects per revision, we need to track the hash for each slot, and then re-calculate the sha1 for each revision. That's expensive especially in terms of bytes-per-database-row, which impacts query performance. So, what do we need the rev_sha1 field for? As far as I know, nothing in core uses it, and I'm not aware of any extension using it either. It seems to be used primarily in offline analysis for detecting (manual) reverts by looking for revisions with the same hash. Is that reason enough for dragging all the hashes around the database with every revision update? Or can we just compute the hashes on the fly for the offline analysis? Computing hashes is slow since the content needs to be loaded first, but it would only have to be done for pairs of revisions of the same page with the same size, which should be a pretty good optimization. Also, I believe Roan is currently looking for a better mechanism for tracking all kinds of reverts directly. So, can we drop rev_sha1? -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

16 29

Proposal regarding the handling of imported usernames
by Brad Jorsch (Anomie) 06 Dec '17

06 Dec '17

Handling of usernames in imported edits in MediaWiki has long been weird (T9240[1] was filed in 2006!). If the local user doesn't exist, we get a strange row in the revision table where rev_user_text refers to a valid name while rev_user is 0 which typically indicates an IP edit. Someone can later create the name, but rev_user remains 0, so depending on which field a tool looks at the revision may or may not be considered to actually belong to the newly-created user. If the local user does exist when the import is done, the edit is attributed to that user regardless of whether it's actually the same user. See T179246[2] for an example where imported edits got attributed to the wrong account in pre-SUL times. In Gerrit change 386625[3] I propose to change that. - If revisions are imported using the "Upload XML data" method, it will be required to fill in a new field to indicate the source of the edits, which is intended to be interpreted as an interwiki prefix. - If revisions are imported using the."Import from another wiki" method, the specified source wiki will be used as the source. - During the import, any usernames that don't exist locally (and can't be auto-created via CentralAuth[4]) will be imported as an otherwise-invalid name, e.g. an edit by User:Example from source 'en' would be imported as "en>Example".[5] - There will be a checkbox on Special:Import to specify whether the same should be done for usernames that do exist locally (or can be created) or whether those edits should be attributed to the existing/autocreated local user. - On history pages, log pages, and the like, these usernames will be displayed as interwiki links, much as might be generated by wikitext like " [[:en:User:Example|en>Example]]". No parenthesized 'tool' links (talk, block, and so on) will be generated for these rows. - On WMF wikis, we'll run a maintenance script to clean up the existing rows with valid usernames and rev_user = 0. The current plan there is to attribute these edits to existing SUL users where possible and to prefix them with a generic prefix otherwise, but we could as easily prefix them all. - Unfortunately it's impossible to retroactively determine the actual source of old imports automatically or to automatically do anything about imports that were misattributed to a different local user in pre-SUL times (e.g. T179246[2]). - The same will be done for CentralAuth's global suppression blocks. In this case, on WMF wikis we can safely point them all at Meta. If you have comments on this proposal, please reply here or on https://gerrit.wikimedia.org/r/#/c/386625/. Background: The upcoming actor table changes[6] require some change to the handling of these imported names because we can't have separate attribution to "Example as a non-registered user" and "Example as a registered user" with the new schema. The options we've identified are: 1. This proposal, or something much like it. 2. All the existing rows with rev_user = 0 would have to be attributed to the existing local user (if any), and in the future when a new user is created any existing edits attributed to that name will be automatically attributed to that new account. 3. All the existing rows with rev_user = 0 and an existing local user would have to be re-attributed to different *valid* usernames, probably randomly-generated in some manner, and in the future when a new user is created any existing edits for that name would have to be similarly re-attributed. 4. Like #2, except the creation (including SUL auto-creation) of the same-named account would not be allowed. Thus, an import before the local name exists would forever block that name from being used for an actual local account. 5. Some less consistent combination of the "all the existing rows" and "when a new user is created" options from #2–4. Of these options, this proposal seems like the best one. [1]: https://phabricator.wikimedia.org/T9240 [2]: https://phabricator.wikimedia.org/T179246 [3]: https://gerrit.wikimedia.org/r/#/c/386625/ [4]: https://phabricator.wikimedia.org/T111605 [5]: ">" was chosen rather than the more typical ":" because the former is already invalid in all usernames (and page titles). While a colon is *now* disallowed in new usernames, existing names created before that restriction was added can continue to be used (and there are over 12000 such usernames in WMF's SUL) and we decided it'd be better not to suddenly break them. [6]: https://phabricator.wikimedia.org/T167246 -- Brad Jorsch (Anomie) Senior Software Engineer Wikimedia Foundation

3 5

Bugfix for diff engine & new Feature in testing
by Michael Schönitzer 06 Dec '17

06 Dec '17

Hi there, The Technical Wishes Team of Wikimedia Germany is currently working on a soon-to-be deployed major improvement of the diff view engine. [1] It is now available on test wiki [2] and Mediawiki [3] and it would be great if you could test it and tell us if you encounter any bugs. [4] The deployment to all other wikis will hopefully happen in the near future. While working on this, we also found an old bug in the diff engine: Sometimes two paragraphs which are not related at all are considered a changed paragraph: example [5]. We have fixed this bug now and it was already deployed on all wikis by the end of October. [6] [7] The bigger improvement of the diff view will be announced separately, once the date for its deployment is fixed – expect it in a few weeks. Regards, Michael for the Technical Wishes Team [1] https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Show_text_changes_whe… [2] http://test.wikipedia.org [3] http://mediawiki.org [4] https://phabricator.wikimedia.org/T146781 [5] https://meta.wikimedia.org/wiki/File:ImprovedDiff_-_Example_1_-_before2.png [6] https://gerrit.wikimedia.org/r/#/c/356582/, [7] https://phabricator.wikimedia.org/T146781 -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

3 3

Proposal for a developer support channel
by Quim Gil 02 Dec '17

02 Dec '17

Hi, The Technical Collaboration team proposes the creation of a developer support channel focusing on newcomers, as part of our Onboarding New Developer program. We are proposing to create a site based on Discourse (starting with a pilot in discourse-mediawiki.wmflabs.org) and to point the many existing scattered channels there. This is an initial proposal for a pilot. If the pilot is successful, we will move it production. For that to happen we still need to sync well with Wikimedia Cloud Services, Operations and the Wikimedia technical community. Please check https://www.mediawiki.org/wiki/Discourse and share your feedback. -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

14 22

AdvancedSearch beta feature now on Mediawiki
by Birgit Müller 30 Nov '17

30 Nov '17

(sorry for cross-posting ...) Hello all, we're really happy to announce that the new AdvancedSearch interface got deployed as a beta feature to Mediawiki.org and test just 2 hours ago. [1] AdvancedSearch enhances Special:Search through an advanced parameters form and aims to make existing search options more visible and accessible for everyone. [2] The feature is a project by WMDE's Tech team and originates from the Technical Wishes project. Many other people and teams helped making this project happen, so stay tuned for the thank you section at the end of this email :-) [3] *Why AdvancedSearch* The Search has great options to perform advanced queries, but often even experienced editors don't know about it - this is what we found out when we were conducting a workshop series on advanced search in several cities in Germany in 2016. Together with contributors from German Wikipedia we've discussed their desired search queries, explained how the syntax works and how keywords like "hastemplate", "filetype" or "intitle" can be used and combined to get the desired results. The idea for the AdvancedSearch feature results out of these workshops, where we not only discussed search but also designed first mocks for an advanced search interface. [4] *The first version, more deployments and other next steps*AdvancedSearch supports some of the special search options the WMF's search team has been implemented in the last years. The way the interface works, users don't have to know the syntax behind each search field, but they can learn about it if they want to. The first version of the feature comes with a first selection of advanced search options, e.g. including support for "hastemplate" or "intitle". The WMF's search team has started to work on a 'deepcat' functionality to make sub category search happen. We plan to add support for this in the future, too. If all goes well, we plan to deploy the beta feature on German and Arabic Wikipedia by Wednesday, Nov 29. [5] In the next 2-3 months we'd love to invite everyone to test the new feature on those wikis: Comments, thoughs, bug reports ... - any feedback is much appreciated! Please see https://www.mediawiki.org/wiki/Help:AdvancedSearch for how to use AdvancedSearch. Deployments to other wikis can follow later. *Thank you (finally!)* This project was developed with the support from many different people from the start, and this makes it very special to us: We would like to thank the folks from the German Wikipedia community who participated in the workshops and who were discussing and designing first ideas together with us. We would like to thank all other people who gave valuable feedback on Phabricator, onwiki and in real life. We would like to thank the Arabic Wikipedia community for getting interested in AdvancedSearch, and for offering to test this feature in an early stage. The credit for the development work on AdvancedSearch goes to our FUN team, supported by the team that usually works on Technical Wishes. Last but really not least we would like to thank the great folks from the WMF's search team who have supported this project from the start and who have done (and still do) the necessary backend work that now got supported by the AdvancedSearch extension. Sorry for the long email, and thanks for reading :-) Birgit (for the Technical Wishes team) [1] https://phabricator.wikimedia.org/T180147 (Deployment ticket group 0) [2] https://www.mediawiki.org/wiki/Help:AdvancedSearch [3] https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/AdvancedSearch [4] https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/AdvancedSearch/Worksh… [5] https://phabricator.wikimedia.org/T180128 (Deployment ticket arwiki and dewiki) -- Birgit Müller Community Communications Manager Software Development and Engineering Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

6 14

2017-11-29 Scrum of Scrums meeting notes
by Grace Gellerman 29 Nov '17

29 Nov '17

*https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-11-29 <https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-11-29>* *= 2017-11-29=* == Callouts == * Reminder! This is your last few weeks of deployments for the year/quarter! Deployment freeze starts the week of December 18th. * Reminder! This is also the first week of the big fundraising campaign. Please be extra careful not to break CentralNotice (js/skin changes, resourceloader changes) or donate.wikimedia.org. * s8 replica set of servers for wikidata (T177208) to be live on 9th January- check scripts/config that assume wikidata is on s5!). We will have a small period of read only that day. * databases no longer guaranteed to be on port 3306 only (T178359)- check also scripts that assume default/no port (should be transparent for most people) * Your performance metrics (esp first paint) may have changed on Nov 2nd. This is due to a bug in our performance monitoring that we fixed on that day. * Please provide feedback on WikimediaUI (OOUI) icon overhaul at https://phabricator.wikimedia.org/M229 – plan to settle on set by next week * Last name of thiemowmde changed, can I get an LDAP update? https://phabricator.wikimedia.org/T181130 * Special:Preferences was migrated to OOjs UI in wmf.10 (this week's train), this changes the prefs UI substantially == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: still working on 5.8 https://phabricator.wikimedia.org/project/view/2913/ ==== Reading Infrastructure ==== * Blocked by: * Blocking: * Updates: ** working on Reading Lists performance ** continuing work on media, summary, and references endpoints ** considering undeploying trending-edits service https://phabricator.wikimedia.org/T180384 ==== Web ==== * Blocked by: * Blocking: * Updates: ** Business as usual (Chromium-based PDF service)! Next up: Performance https://phabricator.wikimedia.org/T178278 ** Started work on updating/tidying the design of special pages on mobile: https://phabricator.wikimedia.org/T169162 ==== Multimedia ==== * Blocked by: N/A * Blocking: N/A * Updates ** MP3 uploads enabled Soon™ on Commons ** 3D product nearly ship-ready, heads-up for deploy coming at some point ** Wikibase/MediaInfo work for SDoC is progressing slowly ===== Maps ===== * Blocked by: None * Blocking: None * Updates: ** Ops person back this week ===== Discovery ===== * Blocked by: None * Blocking: None * Updates: ** automated Jenkins build for wikipedia.org portal working well ** updating documentation === Contributors === ==== Global Collaboration ==== * Blocked by: nobody * Blocking: ops for Flow dumps (talked to Ariel and came to an agreement, but didn't have time to work on it last week with the holidays) * Updates: ** RCFilters "live update" graduating out of beta in this week's train ** On behalf of Editing: Special:Preferences migration to OOjs UI in this week's train, UI of Special:Preferences will change substantially ==== Parsing ==== * <section> tag code being tested. ** Code in beta cluster. ** Some fixes in progress based on tests. Unlikely to be deployed this week, unless we get everything merged and tested today. === Community Tech === * Blocked by TechCom on global preferences * Not blocking * Our wishlist survey is in voting stage ==== UI Standardization ==== * Blocked: icons overhaul, pls see below. New iteration got out end of second-last week, plan to settle by end of week – pls provide feedback ** OOUI v0.24.3 released https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v…; among changes: 7 new features, among those: *** Allow adding virtual viewport spacing (Bartosz Dziewoński) *** DropdownInputWidget: Generate a hidden `<select>` in JS (Bartosz Dziewoński), *** MenuSelectWidget, PopupWidget: Automatically change popup direction if there is no space (Bartosz Dziewoński) 4 design fixes and removing Opera < 15 special treatment * Ongoing: ** OOUI & based products: *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide https://phabricator.wikimedia.org/T177432 - Provide specific feedback at https://phabricator.wikimedia.org/M229 *** Make Special:Preferences use OOUI – related UI/UX amendments https://phabricator.wikimedia.org/T180538 ** Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867 == Technology == ==== Analytics ==== * Blocked by: * Blocking: * Updates: ** Wikistats vetting almost complete, announcing next week ** Prometheus driven metrics on druid - https://grafana.wikimedia.org/dashboard/db/prometheus-druid?orgId=1 ** Working on jsonrefine on eventlogging so eventlogging data that was up to now only available on MYSQL is available on Hadoop ** Productionising Superset ** Data on eventlogging slave back in compliance with privacy policy, master data still being worked on. ==== Services ==== * Not attending personally * Blockers: none * Updates: ** Still reshaping cassandra cluster to migrate to Cassandra 3 ** wikibase-addUsagesForPage job switched to kafka queue === Performance === * Blocking * Blocked * Updates ** Callout above re: performance metrics changing on Nov 2nd. ** Working with Mozilla on some perf regressions related to FF57 ** Working on identifying some issues related to Chrome 62 === Release Engineering === * Blocking ** * Blocked ** [SSD] blocked on ops updating nodejs-devel base image: < https://phabricator.wikimedia.org/T180524> * Updates ** [MW Train] Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th. ** [MW Train] The post mortem for T181006 (Watchlist and RecentChanges failure due to ORES on frwiki and ruwiki) is scheduled for December 7th. ** [nodejs browser tests] CirrusSearch and Mobile are active with migrations. ** [nodejs browser tests] Investigation of using mwbot instead of nodemw (javascript api interfaces to MW). CirrusSearch uses mwbot. We use nodemw (maintained by a Wikia engineer). ** [ruby] We will upgrade rubocop (Ruby linter/static analyzer) across the corpus of extensions due to a security issue (pointed out by the new Github security issue notification service). We (WMF production and CI) are not affected by the issue AND we have officially deprecated our ruby browser test stack (where 99% of all rubocup uses/dependencies are) BUT we’ll do it :) ** [techdebt] Wider conversations regarding SLAs/stewardship of components/software: *https://www.mediawiki.org/wiki/Talk:Development_policy/Code_Stewardship* <https://www.mediawiki.org/wiki/Talk:Development_policy/Code_Stewardship> ** [techdebt] 2nd of the 3 tech debt series of blog posts (after the 0th introduction one on Code Health) is in drafting. ** [CI] We’ve migrated almost all of the tox (python) jobs to the new container based CI infra. === Scoring Platform === * Blocked by: ** scap and network issues preventing us from continuing to test new ORES cluster. *** https://phabricator.wikimedia.org/T181552 *** https://phabricator.wikimedia.org/T181661 * Blocking: * Updates: ** We've been having fun with heavy outages for the last day, seems to have been stabilized a few hours ago. *** https://phabricator.wikimedia.org/T181538 === Search Platform === * Blocked by: none * Blocking: none * Updates: * Merged WIkidata description indexing, starting reindex after deploy * Finished review of Serbian morphological libraries, some promise but needs bugfixes https://phabricator.wikimedia.org/T178926#3790458 * Working on improvements to LTR training https://phabricator.wikimedia.org/T177520 * Improving completion suggester interaction with namespaced prefix search https://phabricator.wikimedia.org/T178474 * Working on porting Selenium tests from Ruby to JS * Working on upgrade to Elastic 5.5 * Working on Wikidata fulltext search === Technical Operations === * Blocking ** [Releng] blocked on ops updating nodejs-devel base image: < https://phabricator.wikimedia.org/T180524> * Blocked ** * Updates ** s8 replica set of servers for wikidata ( https://phabricator.wikimedia.org/T177208 ) to be live on 9th January- check scripts/config that assume wikidata is on s5!). We will have a small period of read only that day. ** databases no longer guaranteed to be on port 3306 only ( https://phabricator.wikimedia.org/T178359 )- check also scripts that assume default/no port (should be transparent for most people) ** service-running was spamming statsd, incident report https://wikitech.wikimedia.org/wiki/Incident_documentation/20171125-statsd == Wikidata == * Fiddling with the concept of "sub-entities" in the Wikibase storage layer: https://phabricator.wikimedia.org/T163724 * We are aware of the blocker for the MediaInfo team, planning to tackle it this week: https://phabricator.wikimedia.org/T177022 * Last name of thiemowmde changed, can I get an LDAP update? https://phabricator.wikimedia.org/T181130 === Fundraising Tech === * Banners are up! Watching the money roll in * Mostly working on visualizations - our funky internal dashboard and more data for grafana * Helping fr-not-tech get good info, debugging campaign configuration * Deployed one low-level CiviCRM thing to deal better with database contention === MediaWiki Platform === * Continuing on schema changes for MCR, etc...

1 0

Re: [Wikitech-l] AdvancedSearch beta feature now on Mediawiki
by יגאל חיטרון 29 Nov '17

29 Nov '17

You are right. I just checked, vector works for me either. Thank you all. Igal On Nov 22, 2017 01:50, "Stas Malyshev" <smalyshev(a)wikimedia.org> wrote: Hi! > Hello, Birgit. Unfortunately, I can't open a phab ticket, because I'm on > mobile, and there is no way to upload a file to phabricator from mobile. > So, I'll answer you here. I'm on Lollipop, use internal browser, timeless > skin. I can't make a screenshot from this device, so I did it on another Likely there's a problem with timeless, see: https://phabricator.wikimedia.org/T181106 Vector works fine for me though. I guess that's why it's "beta" :) -- Stas Malyshev smalyshev(a)wikimedia.org

6 7

Tomorrow: Weekly Technical Advice IRC Meeting
by Michael Schönitzer 28 Nov '17

28 Nov '17

Sorry for cross-posting! Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 4-5 pm UTC** on #wikimedia-tech. The Technical Advice IRC meeting is open for all volunteer developers, topics and questions. This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki /Technical_Advice_IRC_Meeting This meeting is an offer by WMDE’s tech team. Hosts of tomorrows meeting are: @Thiemo_WMDE & @Amir1. Hope to see you there! Michi (for WMDE’s tech team) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

Google Code-in 2017 just started. Please help!
by Andre Klapper 28 Nov '17

28 Nov '17

Google Code-in 2017 just started: https://www.mediawiki.org/wiki/Google_Code-in/2017 You might remember the announcement email from a few weeks ago.[1] In the next seven weeks, many young people are going to make their first contributions to Wikimedia. Expect many questions on IRC and on mailing lists by newcomers who have never used IRC or lists before. Your help and patience is welcome to provide a helping hand! Thanks to all mentors who have already registered & provided tasks! You have not become a mentor yet? Please consider it! You get some smaller tasks fixed and it is fun! * Think of easy tasks in your area that you could mentor. Areas are: Code, docs/training, outreach/research, quality assurance, and user interface. "Easy" means 2-3h to complete for you, or less technical ~30min "beginner tasks" for onboarding). * OR: provide an easy 'clonable' task (a task that is generic and could be repeated many times by different students). * Note that you commit to answer to students' questions and to evaluate their work within 36 hours (but the better your task description the less questions. No worries, we're here to help!) For the full info, please check out https://www.mediawiki.org/wiki/Google_Code-in/Mentors and ask if something is unclear. Thank you again for giving young contributors the opportunity to learn about and work on all aspects of Free & Open Source Software projects. Now bring your tasks! Cheers, andre [1] https://lists.wikimedia.org/pipermail/wikitech-l/2017-October/088956.html -- Andre Klapper | Wikimedia Bugwrangler http://blogs.gnome.org/aklapper/

1 0

Best practices for WebAssembly and compiled JS in front-end modules?
by Brion Vibber 27 Nov '17

27 Nov '17

The latest version of all major browsers support C/C++/etc code compiled to WebAssembly <http://webassembly.org/> ("wasm") format, in addition to the older "asm.js" style of compilation targeting a subset of JavaScript directly. As far as I know the first library that MediaWiki uses to support WebAssembly is ogv.js, my codec & playback library which we use for playing media on Safari, IE, and Edge. ogv.js is packaged with the TimedMediaHandler extension, and we've used its asm.js mode for about two years now. I'd like to enable the WebAssembly mode in production for the faster compilation times, but want to double-check that we've established some best practices for WebAssembly usage first. https://phabricator.wikimedia.org/T181451 -- I've started up a provisional RFC, currently more of a document with a few pain points called out: * packaging (check in binaries from a library, or use a package manager like npm?) * loading (loading .wasm or .js files as assets, vs what can use RL) * security (where to concentrate for security reviews, etc) This may grow into a recommendation on how to package & load things and some tweaks to RL. If anybody has any questions about the scariness that is wasm, give a shout. I want to make it not-too-scary. :) -- brion

1 0

Re: [Wikitech-l] [AI] New ORES FAQ
by Gergo Tisza 22 Nov '17

22 Nov '17

Hi Sarah, thanks for working on this! On Tue, Nov 21, 2017 at 2:31 PM, Sarah R <srodlund(a)wikimedia.org> wrote: > Documentation like this is critical for our open software environment to > work. I'm organizing a special interest group[4] to address > documentation needs. If you have any questions or comments, please dont > hesitate to reach out and share! > [4] https://wikitech.wikimedia.org/wiki/SIGDOCS > Is there any particular reason to do it on wikitech? Most of the other SIGs [1] and most of our documentation is on mediawiki.org (which is in general a better maintained place). The ORES docs live there too, apart from a few pages that are concerned with operations issues. [1] https://www.mediawiki.org/wiki/Special_Interest_Groups

3 4

2017-11-22 Scrum of Scrums meeting notes
by Grace Gellerman 22 Nov '17

22 Nov '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-11-22 = 2017-11-22 = == Callouts == * If you use and process Parsoid HTML, please test your code with Parsoid master (merged patches for section tag addition + use of figure-inline tag) * It looks like Remex got turned off on the cluster on Oct 26 as part of https://phabricator.wikimedia.org/T178632 .. I didn't know about this till last week (because of some gaps in Phab notifications). When you switch off a feature in production, please make sure the responsible folks know about it. :-) * [Performance] Seeing some performance regressions being reported under Firefox 57; working with Mozilla to understand whether these are real or if they're due to a bug in FF57 NavTiming. * node service-runner provides node GC metrics to statsd from v2.4.3 so all node services can have it out of the box * Biggest fundraising campaign of the year starts Tuesday - please double-check with fr-tech if you're about to do anything that might affect CentralNotice or donatewiki. == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: still working on 5.8 https://phabricator.wikimedia.org/project/view/2913/ ==== Android native app ==== * Blocked by: * Blocking: * Updates: ==== Reading Web ==== * Blocked by: * Blocking: * Updates: ** A print button was added to the mobile site ** We made progress with working out how to deploy Chromium inside puppeteer and are continuing to work with services to get the pdf render service up and running ==== Reading Infrastructure ==== * Blocked by: * Blocking: * Updates: ** working on Reading Lists performance improvements, hoping to deploy next week ** work continues on summary and media endpoints ==== Multimedia ==== * Blocked by: * Blocking: * Updates: WMDE is moving on some work for MediaInfo, in the meantime Multimedia is focusing on 3D, which should be ready to move out in early December. ==== Discovery ==== * Blocked by: * Blocking: * Updates: ** Deployed SVGs on www.wikipedia.org portal ===== Maps ===== * Blocked by: None * Blocking: None * Updates: ** Paul attended training at PGConf Seattle and intends to put together a learning session ** Ops person out this week === Community Tech === * Blocked by: * Blocking: * Updates: ** we were mostly conducting our survey. === Contributors === ==== Editing ==== * Blocked by: * Blocking: * Updates: ==== Parsing ==== * Blocked by: * Blocking: * Updates: ** Next big Parsoid deploy *** section-tag and figure-inline-tag patches merged; YET to bump HTML version number to 1.6.0 *** html5 ids patch in WIP; depending on how that goes, we may bundle that with the next deploy with a HTML version bump from 1.5.0 -> 1.6.0; *** If you use and process Parsoid HTML, please test your code with Parsoid master *** Parsoid should be backward compatible for the html -> wikitext direction; we should be able to process v1.5.0 or the newer HTML (or some combination of the two as well :)). Please report bugs if you see any problems. *** If testing early next week goes okay and we get a green from everyone, we will deploy next Thursday. If not, will push to the following week ** Tidy replacement *** It looks like Remex got turned off on the cluster on Oct 26 as part of https://phabricator.wikimedia.org/T178632 -- for whatever reason, Phab didn't notify me when James added me to the ticket, and Tim may not have known either. I've added both of us to the RemexHTML project .. so hopefully will receive project notifications in the future. But, when you switch off a feature in production, please make sure the responsible folks know about it. :-) *** Tim has fixed the problem and the fix will go out next week. ** Linter *** Over the last week, I finished running a script to re-generate linter issues for all pages on all (active) wikis. This should hopefully have initialized linter issues for all wikis (and removed false positives based on Parsoid fixes). ==== Global Collaboration ==== * Blocked by: nobody * Blocking: ops on dumps, Roan STILL hasn't talked to Ariel, apologies again, talking to Ariel as we speak * Updates: ** Translate extension's RC filters are now converted to the new system, that means almost all WMF wikis are fully converted except super edge cases like incubator ** Live update on RC will probably be deployed next week, stay tuned for announcement ==== Contributors Design ==== * Blocked by: * Blocking: * Updates: == Technology == === Analytics === * Blocked by: * Blocking: * Updates: === Cloud Services === * Blocked by: * Blocking: * Updates: ** Annual Toolforge survey kicked off on Monday; will run for 2 weeks -- https://phabricator.wikimedia.org/T177126 ** After much load testing we think Labvirt1015 is fit for duty. ** Andrew back from vacation ** Cloud VPS instances were switched to use Puppet's future parser on Monday === MediaWiki Platform === * Blocked by: * Blocking: * Updates: === Performance === * Blocked by: * Blocking: * Updates: ** Seeing some performance regressions being reported under Firefox 57; working with Mozilla to understand whether these are real or if they're due to a bug in FF57 NavTiming. === Release Engineering === * Blocked by: ** ops: [https://phabricator.wikimedia.org/T180524 Upgrade latest docker-registry.wikimedia.org/nodejs-devel to stretch] * Blocking: * Updates: * [Team vacations] Mukunda is out all week. Dan is only working on Monday. And of course Thurs/Fri for all US staff. * [MediaWiki Train] Catching up after last week’s delay due to the wikidata database crash. No MW train this week. Played catchup yesterday. All on wmf.8 * [Offsite] More team offsite logistics. Will be doing some cross team gathering with Reading (who we’re sharing a hotel/couple days overlap with). * A lot of Gerrit repository and Github mirror administration this past week. * [Phabricator] Caused and fixed a degradation of Phabricator’s search functionality last Wednesday night/Thursday morning (15th and 16th). We apologize if you were affected by this. [wiki] [email] === Research === * Blocked by: * Blocking: * Updates: === Scoring Platform === * Blocked by: * Blocking: * Updates: ** https://mediawiki.org/wiki/ORES/FAQ & https://wikitech.wikimedia.org/wiki/SIGDOCS ** draft_quality data coming to English Wikipedia, doing small DB key things. ** wp10 (article quality) data is going to be available for English Wikipedia ** Nov 20 outage on ruwiki and frwiki, Special:RecentChanges and Special:W atchList caused by ORES. We're drafting an incident report by the end of the day. ** We'd love to work with Global Collaboration to improve robustness on the pages that rely on ORES, to provide graceful degradation. ** Perhaps Global Collaboration wants to share QA resources? *** Roan: Happy to, on both. === Search Platform === * Blocked by: none * Blocking: none * Updates: ** Working on improvements to LTR training https://phabricator.wikimedia.org/T177520 ** Improving completion suggester interaction with namespaced prefix search https://phabricator.wikimedia.org/T178474 ** Preparing for next relevancy survey https://phabricator.wikimedia.org/T176428 ** Reviewing Serbian morphological libraries https://phabricator.wikimedia.org/T178926 ** Working on porting Selenium tests from Ruby to JS ** Working on upgrade to Elastic 5.5 - search clusters upgraded, logstash next ** Working on indexing Wikidata descriptions and adding them to fulltext search === Security === * Blocked by: * Blocking: * Updates: ** Reviews: *** mediawiki-services-chromium-render *** Ex:WikipediaExtracts === Services === * Blocked by: none * Blocking: none * Updates: ** Continuing to reshape cassandra cluster and moving more nodes to Cassandra 3 cluster ** service-runner provides node GC metric now to statsd *** After a deploy you will be able to make a GC graph this for your service *** https://grafana.wikimedia.org/dashboard/db/jobqueue-eventbus?panelId=13&ful… === Technical Operations === * Blocked by: * Blocking: * Updates: == Wikidata == * We are aware of review requests for description indexing for Search, as well as the request by the Structured Commons team. * RDF exports support foreign repository now, needed for Structured Data on Commons: https://phabricator.wikimedia.org/T161592 * Worked on making statements on Forms editable and persistent: https://phabricator.wikimedia.org/T163724 * Worked on fine-grained usage tracking in Wikibase' Lua library: https://phabricator.wikimedia.org/T172914 * Worked on caching constraint check results: https://phabricator.wikimedia.org/T179849 == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ==== Fundraising Tech ==== * CentralNotice: ** Aware that the Selenium tests are still on Ruby, would like to put off the nodejs update till January. ** Deployed a little bit of debugging code for the large banner switch campaign mixin * Payments forms: deployed little style fixes to buttons, putting off deploying validation changes till January * CiviCRM: contact de-duplication improvements for large dollar amount donors, de-duplicating organization records * More refinements to stats reported to Prometheus

1 0

Help wanted: "Web API" vs. "Action API" vs. "REST API" in docs on mediawiki.org
by Andre Klapper 22 Nov '17

22 Nov '17

Hi, I'm looking for someone who has ~15 minutes to check (and potentially update) the ten mediawiki.org pages listed in https://phabricator.wikimedia.org/T173815#3673839 I'd like to see the term "web API" replaced by either "action API" or "REST API" to clarify for developers which API those pages refer to. So if you feel knowledgeable enough in this area: Your help on that Phabricator task is welcome! Thanks in advance, andre -- Andre Klapper | Wikimedia Bugwrangler http://blogs.gnome.org/aklapper/

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l November 2017