Wikitech-l November 2016

wikitech-l@lists.wikimedia.org

94 participants
99 discussions

Triaging Summit proposals for "How to grow our technical community"
by Quim Gil 23 Nov '16

23 Nov '16

The Wikimedia developer Summit main topic "How to grow our technical community <https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/How_to_grow_…>" has nine proposals aiming to be accepted as pre-scheduled sessions. Mathematically speaking, it is very unlikely that we will have slots for all of them. Please help making a good selection by expressing your interest in your preferred candidates, by commenting/subscribing to your preferred tasks, or awarding them a token. Pre-scheduled sessions are supposed to be preceded by active discussions, so the best way to support your preferred topics is by joining the discussion now. Proposals: - Actions to grow the diversity of our technical community <https://phabricator.wikimedia.org/T148598> - Drawing in new MW tech volunteers with Hackathons, and then what <https://phabricator.wikimedia.org/T148557> - Better recommending of tasks suitable for new technical contributors <https://phabricator.wikimedia.org/T149564> - Better code review processes for reviewers <https://phabricator.wikimedia.org/T149639> - IdeaLab for Wikimedia Foundation technical projects <https://phabricator.wikimedia.org/T149948> - How to unleash the power of JavaScript <https://phabricator.wikimedia.org/T148911> - MediaWiki Documentation <https://phabricator.wikimedia.org/T149372> - How to unleash the power of 3rd party extension developers <https://phabricator.wikimedia.org/T147892> - Bringing "Enterprise MediaWiki" to the Wikimedia Foundation <https://phabricator.wikimedia.org/T149612> -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

1 0

About the frontend development tools we use
by Niklas Laxström 23 Nov '16

23 Nov '16

I was reading http://stateofjs.com/2016/introduction/#sections and could not avoid noticing that the frameworks or technologies we use are not among the most popular or most liked among the participants of this survey. Examples: * Frontend frameworks: We use jQuery and OOjs UI. The latter does not appear at all in the list, jQuery is not in the top ten. This question might be biased though on what people perceive as a framework. * Testing framework: We mostly use qUnit, Cucumber, Selenium. Of these only Cucumber appears in the top 6 and it has very low satisfaction (people who have used do not like it). * CSS tools: We use plain CSS and Less. Less has considerable lower satisfaction than SASS/SCSS and is less popular. * Build tools: We don't use these in core to my knowledge, but many extensions seem to use Grunt for running linting tools. Again, Grunt has very low satisfaction compared to other tools. It is natural, that as large and complex project we do not jump to the latest cool thing. I am not advocating to change tools that work well for us, but I don't remember seeing a public discussion whether they work well or not. Though, I am seeing some changes, for example jscs+jshint being replaced with eslint. We could possibly go faster or write better software with better tools (of course this would need a careful evaluation). And while doing that we could perhaps lower the barrier for new developers by using something they already know. The topic of how to attract new developers to our movement has been popular lately (for example [1]). [1] https://phabricator.wikimedia.org/T148911 ---- For me one pain point is automated testing of JavaScript code. It seems that testing frameworks, development practices and the way code is written could all be improved to make automated testing easier. Would there be interest in sharing comments how you do this and does what you do work well for you? -Niklas

3 2

2016-11-23 Scrum of Scrums meeting notes
by Grace Gellerman 23 Nov '16

23 Nov '16

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-11-23 = 2016-11-23 = == Technology == === Analytics === * Blockers: none * on track for quaterly goals * main project about edit data (mediawiki edit history reconstruction) progressing, * we are now calculating standard edit metrics for all wikis since the beginning of time using denormalized edit history: https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Denormalized_edit_h… experimental dataset on pivot: https://goo.gl/XAMVsh * working on productionizing infrasructure for event streams * waiting for hardware for pageview API * owning now statsv together with ops (utility that can consume kafka data and report to graphana) * Thanking discovery for contributing to our metric reporting tools Upcoming: Start design work to revamp information architecture of http://stats.wikimedia.org === Performance === Not blocking, not blocked * thanks everyone who attended the active/active DC meeting after I flagged it here, it has helped getting the ball rolling on two blockers * hidden tabs confirmed as messing with timing data, now excluded from perf metrics * investigating little-known legacy features in mediawiki thumbnailing to decide whether we continue supporting them on Thumbor (302 redirects) * second view tests added for firefox and IE in WebPageTest (was previously only looking at Chrome) * still active on thumbnail URL/API RFC discussion * briefly discussed witth multimedia team setting up Thumbor for them to leverage in their ImageTweaks extension === Security === * Security Reviews * Linter review complete * LoginNotify schedule for this week * Continuing work on wiki account compromise remediation (T150554) * Assistance needed -- e-mail to engineering@ is forthcoming with request === Services === * Blockers: none * Updates: ** PDF render service deployed in codfw, eqiad and public exposure next week ** New version of service-template-node: ES6 and ESLint are coming === Technical Operations === * '''Blockers''' ** IOS native app *** Requesting timeline for Wikipedia iOS app requesting 0px thumbs: https://phabricator.wikimedia.org/T147648 https://phabricator.wikimedia.org/T151078 iOS 5.3.0 was shipped last week ** Performance ? *** MW fix to return 400 on 0px thumbs https://phabricator.wikimedia.org/T147784 * '''Blocking''' ** None * Updates ** jobqueue woes https://phabricator.wikimedia.org/T151196 ** kubernetes/calico work ongoing, goal on track ** dropping varnish 3 compatibility code from our puppet repos ** labsdb goals on track as well === Release Engineering === * Blocking ** * Blocked ** * Updates ** Mediawiki 1.28 tarball release this week! == Product == === Reading === ==== Mobile Content Service (MCS) ==== * Board: https://phabricator.wikimedia.org/tag/mobile-content-service/ * Added announcements feed endpoint (public now). More info and request URL at https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/RESTBase_services_for_ap… : ==== Android native app ==== * Last week: ** Continuing Q2 goals for Wikidata descriptions ** New fundraising announcement explore feed card in progress ** Now building against Android Nougat 7.1 API 25 ** Fixing login and editing issues ** Lots of unit tests * Next week (https://phabricator.wikimedia.org/project/view/2352/): ** More Q2 goals for Wikidata descriptions (tutorial and polish) ==== iOS Native App ==== * Last Week: ** Shipped 5.3.0 (In the news notifications & feed content, MCS backed feed, language variant support, other bug fixes and enhancements) https://phabricator.wikimedia.org/project/view/2220/ ** Added announcement card to the feed (for user research and fundraising) ** Started update of data layer to fix issue with data access & modification from widgets and notifications ** Started dynamic font size updates to the app * This week: https://phabricator.wikimedia.org/project/view/2357/ ** Finishing data layer update ** Continuing dynamic font size updates ** Other minor bug fixes for 5.3.1 ==== Web ==== * Current sprint: https://phabricator.wikimedia.org/tag/reading-web-sprint-86-%F0%9F%94%AA%F0… ** Stopping HoverCards A/B tests from Russian and Italian wikis ** New readers work ** Make PageImages return the image in the lead section ** MobileFrontend tech debt ** Trending service ** Hovercards rewrite * Next week: probably the same stuff as the current week. ==== Reading Infrastructure ==== * not blocking * Still looking for reviews on the API error/warning i18n patches ** https://gerrit.wikimedia.org/r/#/c/321402/ - Improve handling of Message objects as Message parameters ** https://gerrit.wikimedia.org/r/#/c/321404/ - Add Message::listParam() ** https://gerrit.wikimedia.org/r/#/c/321405/ - Fix MediaTransformError message handling ** *https://gerrit.wikimedia.org/r/#/c/321406/* <https://gerrit.wikimedia.org/r/#/c/321406/> - API: i18n for warnings and errors ** The extension patches depended on by that change are next in importance. These are for OAuth, TitleBlacklist, GlobalBlocking, Translate, and ConfirmEdit. ** All other WMF-deployed extensions affected by this change have patches too, see *https://gerrit.wikimedia.org/r/#/q/topic:api-error-i18n/T47843* <https://gerrit.wikimedia.org/r/#/q/topic:api-error-i18n/T47843>. Non-WMF-deployed extensions are (mostly) not touched at this time, the worst that should happen to them is wfDeprecated warnings eventually. * https://gerrit.wikimedia.org/r/#/c/312865/ is blocked on review by Security === Community Tech === * Not blocking * Blocker: Need a security review for https://phabricator.wikimedia.org/T150832 to proceed with exposing a couple of table views on tool labs db * Community Wishlist survey proposal phase over. https://meta.wikimedia.org/wiki/2016_Community_Wishlist_Survey * Did bug-fixing for Copypatrol (plagiarism detection tool) and launched French version: https://tools.wmflabs.org/copypatrol/fr * Ongoing RfC about changing default collation on Meta: https://meta.wikimedia.org/wiki/Requests_for_comment/Switch_default_categor… * Ongoing RfC about abandoned labs tools takeover: https://meta.wikimedia.org/wiki/Requests_for_comment/Abandoned_Labs_tools * Ongoing work with programs dashboard === Discovery === * BM25 scoring enabled on 10 larges wikis * Discovery mission & roadmap presentation: https://docs.google.com/presentation/d/1ctlqdLA__0OxDuO7mJEIDLP-xt9a7E4jv4I… * Load-testing crosswiki searching backend code * Portal updates per-language article count stats, dewiki joins the lucrative 2M+ club :) === Editing === ==== Language ==== * Blocked: T150512: WikiBase Repo tests failing with UsageException ** This is making it difficult to merge Translate patches. Issue seems to be in database clearing in tests. QA/RelEng? ==== Collaboration ==== * Blocked: None * Blocking: None * Updates ** No deployments this week. Ongoing work on: *** Mobile support for left nav of Special:Notifications *** RecentChanges filters and filter framework for Edit Review Improvements === Fundraising Tech === * Big English fundraiser starts next week! ** repeating Greg's emailed plea: https://lists.wikimedia.org/pipermail/engineering/2016-November/000331.html ** Stuff that could impact the fundraiser: GeoIP, ResourceLoader, MessageCache, EventLogging, Hive webrequest tables * CentralNotice: reviewing Aaron Schultz's latest MessageCache patch: https://gerrit.wikimedia.org/r/#/c/318489 ** We want to understand it really well before we deploy anything that could affect banners ** If anyone with deep knowledge of MessageCache (Aaron, Gilles?) has time for a quick video chat, Andrew Green has a few questions ** As always, more scrutiny and comments are welcome. * Deployed Nirzar's mobile CSS fixes, looking great so far * Minor caching optimizations for CiviCRM jobs

1 0

Dev Summit 2017 session: DOM-based semantics for wikitext
by Subramanya Sastry 22 Nov '16

22 Nov '16

Hello everyone, I have proposed a 2017 Dev Summit session titled "Improved editability, tooling, reasoning, and performance by adopting DOM-based semantics for wikitext" [1]. The TL:DR; summary is to treat a wikitext article as made up of document fragments that are composed together instead of a concatenation of strings which may (but often does not) yield well-formed HTML markup. Rather than repeat what is already there on the task, I'll mention a couple of benefits if you are an editor: 1. If you are a template editor, and you edit a template, it would be much more efficient to update all affected articles. For the common use case, you would simply replace the old output in an article with the new output without having to reparse the article. 2. It is possible to think of edit tooling that lets you edit the document at much finer granularities than sections without stepping on each other's toes. There are more details in a draft note [2]. Please don't be alarmed by the "wikitext 2.0" term there. That is an umbrella term that we've been throwing around in the Parsing Team for a bunch of related but different proposals related to wikitext syntax and semantics. This phab task and the wiki page is one proposal that concerns itself primarily with semantics of wikitext markup and less about syntax. The only syntactic feature considered is the heredoc-style syntax for some transclusions [3] which would immensely benefit this approach. Please comment on the task with your thoughts, ideas, concerns, approval, criticism, or award tokens or subscribe to indicate interest. :-) Thanks, Subbu. [1] https://phabricator.wikimedia.org/T149282 [2] https://www.mediawiki.org/wiki/Parsing/Notes/Wikitext_2.0 [3] https://phabricator.wikimedia.org/T114432

1 0

Fwd: [Ops] mw1017, mw1099 are gone
by Greg Grossmeier 22 Nov '16

22 Nov '16

FYI to those who use these hosts during SWAT deploys. ----- Forwarded message from Giuseppe Lavagetto <glavagetto(a)wikimedia.org> ----- > Date: Tue, 22 Nov 2016 14:53:50 +0100 > From: Giuseppe Lavagetto <glavagetto(a)wikimedia.org> > To: Operations Engineers <ops(a)lists.wikimedia.org> > Subject: [Ops] mw1017, mw1099 are gone > > Hi all, > > due to an unfortunate chain of events, mw1017 and mw1099, the MediaWiki > test appservers in eqiad, had to be decommissioned this morning. Their > substitutes are mwdebug1001 and mwdebug1002 > > So, not to break the deployers workflow, what is happening is what follows: > > - when you select mw1017 from the browser extension, the request will go do > mwdebug1001 > - when you select mw1099, the request will go to mwdebug1002 > > the extensions will be updated as soon as possible (in a way that will > allow us an easy change of the server list without any extension release), > and I am in the process of upgrading the documentation on wikitech. > > I apologize for the inconvenience, > > Giuseppe > -- > Giuseppe Lavagetto, Ph.d. > Senior Technical Operations Engineer, Wikimedia Foundation > _______________________________________________ > Ops mailing list > Ops(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/ops ----- End forwarded message ----- -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

1 0

Bash API edit code examples needed, please
by Danny B. 22 Nov '16

22 Nov '16

Hi, a friend of mine is looking for examples of Bash code to edit MediaWiki via API (as far as I understood, login, logout, get page text, put text to the page (preferrably with custom tag); nothing more (unless needed)) for his project of sharing some open data. He has some other Bash scripts which generate some text which he needs to save on wiki. Such text can be provided to the API edit part either via pipe, as a variable content or saved in file (that's the preferred order of possibilities). If you have or know about any examples, please let me know (feel free to reply off-list with attachments), so I can forward it to him. Thanks for any help in advance. Kind regards Danny B.

2 1

Phabricator inbound email outage.
by Mukunda Modell 22 Nov '16

22 Nov '16

Today it came to my attention, via https://phabricator.wikimedia.org/T151229, that Phabricator's inbound email has been failing silently for several days. Since no error messages were being delivered it easily went unnoticed over the weekend. I just fixed the root cause, however, an unknown number of replies to Phabricator tasks have gone into the void without being recorded on the intended tasks. If you regularly interact with phabricator via email, I would like to ask you to check your sent-mail to be sure your comments made it into phabricator. You will likely need to re-send any messages that occurred since at least Friday the 18th. I apologize for the inconvenience. As a user of Phabricator's built-in notifications I never even notice the email functionality, and as a result, it went untested for too long. Regretfully, Mukunda Modell Maintainer: Wikimedia Phabricator

2 2

Invitation to review: Design Statement of Purpose
by Keegan Peterzell 21 Nov '16

21 Nov '16

Hello all, Over the past few months the Design team members at the Wikimedia Foundation (user experience [UX] designers, design researchers, user experience engineers, and communications) have been working with Arthur Richards from the Team Practices Group to identify the high-level themes that motivate design at the WMF. These themes have been turned into a brief statement of purpose, whose intent is to articulate the vision and purpose behind design at the WMF. This statement will influence the future direction of design work. At this point the stakeholders are ready for a review of the draft statement. The purpose of this review is to gather a common understanding of its purpose, and to identify any key themes that may be missing from the high-level discussion. On the wiki page for the statement, you'll find these themes and what they encompass in the "Background" section. If you have an observation, comment, or concern about what is listed there, please bring it up on the talk page. If it is relevant to the review and understanding of the statement, it will be looked at for future drafts. If there are comments about design and the design process in general, we'll hold on to those until a time when they can be addressed for the broader discussion of design in general. All that said, here are the links: * <https://www.mediawiki.org/wiki/Design/Statement_of_purpose> * <https://www.mediawiki.org/wiki/Talk:Design/Statement_of_purpose> We look forward to seeing you on the wiki. -- Keegan Peterzell Technical Collaboration Specialist Wikimedia Foundation

2 1

[Announcement] Search improvements for file properties
by Chris Koerner 21 Nov '16

21 Nov '16

Hello, You can now search for file properties such as file size and and file type on Commons. This includes file media type, MIME type, size, width, height, resolution, and bit depth. A few quick examples: * A search for 'shark' videos [0] * A search for 'flower' where the files are 16 bit [1] * A search for 'stars' where the files are PDFs [2] Please see the documentation for more information. [3] [0] https://commons.wikimedia.org/w/index.php?title=Special:Search&profile=defa… [1] https://commons.wikimedia.org/w/index.php?title=Special:Search&profile=defa… [2] https://commons.wikimedia.org/w/index.php?title=Special:Search&profile=defa… [3] https://www.mediawiki.org/wiki/Help:CirrusSearch#File_properties_search Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation

4 3

Update on WMF account compromises
by Tim Starling 21 Nov '16

21 Nov '16

Since Friday, we've had a slow but steady stream of admin account compromises on WMF projects. The hacker group OurMine has taken credit for these compromises. We're fairly sure now that their mode of operation involves searching for target admins in previous user/password dumps published by other hackers, such as the 2013 Adobe hack. They're not doing an online brute force attack against WMF. For each target, they try one or two passwords, and if those don't work, they go on to the next target. Their success rate is maybe 10%. When they compromise an account, they usually do a main page defacement or similar, get blocked, and then move on to the next target. Today, they compromised the account of a www.mediawiki.org admin, did a main page defacement there, and then (presumably) used the same password to log in to Gerrit. They took a screenshot, sent it to us, but took no other action. So, I don't think they are truly malicious -- I think they are doing it for fun, fame, perhaps also for their stated goal of bringing attention to poor password security. Indications are that they are familiarising themselves with MediaWiki and with our community. They probably plan on continuing to do this for some time. We're doing what we can to slow them down, but admins and other users with privileged access also need to take some responsibility for the security of their accounts. Specifically: * If you're an admin, please enable two-factor authentication. <https://meta.wikimedia.org/wiki/H:2FA> * Please change your password, if you haven't already changed it in the last week. Use a new password that is not used on any other site. * Please do not share passwords across different WMF services, for example, between the wikis and Gerrit. (Cross-posted to wikitech-l and wikimedia-l, please copy/link elsewhere as appropriate.) -- Tim Starling

26 37

← Newer
1
2
3
4
5
6
7
8
9
10
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l November 2016