Wikitech-l May 2017

wikitech-l@lists.wikimedia.org

82 participants
74 discussions

RFC discussion today: how to store long(er) revision comments
by Daniel Kinzler 03 May '17

03 May '17

This is a reminder of today's RFC discussion about how to store longer revision comments in the database: <https://phabricator.wikimedia.org/T153333> One of the main discussion points is whether the necessary schema change should be made as part of a larger refactoring of the revision table, or whether it should be done in an isolated way, so it is not blocked on other projects. Another point is if and how deduplication should be applied. The discussion will take place on the #wikimedia-office channel on Freenode, at 21:00 UTC (2pm PDT, 23:00 CEST). -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

2017-05-03 Scrum of Scrums meeting notes
by Grace Gellerman 03 May '17

03 May '17

*https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-05-03 <https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-05-03>* *= 2017-05-03 =* contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering == Call outs:== * Reminder: no train week of May 15th * Ryan Kaldari is requesting code review of https://gerrit.wikimedia.org/r/#/c/343878/ especially from Editing team * https://gerrit.wikimedia.org/r/#/c/349432/ is waiting on security review === Reading === ==== iOS ==== * Last Week ** Final bug fixes for 5.4.1 - https://phabricator.wikimedia.org/project/view/2600/ *** Another round of public betas, regression testing (after crash fixes) ** 5.5 - https://phabricator.wikimedia.org/project/view/2602/ *** Places changes from user testing feedback *** Tweaks to article footer content rendered in HTML/CSS rather than native views * This Week ** Submit 5.4.1 to the App Store (really this time) ** Continue work on 5.5 *** Another round of places user testing *** TWN localization pluralization support ==== Android ==== * Discussing reading list syncing technical requirements * We plan to enable user opt-in OS-level backups of the app database contents * Continuing work on cross-platform consolidation of CSS & JS * Current release board: https://phabricator.wikimedia.org/project/view/2352/ ==== Reading Infrastructure ==== * ORES: fixes for abuse and DB size issues with api.php, will reenable with next train * Developing spec for new HTML markup to share with Parsoid team https://phabricator.wikimedia.org/T164032 ==== Multimedia ==== * 3D out on beta - will conduct some analysis over the next few weeks to determine rollout plan. * Media Viewer work - esp. focused on mobile support - to come in the next few weeks also. ==== Web ==== Highlights for next 2 weeks: * Deploying print styles for mobile view * Related pages rolled out to remaining 10% * We will move lead paragraph above infoboxes * Page previews to be rolled out to all wikipedias except DE and EN * Tooling and bug fixes === Community Tech ==== * CodeMirror - needs design review, integration with VE ongoing, ** Ryan Kaldari is requesting code review of https://gerrit.wikimedia.org/r/#/c/343878/ especially from Editing team * LoginNotify - will be deployed to Test Wikipedia next week * Popular page reports - working on improving speed * XTools rewrite - in progress === Editing === ==== Parsing ==== * https://gerrit.wikimedia.org/r/#/c/349432/ is waiting on security review * A bunch of bug fixing ongoing wrt link handling in Parsoid * Redlink support in Parsoid is in progress * Language variant support in Parsoid is in progress * Linter fixes ongoing -- hope to re-enable on large wikis in the coming week or two. ==== UI Standardization ==== * This week: ** Work on bringing automatic Sketch export to Wikimedia UI Style Guide to always be up-to-date with design changes ** Work on improving OOjs UI library's CSS and debugging by further widely reducing specificity ** Preparation on * Updates: ** OOjs UI: *** UI and accessibilit improvements on Tag*Widgets ** Mobile Frontend/core: Consolidation of Less mixins ==== Language ==== * No blockers. * Work on OOjs UI migration continue. ==== Collaboration ==== * Continued work on RC Filters, including saved filters and the guided tour. * Improving RC Filters documentation and fixing a couple Doxygen issues along the way. * Working on the user experience level code coverage issue. * Had to make a couple more FlaggedRevs fixes. === German Technical Wishlist === * Working on FileExporter/Importer extensions for moving files to Commons. * Teaching diffs to understand when a paragraph was moved. * Investigating RevisionSlider touch support: https://phabricator.wikimedia.org/T164249 * Minor fixes on mention notifications: https://phabricator.wikimedia.org/T164115 === Wikidata === * Echo notifications about page being connected/disconnected via Wikidata will go live soon: https://phabricator.wikimedia.org/T142102 * Making constraint checks more visible via a new API and gadget. * Lexeme UI for Wiktionary gets more complete: http://wikidata-lexeme.wmflabs.org/index.php/Lexeme:L2 * Starting to mix OOJS UI widgets into our jQuery UI frontend: https://phabricator.wikimedia.org/T163971 === Discovery === * No blockers * Enabled template boosting on media searches * Building infrastructure for machine learning assisted ranking (aka MjoLniR) * Chinese analyzer seems to be doing well, deploying soon. Working on Hebrew analyzer. * Working on Wikidata search improvement * Working on Mediawiki API integration for WDQS === Analytics === * Ongoing: EventLogging to Hive refinement * Detailing architectural design of Wikistats2.0 backend * Ongoing: Better bot detection in EventLogging * Ongoing: Improvements to Banner Activity Pivot data set: efficiency and sanitization * Waiting for the deployment freeze to end, to enable annotations in the reportcard === Security === * Reviews ** TemplateStyles ** https://gerrit.wikimedia.org/r/#/c/349432/ === Services === * Blockers: none * Updates: ** Deprecation of citoid.wikimedia.org domain ** New properties in the summary endpoint ** Removing deprecated RESTBase endpoints *** https://phabricator.wikimedia.org/T158100 *** https://phabricator.wikimedia.org/T158128 === RelEng=== * '''Blocked''' ** None * '''Blocking''' ** None? * '''Updates''' ** Week of May 15th is our team offsite, no train ** 1.29 is branched === Technical Operations === * '''Blocked''' ** None * '''Blocking''' ** No-one * '''Updates''' ** Datacenter switchover (back to eqiad) week === Fundraising Tech === * Getting new hire Maggie up and coding * PayPal Express Checkout fixes * CiviCRM: dedupe fixes, 3rd party mailer integration * Ingenico re-build / library-ization planning * More rounds of review for CentralNotice banner sequence

1 0

CREDIT showcase, Wednesday, 3-May-2017
by Lani Goto 03 May '17

03 May '17

Hi! I'd like to welcome you to join us at the CREDIT showcase next week, Wednesday, 3-May-2017 at 1800 UTC / 1100 Pacific Time. We'd like to see your demos, whether they're rough works in progress or polished production material, or even just a telling of something you've been studying recently. For more information on the upcoming event, as well as recordings of previous events, please visit the following page: https://www.mediawiki.org/wiki/CREDIT_showcase And if you'd like to share the news about the upcoming CREDIT showcase, here's some suggested verbiage. Thanks! *Hi <FNAME>* *I hope all is well with you! I wanted to let you know about CREDIT, a monthly demo series that we’re running to showcase open source tech projects from Wikimedia’s Community, Reading, Editing, Discovery, Infrastructure and Technology teams. * *CREDIT is open to the public, and we welcome questions and discussion. The next CREDIT will be held on May 3rd at 11am PT / 2pm ET / 18:00 UTC. * *There’s more info on MediaWiki <https://www.mediawiki.org/wiki/CREDIT_showcase>, and on Etherpad <https://etherpad.wikimedia.org/p/CREDIT>, which is where we take notes and ask questions. You can also ask questions on IRC in the Freenode chatroom #wikimedia-office (web-based access here <https://webchat.freenode.net/?channels=%23wikimedia-office>). Links to video will become available at these locations shortly before the event.* *Please feel free to pass this information along to any interested folks. Our projects tend to focus on areas that might be of interest to folks working across the open source tech community: language detection, numerical sort, large data visualizations, maps, and all sorts of other things.* *If you have any questions, please let me know! Thanks, and I hope to see you at CREDIT.* *YOURNAME* -- Lani Goto Project Assistant, Engineering Admin

1 1

Re: [Wikitech-l] [Mediawiki-api-announce] [ANNOUNCEMENT] REST API page metadata endpoints deprecation
by Marko Obrovac 03 May '17

03 May '17

Hello, This is just a friendly reminder that the page metadata end points mentioned below will be completely removed on 2017-05-08. Please update your clients if you are still using any of them. Cheers, Marko Obrovac, PhD Senior Services Engineer Wikimedia Foundation On 23 February 2017 at 18:28, Petr Pchelko <ppchelko(a)wikimedia.org> wrote: > Hello, > > Since the early days of REST API it provides two features that haven’t > been widely used neither internally in the WMF nor by the community. Today > in a clean-up pass over the API we have decided to deprecate and eventually > remove those features to allow some long-needed refactorings and stability > improvements of other, more important, endpoints. > > The first one is the ability to query metadata about the page via the > `/page/title/{title}`~[1] endpoint. The metadata includes properties like > the latest revision number of the page, user who have made the last edit, > whether the page is a redirect and similar. The backend storage model used > to power the feature is quite unique in the system and has a significant > maintenance cost without providing a clear benefit to users. > > Another feature that’s never found it’s audience is the ability to get > listings of revisions, titles and renders stored in RESTBase. These > listings suffer from scaling issues and cannot work reliably with the data > model we have. > > We have, hence, opted to remove these unused and complex endpoints until > there is some actual need for this data in the REST API when we can design > and implement them better. Here’s the list of endpoints that are now > deprecated and will be removed on May, 1st 2017: > > • /page/title/ > • /page/title/{title} > • /page/title/{title}/ > • /page/revision/ > • /page/revision/{revision} > > In case you are using them please switch to using the MediaWiki Action > API. In case you need assistance or have questions, feel free to reply to > this e-mail or contact the Wikimedia Services team~[2]. > > Best regards, > Petr Pchelko > Software Engineer > Wikimedia Foundation > > [1] https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_ > page_title_title > [2] https://www.mediawiki.org/wiki/Wikimedia_Services > > > _______________________________________________ > Mediawiki-api-announce mailing list > Mediawiki-api-announce(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce >

1 0

[ANNOUNCEMENT] REST API /page/mobile-text endpoint deprecation
by Marko Obrovac 03 May '17

03 May '17

Hello, The mobile-text endpoint~[1] was introduced in the REST API as an experiment in the early days of the Mobile-Content-Service~[2] to provide page content optimised for use in very lightweight mobile clients. The route was an experiment towards a new-generation lite app, initially targeted for low-powered, older Android devices. The idea was to use native Android UI components to show the page contents instead of in a WebView; alas, it was quickly surpassed by a new set of mobile-sections endpoints~[3]. The route is not maintained, there are no plans to use it within the WMF and according to our metrics the endpoint is not used by the community. Therefore, as part of the effort to consolidate the useful endpoints and focus on new features, we have decided to deprecate the route and remove it on May, 1st 2017. If you’re using the endpoint, please use the alternative mobile-sections endpoints~[3,4,5] instead. Should you have any questions or concerns, please reply to this e-mail or contact the Wikimedia Services Team~[6]. Cheers, Marko Obrovac, PhD Senior Services Engineer Wikimedia Foundation [1] https://en.wikipedia.org/api/rest_v1/#!/Mobile/get_page_mobile_text_title [2] https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/RESTBase_services_for_ap… [3] https://en.wikipedia.org/api/rest_v1/#!/Mobile/get_page_mobile_sections_tit… [4] https://en.wikipedia.org/api/rest_v1/#!/Mobile/get_page_mobile_sections_lea… [5] https://en.wikipedia.org/api/rest_v1/#!/Mobile/get_page_mobile_sections_rem… [6] https://www.mediawiki.org/wiki/Wikimedia_Services

1 1

get ready for MediaWiki Documentation Day on May 12th!
by Ryan Kaldari 03 May '17

03 May '17

Mark your calendars! Friday, May 12th has been designated MediaWiki Documentation Day. This is a day to put aside normal development work and focus on improving MediaWiki documentation. This includes documentation for users, wiki administrators, and developers. Anyone with a good understanding of any particular area of MediaWiki software (broadly construed to include gadgets, bots, libraries, API frameworks, etc.) is encouraged to add or update on-wiki documentation, in-code documentation, READMEs, etc. And even if you don't feel like you know enough to improve the documentation, you can also participate by flagging out-dated documentation or creating documentation requests on the wiki page. Coordination for Documentation Day is happening at: * https://www.mediawiki.org/wiki/MediaWiki_Documentation_Day_2017 * https://phabricator.wikimedia.org/T126500 For more details, please refer to the wiki page above. This is a completely ad hoc event, so feel free to adapt it, evolve it, expand it, hijack it, promote it, or ignore it. Cheers!

2 1

"must be of type int, int given"
by Denny Vrandečić 02 May '17

02 May '17

I'm running into a weird problem, which made me reset my whole vagrant. I assume this is not strictly a MediaWiki issue, but probably an HHVM problem, but maybe someone can help me here. So, if I start a fresh MediaWiki Vagrant installation, and the vagrant ssh into the virtual machine, the fastest way to reproduce the issue is to start hhvmsh and type function f():int { return 1; }; =f() I get: Hit fatal : Value returned from function f() must be of type int, int given #0 at [:1] #1 f(), called at [:1] #2 include(), called at [:1] Any way to circumvent this (besides getting rid of the type hint, obviously)? Update to a new HHVM version (According to Special:Version, the HHMV version is 3.12.14 (srv)?

6 15

Historical use of latin1 fields in MySQL
by Mark Clements (HappyDog) 02 May '17

02 May '17

Hi all, I seem to recall that a long, long time ago MediaWiki was using UTF-8 internally but storing the data in 'latin1' fields in MySQL. I notice that there is now the option to use either 'utf8' or 'binary' columns (via the $wgDBmysql5 setting), and the default appears to be 'binary'.[1] I've come across an old project which followed MediaWiki's lead (literally - it cites MediaWiki as the reason) and stores its UTF-8 data in latin1 tables. I need to upgrade it to a more modern data infrastructure, but I'm hesitant to simply switch to 'utf8' without understanding the reasons for this initial implementation decision. Can anyone confirm that MediaWiki used to behave in this manner, and if so why? If it was due to MySQL bugs, does anyone know in what version these were fixed? Finally, is current best-practice to use 'binary' or 'utf-8' for this? Why does MediaWiki make this configurable? I have a very good understanding of character encodings and have no problems with performing whatever migrations are necessary - and the code itself is fully utf-8 compliant except for the database layer - but I'm just trying to understand the design choices (or technical limitations) that resulted in MediaWiki handling character encodings in this manner. - Mark Clements (HappyDog) [1] https://www.mediawiki.org/wiki/Manual:$wgDBmysql5

8 8

Security announcement: XSS when Kartographer is used with JsonConfig
by Max Semenik 02 May '17

02 May '17

A stored XSS vulnerability was discovered when Kartographer is configured to receive map data from wiki pages via JsonConfig. Unless your wiki has both extensions installed and JsonConfig is configured to provide map data, it is safe. Otherwise, you're encouraged to upgrade both extensions IMMEDIATELY. Affected versions: * Versions for latest MediaWiki release, 1.28, don't support the aforementioned functionality and therefore are not vulnerable. * Versions for pre-release 1.29 and alpha 1.30 are affected and have fixes applied in source control. Upgrading: You can download latest sources from Git[1] or ExtensionDistributor[2] See this ticket for more information: https://phabricator.wikimedia.org/T163166 ---- [1] https://www.mediawiki.org/wiki/Download_from_Git#Using_Git_to_download_Medi… [2] https://www.mediawiki.org/wiki/Special:ExtensionDistributor -- Best regards, Max Semenik ([[User:MaxSem]])

1 0

Phabricator monthly statistics - 2017-04
by communitymetrics＠wikimedia.org 02 May '17

02 May '17

Hi Community Metrics team, This is your automatic monthly Phabricator statistics mail. Accounts created in (2017-04): 345 Active users (any activity) in (2017-04): 879 Task authors in (2017-04): 476 Users who have closed tasks in (2017-04): 274 Projects which had at least one task moved from one column to another on their workboard in (2017-04): 278 Tasks created in (2017-04): 2239 Tasks closed in (2017-04): 2391 Open and stalled tasks in total: 34180 Median age in days of open tasks by priority: Unbreak now: 22 Needs Triage: 271 High: 462 Normal: 659 Low: 932 Lowest: 868 (How long tasks have been open, not how long they have had that priority) TODO: Numbers which refer to closed tasks might not be correct, as described in https://phabricator.wikimedia.org/T1003 . Yours sincerely, Fab Rick Aytor (via community_metrics.sh on iridium at Mon May 1 00:00:17 UTC 2017)

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l May 2017