Wikitech-l July 2016

wikitech-l@lists.wikimedia.org

81 participants
83 discussions

Search: How should question marks be handled?
by Deborah Tankersley 07 Jul '16

07 Jul '16

Hello, The Wikimedia Foundation Discovery Search team <https://www.mediawiki.org/wiki/Wikimedia_Discovery/Search> has recently discovered <https://commons.wikimedia.org/wiki/File:From_Zero_to_Hero_-_Anticipating_Ze…> that search queries that end with a question mark (i.e. "*how old is Tom Cruise?*") can sometimes lead to zero (or unusable) results being returned. This zero result rate is one of the primary ways that the Search team determines how much our users are satisfied with their query results <http://discovery.wmflabs.org/metrics/#kpi_zero_results>. In order to improve the results that queries containing a questions mark, we'd like to change the behavior of the search on the backend. However, we would love to have feedback from the community to make sure that this is a smart change to do. If you are interested in how search works, or see this change as a possible disruption for your work, please learn more about this potential change <https://meta.wikimedia.org/wiki/Discovery/Handling_question_marks_in_search…> and let us know your thoughts. Cheers from the Discovery Search team! -- Deb Tankersley Product Manager, Discovery IRC: debt Wikimedia Foundation

1 0

CI outage - ongoing
by Chad Horohoe 07 Jul '16

07 Jul '16

Hi folks, Right now our CI infrastructure (Zuul/Jenkins/Nodepool) are having a bad day and aren't able to spawn new instances to perform tests. The outage is ongoing and there isn't an ETA for restoration of service just yet. In the meantime: please avoid force-merging (doing the Verified+2 check yourself) and skipping Jenkins unless you're dealing with an urgent production issue that must land today. Doing so makes Zuul get extra noisy which makes further diagnosis difficult. Thanks for your patience! -Chad & rest of RelEng

3 2

Crediting the work of our community members
by Jon Robson 07 Jul '16

07 Jul '16

"I came across a patch from a user who was keen to move himself from "Patch contributors" to "Developers" in the MediaWiki CREDITS file [1]. It had been sitting there for over a year. He doesn't seem to have been active since. I don't know what to do with it. It made me think. Do we have it documented anywhere how we use this credits file and why we feel the need to distinguish between Developers and Patch Contributors? It seems like a recipe for disaster in my opinion as it can only lead to hurt feelings due to contributors feeling unfairly treated. https://www.mediawiki.org/wiki/Special:Version/Credits leads with 'We would like to recognize the following persons for their contribution to MediaWiki." - if someone is not in that list are they not as important? If we keep these files we should probably explain the rules to what adding names looks like within these files and what the process to adding your name is (can I add myself? Is there a process like getting +2?) To take another extreme, we might consider abandoning such a file in favour of something automatically generated. Things like https://github.com/wikimedia/mediawiki/graphs/contributors do a far better job at allowing people to see who contributed to a tool and making people feel like their work is rewarded. On a slightly related note, can we abandon the practice of putting names inside files themselves? I see this practice in JavaScript and PHP files throughout core (grep for @author). As Team Geek [2] (great read btw) says "unlike other collaborative pieces of creative work... software keeps changing even after it's "done". So while listing contributors credits at the end of a movie is a safe and static thing, attempting to add and remove names from a source file is a never-ending exercise in insanity". For similar reasons this practice gives an impression of ownership of a file/code review responsibilities (which are not always true) and risks hurt feelings. [1] https://github.com/wikimedia/mediawiki/blob/master/CREDITS [2] http://www.amazon.com/gp/search?index=books&linkCode=qs&keywords=9781449302…

9 10

2016-07-06 Scrum of Scrums meeting notes
by Grace Gellerman 06 Jul '16

06 Jul '16

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-07-06 = 2016-07-06 = == Product == === Reading === ==== iOS ==== * 5.0.5 has been delayed for 2 more weeks due to new regressions. ** We are restarting the entire beta process because the 5.0.5 release diverged significan;ty from the trunk ** A new 5.0.5 with changes more recent bugfixes from the main trunk is being redeployed to beta this week ** 5.0.5 will re-enter regression next week * 5.1 features are being re-evaluated due to the long development time of 5.0.5 ** Some 5.1 features will likely be moved to 5.2 * Discovery is helping us improe the map UI first developed on Android by implementing some Geodata API enhancements ( https://phabricator.wikimedia.org/T139378) ==== Android ==== Finishing touches on the feed. ==== Mobile Content Service ==== * Experienced more than a dozen Nagios checker failures (timeouts) due to increased memory usage by ORES last weekend. Got fixed on Monday. https://phabricator.wikimedia.org/T139177 * Feed endpoint improvements: * redirect to summary endpoint for random and TFA if possible. https://phabricator.wikimedia.org/T139424 + https://phabricator.wikimedia.org/T136960 * adding date input validation for feed endpoints which accept dates Blocked on Services team: * We need the public feed endpoints (aggregated feed + random page) to enable the feed functionality for a beta Android app release. This is blocking the Android teams quarterly goal. ==== Reading Web ==== * Wikidata descriptions to mobile web stable in ca.wiki and pl.wiki * Deployed lazy references and images to mobile web tl.wiki * Fixing some bugs with hovercards to improve data analysis from A/B test * Started working in improvements to language switcher in mobile web ==== Reading Infrastructure ==== === Community Tech == * Deploying PageAssessments extension to test.wikipedia ( https://www.mediawiki.org/wiki/Extension:PageAssessments) * Finishing work on CopyPatrol Tool (https://tools.wmflabs.org/copypatrol) ** Replication lag on Tool Labs has been a major problem. We are migrating from local DB queries to API requests to avoid the problem. * Figuring the out the best implementation plan for switching to UCA collation + numerical sorting for categories * No blockers === Editing === ==== Multimedia ==== * https://phabricator.wikimedia.org/T138273#2410743 MatmaRex needs review from *anomie, tgr, and AaronSchulz*. * Still waiting for Thumbor deployment to carry ImageTweaks forward, Gilles says he's working on Debian packaging * Continuing work on FileAnnotations * Continuing work on getting rid of fatal errors in UploadWizard ==== Collaboration ==== * Blocking - No change * Blocked - None * Updates ** Deployed to group 0 new enhanced Echo bunding (with the ability to expand and collapse bundles of notifications and act on individual notifications), changed sorting of Alerts and Messages ** Deployed new version of Special:Notifications with per-page notification lists ==== Parsing ==== * Co-ordinating with Services to resume work on Parsoid migration to use service-runner utility. This blocks upgrade of parsoid production cluster to node 4.x (from 0.10) and deployment process to scap3 (from trebuchet) * Continuing to work on a bunch of bug fixes in Parsoid. * Work ongoing (Tim & Scott) to land a PHP HTML5 parser in core which can serve as a default Tidy replacement solution for mediawiki. == Wikidata == * Focus on refactoring the Sites mess in core, starting with: https://phabricator.wikimedia.org/T137537, tracked in: https://phabricator.wikimedia.org/T113034 * Core support for multilingual wikis: https://phabricator.wikimedia.org/T114640 * Ongoing communication about Multi Content Revision support in core: https://phabricator.wikimedia.org/T107595 == Technology == === Analytics === * Will start working on migrating refinery to SCAP 3 tomorrow (cc: release team) * Ongoing work on Event Bus, scaling AQS and the Pageview API, reconstructing mediawiki history, and loading Druid with pageview data === Research === * ores.wikimedia.org online (woo ops). ores.wmflabs.org will remain online and a deprecation plan will be announced "soon" * Major ORES downtime was exacerbated by icinga config failure (301 is a good response and not followed by default) https://wikitech.wikimedia.org/wiki/Incident_documentation/20160610-ORES * ORES extension deployed to fawiki and wikidatawiki. Large wikis coming soon. === Security === * Other security reviews are being schedule; if you have need, please submit your request: https://www.mediawiki.org/wiki/Wikimedia_Security_Team/Security_reviews * A security release will be prepared soon * Reviews: 3d2png (cont.), Tool Labs Console === Services === * out for two weeks - Wikimania + Offsite ** lots of interesting discussions * Auth / session storage service ** discussed the plan with Darian ** will draft an Arch RfC soon * Nodejs migration to 4.4.6 ** https://phabricator.wikimedia.org/T138561 ** scheduled for next wednesday ** test your services and update your package.json to use node v4.4.6 ** especially important for SCB services === Fundraising tech === * Building and deploying new payments servers to get off precise and php 5.3 * Also upgrading mediawiki and CiviCRM, planning how we'll more closely follow mw master * More work towards de-duplicating donor database * CentralNotice: don't load anything on Special: or action=edit * Still investigating what looks like missing log entries (delivered via Kafkatee): https://phabricator.wikimedia.org/T132500 === Discovery === * '''Blocking''': none * '''Blocked''': none * Hired new data analyst, she starts end of July * TextCat A/B finished, analyzing results * Working on enabling boxed search by coordinates with near: and neartitle: - https://gerrit.wikimedia.org/r/#/c/297524/ * Logstash elasticsearch upgrade planned for July 7th ( https://phabricator.wikimedia.org/T136001). This also includes upgrade to Kibana 4. * Portal stats updated on June 30th * Portal survey on why people come to Portal are published: https://phabricator.wikimedia.org/T136874#2418095 ==== Maps/Graphs ===== * Geoshapes service is integrated with WDQS and can provide both wikidata data and shapes: http://data.wmflabs.org/wiki/Regional_maps === RelEng === * Blocking: none? * Blocked: ** Target architecture without gallium (email followup) https://phabricator.wikimedia.org/T133300 * Updates ** Browser Testing survey still open https://goo.gl/xS6mmV (phab task for info: https://phabricator.wikimedia.org/T131123) ** CI reduced yesterday, should be good now ** Scap3 migration - new guide posted on wikitech https://wikitech.wikimedia.org/wiki/Scap3/Migration_Guide === Technical Operations === * '''Blocking''' ** None * '''Blocked''' ** By noone * Updates ** All image scalers are now on jessie with updated librsvg, fixes plenty of bugs (also related to changes to fonts while at it) ** Enabled TCP Fast Open on some caching clusters ** gitblit is finally dead ** Working with LE to get apertium packages reviewed/uploaded on apt.wikimedia.org. Some 20% done

1 0

Wednesday, 6-July CREDIT
by Adam Baso 06 Jul '16

06 Jul '16

Hi all, The next CREDIT showcase will be Wednesday, 6-July-2016 at 1800 UTC (1100 SF). https://www.mediawiki.org/wiki/CREDIT_showcase Please add your demos to the Etherpad. For this one we'll use the YouTube stream for viewers and Hangouts on Air for presenters. See you next week! -Adam

1 1

Commons read-only period
by Jaime Crespo 06 Jul '16

06 Jul '16

Hi, Yesterday, 5 July 2016 there was a period of read-only mode affecting mostly Commons wiki, but also to some deletions on other wikis, starting at 10:22 UTC. You have the details of the issue at: <url:https://wikitech.wikimedia.org/wiki/Incident_documentation/20160705-com…> Regards, -- Jaime Crespo <http://wikimedia.org>

1 0

Etherpad outage
by Jaime Crespo 06 Jul '16

06 Jul '16

Hi all, Etherpad [0], our real-time colaborative editing tool suffered an outage due to what we only know for now was database corruption. This was detected shortly after it happened 14:27 UTC and we (ops in charge of the service and the database) worked to reestablish the service. As the service continued crashing despite our efforts, we decided to recover a database backup from 2016-06-22 01:00:01 UTC. The service is now back up and working since 16:11 UTC, but that means that you may have lost a day and a half of edits in the current available etherpad [0]. I understand that that may cause a lot of inconveniences, specially for the people at Wikimania. *We are now trying to recover more than that*, but as the corruption could come back, or not all could be recovered, and people need the service the plan is the following: - Keep the current pads as is, not delete or add anything from now. You can continue using etherpad now as usual. - If possible, recover the last days of edits on a separate location. See [1] for progress if you are affected. Sorry for the inconveniences. Please, more than ever, follow the recommendation we added at the beginning of every empty pad: > "Keep in mind as well that there is no guarantee that a pad's contents will always be available. A pad may be corrupted, deleted or similar. Please keep a copy of important data somewhere else as well" The reason for this is that wiki content has proper HA and redundancy, etherpad does not. Again, my most sincere apologies, [0] <https://etherpad.wikimedia.org/> [1] <https://phabricator.wikimedia.org/T138516> -- Jaime Crespo <http://wikimedia.org>

3 4

2016W27 ArchCom-RFC meeting: programming languages in MediaWiki
by Rob Lanphier 06 Jul '16

06 Jul '16

Hi everyone, This week, here's the planned agenda for the ArchCom office hour: ArchCom-RFC office hour 2016W27: 2016-06-29: E226 (E66/42)[1] - T136866: Improve the per-programming-language listings for our tools[2] - Let's figure out how newcomers should identify programming languages used on the Wikimedia cluster that align with their interests/skills."What can I work on in language foo" or "I'm really great with language foo, bar, and baz; anything for me to help out with?" - Is the [[mw:Programming_languages]][3] page a good start? Is this the page we should easier to find? Are there better landing pages for someone trying to answer the questions above? The ArchCom status page[4] has these links and other information too (e.g. last week's decision to deprecate running MediaWiki without curl)[5] Rob [1]: https://phabricator.wikimedia.org/E226 [2]: https://phabricator.wikimedia.org/T136866 [3]: https://www.mediawiki.org/wiki/Programming_languages [4]: https://www.mediawiki.org/wiki/Architecture_committee/Status [5]: https://phabricator.wikimedia.org/T137926

1 0

The Revision Scoring weekly update
by Amir Ladsgroup 06 Jul '16

06 Jul '16

Hey, This is the 11th weekly update from revision scoring team that we have sent to this mailing list. *New developments:* - ORES review tool as a beta feature is enabled in Dutch Wikipedia. More wikis to come soon this week [1]. - We have basic edit quality model for Czech Wikipedia ready and merged. To be deployed this week [2]. - We also have basic models for English Wiktionary too. This is the second non-Wikipedia project we support after Wikidata [3]. - Thanks to Tar Lócesilion, we have Polish edit quality campaign completed, We are working on building damaging and goodfaith models at the moment [4]. *Maintenance and robustness:* - We decreased our web capacity in order to reduce memory pressure on scb nodes. You should not get any overload error since our capacity is still very high but if you do, please contact us immediately and we will bring it back up [5]. - We improved documentation on ores.wikimedia.org page a little bit. To be deployed this week [6]. We are working on a rather big refactor on ores which will give us performance boost on scoring multiple models at the same time [7] and reduce memory usage [8]. Feel free to chime in and give us feedback [9]. 1. https://phabricator.wikimedia.org/T139432 2. https://phabricator.wikimedia.org/T138885 3. https://phabricator.wikimedia.org/T138630 4. https://phabricator.wikimedia.org/T130269 5. https://phabricator.wikimedia.org/T139177 6. https://phabricator.wikimedia.org/T138089 7. https://phabricator.wikimedia.org/T134606 8. https://phabricator.wikimedia.org/T139407 9. https://phabricator.wikimedia.org/T139408 Sincerely, Amir from the Revision Scoring team.

1 0

Goals for for Fiscal Year 2016-17, Q1 (July–June)
by Wes Moran 05 Jul '16

05 Jul '16

Hello everyone, The goals for the Technology and Product teams have been posted on Mediawiki.org <https://www.mediawiki.org/wiki/Wikimedia_Engineering/2016-17_Q1_Goals> for FY2016-17, Q1 (July–September) [1] Please feel free to engage with the respective owners identified in each area for feedback or additional information. Also teams provide additional information on their respective Mediawiki.org pages as well. Thank you, Wes [1] https://www.mediawiki.org/wiki/Wikimedia_Engineering/2016-17_Q1_Goals

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l July 2016