Wikitech-l September 2016

wikitech-l@lists.wikimedia.org

105 participants
116 discussions

Long running tasks/scripts now included on [[wikitech:Deployments]]
by Greg Grossmeier 22 Sep '16

22 Sep '16

In an effort to reduce surprises and potential mishaps it is now required to include any long running tasks in the deployment calendar[0]. "Long running tasks" include any script that is run on production 'work machines' such as terbium that last for longer than ~1 hour. Think: migration and maintenance scripts. This was discussed and proposed in T144661[1]. Best, Greg [0] https://wikitech.wikimedia.org/wiki/Deployments Relevant diff: https://wikitech.wikimedia.org/w/index.php?diff=850923&oldid=850244 [1] https://phabricator.wikimedia.org/T144661 -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

2 2

REMINDER: No deploys next week (week of Sept 26th)
by Greg Grossmeier 22 Sep '16

22 Sep '16

Due to the Wikimedia Technical Operations Team having their team offsite that week and generally being less than normally available there will be no non-emergency deploys the week of September 26th (aka: next week). See also: https://wikitech.wikimedia.org/wiki/Deployments#Week_of_September_26th Normal schedule will resume the following week. Teaser: The week of October 17th will be a "no train deploys" weeks as the Wikimedia Release Engineering Team will be at their offsite that week. I'll send a reminder about this as well closer to that date. Best, Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

2 1

ORES article quality data as a database table
by Amir Ladsgroup 22 Sep '16

22 Sep '16

One of ORES [1] applications is determining article quality. For example, What would be the best assessment of an article in the given revision. Users in wikiprojects use ORES data to check if articles need re-assessment. e.g. if an article is in "Start" level and now good it's enough to be a "B" article. As part of Q4 goals, we made a dataset of article quality scores of all articles in English Wikipedia [2] (Here's the link to download the dataset [3]) and we are publishing it in figshare as something you can cite [4] also we are working on publishing monthly data for researchers to track article quality data change over time. [5] As a pet project of mine, I always wanted to put these data in a database. So we can query the database and get much more useful data. For example quality of articles in category 'History_of_Essex' [6] [7]. The weighed sum is a measure of quality which is a decimal number between 0 (really stub) to 5 (a definitely featured article). We have also prediction column which is a number in this map [8] for example if prediction is 5, it means ORES thinks it should be a featured article. I leave more use cases to your imagination :) I'm looking for a more permanent place to put these data, please tell me if it's useful for you. [1] ORES is not a anti-vandalism tool, it's an infrastructure to use AI in Wikipedia. [2] https://phabricator.wikimedia.org/T135684 [3] (117 MBs) https://datasets.wikimedia.org/public-datasets/enwiki/article_quality/wp10-… [4] https://phabricator.wikimedia.org/T145332 [5] https://phabricator.wikimedia.org/T145655 [6] https://quarry.wmflabs.org/query/12647 [7] https://quarry.wmflabs.org/query/12662 [8] https://github.com/wiki-ai/wikiclass/blob/3ff2f6c44c52905c7202515c5c8b525fb… Have fun! Amir

2 1

MediaWiki-Codesniffer 0.8.0-alpha.1 released
by Legoktm 22 Sep '16

22 Sep '16

Hi, As the subject line hints, we're doing something a little different with this release. There was a lot of work done on MW-CS over the summer by our GSoC student, Lethexie. There are a significant number of changes, especially to do with documentation comments. So we are releasing this as alpha quality to ask for more feedback on the changes to the ruleset. You can upgrade in the same manner as normal - by updating the version number in the composer.json file. Feel free to provide feedback by responding to this thread or by filing a bug in the MediaWiki-Codesniffer Phabricator project. We would like to release 0.8.0 by mid-October. Thanks! -- Legoktm

1 0

Upgrade of 1.28.0-wmf.19 to group 1 is on hold
by Antoine Musso 22 Sep '16

22 Sep '16

Hello, We are holding the deployment of MediaWiki version 1.28.0-wmf.19 due to a couple of bugs that have surfaced. The first one is that renaming a user was blocked [T145596]. Reported by K6ka, triaged by MarcoAurelio. The issue is fixed now thanks to Aaron Schulz and Kunal Mehta. The second blocker is way nastier. I have pushed the upgrade to group1. matanya (a long volunteer with a lot of technical patches) immediately reported the infoboxes on the Hebrew Wiki were on the wrong side, which prompted a rollback. [T145673]. Timo Tijhof and Kunal Mehta have found the root cause. I can speak for them as to who/when we will get a solution. For now. I am holding the train. Will reassess tomorrow and ideally push group1 at 19:00 UTC then follow with group2 at 20:00UTC. Stay tuned! Up-to-date status: https://tools.wmflabs.org/versions/ MW-1.28.0-wmf.19 deployment blockers https://phabricator.wikimedia.org/T143328 [T145596] Renames getting stuck on mediawiki.org (Sept 13, 2016) https://phabricator.wikimedia.org/T145596 [T145673] 1.28.0-wmf.19 broke template RTL placement https://phabricator.wikimedia.org/T145673 -- Antoine "hashar" Musso

4 7

[Discussion needed] Process for taking over abandoned tools
by Bryan Davis 22 Sep '16

22 Sep '16

In 2015, a phabricator task [0] and RfC discussion on meta [1] were started to create a process for determining when a tool has been abandoned by it's original maintainer(s) and how to hand control of the tool over to interested volunteers. The process stalled out without resolution. Our on-wiki communities are still highly dependent on volunteer developed tools and vulnerable to disruption when the original developers move on. I have drafted two straw dog [2] policies that attempt to define fair and workable solutions to the general problem. The proposals take two different but compatible approaches to solving the problem of abandonment. The Tool Labs developer community could choose to adopt either or both policies as protection for the communities that they serve. The first policy describes a *right to fork* for all Tool Labs hosted software. This policy clarifies the existing Tool Labs Open Source and Open Data requirements and defines a process for requesting access to code and data that are not already published publicly. The second policy is a more aggressive *abandoned tool policy* that describes a process for adding new maintainers to a tool account (adoption) with a future possibility of removing the original maintainers (usurpation). This policy is based primarily on the discussions that happened on Meta in 2015. Both policies propose creating a new committee of volunteers to evaluate requests and perform cleanup of sensitive data in the tool before providing the source code or direct access to the tool account. This provision is key actually implementing both proposals. Paid administration and management does not scale any better than paid editing. To continue to grow and thrive, the Tool Labs developer community needs to become more active in enforcing and expanding their own policies. Membership in the committees would require signing the Wikimedia Foundation's Volunteer NDA [3] to ensure that sensitive data is handled appropriately. If both polices are adopted the two committees should be collapsed into a single group with authority to handle both types of requests. The straw dog policies are posted on Wikitech: * https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Right_to_fork_policy * https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Abandoned_tool_policy Discussion of the particulars of each proposal should happen on their associated talk pages. As an example it would be appropriate to debate whether the 14 day non-functional waiting period is too short or too long on the Abandoned tool policy talk page. Discussion of the process in general can happen on Meta [1]. I would like discussion to remain open through *2016-10-12* (3 weeks from date of posting). Following the discussion period I hope to call for an approval vote of some sort to make the policies official. Wikitech and Tool Labs do not currently have well defined policies for establishing consensus, but I'm sure we can collectively come up with something reasonable. [0]: https://phabricator.wikimedia.org/T87730 [1]: https://meta.wikimedia.org/wiki/Requests_for_comment/Abandoned_Labs_tools [2]: https://en.wikipedia.org/wiki/Straw_man_proposal [3]: https://wikitech.wikimedia.org/wiki/Volunteer_NDA Bryan -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855

1 0

WikiDev17 Collaboration topic (Re: Opening up MediaWiki dev summit in January?)
by Rob Lanphier 22 Sep '16

22 Sep '16

Scott suggested the following as one of three suggested topic ideas for WikiDev17. The three ideas: 1) Collaboration 2) Wikitext Maintenance 3) Machine Translation More inline about "1) Collaboration" below: On Tue, Sep 20, 2016 at 10:05 AM, C. Scott Ananian <cananian(a)wikimedia.org> wrote: > *1. *(A unified vision for) *Collaboration* > > - Real-time collaboration (not just editing, but chatting, curation, > patrolling) > - WikiProject enhancements: User groups, finding people to work with, > making these first class DB concepts > - Civility/diversity/inclusiveness, mechanisms to handle/prevent > harassment, vandalism, trolling while working together > - Real-time reading -- watching edits occur in real time > - Integration with WikiEdu > - Broadening notion of "an edit" in DB -- multiple contributors, > possibly multiple levels of granularity > - Tip-toeing toward "draft"/"merge" models of editing > - Better diff tools: refreshed non-wikitext UX, timelines, authorship > maps, etc. I've copied this wholesale into the "Collaboration" area on [[WikiDev17/Topic ideas]], and quoted it directly here: <https://www.mediawiki.org/wiki/Topic:Tbypptt9myumu7q7> Let's use this thread to focus on this part of Scott's proposal. A lot of these seems in scope for the Wikimedia Collaboration team. Does the scope that you're thinking of align with what the team has published on their page: <https://www.mediawiki.org/wiki/Collaboration> Rob (p.s. please feel free to start separate threads with the other parts of Scott's proposal)

2 1

2016-09-21 Scrum of Scrums meeting notes
by Grace Gellerman 22 Sep '16

22 Sep '16

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-09-21 = 2016-09-21 = == Product == === Editing === ==== Collaboration ==== * Blocking * Blocked ** Services is working with us on the ReviewStream feed we're developing. * Updates ** Continued work on MessagePoster. PageTriage change to use it up, but not merged yet ** Fixed moderation (e.g. when a page or topic is deleted) of Echo notifications ** Some other Echo changes, such as to how seen time works, GENDER i18n, and to the page-linked notification ==== Language ==== * Blocked: none * Blocking: none * Updates: ** Content Translation work continue; templates and other fixes. ** Jessie migration for Apertium is scheduled on October first week. ==== Parsing (Subbu not attending because of a conflict) ==== * Blocked: Parsoid's native <gallery> implementation is blocked on getting feedback from the VE team. * Patches in gerrit to parse language variant markup in Parsoid -- need to resolve a few questions before we can deploy this * Magic-word based optout of global user pages merged (__NOGLOBAL__) * Kunal has filed an RFC to remove magic link functionality in the parser: https://www.mediawiki.org/wiki/Requests_for_comment/Future_of_magic_links * Evaluation ongoing of a (already merged in core) PHP version of a HTML5 parser (as a potential replacement for Tidy). === Community Tech === * No blockers * Copyvio tools for French Wikipedia https://phabricator.wikimedia.org/T145431 * More performance testing in hopes to get PageAssessments deployed to English Wikipedia * Education Programs Dashboard https://phabricator.wikimedia.org/T125546 * Changes to CentralAuth for cross-wiki watchlists * Investigating work on improving blocking and harassment prevention tools === Reading === ==== Reading Web ==== * Current sprint: https://phabricator.wikimedia.org/project/view/2186/ * Preparing to enable hovercards ab test in italian and russian * Preparing to enable related articles in mobile in more wikis * Lazy images blog post * Next sprint: https://phabricator.wikimedia.org/project/view/2221/ * Enable related articles in mobile in more wikis ==== Mobile Content Service (MCS) ==== * Getting the aggregated feed endpoint working on Beta Cluster since feed aggragation happens in RB now. ==== Android native app ==== * Current sprint: https://phabricator.wikimedia.org/project/view/2216/ * Next sprint: https://phabricator.wikimedia.org/project/view/2238/ * Released to beta Monday 9/19. Plan is to get feedback, fix bugs, and release nav overhaul to the stable app by EOM. ==== iOS native app ==== * Current release board: https://phabricator.wikimedia.org/project/view/2188/ ** In regression ** Expected to be released Sept 21 * Next release board: https://phabricator.wikimedia.org/project/board/2220/ ** In progress ** MIgrating iOS Feed over to the MCS ** Adding In the news and a local notification ** Expected Delivery ~3 weeks ==== Reading Infrastructure ===== * no === Fundraising Tech === * Almost done getting rid of ActiveMQ! * Fixing CentralNotice db calls for strict mode * More refinements to CiviCRM contact de-duplication === RelEng === * Blocking ** no * Blocked ** wmf.18 load-time regression https://phabricator.wikimedia.org/T146099 * Updates ** Skipping wmf.19 entirely due to undiagnosed load-time regression https://wikitech.wikimedia.org/wiki/Incident_documentation/20160915-MediaWi… *** Tentative plan is to cut wmf.20 and deploy to group0 today and, if all goes well, adjust the remainder of this week's schedule accordingly *** https://phabricator.wikimedia.org/T144644#2654240 ** Completed Beta Cluster database migration to jessie/MariaDB 10 yesterday ** Reminder: no deploys week of Sept 26th ** Reminder: no train week of Oct 17th (but SWATs OK) === Analytics === * Blocking: None * Blocked: None * Updates: ** New AQS cluster is up! Massive reduction of latency. We'll increase the throttling threshold. ** Public Event Streams: Finalized Kasocki prototype. Considering using server side events / streaming instead of web-sockets. ** Edit history reconstruction: Some improvements on the quality of data, still a couple issues to fix. ** Pivot (druid based UI): More work on productionizing it using servicerunner ** A/B testing design - Evaluating feasibility of design ideas. ** Nuria participated in the Technology off-site, very productive. === Technical Operations === * '''Blocking''' ** None * '''Blocked''' ** None * Updates ** TechOps offsite happening week of Sept 25 (last week of quarter), please work around this for deployments ** Will not be able to attend SoS for the same reason ** Puppet migration complete ** Varnish upload cluster upgraded to version 4. There are some caveats, but looking good https://phabricator.wikimedia.org/T145661 ** Apertium migration to Jessie scheduled for Oct 3 ** OpenStackManager in wikitech is being deprecated. There will be information coming your way in the next weeks. === Security === * Patch deployed for CentralAuth on monday; let us know if you experience weirdness * Will follow-up on Youdao review ( https://phabricator.wikimedia.org/T143185) * Catching up on e-mail, Gerrit, and Phab traffic from last week === Services === * Blocking: none * Blocked: none * Updates: ** Huge RESTBase deployment, VE now usable on undeleted articles ** Change-Prop: separated transclusions topics - decrease user visibility of lagging ** Trending service planning with reading ** Bad articles stream planning === Discovery === * No blockers * Analyzing BM25 A/B test, if results are positive deployment will be somewhere early October * Working on multiwiki indexes, public comments sought at: ** https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements ** https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Design * Working on file properties indexing (size, type, dimensions, etc.) * WDQS servers on codfw deployed, looking into how to add them to production setup * Portal: updated portal strings to "Read Wikipedia in your language" and language wiki descriptions. == Wikidata == * Blocker: Can not connect to the Hangout on https://hangouts.google.com/hangouts/_/wikimedia.org/scrum-of-scrums. Get error "HJR: 2-RNF". * No other ;-) blockers at the moment. * Making usage information (which page on which wiki uses which Wikidata entity) visible to users: https://phabricator.wikimedia.org/T103091 * Introduced batching in the job that notifies wikis about changes on wikidata.org: https://gerrit.wikimedia.org/r/309972, https://phabricator.wikimedia.org/T107722 * Working on cleaning up several dependencies on legacy stuff in core, e.g. https://phabricator.wikimedia.org/T144977 * Pushing multi content revisions forward.

1 0

Re: [Wikitech-l] Opening up MediaWiki dev summit in January?
by Rob Lanphier 21 Sep '16

21 Sep '16

On Tue, Sep 20, 2016 at 10:05 AM, C. Scott Ananian <cananian(a)wikimedia.org> wrote: > Here are three topic suggestions, cc'ed here in case folks aren't following > the Flow, with an illustrative (but not exhaustive) list of sessions that > could fit under each Thanks for doing that breakdown! I incorporated your suggestions into this page: <https://www.mediawiki.org/wiki/WikiDev17/Topic_ideas> I suspect each of the three ideas you presented here are worthy of separate threads (and perhaps separate on-wiki conversations), so I'll reply to at least the first part under separate cover. Rob

1 0

Re: [Wikitech-l] [Engineering] Long running tasks/scripts now included on [[wikitech:Deployments]]
by Greg Grossmeier 21 Sep '16

21 Sep '16

<quote name="Aaron Schulz" date="2016-09-20" time="15:31:44 -0700"> > How far ahead of time must entries be added? I trust most people's best judgement here :) But, as soon as you have an idea of when you'll do it is the best time to do it. Obviously if the task will impact others and/or require no other deploys at the same time that should be scheduled further in advance (~1 week) than something that is low-risk and will be completed in 2-3 hours. -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

1 0

← Newer
1
2
3
4
5
6
7
8
9
...
12
Older →

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l September 2016