Wikitech-l July 2016

wikitech-l@lists.wikimedia.org

81 participants
84 discussions

Discovery Weekly Update for the week starting 2016-07-11
by Chris Koerner 16 Jul '16

16 Jul '16

Hello again, I hope your day is going well. Here is this week's update from the Discovery department. == Search == * Mikhail completed a review of the language-identification A/B test (using TextCat). The full report is on Commons. [1] * Trey has more language-identification–related stuff: notes on how to introduce confidence to TextCat [2] (comments and ideas especially welcome!), a re-review of the languages of poorly performing queries on English Wikipedia [3] using our now-standard corpus creation process (TL; DR: quite similar to earlier results); and similar first-time reports for Portuguese and Russian Wikipedias [4], with Japanese and Indonesian in the pipeline. * Trey and Jan collaborated on a new design of the TextCat demo [5] * Team discussing deploying the new TextCat language detection to wikis: en, it, fr, de, es with more language wikis to come in the future [6] * a cute picture of a cat and dog, just because [7] == Maps == * Map's geoshapes service is now available in graphs: Interactive maps with regions [8] * Wikivoyage map customizations have moved into the extension code, now available on all WV languages. Customize it via MediaWiki:Kartographer.js and MediaWiki:Kartographer.css [9] [10] == Wikipedia.org Portal== * Conversation with the community continuing for new Wikipedia.org portal page layout with a new language by article count dropdown. [11] ** The page has been translated, so far, in 8 languages and posted on various Village Pump sites; ** a new working prototype has been added [12] ** and a new mock up design has been added which was based off of community feedback. [13] * Released findings and conclusions of the recent survey ran on the Wikipedia.org portal page - how did you arrive at Wikipedia - that also contains good feedback on the portal design page, the sister project links with descriptive text, search and usage on various platforms. [14] * Conversation with the community on handling search queries with question marks also continues [15] ** this page has been translated into 5 additional languages * Team conversations started about how to package portions of the wikipedia portal page to make it easier to re-use across all projects that want it. [16] * Team conversations started about how to automate stats updates on the wikipedia portal page [17] == Events and News== * Erik was promoted to Senior Software Engineer, yay! * Discovery Quarterly review was held [18] * Geo Search was launched by the Discovery Search team on behalf of the Reading team [19] [1] https://commons.wikimedia.org/wiki/File:Report_on_Cirrus_Search_TextCat_AB_… [2] https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/TextCat_and_Confiden… [3] https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/TextCat_Re-optimizat… [4] https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/TextCat_Optimization… [5] https://tools.wmflabs.org/textcatdemo/ [6] https://phabricator.wikimedia.org/T121541 [7] https://commons.wikimedia.org/wiki/File:Foxteriér_a_kocúr.jpg [8] https://www.mediawiki.org/wiki/Template:Graph:Country_with_regions_and_capi… [9] https://en.wikivoyage.org/wiki/MediaWiki:Kartographer.js [10] https://en.wikivoyage.org/wiki/MediaWiki:Kartographer.css [11] https://www.mediawiki.org/wiki/Wikipedia.org_updated_page_layout [12] https://people.wikimedia.org/~jdrewniak/collapsed-languages/index.html [13] https://www.mediawiki.org/wiki/File:Wikipedia.org_portal_more_languages_but… [14] https://www.mediawiki.org/wiki/File:Discovery_-_Wikipedia.org_Portal_Study_… [15] https://meta.wikimedia.org/wiki/Discovery/Handling_question_marks_in_search… [16] https://phabricator.wikimedia.org/T136151 [17] https://phabricator.wikimedia.org/T140159 [18] https://docs.google.com/presentation/d/1gVXPwLtUZc1H5V82wv1TJU-abmmfaZkXYlo… [19] https://www.mediawiki.org/wiki/Help:CirrusSearch#Geo_Search ---- The full update, and archive of past updates, can be found on Mediawiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates -- Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation

1 0

Re: [Wikitech-l] Gerrit 2.12.2 test instance - PLEASE TEST
by Markus Glaser 15 Jul '16

15 Jul '16

This update will definitely improve our experience with git. As others have said before, the inline editing will greatly speed up the review process and improve code quality. This is especially true for the extensions, as you now do not have to clone a repo or reject the commit just to fix a typo or some coding conventions. Thanks for making this happen! I found that the behavior of my old search filters has changed. I used to look for bluespice status:open to get all open commits in all bluespice repos. This however does not work in the test instance, I only get a small subset of the commits I need to see. There are ways around this, e.g. ownerin:bluespice status:open, so I am not too concerned about it. Am not sure whether it affects other extension writers, though. Cheers, Markus (mglaser)

1 0

The Revision Scoring weekly update
by Amir Ladsgroup 15 Jul '16

15 Jul '16

Hey, This is the 12th weekly update from revision scoring team that we have sent to this mailing list. New developments: - We deployed ORES review tool in Russian and Portuguese Wikipedia! [1] [2] - Basic support for English Wiktionary and Czech Wikipedia is live in labs.[3] Soon production will follow. - Damaging model for Polish Wikipedia is ready and soon be available (then we can enable ORES review tool as well) [4] - ORES extension now highlights the whole row for Enhanced recent changes too. It will be live in two weeks [5] - We built prototype of a small web app to show progress of wikilabels campaigns at https://tools.wmflabs.org/dexbot/tools/wikilabels_stats.php. Feedback is welcome [6] Maintenance and robustness: - If a web node goes down in labs, we get icigna warnings [7] - We re-launched wikilabels campaigns for French and Azeri Wikipedia [8] [9] - We are starting to use tsv2json to load the whole data when adding new campaigns in wikilabels [10] - "Hide/Show good edits" soon will be changed to "Hide/Show probably good edits" in recent changes and watchlists to reflect ORES scores more accurately [11] 1. https://phabricator.wikimedia.org/T139541 2. https://phabricator.wikimedia.org/T139692 3. https://phabricator.wikimedia.org/T139789 4. https://phabricator.wikimedia.org/T139207 5. https://phabricator.wikimedia.org/T139924 6. https://phabricator.wikimedia.org/T139874 7. https://phabricator.wikimedia.org/T134782 8. https://phabricator.wikimedia.org/T107730 9. https://phabricator.wikimedia.org/T139875 10. https://phabricator.wikimedia.org/T139939 11. https://phabricator.wikimedia.org/T139754 Sincerely, Amir from the Revision Scoring team

3 4

Phabricator monthly statistics - 2016-06
by communitymetrics＠wikimedia.org 15 Jul '16

15 Jul '16

Hi Community Metrics team, This is your automatic monthly Phabricator statistics mail. Accounts created in (2016-06): 240 Active users (any activity) in (2016-06): 836 Task authors in (2016-06): 471 Users who have closed tasks in (2016-06): 252 Projects which had at least one task moved from one column to another on their workboard in (2016-06): 199 Tasks created in (2016-06): 2478 Tasks closed in (2016-06): 2141 Open and stalled tasks in total: 30571 Median age in days of open tasks by priority: Unbreak now: 18 Needs Triage: 173 High: 318 Normal: 470 Low: 777 Lowest: 594 (How long tasks have been open, not how long they have had that priority) TODO: Numbers which refer to closed tasks might not be correct, as described in https://phabricator.wikimedia.org/T1003 . Yours sincerely, Fab Rick Aytor (via community_metrics.sh on iridium at Thu Jul 14 22:31:51 UTC 2016)

1 0

2016W29 ArchCom-RFC meeting: Plan for a cross-wiki watchlist back-end
by Rob Lanphier 14 Jul '16

14 Jul '16

Hi all, The plan for next week's ArchCom office hour is to discuss T126641: [RFC] Devise plan for a cross-wiki watchlist back-end. There's been a fair amount of discussion already on T126641 which is trying to sort out the following questions: * Is keeping a central table of all recent changes for all wikis a doable option? * Is doing separate queries for each wiki a better or worse idea? * Are there ways that we could mitigate the impact of either of those options? * Are there other options that we haven't even thought about yet? #wikimedia-office: 2016-07-20 (Wednesday) 21:00 UTC (2pm PDT, 23:00 CEST) More details about the meeting: <https://phabricator.wikimedia.org/E235> More details about the topic: <https://phabricator.wikimedia.org/T126641> Of course, no one needs to wait for the meeting to comment. Please make your thoughts known on this list, or in Phab on T126641. Rob

1 0

geo search is now live
by Erik Bernhardson 14 Jul '16

14 Jul '16

This week we shipped a new feature at the request of the mobile apps team, geo search. (bare bones) Documentation: https://www.mediawiki.org/wiki/Help:CirrusSearch#Geo_Search Quick example usage for bounded geo search: https://people.wikimedia.org/~ebernhardson/mapsearch.html#Pittsburgh|intitl… https://people.wikimedia.org/~ebernhardson/mapsearch.html#San Francisco|hastemplate:"National Register of Historic Places in California" https://people.wikimedia.org/~ebernhardson/mapsearch.html#200km,Moscow|inca… Direct search example: https://en.wikipedia.org/w/index.php?fulltext=1&search=incategory%3AKremlin… Also implemented a "boost" version which increases the search score of pages within a particular area, rather than the default which limits to a geographic area. Compare: https://en.wikipedia.org/w/index.php?fulltext=1&search=boost-neartitle%3A%2… vs. https://en.wikipedia.org/w/index.php?fulltext=1&search=museum

1 0

Re: [Wikitech-l] [action required] We are removing inactive projects.
by Andrew Bogott 14 Jul '16

14 Jul '16

Reminder: There are still 73 unclaimed projects listed on https://wikitech.wikimedia.org/wiki/Purge_2016. Please claim your projects today! (Or, better yet, mark them as unused so I can start reclaiming space.) Thanks! -Andrew On 7/8/16 11:09 AM, Andrew Bogott wrote: > If you are exclusively a user of tool labs, you can ignore this email. > If you use or administer another labs project, this email REQUIRES > ACTION ON YOUR PART. > > We are reclaiming unused resources due to an ongoing shortage. > > Visit this page and add a signature under projects you know to be active: > > https://wikitech.wikimedia.org/wiki/Purge_2016 > > Associated wmflabs.org <http://wmflabs.org> domains are included to > identify projects by offered services. > > We are not investigating why projects are needed at this time. If one > person votes to preserve then we will do so in this round of cleanup. > > In a month, projects and associated instances not claimed will be > suspended or shutdown. A month later if no one complains these > projects will be deleted.. > > - Andrew (on behalf of all Labs Admins everywhere)

1 0

What's the "correct" content model when rev_content_model is NULL?
by Daniel Kinzler 14 Jul '16

14 Jul '16

It seems there is disagreement about what the correct interpretation of NULL in the rev_content_model column is. Should NULL there mean (a) "the current page content model, as recorded in page_content_model" or should it mean (b) "the default for this title, no matter what page_content_model says"? Kunal and I have had an unintentional edit war about this question in Revision.php: Kunal changed it from (a) to (b) in https://gerrit.wikimedia.org/r/#/c/222043/ I later changed it from (b) to (a) in https://gerrit.wikimedia.org/r/#/c/297787/ Kunal reverted me from (a) to (b) in https://gerrit.wikimedia.org/r/#/c/298239/ So, which way do we want it? The conflict seems to arise from (at least) three competing use cases: I) re-interpreting page content. For instance, a user may move a misnamed User:Foo.jss to User:Foo.js. In this case, the content should be re-interpreted as JavaScript, including all old revisions. This would be in favor of behavior (a), though it still works with (b), because the default model changes based on the suffix ".js". I think it would however be better to only rely on title parsing magic once, when creating the page, not later, when rendering old revisions. II) converting page content. For instance, if a talk page gets converted to using Flow, new revisions (and page_content_model) will have the Flow model, while old revisions need to keep their original wikitext model (even though their rev_content_model is null). That would need behavior (b). III) changing a namespace's default content model. E.g. when installing an extension that changes the default content model of a namespace (such as Wikibase with Items in the main namespace, or Flow-per-default for Talk pages), existing pages that were already in that namespace should still be readable. With (b), this would fail: even though page_content_model has the correct model for reading the page, rev_content_model is null, so the new namespace default is used, which will fail. With (a), this would simply work: the page will be rendered according to page_content_model. In all cases it's possible to resolve the issue by replacing the NULL entries for all revisions of a page with the current model id. The question is just when and how we do that, and when and how we can even detect that this needs doing. There is also an in-between option, let's call it a/b: fall back to page_content_model for the latest revision (that should *always* be right), but to ignore page_content_model for older revisions. That would cater to use case III at least in so far as it would be possible to view the "misplaced" pages. But viewing old revisions or diffs would still fail with a nasty error. This option may look better on the surface, but I fear it will just add to the confusion. There's another fix: never write null into rev_content_model. Always put the actual model ID there. That's pretty wasteful, but it's robust and reliable. When we decided to use null as a placeholder for the default, we assumed the default would never change. But as we now see, it sometimes does... So, what should it be, option (a) or (b)? And how do we address the use case that is then broken? What should we write into rev_content_model in the future? I personally think that option (a) makes more sense, because the resolutions of defaults is then local to the database. It could even be done within the SQL query. It's easier to maintain consistency that way. For use case II, that would require us to "fill in" all the rev_content_model fields in old revisions when converting a page. I think it would be a good thing to do that. If we have the content model change between revisions, it seems prudent to record it explicitly. -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

9 24

Re: [Wikitech-l] phpunit: "Extension mysql is required"?
by Daniel Barrett 13 Jul '16

13 Jul '16

Never mind - solved it. I discovered that one of my phpunit test files contains @requires extension mysql and this extension is gone in PHP 7.0. DanB

2 1

2016-07-13 Scrum of Scrums meeting notes
by Grace Gellerman 13 Jul '16

13 Jul '16

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-07-13 = 2016-07-13 = == Product == === Reading === ==== Reading Infrastructure ==== * No update, but there will be a follow up meeting on Echo auth impl bug 20-July-2016 (multistakeholder bug) ==== Reading Web ==== * No update, working on language switcher on mobile web ==== iOS native app ==== * 5.0.5 is currently in beta - heading to regression on Tuesday * Setup new plan to incrementally improve regression testing * Development of 5.1 is in progress * Planning of 5.2 is in progress * Discovery implemented https://phabricator.wikimedia.org/T139378 to enable iOS to do https://phabricator.wikimedia.org/T130889 * Filed ticket to start work on a real time trending API: https://phabricator.wikimedia.org/T140102 ==== Android native app ==== ==== Mobile Content Service ==== * Getting close to having public endpoints for the random and aggregated feed endpoints. Removing blockers for that. * Service had been flapping quite a bit lately. Log files indicate heap memory issues. === Community Tech === * Investigating cross-wiki watchlist, how to implement back-end ** Refered to ArchComm as RFC * Will be deploying PageAssessments to English WIkipedia soon ( https://www.mediawiki.org/wiki/Extension:PageAssessments) * Also will be switching English Wikipedia to UCA collation with numerical sorting === Editing === ==== Collaboration ==== * Blocked: None * Blocking: No change * Update ** Finished rollout out bundling and re-sorting changes. Had a couple issues: *** We forgot to change Alerts/Messages to Alerts/Notices, so had to do a SWAT for this. This had complications initially: https://wikitech.wikimedia.org/wiki/Incident_documentation/20160707-Echo *** We learned yesterday there was a bug in Echo that was causing users to have inconsistent state, which prevented login. This was fixed yesterday: https://wikitech.wikimedia.org/wiki/Incident_documentation/20160712-EchoCen… ==== Multimedia ==== * Blocked: None? * Blocking: None? * Update: None? ==== Language ==== * Blocked: None? * Blocking: None? * Update: None? ==== Parsing ==== * Blocked: service-runner migration * Blocking: None? * Update: Services is helping with migrating Parsoid to the service-runner framework. Parsoid deployments are on hold till that time. Migration to node v4, jessie, scap3 will follow that migration. ==== VisualEditor ==== * Blocked: None. * Blocking: None known. * Update: Lots of work on bugs and regressions this week; https://gerrit.wikimedia.org/r/#/c/298392/ / T129360 in MobileFrontend is the remaining focus. == Technology == === Services === '''Blocking''' ** None '''Blocked''' ** None * Node security upgrade: RB, AQS and SCB on Nodejs 4.4.6, Parsoid pending move to Jessie * RESTBase Cassandra cluster upgraded to 2.2.6 * Preparing deploy of new feed API end points (ex: featured article / image) with Reading. ETA likely this week. * Parsoid move to service-runner - *https://phabricator.wikimedia.org/T90668* <https://phabricator.wikimedia.org/T90668> ** Currently testing in BetaCluster **Ping for Ops: need to coordinate the transition in prod === Fundraising Tech === * Civi upgrade complete * CentralNotice deployed * Upgrading payments to MW 1.26 * Building out new servers * Kafka sampling problem solved * No blockers === TechOps === '''Blocking''' ** None * '''Blocked''' ** https://phabricator.wikimedia.org/T135483 - HHVM crashes - raised to UBN! after issue recurrence. Currently no one owns the ticket. * Updates ** About 25% of the MediaWiki clusters run on jessie, we will start reimaging old servers during this quarter, a schedule will be announced. ** Enabled TCP Fast Open on some caching clusters ** Insecure POST traffic cutoff: All external traffic is prohibited, but labs is still permitted (with a 20% failure rate) === Security === * Node.js dependencies are now being checked nightly for disclosed vulnerabilities; Darian will be manually creating Phab tickets about this initially, with automation to follow * Request security reviews: https://www.mediawiki.org/wiki/Wikimedia_Security_Team/Security_reviews * A security release will be prepared soon, possibly July 20 * Reviews: Tool Labs Console (cont.) ===Wikidata / WMDE=== * Daniel is working on multi-content revisions https://phabricator.wikimedia.org/T107595 * Continuing work on structured data support for commons * Preparing deploy of RevisionSlider to test wikis and then as a beta feature on German Wikipedia https://phabricator.wikimedia.org/T140232 === Discovery === * '''Blocking''': none * '''Blocked''': none * Logstash upgrade postponed to Jul 18th, 19-22 UTC * Initiated discussion on using ? in searches: https://meta.wikimedia.org/wiki/Discovery/Handling_question_marks_in_search… * Geosearch (near*:) is ready to roll out in production: https://phabricator.wikimedia.org/T139378 in 1.28.0-wmf.10 * Started discussion on new wikipedia.org portal layout: https://www.mediawiki.org/wiki/Wikipedia.org_updated_page_layout * Geoshapes in graphs and (soon) maps: http://data.wmflabs.org/wiki/Regional_maps === RelEng === * '''Blocking''' ** None * '''Bocked''' ** None * '''Updates''' ** New gerrit update needs testing: https://gerrit-new.wikimedia.org/r/ <https://gerrit-new.wikimedia.org/r/> ** wmf.9 was reverted, wmf.10 will get pushed to group0 and group1 today on a short schedule *** Retrospective to come https://wikitech.wikimedia.org/wiki/Incident_documentation/20160712-EchoCen… === Architecture === * ArchCom weekly meetings today: ** 1pm PDT Planning meeting (private): [[Phab:E227]] ** 2pm PDT Discussion (public) IRC #wikimedia-office: [[Phab:E228]] *** Topic: [[Phab:T589]] - image and oldimage tables * Anything stalled? Comment in Phab here: [[Phab:Z425]]

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l July 2016