Wikitech-l March 2016

wikitech-l@lists.wikimedia.org

109 participants
102 discussions

Reading Team Q4 (April-Jun) Goals
by Jon Katz 25 Mar '16

25 Mar '16

Hi Folks, The Reading team's Q4 goals for 2015-16 are posted on Media Wiki: https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q4_Goals#Readi…. We arrived at these goals by taking the results of the Q4 brainstorming <https://www.mediawiki.org/wiki/Reading/Quarterly_planning/Q4> and matching them against our strategy, speed and strengths. Below is a summary of what we aim to do in the April-June Quarter with more context than what you see on our goals page. You will notice a lot of cross-platform migration. That is no accident: we learn faster by launching on one platform and then migrating the idea to others. The goals are broken out by team or strategic theme. *Community Tech: Community Wishlist items* Ship features and fixes related to three wishes in the Wishlist Survey top 10 <https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey#The_top_10_w…> *Infrastructure: two-factor authentication*The infrastructure team will enable two-factor authentication <https://en.wikipedia.org/wiki/Two-factor_authentication> via AuthManager on the Wikimedia cluster. The goal here is to add another layer of security for accounts with important privileges. *New Readers (was: "reach new users in the global south")* *Web: decrease load time and cost for low-resource environments on mobile*This will be accomplished through lazy loading <https://en.wikipedia.org/wiki/Lazy_loading> of images, and cutting default html size. This is in beta mobile channel by the end of this quarter and we plan to measure results and roll out to all users over the course of Q4. *Make a better encyclopedia experience* *Web: hovercards*Specifically, enable hovercards <https://www.mediawiki.org/wiki/Beta_Features/Hovercards> as default for logged-out desktop users on more than one wiki (this one is dependent on community approval). Hovercards represent a faster way of browsing Wikipedia that most readers prefer (as indicated by this survey <https://www.mediawiki.org/wiki/Beta_Features/Hovercards#Qualtrics_Survey_Re…>). It has been in beta for more than a year and the mobile version was recently launched to all users on our Android app. This is not as simple as turning on a switch, there are several improvements that will have to be made. One reason we are launching this is to clean up our beta site before picking up any new work. This is arguably "New Readers" as well, due to the performance gains from loading fewer pages. A community initiated proposal is being discussed to enable this on English Wikipedia. Please chime in! https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)#Proposal:_… *Android: launch the feed*This quarter, the iOS team launched a feed on the app home screen and it looks great and early results suggest users are responding well to it. The goal is to drive user retention by giving readers a reason to open up Wikipedia even if they don't have a specific query in mind. The Android team will also be implementing backend services to support the feed across platforms. *iOS: Universal links*"Universal Links" (aka deep links) provide convenient re-entry to the app from links and OS search, but do not advertise or promote the app. This is something that is already live on Android and partially supported on iOS. It allows users who prefer the app to open it from links or an OS query, since that is how most users get to Wikipedia. *A Community of Readers (interactive Wikipedia)* This is the third pillar that came out of our strategy process. No active development work is planned here. Future direction for this part of our strategy is currently being explored via a community consultation here: https://www.mediawiki.org/wiki/User_Interaction_Consultation Your feedback or questions are welcome. Thanks, J

1 0

Goals for for Fiscal Year 2015-16, Q4 (April–June)
by Wes Moran 25 Mar '16

25 Mar '16

Hello everyone, The goals for the Technology and Product teams have been posted on Mediawiki.org <https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q4_Goals> for FY2015-16, Q4 (April–June) [1] Please feel free to engage with the respective owners identified in each area for feedback or additional information. Also teams provide additional information on their respective Mediawiki.org pages as well. Thank you, Wes [1] https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q4_Goals

1 0

2016-03-23 Scrum of Scrums notes
by Grace Gellerman 24 Mar '16

24 Mar '16

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-03-23 = *2016-03-2**3* = == Product == === Community Tech === * '''Blocking''': ** None * '''Blocked''': ** Waiting for jcrespo to do schema change for category sorting: https://phabricator.wikimedia.org/T130692 * '''Updates''': ** Pageviews Analytics tool is feature complete ( http://tools.wmflabs.org/pageviews/), talking to communities about adding links to it ** Deadlink logging API is finished, adding integration to Cyberbot === Reading === ==== Reading Infrastructure ==== * *AuthManager is still coming!* (see three weeks ago for details) ** Behind schedule though, but we think we're about there on the core patches. * It looks like we're getting close on the load.php thing, too. ==== Mobile Content Service ==== * '''Updates''': ** Fixed issue with lead paragraph transformation https://phabricator.wikimedia.org/T65874 ** Starting rollout of Mobile Content Service to Android production app this Thursday https://phabricator.wikimedia.org/T126934 ==== Android ==== * 2.1.143 beta published. Expected to be promoted to production soon. ==== iOS ==== * '''Updates''': ** 5.0.1 was released today - all bug fix updates ** Hockey app is currently much happier - less crashes * '''Upcoming''': ** Enabling handoff from Mac browser to iOS app and credentials sharing by updating a file on the server ** No time frame for next release (but expect it within 3 weeks) - currently deciding whether to do another bug fix or do 5.1 with features * '''Blocking''': ** Nothing ====Web==== =====Done===== * Lazy loaded images rolling to beta mobile web * Structured Language Overlay - trying to push it to 100% stable * Language Switcher button - beta for now * Lazy loading references =====Upcoming===== * Bug fixes and tweaks to the Language Overlay * Ship the language switcher * Semi-structured sprint === Editing === ==== Collaboration ==== * '''Blocking''': ** External store work * '''Blocked''': * '''Updates''': ** Working on support for Flow notifications being properly hidden on moderation ** Work on the Echo special page ** Changed MobileFrontend to reuse Echo code ** Working on Flow bug fixes ==== Language ==== * '''Blocking''': ** None * '''Blocked''': ** New MT deployments blocked due to https://gerrit.wikimedia.org/r/#/c/277463/ (WIP) * '''Updates''': ** Work on ContentTranslation continue. ==== Multimedia === * '''Blocking''': ** None * '''Blocked''': ** None * '''Updates''': ** UploadWizard fixes and performance stuff on the way, currently in the works ** ImageTweaks is slow going, currently there's a patch in core that it depends on, probably won't be moving as quickly as we thought ==== Parsing === * '''Blocking''': ** None? * '''Blocked''': ** None * '''Updates''': ** Somewhat of a lull in activity last couple weeks (with vacation, time off, 20% work) ** But, heads up for VE, CX, Flow, Reading teams about https://phabricator.wikimedia.org/T78676 -- been filing dependent tasks for various teams. Parsoid side of the work should mostly be ready for merge in 2-3 weeks. But, other dependent tasks need to be resolved, performance and client blockers will be considered before enabling this in production. ==== VisualEditor === * No update. == Advancement == === Fundraising Tech === * No blocking/blockers * DonationInterface mw1.27 tests are failing on some i18n core change, but they're no longer voting * Raft of CiviCRM enhancements to make logging more robust and enable reverting contact merges * Testing new Latin American payment methods in previously unpestered countries * Pondering how to make payments use a more recent version of mediawiki https://phabricator.wikimedia.org/T130658 * More DonationInterface refactoring == Technology == === Release Engineering === * '''Blocking''': ** ??? * '''Blocked''': ** None * '''Updates''': ** Reminder: Code freeze & Dallas switchover week of April 18 ** Trebuchet -> Scap migration! Get in touch if you need it *** There is a super secret irc channel on freenode, cryptically named #scap3 *** scap 3.1 should be deployed soon, now with moar git-fat support. This will unblock some services switching from Trebuchet to scap. === Research === ==== Research and Data ==== * '''Blocking''': ** none * '''Blocked''': ** ORES blocked on MW core code review *** https://etherpad.wikimedia.org/p/ores_extension_blockers *** What is the normal workflow for getting code reviewed? **** typical turn-around time? **** documentation for engaging +2 people? * '''Updates''': ** ORES downtime over the weekend *** https://wikitech.wikimedia.org/wiki/Incident_documentation/20160319-Ores *** tl;dr pip doesn't remove old versions when installing new wheels === Security === * '''Blocking''': ** ??? * '''Blocked''': ** ??? * '''Updates''': ** 2FA updates for wikitech today. 2FA available on SUL wikis to Staff next week. === Services === * '''Blocking''': ** none * '''Blocked''': ** Ops: change-prop deploy - https://phabricator.wikimedia.org/T128463 * '''Updates''': ** RESTBase to start redirecting non-normalised titles and commons URLs ** DC switch-over for RESTBase && node services successful - https://phabricator.wikimedia.org/T127974 ** Mathoid and RESTBase now support PNG generation ** Scap3: should have a guide soon for node.js services === Technical Operations === * '''Blocking''': ** Services on changepropagation deploy https://phabricator.wikimedia.org/T128463 * '''Blocked''': ** none * '''Updates''': ** All the rdb100* hosts (Redis Job Queue) have been re-imaged with Debian. ** ORES database hosts ready, looking for some help with scap3 and ORES ==== Discovery ==== * '''Blocking''': ** none * '''Blocked''': ** Ops: https://phabricator.wikimedia.org/T127014 ** Security review: SVG sanitizer for graphs - https://phabricator.wikimedia.org/T125382 * '''Updates''': ** ZRR down following completion suggester deployment: http://discovery.wmflabs.org/metrics/#failure_suggestions ** Working on scoring improvement for search and suggester ** Working on connection pooling for ES ** Join #wikimedia-interactive for interactions about interactive content!

1 0

Internship opening: product management at WMDE
by Pine W 24 Mar '16

24 Mar '16

Forwarding info about an opportunity to intern in product management with Wikidata and the good people at Wikimedia Deutschland. Pine ---------- Forwarded message ---------- From: "Lydia Pintscher" <Lydia.Pintscher(a)wikimedia.de> Date: Mar 24, 2016 07:24 Subject: [Wikidata] internship opening: product management at WMDE To: "Discussion list for the Wikidata project." < wikidata(a)lists.wikimedia.org> Cc: Hey folks :) I am looking for someone to support me me with my product management work at WMDE as an intern. If you'd love to work with me and the rest of the team, love Wikidata and want to learn, this might be the thing for you. More details are here: https://wikimedia.de/wiki/Internship_Product_Management Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

2 1

ArchCom RFC update
by Gabriel Wicke 24 Mar '16

24 Mar '16

Hi, please have a look at this week's summary of new and ongoing RFC discussions. There are several new RFCs, and some existing ones are moving close to a decision. No RFCs were decided finally this week. Because of the parallel Hackathon, no IRC discussion is scheduled for next week. Gabriel New RFCs: T130663 WIP RFC: Reference API requirements and options <https://phabricator.wikimedia.org/T130663> (Timo): API access and component markup for references; focus on gathering use cases / requirements. T122942 RFC: Support language variants in the REST API <https://phabricator.wikimedia.org/T122942> (Gabriel): Different options for supporting languange variant selection in the REST API. Needed for languages like Chinese. T122825 Service Ownership and Maintenance <https://phabricator.wikimedia.org/T122825> (Gabriel): Ownership and minimum maintenance requirements for production services. Strongly driven by unclear ownership of OCG (PDF renderer). T39902 RFC: Implement rendering of redlinks (in a post-processor?) <https://phabricator.wikimedia.org/T39902> (Gabriel): Solutions for highlighting links to non-existing pages in Parsoid HTML. Main question is preprocessing vs. separate metadata processed on client. T130528 RFC: PSR-6 Cache interface in Mediawiki core <https://phabricator.wikimedia.org/T130528> (No shepherd): Exploring use of standard PHP cache interface. Today's IRC session: T124792 Service Locator for MediaWiki core <https://phabricator.wikimedia.org/T124792> (Daniel): Introduce a service locator (aka DI container) to allow code in mediaWiki core to make use of the Dependency Injection (DI) and Service Locator (SL) patterns. The discussion showed general support. Several participants expressed a desire to write more code with it before making a final call. Concrete suggestions on areas would be welcome. Tentative working group forming, aiming to discuss at Jerusalem Hackathon. Under discussion: T129435 RFC: drop support for running without mbstring <https://phabricator.wikimedia.org/T129435> (Gabriel): Very focused RFC by Max. Main question in discussion so far is whether polyfilling is worth it. Max reaching out to mediawiki-l. T108655 Standardise on how to access/register JavaScript interfaces <https://phabricator.wikimedia.org/T108655> (Roan): No update since last week, I need to split this task but I haven’t had time to yet. Last week’s update: Considering to split out contentious part (file-based require, or something like it; to support embedding libraries), move forward on less controversial part (basic module-name-based require infrastructure) T18691 RFC: Section headings should have a clickable anchor <https://phabricator.wikimedia.org/T18691> (Timo): Under discussion with Volker and Frontend Standards Group. Volker and team to collect different benefits and concerns to determine whether this is generally a desirable feature. And to explore other conceptually different solutions to the underlying use case of “sharing a link to a section” (e.g. a better table of contents, or live address bar). T124504 Transition WikiDev '16 working areas into working groups <https://phabricator.wikimedia.org/T124504> (RobLa): No concrete progress; MZMcBride advocates for organic growth. T128351 RfC: Notifications in core <https://phabricator.wikimedia.org/T128351> (Brion): No movement last week. Needs clarification of interfaces & scope as follow-up from IRC meeting. T66214 Use content hash based image / thumb URLs & define an official thumb API <https://phabricator.wikimedia.org/T66214> (Brion): Clarified requirements & priorities in last week's IRC discussion. Needs update to reflect discussion. T118517 [RFC] Use <figure> for media <https://phabricator.wikimedia.org/T118517> (Brion): Revisit soon. T88596 Improving extension management <https://phabricator.wikimedia.org/T88596> (Daniel): Discussion is picking up again, patch for review. T113034 RFC: Overhaul Interwiki map, unify with Sites and WikiMap <https://phabricator.wikimedia.org/T113034> (Daniel): Has been discussed before, needs somebody to actually take this on. T114444 [RFC] Introduce notion of DOM scopes in wikitext <https://phabricator.wikimedia.org/T114444> (Tim): Active related discussion and prototyping at Balanced templates <https://phabricator.wikimedia.org/T114445> and Hygienic transclusions for WYSIWYG, incremental parsing & composition: Options and trade-offs <https://phabricator.wikimedia.org/T130567>.

2 1

Tech Talk: New Readership Data: March 18th
by Rachel Farrand 24 Mar '16

24 Mar '16

Please join for the following tech talk: *Tech Talk**:* New readership data: Some things we've been learning recently about how Wikipedia is read *Presenter:* Tilman Bayer *Date:* March 18th, 2016 *Time: *18:00 UTC <http://www.timeanddate.com/worldclock/fixedtime.html?msg=Tech+Talk%3A+New+r…> Link to live YouTube stream <http://www.youtube.com/watch?v=Qo4XIzCJZVs> *IRC channel for questions/discussion:* #wikimedia-office *Summary: *This talk will highlight various recent insights and new sources of data on how readers read Wikipedia, going beyond the familiar pageview numbers (that tell us which topics are popular and how overall traffic is developing, but not e.g. which parts of articles are being read). While we are still only beginning to understand some of these aspects, we now know more than a year or two ago. The presentation is centered around data analysis done by the Reading team, but will also include findings by other WMF teams and by external researchers.

2 2

Review Service Locator / DI Container proposal (upcoming RFC discussion on March 23)
by Daniel Kinzler 24 Mar '16

24 Mar '16

Hi all! Over the last couple of months, I have worked on introducing a dependency injection mechanism into MediaWiki core (don't fear, no auto-wiring). My proposal is described in detail at <https://phabricator.wikimedia.org/T124792> (yea, TL;DR - just read the top and search the rest if you have a question). Before we discuss this again on IRC at the RFC meeting on Wednesday (March 23, 2pm PST / 22:00 CEST due to daylight confusion), I would like to invite you to review the proposal as well as the patches that are up on gerrit. In particular, any feedback would be appreciated on: * Introduce top level service locator <https://gerrit.wikimedia.org/r/#/c/264403/29>. * Allow reset of global services <https://gerrit.wikimedia.org/r/#/c/270020/> * WIP: Make storage layer services injectable. <https://gerrit.wikimedia.org/r/#/c/267692/> Perhaps also have a look at the documentation included in the change, in particular the migration part: <https://gerrit.wikimedia.org/r/#/c/264403/29/docs/injection.txt> Before commenting on design choices on gerrit, please have a look at T124792 and see whether I have written something about the issue in question there. I would like to focus conceptual discussion on the RFC ticket on phabricator, rather than on gerrit. On gerrit, we can talk about the implementation. I very much want this to move forward. Perhaps we can even get the first bits of this merged at the hackathon. So, criticize away! Thanks for your help! -- daniel PS: phabricator event page (still blank, we'll fix that soon): <https://phabricator.wikimedia.org/E66/27>

1 2

Google Summer of Code 2016 portal now accepting applications
by Tony Thomas 24 Mar '16

24 Mar '16

Hello all, Google announced the start of accepting proposals for GSoC 2016 few hours ago. Interested and eligible candidates should submit their proposals at http://g.co/gsoc before the deadline of Friday, March 25 at 19:00 UTC. Wikimedia evaluates your proposal Phabricator task, but it is required that you have a copy of the same in the GSoC portal too, to make sure it gets a slot ( if eligible ). By March 25th, every possible application should have *2 mentors* connected with it, and should have a proposal copy in Phabricator, as well as the GSoC portal. Please make sure you mention the phab task details in your proposal, for convenience. If you are planning to apply, you should be looking at Life_of_a_successful_project#Coming_up_with_a_proposal <https://www.mediawiki.org/wiki/Outreach_programs/Life_of_a_successful_proje…> As of today, we have *8* projects featured for this round ( strong idea + 2 mentors connected ), and *13* projects missing one among the two mentors. Interested in mentoring ? see https://phabricator.wikimedia.org/tag/possible-tech-projects/ and add yourself as one. The Outreachy round May - August 2016 is open, with a deadline of *March 22, 2016 *and eligible applicants are advised to apply for *both* GSoC and Outreachy, so that the project can still make it, in case we are missing a slot with a strong applicant. Thinking of motivating someone in your locality to take part in ? Find flyers and presentations here <https://developers.google.com/open-source/gsoc/resources/media#logos_and_ar…> for GSoC 2016 round! Thanks, Tony Thomas <https://www.mediawiki.org/wiki/User:01tonythomas> Home <http://www.thomastony.me> | Blog <http://blog.thomastony.me> | ThinkFOSS <http://www.thinkfoss.com>

2 3

Re: [Wikitech-l] [Analytics] [Data Release] [Data Deprecation] [Analytics Dumps]
by Dan Andreescu 23 Mar '16

23 Mar '16

On Wed, Mar 23, 2016 at 1:06 PM, Federico Leva (Nemo) <nemowiki(a)gmail.com> wrote: > Dan Andreescu, 23/03/2016 15:58: > >> >> *Clean-up:* Analytics data on dumps was crammed into /other with >> unrelated datasets. We made a new page to receive current and future >> datasets [3] and linked to it from /other and /. Please let us know if >> anything there looks confusing or opaque and I'll be happy to clarify. >> > > I assume the old URLs will redirect to the new ones, right? > Good question, we didn't change any old URLs actually, so if you're trying to get to other/pagecounts-ez, other/pagecounts-raw and all that, they're all still there, just linked-to from /analytics. We did it this way because we figured people had scripts that depended on those URLs. We thought about moving and symlinking but it's probably unlikely that we'll ever be able to delete the other/** location. So mainly we just have a new page where we can do a better job of focusing on the analytics datasets.

1 0

Re: [Wikitech-l] [Data Release] [Data Deprecation] [Analytics Dumps]
by Dan Andreescu 23 Mar '16

23 Mar '16

cc-ing our friends in research and wikitech (sorry I forgot initially) We're happy to announce a few improvements to Analytics data releases on > dumps.wikimedia.org: > > * We are releasing a new dataset, an estimate of Unique Devices accessing > our projects [1] > * We are officially making available a better Pageviews dataset [2] > * We are deprecating two older pageview statistics datasets > * We moved Analytics data from /other to /analytics [3] > > Details follow: > > > *Unique Devices:* Since 2009, the Wikimedia Foundation used comScore to > report data about unique web visitors. In January 2016, however, we > decided to stop reporting comScore numbers [4] because of certain > limitations in the methodology, these limitations translated into > misreported mobile usage. We are now ready to replace comscore numbers with > the Unique Devices Dataset [5][1]. While unique devices does not equal > unique visitors, it is a good proxy for that metric, meaning that a major > increase in the number of unique devices is likely to come from an increase > in distinct users. We understand that counting uniques raises fairly big > privacy concerns and we use a very private conscious way to count unique > devices, it does not include any cookie by which your browser history can > be tracked [6]. > > We invite you to explore this new dataset and hope it’s helpful for the > Wikimedia community in better understanding our projects. This data can > help measurethe reach of wikimedia projects on the web. > > *Pageviews:* This [2] is the best quality data available for counting the > number of pageviews our projects receive at the article and project level. > We've upgraded from pagecounts-raw to pagecounts-all-sites, and now to > pageviews, in order to filter out more spider traffic and measure something > closer to what we think is a real user viewing content. A short history > might be useful: > > * pagecounts-raw: was maintained by Domas Mituzas originally and taken > over by the analytics team. It was and still is the most used dataset, > though it has some majore problems. It does not count access to the mobile > site, it does not filter out spider or bot traffic, and it suffers from > unknown loss due to logging infrastructure limitations. > * pagecounts-all-sites: uses the same pageview definition as > pagecounts-raw, and so also does not filter out spider or bot traffic. But > it does include access to mobile and zero sites, and is built on a more > reliable logging infrastructure. > * pagecounts-ez: is derived from the best data available at the time. > So until December 2015, it was based on pagecounts-raw and > pagecounts-all-sites, and now it's based on pageviews. This dataset is > great because it compresses very large files without losing any > information, still providing hourly page and project level statistics. > > So the new dataset, pageviews, is what's behind our pageview API and is > now available in static files for bulk download back to May 2015. But the > multiple ways to download pageview data is confusing for consumers, so > we're keeping only pageviews and pagecounts-ez and deprecating the other > two. If you'd like to read more about the current pageview definition, > details are on the research page [7]. > > *Deprecating:* We are deprecating the pagecounts-raw and > pagecounts-all-sites datasets in May 2016 (discussion here: > https://phabricator.wikimedia.org/T130656 ). This data suffers from many > artifacts, lack of mobile data, and/or infrastructure problems, and so is > not comparable to the new way we track pageviews. It will remain here > because we have historical data that may be useful, but it will not be > maintained or updated beyond May 2016. > > *Clean-up:* Analytics data on dumps was crammed into /other with > unrelated datasets. We made a new page to receive current and future > datasets [3] and linked to it from /other and /. Please let us know if > anything there looks confusing or opaque and I'll be happy to clarify. > > > [1] http://dumps.wikimedia.org/other/unique_devices > [2] http://dumps.wikimedia.org/other/pageviews > [3] http://dumps.wikimedia.org/analytics/ > [4] https://meta.wikimedia.org/wiki/ComScore/Announcement > [5] https://meta.wikimedia.org/wiki/Research:Unique_Devices > [6] > https://meta.wikimedia.org/wiki/Research:Unique_Devices#How_do_we_count_uni… > [7] https://meta.wikimedia.org/wiki/Research:Page_view >

1 0

← Newer
1
2
3
4
5
6
7
...
11
Older →

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l March 2016