Discovery June 2016

discovery@lists.wikimedia.org

12 participants
21 discussions

Round 2: Wikipedia.org portal survey results
by Deborah Tankersley 30 Jun '16

30 Jun '16

Hello, The Discovery Portal Team recently completed a second running of a survey on the Wikipedia.org portal, using a non-intrusive banner, from June 13 through June 28, 2016. Our goal was to target those visitors that arrive at the portal page but don't actually do anything, as these "non-action" page visits account for about 43% of the portal's daily pageviews. We also wanted to see if we could get additional respondents to the survey by using a different bucketing system and running it for a longer timeframe. We wanted to see if the results would indicate that many visitors have wikipedia.org set as their browser's default home page <https://en.wikipedia.org/wiki/Home_page>. Our thinking was that the non-action pageviews are occurring because each time a visitor opened up their preferred browser, their homepage would display the Wikipedia portal page but the user simply goes onto a different site without using the portal first. The resulting data was very similar to our first running of the survey in May 2016, as shown in the attached graphs - displaying the breakdown of the June survey [1] and also displaying a comparison of May vs June survey results [2]. The results overall were that most visitors had bookmarked the site in their browser and the second most popular selection was that they typed 'wikipedia' to get to the portal. I've also included a sanitized pdf [3] of all the *'other'* answers to our survey, for those curious about the content of the free text answers that were collected. Some of the responses are fairly humorous *"I took the 5 oclock train"* and some were a bit more serious *"I typed in the URL of the website. Not the word "wikipedia" but "http://www.wikipedia.com <http://www.wikipedia.com/>". It's how I **normally navigate to a website I know the name of." *and we received plenty of love too *"**I love ❤ your site"*. Our goal of trying to get those non-action visitors to do an action (namely to take our quick survey) probably failed miserably but, on the other hand, we received a lot of other really good information that will be quite useful to our future updates to the Wikipedia portal. Cheers from the Wikimedia Discovery Portal Team, Deb, Jan and Mikhail [1] Survey results (graph) <https://commons.wikimedia.org/wiki/File:Graph_-_Screenshot_of_a_Wikipedia.o…> [2] Comparison of May vs June 2016 survey results <https://commons.wikimedia.org/wiki/File:Graph_-_Comparision_of_two_Wikipedi…> [3] Text answers from survey <https://commons.wikimedia.org/wiki/File:PDF_-_Text_answers_from_a_Wikipedia…> -- Deb Tankersley Product Manager, Discovery IRC: debt Wikimedia Foundation

2 1

Typing on the Wrong Keyboard in Russian and English
by Trey Jones 25 Jun '16

25 Jun '16

I spent today looking at identifying and converting queries typed on the wrong keyboard on the English and Russian Wikipedias. *Highlights* Looking for mis-keyboarded queries in the "right" character set (ie., Latin on English Wikipedia or Cyrillic on Russian Wikipedia) can explain some gibberish queries and give some improvement in results, but it's very expensive because there are so many candidate queries. Looking for mis-keyboarded queries in the "wrong" character set (ie., Cyrillic on English Wikipedia or Latin on Russian Wikipedia) can explain a lot of gibberish queries and give better results, especially on Russian Wikipedia, where possibly more than 1% of queries are accidentally typed on the wrong keyboard! Limiting the scope to only zero-result queries or perhaps poorly performing (fewer than three results) queries could be computationally less expensive and much more effective! More details are available <https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Typing_on_the_Wrong_…> . —Trey Trey Jones Software Engineer, Discovery Wikimedia Foundation

2 1

Discovery Weekly Update for the week starting 2016-06-20
by Chris Koerner 24 Jun '16

24 Jun '16

Hello again, >From the Wikimedia Foundation Discovery department here's this week's updates. * Trey took a quick look at detecting when users on English and Russian Wikipedias were using the wrong keyboard. It looks like more than 1% of Russian Wikipedia queries may be in Russian but typed on the English keyboard! [1] * We're still discussing how to handle query-final question marks. Comments and suggestions welcome on the Phab ticket. [2] * Testing software upgrade for logstash.wikimedia.org at kibana4.wmflabs.org * Installed 16 new servers in our elasticsearch cluster to replace the older ones * Katie, Deb and Jan are in Esino Lario for Wikimania 2016 along with many other WMF members and about 1,000 of our closest friends all sharing in the sun, fun and humidity. [3] [4] ** The Hackathon was well attended: about 160 people all collaborating together for two days; there will be a showcase of some of the efforts tomorrow night. [5] [6] ** Friday, Saturday and Sunday are all about the main program and it's gonna be grand! [7] *** Best highlight so far (to be fair, it's only end of day Friday as I'm writing this): Simone Cortesi talking about OSM and some of the great work that the Discovery Interactive team has done to get maps into the wiki projects, specifically Wikivoyage. [8] * <mapframe/maplink> no longer need zoom and location - it can figure out the location on its own if missing, based on the geojson data. * Most of the geoshapes backend code is now in place, and should be available soon - it will allow both graphs and maps to get "geoshapes" - location outlines (e.g. state of New York, or a park, or the whole country - as a geometry, based on Wikidata ID. [1] https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Typing_on_the_Wrong_… [2] https://phabricator.wikimedia.org/T133711 [3] http://umap.openstreetmap.fr/it/map/wikimaniaesino_45461#17/45.99447/9.33327 [4] https://wikimania2016.wikimedia.org/wiki/Main_Page [5] https://wikimania2016.wikimedia.org/wiki/Hackathon/Program [6] https://wikimania2016.wikimedia.org/wiki/Hackathon/Program#Showcase [7] https://wikimania2016.wikimedia.org/wiki/Programme [8] https://wikimania2016.wikimedia.org/wiki/User_digest/Projects:_Wikisource,_… ---- Feedback and suggestions on this weekly update are welcome! The full update, and archive of past updates, can be found on Mediawiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates -- Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation

1 0

Redirects not showing in search suggestions
by Pine W 24 Jun '16

24 Jun '16

Hi Discovery, I accidentally discovered ( ;) ) that on ENWP, both mobile and desktop, redirects aren't included in search suggestions. This a bit important when ENWP has nothing on our main page about Brexit, and when I try to search for Brexit the lack of a search result implies to me that we have no such article. Actually, we do have an article under a much longer name, and "Brexit" is a redirect. So I'm wondering if it would be good to include redirects in search results, perhaps appended with "(Redirects to <article name>)". What do you think? Thanks, Pine

4 3

Re: [discovery] [Ops] Index management for Maps
by Guillaume Lederrey 22 Jun '16

22 Jun '16

On Wed, Jun 22, 2016 at 10:15 AM, Alexandros Kosiaris <akosiaris(a)wikimedia.org> wrote: > From my understanding back then, those indices are > kartotherian/tilerator specific. So bundling any schema changes that > suite of software requires with it makes sense. Anything non > kartotherian/tilerator specific we could/should try to upstream it of > course. I have the same understanding. > On Wed, Jun 22, 2016 at 9:40 AM, Giuseppe Lavagetto > <glavagetto(a)wikimedia.org> wrote: >> On Tue, Jun 21, 2016 at 9:02 PM, Guillaume Lederrey >> <glederrey(a)wikimedia.org> wrote: >>> Hello! >>> >>> I need some feedback from my fellow Ops on how to manage indexes on Maps. >>> >>> Context: >>> >>> Maps imports OpenStreetMap data in a Postgresql database. This import >>> is done with osm2pgsql, which takes care of creating the schema, >>> populating tables and creating a few indexes. >>> >>> Some additional indexes are required to support the specific >>> functionalities of Tilerator. So far, those indexes have been created >>> manually and have not been tracked. >>> >>> The enhancements to the schema created by osm2pgsql are minor (a few >>> index and functions), so we probably need a lightweight solution. >>> >>> Proposal: >>> >>> A few idempotent scripts are versionned in osm-bright.tm2source [1]. >>> Those scripts are executed after review by Ops, at the request of the >>> project. We don't use a full schema migration process, because at this >>> point there isn't really a need for it on this project. >>> >>> Does this look reasonable to you? Feedback welcomed, shoot me down if >>> you have to... >> >> My very naive first level suggestion would be to run a wrapper around >> osm2pgsql and (re-)add the indexes via this wrapper once osm2pgsql has >> completed successfully. osm2pgsql is creating the schema on initial data load. Creating additional index / functions should be done at that time. We have a script that takes care of this initial data load and some operations that needs to happen at the same time (work in progress [1]). But after initial data load, we might (and will) still need to evolve the index and need to track those evolutions. On the media wiki side, we have a more elaborate solution, with a real schema migration process. I think that in the case of Maps this is not really needed as the changes to the initial schema are fairly trivial. Idempotent scripts are sufficient IMHO. I'd like validation on this specific point, but at least I did not see anyone jump at the mention of idempotent scripts ... [1] https://gerrit.wikimedia.org/r/#/c/293105/ >> Also, if such schema optimizations have general interest, it would be >> nice to try to get them integrated into osm2pgsql. >> >> But I can be missing some context here! >> >> Cheers >> Giuseppe >> -- >> Giuseppe Lavagetto, Ph.d. >> Senior Technical Operations Engineer, Wikimedia Foundation >> >> _______________________________________________ >> Ops mailing list >> Ops(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/ops > > > > -- > Alexandros Kosiaris <akosiaris(a)wikimedia.org> -- Guillaume Lederrey Operations Engineer, Discovery Wikimedia Foundation

1 0

Index management for Maps
by Guillaume Lederrey 22 Jun '16

22 Jun '16

Hello! I need some feedback from my fellow Ops on how to manage indexes on Maps. Context: Maps imports OpenStreetMap data in a Postgresql database. This import is done with osm2pgsql, which takes care of creating the schema, populating tables and creating a few indexes. Some additional indexes are required to support the specific functionalities of Tilerator. So far, those indexes have been created manually and have not been tracked. The enhancements to the schema created by osm2pgsql are minor (a few index and functions), so we probably need a lightweight solution. Proposal: A few idempotent scripts are versionned in osm-bright.tm2source [1]. Those scripts are executed after review by Ops, at the request of the project. We don't use a full schema migration process, because at this point there isn't really a need for it on this project. Does this look reasonable to you? Feedback welcomed, shoot me down if you have to... Thanks for your help! Guillaume [1] https://github.com/kartotherian/osm-bright.tm2source/tree/master/sql -- Guillaume Lederrey Operations Engineer, Discovery Wikimedia Foundation

2 1

Re: [discovery] [Ops] Kitchen Hackathon in Lausanne
by Deborah Tankersley 20 Jun '16

20 Jun '16

Meet up in Warsaw or Dublin? :) I've got such good food and drink to try in my two weeks across the pond! w00t! -- Deb Tankersley Product Manager, Discovery IRC: debt Wikimedia Foundation On Mon, Jun 20, 2016 at 9:12 AM, Antoine Boegli <antoine.boegli(a)gmail.com> wrote: > Next time, perhaps we'll try chocolate aquafaba mousse for the dessert... > Or something stranger... Mmmh.... don't know for the moment... > > 2016-06-20 17:08 GMT+02:00 Guillaume Lederrey <glederrey(a)wikimedia.org>: > >> @Deb: I told you that you should stop in Lausanne on the way back from >> Wikimania... >> >> On Mon, Jun 20, 2016 at 5:07 PM, Antoine Boegli >> <antoine.boegli(a)gmail.com> wrote: >> > Oh ! I forgot the artichoke tapenade ! shame on me ! >> > >> > We also had some carrots, cucumbers and tomatoes with artichoke >> tapenade :-) >> > >> > 2016-06-20 17:05 GMT+02:00 Antoine Boegli <antoine.boegli(a)gmail.com>: >> >> >> >> Yes, there was food ! >> >> >> >> Guillaume treated us with pasta and pesto, and there was also salad, >> >> beers, cafe, tea, and dessert (vanilia chia cream with stewed rhubarb) >> >> >> >> Almost everything was house made, so it was a bit of a food hackathon >> too >> >> :-) >> >> >> >> >> >> >> >> >> >> 2016-06-20 16:40 GMT+02:00 Deborah Tankersley < >> dtankersley(a)wikimedia.org>: >> >>> >> >>> That's awesome, but, was there food too? :) >> >>> >> >>> Thanks for all your hard work, good job! >> >>> >> >>> >> >>> -- >> >>> Deb Tankersley >> >>> Product Manager, Discovery >> >>> IRC: debt >> >>> Wikimedia Foundation >> >>> >> >>> On Mon, Jun 20, 2016 at 5:33 AM, David Causse <dcausse(a)wikimedia.org> >> >>> wrote: >> >>>> >> >>>> That's certainly the most productive kitchen I've ever seen! >> >>>> >> >>>> Thank you everyone! >> >>>> >> >>>> >> >>>> Le 20/06/2016 12:55, Guillaume Lederrey a écrit : >> >>>>> >> >>>>> Hello all! >> >>>>> >> >>>>> I had a few friends coming over this Sunday for a mini Hackathon in >> my >> >>>>> kitchen. The goal was mainly to introduce them to what we do around >> >>>>> Wikipedia, have fun, exchange ideas and maybe do actual work on a >> few >> >>>>> tickets. Quick summary of what we did: >> >>>>> >> >>>>> * Fix a few puppet unit tests [1][2] >> >>>>> * Try to fix sending logs to logstash from elasticsearch [3][4]. We >> >>>>> could not reproduce the issue with security manager with a local >> >>>>> installation of elasticsearch and I did not prepare access to >> >>>>> deployment-prep where we could have validated the issue. >> >>>>> * Added a few tests to our elasticsearch admin tool (estool) [5] >> >>>>> * Refactoring of Cassandra monitoring [6][7] (thanks to Luca for the >> >>>>> fast code review!). This one prompted a great conversation. It was >> >>>>> fairly unclear to us how to best integrate monitoring in a DRY way, >> >>>>> while keeping the core of the Cassandra module free from external >> >>>>> dependencies. We are all used to have a technical module >> >>>>> (puppet-cassandra) that only manage core Cassandra and a higher >> level >> >>>>> module (puppet-wmf_cassandra) that configure Cassandra in the >> context >> >>>>> of WMF, with the all peripheral fonctions (monitoring, firewall, >> >>>>> logging, ...). This higher level abstraction kind of exists in the >> >>>>> roles, but multiple roles might use Cassandra, and might lead to >> some >> >>>>> duplication. The change [6] is probably more the start of a >> >>>>> conversation than something that can be merged as is. >> >>>>> * We tried to have a go at adding unit tests to Kartotherian [8], >> but >> >>>>> ran into an issue with mapnik and were not able to build the >> project. >> >>>>> None of us had any experience with Node development or Kartotherian. >> >>>>> We moved to other stuff. >> >>>>> >> >>>>> The day was fun! Thanks to all who joined! Special +1 to Mara who >> put >> >>>>> a lot of energy in learning Python for the first time (and >> >>>>> succeeded!). >> >>>>> >> >>>>> I'll try to followup on the changed opened during that day, make >> sure >> >>>>> that those contributions, however small, are not forgotten. >> >>>>> >> >>>>> We will be back! >> >>>>> >> >>>>> MrG >> >>>>> >> >>>>> [1] https://gerrit.wikimedia.org/r/#/c/295127/ >> >>>>> [2] https://gerrit.wikimedia.org/r/#/c/295130/ >> >>>>> [3] https://gerrit.wikimedia.org/r/#/c/295129/ >> >>>>> [4] https://phabricator.wikimedia.org/T136696 >> >>>>> [5] https://gerrit.wikimedia.org/r/#/c/290765/ >> >>>>> [6] https://gerrit.wikimedia.org/r/#/c/295123/ >> >>>>> [7] https://phabricator.wikimedia.org/T137422 >> >>>>> [8] https://phabricator.wikimedia.org/T111950 >> >>>>> >> >>>> >> >>>> >> >>>> _______________________________________________ >> >>>> discovery mailing list >> >>>> discovery(a)lists.wikimedia.org >> >>>> https://lists.wikimedia.org/mailman/listinfo/discovery >> >>> >> >>> >> >> >> >> >> >> >> >> -- >> >> Antoine Boegli >> >> software engineer & linux expert >> > >> > >> > >> > >> > -- >> > Antoine Boegli >> > software engineer & linux expert >> >> >> >> -- >> Guillaume Lederrey >> Operations Engineer, Discovery >> Wikimedia Foundation >> > > > > -- > Antoine Boegli > software engineer & linux expert >

1 0

Re: [discovery] [Ops] Kitchen Hackathon in Lausanne
by Guillaume Lederrey 20 Jun '16

20 Jun '16

@Deb: I told you that you should stop in Lausanne on the way back from Wikimania... On Mon, Jun 20, 2016 at 5:07 PM, Antoine Boegli <antoine.boegli(a)gmail.com> wrote: > Oh ! I forgot the artichoke tapenade ! shame on me ! > > We also had some carrots, cucumbers and tomatoes with artichoke tapenade :-) > > 2016-06-20 17:05 GMT+02:00 Antoine Boegli <antoine.boegli(a)gmail.com>: >> >> Yes, there was food ! >> >> Guillaume treated us with pasta and pesto, and there was also salad, >> beers, cafe, tea, and dessert (vanilia chia cream with stewed rhubarb) >> >> Almost everything was house made, so it was a bit of a food hackathon too >> :-) >> >> >> >> >> 2016-06-20 16:40 GMT+02:00 Deborah Tankersley <dtankersley(a)wikimedia.org>: >>> >>> That's awesome, but, was there food too? :) >>> >>> Thanks for all your hard work, good job! >>> >>> >>> -- >>> Deb Tankersley >>> Product Manager, Discovery >>> IRC: debt >>> Wikimedia Foundation >>> >>> On Mon, Jun 20, 2016 at 5:33 AM, David Causse <dcausse(a)wikimedia.org> >>> wrote: >>>> >>>> That's certainly the most productive kitchen I've ever seen! >>>> >>>> Thank you everyone! >>>> >>>> >>>> Le 20/06/2016 12:55, Guillaume Lederrey a écrit : >>>>> >>>>> Hello all! >>>>> >>>>> I had a few friends coming over this Sunday for a mini Hackathon in my >>>>> kitchen. The goal was mainly to introduce them to what we do around >>>>> Wikipedia, have fun, exchange ideas and maybe do actual work on a few >>>>> tickets. Quick summary of what we did: >>>>> >>>>> * Fix a few puppet unit tests [1][2] >>>>> * Try to fix sending logs to logstash from elasticsearch [3][4]. We >>>>> could not reproduce the issue with security manager with a local >>>>> installation of elasticsearch and I did not prepare access to >>>>> deployment-prep where we could have validated the issue. >>>>> * Added a few tests to our elasticsearch admin tool (estool) [5] >>>>> * Refactoring of Cassandra monitoring [6][7] (thanks to Luca for the >>>>> fast code review!). This one prompted a great conversation. It was >>>>> fairly unclear to us how to best integrate monitoring in a DRY way, >>>>> while keeping the core of the Cassandra module free from external >>>>> dependencies. We are all used to have a technical module >>>>> (puppet-cassandra) that only manage core Cassandra and a higher level >>>>> module (puppet-wmf_cassandra) that configure Cassandra in the context >>>>> of WMF, with the all peripheral fonctions (monitoring, firewall, >>>>> logging, ...). This higher level abstraction kind of exists in the >>>>> roles, but multiple roles might use Cassandra, and might lead to some >>>>> duplication. The change [6] is probably more the start of a >>>>> conversation than something that can be merged as is. >>>>> * We tried to have a go at adding unit tests to Kartotherian [8], but >>>>> ran into an issue with mapnik and were not able to build the project. >>>>> None of us had any experience with Node development or Kartotherian. >>>>> We moved to other stuff. >>>>> >>>>> The day was fun! Thanks to all who joined! Special +1 to Mara who put >>>>> a lot of energy in learning Python for the first time (and >>>>> succeeded!). >>>>> >>>>> I'll try to followup on the changed opened during that day, make sure >>>>> that those contributions, however small, are not forgotten. >>>>> >>>>> We will be back! >>>>> >>>>> MrG >>>>> >>>>> [1] https://gerrit.wikimedia.org/r/#/c/295127/ >>>>> [2] https://gerrit.wikimedia.org/r/#/c/295130/ >>>>> [3] https://gerrit.wikimedia.org/r/#/c/295129/ >>>>> [4] https://phabricator.wikimedia.org/T136696 >>>>> [5] https://gerrit.wikimedia.org/r/#/c/290765/ >>>>> [6] https://gerrit.wikimedia.org/r/#/c/295123/ >>>>> [7] https://phabricator.wikimedia.org/T137422 >>>>> [8] https://phabricator.wikimedia.org/T111950 >>>>> >>>> >>>> >>>> _______________________________________________ >>>> discovery mailing list >>>> discovery(a)lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/discovery >>> >>> >> >> >> >> -- >> Antoine Boegli >> software engineer & linux expert > > > > > -- > Antoine Boegli > software engineer & linux expert -- Guillaume Lederrey Operations Engineer, Discovery Wikimedia Foundation

1 0

Re: [discovery] [Ops] Kitchen Hackathon in Lausanne
by Deborah Tankersley 20 Jun '16

20 Jun '16

oh my gosh, you're making me drool....it all sounds so good! :) -- Deb Tankersley Product Manager, Discovery IRC: debt Wikimedia Foundation On Mon, Jun 20, 2016 at 9:05 AM, Antoine Boegli <antoine.boegli(a)gmail.com> wrote: > Yes, there was food ! > > Guillaume treated us with pasta and pesto, and there was also salad, > beers, cafe, tea, and dessert (vanilia chia cream with stewed rhubarb) > > Almost everything was house made, so it was a bit of a food hackathon too > :-) > > > > > 2016-06-20 16:40 GMT+02:00 Deborah Tankersley <dtankersley(a)wikimedia.org>: > >> That's awesome, but, was there food too? :) >> >> Thanks for all your hard work, good job! >> >> >> -- >> Deb Tankersley >> Product Manager, Discovery >> IRC: debt >> Wikimedia Foundation >> >> On Mon, Jun 20, 2016 at 5:33 AM, David Causse <dcausse(a)wikimedia.org> >> wrote: >> >>> That's certainly the most productive kitchen I've ever seen! >>> >>> Thank you everyone! >>> >>> >>> Le 20/06/2016 12:55, Guillaume Lederrey a écrit : >>> >>>> Hello all! >>>> >>>> I had a few friends coming over this Sunday for a mini Hackathon in my >>>> kitchen. The goal was mainly to introduce them to what we do around >>>> Wikipedia, have fun, exchange ideas and maybe do actual work on a few >>>> tickets. Quick summary of what we did: >>>> >>>> * Fix a few puppet unit tests [1][2] >>>> * Try to fix sending logs to logstash from elasticsearch [3][4]. We >>>> could not reproduce the issue with security manager with a local >>>> installation of elasticsearch and I did not prepare access to >>>> deployment-prep where we could have validated the issue. >>>> * Added a few tests to our elasticsearch admin tool (estool) [5] >>>> * Refactoring of Cassandra monitoring [6][7] (thanks to Luca for the >>>> fast code review!). This one prompted a great conversation. It was >>>> fairly unclear to us how to best integrate monitoring in a DRY way, >>>> while keeping the core of the Cassandra module free from external >>>> dependencies. We are all used to have a technical module >>>> (puppet-cassandra) that only manage core Cassandra and a higher level >>>> module (puppet-wmf_cassandra) that configure Cassandra in the context >>>> of WMF, with the all peripheral fonctions (monitoring, firewall, >>>> logging, ...). This higher level abstraction kind of exists in the >>>> roles, but multiple roles might use Cassandra, and might lead to some >>>> duplication. The change [6] is probably more the start of a >>>> conversation than something that can be merged as is. >>>> * We tried to have a go at adding unit tests to Kartotherian [8], but >>>> ran into an issue with mapnik and were not able to build the project. >>>> None of us had any experience with Node development or Kartotherian. >>>> We moved to other stuff. >>>> >>>> The day was fun! Thanks to all who joined! Special +1 to Mara who put >>>> a lot of energy in learning Python for the first time (and >>>> succeeded!). >>>> >>>> I'll try to followup on the changed opened during that day, make sure >>>> that those contributions, however small, are not forgotten. >>>> >>>> We will be back! >>>> >>>> MrG >>>> >>>> [1] https://gerrit.wikimedia.org/r/#/c/295127/ >>>> [2] https://gerrit.wikimedia.org/r/#/c/295130/ >>>> [3] https://gerrit.wikimedia.org/r/#/c/295129/ >>>> [4] https://phabricator.wikimedia.org/T136696 >>>> [5] https://gerrit.wikimedia.org/r/#/c/290765/ >>>> [6] https://gerrit.wikimedia.org/r/#/c/295123/ >>>> [7] https://phabricator.wikimedia.org/T137422 >>>> [8] https://phabricator.wikimedia.org/T111950 >>>> >>>> >>> >>> _______________________________________________ >>> discovery mailing list >>> discovery(a)lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/discovery >>> >> >> > > > -- > Antoine Boegli > software engineer & linux expert >

1 0

Kitchen Hackathon in Lausanne
by Guillaume Lederrey 20 Jun '16

20 Jun '16

Hello all! I had a few friends coming over this Sunday for a mini Hackathon in my kitchen. The goal was mainly to introduce them to what we do around Wikipedia, have fun, exchange ideas and maybe do actual work on a few tickets. Quick summary of what we did: * Fix a few puppet unit tests [1][2] * Try to fix sending logs to logstash from elasticsearch [3][4]. We could not reproduce the issue with security manager with a local installation of elasticsearch and I did not prepare access to deployment-prep where we could have validated the issue. * Added a few tests to our elasticsearch admin tool (estool) [5] * Refactoring of Cassandra monitoring [6][7] (thanks to Luca for the fast code review!). This one prompted a great conversation. It was fairly unclear to us how to best integrate monitoring in a DRY way, while keeping the core of the Cassandra module free from external dependencies. We are all used to have a technical module (puppet-cassandra) that only manage core Cassandra and a higher level module (puppet-wmf_cassandra) that configure Cassandra in the context of WMF, with the all peripheral fonctions (monitoring, firewall, logging, ...). This higher level abstraction kind of exists in the roles, but multiple roles might use Cassandra, and might lead to some duplication. The change [6] is probably more the start of a conversation than something that can be merged as is. * We tried to have a go at adding unit tests to Kartotherian [8], but ran into an issue with mapnik and were not able to build the project. None of us had any experience with Node development or Kartotherian. We moved to other stuff. The day was fun! Thanks to all who joined! Special +1 to Mara who put a lot of energy in learning Python for the first time (and succeeded!). I'll try to followup on the changed opened during that day, make sure that those contributions, however small, are not forgotten. We will be back! MrG [1] https://gerrit.wikimedia.org/r/#/c/295127/ [2] https://gerrit.wikimedia.org/r/#/c/295130/ [3] https://gerrit.wikimedia.org/r/#/c/295129/ [4] https://phabricator.wikimedia.org/T136696 [5] https://gerrit.wikimedia.org/r/#/c/290765/ [6] https://gerrit.wikimedia.org/r/#/c/295123/ [7] https://phabricator.wikimedia.org/T137422 [8] https://phabricator.wikimedia.org/T111950 -- Guillaume Lederrey Operations Engineer, Discovery Wikimedia Foundation

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Discovery June 2016