Hello,
The Discovery Portal Team recently completed a second running of a survey
on the Wikipedia.org portal, using a non-intrusive banner, from June 13
through June 28, 2016.
Our goal was to target those visitors that arrive at the portal page but
don't actually do anything, as these "non-action" page visits account for
about 43% of the portal's daily pageviews. We also wanted to see if we
could get additional respondents to the survey by using a different
bucketing system and running it for a longer timeframe.
We wanted to see if the results would indicate that many visitors have
wikipedia.org set as their browser's default home page
<https://en.wikipedia.org/wiki/Home_page>. Our thinking was that the
non-action pageviews are occurring because each time a visitor opened up
their preferred browser, their homepage would display the Wikipedia portal
page but the user simply goes onto a different site without using the
portal first.
The resulting data was very similar to our first running of the survey in
May 2016, as shown in the attached graphs - displaying the breakdown of the
June survey [1] and also displaying a comparison of May vs June survey
results [2]. The results overall were that most visitors had bookmarked the
site in their browser and the second most popular selection was that they
typed 'wikipedia' to get to the portal.
I've also included a sanitized pdf [3] of all the *'other'* answers to our
survey, for those curious about the content of the free text answers that
were collected. Some of the responses are fairly humorous *"I took the 5
oclock train"* and some were a bit more serious *"I typed in the URL of the
website. Not the word "wikipedia" but "http://www.wikipedia.com
<http://www.wikipedia.com/>". It's how I **normally navigate to a website I
know the name of." *and we received plenty of love too *"**I love ❤ your
site"*.
Our goal of trying to get those non-action visitors to do an action (namely
to take our quick survey) probably failed miserably but, on the other hand,
we received a lot of other really good information that will be quite
useful to our future updates to the Wikipedia portal.
Cheers from the Wikimedia Discovery Portal Team,
Deb, Jan and Mikhail
[1] Survey results (graph)
<https://commons.wikimedia.org/wiki/File:Graph_-_Screenshot_of_a_Wikipedia.o…>
[2] Comparison of May vs June 2016 survey results
<https://commons.wikimedia.org/wiki/File:Graph_-_Comparision_of_two_Wikipedi…>
[3] Text answers from survey
<https://commons.wikimedia.org/wiki/File:PDF_-_Text_answers_from_a_Wikipedia…>
--
Deb Tankersley
Product Manager, Discovery
IRC: debt
Wikimedia Foundation
I spent today looking at identifying and converting queries typed on the
wrong keyboard on the English and Russian Wikipedias.
*Highlights*
Looking for mis-keyboarded queries in the "right" character set (ie., Latin
on English Wikipedia or Cyrillic on Russian Wikipedia) can explain some
gibberish queries and give some improvement in results, but it's very
expensive because there are so many candidate queries.
Looking for mis-keyboarded queries in the "wrong" character set (ie.,
Cyrillic on English Wikipedia or Latin on Russian Wikipedia) can explain a
lot of gibberish queries and give better results, especially on Russian
Wikipedia, where possibly more than 1% of queries are accidentally typed on
the wrong keyboard!
Limiting the scope to only zero-result queries or perhaps poorly performing
(fewer than three results) queries could be computationally less expensive
and much more effective!
More details are available
<https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Typing_on_the_Wrong_…>
.
—Trey
Trey Jones
Software Engineer, Discovery
Wikimedia Foundation
Hello again,
>From the Wikimedia Foundation Discovery department here's this week's
updates.
* Trey took a quick look at detecting when users on English and Russian
Wikipedias were using the wrong keyboard. It looks like more than 1% of
Russian Wikipedia queries may be in Russian but typed on the English
keyboard! [1]
* We're still discussing how to handle query-final question marks. Comments
and suggestions welcome on the Phab ticket. [2]
* Testing software upgrade for logstash.wikimedia.org at kibana4.wmflabs.org
* Installed 16 new servers in our elasticsearch cluster to replace the
older ones
* Katie, Deb and Jan are in Esino Lario for Wikimania 2016 along with many
other WMF members and about 1,000 of our closest friends all sharing in the
sun, fun and humidity. [3] [4]
** The Hackathon was well attended: about 160 people all collaborating
together for two days; there will be a showcase of some of the efforts
tomorrow night. [5] [6]
** Friday, Saturday and Sunday are all about the main program and it's
gonna be grand! [7]
*** Best highlight so far (to be fair, it's only end of day Friday as I'm
writing this): Simone Cortesi talking about OSM and some of the great work
that the Discovery Interactive team has done to get maps into the wiki
projects, specifically Wikivoyage. [8]
* <mapframe/maplink> no longer need zoom and location - it can figure out
the location on its own if missing, based on the geojson data.
* Most of the geoshapes backend code is now in place, and should be
available soon - it will allow both graphs and maps to get "geoshapes" -
location outlines (e.g. state of New York, or a park, or the whole country
- as a geometry, based on Wikidata ID.
[1]
https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Typing_on_the_Wrong_…
[2] https://phabricator.wikimedia.org/T133711
[3]
http://umap.openstreetmap.fr/it/map/wikimaniaesino_45461#17/45.99447/9.33327
[4] https://wikimania2016.wikimedia.org/wiki/Main_Page
[5] https://wikimania2016.wikimedia.org/wiki/Hackathon/Program
[6] https://wikimania2016.wikimedia.org/wiki/Hackathon/Program#Showcase
[7] https://wikimania2016.wikimedia.org/wiki/Programme
[8]
https://wikimania2016.wikimedia.org/wiki/User_digest/Projects:_Wikisource,_…
----
Feedback and suggestions on this weekly update are welcome!
The full update, and archive of past updates, can be found on Mediawiki.org:
https://www.mediawiki.org/wiki/Discovery/Status_updates
--
Yours,
Chris Koerner
Community Liaison - Discovery
Wikimedia Foundation
Hi Discovery,
I accidentally discovered ( ;) ) that on ENWP, both mobile and desktop,
redirects aren't included in search suggestions. This a bit important when
ENWP has nothing on our main page about Brexit, and when I try to search
for Brexit the lack of a search result implies to me that we have no such
article. Actually, we do have an article under a much longer name, and
"Brexit" is a redirect. So I'm wondering if it would be good to include
redirects in search results, perhaps appended with "(Redirects to <article
name>)". What do you think?
Thanks,
Pine
On Wed, Jun 22, 2016 at 10:15 AM, Alexandros Kosiaris
<akosiaris(a)wikimedia.org> wrote:
> From my understanding back then, those indices are
> kartotherian/tilerator specific. So bundling any schema changes that
> suite of software requires with it makes sense. Anything non
> kartotherian/tilerator specific we could/should try to upstream it of
> course.
I have the same understanding.
> On Wed, Jun 22, 2016 at 9:40 AM, Giuseppe Lavagetto
> <glavagetto(a)wikimedia.org> wrote:
>> On Tue, Jun 21, 2016 at 9:02 PM, Guillaume Lederrey
>> <glederrey(a)wikimedia.org> wrote:
>>> Hello!
>>>
>>> I need some feedback from my fellow Ops on how to manage indexes on Maps.
>>>
>>> Context:
>>>
>>> Maps imports OpenStreetMap data in a Postgresql database. This import
>>> is done with osm2pgsql, which takes care of creating the schema,
>>> populating tables and creating a few indexes.
>>>
>>> Some additional indexes are required to support the specific
>>> functionalities of Tilerator. So far, those indexes have been created
>>> manually and have not been tracked.
>>>
>>> The enhancements to the schema created by osm2pgsql are minor (a few
>>> index and functions), so we probably need a lightweight solution.
>>>
>>> Proposal:
>>>
>>> A few idempotent scripts are versionned in osm-bright.tm2source [1].
>>> Those scripts are executed after review by Ops, at the request of the
>>> project. We don't use a full schema migration process, because at this
>>> point there isn't really a need for it on this project.
>>>
>>> Does this look reasonable to you? Feedback welcomed, shoot me down if
>>> you have to...
>>
>> My very naive first level suggestion would be to run a wrapper around
>> osm2pgsql and (re-)add the indexes via this wrapper once osm2pgsql has
>> completed successfully.
osm2pgsql is creating the schema on initial data load. Creating
additional index / functions should be done at that time. We have a
script that takes care of this initial data load and some operations
that needs to happen at the same time (work in progress [1]). But
after initial data load, we might (and will) still need to evolve the
index and need to track those evolutions.
On the media wiki side, we have a more elaborate solution, with a real
schema migration process. I think that in the case of Maps this is not
really needed as the changes to the initial schema are fairly trivial.
Idempotent scripts are sufficient IMHO. I'd like validation on this
specific point, but at least I did not see anyone jump at the mention
of idempotent scripts ...
[1] https://gerrit.wikimedia.org/r/#/c/293105/
>> Also, if such schema optimizations have general interest, it would be
>> nice to try to get them integrated into osm2pgsql.
>>
>> But I can be missing some context here!
>>
>> Cheers
>> Giuseppe
>> --
>> Giuseppe Lavagetto, Ph.d.
>> Senior Technical Operations Engineer, Wikimedia Foundation
>>
>> _______________________________________________
>> Ops mailing list
>> Ops(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/ops
>
>
>
> --
> Alexandros Kosiaris <akosiaris(a)wikimedia.org>
--
Guillaume Lederrey
Operations Engineer, Discovery
Wikimedia Foundation
Hello!
I need some feedback from my fellow Ops on how to manage indexes on Maps.
Context:
Maps imports OpenStreetMap data in a Postgresql database. This import
is done with osm2pgsql, which takes care of creating the schema,
populating tables and creating a few indexes.
Some additional indexes are required to support the specific
functionalities of Tilerator. So far, those indexes have been created
manually and have not been tracked.
The enhancements to the schema created by osm2pgsql are minor (a few
index and functions), so we probably need a lightweight solution.
Proposal:
A few idempotent scripts are versionned in osm-bright.tm2source [1].
Those scripts are executed after review by Ops, at the request of the
project. We don't use a full schema migration process, because at this
point there isn't really a need for it on this project.
Does this look reasonable to you? Feedback welcomed, shoot me down if
you have to...
Thanks for your help!
Guillaume
[1] https://github.com/kartotherian/osm-bright.tm2source/tree/master/sql
--
Guillaume Lederrey
Operations Engineer, Discovery
Wikimedia Foundation
Meet up in Warsaw or Dublin? :)
I've got such good food and drink to try in my two weeks across the pond!
w00t!
--
Deb Tankersley
Product Manager, Discovery
IRC: debt
Wikimedia Foundation
On Mon, Jun 20, 2016 at 9:12 AM, Antoine Boegli <antoine.boegli(a)gmail.com>
wrote:
> Next time, perhaps we'll try chocolate aquafaba mousse for the dessert...
> Or something stranger... Mmmh.... don't know for the moment...
>
> 2016-06-20 17:08 GMT+02:00 Guillaume Lederrey <glederrey(a)wikimedia.org>:
>
>> @Deb: I told you that you should stop in Lausanne on the way back from
>> Wikimania...
>>
>> On Mon, Jun 20, 2016 at 5:07 PM, Antoine Boegli
>> <antoine.boegli(a)gmail.com> wrote:
>> > Oh ! I forgot the artichoke tapenade ! shame on me !
>> >
>> > We also had some carrots, cucumbers and tomatoes with artichoke
>> tapenade :-)
>> >
>> > 2016-06-20 17:05 GMT+02:00 Antoine Boegli <antoine.boegli(a)gmail.com>:
>> >>
>> >> Yes, there was food !
>> >>
>> >> Guillaume treated us with pasta and pesto, and there was also salad,
>> >> beers, cafe, tea, and dessert (vanilia chia cream with stewed rhubarb)
>> >>
>> >> Almost everything was house made, so it was a bit of a food hackathon
>> too
>> >> :-)
>> >>
>> >>
>> >>
>> >>
>> >> 2016-06-20 16:40 GMT+02:00 Deborah Tankersley <
>> dtankersley(a)wikimedia.org>:
>> >>>
>> >>> That's awesome, but, was there food too? :)
>> >>>
>> >>> Thanks for all your hard work, good job!
>> >>>
>> >>>
>> >>> --
>> >>> Deb Tankersley
>> >>> Product Manager, Discovery
>> >>> IRC: debt
>> >>> Wikimedia Foundation
>> >>>
>> >>> On Mon, Jun 20, 2016 at 5:33 AM, David Causse <dcausse(a)wikimedia.org>
>> >>> wrote:
>> >>>>
>> >>>> That's certainly the most productive kitchen I've ever seen!
>> >>>>
>> >>>> Thank you everyone!
>> >>>>
>> >>>>
>> >>>> Le 20/06/2016 12:55, Guillaume Lederrey a écrit :
>> >>>>>
>> >>>>> Hello all!
>> >>>>>
>> >>>>> I had a few friends coming over this Sunday for a mini Hackathon in
>> my
>> >>>>> kitchen. The goal was mainly to introduce them to what we do around
>> >>>>> Wikipedia, have fun, exchange ideas and maybe do actual work on a
>> few
>> >>>>> tickets. Quick summary of what we did:
>> >>>>>
>> >>>>> * Fix a few puppet unit tests [1][2]
>> >>>>> * Try to fix sending logs to logstash from elasticsearch [3][4]. We
>> >>>>> could not reproduce the issue with security manager with a local
>> >>>>> installation of elasticsearch and I did not prepare access to
>> >>>>> deployment-prep where we could have validated the issue.
>> >>>>> * Added a few tests to our elasticsearch admin tool (estool) [5]
>> >>>>> * Refactoring of Cassandra monitoring [6][7] (thanks to Luca for the
>> >>>>> fast code review!). This one prompted a great conversation. It was
>> >>>>> fairly unclear to us how to best integrate monitoring in a DRY way,
>> >>>>> while keeping the core of the Cassandra module free from external
>> >>>>> dependencies. We are all used to have a technical module
>> >>>>> (puppet-cassandra) that only manage core Cassandra and a higher
>> level
>> >>>>> module (puppet-wmf_cassandra) that configure Cassandra in the
>> context
>> >>>>> of WMF, with the all peripheral fonctions (monitoring, firewall,
>> >>>>> logging, ...). This higher level abstraction kind of exists in the
>> >>>>> roles, but multiple roles might use Cassandra, and might lead to
>> some
>> >>>>> duplication. The change [6] is probably more the start of a
>> >>>>> conversation than something that can be merged as is.
>> >>>>> * We tried to have a go at adding unit tests to Kartotherian [8],
>> but
>> >>>>> ran into an issue with mapnik and were not able to build the
>> project.
>> >>>>> None of us had any experience with Node development or Kartotherian.
>> >>>>> We moved to other stuff.
>> >>>>>
>> >>>>> The day was fun! Thanks to all who joined! Special +1 to Mara who
>> put
>> >>>>> a lot of energy in learning Python for the first time (and
>> >>>>> succeeded!).
>> >>>>>
>> >>>>> I'll try to followup on the changed opened during that day, make
>> sure
>> >>>>> that those contributions, however small, are not forgotten.
>> >>>>>
>> >>>>> We will be back!
>> >>>>>
>> >>>>> MrG
>> >>>>>
>> >>>>> [1] https://gerrit.wikimedia.org/r/#/c/295127/
>> >>>>> [2] https://gerrit.wikimedia.org/r/#/c/295130/
>> >>>>> [3] https://gerrit.wikimedia.org/r/#/c/295129/
>> >>>>> [4] https://phabricator.wikimedia.org/T136696
>> >>>>> [5] https://gerrit.wikimedia.org/r/#/c/290765/
>> >>>>> [6] https://gerrit.wikimedia.org/r/#/c/295123/
>> >>>>> [7] https://phabricator.wikimedia.org/T137422
>> >>>>> [8] https://phabricator.wikimedia.org/T111950
>> >>>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> discovery mailing list
>> >>>> discovery(a)lists.wikimedia.org
>> >>>> https://lists.wikimedia.org/mailman/listinfo/discovery
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Antoine Boegli
>> >> software engineer & linux expert
>> >
>> >
>> >
>> >
>> > --
>> > Antoine Boegli
>> > software engineer & linux expert
>>
>>
>>
>> --
>> Guillaume Lederrey
>> Operations Engineer, Discovery
>> Wikimedia Foundation
>>
>
>
>
> --
> Antoine Boegli
> software engineer & linux expert
>
@Deb: I told you that you should stop in Lausanne on the way back from
Wikimania...
On Mon, Jun 20, 2016 at 5:07 PM, Antoine Boegli
<antoine.boegli(a)gmail.com> wrote:
> Oh ! I forgot the artichoke tapenade ! shame on me !
>
> We also had some carrots, cucumbers and tomatoes with artichoke tapenade :-)
>
> 2016-06-20 17:05 GMT+02:00 Antoine Boegli <antoine.boegli(a)gmail.com>:
>>
>> Yes, there was food !
>>
>> Guillaume treated us with pasta and pesto, and there was also salad,
>> beers, cafe, tea, and dessert (vanilia chia cream with stewed rhubarb)
>>
>> Almost everything was house made, so it was a bit of a food hackathon too
>> :-)
>>
>>
>>
>>
>> 2016-06-20 16:40 GMT+02:00 Deborah Tankersley <dtankersley(a)wikimedia.org>:
>>>
>>> That's awesome, but, was there food too? :)
>>>
>>> Thanks for all your hard work, good job!
>>>
>>>
>>> --
>>> Deb Tankersley
>>> Product Manager, Discovery
>>> IRC: debt
>>> Wikimedia Foundation
>>>
>>> On Mon, Jun 20, 2016 at 5:33 AM, David Causse <dcausse(a)wikimedia.org>
>>> wrote:
>>>>
>>>> That's certainly the most productive kitchen I've ever seen!
>>>>
>>>> Thank you everyone!
>>>>
>>>>
>>>> Le 20/06/2016 12:55, Guillaume Lederrey a écrit :
>>>>>
>>>>> Hello all!
>>>>>
>>>>> I had a few friends coming over this Sunday for a mini Hackathon in my
>>>>> kitchen. The goal was mainly to introduce them to what we do around
>>>>> Wikipedia, have fun, exchange ideas and maybe do actual work on a few
>>>>> tickets. Quick summary of what we did:
>>>>>
>>>>> * Fix a few puppet unit tests [1][2]
>>>>> * Try to fix sending logs to logstash from elasticsearch [3][4]. We
>>>>> could not reproduce the issue with security manager with a local
>>>>> installation of elasticsearch and I did not prepare access to
>>>>> deployment-prep where we could have validated the issue.
>>>>> * Added a few tests to our elasticsearch admin tool (estool) [5]
>>>>> * Refactoring of Cassandra monitoring [6][7] (thanks to Luca for the
>>>>> fast code review!). This one prompted a great conversation. It was
>>>>> fairly unclear to us how to best integrate monitoring in a DRY way,
>>>>> while keeping the core of the Cassandra module free from external
>>>>> dependencies. We are all used to have a technical module
>>>>> (puppet-cassandra) that only manage core Cassandra and a higher level
>>>>> module (puppet-wmf_cassandra) that configure Cassandra in the context
>>>>> of WMF, with the all peripheral fonctions (monitoring, firewall,
>>>>> logging, ...). This higher level abstraction kind of exists in the
>>>>> roles, but multiple roles might use Cassandra, and might lead to some
>>>>> duplication. The change [6] is probably more the start of a
>>>>> conversation than something that can be merged as is.
>>>>> * We tried to have a go at adding unit tests to Kartotherian [8], but
>>>>> ran into an issue with mapnik and were not able to build the project.
>>>>> None of us had any experience with Node development or Kartotherian.
>>>>> We moved to other stuff.
>>>>>
>>>>> The day was fun! Thanks to all who joined! Special +1 to Mara who put
>>>>> a lot of energy in learning Python for the first time (and
>>>>> succeeded!).
>>>>>
>>>>> I'll try to followup on the changed opened during that day, make sure
>>>>> that those contributions, however small, are not forgotten.
>>>>>
>>>>> We will be back!
>>>>>
>>>>> MrG
>>>>>
>>>>> [1] https://gerrit.wikimedia.org/r/#/c/295127/
>>>>> [2] https://gerrit.wikimedia.org/r/#/c/295130/
>>>>> [3] https://gerrit.wikimedia.org/r/#/c/295129/
>>>>> [4] https://phabricator.wikimedia.org/T136696
>>>>> [5] https://gerrit.wikimedia.org/r/#/c/290765/
>>>>> [6] https://gerrit.wikimedia.org/r/#/c/295123/
>>>>> [7] https://phabricator.wikimedia.org/T137422
>>>>> [8] https://phabricator.wikimedia.org/T111950
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> discovery mailing list
>>>> discovery(a)lists.wikimedia.org
>>>> https://lists.wikimedia.org/mailman/listinfo/discovery
>>>
>>>
>>
>>
>>
>> --
>> Antoine Boegli
>> software engineer & linux expert
>
>
>
>
> --
> Antoine Boegli
> software engineer & linux expert
--
Guillaume Lederrey
Operations Engineer, Discovery
Wikimedia Foundation
oh my gosh, you're making me drool....it all sounds so good! :)
--
Deb Tankersley
Product Manager, Discovery
IRC: debt
Wikimedia Foundation
On Mon, Jun 20, 2016 at 9:05 AM, Antoine Boegli <antoine.boegli(a)gmail.com>
wrote:
> Yes, there was food !
>
> Guillaume treated us with pasta and pesto, and there was also salad,
> beers, cafe, tea, and dessert (vanilia chia cream with stewed rhubarb)
>
> Almost everything was house made, so it was a bit of a food hackathon too
> :-)
>
>
>
>
> 2016-06-20 16:40 GMT+02:00 Deborah Tankersley <dtankersley(a)wikimedia.org>:
>
>> That's awesome, but, was there food too? :)
>>
>> Thanks for all your hard work, good job!
>>
>>
>> --
>> Deb Tankersley
>> Product Manager, Discovery
>> IRC: debt
>> Wikimedia Foundation
>>
>> On Mon, Jun 20, 2016 at 5:33 AM, David Causse <dcausse(a)wikimedia.org>
>> wrote:
>>
>>> That's certainly the most productive kitchen I've ever seen!
>>>
>>> Thank you everyone!
>>>
>>>
>>> Le 20/06/2016 12:55, Guillaume Lederrey a écrit :
>>>
>>>> Hello all!
>>>>
>>>> I had a few friends coming over this Sunday for a mini Hackathon in my
>>>> kitchen. The goal was mainly to introduce them to what we do around
>>>> Wikipedia, have fun, exchange ideas and maybe do actual work on a few
>>>> tickets. Quick summary of what we did:
>>>>
>>>> * Fix a few puppet unit tests [1][2]
>>>> * Try to fix sending logs to logstash from elasticsearch [3][4]. We
>>>> could not reproduce the issue with security manager with a local
>>>> installation of elasticsearch and I did not prepare access to
>>>> deployment-prep where we could have validated the issue.
>>>> * Added a few tests to our elasticsearch admin tool (estool) [5]
>>>> * Refactoring of Cassandra monitoring [6][7] (thanks to Luca for the
>>>> fast code review!). This one prompted a great conversation. It was
>>>> fairly unclear to us how to best integrate monitoring in a DRY way,
>>>> while keeping the core of the Cassandra module free from external
>>>> dependencies. We are all used to have a technical module
>>>> (puppet-cassandra) that only manage core Cassandra and a higher level
>>>> module (puppet-wmf_cassandra) that configure Cassandra in the context
>>>> of WMF, with the all peripheral fonctions (monitoring, firewall,
>>>> logging, ...). This higher level abstraction kind of exists in the
>>>> roles, but multiple roles might use Cassandra, and might lead to some
>>>> duplication. The change [6] is probably more the start of a
>>>> conversation than something that can be merged as is.
>>>> * We tried to have a go at adding unit tests to Kartotherian [8], but
>>>> ran into an issue with mapnik and were not able to build the project.
>>>> None of us had any experience with Node development or Kartotherian.
>>>> We moved to other stuff.
>>>>
>>>> The day was fun! Thanks to all who joined! Special +1 to Mara who put
>>>> a lot of energy in learning Python for the first time (and
>>>> succeeded!).
>>>>
>>>> I'll try to followup on the changed opened during that day, make sure
>>>> that those contributions, however small, are not forgotten.
>>>>
>>>> We will be back!
>>>>
>>>> MrG
>>>>
>>>> [1] https://gerrit.wikimedia.org/r/#/c/295127/
>>>> [2] https://gerrit.wikimedia.org/r/#/c/295130/
>>>> [3] https://gerrit.wikimedia.org/r/#/c/295129/
>>>> [4] https://phabricator.wikimedia.org/T136696
>>>> [5] https://gerrit.wikimedia.org/r/#/c/290765/
>>>> [6] https://gerrit.wikimedia.org/r/#/c/295123/
>>>> [7] https://phabricator.wikimedia.org/T137422
>>>> [8] https://phabricator.wikimedia.org/T111950
>>>>
>>>>
>>>
>>> _______________________________________________
>>> discovery mailing list
>>> discovery(a)lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/discovery
>>>
>>
>>
>
>
> --
> Antoine Boegli
> software engineer & linux expert
>
Hello all!
I had a few friends coming over this Sunday for a mini Hackathon in my
kitchen. The goal was mainly to introduce them to what we do around
Wikipedia, have fun, exchange ideas and maybe do actual work on a few
tickets. Quick summary of what we did:
* Fix a few puppet unit tests [1][2]
* Try to fix sending logs to logstash from elasticsearch [3][4]. We
could not reproduce the issue with security manager with a local
installation of elasticsearch and I did not prepare access to
deployment-prep where we could have validated the issue.
* Added a few tests to our elasticsearch admin tool (estool) [5]
* Refactoring of Cassandra monitoring [6][7] (thanks to Luca for the
fast code review!). This one prompted a great conversation. It was
fairly unclear to us how to best integrate monitoring in a DRY way,
while keeping the core of the Cassandra module free from external
dependencies. We are all used to have a technical module
(puppet-cassandra) that only manage core Cassandra and a higher level
module (puppet-wmf_cassandra) that configure Cassandra in the context
of WMF, with the all peripheral fonctions (monitoring, firewall,
logging, ...). This higher level abstraction kind of exists in the
roles, but multiple roles might use Cassandra, and might lead to some
duplication. The change [6] is probably more the start of a
conversation than something that can be merged as is.
* We tried to have a go at adding unit tests to Kartotherian [8], but
ran into an issue with mapnik and were not able to build the project.
None of us had any experience with Node development or Kartotherian.
We moved to other stuff.
The day was fun! Thanks to all who joined! Special +1 to Mara who put
a lot of energy in learning Python for the first time (and
succeeded!).
I'll try to followup on the changed opened during that day, make sure
that those contributions, however small, are not forgotten.
We will be back!
MrG
[1] https://gerrit.wikimedia.org/r/#/c/295127/
[2] https://gerrit.wikimedia.org/r/#/c/295130/
[3] https://gerrit.wikimedia.org/r/#/c/295129/
[4] https://phabricator.wikimedia.org/T136696
[5] https://gerrit.wikimedia.org/r/#/c/290765/
[6] https://gerrit.wikimedia.org/r/#/c/295123/
[7] https://phabricator.wikimedia.org/T137422
[8] https://phabricator.wikimedia.org/T111950
--
Guillaume Lederrey
Operations Engineer, Discovery
Wikimedia Foundation
