Hello!
Capital Consulting (aka SafeGuard Switzerland) is organizing a few
drinks in Geneva next Thursday (May 26). So I'll be in Geneva for the
evening. I'll make sure I attend our standup, but will take a couple
of hours off in the CEST evening. Exact time to be confirmed...
Take care,
MrG
--
Guillaume Lederrey
Operations Engineer, Discovery
Wikimedia Foundation
Hello,
The Discovery Team recently added descriptive text to the Wikipedia.org
page footer in order to give visitors a better idea of what the sister wiki
projects are really all about. Check it out at www.wikipedia.org or view a
mobile screen capture here
<https://www.mediawiki.org/wiki/File:Descriptive-text_sister_projects-wikipe…>
.
We also wrapped up a quick, one question survey that ran for a week on the
portal to determine how visitors arrive at the page. The results showed
that many of the visitors arrive by clicking on a bookmarked link or by
typing in 'wikipedia' in their browser. We had many encouraging and
uplifting comments as well - see the full results here
<https://commons.wikimedia.org/wiki/File:Wikipedia_Portal_Survey_-_May_2016.…>
.
In the coming weeks and months we'll be reaching out to many of those
survey takers who graciously provided their name and email addresses to
engage in deeper conversations on how they use the portal and Wikipedia in
general.
As always, more detailed information is available on wiki for the Wikipedia
Portal <https://www.mediawiki.org/wiki/Wikipedia.org_Portal> and A/B testing
<https://www.mediawiki.org/wiki/Wikipedia.org_Portal_A/B_testing>.
On behalf of our happy little Wikipedia.org Portal team,
Deb
--
Deb Tankersley
Product Manager, Discovery
Wikimedia Foundation
Yes, of course *processing* the entire history (even with text) has been
done before - but perhaps not storing or indexing it.
BTW is anyone still using "Wikihadoop"?
https://blog.wikimedia.org/2011/11/21/do-it-yourself-analytics-with-wikiped…https://github.com/whym/wikihadoop
On Wed, May 18, 2016 at 3:09 AM, Dan Andreescu <dandreescu(a)wikimedia.org>
wrote:
> Hi Tilman, thanks for pointing to this research. We have indeed worked on
> this kind of project, for both ORES and the WikiCredit system. There are
> many challenges like memory and processing time. Loading the entire history
> without text is what we're working on right now for our Wikistats 2.0
> project. Even this has many challenges.
>
> As far as I can tell right now, any simple attempt to handle all the data
> in one way or one place is going to run into some sort of limit. If anybody
> finds otherwise, it would be useful to our work.
>
> *From: *Tilman Bayer
> *Sent: *Tuesday, May 17, 2016 02:54
> *To: *A mailing list for the Analytics Team at WMF and everybody who has
> an interest in Wikipedia and analytics.
> *Reply To: *A mailing list for the Analytics Team at WMF and everybody
> who has an interest in Wikipedia and analytics.
> *Cc: *A public mailing list about Wikimedia Search and Discovery projects
> *Subject: *[Analytics] University project to make entire English
> Wikipedia history searchable on Hadoop using Solr
>
> Detailed technical report on an undergraduate student project at Virginia
> Tech (work in progress) to import the entire English Wikipedia history dump
> into the university's Hadoop cluster and index it using Apache Solr, to
> "allow researchers and developers at Virginia Tech to benchmark
> configurations and big data analytics software":
>
> Steven Stulga, "English Wikipedia on Hadoop Cluster"
> https://vtechworks.lib.vt.edu/handle/10919/70932 (CC BY 3.0)
>
> IIRC this has rarely or never been attempted due to the large size of the
> dataset - 10TB uncompressed. And it looks like the author here encountered
> an out of memory error that he wasn't able to solve before the end of
> term...
>
> --
> Tilman Bayer
> Senior Analyst
> Wikimedia Foundation
> IRC (Freenode): HaeB
>
>
> --
> Sent from Gmail Mobile
>
>
> _______________________________________________
> Analytics mailing list
> Analytics(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
>
--
Tilman Bayer
Senior Analyst
Wikimedia Foundation
IRC (Freenode): HaeB
Detailed technical report on an undergraduate student project at Virginia
Tech (work in progress) to import the entire English Wikipedia history dump
into the university's Hadoop cluster and index it using Apache Solr, to
"allow researchers and developers at Virginia Tech to benchmark
configurations and big data analytics software":
Steven Stulga, "English Wikipedia on Hadoop Cluster"
https://vtechworks.lib.vt.edu/handle/10919/70932 (CC BY 3.0)
IIRC this has rarely or never been attempted due to the large size of the
dataset - 10TB uncompressed. And it looks like the author here encountered
an out of memory error that he wasn't able to solve before the end of
term...
--
Tilman Bayer
Senior Analyst
Wikimedia Foundation
IRC (Freenode): HaeB
--
Sent from Gmail Mobile
Anyone in Discovery have the cycles to look at the Swift plugin I wrote?
-Chad
---------- Forwarded message ----------
From: "Avik Das" <avdas(a)syr.edu>
Date: May 11, 2016 11:58 PM
Subject: Request for code review and merge for
wikimedia/search-repository-swift
To: "chadh(a)wikimedia.org" <chadh(a)wikimedia.org>
Cc:
Hi,
I was working on elasticsearch plugins and was looking for snapshot
plugins. Recently, I came across your repository and found it very useful
and generic. However I needed plugin for elasticsearch version 2.1.1. I
forked from the main branch and created some changes in my local
repository. I am writing this email to request for code review and for
merging into main branch for elasticsearch 2.1.1 . I tested my plugin on
my local machine and swift repository from Soft Layer Object storage.
Please let me know if you need any more information or action from my side.
Please find the link for my repository below
https://github.com/avik1988/search-repository-swift.git
<https://github.com/avik1988/search-repository-swift.git>
avik1988/search-repository-swift
<https://github.com/avik1988/search-repository-swift.git>
github.com
search-repository-swift - Github mirror of "search/repository-swift" - our
actual code is hosted with Gerrit (please see
https://www.mediawiki.org/wiki/Developer_access for contributing
Thanks,
Avik Das
Forwarding.
---------- Forwarded message ----------
From: Pine W <wiki.pine(a)gmail.com>
Date: Thu, May 12, 2016 at 10:21 AM
Subject: Re: [Wikitech-l] 12-May-2016 CREDIT, going back to Hangouts on
Air/YouTube
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>, Wikimedia
Foundation Multimedia Team <Multimedia(a)lists.wikimedia.org>, mobile-l <
mobile-l(a)lists.wikimedia.org>
FYI, this week's presentations, according to the Etherpad, are:
* *Derk-Jan Hartman:* Video.js progress
* *Dmitry Brant*: Wikidata infoboxes in Android app
* *Joaquin Hernandez*: Vicky chat bot
* *Baha*: mobile printing for offline reading
* *Monte*: "smart random" content service endpoint
* *Erik*: Geo boosting search queries
Cheers,
Pine
On Thu, May 12, 2016 at 9:17 AM, Adam Baso <abaso(a)wikimedia.org> wrote:
> Reminder...
>
> On Thu, Apr 14, 2016 at 12:13 AM, Adam Baso <abaso(a)wikimedia.org> wrote:
>
> > Hi all,
> >
> > The next CREDIT showcase will be Thursday, 12-May-2016 at 1800 UTC (1100
> > SF).
> >
> > https://www.mediawiki.org/wiki/CREDIT_showcase
> >
> > For this one we'll use Hangouts on Air for presenters, and the customary
> > YouTube stream for viewers.
> >
> > See you next month!
> > -Adam
> >
> >
> >
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
Hey!
No, the issue is unrelated. We are having a few issues with Varnish
upgrades. To be honest, I don't entirely understand what the issue is,
so I will not even try to explain it. I expect we will have an
incident report coming soon.
Regards,
Guillaume
On Wed, May 11, 2016 at 9:30 PM, Jan Macura <macurajan(a)gmail.com> wrote:
> Hi Gullaume, all
>
> Not sure if it's connected somehow, but WDQS GUI is not working properly
> this evening. For some time, the https://query.wikidata.org/ was rendering
> blank page with no error, now it crashes with "TypeError: wb.ui.visualEditor
> is undefined".
> Using SPARQL endpoint distantly works fine.
>
> Best
> Jan
>
> 2016-05-10 14:16 GMT+02:00 Guillaume Lederrey <glederrey(a)wikimedia.org>:
>>
>> Hello!
>>
>> I took some notes of the recent Wikidata Query Service instabilities
>> and created an incident report [1]. Some people might have additional
>> insight that I don't have. If that's the case, let me know and I'll
>> update that incident report.
>>
>> Thanks all for your help and your understanding!
>>
>> Guillaume
>>
>>
>> [1]
>> https://wikitech.wikimedia.org/wiki/Incident_documentation/20160503-Wikidat…
>>
>>
>> --
>> Guillaume Lederrey
>> Operations Engineer, Discovery
>> Wikimedia Foundation
>>
>> _______________________________________________
>> Wikidata mailing list
>> Wikidata(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikidata
>
>
>
> _______________________________________________
> Wikidata mailing list
> Wikidata(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata
>
--
Guillaume Lederrey
Operations Engineer, Discovery
Wikimedia Foundation
Hello!
Yesterday around 21:43 UTC we experienced a slowdown of Elasticsearch.
In short, one of our elasticsearch server went crazy with its garbage
collector and had to be restarted.
You can find more details in the incident report [1] or the
corresponding Phabricator task [2].
Thank you all for your understanding!
Guillaume
[1] https://wikitech.wikimedia.org/wiki/Incident_documentation/20160509-CirrusS…
[2] https://phabricator.wikimedia.org/T134829
--
Guillaume Lederrey
Operations Engineer, Discovery
Wikimedia Foundation
Hello!
I took some notes of the recent Wikidata Query Service instabilities
and created an incident report [1]. Some people might have additional
insight that I don't have. If that's the case, let me know and I'll
update that incident report.
Thanks all for your help and your understanding!
Guillaume
[1] https://wikitech.wikimedia.org/wiki/Incident_documentation/20160503-Wikidat…
--
Guillaume Lederrey
Operations Engineer, Discovery
Wikimedia Foundation
