Wiki-research-l

wiki-research-l@lists.wikimedia.org

8 participants
2988 discussions

Re: [Wiki-research-l] Database of all users (Kiril Simeonovski) # of contributions
by Jonathan Kiritharan 07 Jun '19

07 Jun '19

Kiril, I wrote something a while back in java that was able to get the number of contributions per user for a given language in Wikipedia. It could be able to be altered for your purposes if the datastructure of the namespaces is the same or similar. https://github.com/hachacha/wikiParticipants particularly this file https://github.com/hachacha/wikiParticipants/blob/master/src/wikipediansbyn… Altering which contributions would be saved within a specific date range is possible. God Bless, Jonathan On Fri, Jun 7, 2019 at 8:00 AM <wiki-research-l-request(a)lists.wikimedia.org> wrote: > Send Wiki-research-l mailing list submissions to > wiki-research-l(a)lists.wikimedia.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > or, via email, send a message with subject or body 'help' to > wiki-research-l-request(a)lists.wikimedia.org > > You can reach the person managing the list at > wiki-research-l-owner(a)lists.wikimedia.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Wiki-research-l digest..." > > > Today's Topics: > > 1. Fwd: [Wikidata] Scaling Wikidata Query Service (Pine W) > 2. Database of all users (Kiril Simeonovski) > 3. Re: Database of all users (Federico Leva (Nemo)) > 4. Re: Database of all users (Kiril Simeonovski) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 6 Jun 2019 19:35:13 +0000 > From: Pine W <wiki.pine(a)gmail.com> > To: "wikitech-l(a)lists.wikimedia.org" <wikitech-l(a)lists.wikimedia.org>, > Wiki Research-l <wiki-research-l(a)lists.wikimedia.org> > Subject: [Wiki-research-l] Fwd: [Wikidata] Scaling Wikidata Query > Service > Message-ID: > <CAF=dyJiJFXf7Jp8NUUu90Zd2dBT6J= > FhTyjirAWRhN+UV2jLpQ(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Forwarding in case this is of interest. > > Pine > ( https://meta.wikimedia.org/wiki/User:Pine ) > > > ---------- Forwarded message --------- > From: Guillaume Lederrey <glederrey(a)wikimedia.org> > Date: Thu, Jun 6, 2019 at 7:33 PM > Subject: [Wikidata] Scaling Wikidata Query Service > To: Discussion list for the Wikidata project. < > wikidata(a)lists.wikimedia.org> > > > Hello all! > > There has been a number of concerns raised about the performance and > scaling of Wikdata Query Service. We share those concerns and we are > doing our best to address them. Here is some info about what is going > on: > > In an ideal world, WDQS should: > > * scale in terms of data size > * scale in terms of number of edits > * have low update latency > * expose a SPARQL endpoint for queries > * allow anyone to run any queries on the public WDQS endpoint > * provide great query performance > * provide a high level of availability > > Scaling graph databases is a "known hard problem", and we are reaching > a scale where there are no obvious easy solutions to address all the > above constraints. At this point, just "throwing hardware at the > problem" is not an option anymore. We need to go deeper into the > details and potentially make major changes to the current architecture. > Some scaling considerations are discussed in [1]. This is going to take > time. > > Reasonably, addressing all of the above constraints is unlikely to > ever happen. Some of the constraints are non negotiable: if we can't > keep up with Wikidata in term of data size or number of edits, it does > not make sense to address query performance. On some constraints, we > will probably need to compromise. > > For example, the update process is asynchronous. It is by nature > expected to lag. In the best case, this lag is measured in minutes, > but can climb to hours occasionally. This is a case of prioritizing > stability and correctness (ingesting all edits) over update latency. > And while we can work to reduce the maximum latency, this will still > be an asynchronous process and needs to be considered as such. > > We currently have one Blazegraph expert working with us to address a > number of performance and stability issues. We > are planning to hire an additional engineer to help us support the > service in the long term. You can follow our current work in phabricator > [2]. > > If anyone has experience with scaling large graph databases, please > reach out to us, we're always happy to share ideas! > > Thanks all for your patience! > > Guillaume > > [1] > https://wikitech.wikimedia.org/wiki/Wikidata_query_service/ScalingStrategy > [2] https://phabricator.wikimedia.org/project/view/1239/ > > -- > Guillaume Lederrey > Engineering Manager, Search Platform > Wikimedia Foundation > UTC+2 / CEST > > _______________________________________________ > Wikidata mailing list > Wikidata(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikidata > > > ------------------------------ > > Message: 2 > Date: Fri, 7 Jun 2019 08:57:38 +0200 > From: Kiril Simeonovski <kiril.simeonovski(a)gmail.com> > To: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org> > Subject: [Wiki-research-l] Database of all users > Message-ID: > < > CABuEHm5mfDeo7sjrpmW_aK-Mpd2qH0c2JyGnoU9OdB5YtuTheg(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Dear all, > > I was wondering if there is a way to extract a database of all users (or > selection of users according to some criteria) with their contributions to > the Wikimedia projects until a fixed point of time from the XTools. > > Thank you. > > Best regards, > Kiril > > > ------------------------------ > > Message: 3 > Date: Fri, 7 Jun 2019 10:53:30 +0300 > From: "Federico Leva (Nemo)" <nemowiki(a)gmail.com> > To: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org>, Kiril Simeonovski > <kiril.simeonovski(a)gmail.com> > Subject: Re: [Wiki-research-l] Database of all users > Message-ID: <33f8a998-2144-1d49-5347-8c59018e2fcb(a)gmail.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > Kiril Simeonovski, 07/06/19 09:57: > > with their contributions to > > the Wikimedia projects > > Do you mean the *number* of their contributions, or literally all their > contributions? Filtering the stub dumps would be one systematic way to > get all the metadata about edits. > > If you just need aggregate numbers with some filter by date, namespace > or other, the fastest way is probably to write a script which loops > through all the databases on Labs. For instance I made this to list the > users who contribute in a certain language, to find translators for very > small languages: > < > https://gerrit.wikimedia.org/r/plugins/gitiles/labs/tools/lists/+/master/sc… > > > > Federico > > > > ------------------------------ > > Message: 4 > Date: Fri, 7 Jun 2019 09:57:45 +0200 > From: Kiril Simeonovski <kiril.simeonovski(a)gmail.com> > To: "Federico Leva (Nemo)" <nemowiki(a)gmail.com> > Cc: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org> > Subject: Re: [Wiki-research-l] Database of all users > Message-ID: > <CABuEHm7ahWx9P= > xa_km1S+Q3Z0WkOHaxcOunFx3AsA_cfnv-hg(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi Federico, > > Thanks for the straightforward answer. My idea is to extract the number of > contributions across projects and namespaces. > > Best, > Kiril > > On Fri, Jun 7, 2019 at 9:53 AM Federico Leva (Nemo) <nemowiki(a)gmail.com> > wrote: > > > Kiril Simeonovski, 07/06/19 09:57: > > > with their contributions to > > > the Wikimedia projects > > > > Do you mean the *number* of their contributions, or literally all their > > contributions? Filtering the stub dumps would be one systematic way to > > get all the metadata about edits. > > > > If you just need aggregate numbers with some filter by date, namespace > > or other, the fastest way is probably to write a script which loops > > through all the databases on Labs. For instance I made this to list the > > users who contribute in a certain language, to find translators for very > > small languages: > > < > > > https://gerrit.wikimedia.org/r/plugins/gitiles/labs/tools/lists/+/master/sc… > > > > > > > Federico > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > ------------------------------ > > End of Wiki-research-l Digest, Vol 166, Issue 4 > *********************************************** >

1 0

Database of all users
by Kiril Simeonovski 07 Jun '19

07 Jun '19

Dear all, I was wondering if there is a way to extract a database of all users (or selection of users according to some criteria) with their contributions to the Wikimedia projects until a fixed point of time from the XTools. Thank you. Best regards, Kiril

2 2

Fwd: [Wikidata] Scaling Wikidata Query Service
by Pine W 06 Jun '19

06 Jun '19

Forwarding in case this is of interest. Pine ( https://meta.wikimedia.org/wiki/User:Pine ) ---------- Forwarded message --------- From: Guillaume Lederrey <glederrey(a)wikimedia.org> Date: Thu, Jun 6, 2019 at 7:33 PM Subject: [Wikidata] Scaling Wikidata Query Service To: Discussion list for the Wikidata project. <wikidata(a)lists.wikimedia.org> Hello all! There has been a number of concerns raised about the performance and scaling of Wikdata Query Service. We share those concerns and we are doing our best to address them. Here is some info about what is going on: In an ideal world, WDQS should: * scale in terms of data size * scale in terms of number of edits * have low update latency * expose a SPARQL endpoint for queries * allow anyone to run any queries on the public WDQS endpoint * provide great query performance * provide a high level of availability Scaling graph databases is a "known hard problem", and we are reaching a scale where there are no obvious easy solutions to address all the above constraints. At this point, just "throwing hardware at the problem" is not an option anymore. We need to go deeper into the details and potentially make major changes to the current architecture. Some scaling considerations are discussed in [1]. This is going to take time. Reasonably, addressing all of the above constraints is unlikely to ever happen. Some of the constraints are non negotiable: if we can't keep up with Wikidata in term of data size or number of edits, it does not make sense to address query performance. On some constraints, we will probably need to compromise. For example, the update process is asynchronous. It is by nature expected to lag. In the best case, this lag is measured in minutes, but can climb to hours occasionally. This is a case of prioritizing stability and correctness (ingesting all edits) over update latency. And while we can work to reduce the maximum latency, this will still be an asynchronous process and needs to be considered as such. We currently have one Blazegraph expert working with us to address a number of performance and stability issues. We are planning to hire an additional engineer to help us support the service in the long term. You can follow our current work in phabricator [2]. If anyone has experience with scaling large graph databases, please reach out to us, we're always happy to share ideas! Thanks all for your patience! Guillaume [1] https://wikitech.wikimedia.org/wiki/Wikidata_query_service/ScalingStrategy [2] https://phabricator.wikimedia.org/project/view/1239/ -- Guillaume Lederrey Engineering Manager, Search Platform Wikimedia Foundation UTC+2 / CEST _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

1 0

Testing ways to make your Wikipedia thrive: Collaborating with CivilServant in 2020
by J. Nathan Matias 05 Jun '19

05 Jun '19

Hi Wiki Research List! Are you interested to make discoveries that could help your Wiki project thrive? Let's talk! Or join us for a one-day research summit after Wikimania on August 19. Funding is available. CivilServant <http://civilservant.io/> is a research nonprofit that works collaboratively with Wikipedians to test tools and practices designed to reach the goals those communities care about most. We currently are looking to add partnerships that start in 2020. If you are an experienced Wikipedian who has helped lead initiatives in your language community - or other Wiki project - and you want to help learn how to make your community stronger, we would love to hear from you. Are you curious about what tools are effective in reaching your Wikipedia's goals? Is your community thinking about using a new or existing tool or practice? We can work with your community to design and run A/B tests to discover which practices are effective. Along the way, we also hope to contribute to science! If you are interested to speak with us, please introduce yourself here <https://docs.google.com/forms/d/e/1FAIpQLSd2BwuVH34pw244D4ZoEE6MS06ZRj7XWDx…>. We also invite you to participate in a one day Research Summit which will be held in Stockholm on August 19th <https://meta.wikimedia.org/wiki/CivilServant%27s_Wikimedia_studies/Summit_S…>, the day after Wikimania. We hope to welcome any Wikipedian who is interested in learning about CivilServant and how to design A/B tests (given space limitations) . Funding: We have 10 full scholarships to pay for attendance at the Research Summit and Wikimania (including airfare & accommodations) as well as 15 partial scholarships to cover the costs of staying in Stockholm two additional nights to attend the summit. We invite you to apply for either scholarship here <https://docs.google.com/forms/d/e/1FAIpQLSd2BwuVH34pw244D4ZoEE6MS06ZRj7XWDx…> . We look forward to hearing from you. If you have questions about CivilServant's work, please visit our Meta page <https://meta.wikimedia.org/wiki/CivilServant%27s_Wikimedia_studies> or our website <https://civilservant.io/> where you can read about some of our previous <https://civilservant.io/moderation_experiment_r_science_rule_posting.html> studies <https://civilservant.io/do_downvotes_cause_bad_behavior_jan_2018.html> with Reddit communities. - or feel free to reach out directly to CivilServant's research manager, Juliakamin(cs) <https://meta.wikimedia.org/wiki/User:Juliakamin(cs)>. Finally, if you know other Wikipedians who may be interested to talk with us, please feel free to forward this email to them. Thanks! -- J. Nathan Matias <http://natematias.com/> : Princeton University : CivilServant <http://civilservant.io> : MIT Media Lab : Cornell University Fall 2019 <https://medium.com/@natematias/im-joining-the-cornell-university-department…> : @natematias <http://twitter.com/natematias> : blog <https://natematias.com/external-posts/>

1 0

Wikipedia search queries log
by S. Nunes 05 Jun '19

05 Jun '19

Hi, Does anybody knows who to contact to request access to Wikipedia search logs? I am aware of the previous effort made to make this information public and the privacy problems involved [1]. Nonetheless, I think that there is a lot of space for research without publicly disclosing the dataset, namely through NDAs. Thanks in advance for your help. [1] http://blog.wikimedia.org/2012/09/19/what-are-readers-looking-for-wikipedia- search-data-now-available/ -- Sérgio Nunes

2 2

ECIR 2020 Call for Workshops and Tutorials
by S. Nunes 04 Jun '19

04 Jun '19

==================== ECIR 2020 - 42nd European Conference on Information Retrieval Call for Workshops and Tutorials Lisbon, Portugal - April 14-17, 2020 http://www.ecir2020.org/ ==================== ECIR 2020 Workshop and Tutorial proposals deadline is September 1st, 2019 - we welcome your submission! Please find further information below. Workshops The purpose of workshops is to provide a platform for presenting novel ideas and research results in a focused and more interactive way. Workshops can be of either a half-day (3 hours plus breaks) or a full day (6 hours plus breaks). Workshops are encouraged to be as dynamic and interactive as possible and should lead to a concrete outcome, such as the publication of a summary paper and/or workshop proceedings. The information required for a workshop proposal is on the conference website. Workshop proposals will be reviewed by the workshop committee. A summary paper of the workshop will be published in the conference proceedings. Please find more information at http://www.ecir2020.org/call-for-workshops/ Tutorials Tutorials inform the community on recent advances in core IR research, related research, or on novel application areas related to IR. They may focus on specific problems or specific domains in which IR research may be applied. Tutorials can be of either a half-day (3 hours plus breaks) or a full day (6 hours plus breaks). Tutorials are encouraged to be as interactive as possible. Please follow tutorial proposal instructions on the conference website (link below). Tutorial proposals will be reviewed by the tutorial committee. A summary of the tutorial will be published in the conference proceedings. Further information can be found at http://www.ecir2020.org/call-for-tutorials/ Important dates: 1 September 2019 – Workshop and Tutorial submission 1 October 2019 – Workshop and Tutorial notification 14 April 2020 – Workshops and Tutorials Hope to see you in Lisbon!

1 0

Fwd: [OpenAccess] Fwd: Job Posting: Opportunity at the NIH Library: Data Services Informationist
by Pine W 24 May '19

24 May '19

Forwarding a job announcement that may be of interest to some list subscribers. The announced pay range is $69,581 to $128,920 per year. Pine ( https://meta.wikimedia.org/wiki/User:Pine ) ---------- Forwarded message --------- From: Federico Leva (Nemo) <nemowiki(a)gmail.com> Date: Fri, May 24, 2019 at 9:55 PM Subject: [OpenAccess] Fwd: Job Posting: Opportunity at the NIH Library: Data Services Informationist To: Wikimedia & Libraries <libraries(a)lists.wikimedia.org>, Open Access discussions <openaccess(a)lists.wikimedia.org> This looks like something where an experienced wikimedian with interest in libraries and/or open access could fit. Federico -------- Messaggio inoltrato -------- Oggetto: [SCHOLCOMM] Job Posting: Opportunity at the NIH Library: Data Services Informationist Data: Thu, 23 May 2019 13:43:48 +0000 Mittente: "Belter, Christopher (NIH/OD/ORS) [E]" The National Institutes of Health (NIH) Library is pleased to announce an exciting employment opportunity for an exceptional candidate to serve as an informationist and lead the Library’s data services program. If you have experience providing information and data services in a biomedical library setting and want to play a significant role in a dynamic organization, then consider joining the NIH Library team. This is a GS-11/12/13, Technical Information Specialist position. For more information, visit https://www.nihlibrary.nih.gov/services/data. We expect to announce this vacancy on USAJOBS.gov on May 28, 2019, and the application will be open for 10 calendar days. The application period reflects the NIH’s effort to hire talented people quickly. You may preview the draft announcements now at https://www.usajobs.gov/GetJob/ViewDetails/534055900/ and https://www.usajobs.gov/GetJob/ViewDetails/534056100/ but you will not be able to apply until May 28th. We encourage you to create or update your USAJOBS.gov account and profile now so you will be ready to apply when this job is announced. As part of the application process, it may be necessary to upload copies of degree transcripts. We also encourage you to set up email notifications for job announcements. Please visit the USAJOBS.gov Help Center at https://www.usajobs.gov/Help/faq/ for more information. Thank you for helping spread the word to others who may be interested. Major responsibilities of the position are as follows. Responsibilities • Serves as lead and subject matter expert for the Library’s data services program • Assesses NIH researchers’ training needs in the areas of data management, data visualization, data analysis, and R and Python programming • Develops and delivers graduate-level training to meet identified needs • Provides one-on-one advice and consultations in the areas of data science • Assists NIH researchers in using the NIH Library’s data sciences workstation, which offers a suite of tools for data analysis, processing, and visualization for a variety of different types of data • Consults and collaborates with NIH staff to aid them in locating, accessing, identifying, and managing information • Performs outreach across NIH and HHS to understand users’ information needs and markets library services and resources that meet those needs Contact Margaret McGhee at margaret.mcghee(a)nih.gov or (301) 451-9335 for more information. Chris Belter, MLS _______________________________________________ OpenAccess mailing list OpenAccess(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/openaccess

1 0

Supporting editors in creating new articles with references
by Lucie-Aimée Kaffee 23 May '19

23 May '19

Hello everyone! We are working on the project Scribe http://tinyurl.com/WikipediaScribe that aims to support editors in creating new, high-quality articles with a focus on currently under-resourced Wikipedia communities. To start this project, I would like to ask you for two favours: (1) We have a 5-minute survey, that we would like you to fill to give us an insight into your opinions on the project. The survey can be found here: http://tinyurl.com/WikipediaScribeSurvey (2) We are conducting a set of interviews with editors of different Wikipedias to gain an understanding of how you are currently edit, find references etc. The interviews are conducted remotely via Skype and take around 30 minutes. We would talk about your editing and referencing in Wikipedia. Please contact me if you are available to participate in one of our interviews, it would be highly appreciated! You can also sign up for the interview in our sheet here: https://forms.gle/rwRpKH1EqJGM423Y6 More information about the interviews can be found in the participant information sheet: https://github.com/luciekaffee/Scribe/blob/master/Interviews/participant-in… I am very much looking forward to the start of the project and supporting a new great editing experience! If you have any other questions, please feel free to reach out to me. Thanks, Lucie -- Lucie-Aimée Kaffee Web and Internet Science Group School of Electronics and Computer Science University of Southampton

2 1

International Conference on Social Media & Society (#SMSociety, Toronto, Canada; July 19-21, 2019)
by agruzd＠gmail.com 20 May '19

20 May '19

***Invitation to Attend*** The organizing committee of the 10th International Conference on Social Media and Society (#SMSociety) invites you to visit vibrant Toronto on July 19-21, 2019. The 2019 conference will be held at the Ted Rogers School of Management and hosted by the Social Media Lab at Ryerson University. This year's theme is on "Rethinking Privacy and Trust in the Social Media Age". Link: https://socialmediaandsociety.org <https://socialmediaandsociety.org/> The conference offers an intensive 3-day program including 8(!) hands-on workshops, full & work-in-progress papers, panels, and posters featuring the latest in social media research (https://socialmediaandsociety.org/2019-schedule/). Our interdisciplinary community brings researchers from a wide variety of disciplines including Communication, Information Science, Management, Sociology, Education, Journalism, Computer Science, Political Science, Health, and other fields. This year, we are honoured to have a highly distinguished scholars as our keynote speakers: * Tarleton Gillespie is a principal researcher at Microsoft Research New England, and an affiliated associate professor in the Department of Communication and Department of Information Science at Cornell University. His new book, Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions that Shape Social Media (Yale University Press) was published in June 2018. * Valerie Steeves is a Full Professor in the Department of Criminology at the University of Ottawa. As the lead researcher of MediaSmarts' Young Canadians in a Wired World research project, she has been tracking young people's use of new media since 2000. She is also the principal investigator of the eQuality Project, a seven-year SSHRC-funded partnership exploring young people's experiences of privacy and equality in networked spaces. We hope you can join us for this exciting event and contribute to this emerging research area! If you have any questions about the conference, please email us at: ask(a)socialmediaandsociety.org <mailto:ask@socialmediaandsociety.org> or visit our website to register https://socialmediaandsociety.org/registration/ --- #SMSociety'19 Organizing Committee https://socialmediaandsociety.org/about/

1 0

Reboot of stat1004--6-7 and notebook1003-4 happening on May 21st (early EU morning)
by Luca Toscano 20 May '19

20 May '19

Hi everybody, the stat1004-6-7 and notebook1003-4 hosts will be rebooted tomorrow morning, May 21st, during the EU morning for security upgrades (Linux kernel upgrades). Please let me or anybody in the Analytics team know if this is problematic for your work so we can schedule a better maintenance window. Thanks! Luca (on behalf of the Analytics team)

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Wiki-research-l