Hi Leila,
First of all thanks for your input!
>
> > Therefore, we worked on producing sentences from the information on
> > Wikidata in the given language. We trained a neural network model, the
> > details can be found in the preprint of the NAACL paper here:
> > https://arxiv.org/abs/1803.07116
>
> It would be good to do human (both readers and editors, and perhaps
> both sets) evaluations for this research, too, to better understand
> how well the model is doing from the perspective of the experienced
> editors in some of the smaller languages as well as their readers. (I
> acknowledge that finding experienced editors when you go to small
> languages can become hard.)
>
We worked with editors in the follow-up study, to be published at ESWC.
https://2018.eswc-conferences.org/wp-content/uploads/2018/02/ESWC2018_paper…
We also asked native speakers for their input on the fluency of the
sentences. However, I agree it would be interesting to dive more into the
question how the community perceives the ArticlePlaceholder in general and
with the generated summary in particular.
>
> > Furthermore, we would love to hear your input: Do you believe, one
> sentence
> > summaries are enough, can we serve the communities needs better with more
> > than one sentence?
>
> This is a hard question to answer. :) The answer may rely on many
> factors including the language you want to implement such a system in
> and the expectation the users of the language have in terms of online
> content available to them in their language.
>
I agree. The best would probably be therefore to study the current usage of
ArticlePlaceholder and communities targeted and draw conclusions for real
needs from those points.
>
> > Is this still true if longer abstracts would be of lower
> > text quality?
>
> same as above. You are signing yourself up for more experiments. ;)
>
> I would be interested to know:
> * What is the perception of the readers of a given language about
> Wikipedia if a lot of articles that they go to in their language have
> one sentence (to a good extent accurate), a few sentences but with
> some errors, more sentences with more errors, versus not finding the
> article they're interested in at all?
> * Related to the above: what is the error threshold beyond which the
> brand perceptions will turn negative (to be defined: may be by
> measuring if the user returns in the coming week or month.)? This may
> well be different in different languages and cultures.
> * Depending on the result of the above, we may want to look at
> offering the user the option to access that information, but outside
> of Wikipedia, or inside Wikipedia but very clearly labeled as Machine
> Generated as you do to some extent in these projects.
>
The questions are very interesting, and in part formalize what we discussed
already as well. The best way would be to actually study this with the
communities involved, as we started in the ESWC paper, but focus on the
different interest groups in particular: readers of Wikipedia, readers
coming from outside Wikipedia, editors of Wikipedia and new editors.
>
> > What other interesting use cases for such a technology in the
> > Wikimedia world can you imagine?
>
> The technology itself can have a variety of use-cases, including
> providing captions or summaries of photos even without layers of image
> processing applied to them.
>
This sounds like a very interesting idea. I saw that there is work on image
captions by WMF already started, I will be following this with great
curiosity :)
Best,
Lucie
>
> Best,
> Leila
>
> > [1] https://www.mediawiki.org/wiki/Extension:ArticlePlaceholder and
> > https://commons.wikimedia.org/wiki/File:Generating_Article_
> Placeholders_from_Wikidata_for_Wikipedia_-_Increasing_
> Access_to_Free_and_Open_Knowledge.pdf
> > [2]
> > https://eprints.soton.ac.uk/413433/1/Open_Sym_Short_Paper_
> Wikidata_Multilingual.pdf
> >
> > --
> > Lucie-Aimée Kaffee
> > Web and Internet Science Group
> > School of Electronics and Computer Science
> > University of Southampton
> > _______________________________________________
> > Wiki-research-l mailing list
> > Wiki-research-l(a)lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
> _______________________________________________
> Wiki-research-l mailing list
> Wiki-research-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
--
Lucie-Aimée Kaffee
Web and Internet Science Group
School of Electronics and Computer Science
University of Southampton
Wikimedia as a movement has over the years given consideration to small
language Wikipedias.
I would like to point you to a recent study I alongside with Hady Elsahar
of the Université de Lyon and Pavlos Vougiouklis of the University of
Southampton have been pursuing, which has been recently translated to
accepted publications.
My research interest involves mainly underserved languages on Wikidata and
Wikipedia, and how we can support them better.
One of the ways to support small Wikipedias was the ArticlePlaceholder [1].
The idea is to use the existing multilingual information in Wikidata [2]
and display it in a reader friendly way on Wikipedia in the respective
language (if a Wikidata label exists in this language).
However, at the moment the data is given only in a tabular form, which is
not very reader friendly and might not be the ideal way to engage editors
to work on the articles.
Therefore, we worked on producing sentences from the information on
Wikidata in the given language. We trained a neural network model, the
details can be found in the preprint of the NAACL paper here:
https://arxiv.org/abs/1803.07116
Given the promising results of the approach using our neural network, we
extended the work to see how we could fit in this text generation into the
existing ArticlePlaceholder and tested it with the Esperanto and Arabic
Wikipedia communities. The ESWC paper preprint for this work can be found
here:
https://2018.eswc-conferences.org/wp-content/uploads/2018/02/ESWC2018_paper…
We show that our approach is feasible for generating text from Wikidata for
Wikipedia. Editors tend to reuse the sentences, which shows it can be a
good encouragement to create full articles from those summaries.
We would like to implement the work in a test Wikipedia to see if
communities are interested in adopting the technology on a large scale in
their Wikipedias.
Furthermore, we would love to hear your input: Do you believe, one sentence
summaries are enough, can we serve the communities needs better with more
than one sentence? Is this still true if longer abstracts would be of lower
text quality? What other interesting use cases for such a technology in the
Wikimedia world can you imagine? And especially if you are part of a
underserved language Wikipedia community, what is your opinion to the
project?
[1] https://www.mediawiki.org/wiki/Extension:ArticlePlaceholder and
https://commons.wikimedia.org/wiki/File:Generating_Article_Placeholders_fro…
[2]
https://eprints.soton.ac.uk/413433/1/Open_Sym_Short_Paper_Wikidata_Multilin…
--
Lucie-Aimée Kaffee
Web and Internet Science Group
School of Electronics and Computer Science
University of Southampton
Hi everyone,
The Wikimedia Foundation is asking for your feedback in a survey. We want
to know how well we are supporting your work on and off wiki, and how we
can change or improve things in the future. The opinions you share will
affect the current and future work of the Wikimedia Foundation.
If you are volunteer developer, and have contributed code to any pieces of
MediaWiki, gadgets, or tools, please complete the survey. It is available
in various languages and will take between 20 and 40 minutes to complete.
*Follow this link to the Survey:*
https://wikimedia.qualtrics.com/jfe/form/SV_5ABs6WwrDHzAeLr?aud=DEV
If you have already seen a similar message on Phabricator, Mediawiki.org,
Discourse, or other platforms for volunteer developers, please don't take
the survey twice.
You can find more information about this survey on the project page
<https://meta.wikimedia.org/wiki/Community_Engagement_Insights/About_CE_>
and see how your feedback helps the Wikimedia Foundation support
contributors like you. This survey is hosted by a third-party service and
governed by this privacy statement
<https://wikimediafoundation.org/wiki/Community_Engagement_Insights_2018_Sur…>.
Please visit our frequently asked questions page
<https://meta.wikimedia.org/wiki/Community_Engagement_Insights/Frequently_as…>
to find more information about this survey.
Feel free to email me directly with any questions you may have.
Thank you!
Edward Galvez
--
Edward Galvez
Evaluation Strategist, Surveys
Learning & Evaluation
Community Engagement
Wikimedia Foundation