I read the summary of the VE study, and I have a question. Anecdotally, I am hearing from multiple sources that new editors *who attend workshops or editathons in person* prefer VE over wikitext for ease of use. Do we have any data specifically about the productivity and longevity of this population of users when they are introduced to to Wikipedia editing on VE instead of wikitext?

Thanks! Pine On Jul 29, 2015 11:09 AM, "Leila Zia" leila@wikimedia.org wrote:

A friendly reminder that this is happening in 23 min. :-)

YouTube stream: https://www.youtube.com/watch?v=vGyrVg_qKSM IRC: #wikimedia-research

Best, Leila

On Mon, Jul 27, 2015 at 2:47 PM, Leila Zia leila@wikimedia.org wrote:

...

Hi everyone,

The next Research showcase will be live-streamed this Wednesday, July 29 at 11.30 PT. The streaming link will be posted on the lists a few minutes before the showcase starts (sorry, we haven't been able to solve this, yet. :-() and as usual, you can join the conversation on IRC at #wikimedia -research.

We look forward to seeing you!

Leila

This month: *VisualEditor's effect on newly registered users*By *Aaron Halfaker* https://www.mediawiki.org/wiki/User:Halfak_%28WMF%29

It's been nearly two years since we ran an initial study https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_registered_editors/June_2013_study of VisualEditor's effect on newly registered editors. While most of the results of this study were positive (e.g. workload on Wikipedians did not increase), we still saw a significant decrease in the newcomer productivity. In the meantime, the Editing https://www.mediawiki.org/wiki/Editing team has made substantial improvements to performance and functionality. In this presentation, I'll report on the results of a new experiment designed to test the effects of enabling this improved VisualEditor software for newly registered users by default. I'll show what we learned from the experiment and discuss some results have opened larger questions about what, exactly, is difficult about being a newcomer to English Wikipedia.

*Wikipedia knowledge graph with DeepDive* By *Juhana Kangaspunta* and *Thomas Palomares (10-week student project)* Despite the tremendous amount of information present on Wikipedia, only a very little amount is structured. Most of the information is embedded in text and extracting it is a non-trivial challenge. In this project, we try to populate Wikidata, a structured component of Wikipedia, using DeepDive tool to extract relations embedded in the text. We finally extracted more than 140,000 relations with more than 90% average precision. We will present DeepDive and the data that we use for this project, we explain the relations we focused on so far and explain the implementation and pipeline, including our model, features and extractors. Finally, we detail our results with a thorough precision and recall analysis.

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

I watched the video, in which Aaron did discuss social and motivational barriers as being more complex and difficult to solve than technical issues with VisualEditor.

I liked the questions that Aaron asked ("Did you make friends? Did you find the work rewarding? Did you identify with the community?") because my understanding is that in the wide world of volunteer associations, questions like those are strongly related to volunteer retention and activity levels.

I have a hunch that in-person workshops and editathons can do a lot to improve the onboarding and retention experience for new editors. My understanding from WMF Learning and Evaluation is that editathon series and writing contest series are particularly effective at retaining editors. I would guess that this effect happens because people in general may find it easier to make friends and identify with a community when they have face-to-face, positive interactions with other members of that community.

However, also note that most students who write Wikimedia content for their classroom assignments don't remain active contributors after the completion with their assignments, so I speculate that the third issue ("did you find the work rewarding?") may be significantly affected by the intrinsic motives and interests of potential contributors, as well as competition for the time of those potential contributors from other activities (like good grades, fulfilling jobs, or happy activities with family and friends) that also provide rewards.

There is ongoing work to improve the effectiveness of mentorships and wikiprojects on English Wikipedia, which may also help to address the "did you make friends" and "did you identity with the community" questions.

I'm thinking about how I can implicitly take these issues into account when designing the content of the video project that I linked earlier in this thread, and how the video content could help with lowering social-motivational barriers. Suggestions from other participants on Research-l would be most welcome.

Thanks,

Pine

Pine

On Sun, Aug 2, 2015 at 1:20 AM, Kerry Raymond kerry.raymond@gmail.com wrote:

I haven’t yet had the opportunity to watch the YouTube version of the talk, but just taking the question at face value.

I don’t think the data is likely to be able to distinguish people doing their first edits at a training class or edit-a-thon because in general there is nothing to distinguish these folk from any other new contributors. It might be that some events use some system of categories for either the users or the articles (editathons often tag the articles with the event name) so you might be able to spot edits arising from a specific event but in general I don’t think you can tell them apart.

I teach a lot of edit training and, although I have yet to switch to the VE, I am looking forward to being able to do so as soon as possible. Markup is definitely a barrier to some people and I think the VE will be preferred by most users. However, while VE may make editing easier, it does not solve the problem of having newcomers’ good faith contributions being reverted by others. WP:NOBITE is the most ignored policy of Wikipedia.

Kerry

*From:* wiki-research-l-bounces@lists.wikimedia.org [mailto: wiki-research-l-bounces@lists.wikimedia.org] *On Behalf Of *Pine W *Sent:* Sunday, 2 August 2015 3:58 PM *To:* Wiki Research-l wiki-research-l@lists.wikimedia.org *Subject:* Re: [Wiki-research-l] July 2015 Research showcase

I read the summary of the VE study, and I have a question. Anecdotally, I am hearing from multiple sources that new editors *who attend workshops or editathons in person* prefer VE over wikitext for ease of use. Do we have any data specifically about the productivity and longevity of this population of users when they are introduced to to Wikipedia editing on VE instead of wikitext?

Thanks! Pine

On Jul 29, 2015 11:09 AM, "Leila Zia" leila@wikimedia.org wrote:

A friendly reminder that this is happening in 23 min. :-)

YouTube stream: https://www.youtube.com/watch?v=vGyrVg_qKSM IRC: #wikimedia-research

Best,

Leila

On Mon, Jul 27, 2015 at 2:47 PM, Leila Zia leila@wikimedia.org wrote:

Hi everyone,

The next Research showcase will be live-streamed this Wednesday, July 29 at 11.30 PT. The streaming link will be posted on the lists a few minutes before the showcase starts (sorry, we haven't been able to solve this, yet. :-() and as usual, you can join the conversation on IRC at #wikimedia-research.

We look forward to seeing you!

Leila

This month:

*VisualEditor's effect on newly registered users*

By *Aaron Halfaker* https://www.mediawiki.org/wiki/User:Halfak_%28WMF%29

It's been nearly two years since we ran an initial study https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_registered_editors/June_2013_study of VisualEditor's effect on newly registered editors. While most of the results of this study were positive (e.g. workload on Wikipedians did not increase), we still saw a significant decrease in the newcomer productivity. In the meantime, the Editing https://www.mediawiki.org/wiki/Editing team has made substantial improvements to performance and functionality. In this presentation, I'll report on the results of a new experiment designed to test the effects of enabling this improved VisualEditor software for newly registered users by default. I'll show what we learned from the experiment and discuss some results have opened larger questions about what, exactly, is difficult about being a newcomer to English Wikipedia.

*Wikipedia knowledge graph with DeepDive*

By *Juhana Kangaspunta* and *Thomas Palomares (10-week student project)* Despite the tremendous amount of information present on Wikipedia, only a very little amount is structured. Most of the information is embedded in text and extracting it is a non-trivial challenge. In this project, we try to populate Wikidata, a structured component of Wikipedia, using DeepDive tool to extract relations embedded in the text. We finally extracted more than 140,000 relations with more than 90% average precision. We will present DeepDive and the data that we use for this project, we explain the relations we focused on so far and explain the implementation and pipeline, including our model, features and extractors. Finally, we detail our results with a thorough precision and recall analysis.

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Thanks for posting your feedback. I also watched the video but my takeaways were so different from yours that I am tempted to rewatch the whole thing before responding. I do recall thinking that Aaron's presentation was significantly less boring than the student one, but that the students had a few points that need to be addressed. As far as Aaron's presentation goes I was most struck by his comments afterwards (so fast forward to about halfway through the video to catch those) and I remember thinking that I wished he could work on that stuff instead of VE stuff. I promise to revisit these comments later, so stay tuned.

On Mon, Aug 3, 2015 at 9:43 AM, Pine W wiki.pine@gmail.com wrote:

I watched the video, in which Aaron did discuss social and motivational barriers as being more complex and difficult to solve than technical issues with VisualEditor.

I liked the questions that Aaron asked ("Did you make friends? Did you find the work rewarding? Did you identify with the community?") because my understanding is that in the wide world of volunteer associations, questions like those are strongly related to volunteer retention and activity levels.

I have a hunch that in-person workshops and editathons can do a lot to improve the onboarding and retention experience for new editors. My understanding from WMF Learning and Evaluation is that editathon series and writing contest series are particularly effective at retaining editors. I would guess that this effect happens because people in general may find it easier to make friends and identify with a community when they have face-to-face, positive interactions with other members of that community.

However, also note that most students who write Wikimedia content for their classroom assignments don't remain active contributors after the completion with their assignments, so I speculate that the third issue ("did you find the work rewarding?") may be significantly affected by the intrinsic motives and interests of potential contributors, as well as competition for the time of those potential contributors from other activities (like good grades, fulfilling jobs, or happy activities with family and friends) that also provide rewards.

There is ongoing work to improve the effectiveness of mentorships and wikiprojects on English Wikipedia, which may also help to address the "did you make friends" and "did you identity with the community" questions.

I'm thinking about how I can implicitly take these issues into account when designing the content of the video project that I linked earlier in this thread, and how the video content could help with lowering social-motivational barriers. Suggestions from other participants on Research-l would be most welcome.

Thanks,

Pine

Pine

On Sun, Aug 2, 2015 at 1:20 AM, Kerry Raymond kerry.raymond@gmail.com wrote:

...

I haven’t yet had the opportunity to watch the YouTube version of the talk, but just taking the question at face value.

I don’t think the data is likely to be able to distinguish people doing their first edits at a training class or edit-a-thon because in general there is nothing to distinguish these folk from any other new contributors. It might be that some events use some system of categories for either the users or the articles (editathons often tag the articles with the event name) so you might be able to spot edits arising from a specific event but in general I don’t think you can tell them apart.

I teach a lot of edit training and, although I have yet to switch to the VE, I am looking forward to being able to do so as soon as possible. Markup is definitely a barrier to some people and I think the VE will be preferred by most users. However, while VE may make editing easier, it does not solve the problem of having newcomers’ good faith contributions being reverted by others. WP:NOBITE is the most ignored policy of Wikipedia.

Kerry

*From:* wiki-research-l-bounces@lists.wikimedia.org [mailto: wiki-research-l-bounces@lists.wikimedia.org] *On Behalf Of *Pine W *Sent:* Sunday, 2 August 2015 3:58 PM *To:* Wiki Research-l wiki-research-l@lists.wikimedia.org *Subject:* Re: [Wiki-research-l] July 2015 Research showcase

I read the summary of the VE study, and I have a question. Anecdotally, I am hearing from multiple sources that new editors *who attend workshops or editathons in person* prefer VE over wikitext for ease of use. Do we have any data specifically about the productivity and longevity of this population of users when they are introduced to to Wikipedia editing on VE instead of wikitext?

Thanks! Pine

On Jul 29, 2015 11:09 AM, "Leila Zia" leila@wikimedia.org wrote:

A friendly reminder that this is happening in 23 min. :-)

YouTube stream: https://www.youtube.com/watch?v=vGyrVg_qKSM IRC: #wikimedia-research

Best,

Leila

On Mon, Jul 27, 2015 at 2:47 PM, Leila Zia leila@wikimedia.org wrote:

Hi everyone,

The next Research showcase will be live-streamed this Wednesday, July 29 at 11.30 PT. The streaming link will be posted on the lists a few minutes before the showcase starts (sorry, we haven't been able to solve this, yet. :-() and as usual, you can join the conversation on IRC at #wikimedia-research.

We look forward to seeing you!

Leila

This month:

*VisualEditor's effect on newly registered users*

By *Aaron Halfaker* https://www.mediawiki.org/wiki/User:Halfak_%28WMF%29

It's been nearly two years since we ran an initial study https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_registered_editors/June_2013_study of VisualEditor's effect on newly registered editors. While most of the results of this study were positive (e.g. workload on Wikipedians did not increase), we still saw a significant decrease in the newcomer productivity. In the meantime, the Editing https://www.mediawiki.org/wiki/Editing team has made substantial improvements to performance and functionality. In this presentation, I'll report on the results of a new experiment designed to test the effects of enabling this improved VisualEditor software for newly registered users by default. I'll show what we learned from the experiment and discuss some results have opened larger questions about what, exactly, is difficult about being a newcomer to English Wikipedia.

*Wikipedia knowledge graph with DeepDive*

By *Juhana Kangaspunta* and *Thomas Palomares (10-week student project)* Despite the tremendous amount of information present on Wikipedia, only a very little amount is structured. Most of the information is embedded in text and extracting it is a non-trivial challenge. In this project, we try to populate Wikidata, a structured component of Wikipedia, using DeepDive tool to extract relations embedded in the text. We finally extracted more than 140,000 relations with more than 90% average precision. We will present DeepDive and the data that we use for this project, we explain the relations we focused on so far and explain the implementation and pipeline, including our model, features and extractors. Finally, we detail our results with a thorough precision and recall analysis.

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Hey folks,

I'm glad the presentation came across so well. I really appreciate the discussion.

Pine, I really appreciate those plots that you linked. It seems that you can identify the progression through barrier types by following the hexagonal graphs clockwise. Concerns start with complex rules and (to a lesser extend) the difficulty of editing and progress to concerns negative social behavior and access to reference materials.

Regarding editathons, I'm not quite sure the right way to measure their effects. I suspect that one of the biggest effects of editathons are the result of discussions that people have with their friends and family after the event. "I edited Wikipedia and it was fun. It turns out that there's a lot of different types of ways to contribute. You don't have to be an expert." -- is a conversation I imagine is relatively common after an editathon. The awareness (I can edit Wikipedia?!), new registrations and contributions that result from such once-removed discussions would be nearly impossible to track.

Jane, seem more of my work exploring the rising social/motivational barriers here: https://www.youtube.com/watch?v=bozyc1z25aQ#t=24m49s In the conclusion of that talk, I bring up Snuggle[1] as an example of a technological strategy for supporting desirable social behaviors. My recent work on the Revision Scoring[2] was originally inspired by my work to extend Snuggle beyond English Wikipedia -- I needed vandalism prediction scores beyond English Wikipedia! Generally, I think we (as Wikipedian community members) have a lot deeper insight into the types of behaviors (e.g. reactions to newcomer contributions) that are desirable than we had in 2006 and that, if we were to redesign counter-vandalism tools from scratch with these insights in mind, we'd be able to dramatically reduce this type of social/motivational barrier. I think Snuggle is a good example of such a new type of tool and the idea with Revision Scoring is that I'd like to make it *really easy* for others to experiment with their own strategies. The next thing I want to do is to try empowering WikiProjects with automated quality control/socialization tools. I suspect that, WikiProject members will be highly motivated to socialize potential good newcomers and help them work productively within the topical context of their WikiProject -- if they had the means to do so efficiently.

1. https://en.wikipedia.org/wiki/Wikipedia:Snuggle 2. https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service

-Aaron

On Mon, Aug 3, 2015 at 7:36 AM, Jane Darnell jane023@gmail.com wrote:

OK I am replying to this mail, as this one has the link to Youtube in it with the two presentations. I am only responding to the first presentation by Aaron here.

In general I like the idea of focussing attention on the "New Editor Activation Funnel". This area is of course the reason why we have a decline in new editors, and it all has to do with an increase in "barriers to entry" (which btw I am not convinced is the same thing as "technical impediments"). It is useful to split these barriers up into Permission, Literacy (here wikimarkup is lumped together with policies), and Social/Motivational (human interaction) issues, but I think the whole presentation misses the point on the need for more dissection of the reverts problem (shown a bit towards the end).

I personally think that demotivational behavior by experienced Wikipedians is the biggest factor in the decline of new editor contributions, but unlike most people I don't think this has to do with what the experienced Wikipedians do, but rather what they don't do. They don't welcome people in person (because they don't see their edits) and they don't give timely feedback on first edits to pages on their watchlist (no way to see if those edits are first time edits). They don't show them the ropes in that if one wants to make a BLP, or an article about a company or building or place, or an article about an artwork, you should look at existing examples and start from there. Having said this, I do think we spend an inordinate amount of time on things like extending the page about WHAT WIKIPEDIA IS NOT (which btw I have yet to read). It seems that our best way of dealing with newcomers is to throw CAPS at them, though we all hate CAPS.

The point of this study was to prove these two: H1: VE will increase the amount of desirable edits by newbies and H2: VE will increase the amount of undesirable edits by newbies (aka VANDALISM). Guess what? Both H1 & H2 show no significance and if anything, less vandalism came from VE editors. I could have told you that beforehand - yawn. It angers me when people assume that others are not technical enough for Wikipedia. Sorry, but it is not rocket science.

This type of thinking is not just on Wikipedia, I see this also in health occupations, where doctors tell their patients not to go look things up on the Internet. Just trust the doctors because they studied it! Yeah right, like I am going to trust all aspects of my future health and well-being to someone who sees my future health and well-being as a 10-minute interlude in their 9-5 workday. No, I will nod politely (one must always remain friendly) while googling my way to better health, thanks. And if I want to make an article about something that I think needs an article on Wikipedia, I am going to try to do it on my own as far as I can get, and I am probably not interested in talking about it until I am done. The whole AfC queue thing is absolutely horrible because it puts these edits on ice until the person totally forgets what the password was that they dreamed up for their user account. As far as spelling corrections go, if I correct an error and see it deleted (like from Kiev to Kyiv, which will be reverted by a bot probably), then I will probably not come back.

I am very eager to hear more about the revision scoring though! I wish there was a better way to do that than manually however. Jane

On Wed, Jul 29, 2015 at 8:07 PM, Leila Zia leila@wikimedia.org wrote:

...

A friendly reminder that this is happening in 23 min. :-)

YouTube stream: https://www.youtube.com/watch?v=vGyrVg_qKSM IRC: #wikimedia-research

Best, Leila

On Mon, Jul 27, 2015 at 2:47 PM, Leila Zia leila@wikimedia.org wrote:

...

Hi everyone,

The next Research showcase will be live-streamed this Wednesday, July 29 at 11.30 PT. The streaming link will be posted on the lists a few minutes before the showcase starts (sorry, we haven't been able to solve this, yet. :-() and as usual, you can join the conversation on IRC at #wikimedia -research.

We look forward to seeing you!

Leila

This month: *VisualEditor's effect on newly registered users*By *Aaron Halfaker* https://www.mediawiki.org/wiki/User:Halfak_%28WMF%29

It's been nearly two years since we ran an initial study https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_registered_editors/June_2013_study of VisualEditor's effect on newly registered editors. While most of the results of this study were positive (e.g. workload on Wikipedians did not increase), we still saw a significant decrease in the newcomer productivity. In the meantime, the Editing https://www.mediawiki.org/wiki/Editing team has made substantial improvements to performance and functionality. In this presentation, I'll report on the results of a new experiment designed to test the effects of enabling this improved VisualEditor software for newly registered users by default. I'll show what we learned from the experiment and discuss some results have opened larger questions about what, exactly, is difficult about being a newcomer to English Wikipedia.

*Wikipedia knowledge graph with DeepDive* By *Juhana Kangaspunta* and *Thomas Palomares (10-week student project)* Despite the tremendous amount of information present on Wikipedia, only a very little amount is structured. Most of the information is embedded in text and extracting it is a non-trivial challenge. In this project, we try to populate Wikidata, a structured component of Wikipedia, using DeepDive tool to extract relations embedded in the text. We finally extracted more than 140,000 relations with more than 90% average precision. We will present DeepDive and the data that we use for this project, we explain the relations we focused on so far and explain the implementation and pipeline, including our model, features and extractors. Finally, we detail our results with a thorough precision and recall analysis.

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

+1 I've been working with James Hare (WikiProject X) to use the revision scoring system to support their work. Their bot is actually pulling scores from our service right now. :)

Also +1 for Wikiprojects as entry points for newcomers. I've been pitching a WikiProject recommender service for a while. I'd really like to start experimenting with that. I think that might have been one of the next logical steps for the Growth team if it was still around.

-Aaron

On Mon, Aug 3, 2015 at 1:46 PM, Pine W wiki.pine@gmail.com wrote:

Hi Aaron,

Thanks, those sound like good ideas for better quality control and mentoring/socialization pathways.

Are there opportunities to coordinate your work on counter-vandalism tools and empowering wikiprojects into the work that others are doing with wikiprojects, such as "Wikiproject X https://en.wikipedia.org/wiki/Wikipedia:WikiProject_X" and Michael Gilbert's work that he recently mentioned on the Analytics mailing list?

One of my thoughts is that it would be good to encourage newcomers to get involved with active wikiprojects very early in their Wikipedia careers, to get guidance and to develop friendships that might increase editor retention.

Thanks!

Pine

Pine

On Mon, Aug 3, 2015 at 8:10 AM, Aaron Halfaker ahalfaker@wikimedia.org wrote:

...

Hey folks,

I'm glad the presentation came across so well. I really appreciate the discussion.

Pine, I really appreciate those plots that you linked. It seems that you can identify the progression through barrier types by following the hexagonal graphs clockwise. Concerns start with complex rules and (to a lesser extend) the difficulty of editing and progress to concerns negative social behavior and access to reference materials.

Regarding editathons, I'm not quite sure the right way to measure their effects. I suspect that one of the biggest effects of editathons are the result of discussions that people have with their friends and family after the event. "I edited Wikipedia and it was fun. It turns out that there's a lot of different types of ways to contribute. You don't have to be an expert." -- is a conversation I imagine is relatively common after an editathon. The awareness (I can edit Wikipedia?!), new registrations and contributions that result from such once-removed discussions would be nearly impossible to track.

Jane, seem more of my work exploring the rising social/motivational barriers here: https://www.youtube.com/watch?v=bozyc1z25aQ#t=24m49s In the conclusion of that talk, I bring up Snuggle[1] as an example of a technological strategy for supporting desirable social behaviors. My recent work on the Revision Scoring[2] was originally inspired by my work to extend Snuggle beyond English Wikipedia -- I needed vandalism prediction scores beyond English Wikipedia! Generally, I think we (as Wikipedian community members) have a lot deeper insight into the types of behaviors (e.g. reactions to newcomer contributions) that are desirable than we had in 2006 and that, if we were to redesign counter-vandalism tools from scratch with these insights in mind, we'd be able to dramatically reduce this type of social/motivational barrier. I think Snuggle is a good example of such a new type of tool and the idea with Revision Scoring is that I'd like to make it *really easy* for others to experiment with their own strategies. The next thing I want to do is to try empowering WikiProjects with automated quality control/socialization tools. I suspect that, WikiProject members will be highly motivated to socialize potential good newcomers and help them work productively within the topical context of their WikiProject -- if they had the means to do so efficiently.

1. https://en.wikipedia.org/wiki/Wikipedia:Snuggle
2. https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service

-Aaron

On Mon, Aug 3, 2015 at 7:36 AM, Jane Darnell jane023@gmail.com wrote:

...

OK I am replying to this mail, as this one has the link to Youtube in it with the two presentations. I am only responding to the first presentation by Aaron here.

In general I like the idea of focussing attention on the "New Editor Activation Funnel". This area is of course the reason why we have a decline in new editors, and it all has to do with an increase in "barriers to entry" (which btw I am not convinced is the same thing as "technical impediments"). It is useful to split these barriers up into Permission, Literacy (here wikimarkup is lumped together with policies), and Social/Motivational (human interaction) issues, but I think the whole presentation misses the point on the need for more dissection of the reverts problem (shown a bit towards the end).

I personally think that demotivational behavior by experienced Wikipedians is the biggest factor in the decline of new editor contributions, but unlike most people I don't think this has to do with what the experienced Wikipedians do, but rather what they don't do. They don't welcome people in person (because they don't see their edits) and they don't give timely feedback on first edits to pages on their watchlist (no way to see if those edits are first time edits). They don't show them the ropes in that if one wants to make a BLP, or an article about a company or building or place, or an article about an artwork, you should look at existing examples and start from there. Having said this, I do think we spend an inordinate amount of time on things like extending the page about WHAT WIKIPEDIA IS NOT (which btw I have yet to read). It seems that our best way of dealing with newcomers is to throw CAPS at them, though we all hate CAPS.

The point of this study was to prove these two: H1: VE will increase the amount of desirable edits by newbies and H2: VE will increase the amount of undesirable edits by newbies (aka VANDALISM). Guess what? Both H1 & H2 show no significance and if anything, less vandalism came from VE editors. I could have told you that beforehand - yawn. It angers me when people assume that others are not technical enough for Wikipedia. Sorry, but it is not rocket science.

This type of thinking is not just on Wikipedia, I see this also in health occupations, where doctors tell their patients not to go look things up on the Internet. Just trust the doctors because they studied it! Yeah right, like I am going to trust all aspects of my future health and well-being to someone who sees my future health and well-being as a 10-minute interlude in their 9-5 workday. No, I will nod politely (one must always remain friendly) while googling my way to better health, thanks. And if I want to make an article about something that I think needs an article on Wikipedia, I am going to try to do it on my own as far as I can get, and I am probably not interested in talking about it until I am done. The whole AfC queue thing is absolutely horrible because it puts these edits on ice until the person totally forgets what the password was that they dreamed up for their user account. As far as spelling corrections go, if I correct an error and see it deleted (like from Kiev to Kyiv, which will be reverted by a bot probably), then I will probably not come back.

I am very eager to hear more about the revision scoring though! I wish there was a better way to do that than manually however. Jane

On Wed, Jul 29, 2015 at 8:07 PM, Leila Zia leila@wikimedia.org wrote:

...

A friendly reminder that this is happening in 23 min. :-)

YouTube stream: https://www.youtube.com/watch?v=vGyrVg_qKSM IRC: #wikimedia-research

Best, Leila

On Mon, Jul 27, 2015 at 2:47 PM, Leila Zia leila@wikimedia.org wrote:

...

Hi everyone,

The next Research showcase will be live-streamed this Wednesday, July 29 at 11.30 PT. The streaming link will be posted on the lists a few minutes before the showcase starts (sorry, we haven't been able to solve this, yet. :-() and as usual, you can join the conversation on IRC at #wikimedia-research.

We look forward to seeing you!

Leila

This month: *VisualEditor's effect on newly registered users*By *Aaron Halfaker* https://www.mediawiki.org/wiki/User:Halfak_%28WMF%29

It's been nearly two years since we ran an initial study https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_registered_editors/June_2013_study of VisualEditor's effect on newly registered editors. While most of the results of this study were positive (e.g. workload on Wikipedians did not increase), we still saw a significant decrease in the newcomer productivity. In the meantime, the Editing https://www.mediawiki.org/wiki/Editing team has made substantial improvements to performance and functionality. In this presentation, I'll report on the results of a new experiment designed to test the effects of enabling this improved VisualEditor software for newly registered users by default. I'll show what we learned from the experiment and discuss some results have opened larger questions about what, exactly, is difficult about being a newcomer to English Wikipedia.

*Wikipedia knowledge graph with DeepDive* By *Juhana Kangaspunta* and *Thomas Palomares (10-week student project)* Despite the tremendous amount of information present on Wikipedia, only a very little amount is structured. Most of the information is embedded in text and extracting it is a non-trivial challenge. In this project, we try to populate Wikidata, a structured component of Wikipedia, using DeepDive tool to extract relations embedded in the text. We finally extracted more than 140,000 relations with more than 90% average precision. We will present DeepDive and the data that we use for this project, we explain the relations we focused on so far and explain the implementation and pipeline, including our model, features and extractors. Finally, we detail our results with a thorough precision and recall analysis.

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

From doing edit training myself, I would say that there really are technical impediments particularly for older people (and I say that as a retired person so I’m not that young either). I only get to do “one-shot” training (typically half day, sometimes full day) and I think a one-shot workshop isn’t enough to get some people to a level of technical competency. It would be nice to get an opportunity to engage with people over a series of sessions but there are a whole host of reasons why that’s a lot harder to set up (harder to lock in a venue over a number of sessions, harder for people coming along to be free for multiple sessions, harder to find the volunteers to do the training across multiple sessions). It’s hard enough to set up a one-shot at times. I do genuinely believe the VE will help with this impediment. At the end of the day, if people don’t click SAVE, there is no edit in our logs to analyse, so it’s not always easy not to see the basic technical impediment that markup creates. But the VE does not solve the “technical impediment” (cognitive impediment?) of understanding what a citation/template/infobox is. Now in this mailing list, the idea of not knowing about citations is unthinkable. But when I teach people older than me, typically local history groups, I know they are statistically likely to have left school at age 14 having had 8 years of primary school education. They don’t know what a citation is. They type with 2 fingers hunt-and-peck; they don’t know how to copy-and-paste; they don’t know how to open a second tab on their browser. They don’t know the difference between a round/square/angle/curly bracket. They don’t understand why balancing their brackets matters or why most of the article disappears from view because of unbalanced brackets. They don’t know that slash and backslash are different or what a “tilde” is. Some of those things the VE helps with, some not. There’s a lot of impediment out there that I never realised until I saw it in edit training.

And the VE won’t solve the “community” problem. Given the hostility of the en.WP community to the VE (probably it’s not widespread but a rather a very vocal minority), it is not clear to me if tagging edits as being VE is actually a “red flag to a bull” to the VE-haters. That is, might a VE-hater behave (even more) aggressively towards new users using the VE?

Some months ago, we altered the banner for WikiProject Australia to include an email help@wikimedia.org.au mailto:help@wikimedia.org.au as a way to reach out to new editors of Australian content who need help and don’t know the Wikipedia ways of getting help. It was successful in that we did receive emails, so there’s a tip – newbies find it easier to email to get help. Aside: What was unexpected about it was that many of them were a conflict-of-interest situation, where the editor was either the subject of the article or an employee or otherwise affiliated. Now in most cases they hadn’t created the article but they felt something was wrong and needed to be fixed or wanted to add something. Generally the articles weren’t puffery and the edits desired weren’t unreasonable (not white-washing) and generally we’ve helped them. None of them attempted to hide their connection to the article; most went to some trouble to establish their bona fides to make it clear that they were “authorised” to request this change. I don’t think any of these people wanted to learn to edit themselves, they just wanted the article fixed. Perhaps Wikipedia might be better off if we encouraged this kind of email request rather than try to force everyone to become editors.

The other thing that pops up (sometimes CoI, sometimes not) is “I saw these messages about needing more references. I’ve added loads of references but no matter how many I add, those messages just don’t go away. What more can I do?”. Not understanding how Wikipedia works is another “cognitive impediment” for the newbie that the VE won’t solve. Of course, if we added in Aaron’s automated assessment tool, maybe we could provide a better way to give some article quality feedback than persisting with the belief that WikiProjects are alive and well and actively reassessing quality.

So, while I do disagree with Jane about the technical impediments, I do agree 100% about the reverts as a big issue. It happens during edit training and it really upsets the people, even though I am there to hold their hand. It’s really hard when I cannot understand myself why the edit was reverted. And, as vast majority of attendees at edit training are female, I see a very strong reaction from the women that reverting is “not nice” and “very rude” (this is strong stuff, likely to lead to mentions of “little hitlers”). I know the “Club House” paper didn’t detect any difference between male and female reactions, but in terms of verbal reactions to being reverted, the women are quite vocal and all agree on the “not nice”. I suspect the men are just as offended but don’t mention it (the strong silent stereotype would prevent them telling me, a woman). When we look at the gender gap, I have to wonder if we have something very basically wrong with the Bold – Revert – Discuss approach. I don’t think it works for women, who tend to Propose – Discuss – Discuss – Discuss – Eventually Implement. I think the whole “Bold – Revert” is very libertarian ideology so I am unsurprised it doesn’t appeal to women. Women tend to do things more slowly but I think get happier outcomes, probably because women are socialised to keep people happy.

Kerry

From: wiki-research-l-bounces@lists.wikimedia.org [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of Jane Darnell Sent: Monday, 3 August 2015 10:36 PM To: Research into Wikimedia content and communities wiki-research-l@lists.wikimedia.org Subject: Re: [Wiki-research-l] July 2015 Research showcase

OK I am replying to this mail, as this one has the link to Youtube in it with the two presentations. I am only responding to the first presentation by Aaron here.

In general I like the idea of focussing attention on the "New Editor Activation Funnel". This area is of course the reason why we have a decline in new editors, and it all has to do with an increase in "barriers to entry" (which btw I am not convinced is the same thing as "technical impediments"). It is useful to split these barriers up into Permission, Literacy (here wikimarkup is lumped together with policies), and Social/Motivational (human interaction) issues, but I think the whole presentation misses the point on the need for more dissection of the reverts problem (shown a bit towards the end).

I personally think that demotivational behavior by experienced Wikipedians is the biggest factor in the decline of new editor contributions, but unlike most people I don't think this has to do with what the experienced Wikipedians do, but rather what they don't do. They don't welcome people in person (because they don't see their edits) and they don't give timely feedback on first edits to pages on their watchlist (no way to see if those edits are first time edits). They don't show them the ropes in that if one wants to make a BLP, or an article about a company or building or place, or an article about an artwork, you should look at existing examples and start from there. Having said this, I do think we spend an inordinate amount of time on things like extending the page about WHAT WIKIPEDIA IS NOT (which btw I have yet to read). It seems that our best way of dealing with newcomers is to throw CAPS at them, though we all hate CAPS.

The point of this study was to prove these two: H1: VE will increase the amount of desirable edits by newbies and H2: VE will increase the amount of undesirable edits by newbies (aka VANDALISM). Guess what? Both H1 & H2 show no significance and if anything, less vandalism came from VE editors. I could have told you that beforehand - yawn. It angers me when people assume that others are not technical enough for Wikipedia. Sorry, but it is not rocket science.

This type of thinking is not just on Wikipedia, I see this also in health occupations, where doctors tell their patients not to go look things up on the Internet. Just trust the doctors because they studied it! Yeah right, like I am going to trust all aspects of my future health and well-being to someone who sees my future health and well-being as a 10-minute interlude in their 9-5 workday. No, I will nod politely (one must always remain friendly) while googling my way to better health, thanks. And if I want to make an article about something that I think needs an article on Wikipedia, I am going to try to do it on my own as far as I can get, and I am probably not interested in talking about it until I am done. The whole AfC queue thing is absolutely horrible because it puts these edits on ice until the person totally forgets what the password was that they dreamed up for their user account. As far as spelling corrections go, if I correct an error and see it deleted (like from Kiev to Kyiv, which will be reverted by a bot probably), then I will probably not come back.

I am very eager to hear more about the revision scoring though! I wish there was a better way to do that than manually however.

Jane

On Wed, Jul 29, 2015 at 8:07 PM, Leila Zia <leila@wikimedia.org mailto:leila@wikimedia.org > wrote:

A friendly reminder that this is happening in 23 min. :-)

YouTube stream: https://www.youtube.com/watch?v=vGyrVg_qKSM IRC: #wikimedia-research

Best,

Leila

On Mon, Jul 27, 2015 at 2:47 PM, Leila Zia <leila@wikimedia.org mailto:leila@wikimedia.org > wrote:

Hi everyone,

The next Research showcase will be live-streamed this Wednesday, July 29 at 11.30 PT. The streaming link will be posted on the lists a few minutes before the showcase starts (sorry, we haven't been able to solve this, yet. :-() and as usual, you can join the conversation on IRC at #wikimedia-research.

We look forward to seeing you!

Leila

This month:

VisualEditor's effect on newly registered users

By https://www.mediawiki.org/wiki/User:Halfak_%28WMF%29 Aaron Halfaker

It's been nearly two years since we ran an initial study https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_registered_editors/June_2013_study of VisualEditor's effect on newly registered editors. While most of the results of this study were positive (e.g. workload on Wikipedians did not increase), we still saw a significant decrease in the newcomer productivity. In the meantime, the Editing https://www.mediawiki.org/wiki/Editing team has made substantial improvements to performance and functionality. In this presentation, I'll report on the results of a new experiment designed to test the effects of enabling this improved VisualEditor software for newly registered users by default. I'll show what we learned from the experiment and discuss some results have opened larger questions about what, exactly, is difficult about being a newcomer to English Wikipedia.

Wikipedia knowledge graph with DeepDive

By Juhana Kangaspunta and Thomas Palomares (10-week student project)

Despite the tremendous amount of information present on Wikipedia, only a very little amount is structured. Most of the information is embedded in text and extracting it is a non-trivial challenge. In this project, we try to populate Wikidata, a structured component of Wikipedia, using DeepDive tool to extract relations embedded in the text. We finally extracted more than 140,000 relations with more than 90% average precision. We will present DeepDive and the data that we use for this project, we explain the relations we focused on so far and explain the implementation and pipeline, including our model, features and extractors. Finally, we detail our results with a thorough precision and recall analysis.

_______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org mailto:Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l