[feedback requested] Taxonomy of knowledge gaps

List overview All Threads
Download

newer

older

Reminder - Review of the draft for...

Movement Strategy: guidance and...

Leila Zia

10 Sep 2020 10 Sep '20

10:47 p.m.

Hi all, I hope this email finds you well. I'm reaching out to let you know that the Research team [1] at the Wikimedia Foundation has been working on developing a taxonomy of knowledge gaps for the Wikimedia projects. We now have the first draft of the taxonomy ready and we're seeking your input to improve it. ==Why are we contacting you?== The taxonomy of knowledge gaps aims to be a high level representation and grouping of the different knowledge gaps Wikimedia projects face today. Each of you, whether you are a volunteer editor, patroller, organizer, affiliate, etc. have valuable on the ground knowledge of the different types of knowledge gaps. We believe it's important to hear from you before we finalize the taxonomy. ==Material== The material you may need to review is listed at https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Taxonomy#Lear…! . I will list them below as well, for archive completeness: * A summary of the taxonomy and motivation: https://commons.wikimedia.org/wiki/File:The_Knowledge_Gaps_Taxonomy_Summary… * Full paper: https://arxiv.org/abs/2008.12314 * A video presentation: https://www.youtube.com/watch?v=pP3uXA9bfvU or https://commons.wikimedia.org/wiki/File:Knowledge_Gaps_Taxonomy.mp4.webm (same video on two platforms) ==Feedback== Please provide your feedback by answering the 6 questions posted at https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy… . We're collecting feedback until 2020-09-30. ==Talk with us== If you have questions about the taxonomy and you'd like to talk with us in a synchronous set-up, we invite you to join us in the upcoming Research Showcase https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#September_2020 . We will have a very short presentation about it and will leave 15-20 min for any questions you may have. We are also happy to set up more time to answer your questions if there is demand for it. ==Disclaimer== As you're going through the material we have shared with you, you will see imperfections and rooms for improvement. I acknowledge that they exist and they may be numerous. We could spend another month and improve the documents. We made the call to not let perfect be the enemy of good. Please keep that in mind, assume good faith, and ask questions if any part of what you read is not clear to you. We're here to engage and answer your questions, and ultimately learn about your perspective. Thank you! Leila, on behalf of Martin Gerlach, Research Scientist, WMF Isaac Johnson, Research Scientist, WMF Miriam Redi, Senior Research Scientist, WMF Leila Zia, Head of Research, WMF [1] https://research.wikimedia.org/team.html

Show replies by date

Samuel Klein

11 Sep 11 Sep

9:14 p.m.

Wow. Thanks for doing this. a) did you mine emijrp's subconscious yet? b) what meta-gaps are you aware of (areas where the gap analysis itself might have blind spots) c) this seems appropriate for a wikijournal <https://meta.wikimedia.org/wiki/WikiJournal> of knowledge... 🌍🌏🌎🌑 On Thu, Sep 10, 2020 at 6:48 PM Leila Zia <lzia(a)wikimedia.org> wrote:

...

Hi all, I hope this email finds you well. I'm reaching out to let you know that the Research team [1] at the Wikimedia Foundation has been working on developing a taxonomy of knowledge gaps for the Wikimedia projects. We now have the first draft of the taxonomy ready and we're seeking your input to improve it. ==Why are we contacting you?== The taxonomy of knowledge gaps aims to be a high level representation and grouping of the different knowledge gaps Wikimedia projects face today. Each of you, whether you are a volunteer editor, patroller, organizer, affiliate, etc. have valuable on the ground knowledge of the different types of knowledge gaps. We believe it's important to hear from you before we finalize the taxonomy. ==Material== The material you may need to review is listed at https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Taxonomy#Lear… ! . I will list them below as well, for archive completeness: * A summary of the taxonomy and motivation: https://commons.wikimedia.org/wiki/File:The_Knowledge_Gaps_Taxonomy_Summary… * Full paper: https://arxiv.org/abs/2008.12314 * A video presentation: https://www.youtube.com/watch?v=pP3uXA9bfvU or https://commons.wikimedia.org/wiki/File:Knowledge_Gaps_Taxonomy.mp4.webm (same video on two platforms) ==Feedback== Please provide your feedback by answering the 6 questions posted at https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy… . We're collecting feedback until 2020-09-30. ==Talk with us== If you have questions about the taxonomy and you'd like to talk with us in a synchronous set-up, we invite you to join us in the upcoming Research Showcase https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#September_2020 . We will have a very short presentation about it and will leave 15-20 min for any questions you may have. We are also happy to set up more time to answer your questions if there is demand for it. ==Disclaimer== As you're going through the material we have shared with you, you will see imperfections and rooms for improvement. I acknowledge that they exist and they may be numerous. We could spend another month and improve the documents. We made the call to not let perfect be the enemy of good. Please keep that in mind, assume good faith, and ask questions if any part of what you read is not clear to you. We're here to engage and answer your questions, and ultimately learn about your perspective. Thank you! Leila, on behalf of Martin Gerlach, Research Scientist, WMF Isaac Johnson, Research Scientist, WMF Miriam Redi, Senior Research Scientist, WMF Leila Zia, Head of Research, WMF [1] https://research.wikimedia.org/team.html _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Leila Zia

15 Sep 15 Sep

8:46 p.m.

Hi SJ, Please see below. On Fri, Sep 11, 2020 at 2:15 PM Samuel Klein <meta.sj(a)gmail.com> wrote:

...

Wow. Thanks for doing this. a) did you mine emijrp's subconscious yet?

(I assume you are referring to their work in https://github.com/emijrp/all-human-knowledge .) We indeed reviewed the work as a source. There are two aspects in emijrp's work that are very helpful: 1) what constitutes a content gap? 2) how to measure it? Now that you mentioned it: I'll reach out to them to seek their feedback more actively. thanks!

...

b) what meta-gaps are you aware of (areas where the gap analysis itself might have blind spots)

This is a great question. I have some answers for you; we will talk as a team and get back to you on this.

...

c) this seems appropriate for a wikijournal <https://meta.wikimedia.org/wiki/WikiJournal> of knowledge...

noted down.

...

🌍🌏🌎🌑 On Thu, Sep 10, 2020 at 6:48 PM Leila Zia <lzia(a)wikimedia.org> wrote:

https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Taxonomy#Lear…

! . I will list them below as well, for archive completeness: * A summary of the taxonomy and motivation:

https://commons.wikimedia.org/wiki/File:The_Knowledge_Gaps_Taxonomy_Summary…

* Full paper: https://arxiv.org/abs/2008.12314 * A video presentation: https://www.youtube.com/watch?v=pP3uXA9bfvU or https://commons.wikimedia.org/wiki/File:Knowledge_Gaps_Taxonomy.mp4.webm (same video on two platforms) ==Feedback== Please provide your feedback by answering the 6 questions posted at

https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy…

. We're collecting feedback until 2020-09-30. ==Talk with us== If you have questions about the taxonomy and you'd like to talk with us in a synchronous set-up, we invite you to join us in the upcoming Research Showcase

https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#September_2020

. We will have a very short presentation about it and will leave 15-20 min for any questions you may have. We are also happy to set up more time to answer your questions if there is demand for it. ==Disclaimer== As you're going through the material we have shared with you, you will see imperfections and rooms for improvement. I acknowledge that they exist and they may be numerous. We could spend another month and improve the documents. We made the call to not let perfect be the enemy of good. Please keep that in mind, assume good faith, and ask questions if any part of what you read is not clear to you. We're here to engage and answer your questions, and ultimately learn about your perspective. Thank you! Leila, on behalf of Martin Gerlach, Research Scientist, WMF Isaac Johnson, Research Scientist, WMF Miriam Redi, Senior Research Scientist, WMF Leila Zia, Head of Research, WMF [1] https://research.wikimedia.org/team.html _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

_______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Leila Zia

25 Sep 25 Sep

11:52 p.m.

Hi SJ, * I made a note for us to address your question about the meta-gaps as part of the update to the taxonomy paper: https://meta.wikimedia.org/w/index.php?title=Research_talk:Knowledge_Gaps_I… * We did get a chance to discuss this as a team. Here is what I heard: ** We all think this is an important point you have raised. Thank you! We will think more about it and if you have ideas about such meta-gaps, please tell us (ideally in the meta page where we're collecting feedback). ** One meta-gap that quickly surfaced was our (over?)reliance on past research and documentation that is written in English. When we look for references, by default we look for those in English, even when collectively in the team we speak French, German, Italian, Spanish, Persian, and Portuguese, (if not more), too. This answer is complete. I'm sending it your way just so you know we will think more, your thoughts are welcome, and we will add this to the final taxonomy documentation. Thanks, Leila On Tue, Sep 15, 2020 at 1:46 PM Leila Zia <lzia(a)wikimedia.org> wrote:

...

Hi SJ, Please see below. On Fri, Sep 11, 2020 at 2:15 PM Samuel Klein <meta.sj(a)gmail.com> wrote:

Wow. Thanks for doing this. a) did you mine emijrp's subconscious yet?

b) what meta-gaps are you aware of (areas where the gap analysis itself might have blind spots)

This is a great question. I have some answers for you; we will talk as a team and get back to you on this.

c) this seems appropriate for a wikijournal <https://meta.wikimedia.org/wiki/WikiJournal> of knowledge...

noted down. > > > > > On Thu, Sep 10, 2020 at 6:48 PM Leila Zia <lzia(a)wikimedia.org> wrote: > > > Hi all, > > > > I hope this email finds you well. > > > > I'm reaching out to let you know that the Research team [1] at the > > Wikimedia Foundation has been working on developing a taxonomy of > > knowledge gaps for the Wikimedia projects. We now have the first draft > > of the taxonomy ready and we're seeking your input to improve it. > > > > ==Why are we contacting you?== > > The taxonomy of knowledge gaps aims to be a high level representation > > and grouping of the different knowledge gaps Wikimedia projects face > > today. Each of you, whether you are a volunteer editor, patroller, > > organizer, affiliate, etc. have valuable on the ground knowledge of > > the different types of knowledge gaps. We believe it's important to > > hear from you before we finalize the taxonomy. > > > > ==Material== > > The material you may need to review is listed at > > > > https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Taxonomy#Lear… > > ! > > . I will list them below as well, for archive completeness: > > > > * A summary of the taxonomy and motivation: > > > > https://commons.wikimedia.org/wiki/File:The_Knowledge_Gaps_Taxonomy_Summary… > > > > * Full paper: https://arxiv.org/abs/2008.12314 > > > > * A video presentation: https://www.youtube.com/watch?v=pP3uXA9bfvU or > > https://commons.wikimedia.org/wiki/File:Knowledge_Gaps_Taxonomy.mp4.webm > > (same video on two platforms) > > > > ==Feedback== > > Please provide your feedback by answering the 6 questions posted at > > > > https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy… > > . > > > > We're collecting feedback until 2020-09-30. > > > > ==Talk with us== > > If you have questions about the taxonomy and you'd like to talk with > > us in a synchronous set-up, we invite you to join us in the upcoming > > Research Showcase > > https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#September_2020 > > . We will have a very short presentation about it and will leave 15-20 > > min for any questions you may have. We are also happy to set up more > > time to answer your questions if there is demand for it. > > > > ==Disclaimer== > > As you're going through the material we have shared with you, you will > > see imperfections and rooms for improvement. I acknowledge that they > > exist and they may be numerous. We could spend another month and > > improve the documents. We made the call to not let perfect be the > > enemy of good. Please keep that in mind, assume good faith, and ask > > questions if any part of what you read is not clear to you. We're here > > to engage and answer your questions, and ultimately learn about your > > perspective. > > > > Thank you! > > > > Leila, on behalf of > > Martin Gerlach, Research Scientist, WMF > > Isaac Johnson, Research Scientist, WMF > > Miriam Redi, Senior Research Scientist, WMF > > Leila Zia, Head of Research, WMF > > > > [1] https://research.wikimedia.org/team.html > > > > _______________________________________________ > > Wikimedia-l mailing list, guidelines at: > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > > https://meta.wikimedia.org/wiki/Wikimedia-l > > New messages to: Wikimedia-l(a)lists.wikimedia.org > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe> > > > _______________________________________________ > Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l > New messages to: Wikimedia-l(a)lists.wikimedia.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Samuel Klein

26 Sep 26 Sep

5:26 p.m.

Thank you Leila -- I appreciate the reflection and the update here. The paper is thorough and methodical in its approach, which makes it easier for me to see a problem (for my own ideas): I don't see a focus on the primary tremendous *gaps *-- which for content is depth + breadth + freshness, and for contributors is reach, and for readers is reach in much of the world. I do see an excellent discussion of systemic *bias*, but mostly treated as *static* bias of what is there, and less *dynamic* bias of what we exclude or disallow or discourage. I left detailed feedback on meta <https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy#Alternatives>. I would welcome any help in aligning the way I think about this w/ your work (if that's desired). Perhaps best to address there, since it is all about refactoring and may benefit from that. But I am posting the heart of it below for completeness: === Here are the first things I think of around coverage gaps. Only the 0th item seems to directly fit the current taxonomy... 0) exclusion via lack of awareness, interest, or expertise 1) exclusion via deletionism 2) exclusion via topic notability norms (including pop culture + current events) 3) exclusion via source notability + limiting source formats 4) exclusion via license pessimism 5) exclusion via file format (!) and codec pessimism 6) exclusion of dense specialist knowledge via review bottlenecks 7) exclusion via knowledge type [model, dataset, map layer] 8) exclusion / rejection via behavior on the projects 9) exclusion / rejection under 1-4 via differential application of policy Some of these, like file-format and review-bottleneck exclusion, are primarily technical restrictions. Some of these, like the first ~4 above, are social+regulatory+technical restrictions that could be alleviated with simple tools (including extensions, alternatives, and sandboxes) -- just as nupedia's social restrictions were alleviated w/ the technical solution of a wiki for the drafting stage. And the last two are purely social restrictions, projecting systemic bias in the community of practice onto who joins and what contributions are welcomed. I'd like to see that subset of gaps addressed directly, and not split up across other parts of a taxonomy. === Wiki♥, Sam.

Peter Southwood

6:17 p.m.

Good points. Are these maybe covered in a future stage of the project? Cheers, Peter -----Original Message----- From: Wikimedia-l [mailto:wikimedia-l-bounces@lists.wikimedia.org] On Behalf Of Samuel Klein Sent: 26 September 2020 19:26 To: Wikimedia Mailing List Subject: Re: [Wikimedia-l] [feedback requested] Taxonomy of knowledge gaps Thank you Leila -- I appreciate the reflection and the update here. The paper is thorough and methodical in its approach, which makes it easier for me to see a problem (for my own ideas): I don't see a focus on the primary tremendous *gaps *-- which for content is depth + breadth + freshness, and for contributors is reach, and for readers is reach in much of the world. I do see an excellent discussion of systemic *bias*, but mostly treated as *static* bias of what is there, and less *dynamic* bias of what we exclude or disallow or discourage. I left detailed feedback on meta <https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy#Alternatives>. I would welcome any help in aligning the way I think about this w/ your work (if that's desired). Perhaps best to address there, since it is all about refactoring and may benefit from that. But I am posting the heart of it below for completeness: === Here are the first things I think of around coverage gaps. Only the 0th item seems to directly fit the current taxonomy... 0) exclusion via lack of awareness, interest, or expertise 1) exclusion via deletionism 2) exclusion via topic notability norms (including pop culture + current events) 3) exclusion via source notability + limiting source formats 4) exclusion via license pessimism 5) exclusion via file format (!) and codec pessimism 6) exclusion of dense specialist knowledge via review bottlenecks 7) exclusion via knowledge type [model, dataset, map layer] 8) exclusion / rejection via behavior on the projects 9) exclusion / rejection under 1-4 via differential application of policy Some of these, like file-format and review-bottleneck exclusion, are primarily technical restrictions. Some of these, like the first ~4 above, are social+regulatory+technical restrictions that could be alleviated with simple tools (including extensions, alternatives, and sandboxes) -- just as nupedia's social restrictions were alleviated w/ the technical solution of a wiki for the drafting stage. And the last two are purely social restrictions, projecting systemic bias in the community of practice onto who joins and what contributions are welcomed. I'd like to see that subset of gaps addressed directly, and not split up across other parts of a taxonomy. === Wiki♥, Sam. _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe> -- This email has been checked for viruses by AVG. https://www.avg.com

Leila Zia

30 Sep 30 Sep

midnight

All: Thanks to those of you who have provided your feedback. This is your last friendly reminder. If you'd like to share your thoughts and feedback with us about the taxonomy of knowledge gap work, the official deadline for it is 2020-09-30 (Anytime on Earth;). Our team is going to be heads down working on a couple of other projects in the week of October 5. This means, if you wanted to provide feedback to us and you didn't get a chance, you're more than welcome to do so on or before October 11. We're collecting your feedback at https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy… . Sam, Thanks for sharing your feedback here and https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy… . Someone from our team or I will write to you more on the meta page. Peter, I hope you don't mind if we continue the conversation with Sam on the meta page now that the content is in both places. I see you've started asking questions from SJ there. Thanks all! Leila Leila Zia Head of Research Wikimedia Foundation On Sat, Sep 26, 2020 at 11:18 AM Peter Southwood <peter.southwood(a)telkomsa.net> wrote:

...

Leila Zia

17 Sep 17 Sep

11:21 p.m.

Hi all, I'd like to give an update about where we are and what's next. * We are one week in the feedback collection. You have almost 2 more weeks if you're interested to provide feedback. * A big thank you to those of you who have already started engaging with the feedback process and thanks to those of you who intend to do it. We know this is no small ask and we appreciate your generosity to share your perspectives with us and help us improve the work. You can share your feedback at https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy… . * For those of you who are interested to talk with us in a more high bandwidth set-up where we can do more back-and-forths: we have dedicated our upcoming Research Showcase to this topic. You will hear a presentation about this work followed by at least 30-min time for Q&A. More information in a separate email sent about the showcase and at https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#September_2020 . If you have questions, please let us know. (SJ: we have not forgotten you. We still intend to get back to you.) Thank you! Leila on behalf of Martin Gerlach, Research Scientist, WMF Isaac Johnson, Research Scientist, WMF Miriam Redi, Senior Research Scientist, WMF Leila Zia, Head of Research, WMF On Thu, Sep 10, 2020 at 3:47 PM Leila Zia <lzia(a)wikimedia.org> wrote:

...

1304

days inactive

1324

days old

wikimedia-l@lists.wikimedia.org

Manage subscription

7 comments

3 participants

tags (0)

participants (3)

Leila Zia
Peter Southwood
Samuel Klein