An interesting question, and one which I was discussing recently with Kristofer Erikson of https://www.create.ac.uk/. He pointed me in the direction of this paper which he co-authored in 2018: https://dl.acm.org/doi/10.1145/3233391.3233533 - (abstract below), which folks here may find interesting if you've not come across it before. My understanding is that Kris would be open to further work in this area.
*The Wikimedia Commons (WC) is a peer-produced repository of freely licensed images, videos, sounds and interactive media, containing more than 45 million files. This paper attempts to quantify the societal value of the WC by tracking the downstream use of images found on the platform. We take a random sample of 10,000 images from WC and apply an automated reverse-image search to each, recording when and where they are used 'in the wild'. We detect 54,758 downstream uses of the initial sample and we characterise these at the level of generic and country-code top-level domains (TLDs). We analyse the impact of specific variables on the odds that an image is used. The random sampling technique enables us to estimate overall value of all images contained on the platform. Drawing on the method employed by Heald et al (2015), we find a potential contribution of USD $28.9 billion from downstream use of Wikimedia Commons images over the lifetime of the project.*
Sara Thomas Programme Manager, Wikimedia UK
On Mon, 10 Jun 2024 at 15:17, Andrew Lih andrew.lih@gmail.com wrote:
Hi all,
This was posed as a question on the GLAM Wiki Global Telegram group, but I thought I'd send it to the WREN list for thoughts and comments. Feel free to respond in email or at the original thread: https://t.me/c/1723589369/4942 https://eu-west-1.protection.sophos.com?d=t.me&u=aHR0cHM6Ly90Lm1lL2MvMTcyMzU4OTM2OS80OTQy&i=NjVhZmE0ZjY3MDExZmY1Y2Q3NDE3Yjkx&t=VzRENHlmMUo3UjluYnFxK1JRdTliUGtWMGh6L3pGOTNKRldEQ3MyTUVHVT0=&h=d1b2ecafe3ba428f96c5679fd3ac9be9&s=AVNPUEhUT0NFTkNSWVBUSVYH9BwpN1vq0wfWrM95H--uNoJkZSu-S-SEneaeW0G6uP_c8mPWnSChQva98tFLw1E
This was in response to the current draft of the annual plan, and a discussion around Selena Deckelmann's (WMF CPTO) response to resourcing Wikimedia Commons.
Thanks. -Andrew
I'd like to put out a question that may be key to understanding the debate seen around "knowledge dissemination," as was mentioned in Selena Deckelmann's response to the resourcing Commons statement: https://commons.wikimedia.org/wiki/Commons_talk:Media_knowledge_beyond_Wikip... https://eu-west-1.protection.sophos.com?d=wikimedia.org&u=aHR0cHM6Ly9jb21tb25zLndpa2ltZWRpYS5vcmcvd2lraS9Db21tb25zX3RhbGs6TWVkaWFfa25vd2xlZGdlX2JleW9uZF9XaWtpcGVkaWEjUmVzcG9uc2VfZnJvbV90aGVfU2VsZW5hX0RlY2tlbG1hbm4sX0NQVE9fYXRfdGhlX1dpa2ltZWRpYV9Gb3VuZGF0aW9u&i=NjVhZmE0ZjY3MDExZmY1Y2Q3NDE3Yjkx&t=TUxnWHFRVzJVY2ZCR3FvOWdOZlhuT2kzTm1lTkM5R1RCSHEwSFZVVko3dz0=&h=d1b2ecafe3ba428f96c5679fd3ac9be9&s=AVNPUEhUT0NFTkNSWVBUSVYH9BwpN1vq0wfWrM95H--uNoJkZSu-S-SEneaeW0G6uP_c8mPWnSChQva98tFLw1E
How do you evaluate or measure the impact of open content?
By its very nature, open content (text, data, code, media) is out of the control of its originator. The editor, uploader, file host makes it available, and others copy it and use it somewhere down the path of replication. Therefore, evaluating the use and impact of any open content can be extremely hard – free/libre content encourages dissemination, but removes the ability to measure it. It may be possible through secondary indicators (e.g. Github stars) or best-effort research (e.g. Google search for image credits, hopefully finding mentions of Commons or a user). However, all too often measuring impact is a qualitative educated guess. This is not a new problem. But I believe this lack of sound methodology for evaluating impact, and our lack of evolving our thinking, has become a blocker in multiple ways.
Some questions to ponder:
- Some GLAM partners have mentioned this problem specifically, and it
would be interesting to hear if others heard the same issues. For example, Cleveland Museum of Art were early adopters of open access, uploading Commons images and creating Wikidata items, but are finding it hard to justify more activity without understanding the impact of these efforts. How might we do better to discover the impact of these GLAM engagements outside of our limited Commons/Wikipedia metrics?
- We know that both Commons and Wikidata have superlatives of their own
– Wiki Loves Monuments is the world's largest public photo contest and Wikidata has been the project that racks up the most edits of all the Wikimedia projects. But outside of our Wikimedia servers, it's hard to tell. We know anecdotally that libraries and archives use Wikidata extensively now for linked open data, with organizations like LD4 and IFLA asking for more Wikidata engagement. All major library databases in the world use Wikidata Q identifiers. But how might we capture that impact better, and on a regular basis?
- Finally, we come to the essay and Talk page mentioned above. My concern
is that since Wikipedia metrics are so easily available and are of such huge magnitude, the appeal to be fixated on them is overwhelming. It's easy to put Commons and Wikidata in a lesser corner as we don't have good methods to estimate their downstream impact. Much of the value of Commons and Wikidata are not people visiting the Commons image page, or the Wikidata item page, but rather the linkages and re-use in the open ecosystem – federated search, authority control records, multimedia visualizations. And because that's hard to measure, we risk always putting Commons and Wikidata into second-class status, when they should be seen as a different beast.
Apologies for the long-winded examples, but it does come down to this:
How might we make a more complete evaluation of the impact of Wikimedia's open content, so we can make better decisions for the future?
Wren mailing list -- wren@lists.wikimedia.org To unsubscribe send an email to wren-leave@lists.wikimedia.org