Dear all,
TL;DR: Today the Wikimedia Foundation published a press release[1] about
the “Wikimedia Enterprise” API project - announcing our first set of
customers, as well as a new self-signup system. This is a significant
milestone because it fulfills several promises we have made to ourselves
and to the movement. There will be a community open meeting on Thursday 23
June @ 1700 UTC as documented on our Meta page.[2] The text of this email
is also published on the talkpage - please centralise any comments/feedback
there.[3]
Details
I am writing today with details of the latest developments in the
“Wikimedia Enterprise” API project. This follows the project’s
community-discussion phase, which began approximately one year ago, focused
on Meta [and also this wikimedia-l thread]. Then, this past October, we
issued a press release announcing that we were “open for business” on the
project’s new site.[4] This wikimedia-l email thread contains the details
of those previous phases.
Now is the third and final major announcement in this journey from “idea”
to “reality”. Today’s press release,[1] and associated story on the
project’s new “news” page,[5] states that:
-
Two well known organisations will be announced as the first customers of
the project. One is a major social/search corporation [Google], as our
first official paying customer. This also means that the project is now
covering its current operating costs. The other is a movement partner and
nonprofit organization [The Internet Archive] that will receive access at
no cost.
-
Anyone will be able to sign up for an account and use/access the service
[but not at a commercial scale] for free. Furthermore, payments for
usage above that threshold will be calculated simply and publicly based on
the number of API requests and gigabytes of data used. (Other free access
methods for the dataset continue to exist, as documented on our Meta FAQ)[6]
-
The API’s metadata has been expanded to include the beta version of what
we are calling “credibility signals”. This is already public information
(such as pageviews, edit-rates, and page-protection status changes)
packaged within the single data feed to help users make more informed
decisions about when they should refresh their copy of the dataset.
(Emphasis on ‘beta’, as this is not available on all versions of the
product yet.)
This announcement is a significant milestone because it fulfills several
promises we have made to ourselves and to the movement, namely that:
-
We have built something that commercial organisations who are already
heavy users of Wikimedia content and Wikimedia Foundation services are
willing to invest in. The pricing is based on estimated usage, resulting in
a more manageable and transparent cost structure. The project is now
covering its current operating expenses. In addition, we requested and
received a public affirmation/support letter from the Board for the
project’s financial operating principles, ensuring that commercial revenue
will only ever be a minority of the total and their oversight for any
future high value contracts.[7]
-
The nonprofit partner will receive access at no cost, demonstrating a
first practical example of how this project supports the mission of
knowledge access while also providing a new revenue stream.
-
The ‘trial’ tier of the service is primarily designed to allow potential
customers to determine whether they want to use it in commercial production
environments, but it also allows anyone to see what is ‘in’ the API.
Moreover, it will allow volunteers or researchers to access the service for
free at a non-commercial scale. If those people have a mission-relevant
use-case that requires them to continue to use the Enterprise API above
that scale (i.e. that isn’t viable using other APIs/dumps), we will
continue to provide them with free access.
-
The ‘credibility signals’ concept means that vandalism and errors should
appear less often and/or be removed more quickly in downstream services
such as search engines. [Note, how and when customers incorporate features
is their own decision].
While we are proud to announce these customers, it is important to note
that our market research has identified a significant gap in our movement’s
ability to have Wikimedia knowledge used. The world’s largest companies are
already using Wikimedia; we’re just providing a better way for them to do
so. But for everyone else, it is often too hard and they do not have the
resources (financial, technical, and human) to incorporate Wikimedia
information – even though they want to. In short: simply providing
legally-reusable knowledge is insufficient to enable reuse for a very large
portion of society.
And so, we are focusing a lot of our future product development on
this Knowledge
as a Service model - consistent with the Movement Strategy’s “strategic
direction”.[8] This is what OpenFuture.eu’s interview with the Enterprise
team referred to as our attempt at “lowering the playing field” – a term we
quite like.[9]
We are increasingly realising that the future of Wikimedia Enterprise is
much more nuanced than merely “making big tech pay”, but is about enabling
access to the many companies who want to use Wikimedia knowledge in their
own products but currently can’t. These organisations are willing to pay us
to find ways to better support their specific use of Wikimedia content,
both through more accessible technology, contractual guarantees of service
availability, and professional services to help them make the best use of
our content in their systems. As per our enumerated Principles,[10] all
customers get the same product - there are no exclusive or bespoke features
- they only pay for the volume of usage. This will allow smaller companies
to compete and will ensure that Wikimedia knowledge is more widely
available. Our goal is for the future business model of Enterprise to
resemble “many paying a little” rather than “few paying a lot” – an
approach similar to our movement’s “many small donors” fundraising
methodology.
Still to come later this year will be:
-
Exploring options to integrate Wikidata in the dataset, which is a
common customer request. We are working closely with Wikimedia-Deutschland
to discuss how to best do this.
-
Small, and non-U.S. based customers. This is crucial to demonstrating
the Knowledge as a Service value of the project. We already work with
relevant Chapters when we have a potential ‘local’ customer who has
expressed strong interest.
-
Publishing aggregate revenue/expense data, but only after there’s enough
aggregate financial data collected, over a sufficient period of time, and
with enough customers to be informative.
-
The “news” page on the Enterprise website itself will be where future
software updates, customer case-studies, etc. will be published. This
ensures that the information is available, while not detracting from
community-focused places like the Diff blog. The first post on that news
page is available today.[5]
In order to not distract from community-focused discussions, and of course
unless there’s something important for the community to know, in the future
we will announce new customers product updates etc. on the project
website’s news page and on Meta rather than on this mailing list etc. - but
we felt it was important to do it this time.
Finally, I would also like to ask you to keep your eyes and ears open for
anyone in your corner of the Wikimedia community who has questions or
concerns about the project. Please ask them to read and comment on our Meta
FAQ,[6] to contact me directly, or to attend the public community open
meeting on Thursday, which is already announced on Meta on the project’s
page & the mainpage calendar.
Sincerely, and on behalf of the Wikimedia Enterprise team,[11]
* Liam Wyatt [Wittylama]*
Program Manager
for *Enterprise,* *Newcomer Experience, & WikiCite*
Wikimedia Foundation
[1]
https://wikimediafoundation.org/news/2022/06/21/wikimedia-enterprise-announ…
[2] https://meta.wikimedia.org/wiki/Wikimedia_Enterprise
[3]
https://meta.wikimedia.org/wiki/Talk:Wikimedia_Enterprise#Update:_First_cus…
[4] https://enterprise.wikimedia.com/
[5] https://enterprise.wikimedia.com/news/hello-world/
[6]
https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/FAQ#Will_the_community…
[7]
https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Board_noticeboard/Wiki…
[8]
https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2017#Our_strate…
[9] https://openfuture.eu/blog/wikimedia-enterprise/
[10] https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/Principles
[11] https://meta.wikimedia.org/wiki/Wikimedia_Enterprise#Team
On Mon, 11 Oct 2021 at 19:01, Liam Wyatt <lwyatt(a)wikimedia.org> wrote:
> Dear all, A lot has happened since March when we Introduced the Wikimedia
> Enterprise API
> <https://diff.wikimedia.org/2021/03/16/introducing-the-wikimedia-enterprise-…>
> and began community conversations about the project’s development. Now is
> an important milestone to give everyone:
> 1) an update responding to community advice we’ve received; and
> 2) to describe what is happening next.
>
>
> The idea of an API for the specific needs of the commercial sector had
> been discussed for more than a decade
> <https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/FAQ#Where_has_this_pre…>
> (both for the purposes of improving user experience, and also to diversify
> revenue). The announcement in March introducing the Wikimedia Enterprise
> API generated a lot of Wikimedia-community and mainstream-media attention -
> most notably in WIRED
> <https://www.wired.com/story/wikipedia-finally-asking-big-tech-to-pay-up/>.
> Since then, the team has been hard at work building the actual product and
> hosting many conversations (regular public meetings and participation in
> events including SWAN, Wikimania, EMWCon, Clinic) - as well as a
> considerable volume of discussions on our meta talkpage
> <https://xtools.wmflabs.org/articleinfo/meta.wikimedia.org/Talk:Wikimedia%20…>.
> All of this has generated lots of suggestions, which we have endeavoured to
> incorporate and respond to before the actual commercial launch. On behalf
> of the whole team, I thank the many many people who have been willing and
> able to share constructive feedback with us over these months. Links to
> recordings from those meetings/presentations can be found on our meta
> homepage:
>
> https://meta.wikimedia.org/wiki/Wikimedia_Enterprise
>
>
> 1. To that end, here are some updates in response to community advice:
>
>
> WMF Board statement.
>
> Subsequent to the most recent WMF Board of Trustees meeting, a statement
> reaffirming their support of the project, and in particular its operating
> principles relating to its future revenue, has been published. You can find the
> board Statement here
> <https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Board_noticeboard/Wiki…>,
> and the Enterprise operating principles on Meta here
> <https://meta.wikimedia.org/wiki/Special:MyLanguage/Wikimedia_Enterprise/Pri…>.
> Consistent community feedback was that our published principles were good
> and sensible, but for such a new and unusual thing in our movement, an
> overt statement from the Board of Trustees was requested. This statement
> affirms that:
>
> - Revenues of the WMF obtained from commercial activities shall not
> surpass 30% of the total planned revenue via all sources (including
> donations) in that fiscal year, and no further revenue would be sought
> beyond that limit;
> - The Board of Trustees will be notified in advance of any large
> commercial agreements, giving them time for review - exactly mirroring the
> procedure for large gifts;
> - Revenue obtained from Wikimedia Enterprise services is under the
> oversight and control of procedures for revenue raised by the Wikimedia
> Foundation and the revenue will not be earmarked for a specific program.
>
> Each of these things were already noted on-wiki, but they were very much
> worthwhile re-stating formally. Equally it bears repeating: the existing
> APIs and methods of accessing Wikimedia sites remain. The creation of this
> optional commercial service, designed for those with specific high
> data-volume demands, does not change the experience (legally or
> technically) for anyone else.
> Relatedy, and also in response to community suggestions, the formal
> contracts which define the legal relationship of the non-profit Wikimedia
> Foundation to this commercial activity, have now been published on the
> governance wiki and linked from the related section on the project’s
> <https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/FAQ#Why_is_this_being_…>
> FAQ
> <https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/FAQ#Why_is_this_being_…>.
>
>
> Free technical access for the community.
>
> You will soon be able to access a copy of the Enterprise dataset,
> refreshed each fortnight, at the Wikimedia dumps portal
> <http://dumps.wikimedia.org/>. Furthermore a ‘daily dump + hourly diff’
> version is also already available, via Wikimedia Cloud Services
> <https://wikitech.wikimedia.org/wiki/Help:Cloud_Services_Introduction> to
> any users of Toolforge, Cloud VPS, or PAWS. Both of these are provided to
> anyone, for free (in both ‘gratis’ and ‘libre’ senses of the word).
> Importantly, and consistent with community feedback, neither of these
> access methods require any special request process to access them, other
> than the existing terms of service on those platforms.
>
> Software development updates are published monthly on our page on
> MediaWiki <https://www.mediawiki.org/wiki/Wikimedia_Enterprise#Updates>
> (as is the API documentation
> <https://www.mediawiki.org/wiki/Wikimedia_Enterprise/Documentation>), and
> the work is coordinated on our Phabricator board
> <https://phabricator.wikimedia.org/tag/wikimedia_enterprise/>.
>
>
> Next public meeting.
> As mentioned, the team has been holding regular public calls. If you would
> like to meet with, and ask any questions of, the Enterprise team (a.k.a.
> “Office hours”):
>
> Friday October 22 @ 1500 UTC on Zoom
> <https://wikimedia.zoom.us/j/88994018553>. If you would like to arrange a
> conversation about this project with a group in the community that you are
> part of (at a time, language, and meeting-software platform of your
> choice), please contact me directly.
>
>
> 2. What is happening next:
>
>
> The launch of the project’s standalone website, denoting the service as
> “open for business”, will take place early next week.
>
>
> In parallel we will also be announcing the first customers - both from the
> commercial and non-profit sectors. We expect this will generate some
> attention in the media. Despite our best efforts to be visible across the
> wikiverse, reading about the Wikimedia Enterprise API in the media will
> probably be the first time that some Wikimedians hear about it - which
> might be surprising for them. So, next week, if you see any Wikimedians
> asking about this project on community forums, please notify me by email,
> on-wiki [as user:LWyatt (WMF) ], and/or direct them to the project FAQ on
> Meta
> <https://meta.wikimedia.org/wiki/Special:MyLanguage/Wikimedia_Enterprise/FAQ>
> (currently available in 7 languages).
>
> Once again, thank you to all who have been involved in the development of,
> or given feedback to, *Enterprise*. It’s been a long road getting from
> there to here...
>
>
> Sincerely,
>
> * Liam Wyatt [Wittylama]*
>
> Program Manager for *Enterprise, WikiCite,* and *Newcomer Experience*
> Wikimedia Foundation
>
>
> On Tue, 16 Mar 2021 at 17:02, Liam Wyatt <lwyatt-ctr(a)wikimedia.org> wrote:
>
>> Dear all,
>>
>> Over the last few months, a small team at the Wikimedia Foundation has
>> been working on a project that has been discussed by many people in our
>> movement for many years: building ‘enterprise grade’ services for the
>> high-volume commercial reusers of Wikimedia content. I am pleased to say
>> that in a remarkably short amount of time (considering the complexity of
>> the issues: technical, strategic, legal, and financial) we now have
>> something worthy of showing to the community, and we are asking for your
>> feedback. Allow me to introduce you to the Wikimedia Enterprise API project
>> – formerly codenamed “okapi”.
>>
>> While the general idea for Wikimedia Enterprise predates the current
>> movement strategy process, its recommendations identify an enterprise API
>> as one possible solution to both “Increase the sustainability of our
>> movement” and “Improve User Experience.”[0] That is, to simultaneously
>> create a new revenue stream to protect Wikimedia’s sustainability, and
>> improve the quality and quantity of Wikimedia content available to our many
>> readers who do not visit our websites directly (including more consistent
>> attribution). Moreover, it does so in a way that is true to our movement’s
>> culture: with open source software, financial transparency, non-exclusive
>> contracts or content, no restrictions on existing services, and free access
>> for Wikimedia volunteers who need it.
>>
>> The team believes we are on target to achieve those goals and so we have
>> written a lot of documentation to get your feedback about our progress and
>> where it could be further improved before the actual product is ‘launched’
>> in the next few months. We have been helped in this process over the last
>> several months by approximately 100 individual volunteers (from many
>> corners of the wikiverse) and representatives of affiliate organisations
>> who have reviewed our plans and provided invaluable direction, pointing out
>> weaknesses and opportunities, or areas lacking clarity and documentation in
>> our drafts. Thank you to everyone who has shared your time and expertise to
>> help prepare this new initiative.
>>
>> A essay describing the “why?” and the “how?” of this project is now on
>> Meta:
>>
>> https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/Essay
>>
>> Also now published on Meta are an extensive FAQ, operating principles,
>> and technical documentation on MediaWiki.org. You can read these at [1] [2]
>> and [3] respectively. Much of this documentation is already available in
>> French, German, Italian, and Spanish.
>>
>> The Wikimedia Enterprise team is particularly interested in your feedback
>> on how we have designed the checks and balances to this project - to ensure
>> it is as successful as possible at achieving those two goals described
>> above while staying true to the movement’s values and culture. For example:
>> Is everything covered appropriately in the “Principles” list? Is the
>> technical documentation on MediaWiki.org clear? Are the explanations in the
>> “FAQ” about free-access for community, or project’s legal structure, or the
>> financial transparency (etc.) sufficiently detailed?
>>
>> Meet the team and Ask Us Anything:
>>
>> The central place to provide written feedback about the project in
>> general is on the talkpage of the documentation on Meta at:
>> https://meta.wikimedia.org/wiki/Talk:Wikimedia_Enterprise
>>
>> On this Friday (March 19) we will be hosting two “Office hours”
>> conversations where anyone can come and give feedback or ask questions:
>>
>> -
>>
>> 13:00 UTC via Zoom at https://wikimedia.zoom.us/j/95580273732
>> -
>>
>> 22:00 UTC via Zoom at https://wikimedia.zoom.us/j/92565175760 (note:
>> this is Saturday in Asia/Oceania)
>>
>> Other “office hours” meetings can be arranged on-request on a technical
>> platform of your choosing; and we will organise more calls in the future.
>>
>> We will also be attending the next SWAN meetings (on March 21)
>> https://meta.wikimedia.org/wiki/Strategic_Wikimedia_Affiliates_Network,
>> and also the next of the Wikimedia Clinics
>> https://meta.wikimedia.org/wiki/Wikimedia_Clinics
>>
>> Moreover, we would be very happy to accept any invitation to attend an
>> existing group call that would like to discuss this topic (e.g. an
>> affiliate’s members’ meeting).
>>
>>
>> On behalf of the Wikimedia Enterprise team,
>>
>> Peace, Love & Metadata
>> -- Liam Wyatt [Wittylama], Wikimedia Enterprise project community liaison.
>>
>> [0]
>> https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2018-20/Recomme…
>>
>> [1] https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/FAQ
>>
>> [2] https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/Principles
>>
>> [3] https://www.mediawiki.org/wiki/Wikimedia_Enterprise
>>
>> *Liam Wyatt [Wittylama]*
>> WikiCite <https://meta.wikimedia.org/wiki/WikiCite> Program Manager & Wikimedia
>> Enterprise <https://meta.wikimedia.org/wiki/Okapi> Community Liaison
>> Wikimedia Foundation
>>
>>