On 1 Mar 2016, at 11:12 AM, Kevin Smith ksmith@wikimedia.org wrote:
I think some people aren't realizing the difference between the leaked presentation (which outlined a general search engine) and the actual grant. The former was just an idea, while the latter is official. By my reading, the grant clearly is NOT for a general internet search engine, although it (unfortunately) did retain a bit of the language from earlier documents.
With the greatest of respect, I'm not sure how could come to the conclusion that general Internet search was not a core component of the Knowledge Engine.
I'm just going to quote directly from the Grant application here [1]:
Knowledge Engine By Wikipedia will democratize the discovery of media, news and information—it will make the Internet's most relevant information more accessible and openly curated, and it will create an open data engine that's completely free of commercial interests. Our new site will be the Internet’s first transparent search engine, and the first one that carries the reputation of Wikipedia and the Wikimedia Foundation.
So to reiterate the words that make it hard for the WMF to deny that they were pitching for an Internet search engine:
Our new site will be the Internet's first transparent search engine, and the first one that carries the reputation of Wikipedia and the Wikimedia Foundation.
For context, this is the answer to the grant application question "Opportunity: What is the overall challenge being addressed? What is the proposed approach? And what evidence is there that this approach will work?"
The grant application also states that one challenge that could disrupt the project is:
Third-party influence or interference. Google, Yahoo or another big commercial search engine could suddenly devote resources to a similar project, which could reduce the success of the project. This is the biggest challenge, and an external one.
It truly strains credibility that an internal search engine merely indexing internal sites could be threatened by either Google or Yahoo devoting resources equal to or greater than the grant money allocated to this project, just to index Wikimedia properties. Similarly, it makes no sense to me how you can "democratize the discovery of media, news, and information" to "make the Internet's most relevant information more accessible and openly curated" without pulling that information from...the Internet!
And of course, to risk repeating myself, the next line states that "our new site will be the Internet's first transparent search engine".
You can tell me the scope was intended to be only for Wikimedia projects, but that isn't what is said in that grant application. That document as it stands literally states that it is to be an Internet search engine. No, I correct myself. It says it is to be THE Internet's search engine.
So when you say than there is confusion between the internal presentation and the official external grant application, I must respectfully disagree with you. There is no such confusion. The two parts of the application I have quoted cover almost a third of the grant application and I'd argue are the key parts of the application.
If fully one third of the grant application seem to be ambiguous or even flat wrong - and key parts at that! - then it's not just "unfortunate" that a "bit" of the language of the presentation remained in the grant application accidentally. That's sheer downright incompetence. Lila signed off on this document, and it was reviewed by others. I don't know who vetted and drafted this, but the buck stops with Lila, and she has never acknowledged her part in the language and scope of this application aside from once stating in a Discovery team meeting [2] that:
How do we explain the story now? The original idea was a broader concept. Never a crawler. We abandoned some ideas during the ideation phase, but we haven’t been clear what/when we abandoned.
I mean, we have here an admission from Lila that it is unclear to the wider community and even WMF staff what they have and haven't abandoned! Why have they assumed that the Knight Foundation would take anything from that grant application that most of us here, the Press, and interested members of the general public would not conclude from merely reading the document?
There has been some handwaving going on from a variety of different parties that "oh, it's just a Grant application, these things are very high level and vague, it doesn't really matter what we write in it lets just put the broadest possible objectives and vision for this thing and we'll deal the scope later on after we've been given the grant money".
Others may not think this is not a concern. I do though, and I'm very concerned that we are making grant applications and not really disclosing our full intentions, and we are not making it clear what are the corresponding scope limitations. Before someone objects, it's even worse when I have asked about the first challenge that could threaten the project and the response [3] is, in part:
Why is Google mentioned? Because they are the undisputed giants of search. If they wanted to dedicated even a small amount of their resources to creating a “Wikisearch” or “Free Knowledge Search” they could do so with ease. This is a risk because the foundation could invest both money and time into improving our search capabilities in an attempt to better surface Wikimedia content, only to be upended by those with more fiscal, staff, and technical resources. When submitting grants you have to be honest about stuff like this so that the grantor doesn’t get surprised down the road.
[...]
Please do understand that grant language is not the same as technical writing. The knight foundation is not a technical organization. The grant, again written my many authors over a substantial period of time, is written to entice and explain simply how we might use the funds.
This is very, very worrying. The explanation given in that comment is given in no more than three sentences, yet if it is a clearer and more accurate reflection of the stated threat than that given in the original document! I mean, we aren't talking about paragraphs of written, turgid prose written by someone like myself on internal Wikimedia mailing lists. It's a pithy, well written and rather excellent explanation on the apparent threat to the project.
And this comment that grant writing isn't like technical writing... Well yes and no. More yes than no. But grant applications - especially ones that are applying for millions of dollars in funds for charitable endeavours - must be clear and not ambiguous about the purpose(s) of the funding.
Nobody can say that the Knight Foundation grant application was clear.
So... I think I've made myself clear now. We can and do understand the differences between those two documents. I don't think anyone is terrible confused by them.
Chris
1. https://upload.wikimedia.org/wikipedia/foundation/a/a7/Knowledge_engine_gran...
2. https://m.mediawiki.org/wiki/Discovery/2016-02-16_Discussing_Knowledge_Engin...
3. https://blog.wikimedia.org/2016/02/16/wikimedia-search-future/#comment-25090