Thomas, Denny, Sebastian, Thomas, and I have published a paper which was accepted for the industry track at WWW 2016. It covers the migration from Freebase to Wikidata. You can now read it here: http://research.google.com/pubs/archive/44818.pdf

Concluding, in a fairly short amount of time, we have been able to provide the Wikidata community with more than 14 million new Wikidata statements using a customizable

after removing dupli cates and facts already contained in Wikidata, we obtain 14 million new statements. If all these statements were added to Wikidata, we would see a 21% increase of the num- ber of statements in Wikidata.

Hey everyone :) Thomas, Denny, Sebastian, Thomas, and I have published a paper which was accepted for the industry track at WWW 2016. It covers the migration from Freebase to Wikidata. You can now read it here: http://research.google.com/pubs/archive/44818.pdf Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Hoi, I add statements of the primary sources tool in preference to add them myself (Primary Sources takes more time). I am still of the strongest opinion that given the extremely disappointing number of added statements the Primary Sources tool is a failure.

It is sad that all the good work of Freebase is lost in this way. It is sad that we cannot even discuss this and consider alternatives. Thanks, GerardM On 18 February 2016 at 18:07, Maximilian Klein &lt;isalix(a)gmail.com <mailto:isalix@gmail.com>> wrote: Congratulations on a fantastic project and a your acceptance in WWW2016. Make a great day, Max Klein ‽ http://notconfusing.com/ On Thu, Feb 18, 2016 at 10:54 AM, Federico Leva (Nemo) &lt;nemowiki(a)gmail.com <mailto:nemowiki@gmail.com>> wrote: Lydia Pintscher, 18/02/2016 15:59: Thomas, Denny, Sebastian, Thomas, and I have published a paper which was accepted for the industry track at WWW 2016. It covers the migration from Freebase to Wikidata. You can now read it here: http://research.google.com/pubs/archive/44818.pdf Nice! I must admit that, despite knowing the context, I wasn't able to understand whether this is the number of "mapped"/"translated" statements or the number of statements actually added via the primary sources tool. I assume the latter given paragraph 5.3: I was confused about that too. "the [Primary Sources] tool has been used by more than a hundred users who performed about 90,000 approval or rejection actions. More than 14 million statements have been uploaded in total." I think that means that ≤ 90,000 items or statements were added of 14 million available to be add through Primary Sources tool. Nemo _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Now that we're at it, I'll highlight a usability problem I have with the primary source tool : it reloads the page every time we approve something, which can take a lot of times for heavy pages, and is really a blocker to approve a large number of claims :)

On 18.02.2016 15:59, Lydia Pintscher wrote: Is it possible that you have actually used the flawed statistics from the Wikidata main page regarding the size of the project? 14.5M items in Aug 2015 seems far too low a number. Our RDF exports from mid August already contained more than 18.4M items. It would be nice to get this fixed at some point. There are currently almost 20M items, and the main page still shows only 16.5M.

On 21.02.2016 20:37, Tom Morris wrote: Obviously, any tool can only import statements for which we have items and properties at all, so the number of importable facts is much lower.

On 22.02.2016 18:28, Tom Morris wrote: That estimate might be a bit off. The paper contains a detailed discussion of this aspect.

The total number of statements that could be translated from Freebase to Wikidata is given as 17M, of which only 14M were new. So this seems to be the current upper bound of what you could import with PS or any other tool.

The authors mention that this already includes more than 90% of the "reviewed" content of Freebase that refers to Wikidata items. The paper seems to suggest that these mapped+reviewed statements were already imported directly -- maybe Lydia could clarify if this was the case.

It seems that if you want to go to the dimensions that you refer to (50M/300M/3200M) you would need to map more Wikidata items to Freebase topics in some way. The paper gives several techniques that were used to obtain mappings that are already more than what we have stored in Wikidata now. So it is probably not the lack of mappings but the lack of items that is the limit here. Data can only be imported if we have a page at all ;-)

Btw. where do the 100K imported statements come from that you mentioned here? I was also interested in that number but I could not find it in the paper.

Or the paper might be off. Addressing the flaws in the paper would require a full paper in its own right.

The paper says in section 4, "At the time of writing (January, 2016), the tool has been used by more than a hundred users who performed about 90,000 approval or rejection actions." which probably means ~80,000 new statements (since ~10% get rejected). My 106K number is from the current dashboard <https://tools.wmflabs.org/wikidata-primary-sources/status.html>.

Yes, I do that sometimes too - if there is a statement saying "spouse: X" on wikidata, and statement in Freebase saying the same but with the start date, or the Freebase one has more precise date than the Wikidata one, such as full date instead of just year, I will modify the original statement and reject the Freebase one.

I'm not sure this is the best practice with regard to tracking numbers but it's easiest and even if my personal numbers do not matter too much I imagine other people do this too. So rejection does not really mean the data was not entered - it may mean it was entered in a different way. Sometimes also while the data is already there, the reference is not, so the reference gets added.