By this reasoning we should also delete items about categories or disambiguation pages. Thad Guidry, 15/06/2015 17:21: What's the issue here? The item doesn't actually contain any list, there is no duplication or information "clumped together". Nemo

This is an important question. There are apparently 196,839 known list items based on a query for instanceOf Wikipedia list item (CLAIM[31:13406463]) http://tools.wmflabs.org/autolist/autolist1.html?q=CLAIM%5B31%3A13406463%5D I tend to agree with Thad that these kinds of items aren't really what we want filling in WikiData. In fact replacing them with the ability to generate them automatically based on queries is a primary use case for wikidata. But just deleting them doesn't entirely make sense either because they are key signposts into things that ought to be brought into wikidata properly. The items in these lists clearly matter.. Ideally we could generate a bot that would examine each of these lists and identify the unifying properties that should be added to the items within the list that would enable the list to be reproduced by a query. I disagree that this reasoning suggests deleting items about categories and disambiguation pages. - both of these clearly have functions in wikidata. I'm not sure what the function of a list entity is. On Mon, Jun 15, 2015 at 8:47 AM, Federico Leva (Nemo) <nemowiki(a)gmail.com> wrote:

Also the list entity has a function. The function of *instance of* is to identify what a page is about. A database is built on consistency, the list entity does do that for lists. A list is a very special type of a subject in comparison to other articles. It isn't linked through topic type properties. By using a list entity this kind of items are identified as such. Likewise for dps, categories, templates, etc. Romaine 2015-06-15 17:53 GMT+02:00 Benjamin Good <ben.mcgee.good(a)gmail.com>om>:

I think this is clearly an evolutionary process. In the short term, wikidata needs to support Wikipedia use cases as Andrew mentioned above (thank you for the clarification). In the long term, this function and all other functions will (in my opinion) best be served by a transition into more and more of an entity graph where claims are made about things in the world rather than about constructs in a database. Perhaps there is some form of the WikiData game that could be generated to support this process for lists. The intervening period is going to be a challenge in terms of modeling and in application-level hiding of weird ontological situations where objects are being described like (item1: instanceOf, WikipediaList AND item1: subclassOf moons of jupiter), but there is no way around it. And its 100% worthwhile to do whatever it takes to keep things integrated with Wikipedia and to further establish wikidata as indispensable there. -Ben On Mon, Jun 15, 2015 at 9:11 AM, Thad Guidry <thadguidry(a)gmail.com> wrote:

Thad Guidry, 15/06/2015 18:11: I understand the question, but why are the two things in contradiction? Nemo

Am 15.06.2015 um 20:09 schrieb Thad Guidry: I would also like that, for convenience. But conceptually, Wikipedia pages are things in the world, and are "real" in that sense. So if we don't want to introduce a nasty hack into the data model, you'd have to do this by saying "exclude everything that is an instance of MediaWiki page (Q15474042)". That's a rather expensive operation... Can you think of a way to do this nicely, that doesn't need special case hacks in the software? -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

Hoi, Wikidata is not Freebase and no thanks. Thanks, GerardM On 15 June 2015 at 18:07, Thad Guidry <thadguidry(a)gmail.com> wrote:

Hi! Why delete them? Wikidata has a number of things which are not your standard "entity" - lists, sources, news, quotes, service entries, narrative articles (e.g. https://en.wikipedia.org/wiki/Control_of_fire_by_early_humans - it's not exactly "entity" like "human" or "fire"), etc. So I don't think the approach that singles out and excludes lists would help much - if you have an application that needs "individual entities" like "Douglas Adams" or "London" and exclude other types will have to exclude much more than just lists - but I think the approach of asking for exactly what you need and ignoring the rest may prove more efficient. I'm not sure there's really well-defined criteria to specify what "individual entity" actually is - I'm sure you have one that matches your application, but some other application may have completely different one. Generally, this can be solved by better classification I think, but so far I'm not sure what to base this classification on. -- Stas Malyshev smalyshev(a)wikimedia.org

Stas, Always agreed, it's a classification problem. So what claims/statements do I rule out ? Or what should I only rule in (claims/statements) when wanting to return only "real" entities ? Can someone help with those negative claims/statements that I am looking for ? So far, I only have got ​1. ​ filtering out any entry with P31:Q13406463 should omit most ​ ​ of them from your results. All, Freebase simply decided to not keep Wikipedia topic pages that simply held "lists of entities", but instead Freebase liked to easily generate those same "lists of entities" by using queries. There was no need to have hand coded lists in Freebase. It was a graph database and could generate all kinds of lists programmaticlly for a user, and keep those lists as views against our user profile for easy tweaking or re-use when we wanted to. (stored user queries) Thad +ThadGuidry <https://www.google.com/+ThadGuidry> On Mon, Jun 15, 2015 at 2:56 PM, Stas Malyshev &lt;smalyshev(a)wikimedia.org&gt; wrote:

Thanks Stas, Those are useful. Thad +ThadGuidry <https://www.google.com/+ThadGuidry> On Mon, Jun 15, 2015 at 7:46 PM, Stas Malyshev &lt;smalyshev(a)wikimedia.org&gt; wrote:

Il 15/06/2015 23:22, Thad Guidry ha scritto: Wikidata is not Freebase. Its primary use is still to link pages between projects, it'll take some time before it can serve as a real data source, with queries and all that stuff.

I'd agree with Stas - it depends immensely what you mean by "real" entities, and it would be best to define your desired subjects explicitly if possible rather than relying on removing (eg if you want to know about people, put in a filter for P31:Q5). To remove things like disambiguation pages, categories, or lists, you would want (to use Magnus's WDQ syntax) something like claim[31:(tree[17379835][][279])] - anything that is an instance of a subclass of Q17379835, "Wikimedia page outside the main knowledge tree". This will remove (probably) all of our internal admin content. However, defining what constitutes a "list" is challenging; many (most?) WP list articles contain a non-trivial amount of content in addition to the list per se. As a result, some are labelled "lists" (like Q6642364, a list of buildings in a city), while others are notionally articles on the class which also include a list of members (eg Q8344876, a list of recipients of an award). This complexity is not likely to go away any time soon, especially given cases where (say) the English article thinks it's a list of the set X, and the Spanish one thinks it's an article about the set X. Andrew. On 15 June 2015 at 22:22, Thad Guidry &lt;thadguidry(a)gmail.com&gt; wrote: -- - Andrew Gray andrew.gray(a)dunelm.org.uk