Hello, I try to get all properties P31 "is instance of" by using Magnus Wikidata query [1]. This was not successfully at different days. I need it to update my geocoordinates projects Wikipedia-world[2] and it was working in May 2014.
I think data access is also important for other projects and I see no alternative, so somebody should work on this problem with high priority.
Greeetings Tim alias Kolossos
[1] wget "wdq.wmflabs.org/api?q=claim[31]&props=31" [2] http://lists.wikimedia.org/pipermail/wikidata-l/2014-May/003838.html
Hoi, There are problems running projects in Labs. It is not that there is no attention when something fails, there is, but it just does not have the priority nor do labs ops have the capacity to do a thorough job.
Yes, tools like WDQ and Autolist, Autolist2 are extremely important but it is seen as a fringe project. For Magnus it is not his day job ... I am extremely grateful for the great job he does. It is just one of those things. Thanks, GerardM
On 15 July 2014 23:01, Kolossos tim@alder-digital.de wrote:
Hello, I try to get all properties P31 "is instance of" by using Magnus Wikidata query [1]. This was not successfully at different days. I need it to update my geocoordinates projects Wikipedia-world[2] and it was working in May 2014.
I think data access is also important for other projects and I see no alternative, so somebody should work on this problem with high priority.
Greeetings Tim alias Kolossos
[1] wget "wdq.wmflabs.org/api?q=claim[31]&props=31" [2] http://lists.wikimedia.org/pipermail/wikidata-l/2014-May/003838.html
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
The problem here is not running the query (which works fine [1]), but the gigantic return string. Right now, I do generate that entire string in memory and then output it. This works fine for most queries, but your example is a good percentage of Wikidata; think "uncompressed dump".
I'll have a look if I find some time; this is unlikely solved with a 5-minute-hack ;-)
In the meantime, either download and parse the latest dump (should only be a few days old), or, depending on your usage, query WDQ live if possible.
Cheers, Magnus
[1] http://wdq.wmflabs.org/api?q=claim%5B31%5D&noitems=1
On Tue, Jul 15, 2014 at 10:08 PM, Gerard Meijssen <gerard.meijssen@gmail.com
wrote:
Hoi, There are problems running projects in Labs. It is not that there is no attention when something fails, there is, but it just does not have the priority nor do labs ops have the capacity to do a thorough job.
Yes, tools like WDQ and Autolist, Autolist2 are extremely important but it is seen as a fringe project. For Magnus it is not his day job ... I am extremely grateful for the great job he does. It is just one of those things. Thanks, GerardM
On 15 July 2014 23:01, Kolossos tim@alder-digital.de wrote:
Hello, I try to get all properties P31 "is instance of" by using Magnus Wikidata query [1]. This was not successfully at different days. I need it to update my geocoordinates projects Wikipedia-world[2] and it was working in May 2014.
I think data access is also important for other projects and I see no alternative, so somebody should work on this problem with high priority.
Greeetings Tim alias Kolossos
[1] wget "wdq.wmflabs.org/api?q=claim[31]&props=31" [2] http://lists.wikimedia.org/pipermail/wikidata-l/2014-May/003838.html
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Update: Found the problem.
Since you can ask for properties unrelated to the actual query, the output generator looks for each item in the result set in the property values. Your query returns about ~10M items, and then get the values for these (from a set of ~10M values). That's 10M queries on a data set with 10M entries. That actually works, and generates the correct output, but by that time the connection has gone bye-bye.
I'll try to fix it tonight. Should be possible to speed up the property lookup considerably.
Cheers, Magnus
On Tue, Jul 15, 2014 at 10:26 PM, Magnus Manske <magnusmanske@googlemail.com
wrote:
The problem here is not running the query (which works fine [1]), but the gigantic return string. Right now, I do generate that entire string in memory and then output it. This works fine for most queries, but your example is a good percentage of Wikidata; think "uncompressed dump".
I'll have a look if I find some time; this is unlikely solved with a 5-minute-hack ;-)
In the meantime, either download and parse the latest dump (should only be a few days old), or, depending on your usage, query WDQ live if possible.
Cheers, Magnus
[1] http://wdq.wmflabs.org/api?q=claim%5B31%5D&noitems=1
On Tue, Jul 15, 2014 at 10:08 PM, Gerard Meijssen < gerard.meijssen@gmail.com> wrote:
Hoi, There are problems running projects in Labs. It is not that there is no attention when something fails, there is, but it just does not have the priority nor do labs ops have the capacity to do a thorough job.
Yes, tools like WDQ and Autolist, Autolist2 are extremely important but it is seen as a fringe project. For Magnus it is not his day job ... I am extremely grateful for the great job he does. It is just one of those things. Thanks, GerardM
On 15 July 2014 23:01, Kolossos tim@alder-digital.de wrote:
Hello, I try to get all properties P31 "is instance of" by using Magnus Wikidata query [1]. This was not successfully at different days. I need it to update my geocoordinates projects Wikipedia-world[2] and it was working in May 2014.
I think data access is also important for other projects and I see no alternative, so somebody should work on this problem with high priority.
Greeetings Tim alias Kolossos
[1] wget "wdq.wmflabs.org/api?q=claim[31]&props=31" [2] http://lists.wikimedia.org/pipermail/wikidata-l/2014-May/003838.html
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l