tool to do multiple searches in Wikidata at once? - Wikidata

Jane Darnell

23 Jul 23 Jul

7:09 a.m.

Good question! This didn't used to be possible but should be today with random access. When I need this it's generally because I want to add something to those items, and if you are certain of the spelling you can use quick_statements to add the statement and then collect the q numbers that way. The problem with this approach is that you can easily add double statements and you theoretically only want to augment the existing statements per item, not duplicate them. Of course if all items share some common statement you may be able to query that statement and filter the results. Another way I do this is with listeria. For example if I want the artists in a collection, I set up a listeria query for all artworks in the collection and include the creator field. Afterwards I collect the result and filter for the artists' names or q numbers. Obvious problem here is that most collections on Wikidata only include the most popular artworks in the collection, because these are coming from the various wikipedia projects. There are still lots and lots of artworks on Commons without wikidata items.Hope it helps. On Thu, Jul 23, 2015 at 8:44 AM, Sandra Fauconnier < sandra.fauconnier(a)gmail.com> wrote:

...

Hi everyone, I’ve been in the situation quite often (edit-a-thons; various to do lists) where I had a list of terms (most usually names of a few hundreds of people, or titles of Wikipedia articles), where I wanted to do a quick search on Wikidata to retrieve each of these concept’s Q number. Does anyone know of a tool that helps me make this easier? Enter a list of, say, 100 of these search terms, and receive Q number suggestions for each of them? I’ve looked around on wmflabs but have not found anything in that direction (also not with the help of Hay’s awesome tool directory <http://tools.wmflabs.org/hay/directory/#/>). Till now, I’ve done all these searches manually - use an excel sheet, look for each term individually, enter Q number for each term - quite accurate but very time-consuming! Would appreciate all help/tips ! Thanks! Sandra (User:Spinster) _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Reply

Jan Ainali

7:51 a.m.

And if you use that tool you can easily paste your excel sheet in wikiformat as source in the big box labeled wikitext. E.g. By entering: [[Search]] [[Term]] the tool will return Q1267283 Q3984436 *Med vänliga hälsningar,Jan Ainali* Verksamhetschef, Wikimedia Sverige <http://wikimedia.se> 0729 - 67 29 48 *Tänk dig en värld där varje människa har fri tillgång till mänsklighetens samlade kunskap. Det är det vi gör.* Bli medlem. <http://blimedlem.wikimedia.se> 2015-07-23 9:29 GMT+02:00 Gerard Meijssen <gerard.meijssen(a)gmail.com>om>:

...

Hoi, When the list is a list of articles in a Wikipedia, try "Linked items" one magnificent tool by Magnus that can be used for this.. Thanks, GerardM https://tools.wmflabs.org/wikidata-todo/linked_items.php On 23 July 2015 at 08:44, Sandra Fauconnier <sandra.fauconnier(a)gmail.com> wrote:

Hi everyone, I’ve been in the situation quite often (edit-a-thons; various to do lists) where I had a list of terms (most usually names of a few hundreds of people, or titles of Wikipedia articles), where I wanted to do a quick search on Wikidata to retrieve each of these concept’s Q number. Does anyone know of a tool that helps me make this easier? Enter a list of, say, 100 of these search terms, and receive Q number suggestions for each of them? I’ve looked around on wmflabs but have not found anything in that direction (also not with the help of Hay’s awesome tool directory <http://tools.wmflabs.org/hay/directory/#/>). Till now, I’ve done all these searches manually - use an excel sheet, look for each term individually, enter Q number for each term - quite accurate but very time-consuming! Would appreciate all help/tips ! Thanks! Sandra (User:Spinster) _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Reply

Ed Summers

8:20 p.m.

Hi Sandra, I suspect this is a bit too nerdy, but I was reconciling some messy CSV data to Wikidata recently and ended up writing a little Python library and command line tool that uses wbgetentities API call. https://github.com/edsu/wikidata_suggest I also created an awkward video I did for Day of Digital Humanities here: https://vimeo.com/128305641 //Ed

...

On Jul 23, 2015, at 3:29 AM, Gerard Meijssen <gerard.meijssen(a)gmail.com> wrote: Hoi, When the list is a list of articles in a Wikipedia, try "Linked items" one magnificent tool by Magnus that can be used for this.. Thanks, GerardM https://tools.wmflabs.org/wikidata-todo/linked_items.php On 23 July 2015 at 08:44, Sandra Fauconnier <sandra.fauconnier(a)gmail.com> wrote: Hi everyone, I’ve been in the situation quite often (edit-a-thons; various to do lists) where I had a list of terms (most usually names of a few hundreds of people, or titles of Wikipedia articles), where I wanted to do a quick search on Wikidata to retrieve each of these concept’s Q number. Does anyone know of a tool that helps me make this easier? Enter a list of, say, 100 of these search terms, and receive Q number suggestions for each of them? I’ve looked around on wmflabs but have not found anything in that direction (also not with the help of Hay’s awesome tool directory). Till now, I’ve done all these searches manually - use an excel sheet, look for each term individually, enter Q number for each term - quite accurate but very time-consuming! Would appreciate all help/tips ! Thanks! Sandra (User:Spinster) _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Reply

Stryn

27 Jul 27 Jul

6:10 p.m.

Oh, what a handy tool! That's what I've been looking for. Only bad is that it doesn't support redirects. Now I'm waiting for getting the arbitrary access enabled on the Finnish Wikipedia so I can try cool things :) *Stryn* *Suomenkielisen Wikipedian ylläpitäjä & osoitepaljastaja / Admin and checkuser on the Finnish Wikipedia* *Wikidatan ylläpitäjä / Admin on Wikidata* *Meta-Wikin ylläpitäjä / Admin on Meta-Wiki* *Ylivalvoja / Steward* 2015-07-23 10:29 GMT+03:00 Gerard Meijssen <gerard.meijssen(a)gmail.com>om>:

...

Hoi, When the list is a list of articles in a Wikipedia, try "Linked items" one magnificent tool by Magnus that can be used for this.. Thanks, GerardM https://tools.wmflabs.org/wikidata-todo/linked_items.php On 23 July 2015 at 08:44, Sandra Fauconnier <sandra.fauconnier(a)gmail.com> wrote:

Hi everyone, I’ve been in the situation quite often (edit-a-thons; various to do lists) where I had a list of terms (most usually names of a few hundreds of people, or titles of Wikipedia articles), where I wanted to do a quick search on Wikidata to retrieve each of these concept’s Q number. Does anyone know of a tool that helps me make this easier? Enter a list of, say, 100 of these search terms, and receive Q number suggestions for each of them? I’ve looked around on wmflabs but have not found anything in that direction (also not with the help of Hay’s awesome tool directory <http://tools.wmflabs.org/hay/directory/#/>). Till now, I’ve done all these searches manually - use an excel sheet, look for each term individually, enter Q number for each term - quite accurate but very time-consuming! Would appreciate all help/tips ! Thanks! Sandra (User:Spinster) _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Reply

Tom Morris

11:43 p.m.

I'm biased, but I think this is a perfect use case for OpenRefine. What you'd need is an OpenRefine compatible "reconcilation service" provided by Wikidata that OpenRefine could use and then it'd allow you to match an entire column in spreadsheet, set a threshold for automatic matches, review candidates for those below the threshold, etc. https://github.com/OpenRefine/OpenRefine/wiki/Reconcilable-Data-Sources https://api.opencorporates.com/documentation/Google-Refine-Reconciliation-A… http://refine.codefork.com/ https://wiki.duraspace.org/display/VIVO/Extending+Google+Refine+for+VIVO Tom On Thu, Jul 23, 2015 at 2:44 AM, Sandra Fauconnier < sandra.fauconnier(a)gmail.com> wrote:

...

Hi everyone, I’ve been in the situation quite often (edit-a-thons; various to do lists) where I had a list of terms (most usually names of a few hundreds of people, or titles of Wikipedia articles), where I wanted to do a quick search on Wikidata to retrieve each of these concept’s Q number. Does anyone know of a tool that helps me make this easier? Enter a list of, say, 100 of these search terms, and receive Q number suggestions for each of them? I’ve looked around on wmflabs but have not found anything in that direction (also not with the help of Hay’s awesome tool directory <http://tools.wmflabs.org/hay/directory/#/>). Till now, I’ve done all these searches manually - use an excel sheet, look for each term individually, enter Q number for each term - quite accurate but very time-consuming! Would appreciate all help/tips ! Thanks! Sandra (User:Spinster) _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Reply

Magnus Manske

8:26 a.m.

On it... On Fri, Jul 24, 2015 at 1:41 AM Thad Guidry <thadguidry(a)gmail.com> wrote:

...

Yeah it would be awesome if Wikidata could hack together a OpenRefine compatible "reconcilation service" ! Librarians use OpenRefine by the way ! (wink wink) ;) More data, more data, more data Thad +ThadGuidry <https://www.google.com/+ThadGuidry> On Thu, Jul 23, 2015 at 6:43 PM, Tom Morris <tfmorris(a)gmail.com> wrote:

I'm biased, but I think this is a perfect use case for OpenRefine. What you'd need is an OpenRefine compatible "reconcilation service" provided by Wikidata that OpenRefine could use and then it'd allow you to match an entire column in spreadsheet, set a threshold for automatic matches, review candidates for those below the threshold, etc. https://github.com/OpenRefine/OpenRefine/wiki/Reconcilable-Data-Sources https://api.opencorporates.com/documentation/Google-Refine-Reconciliation-A… http://refine.codefork.com/ https://wiki.duraspace.org/display/VIVO/Extending+Google+Refine+for+VIVO Tom On Thu, Jul 23, 2015 at 2:44 AM, Sandra Fauconnier < sandra.fauconnier(a)gmail.com> wrote:

Hi everyone, I’ve been in the situation quite often (edit-a-thons; various to do lists) where I had a list of terms (most usually names of a few hundreds of people, or titles of Wikipedia articles), where I wanted to do a quick search on Wikidata to retrieve each of these concept’s Q number. Does anyone know of a tool that helps me make this easier? Enter a list of, say, 100 of these search terms, and receive Q number suggestions for each of them? I’ve looked around on wmflabs but have not found anything in that direction (also not with the help of Hay’s awesome tool directory <http://tools.wmflabs.org/hay/directory/#/>). Till now, I’ve done all these searches manually - use an excel sheet, look for each term individually, enter Q number for each term - quite accurate but very time-consuming! Would appreciate all help/tips ! Thanks! Sandra (User:Spinster) _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Reply

Info WorldUniversity

9:55 a.m.

Great, Magnus, And so wiki-quick! Cheers, Scott On Jul 24, 2015 2:38 AM, "Magnus Manske" <magnusmanske(a)googlemail.com> wrote:

...

Early version: https://tools.wmflabs.org/wikidata-reconcile/ On Fri, Jul 24, 2015 at 9:26 AM Magnus Manske <magnusmanske(a)googlemail.com> wrote:

On it... On Fri, Jul 24, 2015 at 1:41 AM Thad Guidry <thadguidry(a)gmail.com> wrote:

Yeah it would be awesome if Wikidata could hack together a OpenRefine compatible "reconcilation service" ! Librarians use OpenRefine by the way ! (wink wink) ;) More data, more data, more data Thad +ThadGuidry <https://www.google.com/+ThadGuidry> On Thu, Jul 23, 2015 at 6:43 PM, Tom Morris <tfmorris(a)gmail.com> wrote:

I'm biased, but I think this is a perfect use case for OpenRefine. What you'd need is an OpenRefine compatible "reconcilation service" provided by Wikidata that OpenRefine could use and then it'd allow you to match an entire column in spreadsheet, set a threshold for automatic matches, review candidates for those below the threshold, etc. https://github.com/OpenRefine/OpenRefine/wiki/Reconcilable-Data-Sources https://api.opencorporates.com/documentation/Google-Refine-Reconciliation-A… http://refine.codefork.com/ https://wiki.duraspace.org/display/VIVO/Extending+Google+Refine+for+VIVO Tom On Thu, Jul 23, 2015 at 2:44 AM, Sandra Fauconnier < sandra.fauconnier(a)gmail.com> wrote: > Hi everyone, > > I’ve been in the situation quite often (edit-a-thons; various to do > lists) where I had a list of terms (most usually names of a few hundreds of > people, or titles of Wikipedia articles), where I wanted to do a quick > search on Wikidata to retrieve each of these concept’s Q number. > Does anyone know of a tool that helps me make this easier? Enter a > list of, say, 100 of these search terms, and receive Q number suggestions > for each of them? I’ve looked around on wmflabs but have not found anything > in that direction (also not with the help of Hay’s awesome tool > directory <http://tools.wmflabs.org/hay/directory/#/>). > > Till now, I’ve done all these searches manually - use an excel sheet, > look for each term individually, enter Q number for each term - quite > accurate but very time-consuming! > > Would appreciate all help/tips ! > Thanks! Sandra (User:Spinster) > > _______________________________________________ > Wikidata mailing list > Wikidata(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikidata > > _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Reply

Magnus Manske

1:10 p.m.

Ed, now generating an empty type array, so that your query should work; will fill soon-ish ;-) On Fri, Jul 24, 2015 at 1:31 PM Ed Summers <ehs(a)pobox.com> wrote:

...

On Jul 24, 2015, at 5:37 AM, Magnus Manske <magnusmanske(a)googlemail.com>

wrote:

Early version: https://tools.wmflabs.org/wikidata-reconcile/

Wow, that was crazy fast Magnus :) I gave it a try in the latest OpenRefine on a very simple spreadsheet: city Paris New York City OpenRefine threw an exception (see below). I think OpenRefine is expecting the type key in each result. From the documentation [1] it looks like the value for the type key should be a array of types? I don’t know if it’s easy to get at that in your app. Speaking of which is the code available somewhere? Awesome work! //Ed [1] https://github.com/OpenRefine/OpenRefine/wiki/Reconciliation-Service-API#qu… //Ed org.json.JSONException: JSONObject["type"] not found. at org.json.JSONObject.get(JSONObject.java:406) at org.json.JSONObject.getJSONArray(JSONObject.java:482) at com.google.refine.commands.recon.GuessTypesOfColumnCommand.guessTypes(GuessTypesOfColumnCommand.java:216) at com.google.refine.commands.recon.GuessTypesOfColumnCommand.doPost(GuessTypesOfColumnCommand.java:89) at com.google.refine.RefineServlet.service(RefineServlet.java:177) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81) at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:132) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Reply