Lately I've merged Wikipedia usage information with DBpedia information to produce a subjective importance score:
https://github.com/paulhoule/telepath/wiki/SubjectiveEye3D
That particular product has two layers of processing that can be done independently: one is averaging over time, the other is reconciling the {project,page URI} tuples to concepts. In the case of DBpedia, the main thing that happens is that importance is summed up for all of the redirects that point to a particular concept.
The product linked above was built with the English Dbpedia, but it ought to be easy to do the same thing with a different ?lang Dbpedia, and get products that reflect the point of view of different ?lang zones.
I don't see redirect data in Wikidata, so it seems that redirects should be processed against DBpedia of all languages and then these should be merged with the contents of the "Wikipedia pages linked to this Item" that I see on a page like this:
http://www.wikidata.org/wiki/Q1631090
does this make sense? What's the easy way to get bulk access to that information? Who else wants to see this happen?
Hi Paul,
I don't know exactly how your "importance" ranking is defined, but it seems like you are making some "PageRank-y" inferences based on the incoming links, and incoming redirects.
Putting aside the unprovedness of redirects conferring PageRank for a moment.
Your question as I understand it, is how to find all the redirects, of all the sitelinks, of a given WikidataQID. In general I don't think this is easy, but a complicated solution may not be too difficult to compute. Redirect pages are just pages in the main namespace that have the "#REDIRECT" syntax. That means you can find them by transclusion, and some filtering. Like in this API call
http://en.wikipedia.org/w/index.php?title=Special:WhatLinksHere/Deadpool&...
(It also seems that this tool will do it: http://toolserver.org/~dispenser/cgi-bin/rdcheck.py?page=Deadpool )
But if you were going to look at ALL the redirects anyway, another way to do this might be to start looking at redirects and matching them into your preexisting targets. You could do this using the database replicas on Wikimedia labs and the redirect table https://www.mediawiki.org/wiki/Manual:Redirect_table
Maximilian Klein Wikipedian in Residence, OCLC +17074787023
________________________________________ From: wikidata-l-bounces@lists.wikimedia.org wikidata-l-bounces@lists.wikimedia.org on behalf of Paul Houle ontology2@gmail.com Sent: Friday, February 14, 2014 10:21 AM To: wikidata-l@lists.wikimedia.org Subject: [Wikidata-l] Reconciliation of Wikipedia usage data with Wikidata?
Lately I've merged Wikipedia usage information with DBpedia information to produce a subjective importance score:
https://github.com/paulhoule/telepath/wiki/SubjectiveEye3D
That particular product has two layers of processing that can be done independently: one is averaging over time, the other is reconciling the {project,page URI} tuples to concepts. In the case of DBpedia, the main thing that happens is that importance is summed up for all of the redirects that point to a particular concept.
The product linked above was built with the English Dbpedia, but it ought to be easy to do the same thing with a different ?lang Dbpedia, and get products that reflect the point of view of different ?lang zones.
I don't see redirect data in Wikidata, so it seems that redirects should be processed against DBpedia of all languages and then these should be merged with the contents of the "Wikipedia pages linked to this Item" that I see on a page like this:
http://www.wikidata.org/wiki/Q1631090
does this make sense? What's the easy way to get bulk access to that information? Who else wants to see this happen?
-- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2@gmail.com
_______________________________________________ Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
The score that I'm using is derived from the Wikipedia pagecounts data
http://dumps.wikimedia.org/other/pagecounts-raw/
which is (sort of) a direct sample of people's interest in the topics, so it's very different from the PageRank data computed by Andreas Thalhammer for wiki-en, see
http://people.aifb.kit.edu/ath/
Anyway, DBpedia extracts the redirects and even does the transitive closure on them, so it's not a difficult problem (in principle) to resolve the redirects using DBpedia data and then join against Wikidata.
On Mon, Feb 17, 2014 at 12:12 PM, Klein,Max kleinm@oclc.org wrote:
Hi Paul,
I don't know exactly how your "importance" ranking is defined, but it seems like you are making some "PageRank-y" inferences based on the incoming links, and incoming redirects.
Putting aside the unprovedness of redirects conferring PageRank for a moment.
Your question as I understand it, is how to find all the redirects, of all the sitelinks, of a given WikidataQID. In general I don't think this is easy, but a complicated solution may not be too difficult to compute. Redirect pages are just pages in the main namespace that have the "#REDIRECT" syntax. That means you can find them by transclusion, and some filtering. Like in this API call
http://en.wikipedia.org/w/index.php?title=Special:WhatLinksHere/Deadpool&...
(It also seems that this tool will do it: http://toolserver.org/~dispenser/cgi-bin/rdcheck.py?page=Deadpool )
But if you were going to look at ALL the redirects anyway, another way to do this might be to start looking at redirects and matching them into your preexisting targets. You could do this using the database replicas on Wikimedia labs and the redirect table https://www.mediawiki.org/wiki/Manual:Redirect_table
Maximilian Klein Wikipedian in Residence, OCLC +17074787023
From: wikidata-l-bounces@lists.wikimedia.org wikidata-l-bounces@lists.wikimedia.org on behalf of Paul Houle ontology2@gmail.com Sent: Friday, February 14, 2014 10:21 AM To: wikidata-l@lists.wikimedia.org Subject: [Wikidata-l] Reconciliation of Wikipedia usage data with Wikidata?
Lately I've merged Wikipedia usage information with DBpedia information to produce a subjective importance score:
https://github.com/paulhoule/telepath/wiki/SubjectiveEye3D
That particular product has two layers of processing that can be done independently: one is averaging over time, the other is reconciling the {project,page URI} tuples to concepts. In the case of DBpedia, the main thing that happens is that importance is summed up for all of the redirects that point to a particular concept.
The product linked above was built with the English Dbpedia, but it ought to be easy to do the same thing with a different ?lang Dbpedia, and get products that reflect the point of view of different ?lang zones.
I don't see redirect data in Wikidata, so it seems that redirects should be processed against DBpedia of all languages and then these should be merged with the contents of the "Wikipedia pages linked to this Item" that I see on a page like this:
http://www.wikidata.org/wiki/Q1631090
does this make sense? What's the easy way to get bulk access to that information? Who else wants to see this happen?
-- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2@gmail.com
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l