Hi there,
As of yesterday, lib.reviews supports searching Wikidata, so you can quickly write a review (think "1-5 stars") of anything with a Wikidata entry. See more here:
https://lib.reviews/team/developers/post/b2245981-0e59-427f-a7fe-8e685b82765...
lib.reviews is an open source, free content, multilingual, nonprofit platform for reviewing anything (except living people). Registration still needs invite codes (until we have better anti-spam and moderation tools) -- feel free to email me offlist if you want one.
== Implementation ==
Autocompletion is done via the wbsearchentities API, and individual entries are looked up via wbgetentities. We perform client-side filtering to remove results that include phrases such as "Wikimedia disambiguation page", which are not relevant in this context. (This is done via a regex blacklist that can be different for each language.) The client performs a "shallow" lookup for only the information the user cares about, while upon save, the server performs a "deep" lookup, e.g., it gets descriptions in languages the user doesn't care about.
Each review is associated with a "thing" (similar to an item in Wikidata), and when you review via Wikidata, labels and descriptions in all supported lib.reviews languages are loaded automatically. We don't "sync" this data regularly yet -- that's an upcoming feature.
You can either use the search box, or put a Wikidata URL directly into the URL input field -- the information is looked up either way. And if you don't have JavaScript enabled, we still fetch all the item metadata server-side for any Wikidata URL.
== What's Next ==
In addition to automatic synchronization, we'll provide increasing support for metadata that's relevant to reviews. These features are intricately linked: as much as possible, we want to avoid duplicating work, so pulling from free/open databases and keeping that information up to date is our preferred way of handling review subject metadata.
The system we use for plugging into Wikidata is meant to evolve into a generic "adapter" architecture. The next adapter will likely be OpenStreetMap, so it's possible to select an object on a map (e.g., a restaurant) and use that as a starting point for a review. OSM has a lot of data that Wikidata doesn't, e.g., opening times for shops and restaurants.
If more people start using it, it may also make sense to write a bot to back-link from Wikidata to lib.reviews (there's already a property for this: https://www.wikidata.org/wiki/Property:P3308 ).
== Questions and Notes ==
- Is there a better way to exclude disambiguation pages and similar meta-content from an autocompletion than what I'm currently doing (client-side filtering)? I didn't find a way to restrict wbsearchentities results by claim, for example.
- Progress on https://phabricator.wikimedia.org/T97566 would help a lot to clean up descriptions on some items. For applications like this one, items with descriptions like "for the best-known species, see Q10757112; for the genus, see Q8666090" are really problematic.
== Getting Involved ==
The most important thing for the success of the project right now is slowly growing the community of reviewers. So, if you want to review books, movies, etc., please ask me for a code and get involved! If you've written reviews on sites like IMDB or Goodreads, I wrote a browser extension called https://freeyourstuff.cc/ a while ago that lets you export these -- there's no automatic import feature for lib.reviews yet, but it should still save some time when copying things over. Besides, FYS provides online backup to a community-mirrored database, so it's a good way to liberate your content either way.
As for development, the project is fully open and there are lots of areas that any motivated person can work on. For example, improving the rich-text editor component, improving file uploads, adding image thumbnail generation, etc.
Find us here: - Main site: https://lib.reviews/ - GitHub: https://github.com/eloquence/lib.reviews - IRC: #lib.reviews on irc.freenode.net (also a fine place to ask for an invite code)
This is all still in early development & bug reports are always welcome.
Cheers :)
Erik
A small update on this: based on some off-list feedback, I replaced the way I exclude disambiguation pages and the like from the autocomplete list. The autocomplete widget now performs two queries: a MediaWiki API (wbsearchentities) query, and a follow-up WDQS SPARQL query to exclude disambiguation pages, Wikinews articles, and other content that folks are most likely not interested in reviewing.
I didn't find a good example for this in the examples directory, so I figured folks might find the query I'm using useful. Before I add it to the examples, please let me know if you see obvious ways in which it can be improved.
Here's an example query:
# For a list of items, exclude the ones that have "instance of" set to # one from a given set of excluded classes SELECT DISTINCT ?item WHERE { ?item ?property ?value
# Excluded classes: disambiguation pages, Wikinews articles, etc. MINUS { ?item wdt:P31 wd:Q4167410 } MINUS { ?item wdt:P31 wd:Q17633526 } MINUS { ?item wdt:P31 wd:Q11266439 } MINUS { ?item wdt:P31 wd:Q4167836 } MINUS { ?item wdt:P31 wd:Q14204246 }
# Set of items to check against the above exclusion list # wd:Q355362 is a disambiguation page and will therefore not be in # the result set VALUES ?item { wd:Q23548 wd:Q355362 wd:Q1824521 wd:Q309751 wd:Q6952373 } }
Hello Erik,
Congrats on this more efficient way of filtering.
Instead of using "?item ?property ?value" you could maybe use "?item wikibase:sitelinks ?c" to make sure to select here only items and not properties, lexemes... "?item a wikibase:Item" is not supported by the query service.
It could be also interesting to convert the "MINUS { ?item wdt:P31 wd:QXXX }" to "MINUS { ?item wdt:P31/wdt:P279* wd:QXXX }" in order to filter also all items that are instances of subclasses of the filtered class.
Cheers,
Thomas
Le 26 juil. 2017 à 07:26, Erik Moeller eloquence@gmail.com a écrit :
A small update on this: based on some off-list feedback, I replaced the way I exclude disambiguation pages and the like from the autocomplete list. The autocomplete widget now performs two queries: a MediaWiki API (wbsearchentities) query, and a follow-up WDQS SPARQL query to exclude disambiguation pages, Wikinews articles, and other content that folks are most likely not interested in reviewing.
I didn't find a good example for this in the examples directory, so I figured folks might find the query I'm using useful. Before I add it to the examples, please let me know if you see obvious ways in which it can be improved.
Here's an example query:
# For a list of items, exclude the ones that have "instance of" set to # one from a given set of excluded classes SELECT DISTINCT ?item WHERE { ?item ?property ?value
# Excluded classes: disambiguation pages, Wikinews articles, etc. MINUS { ?item wdt:P31 wd:Q4167410 } MINUS { ?item wdt:P31 wd:Q17633526 } MINUS { ?item wdt:P31 wd:Q11266439 } MINUS { ?item wdt:P31 wd:Q4167836 } MINUS { ?item wdt:P31 wd:Q14204246 }
# Set of items to check against the above exclusion list # wd:Q355362 is a disambiguation page and will therefore not be in # the result set VALUES ?item { wd:Q23548 wd:Q355362 wd:Q1824521 wd:Q309751 wd:Q6952373 } }
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Thanks for sharing, Erik!
Combining search and quiery capabilities would needed be useful for quite a few things. We'll probably be working on making this easier soon.
-- daniel
Am 26.07.2017 um 07:26 schrieb Erik Moeller:
A small update on this: based on some off-list feedback, I replaced the way I exclude disambiguation pages and the like from the autocomplete list. The autocomplete widget now performs two queries: a MediaWiki API (wbsearchentities) query, and a follow-up WDQS SPARQL query to exclude disambiguation pages, Wikinews articles, and other content that folks are most likely not interested in reviewing.
I didn't find a good example for this in the examples directory, so I figured folks might find the query I'm using useful. Before I add it to the examples, please let me know if you see obvious ways in which it can be improved.
Here's an example query:
# For a list of items, exclude the ones that have "instance of" set to # one from a given set of excluded classes SELECT DISTINCT ?item WHERE { ?item ?property ?value
# Excluded classes: disambiguation pages, Wikinews articles, etc. MINUS { ?item wdt:P31 wd:Q4167410 } MINUS { ?item wdt:P31 wd:Q17633526 } MINUS { ?item wdt:P31 wd:Q11266439 } MINUS { ?item wdt:P31 wd:Q4167836 } MINUS { ?item wdt:P31 wd:Q14204246 }
# Set of items to check against the above exclusion list # wd:Q355362 is a disambiguation page and will therefore not be in # the result set VALUES ?item { wd:Q23548 wd:Q355362 wd:Q1824521 wd:Q309751 wd:Q6952373 } }
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hi!
On 7/25/17 10:26 PM, Erik Moeller wrote:
Here's an example query:
# For a list of items, exclude the ones that have "instance of" set to # one from a given set of excluded classes
I wonder if this one won't be more efficient:
SELECT DISTINCT ?item WHERE { FILTER NOT EXISTS { # Excluded classes: disambiguation pages, Wikinews articles, etc. { ?item wdt:P31 wd:Q4167410 } UNION { ?item wdt:P31 wd:Q17633526 } UNION { ?item wdt:P31 wd:Q11266439 } UNION { ?item wdt:P31 wd:Q4167836 } UNION { ?item wdt:P31 wd:Q14204246 } } # Set of items to check against the above exclusion list # wd:Q355362 is a disambiguation page and will therefore not be in # the result set VALUES ?item { wd:Q23548 wd:Q355362 wd:Q1824521 wd:Q309751 wd:Q6952373 } }
Generally, patterns like '?item ?property ?value' aren't very good performers, though in this case the query optimizer seems to handle it ok, so it may not matter this much.
Another note is that you can call wbsearchentities from SPARQL: https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual/MWAPI#Find...
Not sure whether it fits your use case or not, but offering it as one more option.
On Wed, Jul 26, 2017 at 4:05 PM, Stas Malyshev smalyshev@wikimedia.org wrote:
Another note is that you can call wbsearchentities from SPARQL:
(...)
Not sure whether it fits your use case or not, but offering it as one more option.
Hi Stas,
Thanks for the tips & pointer to the MW API integration. It may be possible to replace the two queries powering my autocomplete box with a single WDQS query, which seems like a major potential improvement.
My main problem with the example query below is that, with the filter condition, it messes up the original sort order as returned by the API. Can you think of a way to preserve the sort order?
Thanks again for all the helpful tips, Erik
SELECT * WHERE { FILTER NOT EXISTS { # Excluded classes: disambiguation pages, Wikinews articles, etc. { ?item wdt:P31 wd:Q4167410 } UNION { ?item wdt:P31 wd:Q17633526 } UNION { ?item wdt:P31 wd:Q11266439 } UNION { ?item wdt:P31 wd:Q4167836 } UNION { ?item wdt:P31 wd:Q14204246 } } SERVICE wikibase:mwapi { bd:serviceParam wikibase:api "EntitySearch" . bd:serviceParam wikibase:endpoint "www.wikidata.org" . bd:serviceParam mwapi:search "revival" . bd:serviceParam mwapi:language "en" . bd:serviceParam mwapi:uselang "en" . bd:serviceParam mwapi:limit 50 . ?item wikibase:apiOutputItem mwapi:item . ?label wikibase:apiOutput "@label" . ?matchType wikibase:apiOutput "match/@type" . ?matchLang wikibase:apiOutput "match/@language" . ?matchText wikibase:apiOutput "match/@text" . ?description wikibase:apiOutput "@description" . } }