Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate queries, including the ordering (will be supported by the new search backend)
On Jan 24, 2014 1:54 AM, "Yuri Astrakhan" yastrakhan@wikimedia.org wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate
queries,
including the ordering (will be supported by the new search backend) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I like the idea in principle, but think the syntax could use bikeshedding ;)
If we use this as category pages, im a little worried that people could get confused and try to add [[category:Greek philosophers]] to a page, and expect it to work. We would need good error handling in that situation
including the ordering (will be supported by the new search backend)
Cool. I didnt realize search would support this. That's a pretty big deal since people expect there categorirs alphabetized.
Another cool project would be to expand intersection/Dyanamic Page List (Wikimedia) to be able to use search as a different backend (however, that extension would need quite a bit of refactoring to get there)
-bawolff
Sent from my iPhone
On Jan 24, 2014, at 7:55 AM, Brian Wolff bawolff@gmail.com wrote:
On Jan 24, 2014 1:54 AM, "Yuri Astrakhan" yastrakhan@wikimedia.org wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate
queries,
including the ordering (will be supported by the new search backend) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I like the idea in principle, but think the syntax could use bikeshedding ;)
If we use this as category pages, im a little worried that people could get confused and try to add [[category:Greek philosophers]] to a page, and expect it to work. We would need good error handling in that situation
Also, at least at first, you couldn't use one of these synthetic categories to filter search results so you couldn't use them to build other synthetic categories.
including the ordering (will be supported by the new search backend)
Cool. I didnt realize search would support this. That's a pretty big deal since people expect there categorirs alphabetized.
So I looked into it and it would be easy to implement with Cirrus _but_ I'm not yet sure about the memory implications. We have a ton of headroom on memory now so it might not matter but I need to test it before I can be as confident that it is ok as I was last night at 1am. It may have caveats like synthetic categories must be less than 1000 articles or not all results are included.
Another cool project would be to expand intersection/Dyanamic P age List (Wikimedia) to be able to use search as a different backend (however, that extension would need quite a bit of refactoring to get there)
-bawolff _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Jan 24, 2014 at 9:20 AM, Nik Everett neverett@wikimedia.org wrote:
On Jan 24, 2014, at 7:55 AM, Brian Wolff bawolff@gmail.com wrote: If we use this as category pages, im a little worried that people could
get
confused and try to add [[category:Greek philosophers]] to a page, and expect it to work. We would need good error handling in that situation
Also, at least at first, you couldn't use one of these synthetic categories to filter search results so you couldn't use them to build other synthetic categories.
Don't forget the API. You'd probably need to bascally override ApiQueryCategoryMembers to check cmtitle and either use parent::run() or a custom implementation.
You could do something exactly like this very, very easily using Semantic MediaWiki and Concepts. Jamie Thingelstad jamie@thingelstad.com mobile: 612-810-3699 find me on AIM Twitter Facebook LinkedIn
On Jan 24, 2014, at 9:55 AM, Brian Wolff bawolff@gmail.com wrote:
On Jan 24, 2014 1:54 AM, "Yuri Astrakhan" yastrakhan@wikimedia.org wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate
queries,
including the ordering (will be supported by the new search backend) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I like the idea in principle, but think the syntax could use bikeshedding ;)
If we use this as category pages, im a little worried that people could get confused and try to add [[category:Greek philosophers]] to a page, and expect it to work. We would need good error handling in that situation
including the ordering (will be supported by the new search backend)
Cool. I didnt realize search would support this. That's a pretty big deal since people expect there categorirs alphabetized.
Another cool project would be to expand intersection/Dyanamic Page List (Wikimedia) to be able to use search as a different backend (however, that extension would need quite a bit of refactoring to get there)
-bawolff _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Of course - I am actually modeling this feature on Semantic MW. It's just that I doubt WMF will enable SMW on the cluster any time soon :)
On Sat, Jan 25, 2014 at 4:13 PM, Jamie Thingelstad jamie@thingelstad.comwrote:
You could do something exactly like this very, very easily using Semantic MediaWiki and Concepts. Jamie Thingelstad jamie@thingelstad.com mobile: 612-810-3699 find me on AIM Twitter Facebook LinkedIn
On Jan 24, 2014, at 9:55 AM, Brian Wolff bawolff@gmail.com wrote:
On Jan 24, 2014 1:54 AM, "Yuri Astrakhan" yastrakhan@wikimedia.org
wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate
queries,
including the ordering (will be supported by the new search backend) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I like the idea in principle, but think the syntax could use
bikeshedding ;)
If we use this as category pages, im a little worried that people could
get
confused and try to add [[category:Greek philosophers]] to a page, and expect it to work. We would need good error handling in that situation
including the ordering (will be supported by the new search backend)
Cool. I didnt realize search would support this. That's a pretty big deal since people expect there categorirs alphabetized.
Another cool project would be to expand intersection/Dyanamic Page List (Wikimedia) to be able to use search as a different backend (however,
that
extension would need quite a bit of refactoring to get there)
-bawolff _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hoi, Did you consider the use of Wikidata in stead of SMW? It should not only indicate all the articles that have the appropriate statements but also the articles written in other languages. Thanks, GerardM
On 26 January 2014 03:25, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Of course - I am actually modeling this feature on Semantic MW. It's just that I doubt WMF will enable SMW on the cluster any time soon :)
On Sat, Jan 25, 2014 at 4:13 PM, Jamie Thingelstad <jamie@thingelstad.com
wrote:
You could do something exactly like this very, very easily using Semantic MediaWiki and Concepts. Jamie Thingelstad jamie@thingelstad.com mobile: 612-810-3699 find me on AIM Twitter Facebook LinkedIn
On Jan 24, 2014, at 9:55 AM, Brian Wolff bawolff@gmail.com wrote:
On Jan 24, 2014 1:54 AM, "Yuri Astrakhan" yastrakhan@wikimedia.org
wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and
show
result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate
queries,
including the ordering (will be supported by the new search backend) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I like the idea in principle, but think the syntax could use
bikeshedding ;)
If we use this as category pages, im a little worried that people could
get
confused and try to add [[category:Greek philosophers]] to a page, and expect it to work. We would need good error handling in that situation
including the ordering (will be supported by the new search backend)
Cool. I didnt realize search would support this. That's a pretty big
deal
since people expect there categorirs alphabetized.
Another cool project would be to expand intersection/Dyanamic Page List (Wikimedia) to be able to use search as a different backend (however,
that
extension would need quite a bit of refactoring to get there)
-bawolff _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Sun, Jan 26, 2014 at 12:25 AM, Yuri Astrakhan yastrakhan@wikimedia.orgwrote:
Of course - I am actually modeling this feature on Semantic MW. It's just that I doubt WMF will enable SMW on the cluster any time soon :)
That request was already WONTFIXed: https://bugzilla.wikimedia.org/show_bug.cgi?id=8390
Helder
On Fri, Jan 24, 2014 at 10:53 AM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate queries, including the ordering (will be supported by the new search backend)
In the future this will be possible using Wikidata and queries. My dream is that categories will to a very large extend go away then and be replaced with better Wikidata-based tools. I'd _really_ like us not to introduce more incentives to use categories like this one. Also it'd be quite a source for confusion to have both.
Cheers Lydia
On Sun, Jan 26, 2014 at 1:10 AM, Lydia Pintscher < lydia.pintscher@wikimedia.de> wrote:
On Fri, Jan 24, 2014 at 10:53 AM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate
queries,
including the ordering (will be supported by the new search backend)
In the future this will be possible using Wikidata and queries. My dream is that categories will to a very large extend go away then and be replaced with better Wikidata-based tools. I'd _really_ like us not to introduce more incentives to use categories like this one. Also it'd be quite a source for confusion to have both.
Indeed. I think it would be very shortsighted to introduce any sort of new syntax here for this right now as proposed.
There's lots of exciting query opportunities coming with Wikidata as Lydia says (which will very likely end up using Elasticsearch in some manner).
On Sun, Jan 26, 2014 at 10:21 AM, Brian Wolff bawolff@gmail.com wrote:
Category intersection proposals seem to appear on this list about once every 2 years as far as i remember. Usually they dont succede because nobody ends up writing any efficient code to implement it.
Yep, we've been here before. Because category intersection isn't easy with the schema we've got. However, it's incredibly trivial to do in Elasticsearch. So I imagine if we invest more effort in helping Wikidata expand its syntax and uses we'll likely end up with what Yuri's wanting--if maybe in a different form.
I'll also note we've already solved category intersection in Cirrus ;-)
https://en.wikipedia.org/w/index.php?search=incategory%3A%221941+births%22+i...
-Chad
Hoi, In Reasonator we now support calendars [1] It does show you things like people who were born / died on that date, events under way on that date ..
Play with and notice that longer periods are also supported. Some longer periods may take some time but we have a waiting bar for you..It does support any of the languages that we support. It uses Wikidata information so if your favourite person or event is not mentioned. Just add the details in Wikidata. Thanks, GerardM
PS three cheers for Magnus
[1]
On 30 January 2014 19:50, Chad innocentkiller@gmail.com wrote:
On Sun, Jan 26, 2014 at 1:10 AM, Lydia Pintscher < lydia.pintscher@wikimedia.de> wrote:
On Fri, Jan 24, 2014 at 10:53 AM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate
queries,
including the ordering (will be supported by the new search backend)
In the future this will be possible using Wikidata and queries. My dream is that categories will to a very large extend go away then and be replaced with better Wikidata-based tools. I'd _really_ like us not to introduce more incentives to use categories like this one. Also it'd be quite a source for confusion to have both.
Indeed. I think it would be very shortsighted to introduce any sort of new syntax here for this right now as proposed.
There's lots of exciting query opportunities coming with Wikidata as Lydia says (which will very likely end up using Elasticsearch in some manner).
On Sun, Jan 26, 2014 at 10:21 AM, Brian Wolff bawolff@gmail.com wrote:
Category intersection proposals seem to appear on this list about once every 2 years as far as i remember. Usually they dont succede because nobody ends up writing any efficient code to implement it.
Yep, we've been here before. Because category intersection isn't easy with the schema we've got. However, it's incredibly trivial to do in Elasticsearch. So I imagine if we invest more effort in helping Wikidata expand its syntax and uses we'll likely end up with what Yuri's wanting--if maybe in a different form.
I'll also note we've already solved category intersection in Cirrus ;-)
https://en.wikipedia.org/w/index.php?search=incategory%3A%221941+births%22+i...
-Chad _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Thu, Jan 30, 2014 at 10:50 AM, Chad innocentkiller@gmail.com wrote:
I'll also note we've already solved category intersection in Cirrus ;-)
https://en.wikipedia.org/w/index.php?search=incategory%3A%221941+births%22+i...
That should read "in both the old and new search" actually.
It just goes to show you how easy this is in lucene world.
-Chad
I have a vague feeling that a very similar proposal appeared on this very list a few months ago. (No links handy, though.)
On Jan 26, 2014 10:02 AM, "Bartosz Dziewoński" matma.rex@gmail.com wrote:
I have a vague feeling that a very similar proposal appeared on this very
list a few months ago. (No links handy, though.)
-- Matma Rex
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Category intersection proposals seem to appear on this list about once every 2 years as far as i remember. Usually they dont succede because nobody ends up writing any efficient code to implement it.
-bawolff
http://migadv.com/ comes to mind (https://meta.wikimedia.org/wiki/Grants:IEG/MediaWiki_data_browser) but sadly it's barely interactive...
gry
Hey Yuri,
This is a really cool idea. It's almost like set theory for categories! You can define certain base categories (based upon manually entering them into the category using existing category syntax), then define other categories based on set theoretic operations on categories. Certainly, if we could go back in time and build categories from the ground up again, I think this approach would be far superior.
This feature is sufficiently complex from a product standpoint that I think the best thing to do here is to wait to see what happens with CirrusSearch's integration with Wikidata. Many cool features like this may end up being obsoleted by any effort for Wikidata integration. If not, we can revisit this then.
Thanks, Dan
On 24 January 2014 01:53, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate queries, including the ordering (will be supported by the new search backend) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hoi,
With the "autolist" [1] and Widar you can make claims for the entries in a category. So you can first claim "occupation" "philosopher" and then make other claims about such philosophers ... Mind you claiming the Greek nationality for people who spoke Greek is problematic when these people were living before the creation of Greece. Possible claims are that they spoke Greek ... Thanks, GerardM
[1] http://tools.wmflabs.org/wikidata-todo/autolist.html#
On 24 January 2014 10:53, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Hi, I am thinking of implementing a
#CATQUERY <query>
magic keyword for the category pages.
When this keyword is present, the category page would execute a query against the search backend instead of normal category behavior and show result as if those pages were actually marked with this category.
For example, this would allow Greek Philosophers category page to be quickly redefined as a cross-section of greeks & philosophers categories:
#CATQUERY incategory:Greek incategory:Philosopher
Obviously the community will be able to define much more elaborate queries, including the ordering (will be supported by the new search backend) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org