So in a nutshell CirrusSearch is a work alike for MWSearch, the search currently in use on all wikis hosted at Wikimedia. From the perspective of reader or editor the advantages of CirrusSearch are: 1. Changes are reflected in search very quickly. Typically pages update in a few seconds but I'd give templates included in many pages a few minutes to propagate. 2. Tons better language support. I have to leave this vague because every time I try to quantify it I find that I've missed something. 3. Transcluded templates are included in search. That includes text and categories.

At some point it won't just be a work alike. All new search work will be done there. I'm frankly frankly itching to see what searches faceted on category would look like. At some point we'll implement some kind of interwiki search. I have no idea what it'll look like though. In a very real, not marketing speak sense, it is the future of search at WMF and for any other Mediawiki installs that want a really really nice search.

Reading material: What it does: https://www.mediawiki.org/wiki/Search/CirrusSearchFeatures Project related: https://www.mediawiki.org/wiki/Search Minimal extension page: https://www.mediawiki.org/wiki/Extension:CirrusSearch

Nik

On Wed, Nov 13, 2013 at 3:30 PM, Luca Martinelli martinelliluca@gmail.comwrote:

Sorry Nik, where I could read more about Cirrus Search?

Cheers,

2013/11/13 Nikolas Everett neverett@wikimedia.org

...

Ambassadors,

I have three CirrusSearch updates:

1. nl.wikipedia.org is now live as a secondary. You can test it by

adding &srbackend=CirrusSearch to the url of the search results page. If you have any trouble you can send email directly to me or file a bug at https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&...

2. I've scheduled more deployments:

itwiki November 14 plwiktionary November 14 wikimedias November 19 wikimanias November 19 wikisources November 19 wiktionaries November 21 The schedule is here: https://www.mediawiki.org/wiki/Search#Wikis

3. Since I haven't heard of any search problems I think it is time to

switch the wikis running CirrusSearch as a secondary to primary. I certainly don't want to break anything but so far it looks like everything is working very well. So each wiki that has CirrusSearch as a secondary will have a date scheduled to switch it to primary. If we/you/the community that you represent discover a bug that makes CirrusSearch worse than the current search we'll make sure to deploy the fix plenty of time before the date or we'll push the date back.

I think each wiki deserves two weeks to test Cirrus so I'll schedule the date accordingly. For nlwiki it'd be 2013/11/28. For itwiki and plwiktionary it'll be 2013/12/3.

I'd like to switch the wikis that have CirrusSearch as a secondary now to primary on 2013/11/21. All of them will have had two weeks to try it out by then.

Again, we'll push the dates if we think we'll make the experience worse.

Thanks,

Nik Everett

---

Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors

-- Luca "Sannita" Martinelli http://it.wikipedia.org/wiki/Utente:Sannita

---

Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors

As some of you know we had a chain of CirrusSearch outages and ultimately rolled all wikis back to lucene search. We tracked down three out of four major issues yesterday and I'm planning on spending the morning hunting down the last one. We've made a list of bugs we're going to have to take care of before redeploying Cirrus ( https://bugzilla.wikimedia.org/buglist.cgi?list_id=251172&resolution=---...). Looking at it, I've got at least a week of work there. Given that WMF isn't doing any non-emergency deployments next week we're looking at early December before I bring CirrusSearch back.

Obviously you can't test it in the intervening time and all my "please find problems by" deadlines are now defunct. I'll send another update when I have more information.

Sorry for all the trouble,

Nik

On Wed, Nov 13, 2013 at 5:21 PM, Federico Leva (Nemo) nemowiki@gmail.comwrote:

Luca Martinelli, 13/11/2013 21:30:

Sorry Nik, where I could read more about Cirrus Search?

...

• <https://it.wikipedia.org/wiki/Wikipedia:Bar/Discussioni/Lanciamoci_nel_

futuro_motore_di_ricerca_interno> of course. ;)

Nemo

---

Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors

On Tue, Nov 19, 2013 at 11:00 AM, Seb35 seb35wikipedia@gmail.com wrote:

Le Tue, 19 Nov 2013 14:27:52 +0100, Nikolas Everett < neverett@wikimedia.org> a écrit:

As some of you know we had a chain of CirrusSearch outages and ultimately

...

rolled all wikis back to lucene search. We tracked down three out of four major issues yesterday and I'm planning on spending the morning hunting down the last one. We've made a list of bugs we're going to have to take care of before redeploying Cirrus ( https://bugzilla.wikimedia.org/buglist.cgi?list_id=251172&resolution=---... resolution=LATER&resolution=DUPLICATE&status_whiteboard_ type=allwordssubstr&query_format=advanced&status_ whiteboard=cirrus_reenable&product=MediaWiki&product= MediaWiki%20extensions&product=Wikimedia). Looking at it, I've got at least a week of work there. Given that WMF isn't doing any non-emergency deployments next week we're looking at early December before I bring CirrusSearch back.

Obviously you can't test it in the intervening time and all my "please find problems by" deadlines are now defunct. I'll send another update when I have more information.

Sorry for all the trouble,

It’s better to get this type of problems now, before CirrusSearch is set as primary.

I think in the end we learned a lot from this and the silver lining is how much more documentation and utility scripts I've added over the past day.

On Tue, Nov 19, 2013 at 12:43 PM, Federico Leva (Nemo) nemowiki@gmail.comwrote:

Seb35, 19/11/2013 17:00:

It’s better to get this type of problems now, before CirrusSearch is set

...

as primary. Thanks for the very progressive deployment (secondary motor and opt-in wikis).

+1 I was surprised to see that CirrusSearch had apparently "swallowed" million-pages wikis without a glitch; it's nice to see issues were nailed down like that (though it's not nice for those who have to fix them).

We're back now! We fixed the problem that caused the outage two weeks ago. We've added code to prevent glitches in the underlying infrastructure form effecting editing. We've verified that if the infrastructure is half crippled then we'll log warnings at WMF but we'll still serve as many search results as possible. Hopefully we won't break things again like we did two weeks ago, but if we do then we're ready.

We've reenabled Cirrus as a secondary for all wikis to which it was deployed before the outage. None have it as primary at this point. Here is the table of deployments from https://www.mediawiki.org/wiki/Search#Wikis : Wiki Secondary Date Primary Date

mediawikiwiki December 3 (first ???) Unscheduled (first September 11) itwiktionary December 3 (first September 16) Unscheduled (first September 23 ) disabled December 3 (first September 23) Unscheduled (first September 24) cawiki December 3 (first October XX) Unscheduled (first September 23) enwikisource December 3 (first October 15) Unscheduled (first September 23) bnwiki December 3 (first October 23) Unscheduled wikivoyages December 3 (first October 23) Unscheduled sewikimedia December 3 (first November 5) Unscheduled astwiki December 3 (first November 5) Unscheduled guwiki December 3 (first November 5) Unscheduled elwiki December 3 (first November 5) Unscheduled frwikisource December 3 (first November 5) Unscheduled nlwiki December 3 (first November 12) Unscheduled itwiki December 3 Unscheduled plwiktionary December 3 Unscheduled wikimedias Unscheduled Unscheduled wikimanias Unscheduled Unscheduled wikisources Unscheduled Unscheduled wiktionaries Unscheduled Unscheduled All wikis are fully ready except itwiki, nlwiki, and enwikisource. All of the content for those wikis is indexed but the incoming links are being counted as I sent this email. Until they are counted the number of links is assumed to be 0 which will cause uncounted articles to potentially be sorted under counted articles. This process should be finished in a few hours.

Otherwise everything is working well now. I'd like to switch mediawiki, itwiktionary, all the disabled wikis, cawiki, and enwikisource back to running Cirrus as primary soon. If you are related to any of those wikis can you recheck Cirrus by adding the url parameter (srbackend=CirrusSearch)? I'd love that.

I'd like to restart that two week timer. That means that I'll request release windows to switch everything from bnwiki to plwiktionary after December 17th. Please give Cirrus a shot between now and then and let me know if you find anything that would make it worse than what the wiki is running now. You can test it by performing a normal search and then adding &srbackend=CirrusSearch to the end of the URL. At some point in the next few days we'll deploy Cirrus as a beta features option as well.

Finally, a peace offering: https://www.mediawiki.org/wiki/Search/CirrusSearchFeatures#prefer-recent:

Nik

Small Cirrus updates:

You can now enable Cirrus as a beta feature on all wikis on which Cirrus is a secondary. On English wikisource the link is https://en.wikisource.org/wiki/Special:Preferences#mw-prefsection-betafeatur.... The name of the feature is "New search" for now. Yup, we know that it won't be "new" for too long but we hope that it'll become the standard search before it stops being "new".

Mediawiki has cirrus as primary now.

You can now negate incategory: and intitle:. Example: https://en.wikisource.org/w/index.php?search=incategory%3A%22Featured+texts%.... Documentation: https://www.mediawiki.org/wiki/Search/CirrusSearchFeatures#Changes_to_filter... This was another small change the wikinews requested.

I'm still looking to start switching wikis that are currently running cirrus as a secondary to primary starting next Wednesday or so. We've also scheduled adding more wikis to try cirrus as secondary: https://www.mediawiki.org/wiki/Search#Wikis Note that the date on that page is the date that I start indexing so add a day or two for those wikis to finish indexing if you are following along at home.

Nik

Hello ambassadors,

I have more CirrusSearch updates for folks who haven't tuned out this conversation already: 1. Cirrus is now the primary search backend for all wikis that used to have it as primary before our November outage, closed, enwiksource, cawiki, and itwiktionary. 2. Cirrus is available as a betafeature on all wikisources, wikimediawikis, wikimaniawikis, and wiktionaries. 3. We're indexing commons, wikispecies, and all the wikinewses now.

Nik

On Mon, Dec 9, 2013 at 3:42 PM, Nikolas Everett neverett@wikimedia.orgwrote:

Small Cirrus updates:

You can now enable Cirrus as a beta feature on all wikis on which Cirrus is a secondary. On English wikisource the link is https://en.wikisource.org/wiki/Special:Preferences#mw-prefsection-betafeatur.... The name of the feature is "New search" for now. Yup, we know that it won't be "new" for too long but we hope that it'll become the standard search before it stops being "new".

Mediawiki has cirrus as primary now.

You can now negate incategory: and intitle:. Example: https://en.wikisource.org/w/index.php?search=incategory%3A%22Featured+texts%.... Documentation: https://www.mediawiki.org/wiki/Search/CirrusSearchFeatures#Changes_to_filter... This was another small change the wikinews requested.

I'm still looking to start switching wikis that are currently running cirrus as a secondary to primary starting next Wednesday or so. We've also scheduled adding more wikis to try cirrus as secondary: https://www.mediawiki.org/wiki/Search#Wikis Note that the date on that page is the date that I start indexing so add a day or two for those wikis to finish indexing if you are following along at home.

Nik

On Wed, Dec 18, 2013 at 3:00 PM, Andrea Zanni zanni.andrea84@gmail.comwrote:

Hi Nik, thank you.

A question: the new search make the whole system of enabling specific namespaces for the search obsolete? this was a problem for Wikisource (I just noticed that an important ns was not enabled), but with Cirrus now it seems perfectly ok.

Cirrus is a bit more cautious about checking which namespaces the wiki considers content namespaces then the old search. That might be what you are seeing.

Nik

On Fri, Dec 20, 2013 at 8:37 AM, Guillaume Paumier gpaumier@wikimedia.orgwrote:

Hi Nik,

On Wed, Dec 18, 2013 at 8:40 PM, Nikolas Everett neverett@wikimedia.org wrote:

...

2. Cirrus is available as a betafeature on all wikisources,

wikimediawikis,

...

wikimaniawikis, and wiktionaries.

By "wikimediawikis", do you mean chapter wikis ( https://noc.wikimedia.org/conf/highlight.php?file=wikimedia.dblist ) or also Meta-Wiki / Incubator / Strategy etc?

Sorry for the confusion. Everything seems to have more names then I expected. I believe just chapter wikis: arwikimedia bdwikimedia bewikimedia brwikimedia cowikimedia dkwikimedia etwikimedia fiwikimedia ilwikimedia mkwikimedia mxwikimedia nlwikimedia noboard_chapterswikimedia nowikimedia nycwikimedia nzwikimedia pa_uswikimedia plwikimedia rswikimedia ruwikimedia sewikimedia trwikimedia uawikimedia ukwikimedia vewikimedia

Nik