We're back now!  We fixed the problem that caused the outage two weeks ago.  We've added code to prevent glitches in the underlying infrastructure form effecting editing.  We've verified that if the infrastructure is half crippled then we'll log warnings at WMF but we'll still serve as many search results as possible.  Hopefully we won't break things again like we did two weeks ago, but if we do then we're ready.

We've reenabled Cirrus as a secondary for all wikis to which it was deployed before the outage.  None have it as primary at this point.  Here is the table of deployments from https://www.mediawiki.org/wiki/Search#Wikis:

Wiki	Secondary Date	Primary Date
mediawikiwiki	December 3 (first ???)	Unscheduled (first September 11)
itwiktionary	December 3 (first September 16)	Unscheduled (first September 23)
disabled	December 3 (first September 23)	Unscheduled (first September 24)
cawiki	December 3 (first October XX)	Unscheduled (first September 23)
enwikisource	December 3 (first October 15)	Unscheduled (first September 23)
bnwiki	December 3 (first October 23)	Unscheduled
wikivoyages	December 3 (first October 23)	Unscheduled
sewikimedia	December 3 (first November 5)	Unscheduled
astwiki	December 3 (first November 5)	Unscheduled
guwiki	December 3 (first November 5)	Unscheduled
elwiki	December 3 (first November 5)	Unscheduled
frwikisource	December 3 (first November 5)	Unscheduled
nlwiki	December 3 (first November 12)	Unscheduled
itwiki	December 3	Unscheduled
plwiktionary	December 3	Unscheduled
wikimedias	Unscheduled	Unscheduled
wikimanias	Unscheduled	Unscheduled
wikisources	Unscheduled	Unscheduled
wiktionaries	Unscheduled	Unscheduled

All wikis are fully ready except itwiki, nlwiki, and enwikisource.  All of the content for those wikis is indexed but the incoming links are being counted as I sent this email.  Until they are counted the number of links is assumed to be 0 which will cause uncounted articles to potentially be sorted under counted articles.  This process should be finished in a few hours.

Otherwise everything is working well now.  I'd like to switch mediawiki, itwiktionary, all the disabled wikis, cawiki, and enwikisource back to running Cirrus as primary soon.  If you are related to any of those wikis can you recheck Cirrus by adding the url parameter (srbackend=CirrusSearch)?  I'd love that.

I'd like to restart that two week timer.  That means that I'll request release windows to switch everything from bnwiki to plwiktionary after December 17th.  Please give Cirrus a shot between now and then and let me know if you find anything that would make it worse than what the wiki is running now.  You can test it by performing a normal search and then adding &srbackend=CirrusSearch to the end of the URL.  At some point in the next few days we'll deploy Cirrus as a beta features option as well.

Finally, a peace offering:  https://www.mediawiki.org/wiki/Search/CirrusSearchFeatures#prefer-recent:

Nik