Hello all!
The Elasticsearch upgrade is now completed. During the migration process,
we've seen a few issues:
* Search now caps total results count at 10k because of elasticsearch 7
upgrade [1], which is now fixed.
* Some performance degradation was reported on en-wiki village pump [2].
This was observed during the migration when our Search was running on a
single datacenter, with added cross-datacenter latency. Our metrics
currently show a very minor performance degradation compared to before the
upgrade, with the median request time going from between 11 and 12
milliseconds to between 12 and 14 milliseconds [3]. Individual queries
might behave differently.
Please let us know if you spot any outstanding issues, preferably via a
Phabricator task [4] tagged with [Discovery-Search].
Have fun!
Guillaume
[1]
https://phabricator.wikimedia.org/T317374
[2]
https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Search_tot…
[3]
https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1…
[4]
https://phabricator.wikimedia.org/
On Tue, 23 Aug 2022 at 15:46, Guillaume Lederrey <glederrey(a)wikimedia.org>
wrote:
Hello all!
For the last 2 quarters, the Search Platform team has been working on
upgrading our Elasticsearch clusters to version 7.10.2 [1]. Keeping our
software up to date is part of the usual project hygiene, allowing us to
benefit from bugs and security fixes, performance improvements, and new
features. In our case, upgrading to Elasticsearch 7.10.2 is also a required
step towards a potential move to OpenSearch [2].
After much testing, fixes and validations, we are now ready to start the
final migration process. We are anticipating a 3-week migration process,
starting on August 29 2022. You can follow along on Phabricator [3].
What does this mean for you?
For users of Special:Search, Special:MediaSearch and other user-facing
Search interfaces, the upgrade should be fully seamless, and should not
cause any disruptions to normal usage.
For users of Cloudelastic [4] who are accessing the Elasticsearch API
directly, there might be minor API changes that could affect your queries.
Please review the documented breaking changes [5]. Most of the breaking
changes are not related to queries, so it is unlikely that any client code
will break with this upgrade.
If you have any questions about this process, you can find us in
#wikimedia-search on IRC, or at discovery(a)lists.wikimedia.org. Have fun!
The Search Platform team
[1]
https://phabricator.wikimedia.org/T263142
[2]
https://phabricator.wikimedia.org/T280482
[3]
https://phabricator.wikimedia.org/T308676
[4]
https://wikitech.wikimedia.org/wiki/Help:CirrusSearch_elasticsearch_replicas
[5]
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/breaking-chang…
--
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation <https://wikimediafoundation.org/>
--
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation <https://wikimediafoundation.org/>