On 16-08-07 13:50:33, Legoktm wrote:
Does scap/whatever make any requests against those hosts? Or is it just depending upon normal traffic to those hosts to possibly cause errors?
The script that scap is using to query logstash is logstash_checker.py[0]. There are no requests being generated as part of the deployment process, scap relies wholly on normal traffic to spot errors.
There was some discussion on a couple phabricator tickets[1][2] about a pre-canary check step that would still be nice to implement.
While the canary check script was a good step, I still feel that a pre-canary deploy sanity check that consists of requests to known end-points on unpooled servers would be a boon to the prevention of catastrophic deploys.
-- Tyler
[0]. https://github.com/wikimedia/operations-puppet/blob/production/modules/service/files/logstash_checker.py [1]. https://phabricator.wikimedia.org/T136839 [2]. https://phabricator.wikimedia.org/T121597