That is not a good thing! The algorithm does allow for that to happen temporarily without issue, but I appreciate the report. We will keep an eye out for further issues.

Brooke Storm
Senior SRE
Wikimedia Cloud Services
bstorm@wikimedia.org
IRC: bstorm_

On Jan 16, 2020, at 8:57 AM, Roy Smith <roy@panix.com> wrote:

I just got the following error.  I reran the command and it worked the second time.

My limited experience with etcd says this is not a good thing, so figured you'd want to know about it.




tools.spi-tools@tools-sgebastion-08:~/www/python/src$ webservice --backend=kubernetes python3.7 restart                       
Traceback (most recent call last):
  File "/usr/local/bin/webservice", line 318, in <module>
    if stop(job, ""):
  File "/usr/local/bin/webservice", line 142, in stop
    job.request_stop()
  File "/usr/lib/python2.7/dist-packages/toollabs/webservice/backends/kubernetesbackend.py", line 675, in request_stop
    self._delete_obj(pykube.Service, self.webservice_label_selector)
  File "/usr/lib/python2.7/dist-packages/toollabs/webservice/backends/kubernetesbackend.py", line 447, in _delete_obj
    o.delete()
  File "/usr/lib/python2.7/dist-packages/pykube/objects.py", line 96, in delete
    self.api.raise_for_status(r)
  File "/usr/lib/python2.7/dist-packages/pykube/http.py", line 104, in raise_for_status
    raise HTTPError(payload["message"])
pykube.exceptions.HTTPError: client: etcd member https://tools-k8s-etcd-01.tools.eqiad.wmflabs:2379 has no leader

_______________________________________________
Wikimedia Cloud Services mailing list
Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud