Hi,
I'm experiencing issues when trying to SSH into the CloudVPS in order to access our project, to which I do have confirmed access.
I've carefully followed the instructions on Help:Accessing_Cloud_VPS_instances https://wikitech.wikimedia.org/wiki/Help:Accessing_Cloud_VPS_instances, and I've verified that the correct SSH keys are associated with my account. Despite this, I consistently receive the following error message when attempting to connect with command:
*ssh producer.wikispeech.eqiad1.wikimedia.cloud*
Error:
*Connection closed by 185.15.56.87 port 22Connection closed by UNKNOWN port 65535*
Could you maybe help me troubleshoot what might be going wrong?
*Viktoria Hillerud* *Developer* Wikimedia Sverige (WMSE)
Hi!
Two things: 1. What's your shell username? 2. Can you run ssh with the `-v` flag and send the output here?
Taavi
Absolutely!
My shellname is: viktoriahillerudwmse
and the output was: (*and I have replaced my actual SHA-keys with <key>*)
OpenSSH_9.6p1 Ubuntu-3ubuntu13.8, OpenSSL 3.0.13 30 Jan 2024 debug1: Reading configuration data /home/viktoria/.ssh/config debug1: /home/viktoria/.ssh/config line 4: Applying options for *.wikimedia.cloud debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files debug1: /etc/ssh/ssh_config line 21: Applying options for * debug1: Setting implicit ProxyCommand from ProxyJump: ssh -p 22 -v -W '[%h]:%p' bastion.wmcloud.org debug1: Executing proxy command: exec ssh -p 22 -v -W '[producer.wikispeech.eqiad1.wikimedia.cloud]:22' bastion.wmcloud.org debug1: identity file /home/viktoria/.ssh/id_rsa type -1 debug1: identity file /home/viktoria/.ssh/id_rsa-cert type -1 debug1: identity file /home/viktoria/.ssh/id_ecdsa type -1 debug1: identity file /home/viktoria/.ssh/id_ecdsa-cert type -1 debug1: identity file /home/viktoria/.ssh/id_ecdsa_sk type -1 debug1: identity file /home/viktoria/.ssh/id_ecdsa_sk-cert type -1 debug1: identity file /home/viktoria/.ssh/id_ed25519 type 3 debug1: identity file /home/viktoria/.ssh/id_ed25519-cert type -1 debug1: identity file /home/viktoria/.ssh/id_ed25519_sk type -1 debug1: identity file /home/viktoria/.ssh/id_ed25519_sk-cert type -1 debug1: identity file /home/viktoria/.ssh/id_xmss type -1 debug1: identity file /home/viktoria/.ssh/id_xmss-cert type -1 debug1: identity file /home/viktoria/.ssh/id_dsa type -1 debug1: identity file /home/viktoria/.ssh/id_dsa-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.8 OpenSSH_9.6p1 Ubuntu-3ubuntu13.8, OpenSSL 3.0.13 30 Jan 2024 debug1: Reading configuration data /home/viktoria/.ssh/config debug1: /home/viktoria/.ssh/config line 1: Applying options for *. wmcloud.org debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files debug1: /etc/ssh/ssh_config line 21: Applying options for * debug1: Connecting to bastion.wmcloud.org [185.15.56.87] port 22. debug1: Connection established. debug1: identity file /home/viktoria/.ssh/id_rsa type -1 debug1: identity file /home/viktoria/.ssh/id_rsa-cert type -1 debug1: identity file /home/viktoria/.ssh/id_ecdsa type -1 debug1: identity file /home/viktoria/.ssh/id_ecdsa-cert type -1 debug1: identity file /home/viktoria/.ssh/id_ecdsa_sk type -1 debug1: identity file /home/viktoria/.ssh/id_ecdsa_sk-cert type -1 debug1: identity file /home/viktoria/.ssh/id_ed25519 type 3 debug1: identity file /home/viktoria/.ssh/id_ed25519-cert type -1 debug1: identity file /home/viktoria/.ssh/id_ed25519_sk type -1 debug1: identity file /home/viktoria/.ssh/id_ed25519_sk-cert type -1 debug1: identity file /home/viktoria/.ssh/id_xmss type -1 debug1: identity file /home/viktoria/.ssh/id_xmss-cert type -1 debug1: identity file /home/viktoria/.ssh/id_dsa type -1 debug1: identity file /home/viktoria/.ssh/id_dsa-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.8 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4p1 Debian-5+deb11u4 debug1: compat_banner: match: OpenSSH_8.4p1 Debian-5+deb11u4 pat OpenSSH* compat 0x04000000 debug1: Authenticating to bastion.wmcloud.org:22 as 'viktoriahillerudwmse' debug1: load_hostkeys: fopen /home/viktoria/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: algorithm: curve25519-sha256@libssh.org debug1: kex: host key algorithm: ssh-ed25519 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: SSH2_MSG_KEX_ECDH_REPLY received debug1: Server host key: ssh-ed25519 SHA256:<key> debug1: load_hostkeys: fopen /home/viktoria/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: Host 'bastion.wmcloud.org' is known and matches the ED25519 host key. debug1: Found key in /home/viktoria/.ssh/known_hosts:2 debug1: ssh_packet_send2_wrapped: resetting send seqnr 3 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: ssh_packet_read_poll2: resetting read seqnr 3 debug1: SSH2_MSG_NEWKEYS received debug1: rekey in after 134217728 blocks debug1: SSH2_MSG_EXT_INFO received debug1: kex_ext_info_client_parse: server-sig-algs=<ssh-ed25519, sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512, ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521, sk-ecdsa-sha2-nistp256@openssh.com, webauthn-sk-ecdsa-sha2-nistp256@openssh.com> debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey debug1: Next authentication method: publickey debug1: get_agent_identities: bound agent to hostkey debug1: get_agent_identities: agent returned 1 keys debug1: Will attempt key: /home/viktoria/.ssh/id_ed25519 ED25519 SHA256:<key> agent debug1: Will attempt key: /home/viktoria/.ssh/id_rsa debug1: Will attempt key: /home/viktoria/.ssh/id_ecdsa debug1: Will attempt key: /home/viktoria/.ssh/id_ecdsa_sk debug1: Will attempt key: /home/viktoria/.ssh/id_ed25519_sk debug1: Will attempt key: /home/viktoria/.ssh/id_xmss debug1: Will attempt key: /home/viktoria/.ssh/id_dsa debug1: Offering public key: /home/viktoria/.ssh/id_ed25519 ED25519 SHA256:<key> agent debug1: Server accepts key: /home/viktoria/.ssh/id_ed25519 ED25519 SHA256:<key> agent Connection closed by 185.15.56.87 port 22 kex_exchange_identification: Connection closed by remote host Connection closed by UNKNOWN port 65535
*Viktoria Hillerud* *Developer* Wikimedia Sverige (WMSE)
Den fre 11 apr. 2025 kl 15:52 skrev Taavi Väänänen taavi@debian.org:
Hi!
Two things:
- What's your shell username?
- Can you run ssh with the `-v` flag and send the output here?
Taavi
Thanks. Seems like due to https://phabricator.wikimedia.org/T379550 your account wasn't automatically granted access to the bastion when you got access to that project. I've manually added you, can you try again?
Taavi
Hi Viktoria,
I vaguely remember to have had a similar issue a few weeks ago. I removed my ssh key somewhere in the horizon interface and after that everything worked fine. Of course my key must still be present somewhere in Wikimedia server infrastructure but I am not sure where. I was hesitating a bit to write this Email at all since I don't have a precise memory of what I did, but maybe it can be helpful somehow.
Yours Dirk
On 4/11/25 15:29, Viktoria Hillerud wrote:
Hi,
I'm experiencing issues when trying to SSH into the CloudVPS in order to access our project, to which I do have confirmed access.
I've carefully followed the instructions on Help:Accessing_Cloud_VPS_instances https://wikitech.wikimedia.org/wiki/Help:Accessing_Cloud_VPS_instances, and I've verified that the correct SSH keys are associated with my account. Despite this, I consistently receive the following error message when attempting to connect with command:
*ssh /producer.wikispeech/.eqiad1.wikimedia.cloud* Error:* Connection closed by 185.15.56.87 port 22 Connection closed by UNKNOWN port 65535*
Could you maybe help me troubleshoot what might be going wrong?
*Viktoria Hillerud* /Developer/ Wikimedia Sverige (WMSE)
Cloud mailing list --cloud@lists.wikimedia.org List information:https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
Hi all,
I can also confirm that I have occasionally seen this error when connecting to cloud VPS.
Not very often but at least a couple times over the years. Specifically the "closed by UNKNOWN".
Sometimes it happens when an instance is brand new and has just been created.
For the cases where it happens later unfortunately I don't have much detail, but I searched Phabricator for "closed by UNKNOWN" because I had vague memories of previous tickets and see this for example:
https://phabricator.wikimedia.org/T353999 This was when the instance was under really high load.
And then there was this one where it happened in deployment-prep but a long time ago and it was about a local group shadowing an LDAP group. https://phabricator.wikimedia.org/T134777
Could be a temp LDAP lookup failure as well.
BUT the IP that is closing the connection here is the bastion (*185.15.56.87 = *primary.bastion.wmcloud.org)..
And ..I CAN login to this instance producer.wikispeec using root@ access, load seems basically zero and there are no obvious failed logins in the last day or so.
But also I am not using the primary bastion, I am using restricted.bastion and I am not sure if I am supposed to login to primary.bastion to check the logs there.
On Fri, Apr 11, 2025 at 6:58 AM Dirk Hünniger via Cloud < cloud@lists.wikimedia.org> wrote:
Hi Viktoria,
I vaguely remember to have had a similar issue a few weeks ago. I removed my ssh key somewhere in the horizon interface and after that everything worked fine. Of course my key must still be present somewhere in Wikimedia server infrastructure but I am not sure where. I was hesitating a bit to write this Email at all since I don't have a precise memory of what I did, but maybe it can be helpful somehow.
Yours Dirk On 4/11/25 15:29, Viktoria Hillerud wrote:
Hi,
I'm experiencing issues when trying to SSH into the CloudVPS in order to access our project, to which I do have confirmed access.
I've carefully followed the instructions on Help:Accessing_Cloud_VPS_instances https://wikitech.wikimedia.org/wiki/Help:Accessing_Cloud_VPS_instances, and I've verified that the correct SSH keys are associated with my account. Despite this, I consistently receive the following error message when attempting to connect with command:
*ssh producer.wikispeech.eqiad1.wikimedia.cloud*
Error:
- Connection closed by 185.15.56.87 port 22 Connection closed by UNKNOWN
port 65535*
Could you maybe help me troubleshoot what might be going wrong?
*Viktoria Hillerud* *Developer* Wikimedia Sverige (WMSE)
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/