[Labs-l] Instance stuck in SHUTOFF state, unable to boot

[Andrew Bogott]

On 2/19/16 9:19 AM, Fæ wrote:
> Is this related to getting the message "Unable to initialize 
> environment because of error: sge_gethostbyname failed" for queued 
> jobs and that qstat tells me 'error: sge_gethostbyname failed' ... or 
> is this something I ought to raise a phab task for?

the 'shutoff' issue is totally unrelated to toollabs -- different 
project entirely, no relationship to sge.

There was a brief DNS outage which you may have bumped into. 
https://phabricator.wikimedia.org/T124680


>
> Fae
>
> On 19 February 2016 at 14:49, Andrew Bogott <abogott at wikimedia.org 
> <mailto:abogott at wikimedia.org>> wrote:
>
>     Andrew --
>
>     On 2/19/16 2:13 AM, Andrew "FastLizard4" Adams wrote:
>
>         -----BEGIN PGP SIGNED MESSAGE-----
>         Hash: SHA256
>
>         Hello Labs folks,
>
>         One of our Labs instances for the account-creation-assistance
>         project,
>         accounts-db2 (UUID 942cfe3f-8a7b-4e6f-ae5c-e5fcd5f16ed8) seems
>         to be
>         stuck in the SHUTOFF state, and attempting to reboot it yields
>         a "Failed
>         to reboot instance accounts-db2" message.  There's nothing in the
>         console output that indicates why the instance is down in the
>         first place.
>
>     When I try to reboot on the commandline,  I'm told that this
>     instance can't be rebooted
>     because it is in the 'stopped' state.  I did a 'start' on it
>     rather than a 'reboot'
>     and it has accepted that and is now up and running.
>
>     The bad news is that the 'start' command is not currently
>     available to non-Ops.
>     The good news is that my primary project right now is making a
>     greater range
>     of OpenStack commands publicly available via Horizon.  So, in
>     short, you couldn't
>     have fixed this yourself today, but should be able to deal with
>     similar issues yourself
>     in a month or two.
>
>     Let me know if you have any other problems in the meantime!
>
>     -Andrew
>
>
>
>
>         Interestingly, this seems to be unrelated to the glibc-related
>         reboots
>         earlier - users report that the account-creation-assistance
>         instances
>         were unavailable for a while (presumably due to the reboots),
>         then they
>         came back up again, then this particular instance went down
>         again and is
>         still down.
>
>         Any assistance would be greatly appreciated.  Thanks in advance!
>         - -- Sincerely,
>         Andrew "FastLizard4" Adams
>         <https://en.wikipedia.org/wiki/User:FastLizard4>
>         <https://fastlizard4.org>
>         <FastLizard4 at gmail.com <mailto:FastLizard4 at gmail.com>>
>         GPG Key ID: 0x221A627DD76E2616
>         -----BEGIN PGP SIGNATURE-----
>         Version: GnuPG v2.0.17 (MingW32)
>         Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
>         iQIcBAEBCAAGBQJWxs6TAAoJECIaYn3XbiYWA1cQALyzCJjF4CMWVnaMqTkGgXw5
>         PM9CFR0GxzvHEjSmibHM4L3NEoKnoKqZ33YKqMiizxDpSYaZCPCWbp8TWxpIHaGw
>         X3DY1fpsMkHOoi6zTrpA2O20V5ko5udiNlQ1y8KzvqZAOnDuKgndkH2o6WGjff3M
>         kl/JY6VgQWI3PNFohf3qd74MyyrfJIj/m9NWsHQFKsbA36IGcOpiTmtuODHths4j
>         mSsxGhGnzy4hyi4L1W4OW3ttFNCemiW1mjmmh6g9ZCW06OxRz857RtVcNAxafjf5
>         ObPjbmIFGxTni/14bymG38gl2uFpQATdL9HVS31kikDIEh0cQABXpfy5pTBrGSJy
>         6tKPlco53jbMPgiVfhdAuVkMEfeHRGjyR4ud44Qx6Sl+5kUUg9Y9ei9LwN2joEVs
>         x2LrXTxIRV1el/M25FS+EpmjnSFbZpEcXFBwE8SKbU+TV+OgLrob2D1MFsZuBF8I
>         sDsm6l3E1ZpNIE+Scy/CwTE3RP6M9ZvsHFD3aW8ohVZMZiZpGmbDOAExtiWo+6XP
>         fq6LF2Lxj3GugDpwl81kDOgHwMyGHd+OXbdFTzU32QC8qfBxdcdKyN+ffD+E+GlA
>         23xinGVO+4P9F6xoqqAG8+EogZmEmcg7jbc7ihHGDFyNyrOtaLRmgWFRriIdFpor
>         qb/k1U3BFjcDS/lbvVYk
>         =daub
>         -----END PGP SIGNATURE-----
>
>         _______________________________________________
>         Labs-l mailing list
>         Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org>
>         https://lists.wikimedia.org/mailman/listinfo/labs-l
>
>
>
>     _______________________________________________
>     Labs-l mailing list
>     Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org>
>     https://lists.wikimedia.org/mailman/listinfo/labs-l
>
>
>
>
> -- 
> faewik at gmail.com <mailto:faewik at gmail.com> 
> https://commons.wikimedia.org/wiki/User:Fae

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20160219/c7deec5b/attachment.html>