[Labs-l] Reboot post-mortem

[Alex Monk]

On 25 March 2016 at 02:00, Alex Monk <krenair at gmail.com> wrote:

> On 24 March 2016 at 16:34, Bryan Davis <bd808 at wikimedia.org> wrote:
>
>> On Thu, Mar 24, 2016 at 9:06 AM, Stephen E Slevinski Jr
>> <slevin at signpuddle.net> wrote:
>> > Yesterday, during the reboot, one of my instances died.  It reported it
>> was
>> > active, but the web-proxy failed with a 502 bad gateway and ssh failed
>> with
>> > "channel 0: open failed: connect failed: No route to host".  Manual
>> reboots
>> > appeared successful, but the problems continued.  On IRC, Yuvipanda was
>> > unable to debug the problem, so I created a new instance this morning.
>> >
>> > I no longer need instance "signwriting-icon-server-2".  Should I delete
>> the
>> > instance or does someone want to do an autopsy?
>> >
>> > For project SignWriting, I was going to update the documentation to
>> reflect
>> > the new instance name, but I'm having more problems.  The "Edit
>> > documentation" link opens a blank page, rather than the project
>> > documentation form.
>> > https://wikitech.wikimedia.org/wiki/Nova_Resource:Signwriting
>>
>> It looks like Special:FormEdit (part of
>> [[:mw:Extension:Semantic_Forms]] is busted on wikitech. A new
>> deployment branch of MediaWiki and Semantic Forms hit wikitech
>> yesterday. We should probably open a bug and see if we can track down
>> the problem. I'm not seeing anything obvious in the logs at the moment
>> but can recreate the blank page.
>
>
>
> https://wikitech.wikimedia.org/wiki/Special:FormEdit/Nova_Project_Documentation/Nova_Resource:Signwriting/Documentation
> krenair at fluorine:~$ tail -f /a/mw-log/fatal.log -n 0
> 2016-03-25 01:55:03 silver labswiki 1.27.0-wmf.18 fatal ERROR: [6b28a0b2]
> PHP Fatal Error: Call to undefined method SFUtils::getSMWPropertyValues()
> {"exception_id":"6b28a0b2"}
> [Exception ErrorException]
> (/srv/mediawiki/php-1.27.0-wmf.18/extensions/SemanticForms/includes/SF_TemplateField.php:66)
> PHP Fatal Error: Call to undefined method SFUtils::getSMWPropertyValues()
>   #0 [internal function]: MWExceptionHandler::handleFatalError()
>   #1 {main}
>
> Problem introduced in https://gerrit.wikimedia.org/r/#/c/278909/
>

https://phabricator.wikimedia.org/T130989 was filed for this. I just
backported and deployed the fix to the wikimedia cluster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20160326/fad8ca77/attachment.html>