[Labs-l] Debugging web service failure

[Morten Wang]

Hi Merlijn,

Thanks for looking into that. Yes, I've noticed that restarting the web
service seems to fix the problem, but only intermittently. If I now, about
25 mins after you emailed, go to http://tools.wmflabs.org/suggestbot/hw.html
I get a 503 service unavailable. This has appeared to be a consistent
pattern lately, I restart and shortly thereafter it is unavailable again.
Which is also why it's difficult to debug what's going on.


Cheers,
Morten


On 18 April 2016 at 14:00, Merlijn van Deen (valhallasw) <
valhallasw at arctus.nl> wrote:

> Hi Morten,
>
> The 503 suggests the web proxy hit a hitch -- the web server is running
> on tools-webgrid-lighttpd-1415:37083 (qstat -xml, ssh to that host, ps aux
> | grep suggestbot; vim /var/run/lighttpd/suggestbot.conf), but the proxy is
> unaware of that.
>
> I have restarted your webservice job (qmod -rj 5458226), and this seems to
> have resolved the issue (https://tools.wmflabs.org/suggestbot/ now 404s,
> but that's your lighttpd webservice and not the proxy).
>
> Best,
> Merlijn
>
> On 18 April 2016 at 19:00, Morten Wang <nettrom at gmail.com> wrote:
>
>> I'm currently having an issue with SuggestBot's (tools.suggestbot) web
>> services failing after a while, resulting in a 503 (service unavailable).
>> The web service process on the grid appears to be running, but it also
>> appears to be unreachable.
>>
>> There is nothing in the error.log nor the access.log that suggests
>> anything happened to lighttpd, and I am unsure about what debug settings to
>> add to make it possible to log any errors. The fastcgi.debug setting
>> doesn't appear to reveal anything. In other words, I've reached the end of
>> my abilities to figure this out, and thus I wonder:
>>
>> What are some best practices when it comes to debugging what is going on?
>>
>>
>> Cheers,
>> Morten
>>
>>
>> _______________________________________________
>> Labs-l mailing list
>> Labs-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>
>>
>
> _______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20160418/17c805b7/attachment.html>