It looks like the problem started 3/21, see attached image.  I&#39;m about to head to the airport for ~20 hours of travel so can&#39;t investigate further, but may be related to parser changes made that day which line up with the second bigger spike.<br>
<ul><li> 05:23 logmsgbot: tstarling synchronized php-1.19/includes/parser/CoreParserFunctions.php 
</li><li> 05:23 logmsgbot: tstarling synchronized php-1.19/includes/parser/Parser.php 
</li><li> 05:23 logmsgbot: tstarling synchronized php-1.19/includes/parser/StripState.php 
</li><li> 05:22 logmsgbot: tstarling synchronized php-1.19/tests/parser/parserTests.txt 
</li></ul>There was also a the release of a majorly changed MobileFrontend earlier that day / prior night that line up with the first spike.  The MF rewrite doesn&#39;t perform well - MobileFrontend::DOMParse avg time went from 15ms to 150ms (~500ms at 99th) and it wouldn&#39;t be impossible for it to also impact non-mobile performance.<br>
<br>DB, ES, and memcache latency look steady, so I suspect it&#39;s all application side.<br><br>-Asher<br><br><div class="gmail_quote">On Fri, Mar 23, 2012 at 10:00 PM, MZMcBride <span dir="ltr">&lt;<a href="mailto:z@mzmcbride.com">z@mzmcbride.com</a>&gt;</span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi.<br>
<br>
There are over a dozen reports of general slowness with the English<br>
Wikipedia:<br>
<a href="https://en.wikipedia.org/w/index.php?oldid=483631204#is_it_me_or_is_wiki_ver" target="_blank">https://en.wikipedia.org/w/index.php?oldid=483631204#is_it_me_or_is_wiki_ver</a><br>
y_slow.3F<br>
<br>
As I browsed and edited the site today, I experienced similar issues<br>
(painful slowness on intermittent page loads, loading half a page and then<br>
stalling, etc.).<br>
<br>
Is anyone from ops or engineering looking into this? I&#39;ve filed a bug about<br>
this as well: &lt;<a href="https://bugzilla.wikimedia.org/show_bug.cgi?id=35448" target="_blank">https://bugzilla.wikimedia.org/show_bug.cgi?id=35448</a>&gt;.<br>
<br>
It may be that this is a known issue (due to schema changes or whatever<br>
else), but I don&#39;t think anyone from ops or engineering has commented on the<br>
village pump (or elsewhere?). Can someone please take a look at what&#39;s going<br>
on?<br>
<br>
MZMcBride<br>
<br>
<br>
<br>
_______________________________________________<br>
Wikitech-l mailing list<br>
<a href="mailto:Wikitech-l@lists.wikimedia.org">Wikitech-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/wikitech-l" target="_blank">https://lists.wikimedia.org/mailman/listinfo/wikitech-l</a><br>
</blockquote></div><br>