That actually sounds very close to an issue we had after upgrading to 1.22
earlier this year. Pages with a lot of images/thumbnails took a long time
to render (100s of images took over a minute). We eventually tracked it
down to having the default $wgTmpDirectory pointing to the upload/images
directory which was on a NFS share. Each file creation (or access?) on a
NFS share takes a fixed 50ms so you multiply that by multiple accesses and
you get the delay.
We fixed it by simply changing $wgTmpDirectory to point to a path on the
local fixed drive. Since your setup sounds similar to ours it may be worth
trying it out. If this is indeed your issue you can force a "slow" page
load by purging a page with a lot of images on it. Test it before and after
the change.
On 18 November 2015 at 15:42, Justin Lloyd <jlloyd.wiki(a)gmail.com> wrote:
My speculation is that it's image heavy pages, not
one specific php page.
This is for the Guild Wars 2 wikis, specifically the English wiki at
wiki.guildwars2.com. The Game Updates page used to be problematic, causing
a massive backlog because a game update or hotfix was released and people
hammered that page to see the list of changes. Our main editors changed how
the page works, primarily breaking it up into subpages that DPL integrates
the most recent of which into the main page, but also changing the
templates that were used for displaying trait and skill icons.
Further analysis of the Apache logs, after adding the %D field to the log
format, showed a lot of pages taking sometimes minutes to complete, which
ultimately result in 502s. The ones that appear to take the longest are
those with a lot of these thumbnail images, which is why I think it's still
a template issue, but it would be really nice to be able to back up that
hypothesis with actual data from process diagnostics, stack traces, etc.
(I really miss DTrace on Solaris. I know it exists for Linux but I'm wary
of trying it, especially on production systems. Anyone here have experience
with it?)
On Wed, Nov 18, 2015 at 12:25 PM, Dave Humphrey <dave(a)uesp.net> wrote:
My usual strategy is to check server-status and
if I need more detail go
with debugging tools (gdp etc..., see
http://serverfault.com/questions/487530/find-out-what-high-cpu-usage-apache…
).
It seems you have done this, however, and I'm wondering why you haven't
at
least been able to narrow down the issue? You
should at least be able to
know which PHP file is locking up/crashing or the rough area/cause?
Once you know roughly where it is you can add temporary PHP logging
commands in the code to help narrow down the issue further. If you also
know roughly where/how the lockups are you can try testing/replicating
the
behavior to get a bit more control on it.
On 18 November 2015 at 14:59, Justin Lloyd <jlloyd.wiki(a)gmail.com>
wrote:
Hey everyone,
Yesterday I posted this to /r/mediawiki (
https://redd.it/3t2apu) and
cross-posted to /r/apache as well, but unfortunately I've still not
received any feedback other than the one request here for clarification
and
> a couple of suggestions on reddit that I'd already covered in the post.
>
> It's possible no one has any suggestions for me regarding this issue
(it
is
a somewhat complex application stack that could
be requiring
configuration
> and/or tuning in multiple places, for example), but given how severe
of a
problem
this is for my production sites, I wanted to bump it once in
hopes
of possibly getting at least some pointers of
things to consider that I
may
> not have already, especially with respect to diagnostics I could
perform
on
the live web servers beyond just server-status
and the collectd apache
plugin (which is basically the same thing), for example.
On Thu, Nov 12, 2015 at 8:02 AM, Justin Lloyd <jlloyd.wiki(a)gmail.com>
wrote:
> Marcin,
>
> It's the biggest and most heavily trafficked of our wikis because its
the
> > English-language version of the wiki. We also have German, French,
and
> > Spanish, but the English-speaking
community is by far the largest and
> most
> > active. There are some tiny configuration differences between the
wikis
> > (e.g. the value of $wgJobRunRate, the
specific extensions loaded) but
> > nothing very significant I don't believe.
> >
> > I should also add that all four of these wikis (we have a 5th, for 7
> > total, not 6 as I'd originally said) also use Semantic MediaWiki
> > extensively. I believe the other three wikis would run into the same
> > problem if they had same amount of traffic as the English one.
However,
> > since they all are vhosts within the
same Apache instances, the
English
> > one's problems affect all of them.
> >
> > Justin
> >
> >
> > On Thu, Nov 12, 2015 at 1:42 AM, Marcin Cieslak <saper(a)saper.info>
> wrote:
> >
> >> On 2015-11-12, Justin Lloyd <jlloyd.wiki(a)gmail.com> wrote:
> >> > * Six wikis are configured as Vhosts in Apache, load balanced by a
> >> separate
> >> > set of front-end servers, where two of the wikis are for private
> >> internal
> >> > use and the other four are public, though the traffic to one of
the
>>
public
>> > wikis dwarfs the rest and it's the wiki giving me problems.
>>
>> (...)
>>
>> > I'm mainly looking right now for how to troubleshoot the stuck
>> processes,
>> > but any advice regarding this architecture is also welcome, as I
feel
> it
> >> > could use some improvement but I'm not sure how just yet.
> >>
> >> The question that immediately comes to my mind before I start
digging
> any further - how is the wiki making problems
special? Is it just
getting
> most of the traffic (it is the "most
interesting" one) or is its
> configuration slightly different?
>
> Marcin Cieślak
>
https://www.mediawiki.org/wiki/User:Saper
>
>
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
>
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
--
Dave Humphrey -- dave(a)uesp.net
Founder/Server Admin of the Unofficial Elder Scrolls Pages --
www.uesp.net
www.viud.net - Building the world's toughest
USB drive
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
--
Dave Humphrey -- dave(a)uesp.net
Founder/Server Admin of the Unofficial Elder Scrolls Pages --