Since we upgraded to MediaWiki v1.22 we've been having intermittent performance issues on some pages with many images/thumbnails. After a bunch of poking around it seems to be due to thumbnails not being saved to disk. It seems they are saved to a cache (assumably memcached) so the first load a user does is slow and then it is fine for about a day.
As an example we have this page: http://content3.uesp.net/wiki/Online:Ardent_Flame On a "normal" logged-in session load the page takes 0.5 seconds for the article to load. However, on a slow load it takes 8-10 seconds for the article to load. You can force this to happen by doing a purge on the page (?action=purge).
If we enable profiling/debug we see a bunch of lines like:
File::transform: Doing stat for mwstore://local-backend/local-thumb/1/11/ON-icon-skill-Dragonknight-Dragonknight_Standard.png/48px-ON-icon-skill-Dragonknight-Dragonknight_Standard.png BitmapHandler::doTransform: creating 48x48 thumbnail at mwstore://local-backend/local-thumb/1/11/ON-icon-skill-Dragonknight-Dragonknight_Standard.png/48px-ON-icon-skill-Dragonknight-Dragonknight_Standard.png using scaler im BitmapHandler::doTransform: Transforming later per flags.
(see http://www.uesp.net/wiki/User:Daveh/TestProfile1 for more). The strange thing is that a lot of these thumbnails files already exist so I don't understand why they're being created. For thumbnails that don't exist we see lines like:
FileBackendStore::getFileStat: File mwstore://local-backend/local-thumb/b/b4/ON-icon-ActiveFrame.png/64px-ON-icon-ActiveFrame.png does not exist. BitmapHandler::doTransform: creating 64x64 thumbnail at /home/uesp/www/wtest/images/tmp/transform_6fc8d30c9ff9-1.png using scaler im BitmapHandler::doTransform: returning unscaled image
Unfortunately, the specified tmp file doesn't exist nor does the thumbnail file after the page is loaded.
A few "obvious" things I've checked: - Thumbnail files/paths are all readable/writable by Apache - Apache error logs (nothing) - MediaWiki debug output (nothing obvious)
Some of the performance issue for this page could be also related to heavy use of templates but I'd like to figure out the thumbnail issue and rule it out as a cause first.
A few specific questions: 1) Why are thumbnails being created that already exist? 2) Why are thumbnails not being created that don't exist, either as a tmp file or thumbnail file? 3) What does "Transforming later per flags." mean? When is "later"?
Any help or suggestions would be welcome...Thanks!
Is anyone able to offer any advice or suggestions on this? I posted to the list back on 4/23/15 with a very similar issue and was unable to get any suggestions or make any headway with the issue. Using the SupportDesk and IRC also turned up nothing. My original post (which also includes a debug log from my wiki) is included in this email, although the actual wiki is currently down due to a host server issue. I am sure Dave is with me in saying that any help or suggestions are appreciated.
---
Hi there,
I have MW 1.24.1 installed on CentOs7 with nGinx and PHP-FPM 5.4.16. I am not using ImageMagick.
With that stated, I am having an issue where thumbs are not being created for files when they are used in articles with a specific width defined. Image uploads work fine, and a thumb gets made when a file is uploaded (I assume it's what is displayed on the File:) page. However, attempting to use the image with a new size ends up with a 404 due to the thumb not being there. I am not getting any PHP errors, there is nothing in my system log or my web server logs that would indicate a permission issue.
I enabled debugging on a page that attempted to use a resized image, and the log can be found at http://pastebin.com/yeif2YHw
I don't know enough PHP to understand what the TransformationalImageHandler is doing, but I have a strong feeling that...
TransformationalImageHandler::doTransform: creating 150x200 thumbnail at mwstore://shared-backend/shared-thumb/6/67/BFZLogo.png/150px-BFZLogo.png using scaler gd TransformationalImageHandler::doTransform: Transforming later per flags. File::transform transformation deferred. (this appears for 3 different thumbnail sizes)
...isn't a good thing. The specific page that I am using to test this on (and which generated the above debug output) is http://wiki.magicjudges.org/en/w/User:Mumbles
I appreciate in advance any assistance you can give, and will provide whatever information I am able to if there's something that would help diagnose this issue.
Cheers, Adam
On Thu, Sep 17, 2015 at 9:15 AM, Dave Humphrey dave@uesp.net wrote:
Since we upgraded to MediaWiki v1.22 we've been having intermittent performance issues on some pages with many images/thumbnails. After a bunch of poking around it seems to be due to thumbnails not being saved to disk. It seems they are saved to a cache (assumably memcached) so the first load a user does is slow and then it is fine for about a day.
As an example we have this page: http://content3.uesp.net/wiki/Online:Ardent_Flame On a "normal" logged-in session load the page takes 0.5 seconds for the article to load. However, on a slow load it takes 8-10 seconds for the article to load. You can force this to happen by doing a purge on the page (?action=purge).
If we enable profiling/debug we see a bunch of lines like:
File::transform: Doing stat for
mwstore://local-backend/local-thumb/1/11/ON-icon-skill-Dragonknight-Dragonknight_Standard.png/48px-ON-icon-skill-Dragonknight-Dragonknight_Standard.png BitmapHandler::doTransform: creating 48x48 thumbnail at
mwstore://local-backend/local-thumb/1/11/ON-icon-skill-Dragonknight-Dragonknight_Standard.png/48px-ON-icon-skill-Dragonknight-Dragonknight_Standard.png using scaler im BitmapHandler::doTransform: Transforming later per flags.
(see http://www.uesp.net/wiki/User:Daveh/TestProfile1 for more). The strange thing is that a lot of these thumbnails files already exist so I don't understand why they're being created. For thumbnails that don't exist we see lines like:
FileBackendStore::getFileStat: File
mwstore://local-backend/local-thumb/b/b4/ON-icon-ActiveFrame.png/64px-ON-icon-ActiveFrame.png does not exist. BitmapHandler::doTransform: creating 64x64 thumbnail at /home/uesp/www/wtest/images/tmp/transform_6fc8d30c9ff9-1.png using scaler im BitmapHandler::doTransform: returning unscaled image
Unfortunately, the specified tmp file doesn't exist nor does the thumbnail file after the page is loaded.
A few "obvious" things I've checked:
- Thumbnail files/paths are all readable/writable by Apache
- Apache error logs (nothing)
- MediaWiki debug output (nothing obvious)
Some of the performance issue for this page could be also related to heavy use of templates but I'd like to figure out the thumbnail issue and rule it out as a cause first.
A few specific questions:
- Why are thumbnails being created that already exist?
- Why are thumbnails not being created that don't exist, either as a tmp
file or thumbnail file? 3) What does "Transforming later per flags." mean? When is "later"?
Any help or suggestions would be welcome...Thanks!
-- Dave Humphrey -- dave@uesp.net Founder/Server Admin of the Unofficial Elder Scrolls Pages -- www.uesp.net www.viud.net - Building the world's toughest USB drive _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
There's a similar problem documented here: https://www.mediawiki.org/wiki/Topic:Rcda3wyw90xuf503
The solution was to :
You need $wgHashedUploadDirectory = true; $wgGenerateThumbnailOnParse = true;
and $wgMaxShellMemory = 204800; if your images are large.
Worth a shot I guess.
On Mon, Sep 21, 2015 at 7:37 AM, Adam Hubble adamhubble@gmail.com wrote:
Is anyone able to offer any advice or suggestions on this? I posted to the list back on 4/23/15 with a very similar issue and was unable to get any suggestions or make any headway with the issue. Using the SupportDesk and IRC also turned up nothing. My original post (which also includes a debug log from my wiki) is included in this email, although the actual wiki is currently down due to a host server issue. I am sure Dave is with me in saying that any help or suggestions are appreciated.
Hi there,
I have MW 1.24.1 installed on CentOs7 with nGinx and PHP-FPM 5.4.16. I am not using ImageMagick.
With that stated, I am having an issue where thumbs are not being created for files when they are used in articles with a specific width defined. Image uploads work fine, and a thumb gets made when a file is uploaded (I assume it's what is displayed on the File:) page. However, attempting to use the image with a new size ends up with a 404 due to the thumb not being there. I am not getting any PHP errors, there is nothing in my system log or my web server logs that would indicate a permission issue.
I enabled debugging on a page that attempted to use a resized image, and the log can be found at http://pastebin.com/yeif2YHw
I don't know enough PHP to understand what the TransformationalImageHandler is doing, but I have a strong feeling that...
TransformationalImageHandler::doTransform: creating 150x200 thumbnail at mwstore://shared-backend/shared-thumb/6/67/BFZLogo.png/150px-BFZLogo.png using scaler gd TransformationalImageHandler::doTransform: Transforming later per flags. File::transform transformation deferred. (this appears for 3 different thumbnail sizes)
...isn't a good thing. The specific page that I am using to test this on (and which generated the above debug output) is http://wiki.magicjudges.org/en/w/User:Mumbles
I appreciate in advance any assistance you can give, and will provide whatever information I am able to if there's something that would help diagnose this issue.
Cheers, Adam
On Thu, Sep 17, 2015 at 9:15 AM, Dave Humphrey dave@uesp.net wrote:
Since we upgraded to MediaWiki v1.22 we've been having intermittent performance issues on some pages with many images/thumbnails. After a bunch of poking around it seems to be due to thumbnails not being saved to disk. It seems they are saved to a cache (assumably memcached) so the first load a user does is slow and then it is fine for about a day.
As an example we have this page: http://content3.uesp.net/wiki/Online:Ardent_Flame On a "normal" logged-in session load the page takes 0.5 seconds for the article to load. However, on a slow load it takes 8-10 seconds for the article to load. You can force this to happen by doing a purge on the page (?action=purge).
If we enable profiling/debug we see a bunch of lines like:
File::transform: Doing stat for
mwstore://local-backend/local-thumb/1/11/ON-icon-skill-Dragonknight-Dragonknight_Standard.png/48px-ON-icon-skill-Dragonknight-Dragonknight_Standard.png BitmapHandler::doTransform: creating 48x48 thumbnail at
mwstore://local-backend/local-thumb/1/11/ON-icon-skill-Dragonknight-Dragonknight_Standard.png/48px-ON-icon-skill-Dragonknight-Dragonknight_Standard.png using scaler im BitmapHandler::doTransform: Transforming later per flags.
(see http://www.uesp.net/wiki/User:Daveh/TestProfile1 for more). The strange thing is that a lot of these thumbnails files already exist so I don't understand why they're being created. For thumbnails that don't exist we see lines like:
FileBackendStore::getFileStat: File
mwstore://local-backend/local-thumb/b/b4/ON-icon-ActiveFrame.png/64px-ON-icon-ActiveFrame.png does not exist. BitmapHandler::doTransform: creating 64x64 thumbnail at /home/uesp/www/wtest/images/tmp/transform_6fc8d30c9ff9-1.png using scaler im BitmapHandler::doTransform: returning unscaled image
Unfortunately, the specified tmp file doesn't exist nor does the thumbnail file after the page is loaded.
A few "obvious" things I've checked:
- Thumbnail files/paths are all readable/writable by Apache
- Apache error logs (nothing)
- MediaWiki debug output (nothing obvious)
Some of the performance issue for this page could be also related to heavy use of templates but I'd like to figure out the thumbnail issue and rule it out as a cause first.
A few specific questions:
- Why are thumbnails being created that already exist?
- Why are thumbnails not being created that don't exist, either as a tmp
file or thumbnail file? 3) What does "Transforming later per flags." mean? When is "later"?
Any help or suggestions would be welcome...Thanks!
-- Dave Humphrey -- dave@uesp.net Founder/Server Admin of the Unofficial Elder Scrolls Pages -- www.uesp.net www.viud.net - Building the world's toughest USB drive _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Thanks for the response Bill. I saw that topic while attempting to research the issue before posting, and verified those settings were in place.
Cheers, Adam
On Mon, Sep 21, 2015 at 6:52 AM, Bill Traynor btraynor@gmail.com wrote:
There's a similar problem documented here: https://www.mediawiki.org/wiki/Topic:Rcda3wyw90xuf503
The solution was to :
You need $wgHashedUploadDirectory = true; $wgGenerateThumbnailOnParse = true;
and $wgMaxShellMemory = 204800; if your images are large.
Worth a shot I guess.
On Mon, Sep 21, 2015 at 7:37 AM, Adam Hubble adamhubble@gmail.com wrote:
Is anyone able to offer any advice or suggestions on this? I posted to
the
list back on 4/23/15 with a very similar issue and was unable to get any suggestions or make any headway with the issue. Using the SupportDesk and IRC also turned up nothing. My original post (which also includes a debug log from my wiki) is included in this email, although the actual wiki is currently down due to a host server issue. I am sure Dave is with me in saying that any help or suggestions are appreciated.
Hi there,
I have MW 1.24.1 installed on CentOs7 with nGinx and PHP-FPM 5.4.16. I am not using ImageMagick.
With that stated, I am having an issue where thumbs are not being created for files when they are used in articles with a specific width defined. Image uploads work fine, and a thumb gets made when a file is uploaded (I assume it's what is displayed on the File:) page. However, attempting to use the image with a new size ends up with a 404 due to the thumb not
being
there. I am not getting any PHP errors, there is nothing in my system log or my web server logs that would indicate a permission issue.
I enabled debugging on a page that attempted to use a resized image, and the log can be found at http://pastebin.com/yeif2YHw
I don't know enough PHP to understand what the
TransformationalImageHandler
is doing, but I have a strong feeling that...
TransformationalImageHandler::doTransform: creating 150x200 thumbnail at mwstore://shared-backend/shared-thumb/6/67/BFZLogo.png/150px-BFZLogo.png using scaler gd TransformationalImageHandler::doTransform: Transforming later per flags. File::transform transformation deferred. (this appears for 3 different thumbnail sizes)
...isn't a good thing. The specific page that I am using to test this on (and which generated the above debug output) is http://wiki.magicjudges.org/en/w/User:Mumbles
I appreciate in advance any assistance you can give, and will provide whatever information I am able to if there's something that would help diagnose this issue.
Cheers, Adam
On Thu, Sep 17, 2015 at 9:15 AM, Dave Humphrey dave@uesp.net wrote:
Since we upgraded to MediaWiki v1.22 we've been having intermittent performance issues on some pages with many images/thumbnails. After a
bunch
of poking around it seems to be due to thumbnails not being saved to
disk.
It seems they are saved to a cache (assumably memcached) so the first
load
a user does is slow and then it is fine for about a day.
As an example we have this page: http://content3.uesp.net/wiki/Online:Ardent_Flame On a "normal" logged-in session load the page takes 0.5 seconds for the article to load. However, on a slow load it takes 8-10 seconds for the article to load. You can force this to happen by doing a purge on the
page
(?action=purge).
If we enable profiling/debug we see a bunch of lines like:
File::transform: Doing stat for
mwstore://local-backend/local-thumb/1/11/ON-icon-skill-Dragonknight-Dragonknight_Standard.png/48px-ON-icon-skill-Dragonknight-Dragonknight_Standard.png
BitmapHandler::doTransform: creating 48x48 thumbnail at
mwstore://local-backend/local-thumb/1/11/ON-icon-skill-Dragonknight-Dragonknight_Standard.png/48px-ON-icon-skill-Dragonknight-Dragonknight_Standard.png
using scaler im BitmapHandler::doTransform: Transforming later per flags.
(see http://www.uesp.net/wiki/User:Daveh/TestProfile1 for more). The strange thing is that a lot of these thumbnails files already exist so I don't understand why they're being created. For thumbnails that don't
exist
we see lines like:
FileBackendStore::getFileStat: File
mwstore://local-backend/local-thumb/b/b4/ON-icon-ActiveFrame.png/64px-ON-icon-ActiveFrame.png
does not exist. BitmapHandler::doTransform: creating 64x64 thumbnail at /home/uesp/www/wtest/images/tmp/transform_6fc8d30c9ff9-1.png using
scaler
im BitmapHandler::doTransform: returning unscaled image
Unfortunately, the specified tmp file doesn't exist nor does the
thumbnail
file after the page is loaded.
A few "obvious" things I've checked:
- Thumbnail files/paths are all readable/writable by Apache
- Apache error logs (nothing)
- MediaWiki debug output (nothing obvious)
Some of the performance issue for this page could be also related to
heavy
use of templates but I'd like to figure out the thumbnail issue and
rule it
out as a cause first.
A few specific questions:
- Why are thumbnails being created that already exist?
- Why are thumbnails not being created that don't exist, either as a
tmp
file or thumbnail file? 3) What does "Transforming later per flags." mean? When is "later"?
Any help or suggestions would be welcome...Thanks!
-- Dave Humphrey -- dave@uesp.net Founder/Server Admin of the Unofficial Elder Scrolls Pages --
www.uesp.net
www.viud.net - Building the world's toughest USB drive _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
mediawiki-l@lists.wikimedia.org