Whenever a file is linked to with a size specification, e.g. [[File:test.png|thumb|123px]], a new thumbnail is generated in that particular size, and saved to the disk. This is generally a good thing, because it minimises the amount of data the clients need to download without losing quality at that display size. However, this is also an avenue for denial of service - someone could create many links to different images with non-standard sizes, intentionally or unintentionally, and therefore overload computational (temporarily) and storage resources on the server.
Therefore, I propose an option which would either limit the number of stored/generated thumbnails or limit their sizes to a particular set (e.g. powers of two) - however, this should not come at a loss of functionality. Whenever an image link requests a size which can't be generated, for whatever reason, either the next-largest or the next-smallest image is sent, with relevant CSS styles to resize it in the browser. The decision between next-largest and next-smallest would be governed by a user-preferences option which would default to 25% i.e. send the smaller image if the larger image is at least 75% larger than the target size (this should probably use the thumbnail area for comparisons rather than the width, if it's not a major performance hit).
This proposal is especially important for public deployments with large amounts of (especially non-technical) users and/or tight limits on disk space.
On 23 January 2013 13:24, Georgiy Tugai georgiy.tugai@gmail.com wrote:
However, this is also an avenue for denial of service - someone could create many links to different images with non-standard sizes, intentionally or unintentionally, and therefore overload computational (temporarily) and storage resources on the server.
Note that this has been debated at least a few times in the past. See e.g. the thread at http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/63701/
-- [[cs:User:Mormegil | Petr Kadlec]]
I felt the last conversation stalled due to a lack of data and lots of speculation.
It would be great if someone in the analytics team could give an idea of the most commonly requested thumbnails and we could use this as the basis for a new conversation.
I'd suggest that once we have such a list we standardise on several sizes and deprecate the others - returning the closest size match for those and leaving it to css/width and height attributes to perform the scaling wanted.
On Wed, Jan 23, 2013 at 8:25 AM, Petr Kadlec petr.kadlec@gmail.com wrote:
On 23 January 2013 13:24, Georgiy Tugai georgiy.tugai@gmail.com wrote:
However, this is also an avenue for denial of service - someone could create many links to different images with non-standard sizes, intentionally or unintentionally, and therefore overload computational (temporarily) and storage resources on the server.
Note that this has been debated at least a few times in the past. See e.g. the thread at http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/63701/
-- [[cs:User:Mormegil | Petr Kadlec]] _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I'd strongly suggest considering this kind of approach.
-- View this message in context: http://wikimedia.7.n6.nabble.com/Limiting-storage-generation-of-thumbnails-w... Sent from the Wikipedia Developers mailing list archive at Nabble.com.
On Wed, Jan 23, 2013 at 01:53:39PM -0800, Aaron Schulz wrote:
I'd strongly suggest considering this kind of approach.
Ditto. Among other benefits already mentioned, having a predetermined set of sizes would help greatly in the architecture and capacity planning of media storage, as well as in various maintenance tasks that we occasionally need to do (like replicating thumbnails across datacenters).
Regards, Faidon
As the linked thread mentioned, MediaWiki is used on many deployments outside of Wikimedia. For example, I'm running three instances on a server with 10 GB of disk space. Therefore, this should be an option in LocalSettings.php rather than a global change; that way, the installations which are CPU-constrained rather than disk-constrained can use other solutions, from user education to Squid.
On 24/01/13 04:15, Jon Robson wrote:
I felt the last conversation stalled due to a lack of data and lots of speculation.
It would be great if someone in the analytics team could give an idea of the most commonly requested thumbnails and we could use this as the basis for a new conversation.
I'd suggest that once we have such a list we standardise on several sizes and deprecate the others - returning the closest size match for those and leaving it to css/width and height attributes to perform the scaling wanted.
On Wed, Jan 23, 2013 at 8:25 AM, Petr Kadlec petr.kadlec@gmail.com wrote:
On 23 January 2013 13:24, Georgiy Tugai georgiy.tugai@gmail.com wrote:
However, this is also an avenue for denial of service - someone could create many links to different images with non-standard sizes, intentionally or unintentionally, and therefore overload computational (temporarily) and storage resources on the server.
Note that this has been debated at least a few times in the past. See e.g. the thread at http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/63701/
-- [[cs:User:Mormegil | Petr Kadlec]] _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 01/23/2013 10:07 PM, Georgiy Tugai wrote:
As the linked thread mentioned, MediaWiki is used on many deployments outside of Wikimedia. For example, I'm running three instances on a server with 10 GB of disk space. Therefore, this should be an option in LocalSettings.php rather than a global change; that way, the installations which are CPU-constrained rather than disk-constrained can use other solutions, from user education to Squid.
Yeah, I think it should probably an option (we could debate the default later). Besides the obvious bandwidth issue, some browsers may do worse scaling then MediaWiki's library (imagemagick I believe). Of course, others might be better.
Matt Flaschen
On 24/01/13 13:54, Matthew Flaschen wrote:
On 01/23/2013 10:07 PM, Georgiy Tugai wrote:
As the linked thread mentioned, MediaWiki is used on many deployments outside of Wikimedia. For example, I'm running three instances on a server with 10 GB of disk space. Therefore, this should be an option in LocalSettings.php rather than a global change; that way, the installations which are CPU-constrained rather than disk-constrained can use other solutions, from user education to Squid.
Yeah, I think it should probably an option (we could debate the default later). Besides the obvious bandwidth issue, some browsers may do worse scaling then MediaWiki's library (imagemagick I believe). Of course, others might be better.
Matt Flaschen
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Maybe put a CSS box over/next to the image, in preview mode and possibly in normal viewing mode as well, suggesting that the user specifies a standard size for higher quality etc..? Also, a "lightbox" feature may help - expand the image to the full size (as sent by the server) on click, then link to the file page upon a second click (hide the "lightbox" upon a click outside of the image borders). I am unsure how difficult this feature would be to add -- I have not dug around in MediaWiki's code before. Can someone point me in the right direction?
On 01/24/2013 04:28 AM, Georgiy Tugai wrote:
Also, a "lightbox" feature may help - expand the image to the full size (as sent by the server) on click, then link to the file page upon a second click (hide the "lightbox" upon a click outside of the image borders). I am unsure how difficult this feature would be to add -- I have not dug around in MediaWiki's code before. Can someone point me in the right direction?
It shouldn't be too difficult. You could start with a gadget (https://www.mediawiki.org/wiki/Gadget), then if it gets popular, try to move it to core.
I might be able to help if you have questions.
Matt Flaschen
On 25/01/13 08:12, Matthew Flaschen wrote:
On 01/24/2013 04:28 AM, Georgiy Tugai wrote:
Also, a "lightbox" feature may help - expand the image to the full size (as sent by the server) on click, then link to the file page upon a second click (hide the "lightbox" upon a click outside of the image borders). I am unsure how difficult this feature would be to add -- I have not dug around in MediaWiki's code before. Can someone point me in the right direction?
It shouldn't be too difficult. You could start with a gadget (https://www.mediawiki.org/wiki/Gadget), then if it gets popular, try to move it to core.
I might be able to help if you have questions.
Matt Flaschen
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I'm sorry that I wasn't clear enough in my question - I meant the entire issue, not just the "lightbox", i.e. limiting generation of (non-standard) thumbnails, as explained in the "opening post" (if such terminology applies to mailing lists).
On 01/23/2013 12:45 PM, Jon Robson wrote:
It would be great if someone in the analytics team could give an idea of the most commonly requested thumbnails and we could use this as the basis for a new conversation.
Serendipitously, about the same time you mentioned "most commonly requested thumbnails", User:Nicholasjf21 showed up at the MW Support Desk asking for help with the following problem:
We're currently trying to brighten our Main Page with some large images, but could do with them scaling automatically according to the user's screen resolution. Is there any way of doing this?
I pointed to this discussion, but it seems like this would be the perfect time to get some statistics on image scaling. If this sort of scaling became more common, I could see it resulting in a large number of awkward resolution thumbnails that are only rarely used.
Mark.
wikitech-l@lists.wikimedia.org