There've been some issues reported lately with image scaling, where resource usage on very large images has been huge (problematic for batch uploads from a high-resolution source). Even scaling time for typical several-megapixel JPEG photos can be slower than desired when loading up into something like the MMV extension.
I've previously proposed limiting the generatable thumb sizes and pre-generating those fixed sizes at upload time, but this hasn't been a popular idea because of the lack of flexibility and potentially poor client-side scaling or inefficient network use sending larger-than-needed fixed image sizes.
Here's an idea that blends the performance benefits of pre-scaling with the flexibility of our current model...
A classic technique in 3d graphics is mip-mappinghttps://en.wikipedia.org/wiki/Mip-mapping, where an image is pre-scaled to multiple resolutions, usually each 1/2 the width and height of the next level up.
When drawing a textured polygon on screen, the system picks the most closely-sized level of the mipmap to draw, reducing the resources needed and avoiding some classes of aliasing/moiré patterns when scaling down. If you want to get fancy you can also use trilinear filteringhttps://en.wikipedia.org/wiki/Trilinear_filtering, where the next-size-up and next-size-down mip-map levels are combined -- this further reduces artifacting.
I'm wondering if we can use this technique to help with scaling of very large images: * at upload time, perform a series of scales to produce the mipmap levels * _don't consider the upload complete_ until those are done! a web uploader or API-using bot should probably wait until it's done before uploading the next file, for instance... * once upload is complete, keep on making user-facing thumbnails as before... but make them from the smaller mipmap levels instead of the full-scale original
This would avoid changing our external model -- where server-side scaling can be used to produce arbitrary-size images that are well-optimized for their target size -- while reducing resource usage for thumbs of huge source images. We can also still do things like applying a sharpening effect on photos, which people sorely miss when it's missing.
If there's interest in investigating this scenario I can write up an RfC with some more details.
(Properly handling multi-page files like PDFs, DjVu, or paged TIFFs could complicate this by making the initial rendering extraction pretty slow, though, so that needs consideration.)
-- brion
Brion Vibber, 30/04/2014 21:51:
If there's interest in investigating this scenario I can write up an RfC with some more details.
I'd say just go on and update/replace https://www.mediawiki.org/wiki/Requests_for_comment/Standardized_thumbnails_sizes with your proposal. :) (And then mark it as accepted right away? :P)
Nemo
On 04/30/2014 12:51 PM, Brion Vibber wrote:
- at upload time, perform a series of scales to produce the mipmap levels
- _don't consider the upload complete_ until those are done! a web uploader
or API-using bot should probably wait until it's done before uploading the next file, for instance...
- once upload is complete, keep on making user-facing thumbnails as
before... but make them from the smaller mipmap levels instead of the full-scale original
If there's interest in investigating this scenario I can write up an RfC with some more details.
Yes, please do! This is very close to what Aaron & me have been discussing recently on the ops list as well.
Gabriel
That's a great idea. However, the following must be ensured: * The mip-mapped images can be purged manually. * The mip-mapped media files are removed if a software change affects how images are rendered, (e.g. if new metadata are considered) * Mip-mapped media files are updated if an image was overwritten; it is essentially important that *all* of them are updated. Inconsistencies here would cause a lot of confusion. * Currently, JPEG images are sharpened. Does this affect how the end-result looks like if they are sharpened twice? Do you expect other quality-issues, for example for GIF or PNG? Is there a comparison of applying down-scaling twice with doing it just once?
--Rillke
Regarding purges, software updates and overwrites, these images are no different than the thumbnails of arbitrary sizes we currently have. If there's an issue with any of these aspects, it's already affecting us and it's unrelated to the mipmapping discussion.
* Currently, JPEG images are sharpened. Does this affect how the
end-result looks like if they are sharpened twice? Do you expect other quality-issues, for example for GIF or PNG?
Since GIF and PNG use lossless compression, loss of quality for those formats will be even less visible than JPG. That's why I've focused my tests on JPGs so far, it's the worst case scenario. If JPG visual quality can be maintained with this technique despite the lossy nature of the JPG compression, GIF and PNG won't be a problem.
Is there a comparison of
applying down-scaling twice with doing it just once?
I'd like the verify the visual impact of downscaling every step first, because it offers the biggest performance gain. We can always look at the alternative of having only one big as plan B.
I've just sent a survey to wikitech-l [1] to gather some human results and verify if there's any statistically difference in perception between JPGs that have been generated through the existing image scaler and images that have been resized 4-5 times.
[1] http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076286.html
On Fri, May 2, 2014 at 10:14 AM, Rainer Rillke rainerrillke@hotmail.comwrote:
That's a great idea. However, the following must be ensured:
- The mip-mapped images can be purged manually.
- The mip-mapped media files are removed if a software change affects
how images are rendered, (e.g. if new metadata are considered)
- Mip-mapped media files are updated if an image was overwritten; it is
essentially important that *all* of them are updated. Inconsistencies here would cause a lot of confusion.
- Currently, JPEG images are sharpened. Does this affect how the
end-result looks like if they are sharpened twice? Do you expect other quality-issues, for example for GIF or PNG? Is there a comparison of applying down-scaling twice with doing it just once?
--Rillke
Multimedia mailing list Multimedia@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/multimedia
multimedia@lists.wikimedia.org