For the last decade we've supported uploading SVG vector images to MediaWiki, but we serve them as rasterized PNGs to browsers. Recently, display resolutions are going up and up, but so is concern about low-bandwidth mobile users.
This means we'd like sharper icons and diagrams on high-density phone displays, but are leery of adding extra srcset entries with 3x or 4x size PNGs which could become very large. (In fact currently MobileFrontend strips even the 1.5x and 2x renderings we have now, making diagrams very blurry on many mobile devices. See https://phabricator.wikimedia.org/T133496 - fix in works.)
Here's the base bug for SVG client side rendering: https://phabricator.wikimedia.org/T5593 I've turned it into an "epic" story tracking task and hung some blocking tasks off it; see those for more details.
TL;DR stop reading here. ;)
One of the basic problems in the past was reliably showing them natively in an <img>, with the same behavior as before, without using JavaScript hacks or breaking the hamlet caching layer. This is neatly resolved for current browsers by using the "srcset" attribute -- the same one we use to specify higher-resolution rasterizations. If instead of PNGs at 1.5x and 2x density, we specify an SVG at 1x, the SVG will be loaded instead of the default PNG.
Since all srcset-supporting browsers allow SVG in <img> this should "just work", and will be more compatible than using the experimental <picture> element or the classic <object> which deals with events differently. Older browsers will still see the PNG, and we can tweak the jquery.hidpi srcset polyfill to test for SVG support to avoid breaking on some older browsers.
This should let us start testing client-side SVG via a beta feature (with parser cache split on the user pref) at which point we can gather more real-world feedback on performance and compatibility issues.
Rendering consistency across browser engines is a concern. Supposedly modern browsers are more consistent than librsvg but we haven't done a compatibility survey to confirm this or identify problematic constructs. This is probably worth doing.
Performance is a big question. While clean simple SVGs are often nice and small and efficient, it's also easy to make a HUGEly detailed SVG that is much larger than the rasterized PNGs. Or a fairly simple small file may still render slowly due to use of filters.
So we probably want to provide good tools for our editors and image authors to help optimize their files. Show the renderings and the bandwidth balance versus rasterization; maybe provide in-wiki implementation of svgo or other lossy optimizer tools. Warn about things that are large or render slowly. Maybe provide a switch to run particular files through rasterization always.
And we'll almost certainly want to strip comments and white space to save bandwidth on page views, while retaining them all in the source file for download and reediting.
Feature parity also needs more work. Localized text in SVGs is supported with our server side rendering but this won't be reliable in the client; which means we'll want to perform a server side transformation that creates per-language "thumbnail" SVGs. Fonts for internationalized text are a big deal, and may require similar transformations if we want to serve them... Which may mean additional complications and bandwidth usage.
And then there are long term goals of taking more advantage of SVGs dynamic nature -- making things animated or interactive. That's a much bigger question and has implementation and security issues!
-- brion
On Thu, May 5, 2016 at 9:49 AM, Brion Vibber bvibber@wikimedia.org wrote:
Rendering consistency across browser engines is a concern. Supposedly modern browsers are more consistent than librsvg but we haven't done a compatibility survey to confirm this or identify problematic constructs. This is probably worth doing.
There's also the font issue: with the current rasterizing, only certain free fonts are available and we know what it falls back to. Serve the SVG to the client and those free fonts may well be missing (falling back to who-knows-what), while other fonts with different metrics that our rasterizer ignores might be present on the client.
And we'll almost certainly want to strip comments and white space to save bandwidth on page views, while retaining them all in the source file for download and reediting.
There's other stuff that could be stripped, too. For example, Inkscape's default "Inkscape SVG" format saves a lot of extra data under its own "inkscape" and "sodipodi" namespaces and some style rules with a '-inkscape-' vendor prefix that we could probably kill.
Inkscape also likes to put 'id' attributes on absolutely everything (e.g. rect3342, tspan8263), which we could strip if we can somehow determine they're not used somewhere else (including <style> or external reference of some sort). It also likes to be redundant with inline CSS (e.g. reiterating defaults, including all stroke-* properties despite stroke:none meaning they're unused), but that's even more problematic to determine whether it's safe to strip.
Some of the stuff in the svg:metadata node could probably also be killed, although we might want to preserve some pieces (e.g. license, author) even in minimized form and/or add some of our own (e.g. URL of the file description page where the original can be downloaded).
On Thursday, May 5, 2016, Brad Jorsch (Anomie) bjorsch@wikimedia.org wrote:
On Thu, May 5, 2016 at 9:49 AM, Brion Vibber <bvibber@wikimedia.org javascript:;> wrote: There's also the font issue: with the current rasterizing, only certain free fonts are available and we know what it falls back to. Serve the SVG to the client and those free fonts may well be missing (falling back to who-knows-what), while other fonts with different metrics that our rasterizer ignores might be present on the client.
Yep... It's worth experimenting with injecting fonts into the "thumbnail" SVG output. That'll increase total download size again, but common fonts should be shared between files, making them neatly cacheable, and if there are no relevant characters we have no need to inject them.
I think all the CSS @font stuff works in SVG as it does in HTML, so if we have known client-side-safe fonts we can reference them as we do in HTML land... For common cases where text is in the language of the surrounding site, that means we may have loaded the same font already if one was needed.
Making SVG safe for fallback fonts can indeed be tricky if the metrics don't match, though for simple labels it's usually a matter of setting reasonable bounding boxes and making use of the alignment specifications, rather than using the tightest possible bounding box and aligning things manually by x/y.
For extreme cases where exact glyph shapes and metrics matter because the text is mixed with graphical elements directly, files should probably use paths instead of text (or at least embed a specific SVG font made of specific paths). That can be something that good authoring tools can help with.
-- brion
On Thu, May 5, 2016 at 6:49 AM, Brion Vibber bvibber@wikimedia.org wrote:
And then there are long term goals of taking more advantage of SVGs dynamic nature -- making things animated or interactive. That's a much bigger question and has implementation and security issues!
Sorry for the late response (and if this is covered in one of the linked bugs), but getting back to this-- are you envisioning SVG's inlined into the html? Or would they be served from a domain like upload.wm.o, like we currently use for uploaded images?
On Wed, May 11, 2016 at 11:10 AM, Chris Steipp csteipp@wikimedia.org wrote:
On Thu, May 5, 2016 at 6:49 AM, Brion Vibber bvibber@wikimedia.org wrote:
And then there are long term goals of taking more advantage of SVGs
dynamic
nature -- making things animated or interactive. That's a much bigger question and has implementation and security issues!
Sorry for the late response (and if this is covered in one of the linked bugs), but getting back to this-- are you envisioning SVG's inlined into the html?
Maybe someday that might be interesting for interactive bits, but not in the short term.
Or would they be served from a domain like upload.wm.o, like we currently use for uploaded images?
Yes for now.
-- brion
Thanks for starting this conversation Brion!
On mobile we are constantly tackling the trade offs between data shipped (cost for end user) and quality. Hopefully we'll have a better solution for this by the end of the quarter which will allow us to reconsider srcset usage. That said relying on srcset to render SVGs well seems like a hack to me and thus I'd rather we fixed the problem at the source.
A few thoughts * The ResourceLoaderImage module is being used widely to generate SVG icons with png fallbacks. I'd be interested in seeing if we can use this in some way for optimising SVGs and removing meta data. * I wonder if there is an opportunity here to use BetaFeatures and mobile beta to get a sense of performance impact of these changes? This would allow people to opt in and for us to crowdsource feedback in a safe enviroment.
Jon
On Wed, May 11, 2016 at 11:12 AM, Brion Vibber bvibber@wikimedia.org wrote:
On Wed, May 11, 2016 at 11:10 AM, Chris Steipp csteipp@wikimedia.org wrote:
On Thu, May 5, 2016 at 6:49 AM, Brion Vibber bvibber@wikimedia.org wrote:
And then there are long term goals of taking more advantage of SVGs
dynamic
nature -- making things animated or interactive. That's a much bigger question and has implementation and security issues!
Sorry for the late response (and if this is covered in one of the linked bugs), but getting back to this-- are you envisioning SVG's inlined into the html?
Maybe someday that might be interesting for interactive bits, but not in the short term.
Or would they be served from a domain like upload.wm.o, like we currently use for uploaded images?
Yes for now.
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, May 13, 2016 at 1:28 PM, Jon Robson jrobson@wikimedia.org wrote:
Thanks for starting this conversation Brion!
On mobile we are constantly tackling the trade offs between data shipped (cost for end user) and quality. Hopefully we'll have a better solution for this by the end of the quarter which will allow us to reconsider srcset usage.
Note that srcset usage for raster images is a different issue from native SVG, in that the higher-resolution renderings *do* require more bandwidth than the lower-resolution images -- especially for photos.
That said relying on srcset to render SVGs well seems like a hack to me and thus I'd rather we fixed the problem at the source.
The problem is the intersection of: * some older browsers do not support SVG in <img> * <img> provides no system for automatic fallback based on the browser's format support * <picture> does provide that, but the spec is not finalized, browser support is not finalized, and it may have implications for scripting and styling by introducing a new wrapper element
Using <img> with PNG in the src and a "1x" SVG in the srcset resolves the first case by feeding a compatible rasterization to old browsers, solves the second problem by using srcset support as a proxy check for SVG support (native srcset support implies SVG support in all known cases), and avoids the third problem by not using a new, different element for images.
We could jump straight to <picture>, but it'll have bigger QA implications in that it may affect styling and JS gadgets that look for <img> elements specifically.
<picture> would be nice for other things though -- future support of WebP for alternate, more bandwidth-efficient thumbnails, or if we start doing fancier types of responsive images based on "art direction" usages where a narrow column gets a different image than a wide window.
A few thoughts
- The ResourceLoaderImage module is being used widely to generate SVG
icons with png fallbacks. I'd be interested in seeing if we can use this in some way for optimising SVGs and removing meta data.
Definitely yes! Just as we minify CSS and JS source, we should make sure the SVG icons are efficiently packed as well. (There's usually good opportunity to do that at commit time, but it's nice to be able to include comments and such in the source.)
- I wonder if there is an opportunity here to use BetaFeatures and
mobile beta to get a sense of performance impact of these changes? This would allow people to opt in and for us to crowdsource feedback in a safe enviroment.
Yes, I am suggesting exactly that. :)
-- brion
On 2016-05-13 22:28, Jon Robson wrote:
The ResourceLoaderImage module is being used widely to generate SVG icons with png fallbacks. I'd be interested in seeing if we can use this in some way for optimising SVGs and removing meta data.
I don't know what you have in mind, but please remember that ResourceLoaderImage was not written with security in mind. It has a very simplified version of our usual SVG rendering code, and it assumes that any SVG files passed to it is trusted. We traded some caution for some performance. Giving it user-controlled data is going to result in security vulnerabilities (at the very least some denial of service ones).
Another option might be to piggy-back on the current work towards lazy-loaded images [1]. Since this is using JS, it could take into account network performance & screen resolutions, in addition to browser capabilities. Designing this to degrade gracefully without JS might be a bit tricky, though.
Gabriel
[1]: https://phabricator.wikimedia.org/T124390
On Fri, May 13, 2016 at 3:29 PM, Bartosz Dziewoński matma.rex@gmail.com wrote:
On 2016-05-13 22:28, Jon Robson wrote:
The ResourceLoaderImage module is being used widely to generate SVG icons with png fallbacks. I'd be interested in seeing if we can use this in some way for optimising SVGs and removing meta data.
I don't know what you have in mind, but please remember that ResourceLoaderImage was not written with security in mind. It has a very simplified version of our usual SVG rendering code, and it assumes that any SVG files passed to it is trusted. We traded some caution for some performance. Giving it user-controlled data is going to result in security vulnerabilities (at the very least some denial of service ones).
-- Bartosz Dziewoński
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I'll note that wikipedia also currently uses the SVG systemLanguage option in a non-standard way which isn't supported by browsers: https://phabricator.wikimedia.org/T60920
So SVGs which use this feature would have to be blacklisted somehow and always rendered to PNG. --scott
On Fri, May 13, 2016 at 7:58 PM, Gabriel Wicke gwicke@wikimedia.org wrote:
Another option might be to piggy-back on the current work towards lazy-loaded images [1]. Since this is using JS, it could take into account network performance & screen resolutions, in addition to browser capabilities. Designing this to degrade gracefully without JS might be a bit tricky, though.
Gabriel
On Fri, May 13, 2016 at 3:29 PM, Bartosz Dziewoński matma.rex@gmail.com wrote:
On 2016-05-13 22:28, Jon Robson wrote:
The ResourceLoaderImage module is being used widely to generate SVG icons with png fallbacks. I'd be interested in seeing if we can use this in some way for optimising SVGs and removing meta data.
I don't know what you have in mind, but please remember that ResourceLoaderImage was not written with security in mind. It has a very simplified version of our usual SVG rendering code, and it assumes that
any
SVG files passed to it is trusted. We traded some caution for some performance. Giving it user-controlled data is going to result in
security
vulnerabilities (at the very least some denial of service ones).
-- Bartosz Dziewoński
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- Gabriel Wicke Principal Engineer, Wikimedia Foundation
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org