On Sun, May 29, 2016 at 11:22 AM, Gergo Tisza gtisza@wikimedia.org wrote:

On Sun, May 29, 2016 at 7:08 PM, Brion Vibber bvibber@wikimedia.org wrote:

...

We could change all that if there was a stable enough URL/linking API... Can probably fake it for common cases and supported handlers by generating a URL like we would for a local 404-handler repo; my main concern is for future expansion where we add more fancy types on Commons where the local site may not have an extension installed locally.

Not sure if trying to use an unhandled file type via InstantCommons makes sense; the image parameters in the wikitext and the image parameters in the URL (or imageinfo API or hypothetical linking API) don't necessarily use the same syntax; what would transform that? (Granted, you get similar problems when parameter transformation is handled by the local extension and there is a version mismatch between the local and remote installation.)

Yep, it all comes back to T66214 https://phabricator.wikimedia.org/T66214 doesn't it. :D

...

(Iframe embedding as a fallback for unknown handlers sounds possible, could expose a frame URL in the imageinfo data.)

In general that would be a nice idea (see T27854 https://phabricator.wikimedia.org/T27854). Specifically for embedding unknown file types in MediaWiki though, we would probably have to introduce some standard key-value syntax for image parameters first, instead of the current mess where it's parsed by an arbitrary set of regexes defined by the media handlers and generated in a similarly flexible way. And then you still have the problem of handling unknown media types in VisualEditor which would need some kind of parameter metadata at least.

Indeed. iframe embedding of video might be a good place to start for unknown types; since we already have an iframe embedding system for video in TimedMediaHandler, it should be easy to adapt it as a provider so that sites can use videos from Commons without installing TimedMediaHandler locally.

Key issues for options:

* separate the layout parameters from the file rendering parameters (eg things like 'thumb' or 'frame' affect the layout, but the file handler/renderer only cares about the resulting size) * define standardish keys for file options already in use * if extra file options are needed, have a standard option for extended options! * define that extra options will not be localizable or have magical regexes; they should just be key-value pairs with fixed, predictable key names * define a standard way to pass file options to the iframe

Existing file options: * 'width' and 'height' are implied through thumb/frame/upright/default size, or explicitly via /\d+px/ or /\d+x\d+px' etc regexes defined in core. * SvgHandler uses 'lang'; regex defined in core. * PdfHandler and PagedTiffHandler use 'page'; regex defined in core. * PagedTiffHandler uses 'lossy' switch to allow overriding the default thumbnail type (JPG vs PNG); regex defined in extension * TimedMediaHandler adds 'start', 'end', and 'thumbtime'; regex defined in extension

There might be a couple more floating around out there, but that should cover those already in use in Wikimedia content. For compatibility we'd need to keep the core regexes, and might or might not want to move the 'lossy' and 'start/end/thumbtime' ones to core.

I can imagine some extended attributes I'd like for video: * 'loop' * 'mute' * 'autoplay' (only allowed with 'loop' and 'mute', for GIF-like display)

for zoomable panoramic images (flat or spherical): * 'zoom' - initial zoom level * 'center' - coordinates of initial centering (what coord system? separate x/y params or just one? should spherical use lat/lon? etc)

for viewing a 3d model: * 'zoom' - just like panoramas * 'distance' - viewer distance from origin point; this is different from zoom! * 'center' - origin point/center of view, extended to 3 dimensions (for instance to concentrate on a portion of a larger model)

Let's imagine a file type that uses everything! An animated 3d model that's localizable, and has multiple selectable "pages" of which we show one:

[[File:Cool file type.mytype| <!-- layout attributes; keys may be localized --> thumb |alt=A bouncing ball changes colors while a calendar in background counts off seasons in French. <!-- width/height affects both layout and file; regex may be localized --> |640x480px <!-- currently defined core file attributes; keys may be localized --> |lang=fr |page=3 <!-- currently defined extension file attributes; maybe localizable? --> |start=25 |thumbtime=27.5 |end=35 |lossy=1 <!-- extended file attributes; requires key=val pair; not localizable --> |zoom=1.5 |center-x=0.5 |center-y=0.667 |center-z=0.25 |distance=0.75 |loop=1 |muted=1 |autoplay=1 <!-- caption --> |Check out this an awesome [[3d model]] that's [[animation|animated]]! ]]

There is some ambiguity with accepting any unknown key followed by '='... is [[File:Equation.mytype|E=mc^2]] using "E=mc^2" as a caption or passing a parameter "E" with value "mc^2"?

We could add a 'caption=' magic option and use it explicitly:

[[File:Equation.mytype|caption=E=mc^2]]

Or just adding a stub <nowiki> ought to do it as a workaround, since it should break the regex check for the generic key followed by "=":

[[File:Equation.mytype|<nowiki></nowiki>E=mc^2]]

For the media dialog in VE there's a few things we can do... For locally-installed extensions, easy peasy! Extension registers the options it needs along with localized labels and whatever necessaries for validation/input. I'll have to take a look at how it works currently...

For remote files... probably we'd want to have an API method for the remote site to expose a given file's option keys, localized labels, and some metadata about the fields (just like a local extension might supply, but without the ability to inject some JS or a custom widget).

Another possibility is for the remote site to expose a layout option editor "preview" in its own iframe using a postMessage interface to bump data values in/out of the dialog. This might be good for, say, specifying the start/end/thumb times of a video, or the initial zoom/position of a panorama, by scrubbing/dragging/zooming on the actual media instead of manually typing in numbers. (Could use the exact same iframe interface for local handlers to avoid having multiple code paths, or could remove the intermediary frame for that case and have a common JS API for both methods.)

Back to implementation of the viewer: basically we'd take the final file options in normalized key form (width & height and any other regex'd or extended keys) and pass them as query string parameters to an iframe.

So when we look up info about the file itself, 'imageinfo' can return an 'iframe' key with, say, https://commons.wikimedia.org/wiki/File:Curiosity%27s_Seven_Minutes_of_Terro...

we can take that iframe URL from the imageinfo and append our other options, so an invocation like:

[[File:Curiosity's_Seven_Minutes_of_Terror.ogv| 640px|start=3:02|end=3:52|thumbtime=3:14]]

would yield something like

<iframe src="...?embedplayer=yes&width=640&height=360&start=3:02&end=3:52&thumbtime=3:14" width="640" height="360" frameborder="0" allowFullScreen

</iframe>

Now this also brings up questions like did you want to play the video inline vs show a thumbnail that brings up the video in a popup, which may be something where we have to work out in general as well... this would allow using a *standard* popup on the wiki side (say, MultimediaViewer?) to load a remote media item.

If using an iframe to embed something with "plain" thumbnail behavior where click means "link" or "zoom", we have to work out issues like passing click events from the iframe to the parent (or else do some clickjacking ;)

-- brion