Hi guys,
We presented our new Media Viewer tool today at Wikimedia’s monthly metrics meeting, to show off the latest version and discuss next steps.
Here’s a video of our demo, now on YouTube:
http://ur1.ca/gry4u
Here are the slides:
http://ur1.ca/gry2u
Better yet, we invite you to try out the latest version here on Commons:
http://ur1.ca/gry57
This demo page now features 2013 pictures of the year, and these wonderful images really come alive in the Media Viewer.
It looks like our new multimedia browser is coming together pretty nicely, thanks to all your good feedback :)
Let us know what you think of the latest version on this discussion page:
https://www.mediawiki.org/wiki/Talk:Multimedia/About_Media_Viewer
Enjoy …
Fabrice
_______________________________
Fabrice Florin
Product Manager, Multimedia
Wikimedia Foundation
https://www.mediawiki.org/wiki/User:Fabrice_Florin_(WMF)
On Sun, Feb 23, 2014 at 6:43 AM, Brion Vibber <bvibber(a)wikimedia.org> wrote:
> Just an update on this weekend project, see the current demo in your
> browser[1] or watch a video of Theora video playing on an iPhone 5s![2]
>
> [1] https://brionv.com/misc/ogv.js/demo/
> [2] http://www.youtube.com/watch?v=U_qSfHPhGcA
>
> * Got some fixes and testing from one of the old Cortado maintainers --
> thanks Maik!
> * Audio/video sync is still flaky, but everything pretty much decodes and
> plays properly now.
> * IE 10/11 work, using a Flash shim for audio.
> * OS X Safari 6.1+ works, including native audio.
> * iOS 7 Safari works, including native audio.
>
Spent some time tonight on audio/video sync; it's now working both with
native Web Audio API and with the Flash audio shim.
Check out a video with someone talking, and marvel at the synchronization!
https://brionv.com/misc/ogv.js/demo/#file=Sneak_Preview_-_Wikipedia_VisualE…
(Hi Trevor!)
I also snuck in a size selection override, so you can pick the larger 480p
or original files (note that webm originals don't play as only a Theora
decoder is included so far), or down to the tiny 160p, to see the quality
and CPU usage differences.
If there's no objection to use of a Flash shim, I'll keep doing some
experiments in that direction and see if I can rig up a fully-Flash version
that works in old versions of Internet Explorer as well. (The main ogv.js
needs no Flash as long as needed APIs are available, and even runs on iOS
though it's a bit slow for videos.)
-- brion
>
> Audio-only files run great on iOS 7 devices. The 160p video transcodes we
> experimentally enabled recently run *great* on a shiny 64-bit iPhone 5s,
> but are still slightly too slow on older models.
>
>
> The Flash audio shim for IE is a very simple ActionScript3 program which
> accepts audio samples from the host page and outputs them -- no proprietary
> or patented codecs are in use. It builds to a .swf with the open-source
> Apache Flex SDK, so no proprietary software is needed to create or update
> it.
>
> I'm also doing some preliminary research on a fully Flash version, using
> the Crossbridge compiler[3] for the C codec libraries. Assuming it performs
> about as well as the JS does on modern browsers, this should give us a
> fallback for old versions of IE to supplement or replace the Cortado Java
> player... Before I go too far down that rabbit hole though I'd like to get
> peoples' opinions on using Flash fallbacks to serve browsers with open
> formats.
>
> As long as the scripts are open source and we're building them with an
> open source toolchain, and the entire purpose is to be a shim for missing
> browser feature support, does anyone have an objection?
>
> [3] https://github.com/adobe-flash/crossbridge
>
> -- brion
>
>
> On Mon, Oct 7, 2013 at 9:01 AM, Brion Vibber <bvibber(a)wikimedia.org>wrote:
>
>> TL;DR SUMMARY: check out this short, silent, black & white video:
>> https://brionv.com/misc/ogv.js/demo/ -- anybody interested in a side
>> project on in-browser audio/video decoding fallback?
>>
>>
>> One of my pet peeves is that we don't have audio/video playback on many
>> systems, including default Windows and Mac desktops and non-Android mobile
>> devices, which don't ship with Theora or WebM video decoding.
>>
>> The technically simplest way to handle this is to transcode videos into
>> H.264 (.mp4 files) which is well supported by the troublesome browsers.
>> Unfortunately there are concerns about the patent licensing, which has held
>> us up from deploying any H.264 output options though all the software is
>> ready to go...
>>
>> While I still hope we'll get that resolved eventually, there is an
>> alternative -- client-side software decoding.
>>
>>
>> We have used the 'Cortado <http://www.theora.org/cortado/>' Java applet
>> to do fallback software decoding in the browser for a few years, but Java
>> applets are aggressively being deprecated on today's web:
>>
>> * no Java applets at all on major mobile browsers
>> * Java usually requires a manual install on desktop
>> * Java applets disabled by default for security on major desktop browsers
>>
>> Luckily, JavaScript engines have gotten *really fast* in the last few
>> years, and performance is getting well in line with what Java applets can
>> do.
>>
>>
>> As an experiment, I've built Xiph's ogg, vorbis, and theora C libraries
>> cross-compiled to JavaScript using emscripten<https://github.com/kripken/emscripten>and written a wrapper that decodes Theora video from an .ogv stream and
>> draws the frames into a <canvas> element:
>>
>> * demo: https://brionv.com/misc/ogv.js/demo/
>> * code: https://github.com/brion/ogv.js
>> * blog & some details:
>> https://brionv.com/log/2013/10/06/ogv-js-proof-of-concept/
>>
>> It's just a proof of concept -- the colorspace conversion is incomplete
>> so it's grayscale, there's no audio or proper framerate sync, and it
>> doesn't really stream data properly. But I'm pleased it works so far!
>> (Currently it breaks in IE, but I think I can fix that at least for 10/11,
>> possibly for 9. Probably not for 6/7/8.)
>>
>> Performance on iOS devices isn't great, but is better with lower
>> resolution files :) On desktop it's screaming fast for moderate
>> resolutions, and could probably supplement or replace Cortado with further
>> development.
>>
>> Is anyone interested in helping out or picking up the project to move it
>> towards proper playback? If not, it'll be one of my weekend "fun" projects
>> I occasionally tinker with off the clock. :)
>>
>> -- brion
>>
>
>
Oops, that didn't go to the multimedia list.
Juliusz Gonera wrote:
> Gergo Tisza wrote:
>
>> * jquery.color - what for? can we use CSS transitions?
>> * jquery.colorUtil - what for? can we use CSS transitions?
>>
>>
>> Dependencies for doing color transitions with jQuery.animate().
>> Probably can be replaced with CSS; I'll look into that when I have
>> some free time. At any rate, .animate() still runs without those
>> files (just doesn't produce any visible effect) so they can be omitted.
>
> Thanks! In the worst case we could make them a soft dependency, but
> CSS-based transitions would be the best way to go for mobile.
>
>> * moment
>>
>>
>> Used for date localization. Moment is heavy (25K minified) but the
>> browser's built in date formatting is very unreliable.
>> Might be possible doing this on the server side, although it would be
>> ugly from an API design point of view.
>
> Where is it used? The only place I could spot it was "Created on
> [date]." I'm not sure if it pays off to serve 25KB more to a mobile
> user to format one date. Could this, theoretically, be made a soft
> dependency too?
>
>> Probably more preloading? We do a lot of that. You can set
>> mw.mmv.mediaViewer.preloadDistance = 0 to disable. (This only works
>> once you have opened some image already as mw.mmv.mediaViewer is
>> created when the main JS lib is loaded; we don't have a lazy-loading
>> compatible way to configure the viewer yet. Created #275
>> <https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/275> about
>> that.)
>>
>> Also, we request every image twice to get certain performance
>> metrics; this should not result in the image actually being
>> downloaded twice (although we did not test that in mobile browsers),
>> but might confuse the network stats in Chrome. Probably also
>> something that we should make configurable :-)
>
> Yep, that's the major cause of increased size. On my vagrant instance,
> for some reason, those second requests return HTTP 200 and it seems
> that they are fetched from the server again.
>
> Another thing is that I counted 6 API requests. 3 of them are for
> imageinfo, 3 for other APIs. They are small, but as we know each
> request has an HTTP overhead. It would be great to combine them.
>
>> The big win in terms of size would be to make the UI modular and
>> configurable, so you can request a MultimediaViewer instance which
>> does not have most of the information on the metadata panel, for
>> example, and the related files would not be loaded. We have some
>> vague plans about that, but nothing specific yet.
>
> Yes, that's definitely something we should look into when we start
> working on MMV on mobile.
>
> --
> Juliusz
Hi,
As a part of a mobile spike [1], I evaluated the possibility of using
MMV on mobile instead of the custom viewer we have right now in beta.
The good news is that it works. The bad news is that it's quite heavy
compared to what we have right now. We don't know yet when we'd like to
see MMV on mobile, but this moment will eventually come.
Let's start with the additional dependencies I needed to add to mobile
to make it run:
* jquery.fullscreen - not needed on mobile
* jquery.throttle-debounce - make sure that events fire only once, needed?
* jquery.ui.dialog and dependencies - file reuse dialog, can/will be
replaced? (on Multimedia roadmap)
'jquery.ui.core',
'jquery.ui.widget',
'jquery.ui.button',
'jquery.ui.draggable',
'jquery.ui.mouse',
'jquery.ui.position',
'jquery.ui.resizable',
* jquery.color - what for? can we use CSS transitions?
* jquery.colorUtil - what for? can we use CSS transitions?
* moment
Fortunately, Mark told me that removing jquery.ui.dialog is already
planned. I created a simple page with a thumbnail and checked what
Chrome reports in Network tab for no multimedia viewer, the viewer we
currently have on mobile (in beta) and MMV:
* no viewer: 379KB
* mobile viewer:
** 384KB at page load (all JS + LESS + HTML template)
** 390KB when lightbox shows (images used in LESS + API request)
** 561KB when image is loaded (custom size, in this case 171KB for 746px)
** +6KB, +11KB, +182KB
* mmv:
** 395KB at page load (only bootstrap JS loaded)
** 722KB when lightbox shows (the rest of JS and other assets)
** 2.7MB when image is loaded (129KB for 640px and 901KB for 1920px)
** +14KB, +343KB, +2.3MB
** mmv (without jquery.ui.dialog)
** 395KB at page load
** 573KB when lightbox shows
** 2.6MB when image is loaded
** +14KB, +194KB, +2.2MB
Remaining questions/problems:
* There's no indication of overlay loading (on Multimedia roadmap).
* Can we do animations using CSS transitions?
* Why do we load both 640px and 1920px version of images?
* What is loaded at the end apart from images (+2.3MB, images are ~1MB)?
Related patches:
https://gerrit.wikimedia.org/r/#/c/116155/https://gerrit.wikimedia.org/r/#/c/116154/https://gerrit.wikimedia.org/r/#/c/116156/
[1] https://wikimedia.mingle.thoughtworks.com/projects/mobile/cards/1688
--
Juliusz