Today I took a stab at combining ogv.js JavaScript-based media playback with MobileFrontend by adding a loader shim to TimedMediaHandler on the 'mobile' target (the rest of TimedMediaHandler's desktop support code is not loaded, so the UI is mobile-specific but very bare-bones).
Demo page can now play back audio and video in iOS 7's Safari in mobile mode as well as desktop mode: * https://ogvjs-testing.wmflabs.org/
The TMH+ogv.js patch in progress: * https://gerrit.wikimedia.org/r/#/c/145756/
The mobile loader code checks for any <audio> or <video> elements and asks them if they canPlayType() on any of their available sources, so it only loads if non-native playback is actually required. (So for instance it disengages on Android Chrome which can play Ogg Vorbis audio and WebM video, or in theory in Safari if you have locally enabled MP4/H.264 files.)
It needs more work to check for browser compatibility, sufficient JavaScript engine speed, etc, but I find it encouraging that it works so far. :)
Some thoughts and questions:
* Currently ogv.js gets loaded if any audio/video elements are present that require it to play, even if they don't get played. I can delay the loading to when 'play' is clicked fairly easily.
* [[Media:]] or other direct file links, often used for pronunciation markers in Wikipedia articles, are not picked up by this system. Need to extend things a bit to detect clicks on such links and display a player instead of just downloading the file. Same problem occurs in Safari and IE on desktop mode.
* Should we show the video in an overlay like the mobile media viewer for images, instead of playing inline? This is a good place to add additional controls to reach file info details, as with images. If so should I try to extend the same overlay code in MF or create a near-lookalike that lives in TMH?
* If so, that should probably be used for *all* mobile browsers, using the native playback when available instead of loading ogv.js...
* Should we have a manual resolution switcher, as on desktop? (Controlling source selection via code could also fix a problem with Android being unable to play Ogg Theora videos, to force it to WebM which does work natively.)
* On iOS, should the source selector offer to launch higher-res and WebM videos in the external VLC app? 360p is about the limit of good performance in ogv.js on current A7-based iOS devices, and slower models max out at 160p if they can even handle that.
-- brion
This is great progress. Thanks for the update Brion.
Anyone up for helping Brion out with this project?
--tomasz
On Sat, Jul 26, 2014 at 1:29 PM, Brion Vibber bvibber@wikimedia.org wrote:
Today I took a stab at combining ogv.js JavaScript-based media playback with MobileFrontend by adding a loader shim to TimedMediaHandler on the 'mobile' target (the rest of TimedMediaHandler's desktop support code is not loaded, so the UI is mobile-specific but very bare-bones).
Demo page can now play back audio and video in iOS 7's Safari in mobile mode as well as desktop mode:
The TMH+ogv.js patch in progress:
The mobile loader code checks for any <audio> or <video> elements and asks them if they canPlayType() on any of their available sources, so it only loads if non-native playback is actually required. (So for instance it disengages on Android Chrome which can play Ogg Vorbis audio and WebM video, or in theory in Safari if you have locally enabled MP4/H.264 files.)
It needs more work to check for browser compatibility, sufficient JavaScript engine speed, etc, but I find it encouraging that it works so far. :)
Some thoughts and questions:
- Currently ogv.js gets loaded if any audio/video elements are present that
require it to play, even if they don't get played. I can delay the loading to when 'play' is clicked fairly easily.
- [[Media:]] or other direct file links, often used for pronunciation
markers in Wikipedia articles, are not picked up by this system. Need to extend things a bit to detect clicks on such links and display a player instead of just downloading the file. Same problem occurs in Safari and IE on desktop mode.
- Should we show the video in an overlay like the mobile media viewer for
images, instead of playing inline? This is a good place to add additional controls to reach file info details, as with images. If so should I try to extend the same overlay code in MF or create a near-lookalike that lives in TMH?
- If so, that should probably be used for *all* mobile browsers, using the
native playback when available instead of loading ogv.js...
- Should we have a manual resolution switcher, as on desktop? (Controlling
source selection via code could also fix a problem with Android being unable to play Ogg Theora videos, to force it to WebM which does work natively.)
- On iOS, should the source selector offer to launch higher-res and WebM
videos in the external VLC app? 360p is about the limit of good performance in ogv.js on current A7-based iOS devices, and slower models max out at 160p if they can even handle that.
-- brion
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
[responded inline]
On 7/26/14, Brion Vibber bvibber@wikimedia.org wrote:
Today I took a stab at combining ogv.js JavaScript-based media playback with MobileFrontend by adding a loader shim to TimedMediaHandler on the 'mobile' target (the rest of TimedMediaHandler's desktop support code is not loaded, so the UI is mobile-specific but very bare-bones).
Demo page can now play back audio and video in iOS 7's Safari in mobile mode as well as desktop mode:
The TMH+ogv.js patch in progress:
The mobile loader code checks for any <audio> or <video> elements and asks them if they canPlayType() on any of their available sources, so it only loads if non-native playback is actually required. (So for instance it disengages on Android Chrome which can play Ogg Vorbis audio and WebM video, or in theory in Safari if you have locally enabled MP4/H.264 files.)
It needs more work to check for browser compatibility, sufficient JavaScript engine speed, etc, but I find it encouraging that it works so far. :)
Some thoughts and questions:
- Currently ogv.js gets loaded if any audio/video elements are present that
require it to play, even if they don't get played. I can delay the loading to when 'play' is clicked fairly easily.
- [[Media:]] or other direct file links, often used for pronunciation
markers in Wikipedia articles, are not picked up by this system. Need to extend things a bit to detect clicks on such links and display a player instead of just downloading the file. Same problem occurs in Safari and IE on desktop mode.
Perhaps we should just tell users to stop using "Media:" links. Media is supposed to be a direct link to a file. Linking to the image description page would seem more appropriate in such a case (Ideally with autoplay, although allowing users/potential vandals to autoplay files seems like a giant bag of worms...).
To throw out a crazy idea, perhaps we should have a new parameter for TMH files, something like [[File:Foo.ogg|displaylink|Some text here]], which outputs "Some text here" as a link, and then launches the player using js magic when somebody clicks on the link.
- Should we show the video in an overlay like the mobile media viewer for
images, instead of playing inline? This is a good place to add additional controls to reach file info details, as with images. If so should I try to extend the same overlay code in MF or create a near-lookalike that lives in TMH?
There's an overlay viewer in MF? I don't think that's ever popped up on my phone.
Currently we have an overlay dialog on desktop for videos where the thumb is < 800px wide. Personally I think on desktop we should reduce that to something like <400px wide, as for a big thumb I think most people expect inline playing unless the thumb is really tiny (and the desktop overlay thing isn't actually that pretty imo). However on mobile it may make sense to use a more full (perhaps almost "full-screen") overlay. But I think either approach is fine for mobile.
- If so, that should probably be used for *all* mobile browsers, using the
native playback when available instead of loading ogv.js...
Consistency is nice. However, at this stage I'd settle for working video everywhere :)
- Should we have a manual resolution switcher, as on desktop? (Controlling
source selection via code could also fix a problem with Android being unable to play Ogg Theora videos, to force it to WebM which does work natively.)
Doesn't android automatically play the first <source> it knows how to when left to its own devices? Thus skipping the ogg sources?
- On iOS, should the source selector offer to launch higher-res and WebM
videos in the external VLC app? 360p is about the limit of good performance in ogv.js on current A7-based iOS devices, and slower models max out at 160p if they can even handle that.
That seems like the sort of thing that would be a nice feature, but of minor importance.
This is all very awesome. Thanks for working on this.
--bawolff
As far as I remember, one of the primary reasons the Media links are used is for "inline" audio fragments: https://en.wikipedia.org/wiki/Template:Audio
If you properly look at that use case, it comes down to the fact that there is not TMH UI that tools to this use case, So I agree with Brian on this, solve the UI problem and you will likely solve the usage of direct media links in content pages.
DJ
On Wed, Jul 30, 2014 at 5:08 AM, Brian Wolff bawolff@gmail.com wrote:
[responded inline]
On 7/26/14, Brion Vibber bvibber@wikimedia.org wrote:
Today I took a stab at combining ogv.js JavaScript-based media playback with MobileFrontend by adding a loader shim to TimedMediaHandler on the 'mobile' target (the rest of TimedMediaHandler's desktop support code is not loaded, so the UI is mobile-specific but very bare-bones).
Demo page can now play back audio and video in iOS 7's Safari in mobile mode as well as desktop mode:
The TMH+ogv.js patch in progress:
The mobile loader code checks for any <audio> or <video> elements and asks them if they canPlayType() on any of their available sources, so it only loads if non-native playback is actually required. (So for instance it disengages on Android Chrome which can play Ogg Vorbis audio and WebM video, or in theory in Safari if you have locally enabled MP4/H.264 files.)
It needs more work to check for browser compatibility, sufficient JavaScript engine speed, etc, but I find it encouraging that it works so far. :)
Some thoughts and questions:
- Currently ogv.js gets loaded if any audio/video elements are present that
require it to play, even if they don't get played. I can delay the loading to when 'play' is clicked fairly easily.
- [[Media:]] or other direct file links, often used for pronunciation
markers in Wikipedia articles, are not picked up by this system. Need to extend things a bit to detect clicks on such links and display a player instead of just downloading the file. Same problem occurs in Safari and IE on desktop mode.
Perhaps we should just tell users to stop using "Media:" links. Media is supposed to be a direct link to a file. Linking to the image description page would seem more appropriate in such a case (Ideally with autoplay, although allowing users/potential vandals to autoplay files seems like a giant bag of worms...).
To throw out a crazy idea, perhaps we should have a new parameter for TMH files, something like [[File:Foo.ogg|displaylink|Some text here]], which outputs "Some text here" as a link, and then launches the player using js magic when somebody clicks on the link.
- Should we show the video in an overlay like the mobile media viewer for
images, instead of playing inline? This is a good place to add additional controls to reach file info details, as with images. If so should I try to extend the same overlay code in MF or create a near-lookalike that lives in TMH?
There's an overlay viewer in MF? I don't think that's ever popped up on my phone.
Currently we have an overlay dialog on desktop for videos where the thumb is < 800px wide. Personally I think on desktop we should reduce that to something like <400px wide, as for a big thumb I think most people expect inline playing unless the thumb is really tiny (and the desktop overlay thing isn't actually that pretty imo). However on mobile it may make sense to use a more full (perhaps almost "full-screen") overlay. But I think either approach is fine for mobile.
- If so, that should probably be used for *all* mobile browsers, using the
native playback when available instead of loading ogv.js...
Consistency is nice. However, at this stage I'd settle for working video everywhere :)
- Should we have a manual resolution switcher, as on desktop? (Controlling
source selection via code could also fix a problem with Android being unable to play Ogg Theora videos, to force it to WebM which does work natively.)
Doesn't android automatically play the first <source> it knows how to when left to its own devices? Thus skipping the ogg sources?
- On iOS, should the source selector offer to launch higher-res and WebM
videos in the external VLC app? 360p is about the limit of good performance in ogv.js on current A7-based iOS devices, and slower models max out at 160p if they can even handle that.
That seems like the sort of thing that would be a nice feature, but of minor importance.
This is all very awesome. Thanks for working on this.
--bawolff
Multimedia mailing list Multimedia@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/multimedia
On Tuesday, July 29, 2014, Brian Wolff bawolff@gmail.com wrote:
- [[Media:]] or other direct file links, often used for pronunciation
markers in Wikipedia articles, are not picked up by this system. Need to extend things a bit to detect clicks on such links and display a player instead of just downloading the file. Same problem occurs in Safari and
IE
on desktop mode.
Perhaps we should just tell users to stop using "Media:" links. Media is supposed to be a direct link to a file. Linking to the image description page would seem more appropriate in such a case (Ideally with autoplay, although allowing users/potential vandals to autoplay files seems like a giant bag of worms...).
Auto play is scary indeed. :) there are also technical limitations on iOS that prevent auto play for audio; you have to initialize audio context in response to a user input event (like how the default popup blockers prevent creation of new windows).
To throw out a crazy idea, perhaps we should have a new parameter for TMH files, something like [[File:Foo.ogg|displaylink|Some text here]], which outputs "Some text here" as a link, and then launches the player using js magic when somebody clicks on the link.
I quite like that idea -- might be good as a general parameter that brings up MMV or TMH player or links to the file: page as appropriate...
- Should we show the video in an overlay like the mobile media viewer for
images, instead of playing inline? This is a good place to add additional controls to reach file info details, as with images. If so should I try
to
extend the same overlay code in MF or create a near-lookalike that lives
in
TMH?
There's an overlay viewer in MF? I don't think that's ever popped up on my phone.
It may have recently graduated from beta mode; I know it's been getting a ui overhaul recently to feel more consistent with desktop MMV.
I've managed to clone the photo overlay and it looks pretty cool on a phone -- and works for both native and ogv.js playback. But on a tablet view I'm not sure if that's the way to go. Could also consider inline playback with a full screen button that opens the overlay and moves the player into it.
Anyway these are all fungible details that we can work out over time, I'll concentrate on the playback itself for now.
- Should we have a manual resolution switcher, as on desktop?
(Controlling
source selection via code could also fix a problem with Android being unable to play Ogg Theora videos, to force it to WebM which does work natively.)
Doesn't android automatically play the first <source> it knows how to when left to its own devices? Thus skipping the ogg sources?
It looks like I had a broken webm transcode on my test instance; Android is happier playing the webm on my public instance. Yay for consistent behavior! :) I might have to adjust the ram on my vagrant instance...
- On iOS, should the source selector offer to launch higher-res and WebM
videos in the external VLC app? 360p is about the limit of good
performance
in ogv.js on current A7-based iOS devices, and slower models max out at 160p if they can even handle that.
That seems like the sort of thing that would be a nice feature, but of minor importance.
*nod* I might try hacking it up in London if I have free time. Some folks like their HD. :)
This is all very awesome. Thanks for working on this.
It's been a really fun research project, but I'm super happy that it's going to be actually useful!
-- brion
multimedia@lists.wikimedia.org