Dear SMIL and WikiMedia folks, I wanted to bring this to your attention:
The HTML5-driven Wikimedia "sequencer" - a template-based online video editor - stores the EDL (Edit Decision List) in the SMIL format. There's a recent introductory blog post [1] about it.
You can see an example for yourself by opening this page [2] with Firefox 4 (preferably), by clicking on "Edit Sequence", then by clicking on the "View" button in the upper-left corner, followed by "Sequence SMIL XML" from the popup menu. The SMIL 3.0 source will be revealed. For convenience, I quoted the XML source at the bottom of this email.
This appears to be a custom version of what I would normally qualify as SMIL, at least (as far as I can tell) not something based on the normative Scalability Framework [3], nor based on an existing subset of SMIL (see Modularization and Profiling [4]).
For example, the format overrides the normal SMIL namespace for "img" and "ref" with xmlns="http://www.w3.org/1999/xhtml", although I'm a bit confused as I can't see any XHTML constructs there. There's a combination of XML attribute name/value pairs coming from SMIL a well as from some domain-specific markup (none of which seem to belong to the specified namespace). The format also injects arbitrary parameters into media objects using nested "param" tags (and "name" + "value" pairs), thereby adding another layer of extensibility (why mix native attributes and the param element ?).
I am wondering whether there is scope in this "lab" project to discuss the EDL syntax ? It would be great if the final format was compatible with SMIL user agents (playback engines and production tools). I am sure that simple changes could be made to "clean-up" the namespaces and to define an XML schema that draws the boundaries between SMIL and domain-specific markup.
The statement from the blog "The SMIL spec has evolved over time and has become famous for being very large and complicated to implement in a real world player" may be true for the specification as a whole, but not for a domain-specific subset ;) Just look at SVG or DAISY as prime examples of standards that integrate only the strictly-necessary parts of SMIL.
The claim that "Another nice feature of SMIL xml is that it is extensible for custom components" may be true indeed, but this is not quite achieved with the proposed format, as it doesn't conform to the extensibility guidelines.
I am looking forward to hearing your thoughts. Feel free to correct me if I got something wrong ! :)
I assume the "public-smil" and "wikivideo-l" mailing lists are most appropriate to continue this discussion, but I also CC'ed the more mature WikiMedia technical lists to ensure this message reaches "whom it may concern" :)
[1] http://techblog.wikimedia.org/2010/09/video-labs-kaltura-html5-sequencer-ava...
[2] http://commons.wikimedia.org/w/index.php?title=Sequence:Cats&withJS=Medi...
[3] http://www.w3.org/TR/SMIL3/smil-scalabilityFramework.html
[4] http://www.w3.org/TR/SMIL/smil-modules.html
----- SNIP ----8<---------
<?xml version="1.0" encoding="UTF-8"?> <smil baseProfile="Language" version="3.0" xmlns="http://www.w3.org/ns/SMIL "> <head> <meta name="title" content="Cats"/> <transition xmlns="http://www.w3.org/1999/xhtml" id="REF_5_fadeFromColor" type="fade" dur="0:02" fadeColor="#000000" subtype="fadeFromColor"> </transition> </head> <body id="seq_0"> <par id="par_1"> <seq title="Video Track 1" tracktype="video" id="seq_2"> <ref xmlns="http://www.w3.org/1999/ xhtml" type="application/x-wikitemplate" apititlekey="Template:SequenceTitleBlackBG" apiprovider="commons" dur="0:06" id="REF_5" transIn="REF_5_fadeFromColor"> <param name="Title" value="House Catff" /> <param name="SubTitle" value="English House Cat Article <br> ''Read by DollieLlama'' " /> <param name="Image" value="File:Olhos_de_um_gato-3.jpg" /> </ref> <img xmlns="http://www.w3.org/1999/xhtml" dur="0:03" title="Spielendes Kätzchen" src="http://upload.wikimedia.org/wikipedia/commons/3/33/Spielendes_K%C3%A4tzchen.JPG " poster="http://upload.wikimedia.org/wikipedia/commons/thumb/3/33/Spielendes_K%C3%A4tzchen.JPG/80px-Spielendes_K%C3%A4tzchen.JPG " id="IMG_4" panZoom="0%, 0%, 100%, 100%"> <param name="id" value="3494791" /> <param name="apiTitleKey" value="File:Spielendes_Kätzchen.JPG" /> </img> <img xmlns="http://www.w3.org/1999/xhtml" dur="2" title="Woman with Cat" src="http://upload.wikimedia.org/wikipedia/commons/8/8c/Woman_with_Cat.jpg " poster="http://upload.wikimedia.org/wikipedia/commons/thumb/8/8c/Woman_with_Cat.jpg/80px-Woman_with_Cat.jpg " id="IMG_0" panZoom="6%, -13%, 170%, 127%"> <param name="id" value="1410943" /> <param name="apiTitleKey" value="File:Woman_with_Cat.jpg" /> </img> <img xmlns="http://www.w3.org/1999/xhtml" dur="0:03" title="Cat and mouse" src="http://upload.wikimedia.org/wikipedia/commons/thumb/8/82/Cat_and_mouse.jpg/432px-Cat_and_mouse.jpg " poster="http://upload.wikimedia.org/wikipedia/commons/thumb/8/82/Cat_and_mouse.jpg/80px-Cat_and_mouse.jpg " id="IMG_1" panZoom="0%, -81%, 196%, 147%"> <param name="id" value="6979116" /> <param name="apiTitleKey" value="File:Cat_and_mouse.jpg" /> </img> <img xmlns="http://www.w3.org/1999/xhtml" dur="0:05" title="Ancient Egyptian bronze cat" src="http://upload.wikimedia.org/wikipedia/commons/thumb/2/2f/Ancient_Egyptian_bronze_cat.jpg/449px-Ancient_Egyptian_bronze_cat.jpg " poster="http://upload.wikimedia.org/wikipedia/commons/thumb/2/2f/Ancient_Egyptian_bronze_cat.jpg/80px-Ancient_Egyptian_bronze_cat.jpg " id="IMG_5" panZoom="-1%, -35%, 181%, 135%"> <param name="id" value="3804250" /> <param name="apiTitleKey" value="File:Ancient_Egyptian_bronze_cat.jpg" /> </img> <img xmlns="http://www.w3.org/1999/xhtml" dur="2" title="Catstalkprey" src="http://upload.wikimedia.org/wikipedia/commons/thumb/6/6a/Catstalkprey.jpg/800px-Catstalkprey.jpg " poster="http://upload.wikimedia.org/wikipedia/commons/thumb/6/6a/Catstalkprey.jpg/80px-Catstalkprey.jpg " id="IMG_2" panZoom="-18%, -5%, 116%, 87%"> <param name="id" value="3820113" /> <param name="apiTitleKey" value="File:Catstalkprey.jpg" /> </img> <img xmlns="http://www.w3.org/1999/xhtml" dur="2" title="Cat&Pigeon" src="http://upload.wikimedia.org/wikipedia/commons/6/6d/Cat%26Pigeon.jpg " poster="http://upload.wikimedia.org/wikipedia/commons/thumb/6/6d/Cat%26Pigeon.jpg/80px-Cat%26Pigeon.jpg " id="IMG_6" panZoom="-6%, -10%, 135%, 101%"> <param name="id" value="444674" /> <param name="apiTitleKey" value="File:Cat&Pigeon.jpg" /> </img> <img xmlns="http://www.w3.org/1999/xhtml" dur="0:03" title="Cat1" src="http://upload.wikimedia.org/wikipedia/commons/a/ae/Cat1.jpg " poster="http://upload.wikimedia.org/wikipedia/commons/thumb/a/ae/Cat1.jpg/80px-Cat1.jpg " id="IMG_3" panZoom="-1%, -1%, 100%, 100%"> <param name="id" value="2954217" /> <param name="apiTitleKey" value="File:Cat1.jpg" /> </img> <video xmlns="http://www.w3.org/1999/xhtml" durationhint="6.5" dur="0:04" title="Cat claws" src="http://upload.wikimedia.org/wikipedia/commons/a/a6/Cat_claws.ogg " poster="http://upload.wikimedia.org/wikipedia/commons/thumb/a/a6/Cat_claws.ogg/mid-Cat_claws.ogg.jpg " id="VIDEO_7" clipBegin="0:00"> <param name="id" value="532143" /> <param name="apiTitleKey" value="File:Cat_claws.ogg" /> </video> <ref xmlns="http://www.w3.org/1999/xhtml" type="application/x-wikitemplate" apititlekey="Template:SequenceTitleBlackBG" apiprovider="commons" dur="2" id="REF_0"> <param name="Title" value="end title" /> </ref> </seq> <seq title="Audio track 1" tracktype="audio" id="seq_3"> <audio xmlns="http://www.w3.org/1999/ xhtml" durationhint="1628.9262585034" dur="0:29" title="En-Cat (part 1)" src="http://upload.wikimedia.org/wikipedia/commons/d/d5/En-Cat_%28part_1%29.ogg " id="AUDIO_4" clipBegin="1:05"> <param name="id" value="2755934" /> <param name="apiTitleKey" value="File:En- Cat_(part_1).ogg" /> </audio> </seq> </par> </body> </smil>
----- SNIP ----8<---------
Daniel wrote:
I assume the "public-smil" and "wikivideo-l" mailing lists are most appropriate to continue this discussion, but I also CC'ed the more mature WikiMedia technical lists to ensure this message reaches "whom it may concern" :)
I stand corrected: the "public-smil" mailing list doesn't exist, it is in fact "www-smil@w3.org" (I resent the message there). Additionally, the "mediawiki-l" mailing-list requires registration, so my message wasn't sent there at all. Michael Dale, principal developer, will probably get my message via a comment I made in his blog, just in case I used the wrong list addresses.
Kind regards, Daniel
Hi Daneil,
Thanks for your interest in the project. To be sure there is a good amount of work to be done for compatibility with any SMIL like profile or alternate players. Suggestions and bugs that get us going in that direction are good. But it should be noted that it was not a target design consideration for the initial version, rather the format was initially just 'smil like' and loosely defined by the 'player plays' what the 'sequencer outputs'. This is of course not ideal, but is the fastest way to move forward without spending too much time on 'full spec compliance' at the initial stage of development
As the project envolves it would be ideal to become more smil like so other editors can edit and other players can play and they all work together in a relatively predictable way.
On 10/04/2010 02:14 AM, Daniel Weck wrote:
For example, the format overrides the normal SMIL namespace for "img" and "ref" with xmlns="http://www.w3.org/1999/xhtml", although I'm a bit confused as I can't see any XHTML constructs there.
I believe this is a consiquence of manipulating the XML in jQuery. I never explicitly added the alternate xmlns it just showed up when using the transforms functions. Probably have to expliclty state the SMIL xmlns for jQuery not to assume its xhtml. ( should be easy to fix )
There's a combination of XML attribute name/value pairs coming from SMIL a well as from some domain-specific markup (none of which seem to belong to the specified namespace). The format also injects arbitrary parameters into media objects using nested "param" tags (and "name" + "value" pairs), thereby adding another layer of extensibility (why mix native attributes and the param element ?).
I agree its a bit confusing. In some cases like apiTitleKey 'param' it does not have any direct effect on the presentation of the asset. Its only there to help provide a link-back to the asset page in the sequencer and help the sequencer find the asset it it gets moved in the medaiWiki system. If you have suggestions on how to better represent that info I would be interested in hearing that.
I am wondering whether there is scope in this "lab" project to discuss the EDL syntax ?
Sure :)
It would be great if the final format was compatible with SMIL user agents (playback engines and production tools). I am sure that simple changes could be made to "clean-up" the namespaces and to define an XML schema that draws the boundaries between SMIL and domain-specific markup.
Sounds good. I would be happy to work with you or the SMIL community to figure out how things should be represented. I did make a best effort under give time constraints to try and do things SMIL like for example in the spec you see things like:
<ref src="http://www.example.com/herbert.face"> <param name="mood" value="surly" valuetype="data"/> <param name="accessories" value="baseball-cap,nose-ring" valuetype="data"/> </ref>
In the spec here: http://www.w3.org/TR/SMIL3/smil-extended-media-object.html#edef-param
Which lead to templates being represented how they are.. but I understand its not as clean as it could be and I am open to suggestions on how we can improve it and make it more likely a 3rd party real world player could play the format.
The statement from the blog "The SMIL spec has evolved over time and has become famous for being very large and complicated to implement in a real world player" may be true for the specification as a whole, but not for a domain-specific subset ;) Just look at SVG or DAISY as prime examples of standards that integrate only the strictly-necessary parts of SMIL.
Right, at this initial stage of development its not even really ready for a domain-specific subset specification, just help and guidance to make things as smil like as possible, then eventually we can think about efforts in this area.
The claim that "Another nice feature of SMIL xml is that it is extensible for custom components" may be true indeed, but this is not quite achieved with the proposed format, as it doesn't conform to the extensibility guidelines.
Yea I did not mean to say that I strictly conformed to the extensible guidelines only that the 'format is extensible', and that it would be a good direction for the project to go and I am making efforts in that direction.
I am looking forward to hearing your thoughts. Feel free to correct me if I got something wrong ! :)
I assume the "public-smil" and "wikivideo-l" mailing lists are most appropriate to continue this discussion, but I also CC'ed the more mature WikiMedia technical lists to ensure this message reaches "whom it may concern" :)
wikivideo-l is a fine list for this sort of thing.
--michael
Hi Michael, thank you for your prompt response. (CC'ed to the public SMIL mailing-list, so that the experts can provide feedback too)
Point taken about jQuery generating the "wrong" namespace. Easy to fix indeed.
Globally, I think your approach is very good: SMIL looks like a perfect fit for composing the multimedia presentation resulting from the video editing process.
Using "native" XML attributes on your media objects (ref) instead of param (name/value pairs) would make it easier to validate content based on your domain-specific schema, but in practice it may result in your EDL content being rejected by some existing (and confused) SMIL user agents. I don't know, I've never tried this. :)
I would suggest that at the early stages of development, you make your EDL format valid against the DTD of the SMIL 3.0 Language Profile [1] [2]. You will ultimately adopt only a minute portion of SMIL, probably smaller that the Tiny Profile [3] ! Heck, you might even only need features from SMIL 2.0 ! :)
I would suggest being consistent when injecting your domain-specific features:
* use param name/value XOR * use namespace-prefixed attributes (or even elements)
I suggest ignoring the Scalability Framework for now. Instead, focus on separating pure SMIL constructs versus your specific markup. Once your requirements stabilize (i.e. your extensions to SMIL are well- defined), you may consider ensuring compliance with the SMIL extensibility guidelines.
Some of your domain-specific markup strikes me as metadata rather than instructions to the media object renderer. I suggest you consider adding inline meta [4] to your refs, instead of using param.
There's an open-source project called LimSee3 [5], which deals with SMIL document templates. I'm not sure how useful this can be to you...just thinking aloud here. They have clearly been facing similar issues to yours though.
I hope this helps. Regards, Dan
[1] http://www.w3.org/TR/SMIL3/smil-DTD.html#SMIL30Language
[2] http://www.w3.org/2008/SMIL30/SMIL30Language.dtd
[3] http://www.w3.org/TR/SMIL3/smil-tiny-profile.html
[4] http://www.w3.org/TR/SMIL3/smil-metadata.html
[5] http://limsee3.gforge.inria.fr/public-site/
On 4 Oct 2010, at 21:46, Michael Dale wrote:
Hi Daneil,
Thanks for your interest in the project. To be sure there is a good amount of work to be done for compatibility with any SMIL like profile or alternate players. Suggestions and bugs that get us going in that direction are good. But it should be noted that it was not a target design consideration for the initial version, rather the format was initially just 'smil like' and loosely defined by the 'player plays' what the 'sequencer outputs'. This is of course not ideal, but is the fastest way to move forward without spending too much time on 'full spec compliance' at the initial stage of development
As the project envolves it would be ideal to become more smil like so other editors can edit and other players can play and they all work together in a relatively predictable way.
On 10/04/2010 02:14 AM, Daniel Weck wrote:
For example, the format overrides the normal SMIL namespace for "img" and "ref" with xmlns="http://www.w3.org/1999/xhtml", although I'm a bit confused as I can't see any XHTML constructs there.
I believe this is a consiquence of manipulating the XML in jQuery. I never explicitly added the alternate xmlns it just showed up when using the transforms functions. Probably have to expliclty state the SMIL xmlns for jQuery not to assume its xhtml. ( should be easy to fix )
There's a combination of XML attribute name/value pairs coming from SMIL a well as from some domain-specific markup (none of which seem to belong to the specified namespace). The format also injects arbitrary parameters into media objects using nested "param" tags (and "name" + "value" pairs), thereby adding another layer of extensibility (why mix native attributes and the param element ?).
I agree its a bit confusing. In some cases like apiTitleKey 'param' it does not have any direct effect on the presentation of the asset. Its only there to help provide a link-back to the asset page in the sequencer and help the sequencer find the asset it it gets moved in the medaiWiki system. If you have suggestions on how to better represent that info I would be interested in hearing that.
I am wondering whether there is scope in this "lab" project to discuss the EDL syntax ?
Sure :)
It would be great if the final format was compatible with SMIL user agents (playback engines and production tools). I am sure that simple changes could be made to "clean-up" the namespaces and to define an XML schema that draws the boundaries between SMIL and domain-specific markup.
Sounds good. I would be happy to work with you or the SMIL community to figure out how things should be represented. I did make a best effort under give time constraints to try and do things SMIL like for example in the spec you see things like:
<ref src="http://www.example.com/herbert.face"> <param name="mood" value="surly" valuetype="data"/> <param name="accessories" value="baseball-cap,nose-ring" valuetype="data"/> </ref>
In the spec here: http://www.w3.org/TR/SMIL3/smil-extended-media-object.html#edef-param
Which lead to templates being represented how they are.. but I understand its not as clean as it could be and I am open to suggestions on how we can improve it and make it more likely a 3rd party real world player could play the format.
The statement from the blog "The SMIL spec has evolved over time and has become famous for being very large and complicated to implement in a real world player" may be true for the specification as a whole, but not for a domain-specific subset ;) Just look at SVG or DAISY as prime examples of standards that integrate only the strictly- necessary parts of SMIL.
Right, at this initial stage of development its not even really ready for a domain-specific subset specification, just help and guidance to make things as smil like as possible, then eventually we can think about efforts in this area.
The claim that "Another nice feature of SMIL xml is that it is extensible for custom components" may be true indeed, but this is not quite achieved with the proposed format, as it doesn't conform to the extensibility guidelines.
Yea I did not mean to say that I strictly conformed to the extensible guidelines only that the 'format is extensible', and that it would be a good direction for the project to go and I am making efforts in that direction.
I am looking forward to hearing your thoughts. Feel free to correct me if I got something wrong ! :)
I assume the "public-smil" and "wikivideo-l" mailing lists are most appropriate to continue this discussion, but I also CC'ed the more mature WikiMedia technical lists to ensure this message reaches "whom it may concern" :)
wikivideo-l is a fine list for this sort of thing.
--michael
wikivideo-l@lists.wikimedia.org