You may have already seen the FVorbis implementation of Vorbis in Flash 10: http://people.xiph.org/~arek/pg/hx/test.html
This is a more integrated demo of the same approach: http://www.omtk.org/js/demo-mod.html
The FVorbis implementation uses lower CPU on my system, but in any case, this definitely seems like a viable approach to bring at least Vorbis to many more people - especially when combined with native support in Firefox 3.1 and the existing Java player.
On Wed, Nov 19, 2008 at 12:30 AM, Erik Moeller erik@wikimedia.org wrote:
You may have already seen the FVorbis implementation of Vorbis in Flash 10: http://people.xiph.org/~arek/pg/hx/test.html
This is a more integrated demo of the same approach: http://www.omtk.org/js/demo-mod.html
The FVorbis implementation uses lower CPU on my system, but in any case, this definitely seems like a viable approach to bring at least Vorbis to many more people - especially when combined with native support in Firefox 3.1 and the existing Java player.
Along these lines participants may be interested in Adobe Alchemy: http://labs.adobe.com/wiki/index.php/Alchemy:Libraries
Alchemy is a C -> AS3 bytecode compiler. It allows developers to take arbitrary C code and compile it so that it runs in the Flash 10 virtual machine. Adobe is distributing a couple examples, including the Vorbis library and some encryption code (presumably useful for creating Flash media DRM).
The Vorbis in Flash (10) examples that Erik linked to above were created by reimplementing Vorbis decoding in other programming languages specifically targeted at flash. While that approach is likely to produce higher performance results it takes a large amount of developer resources to create and maintain. With Alchemy Flash can make use of the existing reference code.
This also means that a port of other codecs such as Theora, Speex, and Flac should be more straight forward. It should also make things like a flash DJVU browser much easier to create.
A similar c->bytecode process is also possible for Java, through a project called NestedVM (http://nestedvm.ibex.org/), although I'm not aware of any codecs using it yet as it's a bit difficult to interface NestedVM software with regular Java code.
The performance of these VM solutions will never be as good as native support, if for no other reason than the software will have to be transferred to every user. But more compatibility and more options are always good.
2008/11/20 Gregory Maxwell gmaxwell@gmail.com:
The Vorbis in Flash (10) examples that Erik linked to above were created by reimplementing Vorbis decoding in other programming languages specifically targeted at flash. While that approach is likely to produce higher performance results it takes a large amount of developer resources to create and maintain. With Alchemy Flash can make use of the existing reference code.
Both are definitely worth looking into. I think performance should be a key criterion here - we want something that can play on low end systems at reasonable load levels. I've reached out to the developers of the aforementioned implementations to see if they're interested in helping towards getting this to run on Wikimedia; if you see developers working on related projects for Vorbis & Theora, please let me know.
2008/11/20 Erik Moeller erik@wikimedia.org:
2008/11/20 Gregory Maxwell gmaxwell@gmail.com:
The Vorbis in Flash (10) examples that Erik linked to above were created by reimplementing Vorbis decoding in other programming languages specifically targeted at flash. While that approach is likely to produce higher performance results it takes a large amount of developer resources to create and maintain. With Alchemy Flash can make use of the existing reference code.
Both are definitely worth looking into. I think performance should be a key criterion here - we want something that can play on low end systems at reasonable load levels. I've reached out to the developers of the aforementioned implementations to see if they're interested in helping towards getting this to run on Wikimedia; if you see developers working on related projects for Vorbis & Theora, please let me know.
If a .swf served by us works in Gnash even though we know 99.9% of users will in fact be using it in Adobe Flash, is that free software enough for us, morally as well?
- d.
If our images can be viewed under Linux even though we know 90% of users will in fact be viewing them with Windows, nobody has had any moral issues that I know of. Free software/knowledge/images are free, because it is allowed to use them virtually everywhere (and not like big software companies restricting it to work only with some special player software). Why should we force people to use free software?
Regards,
ChrisiPK
2008/11/24 David Gerard dgerard@gmail.com:
2008/11/20 Erik Moeller erik@wikimedia.org:
2008/11/20 Gregory Maxwell gmaxwell@gmail.com:
The Vorbis in Flash (10) examples that Erik linked to above were created by reimplementing Vorbis decoding in other programming languages specifically targeted at flash. While that approach is likely to produce higher performance results it takes a large amount of developer resources to create and maintain. With Alchemy Flash can make use of the existing reference code.
Both are definitely worth looking into. I think performance should be a key criterion here - we want something that can play on low end systems at reasonable load levels. I've reached out to the developers of the aforementioned implementations to see if they're interested in helping towards getting this to run on Wikimedia; if you see developers working on related projects for Vorbis & Theora, please let me know.
If a .swf served by us works in Gnash even though we know 99.9% of users will in fact be using it in Adobe Flash, is that free software enough for us, morally as well?
- d.
Commons-l mailing list Commons-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/commons-l
2008/11/24 ChrisiPK chrisipk@gmail.com:
If our images can be viewed under Linux even though we know 90% of users will in fact be viewing them with Windows, nobody has had any moral issues that I know of. Free software/knowledge/images are free, because it is allowed to use them virtually everywhere (and not like big software companies restricting it to work only with some special player software). Why should we force people to use free software?
Regards,
ChrisiPK
Because the software needed to host flash videos significantly risks costing us money.
On Mon, Nov 24, 2008 at 3:10 PM, ChrisiPK chrisipk@gmail.com wrote:
If our images can be viewed under Linux even though we know 90% of users will in fact be viewing them with Windows, nobody has had any moral issues that I know of. Free software/knowledge/images are free, because it is allowed to use them virtually everywhere (and not like big software companies restricting it to work only with some special player software). Why should we force people to use free software?
We shouldn't, obviously. I don't think anyone here has argued that we should.
When the issue came up on foundation-l a little while back, I stated that I thought a flash file that used only the publicly documented subset of flash (or at least works in Gnash) and doesn't use the patented-encumbered codecs would be okay with the board's proposed file format policy.
So I think David's open ended question is unanswerable. We need to know "For what?" and "How?".
This is the historical high level four fold set of issues with flash:
(1) Flash is executable software. We do not allow executable *content* uploads for a multitude of reasons which are not specific to flash. (Security concerns, the impossibility of validating what non-trivial executable code really does, etc) But trusted executable things might still be useful as wikimedia-provided tools. (we do this with Java for Video today)
(2) The Flash format has (historically) been secret and there has been no freely licensed implementation. Adobe has released some specs although many things are still not disclosed. GNASH is able to run some older flash code. Only a subset of flash can be run on free software systems.
(3) Flash video and audio have been some of the most frequently desired features, but these have *required* the use of patent covered formats. Users can not legally play these formats without licensed decoders, and more importantly other publishers can not use them without paying royalties. We could pay up the licensing offer users a choice of formats, but other publishers wouldn't have the same freedoms. GNASH does nothing to help this problem.
(4) No free software flash authoring tools, if anyone who wants to create or edit flash would need to purchase the expensive editing tools.
Currently these issues have the following answers:
(1) Isn't relevant for things like uploader widgets, player widgets, or shims to give IE equivalents to more modern HTML features.
(2) Is still somewhat problematic. Adobe is still churning out undocumented OPcodes in flash, but if you demonstrate that something operates in Gnash thats probably pretty good.
(3) Is now potentially avoidable because Flash 10 is a powerful enough programming language to send clients decoders written for the flash vm for non-patented formats. All publishers have this option. Using the non-free formats would still be unacceptable since it would be taking a liberty that downstream republishers would lack. There now exists a drop in flash based HTML5 <audio/> replacement for Vorbis decoding. No one has done video yet, though it's clearly possible now. Performance for video accomplished in this manner is still an open question.
(4) Is a problem for many things but for things like uploader widgets, players, and compatibility shims they would most likely be created with haXe (A free software language for targeting the flash virtual machine http://haxe.org/), and while author-ability/editability is a deal-breaker for *content*, it's likely less important for non-content website machinery. Limiting the developer pool is a serious practical issue, but haXe addresses that.
2008/11/24 Gregory Maxwell gmaxwell@gmail.com:
When the issue came up on foundation-l a little while back, I stated that I thought a flash file that used only the publicly documented subset of flash (or at least works in Gnash) and doesn't use the patented-encumbered codecs would be okay with the board's proposed file format policy.
I'm not sure I understand what you're saying - are you saying that if we shipped open Flash code to decode Vorbis that doesn't work on open clients, that would be objectionable? Even if the Vorbis player didn't work in Gnash or it relied on proprietary voodoo in Flash 10, I'm not sure I can see that there is some fundamental problem with sending open code to client-side black box implementations that makes open standards work well on those clients. It doesn't strike me as fundamentally different from supporting, say, proprietary SVG implementations whe available on the client side, or sending code to Internet Explorer to render our web pages correctly. It doesn't exclude anyone or even disincentivize use of open standards.
On Mon, Nov 24, 2008 at 4:31 PM, Erik Moeller erik@wikimedia.org wrote:
2008/11/24 Gregory Maxwell gmaxwell@gmail.com:
When the issue came up on foundation-l a little while back, I stated that I thought a flash file that used only the publicly documented subset of flash (or at least works in Gnash) and doesn't use the patented-encumbered codecs would be okay with the board's proposed file format policy.
I'm not sure I understand what you're saying - are you saying that if we shipped open Flash code to decode Vorbis that doesn't work on open clients, that would be objectionable?
No. We're in agreement. For the reasons you stated.
I'll forward you the longer off-list reply I wrote to David.
2008/11/24 Gregory Maxwell gmaxwell@gmail.com:
I'll forward you the longer off-list reply I wrote to David.
I only just looked up and realised you hadn't sent it to the list :-) You should, it's the first I'd heard of Haxe.
- d.
On Mon, Nov 24, 2008 at 5:06 PM, David Gerard dgerard@gmail.com wrote:
2008/11/24 Gregory Maxwell gmaxwell@gmail.com:
I'll forward you the longer off-list reply I wrote to David.
I only just looked up and realised you hadn't sent it to the list :-) You should, it's the first I'd heard of Haxe.
Meh. You're thwarting my resolution to not be personally responsible for the majority of any mailing list's volume. I'm trying to use off-list replies unless I see a fair amount of discussion on the list. Oh well. There is some redundancy with the shorter public message I posted today after chrisipk's response.
The idea basically being that Flash is probably fine now for particular limited uses, that we should try hard to avoid it for some others, and that we ought not to use it at all for others still, depending on a whole bunch of details.
On Sun, Nov 23, 2008 at 8:05 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Sun, Nov 23, 2008 at 7:12 PM, David Gerard dgerard@gmail.com wrote:
If a .swf served by us works in Gnash even though we know 99.9% of users will in fact be using it in Adobe Flash, is that free software enough for us, morally as well?
"Works in gnash" is insufficient in the general case.
For *content* on the sites it must also be authorable with free tools. You can now create some subsets of flash with tools like haxe, but flash is not generally editable without the fairly expensive Adobe tools.
Gnash is also often combined with non-free codecs, Gnash + non-free bits is an obviously insufficient test.
Specific to this discussion:
These decoders will not work in gnash, nor do I believe there is any near term prospect. Gnash is fairly far behind flash, and Alchemy uses many totally undocumented VM opcodes.
But I do not think it's all that relevant to this discussion because the use of Flash would only be as a fall back for other free technology (<audio/> and <video/> in this case), and it's a fall back that *all* publishers could use (so long as the SWF itself were freely licensed). Worth mentioning is that roughly 1/3 of our traffic is already firefox.
For the moral test you need to consider two groups: (1) Users and (2) publishers. We ought not distribute files that requires users to use non-free software, and we ought not to distribute files that other publishers couldn't distribute without software or patent licensing.
Flash with a free software SWF player as a fallback to HTML5 <video/>/<audio/> that plays Vorbis/Theora passes both those tests. Users can use Firefox 3.1 (or Java, or a future opera version, or Safari+XiphQT, or...) and not use any problematic software, and other publishers can do exactly what we do without any licensing. *Or at least this will be true once these things exist* We're not there yet, it's just that it's clearly possible now.
This wasn't true prior to Flash ~10: Prior versions of flash required media be available in non-free formats (MP3/AAC/H264/etc), so even if we offered an Ogg version, we'd fail the publisher moral-test because other publishers would not have the option of the Flash fallback.
Other places where I think flash would be okay:
- Freely licensed SWF upload widget (so long as we have an equal or
better Java alternative, or a HTML5 feature)
- Freely licensed SWF replacement for the HTML <canvas> tag, so long
as we have equal (or better) support for the real canvas tag.
- Freely licensed SWF interactive graphing widget, so long as we work
equally well using <canvas> and JS.
The idea being that flash is generally okay if it does not create an imposition on users and other publishers could do the same things.
A harder question:
*Flash that was created in HAXE (free authoring) and works in GNASH (without non-free addons), but has no other free equivalent.
I think I'd like to say that such a beast would not be strictly impermissible but actually validating that it didn't require non-free tools to create, and that it could work in GNASH are likely to be a big pain. And that it's would be unfortunate to promote Flash while gnash is such a useless alternative. (Flash ain't done till Gnash won't run, dontcha know?)
2008/11/25 Gregory Maxwell gmaxwell@gmail.com:
The idea basically being that Flash is probably fine now for particular limited uses, that we should try hard to avoid it for some others, and that we ought not to use it at all for others still, depending on a whole bunch of details.
Of course :-) Flash is smelly and evil in various ways, and it's also on over 90% of Internet-connected personal computers.
- d.