[Foundation-l] Strategy document?

[Aryeh Gregor]

On Fri, Oct 3, 2008 at 10:56 AM, Thomas Dalton <thomas.dalton at gmail.com> wrote:
> 2008/10/3 David Gerard <dgerard at gmail.com>:
>> 2008/10/3 geni <geniice at gmail.com>:
>>> 2008/10/3 Nikola Smolenski <smolensk at eunet.yu>:
>>>> Birgitte SB wrote:
>>
>>>>> Ability to transcribe musical scores in wikitext.
>>
>>>> Easily doable.
>>
>>> Close. It's been done but with security issues.
>>
>>
>> o_0 Dare I ask?
>
> I wasn't aware that music could be used as a dangerous weapon
> either... surely it's just putting lots of pictures together?

The problem is that the solution people have proposed, LilyPond,
includes a scripting language.  There's a safe mode to disable some of
the more unpleasant features, but apparently that's not safe enough:
it can still easily be DoS'd by infinite loops.  Info on LilyPond's
safe mode:

"--safe does not detect resource overuse. It is still possible to make
the program hang indefinitely, for example by feeding cyclic data
structures into the backend. Therefore, if using LilyPond on a
publicly accessible webserver, the process should be limited in both
CPU and memory usage."
http://lilypond.org/doc/v2.9/Documentation/user/lilypond/Invoking-lilypond

I.e., people are going to write inefficient (even if well-intentioned)
Scheme programs into their music that will collectively cause the
music rendering servers to run like snails, so people will militate to
get more music servers so that trivial music snippets can render in
less than two minutes, then people will write even *more* inefficient
code, then . . .

Basically, anyone who can execute arbitrary code on Wikimedia servers
(even sandboxed) has to be trusted to not just be well-intentioned,
but *good at writing efficient code*.  And that's not an assumption
that can be extended to even a small fraction of sysops, let alone the
average user typing stuff into the edit page.

It's already enough of a headache that people write absurdly
complicated templates, but that's carefully limited.  You'll notice
that there are no looping or recursion constructs in wikitext, plus
there are length limits on the size of the executed code -- so the run
time of any wikitext parsing is bounded, and making sure it's
efficient enough is just a matter of tweaking the bounds or the
implementation, no matter how lousy the code is that people write.  If
it's too slow it will just refuse to run, skipping template inclusions
and similar.

There is a place in the world for programming languages, and a place
for less powerful languages.  Arbitrary typesetting is *arguably* a
place where real programming languages are needed.  But I'm willing to
bet that typesetting music specifically is not, in the overwhelming
majority of cases, or certainly not for our purposes.


All this is similar to the situation with math markup.  LaTeX would be
extremely unsafe to run on arbitrary user input, so we only run a very
limited subset of it, with allowable codes whitelisted.  This uses an
extension that basically parses the input and throws an error if it
doesn't recognize one of the codes used, and only passes it off to
LaTeX if there are no errors.

If someone were to write a similar extension that parsed LilyPond
input and ensured that it contained no programming constructs (flow
control, loops, functions, . . .) of any kind, it could potentially be
enabled, but not if it just passes things on to LilyPond.  Similarly,
if the LilyPond developers would be willing to create a mode that
actually disables all programming constructs and only allows
simple-minded stuff like the example at
<http://meta.wikimedia.org/wiki/Music_markup#Using_Lilypond> (I'm
assuming that the codes there are simple positional things), then we
might be able to use it directly.

But until then, nobody's written any music extension yet that's usable
for us.  It could certainly be done, and if it's a high priority then
Wikimedia could certainly pay someone to do it, but it's not *quite*
as trivial as you might think.  Although I don't expect it would be
terribly difficult either, if someone who understood the requirements
put in the time.