Hello Conrad,

...

2. Implement an AMS-TeX validator

How different would this be from the current validator?

It should be exactly the same, except written in Python.

...

5. Repackage the entire Math thing as an extension

I might do this if I have time left at the end. I'm sure the project will change over the summer.

This would be very amazing.

Maybe this should be my project, then.

...

Python doesn't have parsing just locked right down the way C does with flex/bison, but there are some good options, I have the most experience with it, and I think I'd be able to complete the port faster in Python than in either of the other languages. I was tempted at first to port to PHP, to conform with the rest of Mediawiki, but there don't seem to be any good parsing packages for PHP. (Please tell me if that's wrong.)

A good PHP parser library would be exceptionally useful for MediaWiki (and many extensions), at the moment we have loads of methods that do regex "parsing", so if you felt like writing one... :D.

Actually...

I've never used PHP for real programming, but how difficult would it be to write a really simple, stupid first pass at a DFA parser? I suspect I'd need much more than three months to make it useful, but would it be possible to implement some coherent subset of the features? E.g., building the LR0 automaton, at least?

...

I'd appreciate any advice or criticism. Since my only previous experience has been using Wikipedia and setting up a test Mediawiki instance for my ACM chapter, I'm only just now learning my way around the code base and it's not always evident why things were done as they are. Does this look like a reasonable and worthwhile project?

Step 5. has been a "we really should do this" for a while, the shipping of OCaml code which many users won't be able to use is very messy. I am less convinced of the utility of a Python port, OCaml is a great language for implementing this, and I fear a lot of your time would be wasted trying to make the Python similarly nice. As you note, MediaWiki is not written in Python, doing this in PHP would be a larger step in the right direction, though without such nice frameworks, maybe less nice to do.

I suggested a Python port because http://www.mediawiki.org/wiki/Summer_of_Code_2010#MediaWiki_core lists it as a potential project idea. I was under the impression that people around here did not want to leave texvc in OCaml. Is this wrong?

Yours, Damon Wang

2010/3/23 Aryeh Gregor Simetrical+wikilist@gmail.com:

...

I've never used PHP for real programming, but how difficult would it be to write a really simple, stupid first pass at a DFA parser? I suspect I'd need much more than three months to make it useful, but would it be possible to implement some coherent subset of the features? E.g., building the LR0 automaton, at least?

I don't think you'd need a "real" parser here.  Mostly we just use preg_split() for this sort of thing.  I'm not familiar with formal grammars and such, so I can't say what the concrete disadvantages of that approach are.

DFAs parse regular languages, which means those languages can also be expressed as regexes. In fact, the regexes accepted by the preg_*() functions allow certain extensions to the language theory definition of regular expressions, allowing them to describe certain non-regular languages as well. In short: preg_split() can do everything a DFA can do, and more. The only reason to use a DFA parser would be performance, but since the preg_*() functions are so heavily optimized I don't think that'll be an issue.

...

I suggested a Python port because    http://www.mediawiki.org/wiki/Summer_of_Code_2010#MediaWiki_core lists it as a potential project idea. I was under the impression that people around here did not want to leave texvc in OCaml. Is this wrong?

No, it's right.  Conrad is crazy.  :P

Having it in a language no one understands is a bad thing and leads to maintenance not happening, so yeah, we definitely want it rewritten in PHP. If the PHP implementation turns out to be too slow to run on WMF, for instance, we could do a C++ port à la wikidiff2 (a C++ port of our ludicrously slow PHP diff implementation).

Roan Kattouw (Catrope)

On Tue, Mar 23, 2010 at 1:00 PM, Roan Kattouw roan.kattouw@gmail.com wrote:

DFAs parse regular languages, which means those languages can also be expressed as regexes. In fact, the regexes accepted by the preg_*() functions allow certain extensions to the language theory definition of regular expressions, allowing them to describe certain non-regular languages as well. In short: preg_split() can do everything a DFA can do, and more. The only reason to use a DFA parser would be performance, but since the preg_*() functions are so heavily optimized I don't think that'll be an issue.

This much I know, but is LaTeX actually a regular language?

On Tue, Mar 23, 2010 at 1:13 PM, Conrad Irwin conrad.irwin@googlemail.com wrote:

And here was me thinking that maintenance didn't happen because making changes to security critical sections of the code is dangerous :).

It's not security-critical. The worst you could possibly do is DoS, and any DoS could be instantly shut off by just turning off math briefly. Furthermore, the part that makes DoS impossible is a quite small portion of the code that would need to change effectively never. No, the problem is that most PHP programmers have never even heard of OCaml, let alone used it.

2010/3/23 Roan Kattouw roan.kattouw@gmail.com:

2010/3/23 Aryeh Gregor Simetrical+wikilist@gmail.com:

...

This much I know, but is LaTeX actually a regular language?

I don't know; I was just making the point that writing a DFA parser in PHP is probably not very useful.

Sorry, I got confused and wrote DFA when I should have written LALR. DFAs cannot parse even the allowed subset of AMS-LaTeX, because there are some permitted environments.

Without claiming to know much formal language theory, a rule of thumb is that languages with matched delimiters were never regular, because of the pumping lemma: http://en.wikipedia.org/wiki/Pumping_lemma_for_regular_languages

So, for example, it's theoretically impossible to check that parentheses nested correctly using regular expressions, and similarly it'd be impossible to check that the \begin and \end commands matched up.

In practice there might be ways to hack around that by using multiple regular expressions and manually tracking how they nest, but at that point we're basically writing half of a bad LALR parser.

Fortunately, though, Python has parser generators! And if we're really concerned about speed, there's PyBison, which does the parsing in C and apparently produces (at least) five-fold improvements over Python-native alternatives.

Yours, Damon Wang

Conrad Irwin wrote:

On 03/23/2010 05:23 PM, Aryeh Gregor wrote:

...

On Tue, Mar 23, 2010 at 1:00 PM, Roan Kattouw roan.kattouw@gmail.com wrote:

...

DFAs parse regular languages, which means those languages can also be expressed as regexes. In fact, the regexes accepted by the preg_*() functions allow certain extensions to the language theory definition of regular expressions, allowing them to describe certain non-regular languages as well. In short: preg_split() can do everything a DFA can do, and more. The only reason to use a DFA parser would be performance, but since the preg_*() functions are so heavily optimized I don't think that'll be an issue.

This much I know, but is LaTeX actually a regular language?

It's not even context free, luckily the subset we are interested in is (as clearly shown by the texvc parser :p).

Just because a language is context-sensitive doesn't mean it will be hard to write a parser for it. That's just a myth propagated by computer scientists who, strangely enough given their profession, have a disdain for the algorithm as a descriptive framework.

In the last few decades, pure mathematicians have been exploring the power of algorithms as a general description of an axiomatic system. And simultaneously, computer scientists have embraced the idea that the best way to process text is by trying to shoehorn all computer languages into some Chomsky-inspired representation, regardless of how awkward that representation is, or how inefficient the resulting algorithm becomes, when compared to an algorithm constructed a priori.

-- Tim Starling

On Wed, Mar 24, 2010 at 9:16 AM, Trevor Parscal tparscal@wikimedia.org wrote:

I think we should really consider LOLCODE for this sort of thing.

http://en.wikipedia.org/wiki/Lolcode

It's just more fun!

• Trevor

Also rewrite parser functions to use it? that would be interesting on en.wiki since they are always complaining about the syntax.

jks

-Peacvhey

On 03/23/2010 10:44 PM, Tim Starling wrote:

Just because a language is context-sensitive doesn't mean it will be hard to write a parser for it. That's just a myth propagated by computer scientists who, strangely enough given their profession, have a disdain for the algorithm as a descriptive framework.

Context free grammars have a strong advantage because they come with documentation built in. Given a BNF-esque description of a language, it is possible to understand, at a high level, how the language works. This means it's easy to write a parser (though much easier to get the parser written automatically), it's also more pleasant to verify properties of the language (no state to keep track of), and to reason about the consequences of modifications. Of course it is possible to give decent documentation for a context-sensitive language; from what I've seen, this just doesn't happen.

Take for example Python and perl, or Markdown and MediaWiki. In both cases the former has a syntax that can be mainly modeled by a context free grammar and there are many implementations that all work. The latter of the pair has a context sensitive grammar defined only by the reference implementation, there are no other parsers with feature completeness. This is certainly not a technical limitation, rather a reflection on the ability of your average human programmer.

Conrad

On Tue, Mar 23, 2010 at 6:28 PM, Conrad Irwin conrad.irwin@googlemail.com wrote:

Many LaTeX installations can be made read/write/execute anything by default.

What does that mean? LaTeX can invoke external programs? Using what commands? Is this functionality actually enabled in practice in stock LaTeX installs?

Even if it were not possible to break out of restricted write18, there will exist installations with write18 enabled, and I can't imagine people remembering to check. Depending on the flavour of LaTeX in use it

If people won't remember, surely either the MediaWiki installer or the extension itself can be made to check this and simply refuse to enable <math> tags.

Aryeh Gregor wrote:

As long as the worst that could happen on a large majority of installations is DoS, I don't think we should be afraid to rewrite the code just because *maybe* it would be less secure. We should obviously check over the new code carefully, but I wouldn't say it's any more security-critical than random pieces of MediaWiki -- which are typically vulnerable to XSS if someone forgets to escape something.

Getting shell access is not a DoS or XSS. Specially for a large majority of installs where it compromises their only account. Does this mean that we shouldn't rewrite it? No. We should rewrite it, and make it more secure. We start it by having enough eyes on the code. I wouldn't be surprised if we found a vulnerability on texvc during the rewrite.

Running the LaTeX interpreter under ulimit -u 1 should be provide a quite safe default against external launches. But take into account that file writes are also a dangerous vector.

Hello Rob,

Just to be really clear, I'm not looking for a "right" answer on any of those questions.  It's not necessary for you to be even interested in getting deeply involved in the Wikipedia user community to have a really successful project.  The purpose of this line of questions is to figure out if we should continue helping you refine your current idea, or suggest some other direction that's a bigger payoff and/or easier sell.

I understand, and that'd be very helpful. To be honest, I'm not passionately committed to any project at all. I've been writing projects for university and for a computer lab I work at, but it's mostly small, one-off sysadmin things and usually the emphasis is more on "xyz server has to be back up before we open tomorrow" than writing good, clean code. So, yes, I'd welcome other suggestions.

As I'm sure you've already gathered from the other responses, this is exactly the right place.  I'm a little skeptical myself that porting that particular piece of code from OCaml to Python is going to be a really big win for us (because it's still a "foreign" language as far as PHP-based MediaWiki is concerned, so integration is still a little clunky and performance may take a hit due to yet another interpreter needing to load), but I'll let others weigh in on whether I'm making too big a deal about that.

There are ways to make this run faster if performance is a concern. For example, mod_python or mod_wscgi, or explicitly pulling the Python out into a standalone daemon that listens for requests from the webserver.

Another possibility be writing it in C to avoid all interpreter overhead, and using a foreign function interface. Unfortunately, I'm not familiar with PHP's FFI. Google takes me to http://wiki.php.net/rfc/php_native_interface which seems to think that as of a year ago there weren't any good ones, but this doesn't look too painful: http://theserverpages.com/php/manual/en/zend.creating.php

Stepping back from the specifics of your proposal (which I think the others on this list have responded to pretty well), I'd like to find out more about what general sorts of projects interest you the most, which may help us figure out if we should keep going in this direction.  Some questions:

1. Are you most interested in having a Python-based project, or would you

be *equally* happy and productive programming something in PHP?

I'm most familiar with Python and C, for whatever that's worth coming from an undergrad who didn't know Python existed five years ago. I learned PHP to maintain the web interfaces of an in-house print system at work, but I haven't used it for anything as involved as what we're discussing here. So, in terms of productivity, yes, if I have to work in PHP my mentor will probably get asked a few more newbie questions.

In terms of happiness, though, it'd be a great opportunity to dig into PHP and finally learn to use it as more than really smart CSS with a database connection. Although I prefer Python or even C because I think I'd be more useful, I wouldn't be very upset at all if it turned out you guys were willing to let me learn PHP on your time.

2. Are you zeroing in on <math> parsing and parsing in general because

that's an area that you're already developing expertise in and/or are deeply interested in getting into, or is that just something that looked kinda interesting to learn about relative to other opportunities you considered?

I like the <math> parsing project because it seems well-suited for a third-year undergrad who knows LaTeX and reads a few other functional languages and has studied lex/yacc before in his coursework. The goals are clear, and I know how to break them down into smaller problems and how to tackle each one. It's a little isolated from the rest of Mediawiki, so I don't need to grok the entire code base.

Basically, this looks like a way to make a concrete contribution despite being a newcomer to the project. That doesn't mean I'm not happy to entertain alternatives, just that they have a pretty high bar to clear.

3. Are you coming at this as someone who is already deep into

Wikipedia/MediaWiki usage who is looking to resolve particular things (like <math> parsing) that are painful as an end user, or are you more casually involved and more interested in applying in this project because it looks like we've got a lot of interesting programming problems to solve?

The second. I just want to tackle a problem that's near but not quite beyond my limits, and if I can help out a site I use daily, so much the better.

Yours, Damon Wang

There's a few Python-based things that might be interesting, but I think you'll get a lot more love for doing something in PHP or C.  Since this is a student internship, you shouldn't be bashful about using this as a learning opportunity.

I'd only caution against convincing yourself (and us) that you'll be more interested in learning something like PHP than you truly are.  It might help you land a spot, but it will work against you in having a successful project, and

this has such high visibility that you'll really want to be successful.

What visibility does this have? I thought it was some abandoned corner of the wiki that nobody has touched in the seven years since it was first written. What happens if I make a hash of this?

So, if you find yourself thinking about doing this in PHP and having your inner voice say "meh", then I'd recommend sticking to your guns and propose doing this or something else in Python and/or C.

Well, now my inner voice says, "I really don't want to make a hash of this texvc port!", so let me explain why I want to do it in Python rather than PHP. I agree that the performance will probably be just fine, and that it would be a great coup for maintainability and installation and usage. The problem is, I don't think PHP has a parser-generator package.

So let me make sure I understand the problem here. You already have a texvc implementation that has worked just fine for the last seven years. TeX is pretty stable at this point, so chances are good you'd make it another seven years without problems. But you're still dissatisfied because OCaml is a hard language to find programmers for, and the existing implementation isn't really maintained. You want it ported to a different language that has more programmers available.

(You also as a Mediawiki extension rather than a core feature; I'm going to do that, but I won't say anything more because it seems fairly uncontroversial.)

Since the subset of TeX you need parsed has a context-free grammar, it needs an LALR parser, not just a bunch of regexes. I know three ways to get an LALR parser:

(1) write a pushdown automaton manually (i.e., be yacc) (2) write input for a parser-generator (3) write a parser-generator, and give it input

Option (2) is the most maintainable and feasible option, and it's precisely the one that cannot be done in PHP. As far as I know, PHP has no parser-generator package. (Please, please let me know if that's incorrect so I can stop embarrassing myself and get on with writing a GSoC proposal.)

I could probably do (1), or some hackish kludge at half of it, by throwing custom control structures into a bucketload of regexes, but I don't think that's in the project's best interests. As has been pointed out, the OCaml implementation is really concise and elegant. A large fraction of that concision and elegance comes from not actually being a parser but rather only a context-free grammar written in a BNF-like syntax common to most parser-generators.

I think it'd be easier to find a programmer who has worked with a parser-generator and can learn a little bit of OCaml, than it would be to find a PHP programmer who has to read himself into a manually implemented parser. After all, how many PHP programmers do you know who have experience mucking around inside an LALR parser?

So that's why, while I'm happy to take it on in PHP as a learning experience for myself, I think it'd be better for Mediawiki to port texvc to Python. That gets us the larger pool of potential maintainers that comes with using a commonly known language, without sacrificing the amazing advantage of only needing to maintain a grammar rather than the parser itself.

And as far as dependencies are concerned, Python is still a much easier dependency to satisfy, both for programmers working with the code and for sysadmins installing it.

What do you guys think?

Also, would anyone be interested in mentoring this project?

Yours, Damon Wang

On Fri, Mar 26, 2010 at 10:48 PM, Damon Wang damonwang@uchicago.edu wrote:

(You also as a Mediawiki extension rather than a core feature; I'm going to do that, but I won't say anything more because it seems fairly uncontroversial.)

I actually disagree with this pretty strongly. It would be a regression in functionality for existing users -- if they upgrade, their wiki breaks unless they install a new extension. There's no reason to remove it from core that I see that outweighs this disadvantage.

Since the subset of TeX you need parsed has a context-free grammar, it needs an LALR parser, not just a bunch of regexes. I know three ways to get an LALR parser:

(1) write a pushdown automaton manually (i.e., be yacc)    (2) write input for a parser-generator    (3) write a parser-generator, and give it input

Option (2) is the most maintainable and feasible option, and it's precisely the one that cannot be done in PHP. As far as I know, PHP has no parser-generator package. (Please, please let me know if that's incorrect so I can stop embarrassing myself and get on with writing a GSoC proposal.)

I could probably do (1), or some hackish kludge at half of it, by throwing custom control structures into a bucketload of regexes, but I don't think that's in the project's best interests. As has been pointed out, the OCaml implementation is really concise and elegant. A large fraction of that concision and elegance comes from not actually being a parser but rather only a context-free grammar written in a BNF-like syntax common to most parser-generators.

Okay, well, maybe you're right. I'd be interested to hear Tim Starling's opinion on this (using parser generators vs. writing by hand). Writing it in Python would certainly be a big step forward from OCaml -- any site with LaTeX accessible to MediaWiki will almost certainly have Python available, so Python vs. PHP should make no difference to end-users. And Python is probably the second-best-known language among MediaWiki hackers.

I think it'd be easier to find a programmer who has worked with a parser-generator and can learn a little bit of OCaml, than it would be to find a PHP programmer who has to read himself into a manually implemented parser. After all, how many PHP programmers do you know who have experience mucking around inside an LALR parser?

The parsing part is unlikely to need much maintenance. There are other things currently in OCaml that make more sense to modify from time to time -- like the whitelist of commands, and (some of?) the code for non-image output formats. So for instance, MathML output is theoretically supported, but I don't know how good the support is. That might become more important in the future, since Firefox is likely to support inline MathML in text/html not too long from now. This sort of thing would be harder if it were Python rather than PHP.

I don't think it would be a big deal if it were rewritten entirely in Python, though. It would be a big step forward in any case, and if it's easier for you, great. So personally I'd be okay with it, although it's perhaps not ideal.

Also, would anyone be interested in mentoring this project?

I probably wouldn't be of any help for this particular project, since I don't know anything about parsers, and my Python and TeX are passable but not great. We could probably come up with a mentor, though.

"Aryeh Gregor" Simetrical+wikilist@gmail.com wrote in message news:7c2a12e21003281059i551c4650p8a8e51e100b62479@mail.gmail.com...

On Fri, Mar 26, 2010 at 10:48 PM, Damon Wang damonwang@uchicago.edu wrote:

...

(You also as a Mediawiki extension rather than a core feature; I'm going to do that, but I won't say anything more because it seems fairly uncontroversial.)

I actually disagree with this pretty strongly. It would be a regression in functionality for existing users -- if they upgrade, their wiki breaks unless they install a new extension. There's no reason to remove it from core that I see that outweighs this disadvantage.

As opposed to their wiki breaking when they upgrade for all the other reasons that we document in the release notes? I have never built a wiki where texvc has been needed, wanted, or even thought harmless. Currently MW users have to compile and configure a binary from a language 99.99% of them cannot understand, and enable the functionality using config variables. Asking them instead to download and install an extension like every other non-ubiquitous feature in MediaWiki is far from being a regression.

--HM

If you make sure to run update.php, it's very rare for your wiki to break, unless you've hacked things or not updated your extensions or such.  We're usually pretty careful to avoid significant regressions when upgrading wikis that are using supported/sane configurations.

Can we make update.php ask the user if he wants to install the new extension?

It's a regression for people who already have math working.  What's the advantage?  We have an awful lot of marginal features in core. When have we ever split a feature that we'd released in core into an extension, when a significant number of people were using it?  I don't see the point.  It's not like we're going to significantly reduce the size of the tarball or anything.

Is there any place we could get usage statistics for the math feature? I think the advantages for new installations justify inconveniencing some existing users, especially if we can automate installation of the new extension, but this discussion would be better with some data.

Yours, Damon Wang

On Mon, Mar 29, 2010 at 12:12 PM, Aryeh Gregor Simetrical+wikilist@gmail.com wrote:

On Sun, Mar 28, 2010 at 11:45 PM, Damon Wang damonwang@uchicago.edu wrote:

...

Can we make update.php ask the user if he wants to install the new extension?

That would be hacky and unreliable.  We'd have to make sure the versions match, automatically alter LocalSettings.php (!), hope that the wiki files are writable to the web server (they probably aren't), hope that it's not on a firewalled intranet, . . . also, update.php doesn't require user interaction, and changing that would break everything.

...

Is there any place we could get usage statistics for the math feature?

No, we don't have this kind of tracking in place.  People would probably object if we did.

...

I think the advantages for new installations justify inconveniencing some existing users, especially if we can automate installation of the new extension, but this discussion would be better with some data.

I'm not so much worried about math specifically as about what would happen if we started systematically moving relatively-unused things from core to extensions.  Few people use math, but a whole lot of people probably use at least one little-used feature that could be moved to an extension.  We generally haven't moved things from core to extensions AFAIK -- if we started doing it, it could cumulatively have repercussions on ease of upgrade, for no real benefit that I see.

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

What if it was written as an extension and moved to /extensions? Then we get the benefit of decoupling Math from the core software, but we don't require users to download a new extension to keep their existing functionality. As long as it's clearly indicated in the RELEASE-NOTES and relevant Manual pages that you might need to update the path to Math, I don't see a huge drawback.

-Chad

Aryeh Gregor wrote:

On Mon, Mar 29, 2010 at 12:46 PM, Chad wrote:

...

What if it was written as an extension and moved to /extensions? Then we get the benefit of decoupling Math from the core software,

What benefit is this? It's not realistically decoupled from the core software unless it avoids using MediaWiki functions and classes. We have some code in core that deliberately does this, like IEContentAnalyzer.php and all of includes/normal/. Making something an extension per se doesn't change how tightly it's coupled with MediaWiki -- it's an orthogonal issue.

The only reasons I see to have extensions in trunk at all, instead of having everything in core, is that 1) it helps keep us honest about ensuring there are extension points for third parties to use, and 2) otherwise the tarball would be huge. Neither point argues for breaking things already in core out to extensions.

<math> was implemented directly in the parser a really long time ago. That's the reason it has always been in core, despite it not being available for 99% users. Only recently, I freed the 'math' from the parser (r57997), so it can be used by another tag hook to provide the same functionality (I gave up on math and wanted to use [[Extension:Mimetex_alternative]]) and then Tim moved it out creating CoreTagHooks (r61913). It logically is an extension, much more than eg. ParserFunctions. I bet there are more users of ParserFunctions than of math. Changing to python will also break for people that compiled math, update without reading the release notes and don't have python. We can as well move it to a separate extension instead of embedding it, although that wouldn't be much an issue. The only way to ensure it will work for everyone would be to provide a PHP implementation, in which case thousands of installs will get a toolbar button suddenly working.

* Bryan Tong Minh bryan.tongminh@gmail.com [Tue, 30 Mar 2010 17:22:09 +0200]:

While this is ofcourse possible, how big is the chance that somebody will have ocaml but not python?

Fedora Linux has ocaml for ages. yum install ocaml or something like that. Compiling texvc is fast and easy - never had any problems. Since ocaml was developed in France, chances are bigger that it has wider spread over there. Dmitriy

On 29 March 2010 01:12, Aryeh Gregor Simetrical+wikilist@gmail.com wrote:

...

I have never built a wiki where texvc has been needed, wanted, or even thought harmless.

Granted, this is not as widely used as some other optional features. There are certainly many wikis that do use it, though -- it's not like no one will be affected.

I'd *like* to use it, but it's such an arse I've never yet got it working ... perhaps it's just me.

- d.

On 23 March 2010 19:17, Rob Lanphier robla@robla.net wrote:

As I'm sure you've already gathered from the other responses, this is exactly the right place.  I'm a little skeptical myself that porting that particular piece of code from OCaml to Python is going to be a really big win for us (because it's still a "foreign" language as far as PHP-based MediaWiki is concerned, so integration is still a little clunky and performance may take a hit due to yet another interpreter needing to load), but I'll let others weigh in on whether I'm making too big a deal about that.

Getting it off Ocaml is an excellent first step. I have tried and failed to get texvc working properly in MediaWiki myself more than a few times, because of Ocaml not wanting to play nice ...

- d.

On Tue, Mar 30, 2010 at 14:34, Victor bobbie@ua.fm wrote:

Actually I completely disagree. Since I've got some experience with both OCaml and PHP the idea to convert Maths processing to PHP looks like a not so good idea at all.

Probably the issues you had were more like a wrong/problematic configuration or something like that. OCaml itself is actively developed and is a mature language and development environment, much better than PHP or Python (IMHO).

It is just interesting to wait a bit and compare the PHP and OCaml implementations of texvc (if there will be anything at all to compare).

c is "better", so is Common Lisp, Scheme, Haskell, Clojure or a number of other languages.

The problem is that worse is better. OCaml isn't widely known among programmers or as easy for PHP programmers to get into as say Perl, Python or Ruby. As a result the math/ directory has been untouched (aside from the stray doc+bug fix) since 2003.

There are many long standing core issues with the texvc component in Bugzilla (http://bit.ly/bsSUPM) that noone is looking at.

I don't think anyone would have a problem with it remaining in OCaml if it was being maintained and these long-standing bugs were being fixed.

Ævar Arnfjörð Bjarmason wrote:

On Wed, Mar 31, 2010 at 10:58, Ævar Arnfjörð Bjarmason avarab@gmail.com wrote:

...

c is "better"

That should have been "OCaml".

No it shouldn't! :)

On 03/31/2010 12:31 PM, Victor wrote:

Hi, now I see...

I've posted a message to the fa.caml newsgroup: http://groups.google.com/group/fa.caml/browse_frm/thread/1593e053759d7679

hopefully somebody will volunteer to fix the issues, thus saving the human resources for better tasks.

With best regards, Victor

While I'll no doubt regret saying this, I am happy to fix some of these bugs. With the majority of these it's harder to decide "should we fix" and "how should we fix" rather than actually being hard to implement.

What I don't want to do is fix things to find that it then gets immediately re-implemented in PHP, which seems to be what people want.

Some of the issues have LaTeX dependencies, particularly "support Unicode", so fixing them could be dangerous unless we conditionally include support for them, i.e. by running a feature-test as part of the installation (we should probably do that anyway, so that we can provide nicer error messages to the user if they are missing one of the other dependencies).

Conrad

On 30 March 2010 16:34, Victor bobbie@ua.fm wrote: ....

...

Getting it off Ocaml is an excellent first step. I have tried and failed to get texvc working properly in MediaWiki myself more than a few times, because of Ocaml not wanting to play nice ...

Actually I completely disagree. Since I've got some experience with both OCaml and PHP the idea to convert Maths processing to PHP looks like a not so good idea at all.

Doing Math in any programming language or digital computer is a bad idea. Anyway.

I could be worse, It could be Math in Javascript:

v = (011 + "1" + 0.1)/3;

303.3666666666667

I could be worse, It could be Math in Javascript: v = (011 + "1" + 0.1)/3; 303.3666666666667

Somebody ought to slap you for mixing four different types in such a horrendous manner to construct something that is supposed to make people who do not understand octal numbers and string concatenation to think JavaScript is insane. *shakes head*

([8, 32, 189].sort()[0]) === (new Boolean(false) ? 189 : 8)

Why such a contrived example?! This all boils down to

new Boolean(false) == false returning true

new Boolean(false) === false returning false

The ?: operator is just not performing an implicit cast. The fact that the Boolean object is different from the Boolean primitive type is... ...well unfortunate. That's why it's use is as an Object is deprecated since JavaScript 1.3 You example gives the expected outcome if you call Boolean as a function: ([8, 32, 189].sort()[0]) === (Boolean(false) ? 189 : 8)

So, can we drop this dreaded debate now?

On 30 March 2010 15:34, Victor bobbie@ua.fm wrote:

On Tue, 30 Mar 2010 16:05:02 +0300, David Gerard dgerard@gmail.com wrote:

...

Getting it off Ocaml is an excellent first step. I have tried and failed to get texvc working properly in MediaWiki myself more than a few times, because of Ocaml not wanting to play nice ...

Actually I completely disagree. Since I've got some experience with both OCaml and PHP the idea to convert Maths processing to PHP looks like a not so good idea at all. Probably the issues you had were more like a wrong/problematic configuration or something like that. OCaml itself is actively developed and is a mature language and development environment, much better than PHP or Python (IMHO).

Oh, I don't doubt that at all. It just didn't work for me :-)

- d.

"Platonides" Platonides@gmail.com wrote in message news:hobfpi$4ud$1@dough.gmane.org...

You seem to be thinking about creating a PHP extension. I don't think you should go that route. A binary is good enough, we don't need it to be in a PHP extension. That glue could be added later if needed, but would increase the complexity to write and debug.

I took it to mean that he wanted to split the math parsing out as a **MediaWiki** extension, implementing <math> as a parser tag hook in the usual way. Which is definitely highly desirable.

--HM