Hello,
As shaihulub told me on irc, it seems the parser using tokenizer.php have been disabled. It looks like we are now using an old parser instead.
Lot of the recent bug reports on sourceforge are related to this swicth: lot of features and bugfixing are no more implemented / fixed :-(
It seems there is a performance issue on the tokenizer, so should we :
1/ get ride of the tokenizer (current situation) and fix bug / reimplement features
2/ improve the tokenizer performances
;)
On Sun, 2004-05-30 at 22:26 +0200, Ashar Voultoiz wrote:
Hello,
As shaihulub told me on irc, it seems the parser using tokenizer.php have been disabled. It looks like we are now using an old parser instead.
Lot of the recent bug reports on sourceforge are related to this swicth: lot of features and bugfixing are no more implemented / fixed :-(
It seems there is a performance issue on the tokenizer, so should we :
1/ get ride of the tokenizer (current situation) and fix bug / reimplement features
2/ improve the tokenizer performances
I think there's no debate that 2) is the way to go, however this will most likely involve writing the parser in c (possibly as php extension)- not very likely to happen in the next days.
Gabriel Wicke
I think there's no debate that 2) is the way to go, however this will most likely involve writing the parser in c (possibly as php extension)- not very likely to happen in the next days.
Speaking of which, Gabriel: has there been any serious deliberation on C?
I'm rewriting the lexer/parser for WikiTeX, and it may be an opportune time to design a system from the ground up; robust enough to take into account, for instance, arbitrary tags and local languages.
Best, Peter
On Tue, 2004-06-01 at 16:29 +0200, Peter Danenberg wrote:
I think there's no debate that 2) is the way to go, however this will most likely involve writing the parser in c (possibly as php extension)- not very likely to happen in the next days.
Speaking of which, Gabriel: has there been any seriousdeliberation on C?
I'm rewriting the lexer/parser for WikiTeX, and it maybe an opportune time to design a system from the ground up; robust enough to take into account, for instance, arbitrary tags and local languages.
One idea is to make the parser available as a php component and lobby to get it included in future php versions- the tokenized parser we used until recently can serve as a fallback for small (= not performance-critical) installations. I think Taw wrote something along these lines already (in ocaml), Magnus has something in C++ iirc and JeLuF already experimented with Taw's parser as a php module. So there are some bits already there to play with.
Gabriel Wicke
wikitech-l@lists.wikimedia.org