One issue I've been having has to do with high level punctuation
getting tangled up in embedded text. In wikitext, it's generally ok to
write a literal ]] - it means two right square brackets in a row. But
of course in an [[image:foo.jpg|caption - ]] means the end of the
image element, not just raw text.
I can see, and have sort of tried, three ways to handle this:
1) Using traditional grammar approaches, backtracking and so forth,
hoping the parser is smart enough to match the right string, and "pull
back" at the right moment. Unfortunately, this seems very difficult
without an extremely good knowledge of the compiler compiler, and is
probably slow to boot.
2) Using bottom up* context flags like "inside image element", so when
an "]]" is found, we know whether or not we can treat them as
literals. Problem: you end up smearing knowledge about the image
element everywhere: why does the RIGHT_SQUARE_BRACKET literal want to
know anything about image elements?
3) Using top down restrictions on literals like "prohibit literal
double right square bracket". Similar to 2), but when a "]]" is found
it just dumbly looks at the corresponding flag to decide whether to
match it as literal.
Method 3 seems the most promising now. I was using 2), but it seemed
to become very complex all of a sudden.
I now have code that looks like this:
image_caption
@init {prohibit_literal_link_end++; prohibit_literal_pipe++;}
: inline_text?
-> ^(TEXT inline_text);
finally {prohibit_literal_link_end--; prohibit_literal_pipe--;}
...
literal_link_end: {prohibit_literal_link_end <= 0}? => link_end;
This seems to be relatively readable too: "An image caption is any
text, except that there can't be an unescaped literal pipe or link_end
(]]) in it." and "A literal link end is whenever you encounter a raw
link_end, unless someone has said you can't."
Seems to keep me a bit saner, too. Anyway, just thought I would share.
Steve
* I'm using the terms 'bottom up' and 'top down' extremely loosely
here.