I've written up an account of how the current parser treats apostrophes here:
http://www.mediawiki.org/wiki/Markup_spec/BNF/Inline_text#Determining_the_b…
All I've done is read the code of doAllQuotes() and translate it from
a procedural style (first replace blah, then iterate through...) into
a more declarative style (four apostrophes end up getting rendered as
X if the following is the case...).
The most interesting case is this one:
Take ''''four''' apostrophes and then throw
'''''five unclosed
apostrophes at them.
Normally, four apostrophes is treated as apostrophe followed by bold.
But when the parser finds unbalanced bold *and* italics on the line,
it goes looking for a bold to split. The first bold, which is now
preceded by an apostrophe, is seen as a good candidate because it
seems to be a single letter followed by a bold (as in the l'''idee''
case). So that bold gets split *again*. Meaning that the four
apostrophes end up getting rendered as two apostrophes followed by
italics.
I suspect this was not planned behaviour.
Steve