Giant mails follow, no panic.
2012/2/6 Gabriel Wicke wicke@wikidev.net:
The enriched HTML DOM we are building (and actually most token stream processing including template expansion) is not tied to any specific syntax or user interface.
It is tied to HTML and it's the same. Even if all of current wikitext features can be represented by HTML (which I doubt) there's no guarantee that this will be true in furutre. This view has probably led to current messy markup.
When I'm saying that HTML can't represent even current wikitext features I imply that we're not talking about microformats and other XHTML tricks. And if we're talking about plain HTML then why not use a completely new format for storing DOM? Or at least clean XML without any standard namespaces that in theory should ease rendering of DOM into HTML (?). It will be parser-specific and won't suffer from future changes of linked namespaces, will be simple to test, etc.
On Future/Parser development diagram HTML DOM is built right after the stream has been parsed into a tree... in other words HTML5 is used to represent wiki. With tricks.
But I'm already venturing into an offtopic discussion here.
But in any case, we first have to implement a solid tokenizer for the current syntax and recreate at least a part of the higher-level functionality (template expansion etc) based on a syntax-independent representation.
I agree on this one.
2012/2/6 Sumana Harihareswara sumanah@wikimedia.org:
Pavel, you're clearly both an intelligent and a technical man - but not all intelligence is of the same, technically-minded type, and it's not always backed up by pertinent and complex knowledge.
I'm flattered with your words, thanks, Oliver.
However, this does not explain why at first Wikipedians had no troubles editing (and even creating) articles and now they are gradually loosing this skill. Is this a result of general degradation? I would hate to think this way and believe it's more what Yury has already said above - the project is just getting mature and, naturally, subjects for new articles that are left require more than a general knowledge while edits for existing articles are either complete, require some special knowledge as well or are plain unmotivated enough - new page patrol, "article babysitting", etc. are all "dirty" work and by definition are not that interesting as adding a new article section, prooflink or even correcting a simple typo.
In other words regular edits can be done by most visitors while maintenance - by small slice of them. It's the same with computers and users: most of people can use the computer but only some of them can type regedit.exe without breaking things. Is it different with Wikipedia? Is it different with most other non-commercial project?
The complexity of our existing markup language is a barrier, but not as much as the presence of any markup language whatsoever as a default.
Now this is something specific to argue about. I must admit that your speech has given me something to think about; perhaps you're right and the fact that initial editors of Wikipedia have come from that "first wave" of the Internet users - with this in mind it's understandable why their number is wearing out.
The usability studies that you have referred to speak with one accord that WYSIWYG is a must. I admit it sounds appropriate in that context. Still, another link suggests that even non-technical people were able to edit and (uh!) format text as bold and italic given a bit of help. And then it notes that even before doing any edits - or seeing an editor's window, be it text or visual - people were confronted with dozens of guideline links and warnings.
Which problem is more important? How you're going to present users with warnings in an inline visual editor? Or is it easier to just put "I've read and understood the rules" fobber-off and consider the matter settled?
More things to ponder about before my peaceful sleep, huh.
p.s: I wonder why people who can actually give answers are quite often not in the mailing lists.
2012/2/6 Amgine amgine@wikimedians.ca:
As I understand it, for the foreseeable future there will be a raw wiki syntax interface available. I hope contributors can be reassured on this point.
Combined with: 2012/2/6 Trevor Parscal tparscal@wikimedia.org:
Make significant changes to what Wikitext is and can do
The problem with this is that if present "raw wiki syntax" will be kept it will ensure that edits continue downfall.
The concern I see being expressed, fundamentally, is "I have developed skills, practices, and efficiencies with current Wiki syntax. Is your new parser going to destroy my investments in learning? am I going to have to start over with this new system?"
I think it's close in words but not in the meaning. What will you choose: cope with an old dusty car of your grandfather with annual repairs, dyeing, cleaning or find a free day, go to a nearby shop, choose a top-notch car with nano-tech-driven automatic repair, dyeing, cleaner that will serve you for the foreseeable future?
How many programmers (given the opportunity) choose to maintain old spaghetti code over refactoring it to something they'll have pleasure working with?
Quite few, as you probably know. It's the same with common folk who'll stick to old printer, scanner and copier than a new all-in-one device. But it's not right and everyone knows that it's better when they break this trend.
2012/2/7 Jay Ashworth jra@baylink.com:
Correct, and it isn't merely investments in learning; there are likely investments in wrap-around-the-outside coding which assume access to markup as well. Not All Mediawikiae Are Wikipedia.
I hope this was not a case for keeping old markup running. Most of the time it's better to provide backward compatibility module running on top of the new system than to fix and repair the old system trying to pursue mythical goal of supporting old versions.
Look at C++ STL and what it has become since '89. Look at Microsoft Windows and if its performance on an 4xi7 core has scaled along with Windows 95 on an 80356.
2012/2/7 Mihaly Heder hedermisi@gmail.com:
the millions of pages we already have is not easy to convert in the absence of a formalized wiki grammar
Indeed, but this can be solved by bringing together all pieces of modern wikitext under one roof and building a new strict grammar apart from that. Then a converter can be written that will seamlessly transform old syntax into new and warn user when this is not possible.
From what I know this is the direction WMF is going.
and some of them are already afraid that this skill will be obsolete because of the new editor, like the thread starter
This is the second time this argument appears in this thread but I don't understand it. Will you be afraid to "lose" your old coat that has worn our in a bin?
By following this list I hope I gathered how they plan to tackle this really hard problem: ... I think this is the smartest thing to do in this situation.
I agree, this is a way to go. If this works out then even terrible markup syntax won't be such a big trouble as it is now because if you disagree you can write your own tokenizer. The point is: why keep terrible syntax? It will be necessary to parse and transform the old wikitext syntax - I hope this is not argued - then why care how much improved markup will look like it or will it be completely different at all?
People will only greet the fact that, while a full-scale visual editor is underway, they at least can read and make occasional edits using more or less humane syntax. The best possible.
Isn't Wikimedia about a world with free knowledge? If so, it deals with texts most of the time. And the tools used have to be top-notch - there is no Microsoft to lobby OpenXML and force it down CEOs' throats.
I might sound rude but I hope for understanding. 5 years are enough for a single person (if he's minimally funded) to carry on the research, create an ideal markup language (as ideal as it can be spanning all cultures and nations), write a parser/serializer/renderer and even attach a text editor with advanced features made from scratch. And then there's even half of the time left.
Everyone at WMF able to hold the sword can get this thing done once and for all in a very short amount of time. No more annual parser rewrites, no more markup hell. After all, MediaWiki and its markup is the main workhorse of the community. It can't be kept on shelf that long...
Signed, P. Tkachenko