Hello!
As many of you probably know, during last weeks (and also a lot of time in 2009 and 2010) I have been working on the WikiScripts extension (formerly InlineScripts), which allows to embed scripts into pages.
The language it uses, as well as the instruction of how to use it, are here: http://www.mediawiki.org/wiki/Extension:WikiScripts. Any clean up of the docs would be appreciated.
(the rest of the mail probably should be read after reading the documentation; right now it's short, because I'm lazy!)
The reason why we want our own language at all: * Code maintainability. Right now we have horrible templates like the legendary {{Citation/core}} nobody can really understand. * I personally hope that it will be more performance-wise and easier to optimize (due to fact that it has pretty straightforward context-free grammar). * It allows us to increase potential complexity of the code, introducing things like string functions
The reason why we use our own language is: * We can restrict is as we want (for example, the for( ; ; ) instruction is not present); * We can make it platform-independent since it is written in PHP * Based on some experience with writing those scripts and my generic experience as well, I have added some nice features absent from different languages, mainly the append keyword (see docs).
The reason why from InlineScripts it was changed to non-inline model: * This is the only way we can adequately implement functions. Previously I wanted to implement them as calling another template directly from script, but parser overhaul seems useless, and this way we have a better control over it. * We can add syntax highlighting * We can add the code editor * It is easier to track inclusions that way * Greater potential for global scripts
== The plan == Right now the extension is almost ready. There are certain features I would like to see there (some are listed below), but no backwards-incompatible dramatic changes. However, you may want to make them, so here is my plan 1. We settle down all issues we will not be able to change later due to b/c. I made a list of some below, you may want to add more. Before we settle them, we cannot move further. 2. At this point I want someone at WMF review the language description and make sure it is OK with them. 3. Once the details are settled, we may start doing internationalization, documentation, cleanup and testing. 4. Probably some live test deployment here. 5. ??? 6. PROFIT. WMF-wide deployment here.
== The questions == Here are some questions I found worth discussing. We need to discuss them before we move on: 1. Case sensitivity. Right now the language is case-sensitive. We may want to change that, since case-insensitivity is more user-friendly. 2. Transclusion from wikitext. I have added two hooks ({{#invoke:}} and {{s:}}) and will probably add two more for global scripts ({{#globalinvoke:}} and {{g:}}). Any better ideas for hook names are welcome. 3. Library. Right now I developed a set of conventions in order to make the function naming more consistent and modular (with ability to add new modules by extension). 4. Your question here.
== Stability and performance == Right now my benchmarking shows that a simple template which uses {{#switch}} is as effective as a simple module using associated arrays. You may benchmark yourself and put the results on the wiki. If we find the problems with performance on the real use cases during live trial (or before, or after), we have a lot possible performance tweaks, including: * Function output caching; * Built-in optimizer; * Separate cache for scripts (there aren't as many as templates), in order to increase hit rate; * Rewrite of parser in C; * Rewrite of whole interpreter in C (probably not necessary, 50% to 75% of time is parsing); * Rewrite of whole MediaWiki in <s>Python</s><s>C</s> HipHop-compatible PHP!
As of stability, we have built-in platform-independent limits (like evaluation count) and most of insecure features (recursion, infinite loops) are disabled (you can still make a deep nested loop, but you better write the scripts as if Domas knew where you live).
== Future == Some possible future tweaks: * switch construction * Built-in code editor (with highlighting and auto-ident) * AJAX-based debugging stuff right in the edit form (syntax check, calling the function from the edit form instead of preview, etc). * Global scripts! That would be (probably) easier than global templates; the only tricky part would be tracking changes and invalidating local caches. * Your suggestion submitted here: https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&...
--vvv
Victor Vasiliev wrote:
As many of you probably know, during last weeks (and also a lot of time in 2009 and 2010) I have been working on the WikiScripts extension (formerly InlineScripts), which allows to embed scripts into pages.
The language it uses, as well as the instruction of how to use it, are here: http://www.mediawiki.org/wiki/Extension:WikiScripts. Any clean up of the docs would be appreciated.
This all looks really great!
I have two questions, which are somewhat related:
* Has there a been a discussion/agreement about using a new scripting language on Wikimedia wikis? I know Lua, JavaScript, StringFunctions and a bunch of other options have been endlessly discussed in the past. Does your e-mail mean that a decision is made and this can move forward?
* What level of Wikimedia sysadmin support is there for this? I know you've worked on large projects in the past and have had difficulty getting code reviewed/deployed. I see Tim's name in some of the page histories (e.g., /Brainstorming), but some direct clarification about this would be great.
MZMcBride
On 18/08/11 13:52, MZMcBride wrote:
- Has there a been a discussion/agreement about using a new scripting
language on Wikimedia wikis? I know Lua, JavaScript, StringFunctions and a bunch of other options have been endlessly discussed in the past. Does your e-mail mean that a decision is made and this can move forward?
In the past week, I've been working on a prototype Lua solution. I wrote an extension for PHP that embeds a Lua interpreter, with some extra features (such as CPU time and memory limits) that weren't present in the previous solutions. I will add support for it to the existing Lua extension for MediaWiki and add some extra features to that extension.
When that's done, we'll be able to compare it with WikiScripts in terms of performance and flexibility and to choose the most suitable solution.
I discussed WikiScripts extensively with Victor at Wikimania. At the start of the conference week, I was very keen on the idea, but as the week went on and I did more research, I became less keen on it and more keen on Lua. I told Victor that this was the situation, and that deployment of WikiScripts would depend on its benchmark performance and other criteria.
I'm very anxious about getting this right. I think the current situation with template programming on Wikimedia wikis is terrible, and I think it's largely my fault. The language is ugly and inaccessible, causing metatemplate editing to be confined to a small group of elite editors, and it's slow, causing save times of 40 seconds or more on popular articles. Perhaps by supporting the right solution here, I can gain some absolution.
https://bugzilla.wikimedia.org/show_bug.cgi?id=26786#c24
http://www.mediawiki.org/wiki/Extension:Lua
-- Tim Starling
On 19/08/11 04:50, Tim Starling wrote: <snip>
I'm very anxious about getting this right. I think the current situation with template programming on Wikimedia wikis is terrible, and I think it's largely my fault. The language is ugly and inaccessible, causing metatemplate editing to be confined to a small group of elite editors, and it's slow, causing save times of 40 seconds or more on popular articles. Perhaps by supporting the right solution here, I can gain some absolution.
At the time you implemented the templates and parserfunctions, the community gave you a huge "thank you!". It was probably better to have barely usable feature than no feature at all :-)
When you wrote your code, I am sure you never thought the community will push your system to its limit. So there is nobody to blame you on that.
For the good part, the functions lowered the pressure imposed to the few core devs by the community since YOU enabled them to create new features!
We probably should have created PHP extensions for the most used templates. The enwiki {{citation}} template comes to mind.
Just to help you on the way to absolution, please remember your day is in roughly 2 months!!! http://en.wikipedia.org/wiki/Wikipedia:Tim_Starling_Day
2011/8/18 Victor Vasiliev vasilvv@gmail.com
Hello!
As many of you probably know, during last weeks (and also a lot of time in 2009 and 2010) I have been working on the WikiScripts extension (formerly InlineScripts), which allows to embed scripts into pages.
Excellent!
Just little bits of constructive criticism:
- Case sensitivity. Right now the language is case-sensitive. We may
want to change that, since case-insensitivity is more user-friendly.
No, case-insensitivity is user-UNfriendly. In MediaWiki links and templates are case-sensitive (except the first letter) and it is a Good Thing. The script language must also be case-sensitive. Hey, it's less work for you.
- Transclusion from wikitext. I have added two hooks ({{#invoke:}}
and {{s:}}) and will probably add two more for global scripts ({{#globalinvoke:}} and {{g:}}). Any better ideas for hook names are welcome.
One-letter names will be very confusing. 's' and 'g' can mean just about anything. I would offer better names, but i don't even understand what are these supposed to mean.
-- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com “We're living in pieces, I want to live in peace.” – T. Moore
wikitech-l@lists.wikimedia.org