Tim Starling tstarling@wikimedia.org variously wrote:
https://fr.wikipedia.org/w/index.php?title=Mod%C3%A8le:Donn%C3%A9es_PyrF1-2009&action=edit That template alone uses 47MB for 37000 #switch cases
I tried converting that template with 37000 switch cases to a Lua array. Lua used 6.5MB for the chunk and then another 2.4MB to execute it, so 8.9MB in total compared to 47MB for wikitext
It's only a 400kb string, and no "key" is a substring of another key. So just match the regexp /|<lookFor>=(.*)$/m , and $1 holds the value. This works great in Perl, PHP, JavaScript...
D'oh, Extension:RegexParserFunctions not enabled on Wikimedia sites.
Fine, use string functions to look for |<lookFor>= , look from there onwards for the next '|', and take the substring. D'oh, $wgPFEnableStringFunctions is set false on Wikimedia sites, bug 6455 (a great read).
Fine, use the string lookup function people have coded in wiki template syntax. e.g. {{Str find0}} – Very fast zero-based substring search with string support up to *90* characters. D'oh, several orders of magnitude too small.
OK, lua and Scribuntu. Reading the fine tutorial https://www.mediawiki.org/wiki/Lua_scripting/Tutorial
local p = {} p.bigStr = [[ |01001=22.4 |01002=17.3 ... 36,000 lines ]]
p.findStr = '|' .. p.lookFor .. '=' p.begin, p.ending = string.find( p.bigStr, p.findStr ) ... something or other...
Amazingly, my browser and the syntaxhighlighter in the Module namespace can handle this 400kB textarea, https://www.mediawiki.org/wiki/Module:SPageBigString.lua, well done!
If I just ask for the string.len( p.bigStr ), Scribuntu loads and executes this module. I dunno how to determine its memory consumption. But when I try to do string.find() I get "Script error", probably because I've never written any Lua before this evening. Assuming it's possible, what are the obvious flaws in string matching that I'm overlooking?
Is there an explanation of how to simulate the Scribuntu/Lua calling environment (the frame setup, I guess) in command-line lua?
This was fun :-) -- =S Page software engineer on E3