On 21/09/12 17:47, Strainu wrote:
2012/9/21 Tim Starling
<tstarling(a)wikimedia.org>rg>:
On 21/09/12 16:06, Strainu wrote:
I'm just curious: would LUA improve memory
usages in this use case?
Yes, it's an interesting question.
I tried converting that template with 37000 switch cases to a Lua
array. Lua used 6.5MB for the chunk and then another 2.4MB to execute
it, so 8.9MB in total compared to 47MB for wikitext. So it's an
improvement, but we limit Lua memory to 50MB and you would hit that
limit long before you loaded 15 such arrays.
I'm not sure on how the Lua code would look like, but perhaps you can
tweak the loading of Lua templates so that you don't load the same
code more than once? I'm totally oblivious on how MediaWiki (or is it
PHP?) is linked to Lua right now, but I'm thinking along the lines of
a C program which loads a library once, then can use it many times
over.
With such an approach, you would have 6.5 + 15*2.4 = 42.5 MB of memory
(assuming memory cannot be reused between calls).
The Lua code looks like this:
a = {
['01001'] = 22.4,
['01002'] = 17.3,
['01004'] = 21.1,
['01005'] = 20.0,
['01006'] = 9.3,
['01007'] = 21.2
...
}
Then presumably you would do something with the "a" table to generate
wikitext. Lua needs 6.5MB per table for the internal representation of
the code itself, i.e. bytecode and supporting structures. Presumably,
most of that space would be in the form of instructions like "add an
element to the current table with key '01007' and value 21.2". When
those instructions are executed, another 2.4MB is needed to store the
resulting table.
The bytecode is cached between #invoke calls, but the table is not. So
if you replaced the individual data template invocations with #invoke
calls, the memory requirement would be something like 15*6.5MB + 2.4MB
= 99.9MB.
-- Tim Starling