New subject: Scribunto, Lua 5.2, and forward compatibility for Module code

14 Jul 2014


      Hi all. I came across a strange issue with regards to Lua 5.1 and Lua 5.2
this weekend. The defect was caused by a change in behavior for "tonumber"
between 5.1 and 5.2. I detail more below for anyone interested.
With that in mind, I wanted to ask a few questions to the group:
* Will Scribunto support Lua 5.2 in the future? According to this comment,
it may:
https://github.com/wikimedia/mediawiki-extensions-Scribunto/blob/master/Scri...
* If so, what is the best approach to ensure forward compatibility of
Module code for 5.2? Note that code that is valid for 5.1 may "break" in
5.2. (Again, see below for details)
** Should there be a reference page on MediaWiki or in the enwiki Wikipedia
Namespace that details these breaking issues for other Module writers?
** Or should 5.1 vs 5.2 issues be corrected on a case-by-case basis for
each Module?
** Or should they be left as is, as any future-proofing may be unnecessary
and / or premature?
* Lastly, is there a general purpose mailing list for Scribunto issues?
I've had a few technical questions in the past that have been painful to
sort out on my own. I'd like to think that there might be other Module
writers who would also be interested in a mailing list as well.
Detail on the "tonumber" issue follows below.
----
== In brief ==
* For a call of "tonumber(213.5616, 10)"
** 5.1 will return back 213.5616
** 5.2 will return back NIL
== In detail ==
* Template:Weather_box uses Module:WeatherBox which eventually calls
Module:BaseConvert
* Module:WeatherBox often calls Module:BaseConvert with decimal values. For
example, here is a sample call:
    {{#invoke:BaseConvert|convert|n=213.5615|base = 16|width = 2|precision
= 0}}
* Module:BaseConvert has the following lines
    from = from or 10
    local num = tonumber(n, from)
* Note that if no {{{from}}} is passed, the default is 10. In general, most
callers do not specify a {{{from}}} (as seen above)
* Lua 5.1 has code that specifically says if tonumber is called with no
base or a base of 10, convert it as a number
    // http://www.lua.org/source/5.1/lbaselib.c.html
    static int luaB_tonumber (lua_State *L) {
        int base = luaL_optint(L, 2, 10);
        if (base == 10) {  /* standard conversion */
* In contrast, Lua 5.2 has code that only cares if a base is not specified.
    // http://www.lua.org/source/5.2/lbaselib.c.html
    static int luaB_tonumber (lua_State *L) {
      if (lua_isnoneornil(L, 2)) {  /* standard conversion */
** If a base is specified, Lua 5.2 will not do the standard conversion, and
instead try to parse the number
** This parse code only accepts alpha-numeric characters. The dot is
considered invalid, and any decimal number is converted to NIL
* As a result, for a call of "tonumber(213.5616, 10)"
** 5.1 will return back 213.5616
** 5.2 will return back NIL
* The NIL removes the background (and sometimes text colors) in weather
boxes.
For those curious, I discovered this in XOWA which uses Luaj -- a Java port
of Lua 5.2. I had to backport the 5.1 behavior to get the Weather box to
show the right colors again.
A proposed fix that would be forward compatible with 5.2 would be as
follows:
local num = 0;
-- {{{from}}} is specified, and {{{from}}} is not base 10
    if (from && from <> 10) then
      -- call overload that will always do parsing
      num = tonumber(n, from )
    -- {{{from}}} is not specified or {{{from}}} is base 10
    else
      -- call overload that will always convert to number
      num = tonumber(n)
from = from or 10
Finally, I've come across a related issue with the varargs operator
("..."). This operator is valid in 5.1, but not in 5.2. I've seen some
Modules use this varargs operator, that will presumably just break in 5.2.
See https://en.wikipedia.org/wiki/Module:Horizontal_timeline and "function
getNotNilValue(...)"