Consider using https://www.mediawiki.org/wiki/Extension:Scribunto/Lua_reference_manual#mw.loadData , keeping in mind that lua isn't really made with the usecase of huge data tables in mind, so there might be limits you run into if your data is really big.

--
Bawolff

On Sun, Mar 15, 2020 at 2:13 PM Deryck Chan <deryckchan@gmail.com> wrote:
Hello Ambassadors - This technical question may be relevant to multiple (particularly CJK) language communities so I'm asking it here.

What is the advice for writing a Lua script that needs to look up data from a big table (~10k rows at first deployment, potentially increasing in the future)? Does one hard-code the data into a Lua script, or is there a recommended data structure for storing those?

The design problem at hand is that the Cantonese Wikipedia wants to re-sort articles by Jyutping rather than Unicode. This will probably involve automating the generation of Jyutping phonetic guides by looking up the Jyutping transcription of common Chinese characters using a Lua module. Where do we store the data?

If another wiki has done similar things, we'd be interested in sharing the infrastructure.

Deryck
On behalf of the Cantonese Wikipedia community

_______________________________________________
Wikitech-ambassadors mailing list
Wikitech-ambassadors@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors