On that note, I'll mention there was a previous attempt by Liangent to do better sorting of categories for zhwiki, that unfortunately never got reviewed so ended up bitrotting.https://phabricator.wikimedia.org/T46667

I agree generally, that if you want to change the sort order for categories, that lua is not the best place to implement that.

--
bawolff

On Sun, Mar 15, 2020 at 8:25 PM Huji Lee <huji.huji@gmail.com> wrote:
Deryck,

I am not sure what you mean by "re-sort" articles, but if what you means is that categories should be sorted differently, then I don't think Lua is the answer, and it would need to be handled on the back end.

In general, also, I think that for a problem like yours, Lua is not the right answer. I would recommend investing the time to create a MediaWiki extension instead, and then work through the WMF processes to have it enabled on Cantonese Wikipedia (and possibly, the entire family of Cantonese WMF wikis).

Lastly, as far as where to store the data, have you considered wikidata? I'm not sure if wikidata already supports storing pronunciations of words or not but I'm assuming that would be of interest to that project anyway.

Hope this helps!
Huji

On Sun, Mar 15, 2020 at 3:55 PM bawolff <bawolff+wn@gmail.com> wrote:
Consider using https://www.mediawiki.org/wiki/Extension:Scribunto/Lua_reference_manual#mw.loadData , keeping in mind that lua isn't really made with the usecase of huge data tables in mind, so there might be limits you run into if your data is really big.

--
Bawolff

On Sun, Mar 15, 2020 at 2:13 PM Deryck Chan <deryckchan@gmail.com> wrote:
Hello Ambassadors - This technical question may be relevant to multiple (particularly CJK) language communities so I'm asking it here.

What is the advice for writing a Lua script that needs to look up data from a big table (~10k rows at first deployment, potentially increasing in the future)? Does one hard-code the data into a Lua script, or is there a recommended data structure for storing those?

The design problem at hand is that the Cantonese Wikipedia wants to re-sort articles by Jyutping rather than Unicode. This will probably involve automating the generation of Jyutping phonetic guides by looking up the Jyutping transcription of common Chinese characters using a Lua module. Where do we store the data?

If another wiki has done similar things, we'd be interested in sharing the infrastructure.

Deryck
On behalf of the Cantonese Wikipedia community

_______________________________________________
Wikitech-ambassadors mailing list
Wikitech-ambassadors@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors
_______________________________________________
Wikitech-ambassadors mailing list
Wikitech-ambassadors@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors