On Fri, Feb 22, 2013 at 4:41 AM, Johnuniq <wp.johnuniq(a)gmail.com> wrote:
On Feb 20, 2013 at 3:54 pm, Tim Starling wrote:
The idea of storing a database in a large string
literal could
be made to be fairly efficient and user-friendly if a helper
module was written to do parsing and a binary search.
I have implemented the above suggestion with some promising results.
Packing a large table in a string and unpacking it on demand appears
to work well, and the data is accessed as if it were stored in a
standard table. Using the table from Wiktionary Module:Languages
mentioned earlier in this thread, testing shows that accessing the
packed data is 20 times faster. Info is at
http://test2.wikipedia.org/wiki/User_talk:Johnuniq#Big_tables
Note that
https://gerrit.wikimedia.org/r/#/c/50299/ added a
mw.loadData() function that should solve the problem for normal
tables. It works like require, but can only handle simple data (no
functions, tables with metatables, or tables with tables as keys), the
returned data structure is made read-only, and it avoids having to
re-execute the module chunk on every #invoke.
Speaking of which, I need to update the documentation.
--
Brad Jorsch
Software Engineer
Wikimedia Foundation