[Abstract-wikipedia] Re: Newsletter #190: A proposal for types per language and part of speech

تة

في الأربعاء، ١٩ فبراير ٢٠٢٥، ١١:٣٨ م Denny Vrandečić < dvrandecic@wikimedia.org> كتب:

The on-wiki version of this newsletter can be found here: https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2025-02-19

-- A proposal for Types per language and part of speech https://www.wikifunctions.org/wiki/File:TABLE.gif

We have written a proposal for types to represent a part of speech in a language https://www.wikifunctions.org/wiki/Wikifunctions:Type_proposals/Syntactic_type, e.g. a type for English nouns, a type for English adjectives, for Polish nouns, etc.

Even though many of these parts of speech in different languages are quite different, they have certain commonalities. We are proposing a pattern for these types to follow using a “table” type, and a function called “merge” that can help with many tasks of agreements in many different languages.

Based on observing previous natural language generation systems, such as Grammatical Framework, we have extracted this proposal. This proposal doesn’t yet touch on the big question of how to represent abstract content, but only on the question of how to more conveniently grow sentences from lexicographic entries in Wikidata. Over the last week months we have seen quite a few functions being built to construct phrases and sentences. We saw a few challenges, and are trying to address these with the given proposal.

We also discussed the type in the NLG SIG meeting (for a recording, see below), and the discussions on-wiki have already started. Recording of the NLG SIG meeting on February 18

This Tuesday, we had our first public Natural Language Generation Special Interest Group meeting. A recording of the meeting is available on Wikimedia Commons https://commons.wikimedia.org/wiki/File:Abstract_Wikipedia_NLG_SIG_meeting_2025-02.webm. A wiki page for the NLG SIG https://www.wikifunctions.org/wiki/Wikifunctions:NLG_SIG has been created, and agenda items for the meeting next month can already be collected there. Recent Changes in the software

A number of bigger changes landed this week, though not all of them are available immediately.

As part of our work this Quarter on extending Wikidata use, we're adding the new pre-defined function Z6830 https://www.wikifunctions.org/wiki/Z6830?action=edit&uselang=en&redlink=1 and its built-in implementation Z6930 https://www.wikifunctions.org/wiki/Z6930?action=edit&uselang=en&redlink=1 to get the Lexemes related to an Item (T383631 https://phabricator.wikimedia.org/T383631). We're also extending the definition of Wikidata property (Z6002) https://www.wikifunctions.org/view/en/Z6002 to allow more comprehensive fetching of Wikidata properties (T383636 https://phabricator.wikimedia.org/T383636), and provided new front-end UX components for referencing, fetching, and displaying them (T383643 https://phabricator.wikimedia.org/T383643).

On the integrating-Wikifunctions-calls-into-Wikipedias front, we have landed the major, base functionality on the PHP side (T272516 https://phabricator.wikimedia.org/T272516). This is behind a feature flag, so this is not yet available in production to test. It so far includes triggering the relevant display function on the response so it can be displayed in the content (T362252 https://phabricator.wikimedia.org/T362252), the relevant reading functions on the inputs so they can be turned from text into Objects ( T368604 https://phabricator.wikimedia.org/T368604), caching each call result in a cluster-wide system to improve speed and reduce load (T362256 https://phabricator.wikimedia.org/T362256), and the beginning of the cross-wiki notification code that will make sure the content is updated when the Function or Wikidata content changes, and put entries into RecentChanges and Watchlist pages on client wikis, like Wikidata does ( T383156 https://phabricator.wikimedia.org/T383156). We're also close to completion of the base integration with the wikitext and visual editors ( T373118 https://phabricator.wikimedia.org/T373118), which will allow us to experiment with embedding the full Wikifunctions experience for users. The PHP version of the error system can now be asked to output messages in an arbitrary language, rather than just English (T362236 https://phabricator.wikimedia.org/T362236), which will be used to display errors if things go wrong on the reader's page.

We fixed a few minor user-facing issues this week. We've tweaked the 'languages' dialog triggered from the About box to sort the languages based on the user's locale language (T355951 https://phabricator.wikimedia.org/T355951). We fixed some bugs spotted in the use of the new Codex component for the table listing the Implementations and Test cases for Functions – thanks for the help from the community for spotting some of those!

For development purposes, we've added a new staff-only right to bypass the results caching (T379432 https://phabricator.wikimedia.org/T379432). We've re-written our code's MediaWiki registration, so that it now specifies exactly which Codex components we use, so we get a custom build rather than the whole library (T372799 https://phabricator.wikimedia.org/T372799). This reduces the bytes shipped to each user of Wikifunctions.org, which we hope might improve performance when using the site. We've re-built the front-end codebase to use the Pinia state management library instead of Vuex, which has been deprecated (T318630 https://phabricator.wikimedia.org/T318630), and consolidated and renamed files and require() statements to match. We've re-organised the test code to start splitting out the tests that are for 'repo mode', i.e. for Wikifunctions.org, and that for 'client mode', such as Wikipedias. GitHub Video introducing Abstract Wikipedia and Wikifunctions by Danny Thompson

Last week, GitHub invited Danny Thompson, the CTO of this.dot to a conversation in their Open Source and AI series. He gave a great explanation of what Abstract Wikipedia and Wikifunctions is about, and we recommend a the video for a watch, The video is available on YouTube and on Twitch:

• Open Source and AI with Danny Thompson - YouTube

https://www.youtube.com/watch?v=68qYBxBiofE

• Open Source and AI with Danny Thompson - Twitch

https://www.twitch.tv/videos/2380182668

Invitation to a hybrid talk by Denny Vrandečić in London

On Monday, 10 March 2025, Denny Vrandečić will talk at King’s College London https://en.wikipedia.org/wiki/King%27s_College_London on the topic of Knowledge in the Age of AI https://www.eventbrite.co.uk/e/knowledge-in-the-age-of-ai-tickets-1232817453189. Wikidata and Wikifunctions will be topics in that talk. The event will be hybrid. You can join either remotely or locally in London. Free registration via Eventbrite https://www.eventbrite.co.uk/e/knowledge-in-the-age-of-ai-tickets-1232817453189 is requested if you want to attend. The talk will be recorded. Function of the Week: string of numeral digits in order from language

*The Function of the Week is a column written by the community. This week's submission has been written by 99of9 https://www.wikifunctions.org/wiki/User:99of9 and edited and improved by Feeglgeef https://www.wikifunctions.org/wiki/User:Feeglgeef and GrounderUK https://www.wikifunctions.org/wiki/User:GrounderUK. Planning the column and submissions can be made here https://www.wikifunctions.org/wiki/Special:MyLanguage/Wikifunctions:Function_of_the_Week/submissions.*

This week we are discussing: string of numeral digits in order from language https://www.wikifunctions.org/view/en/Z22302.

The function takes one input, a Natural language https://www.wikifunctions.org/view/en/Z60, and returns a string of length 10 with each of the digits of numerals in that language. So, for example, languages using western Arabic numerals https://en.wikipedia.org/wiki/Arabic_numerals have digits "0123456789". On the face of it, this seems like a strange function to write or use. If you already speak a language, you already know these digits, and if you don't know a language, it tells you just one piece of information in a hardcoded inflexible string. But for a multilingual project like ours, it turns out to be such a useful helper function that we are currently using it indirectly every time we use the read functions for a natural number https://www.wikifunctions.org/view/en/Z13518 or floating point number https://www.wikifunctions.org/view/en/Z20838 in Wikifunctions.

Only one implementation is available so far, a Python lookup https://www.wikifunctions.org/view/en/Z22312. It essentially just checks if the input is a language with a known answer and, if so, directly returns the string for that language, defaulting otherwise to "0123456789". This implementation can never be fully completed, but even in its current form, it is very useful. It can be improved by adding additional decimal scripts, or by applying an existing string to another dialect that shares one of the existing scripts. There are about 20 more scripts listed at Hindu-Arabic numeral system https://en.wikipedia.org/wiki/Hindu%E2%80%93Arabic_numeral_system, so don't hesitate to add one. A JavaScript implementation would also be easy to implement here, and may deliver a small efficiency gain, which given how often this function is called, could be useful.

16 tests are available. Isn't linguistic diversity beautiful? Can you spot any patterns?

• French digits are "0123456789"

https://www.wikifunctions.org/view/en/Z22311

• English digits are also "0123456789"

https://www.wikifunctions.org/view/en/Z22310

• Gujarati digits "૦૧૨૩૪૫૬૭૮૯"

https://www.wikifunctions.org/view/en/Z22309

• Kashmiri (Devanagari) digits "०१२३४५६७८९"

https://www.wikifunctions.org/view/en/Z22308

• Bangla digits "০১২৩৪৫৬৭৮৯"

https://www.wikifunctions.org/view/en/Z22307

• Burmese digits "၀၁၂၃၄၅၆၇၈၉"

https://www.wikifunctions.org/view/en/Z22306

• Thai digits "๐๑๒๓๔๕๖๗๘๙"

https://www.wikifunctions.org/view/en/Z22305

• Khmer digits "០១២៣៤៥៦៧៨៩"

https://www.wikifunctions.org/view/en/Z22304

• Malayalam digits "൦൧൨൩൪൫൬൭൮൯"

https://www.wikifunctions.org/view/en/Z22303

• Kannada digits "೦೧೨೩೪೫೬೭೮೯"

https://www.wikifunctions.org/view/en/Z22355

• Odia digits "୦୧୨୩୪୫୬୭୮୯"

https://www.wikifunctions.org/view/en/Z22356

• Punjabi digits "੦੧੨੩੪੫੬੭੮੯"

https://www.wikifunctions.org/view/en/Z22357

• Tibetan digits "༠༡༢༣༤༥༦༧༨༩"

https://www.wikifunctions.org/view/en/Z22360

• Urdu digits "۰۱۲۳۴۵۶۷۸۹"

https://www.wikifunctions.org/view/en/Z22361

• Telugu digits "౦౧౨౩౪౫౬౭౮౯"

https://www.wikifunctions.org/view/en/Z22359

• Tamil digits "௦௧௨௩௪௫௬௭௮௯"

https://www.wikifunctions.org/view/en/Z22358

These tests simply repeat all of the strings represented in the Python implementation so far. So they are not particularly rigorous tests of the cases the function does not cover, but are more to display its current capabilities. Feel free to add tests of other languages or dialects that you know.

https://www.wikifunctions.org/wiki/File:Wikifunctions_Z13521_showing_99_plus_42_in_Bangla.pngAdd natural numbers https://www.wikifunctions.org/view/en/Z13521 showing "৯৯" (99) plus "৪২" (42) in Bangla

What is the point of this function? In scripts that use a decimal https://en.wikipedia.org/wiki/decimal numeral system and positional notation https://en.wikipedia.org/wiki/Positional_notation, the structure of numbers is so well defined that, even if you can't read the language, knowing the decimal digits is all you need to infer what number a numeral represents. This is used to our advantage in three compositions that call this function: a generic natural number reader https://www.wikifunctions.org/view/en/Z14292, a generic float64 reader https://www.wikifunctions.org/view/en/Z21926, and a configurable float64 reader https://www.wikifunctions.org/view/en/Z22353, which uses it to pre-process digits before a language-specific configuration function is used. Although we only have one script-specific number read function, read Malayam natural numbers leniently https://www.wikifunctions.org/view/en/Z22271, these general compositions offer an alternative, broader generic approach, allowing us to read numbers in a much wider range of scripts (see the figure for an example adding natural numbers with a Bangla interface language). Although it's not implemented yet, we will also be able to use this function in similar display function compositions to show numeral outputs in these scripts.

Please consider adding a single extra line of code to this function to support an entire additional numeral system in Wikifunctions! Fresh Functions Weekly

Here is a list of new functions that have been created since last week, with connected implementations and passing tests. Plenty of new functions to celebrate!

• Equal bytes (Z22373) https://www.wikifunctions.org/view/en/Z22373: renvoie

"vrai" si les octets donnés sont égaux, sinon "faux"

• next byte (Z22380) https://www.wikifunctions.org/view/en/Z22380: Returns

the next byte. At overflow, it starts at 0 again.

• get texts of representations of WD lexeme form (Z22396)

https://www.wikifunctions.org/view/en/Z22396: retrieves a list of monolingual texts for all representations of a Wikidata Lexeme Form

• representations of lexeme form (Z22399)

https://www.wikifunctions.org/view/en/Z22399: returns the multilingual text that contains the representations (key Z6004K3) of the specified lexeme form

• Kleenean as integer (Z22417)

https://www.wikifunctions.org/view/en/Z22417: pour un triléen donné, renvoie un nombre entier

• multilingual text includes monolingual text(s) (Z22425)

https://www.wikifunctions.org/view/en/Z22425: returns True if the Z11-list in the Z12/multilingual text is a superset of the list of Z11/monolingual texts

• multilingual text has first monolingual text (Z22430)

https://www.wikifunctions.org/view/en/Z22430: returns True if the first monolingual text is as given.

• multilingual text includes language (Z22431)

https://www.wikifunctions.org/view/en/Z22431: returns True if the Z11-list in the Z12/multilingual text has a string matching the specified language.

• byte as binary string (Z22448)

https://www.wikifunctions.org/view/en/Z22448: no description

• natural number in toki pona (Z22455)

https://www.wikifunctions.org/view/en/Z22455: converts numbers to their (traditional) toki pona representation

• sum the elements of a list of integers (Z22469)

https://www.wikifunctions.org/view/en/Z22469: no description

• value by key (safer) (Z22475)

https://www.wikifunctions.org/view/en/Z22475: calls the built-in value by key function except when that would fail unnecessarily

• string of first representation of lexeme form (Z22478)

https://www.wikifunctions.org/view/en/Z22478: retrieves the string in the first monolingual text amongst all representations of a Wikidata Lexeme Form

• lexeme reference of lexeme form (Z22483)

https://www.wikifunctions.org/view/en/Z22483: gets the lexeme reerence (key Z6004K2) of the supplied lexeme form

• grammatical features of lexeme form (Z22487)

https://www.wikifunctions.org/view/en/Z22487: get the list of Wikidata item references representing the grammatical features (key Z6004K4) of the supplied lexeme form

• is lexeme form plural? (Z22490)

https://www.wikifunctions.org/view/en/Z22490: tests if the lexeme form has a grammatical feature/number of Q146786 (plural)

• same Key reference (Z22499)

https://www.wikifunctions.org/view/en/Z22499: returns True if both Key references have the same reference

• join list of strings with spaces (Z22504)

https://www.wikifunctions.org/view/en/Z22504: joins a list of strings inserting a single space between each

• replace multiple spaces with single spaces (Z22507)

https://www.wikifunctions.org/view/en/Z22507: removes repeated (regular U+0020) spaces in a string, leaving only a single space in place

• capitalise first letter and add full stop (Z22511)

https://www.wikifunctions.org/view/en/Z22511: turn a string of words into a sentence format, with an initial capital, and a full stop at the end.

• sentence from list of words (Z22514)

https://www.wikifunctions.org/view/en/Z22514: takes a list of words, joins with spaces, collapses multiple spaces, then turns to sentence case and adds a full stop at the end

• add word to list then join with spaces (Z22518)

https://www.wikifunctions.org/view/en/Z22518: joins a list of adjectives (as strings) preceding a noun (string) to form a single string with space separators

• bitwise not (Z22529) https://www.wikifunctions.org/view/en/Z22529: Turns

every 1 in a byte into a 0, and every 0 into a 1.

• natural number to byte (Z22535)

https://www.wikifunctions.org/view/en/Z22535: returns a Byte object given a Natural number less than 256 (values over 255 are errors)

• Key reference from String (Z22549)

https://www.wikifunctions.org/view/en/Z22549: returns a Key reference with Z39K1 equal to the given string

• grammatical features inherited from lexeme (Z22556)

https://www.wikifunctions.org/view/en/Z22556: a lexeme form (often?) inherits the features of its parent lexeme

• grammatical features of lexeme (Z22559)

https://www.wikifunctions.org/view/en/Z22559: extracts a list of all grammatical features of a lexeme

• is grammatical number statement (Z22562)

https://www.wikifunctions.org/view/en/Z22562: no description

• is grammatical person statement (Z22565)

https://www.wikifunctions.org/view/en/Z22565: no description

• is grammatical statement (Z22568)

https://www.wikifunctions.org/view/en/Z22568: no description

• toki pona latin to sitelen pona (Z22571)

https://www.wikifunctions.org/view/en/Z22571: converts Latin representations of toki pona words to their sitelen pona glyph as defined in the Under-ConScript Unicode Registry (see https://www.kreativekorp.com/ucsur/charts/sitelen.html)

• Slavic languages cardinal (Z22576)

https://www.wikifunctions.org/view/en/Z22576: no description

• sum a list of floating point numbers (float64) (Z22579)

https://www.wikifunctions.org/view/en/Z22579: no description

• mean of a list of floating point numbers (float64) (Z22583)

https://www.wikifunctions.org/view/en/Z22583: no description

• median of float64 list (Z22588)

https://www.wikifunctions.org/view/en/Z22588: no description

• product of float64 list (Z22592)

https://www.wikifunctions.org/view/en/Z22592: no description

We can see a rich variety of functions – and there were a few more that didn’t make it to the list because they didn’t have tests or implementations. A comprehensive list of all functions sorted by creation date https://www.wikifunctions.org/wiki/Special:ListObjectsByType?type=Z8&orderby=latest can be found on-wiki. _______________________________________________ Abstract-Wikipedia mailing list -- abstract-wikipedia@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/abstract-wikipedia.lists.wikimed...