The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-28
--
Type proposal for natural numbers
<https://www.wikifunctions.org/wiki/File:Three_Baskets_with_Apples.svg>
We are getting close to having all the pieces in place to open up the next
type to the community: natural numbers! Natural numbers, that is the
numbers used for counting, are already in high demand, as witnessed by
functions such as the String length
<https://www.wikifunctions.org/view/en/Z11040> function.
We have prepared a type proposal
<https://www.wikifunctions.org/wiki/Wikifunctions:Type_proposals/Natural_num…>.
This type proposal is also meant as a template for future type proposals,
which should eventually be coming from the community, not the development
team. Input and discussion both of the format of the type proposal as well
as the actual proposal for natural numbers is welcome.
Note that we will launch numbers without immediate support for renderers
and parsers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20>,
which reduces the Wikifunctions community's ability to provide them in
translated form. This is temporary, and we are also working hard on getting
renderers and parsers implemented. That will allow Wikifunctions to display
and understand numbers in the way most appropriate for each language, but
it will not be available immediately when we launch natural numbers.
Recent Changes in the software
This week we have refreshed the design of the "meta-data" dialog which
opens when you click the "Details" control next to a function call result,
or the “info” icon (circle around “i”) next to a test result (T341127
<https://phabricator.wikimedia.org/T341127>). We now group the different
pieces of information together in collapsed "accordions", and summarise
some of the groups to help you understand them at a glance.
Thanks to GrounderUK, who found a mysterious bug that meant the pipe
character ("|") would be swallowed when trying to run tests on draft
Implementations or Test cases, but would work when saved. This was due to a
complication in MediaWiki's API handling, and we've worked around it for
now (T358089 <https://phabricator.wikimedia.org/T358089>). We found and
fixed a UX bug on the initialisation of blank custom objects, where only
the first key would be initialised with the correct type, while the others
were initialised as references (T355497
<https://phabricator.wikimedia.org/T355497>).
As part of our long-term vision for Wikifunctions, we want to support not
just multiple programming languages (currently Python and JavaScript, with
more to come later), but also provide a consistent, understandable way to
upgrade versions of these languages and help the Wikifunctions community
migrate – so we can spot *e.g.* code that works in ES2019 JavaScript
<https://www.wikifunctions.org/view/en/Z607> but needs to be improved to
also work in ES2020 JavaScript <https://www.wikifunctions.org/view/en/Z608>.
An important part of that plan is having consistent references for each
version, so we have been slowly improving our stack to use ZID references
rather than strings, and as of this week our UX now will understand both,
and new code will use ZIDs (T287153
<https://phabricator.wikimedia.org/T287153>). In future, we will use a bot
to mass-move all uses to just ZIDs.
We, along with all Wikimedia-deploy code like the Vector skin's search bar,
are using the latest version of the Codex UX library, v1.3.3, as of this
week. Though we do not believe the library upgrade will have any
user-visible changes in our use on Wikifunctions, please comment on the
Project chat or file a Phabricator task if you spot an issue. This version
of the library provides a new feature in how the lookup control works (
T350946 <https://phabricator.wikimedia.org/T350946>), which unblocks a
minor improvement to pre-populate the Object selector with common types,
which we hope to do soon (phab:T343564
<https://phabricator.wikimedia.org/T343564>).
Wiki Mentor Africa
<https://www.wikifunctions.org/wiki/File:WMA-wikifunctions_Flyer.jpg>
On this Saturday, March 2nd, at 16:00 UTC
<https://zonestamp.toolforge.org/1709395200>, Luca Martinelli and Denny
Vrandečić will present at an event in the Wiki Mentor Africa
<https://www.wikidata.org/wiki/Wikidata:Wiki_Mentor_Africa> program,
and on Sunday,
March 3rd, at 16:00 UTC <https://zonestamp.toolforge.org/1709395200>,
Hogü-456 will follow up with a hands-on demonstration. You can register for
that event on the Wiki Mentor Africa event Website for the Introduction to
Wikifunctions
<https://meta.wikimedia.org/wiki/Event:WMA_Introduction_to_Wikifunctions>.
A research agenda regarding AI and the knowledge commons
Last week, Denny attended a convening to discuss a research agenda
regarding the effect of current progress in machine learning on projects in
the knowledge commons, such as Wikipedia. The resulting draft for a
research agenda and more details on this event are published on Meta
<https://meta.wikimedia.org/wiki/Artificial_intelligence/Bellagio_2024>.
Function of the Week: Or (Z10184)
“Or <https://www.wikifunctions.org/view/en/Z10184>” is a binary Boolean
function. A binary function is one that takes two arguments. A Boolean
function is one where all arguments and the return type are Boolean, that
is, each of them is either true or false. A binary Boolean function is a
function that takes two arguments of type Boolean and returns a Boolean.
“Or” is also known as the logical disjunction
<https://en.wikipedia.org/wiki/Logical_disjunction>, and there are Wikipedia
articles <https://www.wikidata.org/wiki/Q1651704> about it in 46 language
editions of Wikipedia.
“Or” returns false only if both of the arguments are false. Otherwise it
returns true. This is meant to be a rigorous formulation of the natural
language use of “or”, which usually means that either one of the two parts
connected by the or may be given. Or both.
“Or” is useful to connect two conditions together. For example, if we want
to know if a word ends with a vowel
<https://www.wikifunctions.org/view/en/Z11961> or a y.
Since each Boolean argument can only have two possible values, and there
are two arguments, there are exactly four possible combinations of values
the function can take. Accordingly, there are four tests on the function: true
or true <https://www.wikifunctions.org/view/en/Z10192>, true or false
<https://www.wikifunctions.org/view/en/Z10195>, false or true
<https://www.wikifunctions.org/view/en/Z10198>, and false or false
<https://www.wikifunctions.org/view/en/Z10200>.
The function has three implementations:
- in Python <https://www.wikifunctions.org/view/en/Z10186>, using the
Python keyword or
- in JavaScript <https://www.wikifunctions.org/view/en/Z10201>, using
the || operator
- a composition <https://www.wikifunctions.org/view/en/Z13469> using the
if <https://www.wikifunctions.org/view/en/Z802> function: if the first
argument is true, return true, or else return the second argument
Think through the composition and see if you understand the logic: does it
indeed lead to the desired result for all four possible inputs?
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-22U…
the function model
A few weeks ago we started an update of the function model
<https://www.wikifunctions.org/wiki/Wikifunctions:Function_model>. The page
was set up early in the project and captured the original plan for the
function model, and was written prescriptively and aspirationally. The
actual implementation diverged from it in a number of ways. Some of these
divergences might be still aspirational, whereas others have evolved (*e.g.*
Benjamin arrays
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-07-29>)
and we have no intention to go back.
The page as it was before the updates was basically not useful, as it
represented neither the current status nor where we want to end up. We took
the time to update it from beginning to end, with the goal of representing
the function model as it is right now. It has now been updated to the
current state where appropriate, and aspirational content has been removed.
This is one step in improving our documentation. We think that the function
model is useful but not as useful as it could be: one set of feedback we
got was that it is hard to read and understand. We heard that and are
thinking about creating more documentation that is more accessible and can
support more paths for a diverse audience to find their way to
understanding Wikifunctions deeper if they want to do so.
Feedback and comments are welcome.
Recent Changes in the software
After last week's "Fix-It" drive, this week we returned to work on the
focus features as shared two weeks ago
<https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-07>,
as well as some smaller bug fixes. We continued our work on better support
for Types, with work on the way that user input will be transformed into a
value and *vice versa*
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20>,
which we hope to demonstrate soon, and then use in reality so that the next
Type can be created and used, on the way to interacting with Wikidata
lexemes and items.
We made a change to how the load of the software works for readers, which
means that we only load a small part of the Codex library before putting
the language control button on the page, rather than demanding the whole
library needed for rendering and interacting with Objects, which comes in a
few tenths of a second later. This should make browsing Wikifunctions
slightly faster for all readers and editors, especially for non-Object
pages. Our thanks to the Design System team for their work to make this
possible.
We re-wrote our code handling Wikifunctions Objects on the front-end, as it
could sometimes flake out and provide a broken user experience on the site;
we renamed and better documented the methods, and added a number of tests
to demonstrate the previously-broken, now-fixed behaviours (T352799
<https://phabricator.wikimedia.org/T352799>).
We fixed two bugs seen in production; one is an error check when processing
function calls in the API (T357691
<https://phabricator.wikimedia.org/T357691>), and the other is to fallback
to English when trying to change the display text of links to ZIDs and the
language isn't known (T357702 <https://phabricator.wikimedia.org/T357702>).
Spurred on by this, we also fixed the fall-back code when trying to access
a non-existent page with a non-English user language; previously this would
try to redirect you to the page called "Main Page" as translated in your
language, but now it will correctly send you to the actual main page in
your language.
We updated the credits file
<https://www.wikifunctions.org/wiki/Special:Version/Credits/WikiLambda> for
the MediaWiki part of the software, which adds Winston Sung (welcomed last
week).
Function of the Week: Negation
Negation <https://en.wikipedia.org/wiki/Negation> (not (Z10216)
<https://www.wikifunctions.org/view/en/Z10216>) is a unary Boolean
function. A unary function is one with a single argument. Boolean functions
are functions where all arguments are of type Boolean, *i.e.* they can be
either true or false, and the return type is also Boolean.
<https://www.wikifunctions.org/wiki/File:NOT_ANSI_Labelled.svg>The NOT gate
in electrical circuit diagrams
Negation takes one Boolean value and returns the other: if you ask for the
negation of true you get false, and if you ask for the negation of false
you get true. This is also how the composition
<https://www.wikifunctions.org/view/en/Z10537> is implemented: if
<https://www.wikifunctions.org/view/en/Z802> the value is true, return
false, else true. This is one pattern I really like seeing with one
composition: a composition that expresses what the function means in an
almost declarative way, without regards to how efficient the implementation
is. I am still thinking of a good name for these — maybe definitional
compositions? The usefulness of these definitional compositions is that
they serve both to explain what the function does, and also to give a high
confidence implementation to test the more efficient implementations
against.
Beyond the composition, there are three further implementations:
- In Python <https://www.wikifunctions.org/view/en/Z10538>, using the
keyword not
- In JavaScript <https://www.wikifunctions.org/view/en/Z10586>, using
the ! operator
- another one in JavaScript
<https://www.wikifunctions.org/view/en/Z10220> using the ! operator,
using a different syntax
I would recommend that as a community we come up with a JavaScript syntax
guideline that would suggest to only have one or the other JavaScript
implementation.
One interesting thing about Negation is that because the input types can
only have a closed, small set of different values, we can actually create a
complete set of tests, which test every possible input. In this case, in
fact, there are only two possible values: true and false. Accordingly we
have two tests: one that makes sure that the negation of true is false
<https://www.wikifunctions.org/view/en/Z10512>, and one for the opposite,
that the negation of false is true
<https://www.wikifunctions.org/view/en/Z10513>.
You can read much more about Negation on Wikipedia
<https://en.wikipedia.org/wiki/Negation>.
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-14
Recent Changes in the software: Fix-it week
As mentioned in last week's update, we had our regular "Fix-It Week", where
we work on technical debt, design oddities, and other issues that can pile
up when our main daily focus is on delivering larger-scale change for users.
One of main areas of change was in cleaning-up the Vue front-end code,
which has undergone a lot of change in the past few months. We improved our
front-end Vuex "store" code to remove now-unused methods and change the
remaining ones to be safer in edge cases (T329107
<https://phabricator.wikimedia.org/T329107>); we re-organised and split up
the 'mix-ins' in the Vuex store for consistency and better testability (
T328430 <https://phabricator.wikimedia.org/T328430>).
On the back-end services, we made a few improvements to code quality. One
of these was shifting the logging system from a global into our
"Invariants" class, so that it can be dependency-injected and so tested
better (T337785 <https://phabricator.wikimedia.org/T337785>). Our internal
ZWrapper class will now correctly error rather than silently continuing
when asked for a non-existent key (T309809
<https://phabricator.wikimedia.org/T309809>). We migrated the JavaScript
code coverage tool to use the built artefact pipeline rather than running
directly on the repo's code (T355815
<https://phabricator.wikimedia.org/T355815>). We also added pyflakes to our
CI, to improve our Python evaluator code quality, in the course of which we
found some bugs and added tests (T306365
<https://phabricator.wikimedia.org/T306365>).
We switched the Beta Cluster test version of the back-end services to use
the 'split' evaluator model that we rolled out to production last year (
T349008 <https://phabricator.wikimedia.org/T349008>), and so we were able
to decommission the old 'omnibus' image that would process both Python and
JavaScript code (T349557 <https://phabricator.wikimedia.org/T349557>).
After some checking to ensure we were content with the now-fixed results,
we added a test to demonstrate and enforce the current model of
error-handling in the WASM-wrapped JavaScript evaluator (T347898
<https://phabricator.wikimedia.org/T347898>).
We added a little further coverage of the PHP code that mirrors how Objects
work (T302599 <https://phabricator.wikimedia.org/T302599>), finding and
fixing a bug in how incoming results were interpreted in some
circumstances. We also found and fixed a forgotten remnant of a now-removed
piece of our test infrastructure that would cause errors when Wikimedia
production rolled forward to PHP 8.2.
Outside of proper "Fix-It" tasks, we also landed two improvements that will
roll out this week:
Following-on from last week's change to how string values are shown as
outputs, wrapping them in double quote characters, we now also do the same
for the abbreviated form, commonly seen on Test pages. This does not show
all whitespace, but it is a further minor improvement in this vein (T343608
<https://phabricator.wikimedia.org/T343608>).
Winston Sung, a Wikimedia community member very active in Chinese language
support, made their first contribution, fixing our use of whitespace on
each Object page's title headings in Chinese and some other languages (
T356731 <https://phabricator.wikimedia.org/T356731>). Thank you, and
welcome!
Function of the Week: duplicate string
Duplicate string <https://www.wikifunctions.org/view/en/Z10753> is a
function that takes a single string as the input, and returns a single
string as the output: the output is repeating the input twice.
<https://www.wikifunctions.org/wiki/File:Twins_11183454.jpg>
Duplication <https://en.wikipedia.org/wiki/Reduplication> of words or parts
of words, for various different effects, is a feature of many different
languages from all around the world. In some languages it may express
emphasis, in others a plural, but it can have many other semantic effects
too. The Wikipedia article
<https://en.wikipedia.org/wiki/Reduplication> provides
an overview of how duplication is used, with numerous examples. Here are
just two examples stolen from that page:
- In Finnish, *“ruoka”* means "food", but *“ruokaruoka”* means "proper
food", as opposed to snacks.
- In Swahili, *“piga”* means to “to strike”; whereas *“pigapiga”* means
“to strike repeatedly”
In Wikifunctions, duplicate string has five tests, WikiWiki
<https://www.wikifunctions.org/view/en/Z10754>, 1.11.1
<https://www.wikifunctions.org/view/en/Z10760>, ¨¨
<https://www.wikifunctions.org/view/en/Z10763>, the space character
<https://www.wikifunctions.org/view/en/Z10756>, and the empty string
<https://www.wikifunctions.org/view/en/Z10755> (a test that display
correctly as of today, thanks to the fix-it tasks of this week). Those are
great tests, trying out different kinds of characters and inputs. I
particularly enjoyed the empty string as an edge case. What I am missing
are non-English alphabet characters such as Hangul characters,
left-to-right scripts, or emojis. But this is a great set already.
There are four implementations, of which three are unsurprising: JavaScript
<https://www.wikifunctions.org/view/en/Z10759>, Python
<https://www.wikifunctions.org/view/en/Z10758>, and a composition
<https://www.wikifunctions.org/view/en/Z10757> using join strings
<https://www.wikifunctions.org/view/en/Z10000>, all of which are calling
the argument twice. The fourth implementation
<https://www.wikifunctions.org/view/en/Z10913>, another composition, is
using the function duplicate string n-times
<https://www.wikifunctions.org/view/en/Z10911>, which is a great function
to be used here, but requires a number as one of its arguments, but uses a
string (set to the string “2”, for the given composition). Once we actually
have a number type, this would need to be cleaned up.
All in all, this is a neat functionality, which is usually not available
through a standard library, and can exemplify tests and different
implementations.
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-07
--Quarterly planning
Two weeks ago we held an internal planning meeting to sketch out what we
want to work on in the current quarter. Last week we collected and wrote
down the results, and we want to share those results with you.
<https://www.wikifunctions.org/wiki/File:Planning_icon_from_Noun_Project.png>
Our team's overarching goal is to work towards supporting the pieces needed
for Abstract Wikipedia.
- *Type support for Wikidata prototype*. We want to add more types to
Wikifunctions, in particular with an eye on the types needed to be able to
integrate forms from lexicographic data on Wikidata. For this, we want to
work on serializers and deserializers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-27>,
renderers
and parsers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20>,
as well as validators. The types we expect to enable are, besides
the recently
enabled lists
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2024-01-03>,
numbers, enumerations (for, *e.g.* grammatical features), and lexemes.
We will work together with the community on the exact list of types to
support, but in the end we hope to have all types in place so that in the
next quarter we can work towards accessing lexemes from Wikidata.
- *Public API for Wikifunctions*. We want to encourage the creation of
third-party apps using functions from Wikifunctions, such as the
integration with Lucas Werkmeister’s Wikidata Lexeme Forms
<https://www.wikidata.org/wiki/Wikidata:Wikidata_Lexeme_Forms> which is
currently using an internal API. The first step is to design the API and
decide on our approach.
- *Simplify our end-to-end tests and improve the reliability*. Our stack
is a bit complicated compared to other systems at Wikimedia, running an
evaluator and orchestrator services on the backend as well as the wiki
itself with the Wikilambda extension, and relying on certain content in the
wiki. Because of this, parts of our continuous integration test platform
are currently not in a great shape. This task is about improving that
situation and increasing our confidence in our tests.
- *Research for calling Wikifunctions inside Wikipedia content*. One
major goal of our project is to allow the Wikipedia and other Wikimedia
communities to call functions from Wikifunctions within wiki pages, like
they do with Commons media files or local templates and Lua scripts. In
order to prepare for this we will be starting user research to get a better
understanding of user needs and expectations.
- *Develop a baseline understanding of our users and their motivations*.
To make sure that we are developing Wikifunctions in a user-informed way,
we want to make better analysis of who our current users are and why they
are here based on the metrics we have so far.
- *Establish a regular cadence of external communication*. Our weekly
newsletter is good to communicate with our immediate community, but we want
to reach out to a wider audience as well. This will include regular posts
on Diff <https://diff.wikimedia.org/>.
- *Develop a framework to measure the narrative difference between two
Wikipedia articles in different languages*. This is a research task that
will take us on the path towards understanding how any two articles in two
different languages about the same topic differ. Such a tool would help us
with providing guidance for growing Abstract Wikipedia, and at the same
time it could potentially provide us with an early warning system that
might indicate a loss of diversity.
As this was our first planning effort for such a cadence, part of it is
also learning to pick the right size of work. At the end of the quarter we
will look back at what we actually achieved, and will learn from that for
the next quarter.
Thank you, Quiddity!
<https://www.wikifunctions.org/wiki/File:Nick_Wilson_at_Wikimania_2023.jpg>Nick
"Quiddity" Wilson at Wikimania 2023
Last week saw Nick Wilson’s last day on the Abstract Wikipedia team, before
he heads into a well-deserved break. Nick may to some of you be better
known under his wikinym “Quiddity”. Nick was part of the Abstract Wikipedia
project from the very beginning, set up and organized the naming contest
for Wikifunctions, and set up most of our communication structures.
Here are a few words by Nick as a good bye message:
*Thank you, all!*
*It has been fascinating, humbling, motivating, and deeply inspiring to be
a part of these complex projects. I've appreciated getting to know (or
become more deeply familiar with) so many aspects of our movement and our
mission, and more of the people that steadily help everything to move
forwards.*
*As always, I wish I could clone myself in order to continue focusing on
this project forever, in addition to working in new areas. I'm eagerly
looking forward to a function-powered future, and I hope to continue
helping in a volunteer capacity after I take a short wikibreak.‘*
Luca, who joined us in early 2022
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-01-21>,
will continue in the role of Community Relations Specialist on the project
and take over Nick’s responsibilities. Please join me in thanking Nick for
his work!
Recent Changes in the software
As discussed above, part of our bigger work this quarter is type support
leading to Wikidata support. As part of this, we've fixed and now
re-enabled validation for types (T352295
<https://phabricator.wikimedia.org/T352295>), which shouldn't have any
visible effect yet but will unlock the ability to tell the user their input
is wrong when trying to use a type (such as saying that a natural number
cannot have non-digits in it, or a date in February cannot be on the 30th
day).
We've now made a small tweak to the display of strings returned from
functions, wrapping them in double-quotes, which makes it possible to
distinguish "" and " " as different responses. This does not fix all issues
with spaces in inputs, as the nature of HTML makes some possible return
values hard to cope with in a scalable way that doesn't break other use
cases. We plan to come back to this area with further improvements, and
would love your thoughts on improvements you'd like to see.
We've also fixed the interface breaking when using a language that
MediaWiki doesn't support, such as
https://www.wikifunctions.org/view/elx/Z10000 – previously you would end up
in a broken state, but now things work as designed (phab:T356428
<https://phabricator.wikimedia.org/T356428>). This also means that if you
try to use a language that even Wikifunctions doesn't know about, such as
https://www.wikifunctions.org/view/hellothisisnotalanguage/Z10000, the
system will fall back to English (like the rest of MediaWiki) rather than
mysteriously providing a mixture of English and mostly bugs.
In smaller changes, one is a follow-up to last week's change to remove the
noindex directive from our pages so that Google and other search engines
see them (phab:T355441 <https://phabricator.wikimedia.org/T355441>).
Alongside the main links like https://wikifunctions.org/view/ar/Z801, we
still have some URLs that also work due to how MediaWiki controls pages,
like …/wiki/Z801?uselang=ar <https://wikifunctions.org/wiki/Z801?uselang=ar>,
or …/wiki/Special:ViewObject/ar/Z801
<https://wikifunctions.org/wiki/Special:ViewObject/ar/Z801>; these will now
no longer claim to be "canonical" (T355546
<https://phabricator.wikimedia.org/T355546>). Another is that we've moved
all the linked software help pages on MediaWiki.org to be in the same
place, in the "Help:Wikifunctions" hierarchy. We've also made a tweak to
the UX of the language selector on a page when it has a number of
translations already – the padding is now in line with the rest of the
form, instead of much wider (T355946
<https://phabricator.wikimedia.org/T355946>).
We also added two new languages that MediaWiki supports: Ebira (Z1920)
<https://www.wikifunctions.org/view/en/Z1920> and Petjo (Z1921)
<https://www.wikifunctions.org/view/en/Z1921>. These are added manually in
production.
On the code side, we've made some minor improvements to our PHP code's
coverage of the critical ZObject classes, edging up towards 100% (T302599
<https://phabricator.wikimedia.org/T302599>). We've made a series of
clean-ups to our front-end code following-on from previous work, removing
unused components (T301868 <https://phabricator.wikimedia.org/T301868>) and
generally improving our data store code (T329107
<https://phabricator.wikimedia.org/T329107>). This week is a "Fix It" week
for the team, so we'll make further such improvements that will ship next
week.
Function of the Week: is permutation
For this week’s Function of the Week I asked Nick for a favorite function,
and he immediately answered that he loves lists and the unusual, and so he
would like to choose an unusual function that deals with lists. Looking at
the catalog of list functions
<https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue#List_operations>,
most of them were what you would expect from list operations, but one of
them struck us as unusual in the sense that most programming languages
would not have support for that functionality out of the box: is permutation
<https://www.wikifunctions.org/view/en/Z12741> (please feel free to correct
us).
<https://www.wikifunctions.org/wiki/File:Permutations_RGB.svg>Permutations
of three colors
A permutation <https://en.wikipedia.org/wiki/Permutation> of a list is a
second list that has exactly the same elements, but possibly in a different
order. The function is permutation
<https://www.wikifunctions.org/view/en/Z12741> takes two lists and checks
if they are permutations of each other. The function returns a Boolean
value: true if the two lists are permutations of each other, and false if
they are not.
In general, I have a bit of an issue with a function like this one: the
function is defined on lists of objects, and not on lists of a specific
type, such as a list of strings, or a list of Booleans. But not all types
will necessarily have a function to check for equality, which will be a
necessary prerequisite for this function to work. So this might lead to
problems down the road. I am not sure how to handle this best.
The function currently has five tests: three pairs that are not
permutations – (a, a, a) / (a, b, c)
<https://www.wikifunctions.org/view/en/Z12743>, (b,c) / (a,b,c)
<https://www.wikifunctions.org/view/en/Z12744>, and (b,b,a) / (a,a,b)
<https://www.wikifunctions.org/view/en/Z12747> – and two pairs that are
permutations, (b,c,a) / (a,b,c)
<https://www.wikifunctions.org/view/en/Z12742> and (a,b,a) / (a,a,b)
<https://www.wikifunctions.org/view/en/Z12746>. It’s great to see five
tests! I would also go for some edge cases (what about empty lists?) and,
given that the function is defined on lists of any type, also for lists
that have more than single-character strings as elements. But again, yay
for five tests! With five tests, the function is in the top 10% of
functions per number of tests.
The function currently has three implementations, one in Python and two
compositions. The Python implementation
<https://www.wikifunctions.org/view/en/Z12869> has two steps: it first does
a short-cut, checking if the two lists have different lengths. If they do,
it stops the implementation and returns False: two lists that have
different lengths can never be permutations of each other. Once we know
they have the same length, we compare whether the two lists, when sorted,
are the same. If they are, the two lists are permutations of each other, if
not, they are not. The sorting turns each of the input lists into a
canonical version with regards to its elements that is independent of the
order, which is why this implementation works. In theory, we could also
drop the first step from this implementation, but having the check probably
allows for some speed-up.
One of the compositions works by using supersets
<https://www.wikifunctions.org/view/en/Z12869>: we have a function, is
superset <https://www.wikifunctions.org/view/en/Z12846>, that checks
whether one list is a superset of the other. The composition here uses is
superset to check whether the first list is a superset of the second, and
the second is a superset of the first. If both conditions are fulfilled (we
check that by using and <https://www.wikifunctions.org/view/en/Z10174>), we
know that both lists contain all the elements of the other list, and
therefore contain the same elements, and thus they must be permutations of
each other. Two of the tests time out, but unfortunately with an unusable
error message. A bug has been reported (T356556
<https://phabricator.wikimedia.org/T356556>).
The other composition <https://www.wikifunctions.org/view/en/Z12867> is
more elaborate:
- First it checks if the two lists have equal lengths
<https://www.wikifunctions.org/view/en/Z12864>. If not, it returns false.
- If they do, it checks if the first list is empty
<https://www.wikifunctions.org/view/en/Z813>. If it is, it returns true
(because we know that the second list also must be empty, because of the
previous check, and empty lists are permutations of each other).
- Next, we recursively <https://en.wikipedia.org/wiki/Recursion> call is
permutation itself. For the recursion step, we find and remove
<https://www.wikifunctions.org/view/en/Z12856> the first element
<https://www.wikifunctions.org/view/en/Z811> of the second list from the
first list, and compare that to the second list without the first element
<https://www.wikifunctions.org/view/en/Z812>.
In this last step we remove the same element from both lists, and are thus
left with two shorter lists. If those are permutations of each other, the
two lists with the removed elements would also be permutations of each
other, no matter where those two elements are added in those lists.
If the first element of the second list does not exist in the first list,
the first list remains unchanged, a condition checked by this test
<https://www.wikifunctions.org/view/en/Z12859> on remove first matching
element <https://www.wikifunctions.org/view/en/Z12856> function.
The two compositions illustrate how different two compositions can be in
terms of complexity: the composition using superset is a very simple
composition, almost like a definition, combining just two existing
functions. The other composition is much more elaborate, combining quite a
number of different functions, using nested if conditions, and a recursive
call. The latter composition is probably about half-way through the
complexity level I would expect us to support for function compositions: I
wouldn’t expect much more complex compositions than that in Wikifunctions
at all. But, in the end, both compositions are great examples of how to use
composition for creating higher-level functions.
The on-wiki version of this newsletter is from now on to be found on
Wikifunctions, and not on Meta anymore. It contains a video that was
removed from here. The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-01
--
The Igbo imperative!
I was talking with Uzoma Ozurumba
<https://meta.wikimedia.org/wiki/User:Uzoma_Ozurumba>, a Community
Relations Specialist
<https://meta.wikimedia.org/wiki/Communications/Movement_Communications>
working
at the Wikimedia Foundation, and demonstrating Wikifunctions to her. Uzoma
is a founding member of the Igbo Wikimedians User Group
<https://meta.wikimedia.org/wiki/Igbo_Wikimedians_User_Group>. Igbo
<https://en.wikipedia.org/wiki/Igbo_language> is the principal language of
the Igbo people <https://en.wikipedia.org/wiki/Igbo_people> who live in the
South-Eastern part of Nigeria, and is spoken by more than 44 million
people. Igbo is also one of the five focus languages of Abstract Wikipedia
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-04-15>,
Wikifunctions, and Wikidata’s improvements to the lexicographic data.
Igbo is currently represented with more than 3,900 Lexemes in Wikidata,
making it currently the 34th language by number of Lexemes, before Punjabi
and Polish, and after Portuguese and Mandarin. In Wikifunctions, already as
early as September, a first function for Igbo was created, adding the
preposition ‘na’ to a word <https://www.wikifunctions.org/view/en/Z11248>.
When I was talking with Uzoma, I was showing her another function, which
takes the base form of a verb and creates the imperative
<https://www.wikifunctions.org/view/en/Z11353>, the form used to make
commands. The function had been created just days before my meeting with
Uzoma. I don’t remember which verb I used to demonstrate it to Uzoma. I
think I had prepared myself, and memorized the tests. So I opened the page
for the function, entered “gụ” (which means “to read”). I called the
function, and the system came back with “gụọ”.
I wasn’t prepared for Uzoma’s reaction. Her eyes were shining, and she had
a big smile on her face. She was delighted to see a function operating on
her native language. And not only that this was a function that had just
been recently created, but that it was open to everyone in the world.
Everyone could just go to Wikifunctions and call natural language
processing functions in Igbo. Or create new such functions, and share them
with the whole world.
Before Wikifunctions, that wouldn’t have been easily possible. Whoever
created such functions would share them, maybe as a JavaScript library.
Whoever wanted to use them, would need to install the right programs on
their computer, if they even had a computer. How do you try out a Python
library on a mobile phone? Who would create and share a library that only
creates the imperative?
With Wikifunctions, many of these issues have evaporated. Whoever wants to
share even a single morphological function is very welcome in
Wikifunctions. They don’t have to set up a library and share it through npm
or GitHub. Someone who wants to use it doesn’t have to install a whole
development environment. Instead, they can just go to Wikifunctions.org and
start using the function. Wikifunctions removed a huge amount of potential
friction, making it easier to share and collaborate on a library of natural
language processing functions for languages that might not have had any
natural language processing libraries before.
Big thanks to Dolphyb <https://www.wikifunctions.org/wiki/User:Dolphyb> for
kicking off the creation of the Igbo functions! You can find the current
catalog of Igbo functions
<https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue#Igbo> on
Wikifunctions.
Function of the Week: Suffix a verb for an Igbo imperative form
Having talked about the Igbo imperative, I remain with this function, the first
morphological function
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-08>
selected
as Function of the Week. We have discussed what the function does above, so
let’s dive here into how it does it.
<https://www.wikifunctions.org/wiki/File:Igbo_imperative.png>Screenshot of
the function for the Igbo imperative
The first thing to notice is that the function has a label and description
in English, but not in Igbo itself! It would be great to give it a label in
Igbo (and I know I am being a hypocrite here, because the morphological
functions I have created for Croatians also don’t have any Croatian
labels). By giving the functions labels in their own language, we make
Wikifunctions more self-sufficient, and allow for less code switching when
using the system.
The function has two tests, 'gụ' to 'gụọ'
<https://www.wikifunctions.org/view/en/Z11376> (to read/ read!) and ‘ri’ to
‘rie’ <https://www.wikifunctions.org/view/en/Z11377>. I couldn’t confirm
the latter being a valid pair of a verb and its imperative, though, which
would make it a potentially bad test. There is one implementation in
JavaScript <https://www.wikifunctions.org/view/en/Z11354>, which passes
both tests. But by looking at the implementation, we can see that it is
quite complex, and that the two test cases don’t cover the many different
cases the implementation covers. The tests really undersell the
implementation, as the implementation is quite more powerful than what we
would be able to tell from the tests.
By having Tests, we increase our confidence in the Implementations that
pass those tests. Given that we are dealing with a wiki, implementations
can change, and failing tests should give us quick feedback that something
is off and that we should negotiate between the tests and the
implementation. Having more than one implementation can help with that too.
Tests can also play the role of documentation, demonstrating what kind of
values we support and what outputs the function will give on the given
values. This can capture edge cases, or standard values, and should capture
both. If an implementation has many different paths, the different tests
should aim to cover as many of the diverging paths as possible.
The third role tests can play is to define scope and expectations for a
function when it is first proposed and has no implementation yet. It should
be easy (and indeed, we need to work on making this easier) to add new
Tests to a function, including for contributors who are not coders. Igbo
speakers should be able to add more Tests for this function, as long as
they are following the regular paradigm for imperatives. In turn, these
Tests will help the contributor writing the implementation by checking
their implementation and ensuring it works correctly.
This function gave me a really great moment when presenting Wikifunctions!
I am very thankful to the Igbo community, and we are hoping that more
members of the community will embrace the creation of regular morphological
functions for Igbo, be it in the form of tests, implementations, or
maintaining the catalog of functions for Igbo.
Recent Changes in the software
The biggest change this week is the correction of our 'robots' policy when
viewing a page on Wikifunctions. Due to the special way we implemented
Wikifunctions, which allows for every language to get a 'real' link, we had
accidentally blocked robots and other search engines from indexing most of
our content. We've now corrected this (T355441
<https://phabricator.wikimedia.org/T355441>), which means that Bing,
Yahoo!, Google, and other places should slowly start to index our functions
for the first time (T353993 <https://phabricator.wikimedia.org/T353993>).
Beyond this, our planning has helped us focus on a few big tasks, which we
will share next week.
