The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-04-19
--
Welcome, Sharvani
We are happy to welcome Sharvani Haran to the team! Sharvani is currently
an engineer on the Mobile Apps team, and joining us temporarily to explore
the role of the Product Manager. I will let Sharvani introduce herself in
her own words:
<https://www.wikifunctions.org/wiki/File:Mysore_palace_illuminated.jpg>Mysore
palace
"I come from Mysore, India. Being born and raised in this city of palaces,
world-renowned for its Dasara celebrations
<https://en.wikipedia.org/wiki/Mysore_Dasara>, I have always been in awe of
India’s ancient wisdom, diverse culture and peace-loving temperament. I'm
fascinated by different cultures and traditions - marveling at our
similarities and celebrating our differences.
Growing up with my head in books, I spent all my childhood imagining all
things that were so foreign to me like the British cultural references in
‘The Famous Five’, or stepping into Huckleberry Finn’s St. Petersburg, or
wondering how children of the world would feel if they read ‘Malgudi days
<https://en.wikipedia.org/wiki/Malgudi_Days_(short_story_collection)>’ and
its cultural references to India. I have been a lifelong fan of literary
exchange.
Graduating as a computer science engineer, I have had the opportunity to
work on various interesting projects in diverse roles for 18 years. This
amazing adventure on the Abstract Wikipedia team feels like a natural final
destination for all such linguistic and literary passions that have been a
part of me. Thrilled to be here with an amazing team of people who are
trying to make knowledge sharing easy and seamless across languages."
You already met Sharvani when she had taken care of Newsletter #150
<https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-04-03>
two
weeks ago. Please join me in welcoming Sharvani on the team!
Recent Changes in the software
This week has been one of our regular "Fix-It" weeks, where we mostly focus
on technical debt and smaller feature improvements.
When you get an error when publishing an edited or new Object, we now show
the error's message and its context in a simpler way, so it should be
easier to understand, especially when multiple errors are returned (T345557
<https://phabricator.wikimedia.org/T345557>).
The internal APIs which filter by language now list and validate the
requested language against all the known languages to Wikifunctions, rather
than giving no guidance or validation (T362193
<https://phabricator.wikimedia.org/T362193>).
We improved our test coverage for several of our custom base Vue components
(T358207 <https://phabricator.wikimedia.org/T358207>), and corrected an
issue with our test code to ensure it works with asynchronous DOM actions (
T358204 <https://phabricator.wikimedia.org/T358204>).
Finally, we created a new language, now supported by MediaWiki:
Z1229/zgh-latn <https://www.wikifunctions.org/view/en/Z1229>.
Team meeting next week
Next week, the Abstract Wikipedia team is going to have our annual team
meeting. This time we are meeting in the office of the Wikimedia Foundation
in San Francisco. Among other things, we will be planning the next few
months of work. Because of the meeting, we are going to skip the regular
update next week, and will report on the results of our planning afterwards.
Function of the Week: Encode in Morse code
In the 1840s, electrical telegraphs became the standard system to allow for
fast communication. Two stations, connected with an electrical wire, would
send electric pulses along the wire. In order to be able to send and
receive messages, the stations had to agree on how to turn messages into
pulses, and how to interpret a series of pulses as messages again.
<https://www.wikifunctions.org/wiki/File:International_Morse_Code.svg>
For this, Morse code <https://en.wikipedia.org/wiki/Morse_code> was
developed. It consists of two types of pulses, a short and a long one, and
breaks between the letters as well as longer breaks between the words. The
encoding was meant to roughly support efficiency in English: letters that are
frequent in English <https://en.wikipedia.org/wiki/Entropy_coding>, such as
E or I had short codes, letters that are less frequent, such as Q or J, had
longer codes.
There are a number of slightly different codes. Wikifunctions provides
function Z10944 for the international Morse code
<https://www.wikifunctions.org/view/en/Z10944>. Enter a text, using the 26
letters of the English alphabet plus the letter é, and you get the Morse
code back: for example, the word Wiki will be encoded as ".-- .. -.- .." -
a pattern you might recognize in the Wikidata logo. Besides the letters,
the international code also covers some punctuation characters and digits.
The function has two implementations, one in Python
<https://www.wikifunctions.org/view/en/Z10945> and one in JavaScript
<https://www.wikifunctions.org/view/en/Z10968>. Both implementations
consist of a dictionary respectively a map with all the covered codepoints,
and some short code that uses these on the input. A total of seven tests
cover a number of different cases;
- A simple input such as Hello World
<https://www.wikifunctions.org/view/en/Z10947>
- The whole set of letters, both uppercase
<https://www.wikifunctions.org/view/en/Z10969> and lowercase
<https://www.wikifunctions.org/view/en/Z10946>
- Digits <https://www.wikifunctions.org/view/en/Z10948>
- Punctuation marks <https://www.wikifunctions.org/view/en/Z10949>
- Dropping characters which are not covered
<https://www.wikifunctions.org/view/en/Z10950>
- Collapsing multiple spaces
<https://www.wikifunctions.org/view/en/Z10951>
Wikifunctions has also function Z10956 for the inverse operation, decoding
Morse code <https://www.wikifunctions.org/view/en/Z10956>. It can be used
to take a text in Morse code and turn it back into the usual letters. ....
.- ...- . / ..-. ..- -. / .- -. -.. / .-- . / .- .-. . / -... .- -.-. -.- /
.. -. / - .-- --- / .-- . . -.- …
A new API for calling Wikifunctions
This week we are happy to announce the introduction of a new API to call
functions on Wikifunctions! The MediaWiki Action API for Wikifunctions was
extended by a public endpoint to run functions
<https://api.wikimedia.org/wiki/Wikifunctions_API/Reference/Run_Function>.
This allows external tools to call Wikifunctions in order to run a function
with the given arguments, and get the result back.
The new action should mostly work as a drop-in replacement for the existing
internal wikilambda_function_call API action. If there are any tools
currently using the internal action, we ask you to switch over to the
public one, as it’s more stable, reliable, and performant. We plan to
continue swiftly evolving the internal API only in ways that support the
front end better, without announcing such changes or offering guarantees of
stability. Changes to the external API on the other hand will follow
MediaWiki’s
usual policies <https://www.mediawiki.org/wiki/Stable_interface_policy>. We
also documented the new API in the API Portal <https://api.wikimedia.org/>,
in the Wikifunctions API Reference
<https://api.wikimedia.org/wiki/Wikifunctions_API/Reference>.
If you like to get support with switching the API, we are happy to provide
help.
As with everything Wikifunctions at this early stage, we ask that you use
the API gently and with consideration, and to allow us to slowly grow into
the most useful version the project can be.
The first 1000 function
<https://www.wikifunctions.org/wiki/File:Jarnioux_-_Num%C3%A9ro_de_rue_1000_…>
As of Saturday, 6 April 2024, Wikifunctions has grown to 1000 functions!
Congratulations to the community! We present the 1000th function in the
section about the Function of the Week below, here we look at a few
statistics around the functions as we have them so far.
Sixty of the functions come in the predefined space. Those are validators
for the predefined types and a small set of functions with builtins.
Of the first 1000 functions,
- 685 deal with strings
- 223 deal with Booleans
- 225 deal with numbers
- 173 deal with lists and other types
Since a function can be in more than one group, the numbers won’t add up to
a thousand.
As we can see, the majority of our functions are dealing with strings,
which is great for a system that aims to become a comprehensive natural
language generation library. According to the count in our catalogue
<https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue#Natural_language…>,
we currently have functions for 20 languages.
We are excited about this early milestone, and are looking forward to
develop into a comprehensive library of functions to support Wikipedia, the
other Wikimedia projects, and beyond.
Senior Frontend Engineer position open for Wikifunctions
Are you a front end or full stack engineer and want to work with us on
Abstract Wikipedia and Wikifunctions? Then we are a looking for you! Check
out our job description on Greenhouse
<https://boards.greenhouse.io/wikimedia/jobs/5850672>.
Volunteers’ Corner recording is available on Commons
The recording of the last Volunteers’ Corner
<https://commons.wikimedia.org/wiki/File:Abstract_Wikipedia_Volunteer_Corner…>
is
now available for everyone to see on Wikimedia Commons. Thanks to all who
participated!
Recent Changes to the software
This week, we made some follow-up tweaks to the code for the new, public
API (T360359 <https://phabricator.wikimedia.org/T360359>). We adjusted our
example API call for the public to use the shorter, simpler, more familiar
'canonical form', rather than the more expansive 'normal form'.
We deployed the latest version of the back-end services, which we hope will
have fixed a number of minor issues, and helped us move forwards with other
improvements.
We improved the error-handling when using Python to wrap things in
user-readable errors in some further cases (T356715
<https://phabricator.wikimedia.org/T356715>). We no longer hard-code the
internal label for what kind of evaluation is used, but show Z14K2 or
similar (T358571 <https://phabricator.wikimedia.org/T358571>); there is
more work to do to label this nicely. We reduced the level of expansion in
argument calls, which should speed up execution in some cases (T360172
<https://phabricator.wikimedia.org/T360172>). Similarly, we switched the
expansion of Code objects to be ephemeral, so they're returned only by
reference (T359872 <https://phabricator.wikimedia.org/T359872>). Finally,
we fixed an unnecessary expansion of Types in lists (T324661
<https://phabricator.wikimedia.org/T324661>). Collectively, these three
prompted further review of how we can simplify responses (work forthcoming).
We adjusted how the front-end handles API errors (T361598
<https://phabricator.wikimedia.org/T361598>), and consolidated the code
into a central place (T315432 <https://phabricator.wikimedia.org/T315432>).
We changed how the object selector filters out 'disallowed' values so that
it can be used in certain cases which we previously unintentionally
prohibited (T336292 <https://phabricator.wikimedia.org/T336292>).
We tweaked our metrics monitoring code to not emit null values, so that the
new Metrics Platform from our colleagues won't reject the logs (T350497
<https://phabricator.wikimedia.org/T350497>). We simplified some of our
browser tests to be simpler and less likely to break as the code changes (
T349836 <https://phabricator.wikimedia.org/T349836>), and abstracted the
internationalisation label checking so it won't break tests when labels
update; the newly-fixed tests are now part of the daily browser test
regression monitoring suite, as well as the regular CI.
Function of the Week: Padovan number
<https://www.wikifunctions.org/wiki/File:Padovan_triangles_(1).svg>
Since numbers have become available, a lot of new functions have been
created for them. And some of them leave me with mixed feelings, such as
the Function of the Week for this week. I chose it because it is the 1000th
function: Padovan number (Wikifunction Z15075
<https://www.wikifunctions.org/view/en/Z15075>, Wikipedia article
<https://en.wikipedia.org/wiki/Padovan_sequence>).
Named for Richard Padovan <https://en.wikipedia.org/wiki/Richard_Padovan>,
an European architect, the Padovan numbers are a sequence of numbers where
each number is the sum of the two numbers in the sequence preceding the one
before the number we are looking for. That is, the tenth Padovan number is
the sum of the eighth and seventh numbers, the two numbers before the
ninth. That renders the definition quite similar to the far more
famous Fibonacci
sequence <https://en.wikipedia.org/wiki/Fibonacci_sequence> (Wikifunction
Z13835 <https://www.wikifunctions.org/view/en/Z13835>). The Wikipedia
article discusses the sequence in further detail.
Currently, the function offers three tests and two implementations. The
three tests define the zeroth Padovan number as 1
<https://www.wikifunctions.org/view/en/Z15077>, the tenth number as 3
<https://www.wikifunctions.org/view/en/Z15078>, and the hundredth number as
288,627,200,960 <https://www.wikifunctions.org/view/en/Z15079>. As the
Wikipedia article discusses, there are several ways to start the sequence,
and the tests specify how the sequence starts, which follows On-Line
Encyclopedia of Integer Sequences
<https://en.wikipedia.org/wiki/On-Line_Encyclopedia_of_Integer_Sequences>
sequence A000931 <https://oeis.org/A000931>, but is, interestingly,
different from what the Wikipedia article in any of the existing language
editions suggests (by being shifted by five positions). Then we do
have the Padovan
spiral number <https://www.wikifunctions.org/view/en/Z15085>, which aligns
with Wikipedia's definition, and shifts the number accordingly.
*n* *0* *1* *2* *3* *4* *5* *6* *7* *8* *9* *10*
*Wikipedia* 1 1 1 2 2 3 4 5 7 9 12
*Wikifunctions* 1 0 0 1 0 1 1 1 2 2 3
The two existing implementations are both written in Python, and represent
two different ways to calculate the number:
- The recursive implementation
<https://www.wikifunctions.org/view/en/Z15076> following the definition,
and
- A sum of binomial coefficients
<https://www.wikifunctions.org/view/en/Z15091>, as described on the
Wikipedia article
You may ask “what is this function good for?”, and in my opinion these
functions dance around the border of being useful for Wikifunctions. This
is ultimately a discussion that we have to have as a community. I don’t
expect this function to be terribly useful for Abstract Wikipedia, but I
could see that eventually we could interactively embed this function in the
Wikipedia article for the Padovan sequence itself, allowing for a better
experience of our favourite encyclopaedia. I could also see us postponing
the decision of how to define what belongs into Wikifunctions and what does
not for a little bit longer. Or making a decision now that we revise after
a while, with more experience. I am all for allowing a young project such
as Wikifunctions to have a bit more space to breathe in these early days,
to have a bit more space for fun, for exploration. And I certainly don’t
want to make a call at this point in time already about how to circumscribe
the functions which belong in Wikifunctions, but it will be a discussion
and a decision that we will have to face at some point.
I am looking forward to that discussion with curiosity and an open mind.
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-03-28
--
Creating tests is now much easier!
<https://www.wikifunctions.org/wiki/File:Autofilled_tester.png>Automatically
filled Test case for the to lowercase
<https://www.wikifunctions.org/view/en/Z10047> function
Test cases are a crucial part of defining what functions mean and ensuring
all Implementations stick to that definition. And yet, the creation of a
good test has been somewhat challenging:
1. Go to the Function and click on the plus in the box of Test cases
2. Select the Function call you want to test, using search
3. Enter the arguments for the call
4. Select the Function to check the result, using search
5. Enter the arguments for the check
The idea is that the Test works by running the call you have built in steps
2 and 3, and then check the result with the Function built in steps 4 and
5. If the latter check returns true, the test has passed. If it returns
false, the test has failed.
Selecting the right Function to check the result isn’t always
straightforward: you need to be aware of the result Type of the Function
and must either search for, or simply know, the right Function to check the
resulting value.
A few weeks ago, we extended Types with the ability to know their equality
function, *i.e.* what is the Function, if it exists, for testing whether
two values of that Type are equal or not.
With this week’s release, the process of creating Test cases has been
considerably simplified: you click on the plus button, and both the
Function for testing and the Function for checking the result are
automatically selected and filled-in for you. You can still go and change
it – for example, if the pre-selected checking Function is not appropriate,
and you need something else. But you can now just click on plus, enter all
relevant arguments, and publish the Test case.
Thanks to Geno <https://www.wikifunctions.org/wiki/User:Geno_(WMF)> who was
the engineer behind this Quality of Life improvement! We hope that this
will make writing Test cases faster for all, and especially that it will be
simpler and easier for newer community members to understand how to do the
right thing.
Recent Changes in the software
A big piece of work we've been doing this week has been to create a
properly-documented API designed for gadgets, tools, and third parties to
call to run Wikifunctions. Expect details soon! Alongside that, we've made
some feature improvements to make using Wikifunctions easier.
First, as mentioned above, when creating a new Test case, we now auto-fill
the test call to the target Function being tested and, where possible, the
validator's call to the returned Type's equality function (T358214
<https://phabricator.wikimedia.org/T358214>).
Second, when prompted to pick a Type or a Natural language, we now use a
new feature of the Codex UX library to give suggested options rather than
expecting you to know the name of one already (T350037
<https://phabricator.wikimedia.org/T350037>). For Types, if you click into
the field it will suggest string and boolean; for Natural languages, it
will suggest the "United Nations Six" languages. In the future, this may
support more Types, and allow on-wiki control of what options are
presented; please discuss this on-wiki if you have ideas.
In terms of bugs, we fixed an issue where we always expanded new items in
lists, regardless of its Type (T359576
<https://phabricator.wikimedia.org/T359576>). We fixed the special page
listing Objects to use the primary label if it's available, rather than an
alias (T358805 <https://phabricator.wikimedia.org/T358805>). We corrected
the logic that decided whether to show a 'chip' in the page's header for
the Object label and sub-title for its Type, using the relevant language
chain for each label (T360000 <https://phabricator.wikimedia.org/T360000>).
All Wikimedia-deployed code is using the latest version of the Codex UX
library, v1.3.5, as of this week. Though we do not believe the library
upgrade will have any user-visible changes in our use on Wikifunctions,
please comment on the Project chat or file a Phabricator task if you spot
an issue. We have also upgraded the Ace library (from 1.14.0 to 1.32.7)
that provides the editor and syntax highlighter when writing code
implementations (which is also used when editing site JS/CSS pages, user
scripts, and gadgets). Similarly, the upgrade should not make any
significant changes, but please do highlight to us if you notice any issues.
Upcoming Volunteer’s Corner on April 8th[edit source
<https://www.wikifunctions.org/w/index.php?title=Wikifunctions%3AStatus_upda…>
]
The next Volunteer’s Corner will be on 8 April 2024 at 22:30-23:30 UTC
<https://zonestamp.toolforge.org/1712615400>. We scheduled it to a
different time, to accommodate different people than usual. The timing is
for now a one-off. We are still working on the timing in order to
accommodate as many people as possible, and are considering a rotating
timing, possibly having three different times each quarter. We appreciate
feedback and will be taking note of attendance.
The agenda of the Volunteer’s Corner is as usual:
1. Off-the-record time to ask questions
2. Recording starts
3. Update on the project (also shared online afterwards)
4. Time for questions
5. Work together on a function
6. Time for questions
7. Recording ends
8. Off-the-record time to ask questions
We are looking forward to meeting you! Let us know if you would like to
attend but the times so far didn’t work out and which times would work for
you.
Guest editor next week
I’m taking a week off, but we will have a guest editor next week!
Function of the Week: NOR (Z10231)
I have been looking forward to writing about the NOR function (Wikipedia
article <https://en.wikipedia.org/wiki/Logical_NOR>, Wikifunctions function
<https://www.wikifunctions.org/view/en/Z10231>) for a while. In fact,
that’s why I previously introduced the NOT
<https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-22#…>
and
the OR
<https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-28#…>
functions.
Nor is the negation of the or function. Just like the or function, nor is a
binary Boolean function: it takes two arguments of type Boolean and returns
one. Since or returns true if either of the arguments is true, nor returns
true only if neither one *nor* the other argument is true.
Just as the other binary Boolean functions, Nor has four Test cases
(corresponding to Nor's truth table):
1. True NOR true is false <https://www.wikifunctions.org/view/en/Z10233>
2. True NOR false is false <https://www.wikifunctions.org/view/en/Z10234>
3. False NOR true is false <https://www.wikifunctions.org/view/en/Z10235>
4. False NOR false is true <https://www.wikifunctions.org/view/en/Z10236>
These tests are complete in Wikifunctions, meaning that every possible
combination of inputs is covered with the tests. We won’t have that luxury
for many other Types!
So, what’s so special about NOR that makes me excited about it? The
interesting thing about NOR is that any other Boolean function can be
implemented by using just the NOR function. No matter how many arguments,
all you need is the NOR function, possibly several times, and nothing else.
Of the sixteen binary Boolean functions, only two have this special
property.
I am not saying that it’s a good idea to only use NOR, but it is possible!
We could have implementation for all our Boolean functions using only NOR,
which can be an interesting educational resource. The same is true for NAND
<https://www.wikifunctions.org/view/en/Z10243> (not-and). For example, the
NOT <https://www.wikifunctions.org/view/en/Z10216> function can be
implemented by using NOR with the same value in both arguments
<https://www.wikifunctions.org/view/en/Z14580>.
The NOR function has four Implementations:
1. In JavaScript <https://www.wikifunctions.org/view/en/Z10611>,
combining the language's ! (not) and || (or) operators
2. In Python <https://www.wikifunctions.org/view/en/Z10279>, combining
the not and or keywords
3. One composition <https://www.wikifunctions.org/view/en/Z10232> using
NOT
<https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-22#…>
and OR
<https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-28#…>
4. Another composition <https://www.wikifunctions.org/view/en/Z13488> using
two if <https://www.wikifunctions.org/view/en/Z802> functions
It can be an interesting exercise to think through the last composition and
understand why it results in the NOR function.
I always thought that the name of the function, “nor”, came from combining
the words “not” and “or”, similar to how the “nand” function is named for
combining “not” and “and”, but it seems I got my direction wrong: nor
simply comes from the English word “nor
<https://en.wiktionary.org/wiki/nor#English>”, and was then reinterpreted
as “not or” in order to be a pattern for “nand”.
This update is available on-wiki here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-03-21
--
On the way to internationalizing numbers
<https://www.wikifunctions.org/wiki/File:Multiplication_without_renderer_set…>Multiplication
without renderer
We now have the ability to internationalize numbers in Wikifunctions by
implementing renderers and parsers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20>.
For now, this is only configured for natural numbers
<https://www.wikifunctions.org/view/en/Z13518>, but it should work for
future types out of the box.
However, there’s a caveat! It has not been tested at large scale yet, and
it currently runs into some speed issues. But let’s start with how it
should be working.
In the first screenshot, we see how the system used to work before, using
the example of the multiplication function: the instances of natural
numbers are expanded, displaying their type and their value as the string
types they use internally.
The second screenshot displays the status after we configured the renderer
(the display function) and the parser (the reading function) for natural
numbers. With these two functions configured, the UX collapses the values
into single lines, both for input (for the two arguments of the
multiplication) as well as output (for the result).
The configuration itself happens on-wiki, on the natural number type
<https://www.wikifunctions.org/view/en/Z13518>, where we can see that the
renderer is set to display natural number
<https://www.wikifunctions.org/view/en/Z14280> and the parser to read
natural number <https://www.wikifunctions.org/view/en/Z14290>. The right to
edit types will be handed over to the community at a later date, but the
configured functions are editable for you right now.
<https://www.wikifunctions.org/wiki/File:Multiplication_with_renderer_setup.…>Multiplication
with renderer
This makes the display for all functions using natural numbers much more
compact. But that’s not all! Both functions to display and read natural
numbers also take a natural language as a parameter, with the idea that
both displaying and reading the values can be adjusted based on the
language set in the user interface. As of sending this newsletter, this
argument is ignored, but things are almost all set up.
If you are not interested in gory technical details, you might want to skip
the rest of this section.
In order to display a number appropriately for a given language, we first
need to have a function to do so. On Wikifunctions, we already had a function
that formats a large natural number string by adding commas
<https://www.wikifunctions.org/view/en/Z13473>. But since it was defined on
Strings, we went ahead and recreated it for numbers
<https://www.wikifunctions.org/view/en/Z14295>, and made a little change
(more to initiate a discussion than to make an editorial decision), by
putting commas into the resulting display only if the number has more than
four digits. For the implementation, we basically copied
<https://www.wikifunctions.org/view/en/Z14301> the previous implementation
<https://www.wikifunctions.org/view/en/Z13474>.
This gives us a function to format the number for English texts. Now, we
need to somehow connect this function with the renderer function that is
set above. For that, we have created two new types that help us with
configuring which languages should use which function: a configuration of
functions for a given language
<https://www.wikifunctions.org/view/en/Z14294>, which uses the function
option for a list of languages
<https://www.wikifunctions.org/view/en/Z14293>. This has been used to
create a configuration for the display function
<https://www.wikifunctions.org/view/en/Z14302> and one for the reading
function <https://www.wikifunctions.org/view/en/Z14303>.
Furthermore, we need a function that selects the right function given a
language <https://www.wikifunctions.org/view/en/Z14310> from that
configuration structure. This is needed because different languages will
expect different ways of showing values – French and German will use dots
rather than commas as three-digit group separators, Indian English will use
commas like International English but in a different pattern than
three-digit groups, some languages might want to use non-Arabic digits for
familiarity, *etc.*. And finally, we need to apply
<https://www.wikifunctions.org/view/en/Z13036> the selected function to the
number. We discuss the apply function in the Function of the Week section
below.
But alas, as of now, this frequently times out. You can see the two
implementations connected to the display function
<https://www.wikifunctions.org/view/en/Z14280>: the simple one, that just
returns the unformatted string
<https://www.wikifunctions.org/view/en/Z14282>, and the implementation
trying to apply the appropriate function based on the language
<https://www.wikifunctions.org/view/en/Z14340>. As the latter randomly, but
rather frequently times out, it is currently disconnected, in order to
avoid disrupting the site.
What’s the call to action?
1. Create functions that display and read numbers as appropriate for
your language.
2. Add those functions to the configuration object for reading
<https://www.wikifunctions.org/view/en/Z14303> and display
<https://www.wikifunctions.org/view/en/Z14302>.
3. Help us speed up the system.
For the parsers, it is recommended to be lenient. If, for example, it is
common to enter numbers both in a language-specific script as well as with
Western Arabic digits, it is totally OK to allow for both. The current
default parser <https://www.wikifunctions.org/view/en/Z14304> allows for
commas and dots in any place, instead of ensuring that they split exactly
three numbers. It’s up to the community to make the editorial decision on
what they accept as input for a given language. A good quality check is if
the output of a Type's display function for a language is valid for the
reader function for that language – that it 'round-trips' without errors.
All of these are entirely new, early-stage mechanisms working on
Wikifunctions. We are curious to hear your feedback and to hear about what
works and what doesn’t. And also for creating more functions and for help
with getting this rolled out.
Recent Changes in the software
This week we worked on a few smaller features and bug-fixes, as well as
improvements to the back-end services. We adjusted how the new reading and
display functions for Types (parsers and renderers) are called, avoiding
confusing mis-readings of user input depending on how you clicked (T359987
<https://phabricator.wikimedia.org/T359987>). The edit link for old
revisions was mis-configured, and now actually points to the edit page for
that old revision, rather than the current one (T343653
<https://phabricator.wikimedia.org/T343653>); our apologies!
We addressed a missing feature in the table of Implementations on Function
pages, which showed only a '–' rather than the number of Test cases passed,
like '3/5' (T347536 <https://phabricator.wikimedia.org/T347536>). We
improved some of the Object conversion code, which should avoid mysterious
errors breaking the front-end experience in some cases (T357594
<https://phabricator.wikimedia.org/T357594>). We now allow empty strings to
be used in a function selector, unbreaking the experience in some edge
cases (T346006 <https://phabricator.wikimedia.org/T346006>).
We've improved the accessibility experience by adding longer, explanatory
titles for the three dialogs missing them using the 'aria-title' attribute,
and two icon-only buttons, one in the mode selector control, and the other
in the Function Explorer. We rewrote how fetching state is handled in the
front-end, which should handle pending status better when loading Test case
results (T360018 <https://phabricator.wikimedia.org/T360018>).
As part of our migration work for programming language references, we now
have a maintenance script that will update on-wiki content; we will run
this shortly (T287153 <https://phabricator.wikimedia.org/T287153>). We
audited our use of i18n and found a few messages that are no longer used;
their removal will reduce the burden on translators. Finally, we've started
tracking view actions on Function pages, so that we can better understand
who is using Wikifunctions and where we might need to improve (T357069
<https://phabricator.wikimedia.org/T357069>).
Ongoing discussion on identity
Last week we introduced a question on how to work with identity
<https://www.wikifunctions.org/wiki/Wikifunctions:Representing_identity>.
The discussion is ongoing, and the team will devote more time to the
discussion very soon.
Function of the Week: apply (Z13036)
Apply <https://en.wikipedia.org/wiki/Apply> is one of the most foundational
functions in computer science, and yet it is not available in a large
number of programming languages and systems, or only by going through
hoops. Apply, in the way it is defined in Wikifunctions,
<https://www.wikifunctions.org/view/en/Z13036> takes two arguments, a
function and a value, and applies the given function to the given value,
meaning that it runs the function with the given value as the argument.
One good way to understand the function is to look at the existing tests.
Tests always live a triple life:
1. They ensure that the implementations are working correctly.
2. They capture agreement and declare what the function is supposed to
do. If there are two possible ways a function could react, then tests can
be used to resolve that ambiguity.
3. They document for the reader what the function does by providing
examples.
In this case, we can benefit from the tests and read them as great examples
of what the apply function does:
1. The first test <https://www.wikifunctions.org/view/en/Z13047> applies
the reverse <https://www.wikifunctions.org/view/en/Z10012> function
(which was our first Function of the Week
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2024-01-11>)
to the string "abc", and it checks that we get "cba". Using apply with
reverse and "abc" is the same as calling reverse itself with the
argument "abc".
2. The second test <https://www.wikifunctions.org/view/en/Z13049> applies
another Function of the Week
<https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-22>,
not, to the value true, and checks whether we indeed get false. This shows
us that apply works across different types: it works both with strings and
Booleans.
3. The third test <https://www.wikifunctions.org/view/en/Z13347> applies
one of our earliest user-created functions, to uppercase
<https://www.wikifunctions.org/view/en/Z10018>, to the string "hello" and
checks that the result is "HELLO".
4. The fourth test <https://www.wikifunctions.org/view/en/Z14355> applies
the square function <https://www.wikifunctions.org/view/en/Z13663> to
the number 9, and checks that the result is 81. This demonstrates that it
also works with the new types, not just the initial, built-in types.
For now, this function can only be implemented as a composition, not in
JavaScript or Python, because we don’t yet allow code implementations to
call further functions.
The first implementation <https://www.wikifunctions.org/view/en/Z13048> was
created by 99of9 <https://www.wikifunctions.org/wiki/User:99of9>, and it is
entirely composed of built-in functions: it centers around the map
<https://www.wikifunctions.org/view/en/Z873> higher-order function (which
became famous as the first half of Google’s MapReduce
<https://en.wikipedia.org/wiki/MapReduce> technology), a function that
takes a list and a function, and applies the function to each element of
the list, resulting in a new list.
The rest of the function is taking the value and adding
<https://www.wikifunctions.org/view/en/Z810> it to an empty list, in order
to create the list necessary for the map function. Map is then applied to
that one-element list, and then first element
<https://www.wikifunctions.org/view/en/Z811> is used to get the resulting
value out of the list again.
To recapitulate: we take the value, put it in a list, map the list, and
then get the resulting value out of the list again. There’s a task to add
apply as a built-in function <https://phabricator.wikimedia.org/T357858>,
so that hop in and out of a list wouldn’t be necessary.
The second implementation <https://www.wikifunctions.org/view/en/Z14353> copies
the same approach, but skips the use of the function adding the value to an
empty list and instead creates a list directly with the value. I built it
because I expected it to be a bit faster, and hoped it would run the
display function faster.
The humor here is that I have previously scolded
<https://www.wikifunctions.org/w/index.php?title=Wikifunctions:Project_chat&…>
TomT0m <https://www.wikifunctions.org/wiki/User:TomT0m> and 99of9
<https://www.wikifunctions.org/wiki/User:99of9> for working on the apply
function and related functions, pointing out that we currently don’t have
great support for functions using functions. I would still prefer for these
things to be tried out on the Beta Cluster, where I had recreated their
apply function <https://wikifunctions.beta.wmflabs.org/view/en/Z13140>, but
to be entirely honest, without their work and their exploration I would
have been stuck trying to create the routing to the right function based on
the language. Thank you!
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-03-13
--
On identity <https://www.wikifunctions.org/wiki/File:Banana_-_Q503.jpg>A
banana with the QID for bananas. Note that Wikidata currently has no
individual banana <https://w.wiki/9TU7> described.
In the Wikifunctions data model, we are using the notion of "identity"
widely. For example, Types, Functions, and also the two Boolean values all
feature the idea that they have an identity.
But the concept of identity has so far not been an explicit part of the
data model, which prevented us from easily using it, such as in certain
kinds of new types where values are limited, or easily incorporate them
into a better user experience .
We would like to fix this issue for Wikifunctions, and add an explicit
notion of identity to the data model. We have written a document with four
different proposals, and would also like to gather input from the
community: did we overlook advantages or disadvantages of certain
proposals, are there even better ideas out there which we could implement
instead? What did we miss, what could we do better?
You can find the document representing identity
<https://www.wikifunctions.org/wiki/Wikifunctions:Representing_identity> here,
and it also touches on how to represent optional keys on types (which may
also pre-inform on how to represent optional arguments on functions). We
hope to respond to your thoughts and implement a decision in the coming
weeks and months.
Recent Changes in the software
Our big focus this Quarter has been on improving Types so you can use them
easily, on the path to accessing and using Lexemes and other content from
Wikidata. This week, we landed a big milestone around how Types can use
Functions to check inputs as you enter them, and shown based on their value
(T358041 <https://phabricator.wikimedia.org/T358041>). We will discuss
this, with demonstrations, on-wiki and in a later Update as new Types are
developed. You can already try it out on Wikifunctions Beta, for example to
see multiplication work for Roman numerals
<https://wikifunctions.beta.wmflabs.org/view/la/Z11695> (see screenshot).
Please try it out in the Beta and raise issues if you encounter them.
We've also made some improvements related to editing Types (T358136
<https://phabricator.wikimedia.org/T358136> & [[:phab:T358135|]T358135]);
we hope to make the interface good enough for the community to use and so
be able to open up access without risking the site integrity in the future.
Previously, when editing an old revision of an Object (*e.g.* to manually
revert), the 'Publish' button wasn't active until you made a change, like
for normal edits; we now detect this situation and make it active
immediately (T343654 <https://phabricator.wikimedia.org/T343654>). Last
week we tweaked the heading on each Object page to not repeat the type
between the title and the sub-title; this week, we've also removed the ZID
of the type from the sub-title, to make the interface simpler (T357805
<https://phabricator.wikimedia.org/T357805>).
On the back-end services, as well as internal changes, we've made some
fixes to simplify the return value when using generic types (T324661
<https://phabricator.wikimedia.org/T324661>), and to not hard-code an
English value for the meta-data value of what kind of Implementation is run
(T358571 <https://phabricator.wikimedia.org/T358571>). We've dropped an
old, now-unused API that was created to monitor the back-end services'
availability (T359179 <https://phabricator.wikimedia.org/T359179>). A few
Functions have mis-reported Implementations or Testers due to how they were
created and a bug in our code; we've made an adjustment to our code that
might fix this situation as the affected items are edited, but will
investigate further.
We made a fix to stop loading many copies of the style files on a page,
which should slightly speed up the site loading and operation for everyone (
T355919 <https://phabricator.wikimedia.org/T355919>). Last week we tweaked
the Function Evaluator to hide the 'no approved implementations' message;
we've made a follow-up bug-fix, so you can consistently use it again (
T346852 <https://phabricator.wikimedia.org/T346852>). Our apologies!
All Wikimedia-deployed code, including Wikifunctions, are using the latest
version of the Codex UX library, v1.3.4
<https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/refs/heads/…>,
as of this week. We don't expect any user-visible changes, so as always
please comment on the Project chat or file a Phabricator task if you spot
an issue or concern.
Wiki Mentor Africa recording available
The recording of the Wiki Mentor Africa event, that we cited in our last
weekly update, is available on Zoom
<https://us02web.zoom.us/rec/share/Jd1zfT2ADSzACJ8Fyrtn9-svAMxjgzcFjgWAt0n0a…>
(passcode: w%mj$Yp3).
Function of the Week: equality of natural numbers (Z13522
<https://www.wikifunctions.org/view/en/Z13522>)
Since we introduced natural numbers last week, 140 functions using natural
numbers have been created. And since we are talking about identity above,
we can look at a formally related notion, equality. This is why this week I
chose the equality of natural numbers. This function was started in last
week’s Volunteer’s Corner, and is thus one of the functions where we
have a video
of its beginning
<https://www.wikifunctions.org/wiki/File:Abstract_Wikipedia_Volunteer_Corner…>
.
When we talk about humans, there is a huge difference between identity and
equality. But when we talk about numbers, not so much. If a number is equal
to another number, then those two numbers are really the same number,
aren’t they? Maybe, maybe not. This question goes very deep into the philosophy
of mathematics <https://en.wikipedia.org/wiki/Philosophy_of_mathematics> and
the question on what a number <https://en.wikipedia.org/wiki/Number> actually
is. For other types than natural numbers, the question whether two
instances of that type are the same, whether they are equal, and whether
there is a difference between these two questions, can become more
interesting, and is really part of defining the type.
The same discussion about equality and identity becomes relevant for
the JavaScript
implementation <https://www.wikifunctions.org/view/en/Z13526>. JavaScript
also makes a difference between equality and identity: the former is tested
using two equal signs, ==, whereas the latter is tested using three equal
signs, ===. As long as we are comparing two BigInt values, it doesn’t make
a difference. But if we would compare a BigInt value with a Number value
using ===, it would always fail. And if we had chosen to turn BigInt into
JavaScript objects instead of BigInt values, === would also always fail. It
is the interplay between the converter to BigInt
<https://www.wikifunctions.org/view/en/Z13519> that we are using and
the implementation
of equality using === <https://www.wikifunctions.org/view/en/Z13526> that
makes it work.
In Python, we don’t have to make that decision, and just use an implementation
with the == operator <https://www.wikifunctions.org/view/en/Z13533>.
The composition <https://www.wikifunctions.org/view/en/Z14118> uses
the built-in
string equality <https://www.wikifunctions.org/view/en/Z866> on the strings
resulting from casting both values to strings
<https://www.wikifunctions.org/view/en/Z13713>.
The function has four tests:
1. Two equals two <https://www.wikifunctions.org/view/en/Z13523>
2. Two does not equal zero <https://www.wikifunctions.org/view/en/Z13525>
3. 9007199254740992 and 9007199254740993 are not equal
<https://www.wikifunctions.org/view/en/Z14043>
4. A leading zero does not make two values unequal
<https://www.wikifunctions.org/view/en/Z13524>
The first two tests are quite obvious, but what’s up with the other two
tests?
Test 3 looks like two random large numbers, one being one larger than the
other. But in JavaScript, if you type the following code into a JavaScript
console:
9007199254740992 == 9007199254740993
or
9007199254740992 === 9007199254740993
you will get the surprising answer:
true
This is one of the cases where the human reader is usually better than the
computer at Mathematics. Wikifunctions on the other hand gets this right,
as it is set up to deal with arbitrary large natural numbers, as we
discussed last week.
(The reason why JavaScript doesn’t get this right is because JavaScript
does not really have integers, but all numbers are floating point numbers,
and floating point numbers have a well-defined, but limited precision. That
is the same reason why in JavaScript 0.8 minus 0.1 does not result in 0.7,
but in 0.7000000000000001. Recently, JavaScript introduced the BigInt type
for precise arithmetics using arbitrarily large integers. This is why we
are using it as the value to which to convert the natural number type in
JavaScript.)
The last test lead demonstrates that we haven't yet connected the validator
with the type properly. As we have discussed in last week's Function of the
Week
<https://www.wikifunctions.org/wiki/Special:MyLanguage/Wikifunctions:Status_…>,
the validator should not allow for leading zeros.
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-28
--
Type proposal for natural numbers
<https://www.wikifunctions.org/wiki/File:Three_Baskets_with_Apples.svg>
We are getting close to having all the pieces in place to open up the next
type to the community: natural numbers! Natural numbers, that is the
numbers used for counting, are already in high demand, as witnessed by
functions such as the String length
<https://www.wikifunctions.org/view/en/Z11040> function.
We have prepared a type proposal
<https://www.wikifunctions.org/wiki/Wikifunctions:Type_proposals/Natural_num…>.
This type proposal is also meant as a template for future type proposals,
which should eventually be coming from the community, not the development
team. Input and discussion both of the format of the type proposal as well
as the actual proposal for natural numbers is welcome.
Note that we will launch numbers without immediate support for renderers
and parsers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20>,
which reduces the Wikifunctions community's ability to provide them in
translated form. This is temporary, and we are also working hard on getting
renderers and parsers implemented. That will allow Wikifunctions to display
and understand numbers in the way most appropriate for each language, but
it will not be available immediately when we launch natural numbers.
Recent Changes in the software
This week we have refreshed the design of the "meta-data" dialog which
opens when you click the "Details" control next to a function call result,
or the “info” icon (circle around “i”) next to a test result (T341127
<https://phabricator.wikimedia.org/T341127>). We now group the different
pieces of information together in collapsed "accordions", and summarise
some of the groups to help you understand them at a glance.
Thanks to GrounderUK, who found a mysterious bug that meant the pipe
character ("|") would be swallowed when trying to run tests on draft
Implementations or Test cases, but would work when saved. This was due to a
complication in MediaWiki's API handling, and we've worked around it for
now (T358089 <https://phabricator.wikimedia.org/T358089>). We found and
fixed a UX bug on the initialisation of blank custom objects, where only
the first key would be initialised with the correct type, while the others
were initialised as references (T355497
<https://phabricator.wikimedia.org/T355497>).
As part of our long-term vision for Wikifunctions, we want to support not
just multiple programming languages (currently Python and JavaScript, with
more to come later), but also provide a consistent, understandable way to
upgrade versions of these languages and help the Wikifunctions community
migrate – so we can spot *e.g.* code that works in ES2019 JavaScript
<https://www.wikifunctions.org/view/en/Z607> but needs to be improved to
also work in ES2020 JavaScript <https://www.wikifunctions.org/view/en/Z608>.
An important part of that plan is having consistent references for each
version, so we have been slowly improving our stack to use ZID references
rather than strings, and as of this week our UX now will understand both,
and new code will use ZIDs (T287153
<https://phabricator.wikimedia.org/T287153>). In future, we will use a bot
to mass-move all uses to just ZIDs.
We, along with all Wikimedia-deploy code like the Vector skin's search bar,
are using the latest version of the Codex UX library, v1.3.3, as of this
week. Though we do not believe the library upgrade will have any
user-visible changes in our use on Wikifunctions, please comment on the
Project chat or file a Phabricator task if you spot an issue. This version
of the library provides a new feature in how the lookup control works (
T350946 <https://phabricator.wikimedia.org/T350946>), which unblocks a
minor improvement to pre-populate the Object selector with common types,
which we hope to do soon (phab:T343564
<https://phabricator.wikimedia.org/T343564>).
Wiki Mentor Africa
<https://www.wikifunctions.org/wiki/File:WMA-wikifunctions_Flyer.jpg>
On this Saturday, March 2nd, at 16:00 UTC
<https://zonestamp.toolforge.org/1709395200>, Luca Martinelli and Denny
Vrandečić will present at an event in the Wiki Mentor Africa
<https://www.wikidata.org/wiki/Wikidata:Wiki_Mentor_Africa> program,
and on Sunday,
March 3rd, at 16:00 UTC <https://zonestamp.toolforge.org/1709395200>,
Hogü-456 will follow up with a hands-on demonstration. You can register for
that event on the Wiki Mentor Africa event Website for the Introduction to
Wikifunctions
<https://meta.wikimedia.org/wiki/Event:WMA_Introduction_to_Wikifunctions>.
A research agenda regarding AI and the knowledge commons
Last week, Denny attended a convening to discuss a research agenda
regarding the effect of current progress in machine learning on projects in
the knowledge commons, such as Wikipedia. The resulting draft for a
research agenda and more details on this event are published on Meta
<https://meta.wikimedia.org/wiki/Artificial_intelligence/Bellagio_2024>.
Function of the Week: Or (Z10184)
“Or <https://www.wikifunctions.org/view/en/Z10184>” is a binary Boolean
function. A binary function is one that takes two arguments. A Boolean
function is one where all arguments and the return type are Boolean, that
is, each of them is either true or false. A binary Boolean function is a
function that takes two arguments of type Boolean and returns a Boolean.
“Or” is also known as the logical disjunction
<https://en.wikipedia.org/wiki/Logical_disjunction>, and there are Wikipedia
articles <https://www.wikidata.org/wiki/Q1651704> about it in 46 language
editions of Wikipedia.
“Or” returns false only if both of the arguments are false. Otherwise it
returns true. This is meant to be a rigorous formulation of the natural
language use of “or”, which usually means that either one of the two parts
connected by the or may be given. Or both.
“Or” is useful to connect two conditions together. For example, if we want
to know if a word ends with a vowel
<https://www.wikifunctions.org/view/en/Z11961> or a y.
Since each Boolean argument can only have two possible values, and there
are two arguments, there are exactly four possible combinations of values
the function can take. Accordingly, there are four tests on the function: true
or true <https://www.wikifunctions.org/view/en/Z10192>, true or false
<https://www.wikifunctions.org/view/en/Z10195>, false or true
<https://www.wikifunctions.org/view/en/Z10198>, and false or false
<https://www.wikifunctions.org/view/en/Z10200>.
The function has three implementations:
- in Python <https://www.wikifunctions.org/view/en/Z10186>, using the
Python keyword or
- in JavaScript <https://www.wikifunctions.org/view/en/Z10201>, using
the || operator
- a composition <https://www.wikifunctions.org/view/en/Z13469> using the
if <https://www.wikifunctions.org/view/en/Z802> function: if the first
argument is true, return true, or else return the second argument
Think through the composition and see if you understand the logic: does it
indeed lead to the desired result for all four possible inputs?
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-22U…
the function model
A few weeks ago we started an update of the function model
<https://www.wikifunctions.org/wiki/Wikifunctions:Function_model>. The page
was set up early in the project and captured the original plan for the
function model, and was written prescriptively and aspirationally. The
actual implementation diverged from it in a number of ways. Some of these
divergences might be still aspirational, whereas others have evolved (*e.g.*
Benjamin arrays
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-07-29>)
and we have no intention to go back.
The page as it was before the updates was basically not useful, as it
represented neither the current status nor where we want to end up. We took
the time to update it from beginning to end, with the goal of representing
the function model as it is right now. It has now been updated to the
current state where appropriate, and aspirational content has been removed.
This is one step in improving our documentation. We think that the function
model is useful but not as useful as it could be: one set of feedback we
got was that it is hard to read and understand. We heard that and are
thinking about creating more documentation that is more accessible and can
support more paths for a diverse audience to find their way to
understanding Wikifunctions deeper if they want to do so.
Feedback and comments are welcome.
Recent Changes in the software
After last week's "Fix-It" drive, this week we returned to work on the
focus features as shared two weeks ago
<https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-07>,
as well as some smaller bug fixes. We continued our work on better support
for Types, with work on the way that user input will be transformed into a
value and *vice versa*
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20>,
which we hope to demonstrate soon, and then use in reality so that the next
Type can be created and used, on the way to interacting with Wikidata
lexemes and items.
We made a change to how the load of the software works for readers, which
means that we only load a small part of the Codex library before putting
the language control button on the page, rather than demanding the whole
library needed for rendering and interacting with Objects, which comes in a
few tenths of a second later. This should make browsing Wikifunctions
slightly faster for all readers and editors, especially for non-Object
pages. Our thanks to the Design System team for their work to make this
possible.
We re-wrote our code handling Wikifunctions Objects on the front-end, as it
could sometimes flake out and provide a broken user experience on the site;
we renamed and better documented the methods, and added a number of tests
to demonstrate the previously-broken, now-fixed behaviours (T352799
<https://phabricator.wikimedia.org/T352799>).
We fixed two bugs seen in production; one is an error check when processing
function calls in the API (T357691
<https://phabricator.wikimedia.org/T357691>), and the other is to fallback
to English when trying to change the display text of links to ZIDs and the
language isn't known (T357702 <https://phabricator.wikimedia.org/T357702>).
Spurred on by this, we also fixed the fall-back code when trying to access
a non-existent page with a non-English user language; previously this would
try to redirect you to the page called "Main Page" as translated in your
language, but now it will correctly send you to the actual main page in
your language.
We updated the credits file
<https://www.wikifunctions.org/wiki/Special:Version/Credits/WikiLambda> for
the MediaWiki part of the software, which adds Winston Sung (welcomed last
week).
Function of the Week: Negation
Negation <https://en.wikipedia.org/wiki/Negation> (not (Z10216)
<https://www.wikifunctions.org/view/en/Z10216>) is a unary Boolean
function. A unary function is one with a single argument. Boolean functions
are functions where all arguments are of type Boolean, *i.e.* they can be
either true or false, and the return type is also Boolean.
<https://www.wikifunctions.org/wiki/File:NOT_ANSI_Labelled.svg>The NOT gate
in electrical circuit diagrams
Negation takes one Boolean value and returns the other: if you ask for the
negation of true you get false, and if you ask for the negation of false
you get true. This is also how the composition
<https://www.wikifunctions.org/view/en/Z10537> is implemented: if
<https://www.wikifunctions.org/view/en/Z802> the value is true, return
false, else true. This is one pattern I really like seeing with one
composition: a composition that expresses what the function means in an
almost declarative way, without regards to how efficient the implementation
is. I am still thinking of a good name for these — maybe definitional
compositions? The usefulness of these definitional compositions is that
they serve both to explain what the function does, and also to give a high
confidence implementation to test the more efficient implementations
against.
Beyond the composition, there are three further implementations:
- In Python <https://www.wikifunctions.org/view/en/Z10538>, using the
keyword not
- In JavaScript <https://www.wikifunctions.org/view/en/Z10586>, using
the ! operator
- another one in JavaScript
<https://www.wikifunctions.org/view/en/Z10220> using the ! operator,
using a different syntax
I would recommend that as a community we come up with a JavaScript syntax
guideline that would suggest to only have one or the other JavaScript
implementation.
One interesting thing about Negation is that because the input types can
only have a closed, small set of different values, we can actually create a
complete set of tests, which test every possible input. In this case, in
fact, there are only two possible values: true and false. Accordingly we
have two tests: one that makes sure that the negation of true is false
<https://www.wikifunctions.org/view/en/Z10512>, and one for the opposite,
that the negation of false is true
<https://www.wikifunctions.org/view/en/Z10513>.
You can read much more about Negation on Wikipedia
<https://en.wikipedia.org/wiki/Negation>.
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-14
Recent Changes in the software: Fix-it week
As mentioned in last week's update, we had our regular "Fix-It Week", where
we work on technical debt, design oddities, and other issues that can pile
up when our main daily focus is on delivering larger-scale change for users.
One of main areas of change was in cleaning-up the Vue front-end code,
which has undergone a lot of change in the past few months. We improved our
front-end Vuex "store" code to remove now-unused methods and change the
remaining ones to be safer in edge cases (T329107
<https://phabricator.wikimedia.org/T329107>); we re-organised and split up
the 'mix-ins' in the Vuex store for consistency and better testability (
T328430 <https://phabricator.wikimedia.org/T328430>).
On the back-end services, we made a few improvements to code quality. One
of these was shifting the logging system from a global into our
"Invariants" class, so that it can be dependency-injected and so tested
better (T337785 <https://phabricator.wikimedia.org/T337785>). Our internal
ZWrapper class will now correctly error rather than silently continuing
when asked for a non-existent key (T309809
<https://phabricator.wikimedia.org/T309809>). We migrated the JavaScript
code coverage tool to use the built artefact pipeline rather than running
directly on the repo's code (T355815
<https://phabricator.wikimedia.org/T355815>). We also added pyflakes to our
CI, to improve our Python evaluator code quality, in the course of which we
found some bugs and added tests (T306365
<https://phabricator.wikimedia.org/T306365>).
We switched the Beta Cluster test version of the back-end services to use
the 'split' evaluator model that we rolled out to production last year (
T349008 <https://phabricator.wikimedia.org/T349008>), and so we were able
to decommission the old 'omnibus' image that would process both Python and
JavaScript code (T349557 <https://phabricator.wikimedia.org/T349557>).
After some checking to ensure we were content with the now-fixed results,
we added a test to demonstrate and enforce the current model of
error-handling in the WASM-wrapped JavaScript evaluator (T347898
<https://phabricator.wikimedia.org/T347898>).
We added a little further coverage of the PHP code that mirrors how Objects
work (T302599 <https://phabricator.wikimedia.org/T302599>), finding and
fixing a bug in how incoming results were interpreted in some
circumstances. We also found and fixed a forgotten remnant of a now-removed
piece of our test infrastructure that would cause errors when Wikimedia
production rolled forward to PHP 8.2.
Outside of proper "Fix-It" tasks, we also landed two improvements that will
roll out this week:
Following-on from last week's change to how string values are shown as
outputs, wrapping them in double quote characters, we now also do the same
for the abbreviated form, commonly seen on Test pages. This does not show
all whitespace, but it is a further minor improvement in this vein (T343608
<https://phabricator.wikimedia.org/T343608>).
Winston Sung, a Wikimedia community member very active in Chinese language
support, made their first contribution, fixing our use of whitespace on
each Object page's title headings in Chinese and some other languages (
T356731 <https://phabricator.wikimedia.org/T356731>). Thank you, and
welcome!
Function of the Week: duplicate string
Duplicate string <https://www.wikifunctions.org/view/en/Z10753> is a
function that takes a single string as the input, and returns a single
string as the output: the output is repeating the input twice.
<https://www.wikifunctions.org/wiki/File:Twins_11183454.jpg>
Duplication <https://en.wikipedia.org/wiki/Reduplication> of words or parts
of words, for various different effects, is a feature of many different
languages from all around the world. In some languages it may express
emphasis, in others a plural, but it can have many other semantic effects
too. The Wikipedia article
<https://en.wikipedia.org/wiki/Reduplication> provides
an overview of how duplication is used, with numerous examples. Here are
just two examples stolen from that page:
- In Finnish, *“ruoka”* means "food", but *“ruokaruoka”* means "proper
food", as opposed to snacks.
- In Swahili, *“piga”* means to “to strike”; whereas *“pigapiga”* means
“to strike repeatedly”
In Wikifunctions, duplicate string has five tests, WikiWiki
<https://www.wikifunctions.org/view/en/Z10754>, 1.11.1
<https://www.wikifunctions.org/view/en/Z10760>, ¨¨
<https://www.wikifunctions.org/view/en/Z10763>, the space character
<https://www.wikifunctions.org/view/en/Z10756>, and the empty string
<https://www.wikifunctions.org/view/en/Z10755> (a test that display
correctly as of today, thanks to the fix-it tasks of this week). Those are
great tests, trying out different kinds of characters and inputs. I
particularly enjoyed the empty string as an edge case. What I am missing
are non-English alphabet characters such as Hangul characters,
left-to-right scripts, or emojis. But this is a great set already.
There are four implementations, of which three are unsurprising: JavaScript
<https://www.wikifunctions.org/view/en/Z10759>, Python
<https://www.wikifunctions.org/view/en/Z10758>, and a composition
<https://www.wikifunctions.org/view/en/Z10757> using join strings
<https://www.wikifunctions.org/view/en/Z10000>, all of which are calling
the argument twice. The fourth implementation
<https://www.wikifunctions.org/view/en/Z10913>, another composition, is
using the function duplicate string n-times
<https://www.wikifunctions.org/view/en/Z10911>, which is a great function
to be used here, but requires a number as one of its arguments, but uses a
string (set to the string “2”, for the given composition). Once we actually
have a number type, this would need to be cleaned up.
All in all, this is a neat functionality, which is usually not available
through a standard library, and can exemplify tests and different
implementations.
The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-07
--Quarterly planning
Two weeks ago we held an internal planning meeting to sketch out what we
want to work on in the current quarter. Last week we collected and wrote
down the results, and we want to share those results with you.
<https://www.wikifunctions.org/wiki/File:Planning_icon_from_Noun_Project.png>
Our team's overarching goal is to work towards supporting the pieces needed
for Abstract Wikipedia.
- *Type support for Wikidata prototype*. We want to add more types to
Wikifunctions, in particular with an eye on the types needed to be able to
integrate forms from lexicographic data on Wikidata. For this, we want to
work on serializers and deserializers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-27>,
renderers
and parsers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20>,
as well as validators. The types we expect to enable are, besides
the recently
enabled lists
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2024-01-03>,
numbers, enumerations (for, *e.g.* grammatical features), and lexemes.
We will work together with the community on the exact list of types to
support, but in the end we hope to have all types in place so that in the
next quarter we can work towards accessing lexemes from Wikidata.
- *Public API for Wikifunctions*. We want to encourage the creation of
third-party apps using functions from Wikifunctions, such as the
integration with Lucas Werkmeister’s Wikidata Lexeme Forms
<https://www.wikidata.org/wiki/Wikidata:Wikidata_Lexeme_Forms> which is
currently using an internal API. The first step is to design the API and
decide on our approach.
- *Simplify our end-to-end tests and improve the reliability*. Our stack
is a bit complicated compared to other systems at Wikimedia, running an
evaluator and orchestrator services on the backend as well as the wiki
itself with the Wikilambda extension, and relying on certain content in the
wiki. Because of this, parts of our continuous integration test platform
are currently not in a great shape. This task is about improving that
situation and increasing our confidence in our tests.
- *Research for calling Wikifunctions inside Wikipedia content*. One
major goal of our project is to allow the Wikipedia and other Wikimedia
communities to call functions from Wikifunctions within wiki pages, like
they do with Commons media files or local templates and Lua scripts. In
order to prepare for this we will be starting user research to get a better
understanding of user needs and expectations.
- *Develop a baseline understanding of our users and their motivations*.
To make sure that we are developing Wikifunctions in a user-informed way,
we want to make better analysis of who our current users are and why they
are here based on the metrics we have so far.
- *Establish a regular cadence of external communication*. Our weekly
newsletter is good to communicate with our immediate community, but we want
to reach out to a wider audience as well. This will include regular posts
on Diff <https://diff.wikimedia.org/>.
- *Develop a framework to measure the narrative difference between two
Wikipedia articles in different languages*. This is a research task that
will take us on the path towards understanding how any two articles in two
different languages about the same topic differ. Such a tool would help us
with providing guidance for growing Abstract Wikipedia, and at the same
time it could potentially provide us with an early warning system that
might indicate a loss of diversity.
As this was our first planning effort for such a cadence, part of it is
also learning to pick the right size of work. At the end of the quarter we
will look back at what we actually achieved, and will learn from that for
the next quarter.
Thank you, Quiddity!
<https://www.wikifunctions.org/wiki/File:Nick_Wilson_at_Wikimania_2023.jpg>Nick
"Quiddity" Wilson at Wikimania 2023
Last week saw Nick Wilson’s last day on the Abstract Wikipedia team, before
he heads into a well-deserved break. Nick may to some of you be better
known under his wikinym “Quiddity”. Nick was part of the Abstract Wikipedia
project from the very beginning, set up and organized the naming contest
for Wikifunctions, and set up most of our communication structures.
Here are a few words by Nick as a good bye message:
*Thank you, all!*
*It has been fascinating, humbling, motivating, and deeply inspiring to be
a part of these complex projects. I've appreciated getting to know (or
become more deeply familiar with) so many aspects of our movement and our
mission, and more of the people that steadily help everything to move
forwards.*
*As always, I wish I could clone myself in order to continue focusing on
this project forever, in addition to working in new areas. I'm eagerly
looking forward to a function-powered future, and I hope to continue
helping in a volunteer capacity after I take a short wikibreak.‘*
Luca, who joined us in early 2022
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-01-21>,
will continue in the role of Community Relations Specialist on the project
and take over Nick’s responsibilities. Please join me in thanking Nick for
his work!
Recent Changes in the software
As discussed above, part of our bigger work this quarter is type support
leading to Wikidata support. As part of this, we've fixed and now
re-enabled validation for types (T352295
<https://phabricator.wikimedia.org/T352295>), which shouldn't have any
visible effect yet but will unlock the ability to tell the user their input
is wrong when trying to use a type (such as saying that a natural number
cannot have non-digits in it, or a date in February cannot be on the 30th
day).
We've now made a small tweak to the display of strings returned from
functions, wrapping them in double-quotes, which makes it possible to
distinguish "" and " " as different responses. This does not fix all issues
with spaces in inputs, as the nature of HTML makes some possible return
values hard to cope with in a scalable way that doesn't break other use
cases. We plan to come back to this area with further improvements, and
would love your thoughts on improvements you'd like to see.
We've also fixed the interface breaking when using a language that
MediaWiki doesn't support, such as
https://www.wikifunctions.org/view/elx/Z10000 – previously you would end up
in a broken state, but now things work as designed (phab:T356428
<https://phabricator.wikimedia.org/T356428>). This also means that if you
try to use a language that even Wikifunctions doesn't know about, such as
https://www.wikifunctions.org/view/hellothisisnotalanguage/Z10000, the
system will fall back to English (like the rest of MediaWiki) rather than
mysteriously providing a mixture of English and mostly bugs.
In smaller changes, one is a follow-up to last week's change to remove the
noindex directive from our pages so that Google and other search engines
see them (phab:T355441 <https://phabricator.wikimedia.org/T355441>).
Alongside the main links like https://wikifunctions.org/view/ar/Z801, we
still have some URLs that also work due to how MediaWiki controls pages,
like …/wiki/Z801?uselang=ar <https://wikifunctions.org/wiki/Z801?uselang=ar>,
or …/wiki/Special:ViewObject/ar/Z801
<https://wikifunctions.org/wiki/Special:ViewObject/ar/Z801>; these will now
no longer claim to be "canonical" (T355546
<https://phabricator.wikimedia.org/T355546>). Another is that we've moved
all the linked software help pages on MediaWiki.org to be in the same
place, in the "Help:Wikifunctions" hierarchy. We've also made a tweak to
the UX of the language selector on a page when it has a number of
translations already – the padding is now in line with the rest of the
form, instead of much wider (T355946
<https://phabricator.wikimedia.org/T355946>).
We also added two new languages that MediaWiki supports: Ebira (Z1920)
<https://www.wikifunctions.org/view/en/Z1920> and Petjo (Z1921)
<https://www.wikifunctions.org/view/en/Z1921>. These are added manually in
production.
On the code side, we've made some minor improvements to our PHP code's
coverage of the critical ZObject classes, edging up towards 100% (T302599
<https://phabricator.wikimedia.org/T302599>). We've made a series of
clean-ups to our front-end code following-on from previous work, removing
unused components (T301868 <https://phabricator.wikimedia.org/T301868>) and
generally improving our data store code (T329107
<https://phabricator.wikimedia.org/T329107>). This week is a "Fix It" week
for the team, so we'll make further such improvements that will ship next
week.
Function of the Week: is permutation
For this week’s Function of the Week I asked Nick for a favorite function,
and he immediately answered that he loves lists and the unusual, and so he
would like to choose an unusual function that deals with lists. Looking at
the catalog of list functions
<https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue#List_operations>,
most of them were what you would expect from list operations, but one of
them struck us as unusual in the sense that most programming languages
would not have support for that functionality out of the box: is permutation
<https://www.wikifunctions.org/view/en/Z12741> (please feel free to correct
us).
<https://www.wikifunctions.org/wiki/File:Permutations_RGB.svg>Permutations
of three colors
A permutation <https://en.wikipedia.org/wiki/Permutation> of a list is a
second list that has exactly the same elements, but possibly in a different
order. The function is permutation
<https://www.wikifunctions.org/view/en/Z12741> takes two lists and checks
if they are permutations of each other. The function returns a Boolean
value: true if the two lists are permutations of each other, and false if
they are not.
In general, I have a bit of an issue with a function like this one: the
function is defined on lists of objects, and not on lists of a specific
type, such as a list of strings, or a list of Booleans. But not all types
will necessarily have a function to check for equality, which will be a
necessary prerequisite for this function to work. So this might lead to
problems down the road. I am not sure how to handle this best.
The function currently has five tests: three pairs that are not
permutations – (a, a, a) / (a, b, c)
<https://www.wikifunctions.org/view/en/Z12743>, (b,c) / (a,b,c)
<https://www.wikifunctions.org/view/en/Z12744>, and (b,b,a) / (a,a,b)
<https://www.wikifunctions.org/view/en/Z12747> – and two pairs that are
permutations, (b,c,a) / (a,b,c)
<https://www.wikifunctions.org/view/en/Z12742> and (a,b,a) / (a,a,b)
<https://www.wikifunctions.org/view/en/Z12746>. It’s great to see five
tests! I would also go for some edge cases (what about empty lists?) and,
given that the function is defined on lists of any type, also for lists
that have more than single-character strings as elements. But again, yay
for five tests! With five tests, the function is in the top 10% of
functions per number of tests.
The function currently has three implementations, one in Python and two
compositions. The Python implementation
<https://www.wikifunctions.org/view/en/Z12869> has two steps: it first does
a short-cut, checking if the two lists have different lengths. If they do,
it stops the implementation and returns False: two lists that have
different lengths can never be permutations of each other. Once we know
they have the same length, we compare whether the two lists, when sorted,
are the same. If they are, the two lists are permutations of each other, if
not, they are not. The sorting turns each of the input lists into a
canonical version with regards to its elements that is independent of the
order, which is why this implementation works. In theory, we could also
drop the first step from this implementation, but having the check probably
allows for some speed-up.
One of the compositions works by using supersets
<https://www.wikifunctions.org/view/en/Z12869>: we have a function, is
superset <https://www.wikifunctions.org/view/en/Z12846>, that checks
whether one list is a superset of the other. The composition here uses is
superset to check whether the first list is a superset of the second, and
the second is a superset of the first. If both conditions are fulfilled (we
check that by using and <https://www.wikifunctions.org/view/en/Z10174>), we
know that both lists contain all the elements of the other list, and
therefore contain the same elements, and thus they must be permutations of
each other. Two of the tests time out, but unfortunately with an unusable
error message. A bug has been reported (T356556
<https://phabricator.wikimedia.org/T356556>).
The other composition <https://www.wikifunctions.org/view/en/Z12867> is
more elaborate:
- First it checks if the two lists have equal lengths
<https://www.wikifunctions.org/view/en/Z12864>. If not, it returns false.
- If they do, it checks if the first list is empty
<https://www.wikifunctions.org/view/en/Z813>. If it is, it returns true
(because we know that the second list also must be empty, because of the
previous check, and empty lists are permutations of each other).
- Next, we recursively <https://en.wikipedia.org/wiki/Recursion> call is
permutation itself. For the recursion step, we find and remove
<https://www.wikifunctions.org/view/en/Z12856> the first element
<https://www.wikifunctions.org/view/en/Z811> of the second list from the
first list, and compare that to the second list without the first element
<https://www.wikifunctions.org/view/en/Z812>.
In this last step we remove the same element from both lists, and are thus
left with two shorter lists. If those are permutations of each other, the
two lists with the removed elements would also be permutations of each
other, no matter where those two elements are added in those lists.
If the first element of the second list does not exist in the first list,
the first list remains unchanged, a condition checked by this test
<https://www.wikifunctions.org/view/en/Z12859> on remove first matching
element <https://www.wikifunctions.org/view/en/Z12856> function.
The two compositions illustrate how different two compositions can be in
terms of complexity: the composition using superset is a very simple
composition, almost like a definition, combining just two existing
functions. The other composition is much more elaborate, combining quite a
number of different functions, using nested if conditions, and a recursive
call. The latter composition is probably about half-way through the
complexity level I would expect us to support for function compositions: I
wouldn’t expect much more complex compositions than that in Wikifunctions
at all. But, in the end, both compositions are great examples of how to use
composition for creating higher-level functions.
.jpg){kind=link}
.svg){kind=link}
