The on-wiki version of this newsletter is from now on to be found on
Wikifunctions, and not on Meta anymore. It contains a video that was
removed from here. The on-wiki version of this newsletter can be found here:
https://www.wikifunctions.org/wiki/Wikifunctions:Status_updates/2024-02-01
--
The Igbo imperative!
I was talking with Uzoma Ozurumba
<https://meta.wikimedia.org/wiki/User:Uzoma_Ozurumba>, a Community
Relations Specialist
<https://meta.wikimedia.org/wiki/Communications/Movement_Communications>
working
at the Wikimedia Foundation, and demonstrating Wikifunctions to her. Uzoma
is a founding member of the Igbo Wikimedians User Group
<https://meta.wikimedia.org/wiki/Igbo_Wikimedians_User_Group>. Igbo
<https://en.wikipedia.org/wiki/Igbo_language> is the principal language of
the Igbo people <https://en.wikipedia.org/wiki/Igbo_people> who live in the
South-Eastern part of Nigeria, and is spoken by more than 44 million
people. Igbo is also one of the five focus languages of Abstract Wikipedia
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-04-15>,
Wikifunctions, and Wikidata’s improvements to the lexicographic data.
Igbo is currently represented with more than 3,900 Lexemes in Wikidata,
making it currently the 34th language by number of Lexemes, before Punjabi
and Polish, and after Portuguese and Mandarin. In Wikifunctions, already as
early as September, a first function for Igbo was created, adding the
preposition ‘na’ to a word <https://www.wikifunctions.org/view/en/Z11248>.
When I was talking with Uzoma, I was showing her another function, which
takes the base form of a verb and creates the imperative
<https://www.wikifunctions.org/view/en/Z11353>, the form used to make
commands. The function had been created just days before my meeting with
Uzoma. I don’t remember which verb I used to demonstrate it to Uzoma. I
think I had prepared myself, and memorized the tests. So I opened the page
for the function, entered “gụ” (which means “to read”). I called the
function, and the system came back with “gụọ”.
I wasn’t prepared for Uzoma’s reaction. Her eyes were shining, and she had
a big smile on her face. She was delighted to see a function operating on
her native language. And not only that this was a function that had just
been recently created, but that it was open to everyone in the world.
Everyone could just go to Wikifunctions and call natural language
processing functions in Igbo. Or create new such functions, and share them
with the whole world.
Before Wikifunctions, that wouldn’t have been easily possible. Whoever
created such functions would share them, maybe as a JavaScript library.
Whoever wanted to use them, would need to install the right programs on
their computer, if they even had a computer. How do you try out a Python
library on a mobile phone? Who would create and share a library that only
creates the imperative?
With Wikifunctions, many of these issues have evaporated. Whoever wants to
share even a single morphological function is very welcome in
Wikifunctions. They don’t have to set up a library and share it through npm
or GitHub. Someone who wants to use it doesn’t have to install a whole
development environment. Instead, they can just go to Wikifunctions.org and
start using the function. Wikifunctions removed a huge amount of potential
friction, making it easier to share and collaborate on a library of natural
language processing functions for languages that might not have had any
natural language processing libraries before.
Big thanks to Dolphyb <https://www.wikifunctions.org/wiki/User:Dolphyb> for
kicking off the creation of the Igbo functions! You can find the current
catalog of Igbo functions
<https://www.wikifunctions.org/wiki/Wikifunctions:Catalogue#Igbo> on
Wikifunctions.
Function of the Week: Suffix a verb for an Igbo imperative form
Having talked about the Igbo imperative, I remain with this function, the first
morphological function
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-08>
selected
as Function of the Week. We have discussed what the function does above, so
let’s dive here into how it does it.
<https://www.wikifunctions.org/wiki/File:Igbo_imperative.png>Screenshot of
the function for the Igbo imperative
The first thing to notice is that the function has a label and description
in English, but not in Igbo itself! It would be great to give it a label in
Igbo (and I know I am being a hypocrite here, because the morphological
functions I have created for Croatians also don’t have any Croatian
labels). By giving the functions labels in their own language, we make
Wikifunctions more self-sufficient, and allow for less code switching when
using the system.
The function has two tests, 'gụ' to 'gụọ'
<https://www.wikifunctions.org/view/en/Z11376> (to read/ read!) and ‘ri’ to
‘rie’ <https://www.wikifunctions.org/view/en/Z11377>. I couldn’t confirm
the latter being a valid pair of a verb and its imperative, though, which
would make it a potentially bad test. There is one implementation in
JavaScript <https://www.wikifunctions.org/view/en/Z11354>, which passes
both tests. But by looking at the implementation, we can see that it is
quite complex, and that the two test cases don’t cover the many different
cases the implementation covers. The tests really undersell the
implementation, as the implementation is quite more powerful than what we
would be able to tell from the tests.
By having Tests, we increase our confidence in the Implementations that
pass those tests. Given that we are dealing with a wiki, implementations
can change, and failing tests should give us quick feedback that something
is off and that we should negotiate between the tests and the
implementation. Having more than one implementation can help with that too.
Tests can also play the role of documentation, demonstrating what kind of
values we support and what outputs the function will give on the given
values. This can capture edge cases, or standard values, and should capture
both. If an implementation has many different paths, the different tests
should aim to cover as many of the diverging paths as possible.
The third role tests can play is to define scope and expectations for a
function when it is first proposed and has no implementation yet. It should
be easy (and indeed, we need to work on making this easier) to add new
Tests to a function, including for contributors who are not coders. Igbo
speakers should be able to add more Tests for this function, as long as
they are following the regular paradigm for imperatives. In turn, these
Tests will help the contributor writing the implementation by checking
their implementation and ensuring it works correctly.
This function gave me a really great moment when presenting Wikifunctions!
I am very thankful to the Igbo community, and we are hoping that more
members of the community will embrace the creation of regular morphological
functions for Igbo, be it in the form of tests, implementations, or
maintaining the catalog of functions for Igbo.
Recent Changes in the software
The biggest change this week is the correction of our 'robots' policy when
viewing a page on Wikifunctions. Due to the special way we implemented
Wikifunctions, which allows for every language to get a 'real' link, we had
accidentally blocked robots and other search engines from indexing most of
our content. We've now corrected this (T355441
<https://phabricator.wikimedia.org/T355441>), which means that Bing,
Yahoo!, Google, and other places should slowly start to index our functions
for the first time (T353993 <https://phabricator.wikimedia.org/T353993>).
Beyond this, our planning has helped us focus on a few big tasks, which we
will share next week.
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2024-01-25
Due to the formatting, the on-wiki version might be easier to read than the
email.
--
Getting debug information from code
Contributors to Wikifunctions can provide Implementations in two different
ways: either by composing existing Functions, or by providing code in
either JavaScript or Python. And as everyone who has ever created
programming code will confirm, it is unavoidable to have mistakes in the
code one writes, or bugs <https://en.wikipedia.org/wiki/Software_bug> as
they are often called.
<https://meta.wikimedia.org/wiki/File:First_Computer_Bug,_1945.jpg>*An
actual bug that was found in a 1940s computer that caused an error, as told
by Admiral Grace Hopper.*
A very widespread way for debugging is to pepper your code with print
statements, and then remove them again once you figured out and fixed the
bug. A print statement either displays the argument of the statement on the
console or adds it to a logfile. This is usually done in order to inspect
the state of a variable at the point in the code where the print statement
is added. Let’s assume the following JavaScript code was written with the
goal to create a string with five stars:
let result = ""for (let i=0; i=5; i++) { result += "★"}
Some folks may recognize the error immediately, but for others it might be
obscure: why does the computer not return from that code?
One way forward could be to add a print statement to the code. One way to
add a print statement in JavaScript is to write to console.log:
let result = ""for (let i=0; i=5; i++) { console.log(i) result += "★"}
And we see on the console that we get many, many lines with a 5, instead of
the expected 0, 1, 2, 3, 4. This can be an indication of the problem in the
given code, that the condition in the loop should not be i=5, which means
that i is set to the value 5, but should be i!=5, where we compare i with 5
(and even a bit more robust, it should be i<5).
This method is usually called print debugging or tracing
<https://en.wikipedia.org/wiki/Debugging#Techniques>, and although it has a
bad reputation with some folks, it is an often and widely used effective
method to find issues in code and also to just understand code. And it is
now also available in Wikifunctions! You can use the command
Wikifunctions.Debug one or more times with a string argument, and after
running the Implementation your debug strings will be present in the
Details view of the Function result output. Two examples for using
Wikifunctions.Debug can be found for JavaScript
<https://wikifunctions.beta.wmflabs.org/view/en/Z11647> and Python
<https://wikifunctions.beta.wmflabs.org/view/en/Z10057> on the Beta site.
<https://meta.wikimedia.org/wiki/File:Debugger_output_in_Wikifunctions.png>*Details
of a function run. The last bullet point contains the log.*
We would suggest keeping the code free of calls to Wikifunctions.Debug
unless an Implementation is being actively debugged. This will be
particularly helpful once we allow the feature to call one function from
another through code, which could otherwise lead very quickly to very
unreadable log files. We would suggest that the Wikifunctions community
should consider adding this to a style guide for published and connected
functions.
Congratulations to Cory, who shepherded this functionality from idea to
deployment! David assisted by updating the Details view to nicely display
the debugging strings.
Function of the Week: Wrap string
Wrap string <https://www.wikifunctions.org/wiki/Z11145> takes two strings,
the *string to wrap* and the *wrapper*, and returns the former wrapped
between two occurrences of the latter. This can be used to quote some
content with straight quotes, as in the first Test the Function had
<https://www.wikifunctions.org/wiki/Z11147>. Or it can be used for some
artistic effect, to create *l’art
<https://www.wikifunctions.org/wiki/Z12829> pour l’art
<https://en.wikipedia.org/wiki/Art_for_art%27s_sake>*. Or to playfully
create a palindrome letter by letter, such as level
<https://www.wikifunctions.org/wiki/Z12830>.
<https://meta.wikimedia.org/wiki/File:Falafel_%26Tzatziki_Wrap_-_Lavash.jpg>*Wrap(Falafel,
Lavash). Note that the pictured wrap is three dimensional, whereas the
Function we discuss here is only working on a single dimension.*
The other two Tests linked here have been written in preparation for this
post. The Function used to have a single Test, and now has three. Just as
last time, it would be great to have more Tests, in order to ensure that
the Implementations work with a larger number of possible inputs. Does the
Function work well with other scripts, e.g. Chinese or Arabic? Does it work
with Emojis? What about numbers? Does it work with empty strings? In
general, we should strive to have more Tests on our Functions, in order to
increase the confidence in our system and our Implementations.
The three implementations we have are quite straightforward: both JavaScript
<https://www.wikifunctions.org/wiki/Z11148> and Python
<https://www.wikifunctions.org/wiki/Z11149> just use the + operator to
concatenate three strings, referencing the wrapper twice, at the beginning
and at the end. The composition <https://www.wikifunctions.org/wiki/Z11146>
uses join strings <https://www.wikifunctions.org/wiki/Z10000> twice, first
to add the wrapper to the end of the string to wrap, and then once more to
add the string to wrap before the result of that first call.
Recent Changes in the software
This week was mostly a quiet one in terms of new code deployed, as the team
focussed on planning future work. The biggest change was our first
production release of our back-end services this calendar year; none of the
commits included are intended to bring any user-visible changes (beyond
being a little safer and more rigorous). We also fixed a bug with certain
kinds of uses of types that meant the system previously didn't recognise
the call as a valid Object, and improved our code documentation for people
getting started with our code. We also added four new languages that
MediaWiki supports: Rutul (Z1915) <https://www.wikifunctions.org/wiki/Z1915>
, Southeastern Kolami (Z1916) <https://www.wikifunctions.org/wiki/Z1916>, Tooro
(Z1917) <https://www.wikifunctions.org/wiki/Z1917>, and Nupe (Z1919)
<https://www.wikifunctions.org/wiki/Z1919>.
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2024-01-17
--
Refreshing the Function page
<https://meta.wikimedia.org/wiki/File:Wikifunctions_and_Function_page_2013_a…>The
"About" tab of the previous Function page
This week, we are happy to announce that a redesign and rewrite of the
Function page has been deployed to production.
The Function page was always a bit different from the others, as it was
separated into two tabs: “About” and “Details”.
<https://meta.wikimedia.org/wiki/File:Wikifunctions_and_Function_page_2013_d…>The
"Details" tab of the previous Function page
The About page contained an overview of the Function, including its name
and description. It also offered a widget to actually run the Function. The
Details page showed the formal Function definition, the input and output
types, as well as the list of Implementations and Tests.
Feedback for this page was pointing out a few issues: the About page was
(intentionally) rather sparse with information, the second line of tabs
under “Page” and “Discussion” was a bit confusing, and it was not possible
to link the “Details” tab.
<https://meta.wikimedia.org/wiki/File:Wikifunctions_and_Function_page_2014.p…>The
new Functions page
Amin <https://meta.wikimedia.org/wiki/User:AAlhazwani_(WMF)>, our UX lead,
reviewed past research, gathered input from the community and redesigned
the whole experience. He combined the description and names from the front
page with the type information and argument names from the back page in a
new box, the About widget. The widget to try the Function remains in a
prominent spot on the top of the page, followed by the tables for the
Implementations and Tests, which also have been refreshed, incorporating a
“+” for creating new Implementations and Tests. These changes mean that the
Function page is now more consistent with other pages, and the technical
"back-end" details are more prominent. We hope this does not overwhelm
users, or make them feel things are too technical and unwelcoming to them.
We are looking forward to hearing your feedback on the new Function page,
and we want to thank everyone for their hard work on designing it,
gathering the feedback, developing, and testing it.
Function of the Week: ROT13
This week’s Function of the Week is ROT13
<https://www.wikifunctions.org/wiki/Z10627>. ROT13
<https://en.wikipedia.org/wiki/ROT13>, short for “rotate by 13 places”, is
the best known variant of the Caesar cipher
<https://en.wikipedia.org/wiki/Caesar_cipher>, a simple method to encrypt
messages. Every letter gets replaced by the letter 13 places forward in the
English alphabet (if the alphabet was a ring). So the first letter, A, gets
replaced by N, B by O, C by P, *etc.* The special thing about the number 13
is that the English alphabet has 26 letters, and 13 is exactly half of
that, which means that the letter N in turn is replaced by A, O by B, and P
by C, exactly the opposite of what we were saying before.
That means that if you apply ROT13 to a string twice, you move forward 26
letters: every letter becomes itself again! ABC becomes NOP, and applying
ROT13 to NOP becomes ABC again. This way, ROT13 is not only an encryption
function, but also a decryption function! You can use the same function to
encrypt and decrypt messages, which is rather unusual.
ROT13 has been the function I have been using the most outside of Wikimedia
tasks: for example, if I put a spoiler into a discussion, I would use ROT13
to encrypt it, and then link to the Wikifunctions page for ROT13
<https://www.wikifunctions.org/wiki/Z10627>. This allows everyone to
easily skip the spoiler, since most people can’t do ROT13 in their head.
Vg vf abg ernyyl zrnag nf n erny rapelcgvba gurfr qnlf, orvat sne gbb
boivbhf naq rnfl gb qrpelcg.
ROT13 relies heavily on the English alphabet. The Implementation we have
leaves all letters that are not in the English alphabet unmodified. It
would be great to have some variants of the cipher available that also work
for other alphabets. I am sure that books for kids in languages using
Cyrillic for kids contain tables that work on the Cyrillic alphabet, and
probably the same for other scripts. It would be interesting to see a few
more of those implemented.
As of writing, the Function has only a single Test
<https://www.wikifunctions.org/wiki/Z10629>, which is really not that
great. Having more than one Test allows a greater confidence that the
different Implementations work and have the same results, and also that it
handles different cases appropriately, for example empty strings or an
Arabic input passing unmodified, and so on.
The composition relies on the ROT1
<https://www.wikifunctions.org/wiki/Z10846> function, which rotates the
letters by 1. ROT1 turns A into B, B into C, C into D, and so on. If you
apply ROT1 to the name of one of the more famous computers in science
fiction, HAL <https://en.wikipedia.org/wiki/HAL_9000>, you will learn about
the inspiration for that name. If you run ROT1 two times, you effectively
get ROT2. And if you run it 13 times, you get ROT13. The composition of
ROT13 <https://www.wikifunctions.org/wiki/Z10856> thus is using 13 calls to
ROT1.
When we introduce numbers and functions as argument types, we will likely
see the creation of a “repeat” function, which calls a given function a
given number of times on a specific input. That will make it possible to
write a nice alternative composition that is a bit easier to read.
All examples in the text can be run on the wiki, and you can explore other
examples on the linked Function pages.
Recent Changes to the software
The big user-facing change this week is of course the new design for the
Function page, discussed above (T339954
<https://phabricator.wikimedia.org/T339954>). Alongside that, we also added
support for displaying user-controlled debug logs in the front-end user
experience (T353544 <https://phabricator.wikimedia.org/T353544>), drafted
an extensive overview of our front-end code's architecture
<https://www.mediawiki.org/wiki/Extension:WikiLambda/Frontend_Architecture>,
and made a number of clean-up fixes to our code.
The on-wiki version of this newsletter edition can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2024-01-11
--
The Joy of Collaboration
This Monday, we hosted our monthly Volunteers’ Corner. And, as it has been
for the last few times, my personal highlight during this Volunteers’
Corner was the collaborative creation of a new function, following the open
question and answer session.
Given that we just released lists
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2024-01-03>,
this week we unsurprisingly created a function that works on lists: “is any
true <https://www.wikifunctions.org/view/en/Z12698>”, a function that takes
a list of Booleans and figures out if any of them are true. A recording of
the session
<https://commons.wikimedia.org/wiki/File:Abstract_Wikipedia_Volunteer_Corner…>
is
available on Wikimedia Commons.
I want to share a personal observation from these collaborative sessions.
I have been writing software for more than 30 years, and besides a very
small number of pair programming sessions (which I quite enjoyed), I was
always considering it a rather solitary activity. When I got into the flow,
I could code for hours, building up the system I was working on. This was
also true when I was working with others on a system, usually working on an
individual task or fixing a bug. And I usually really enjoyed this work.
Creating functions in Wikifunctions feels very different. And my experience
is very different depending on whether I am editing Wikifunctions by
myself, or whether we are talking about the collaborative sessions.
Let’s talk about the solitary sessions first. When working on Wikifunctions
– and here I also include experience on Beta Wikifunctions and on my local
development machine – one thing is that I like creating compositions, but
at the same time I often notice that I am missing some function that I need
for the composition I want to do. So I usually end up creating a whole
group of functions at once, and for each, I want to create at least three
tests, and three implementations. I quickly end up with creating dozens of
objects, and I sometimes get lost a little bit while doing so. This will
probably get better once we have more of the basic functions in place.
The more interesting experience, though, was when we were doing the
collaborative sessions. And I have to say, these are much more fun than I
expected them to be! Discussing together about the details of the function,
starting some tests, but then there are more tests already there when we
get to the implementations because someone else created other tests, and
while we are doing one implementation, more tests are coming in, and we
discuss them, and by the time we connect the implementations, sometimes
more than one is already ready, and with the tests it feels almost like a
kind of ping-pong coding
<https://openpracticelibrary.com/practice/ping-pong-programming/>, trying
to find edge cases and at the same time ensuring the implementations are
robust enough for them. That also really works well with the tests being
separate from the implementations and functions, which is conducive
for test-driven
development <https://en.wikipedia.org/wiki/Test-driven_development>. I
really enjoy these sessions; they feel a lot more like a collaborative wiki
than 'normal' coding.
I am not yet sure what to think about this experience. Is it just me? Is
this a general experience? Is this something that should flow into our
designs? If so, how? What are your experiences with Wikifunctions so far?
Let us know; we would love to hear your feedback. Maybe we should have
these collaborations more frequently, and have them volunteer-driven?
One thing we decided to try out now, is to extend the next Volunteers’
Corner to an hour. It was very cramped each time. The next Volunteers’
Corner will be on 5 February 2024 from 18:30
<https://zonestamp.toolforge.org/1707157800> to 19:30 UTC. You are all
welcome to join, even for part of the time.
New section: Function of the Week
I want to start highlighting one function each week in these updates.
Suggestions for highlights will be welcome, and if anyone wants to set up a
process that is different from “Denny picks something at his own whim”, I'd
be happy to take those suggestions. Until then, I will just pick something
each week and discuss it here.
<https://meta.wikimedia.org/wiki/File:%D4%B9%D5%A5%D5%AC%D5%A1%D5%AD%D5%A1%D…>Manipulating
strings
And I mean not only to present it quickly, but indeed to discuss it: what
could we do differently with the function, what can we learn from it, how
can we use it?
The function I am kicking this section off with is reverse string
<https://www.wikifunctions.org/wiki/Z10012>. I am doing so because it was
my daughter’s favorite function during the development of Wikifunctions,
admittedly for a rather childish reason (but then again, she does go to
elementary school). She loved it because she could make the computer say
“bad words” even though she didn’t enter the bad words into the system. So
she would give an argument such as “diputs” and laugh uncontrollably for a
while due to the website her daddy is working on showing a naughty word.
<https://meta.wikimedia.org/wiki/File:RevertierungEinesStringsMittelsStackHs…>Reverting
a string using a stack
The function takes a string and returns the string with the first letter
last, the second letter being the second last, and so on, all until the
last letter of the input is the first letter of the output. Some languages
such as C++ have a built-in reverse function for strings, or Java for
StringBuilder. However, neither of the two languages we currently support,
JavaScript and Python, come with a standard reverse function for strings,
which is almost surprising. Both languages offer many different ways on how
to accomplish that task, and the current implementations in Wikifunctions
offer three different ways:
- This JavaScript implementation
<https://www.wikifunctions.org/wiki/Z10017> uses the rather new
deconstruction syntax to turn the input string into an array of characters,
and then uses the reverse function that arrays have, before joining the
characters with empty strings to form a result string again.
- The other JavaScript implementation
<https://www.wikifunctions.org/wiki/Z12662> uses a more traditional
approach, a constructor function, to turn the input into an array, and then
the same series of reversing and joining to get to the result.
- The Python implementation <https://www.wikifunctions.org/wiki/Z10013> uses
an idiomatic, albeit arcane syntax for slicing through an array (and,
applied to a string, it regards the string automatically as an array of
characters).
We also have a new implementation using composition
<https://www.wikifunctions.org/wiki/Z12716>, using a pattern that will
likely be used frequently (basically the same pattern we saw for the
JavaScript implementations): first we turn the string into a list of
codepoints, and then use the brand new reverse a list
<https://www.wikifunctions.org/wiki/Z12668> function created last week,
just after lists were made available. Then we turn that list of codepoints
back into a string again.
The function has six tests, and had for a short time a seventh test, which
was then removed. We’ll get to it in a bit. Three of the tests take simple
Latin-character based strings: abc
<https://www.wikifunctions.org/wiki/Z10014>, stressed
<https://www.wikifunctions.org/wiki/Z10016>, and kayak
<https://www.wikifunctions.org/wiki/Z11028>, one features a longish
hexadecimal number <https://www.wikifunctions.org/wiki/Z10550>, and
one contains
an emoji <https://www.wikifunctions.org/wiki/Z10078>. One uses the string Q1
<https://www.wikifunctions.org/wiki/Z11214>, which should be quite
straightforward, but fails currently due to a bug in Wikifunctions. It
would be great to see more tests with other alphabets, particularly also
right-to-left scripts, or scripts that have many ligatures.
The test that was removed featured an emoji
<https://www.wikifunctions.org/wiki/Z12224>, but from the perspective of
Unicode this is a very different emoji: whereas the test that remains
contains “😂”, the one that was removed contains “🚵🏻♀️”. What’s so
different between these two emojis? This takes us deep into how Unicode
works, but in short, the first one is represented by a single codepoint
<https://en.wikipedia.org/wiki/Face_with_Tears_of_Joy_emoji>, 128514,
whereas the second is represented by a series of five codepoints: 63157,
63157, 8205, 9792, 65039. These five codepoints mean, in turn, “mountain
bicyclist”, “modifier skin tone Fitzpatrick scale
<https://en.wikipedia.org/wiki/Fitzpatrick_scale> 1-2” (*i.e.* white skin),
“zero width joiner <https://en.wikipedia.org/wiki/Zero-width_joiner>”,
“female sign”, “variation selector
<https://en.wikipedia.org/wiki/Variation_Selectors_(Unicode_block)>:
colorful and image like”. All these five codepoints together create a
single emoji, or grapheme as the Unicode standard calls them.
If we turn the order of these five codepoints around, we get a different
result than what we might expect: “♀🏻🚵”. This is being resolved by a new
function that reverses the string on a grapheme level
<https://www.wikifunctions.org/wiki/Z10548>, but that doesn’t have a
working implementation yet. I think that’s the right approach, but I wonder
if the original function should already deal with graphemes, instead of
codepoints (which means we could rename the latter function to become the
actual reverse string function).
This led to discussions on the Project chat
<https://www.wikifunctions.org/wiki/Wikifunctions:Project_chat#Questions_aro…>
about
exactly this topic.
Other considerations could be around capitalization and punctuation. For
example, the following sentence:
*“In the dance of life, find your rhythm!”*
would, with a naïve application of reversion, look like this:
*“!mhtyhr ruoy dnif ,efil fo ecnad eht nI”*
A different reversion might aim to keep the punctuation and capitalization
natural:
*“Mhtyhr ruoy dnif, efil fo ecnad eht ni!”*
Other considerations could be applied to digraphs
<https://en.wikipedia.org/wiki/Digraph_(orthography)> such as “ph”, “th” or
“ch” in English.
Even a seemingly small and simple function such as reversing strings can
take us down several rabbit holes already. And whereas the straightforward
reversal of Unicode codepoints is simple, the more complex functions have
the potential to provide much more value to the end users, as they match
closer with their expectations and are more difficult to find in other
places.
Wikifunctions updates moving to Wikifunctions
These regular Wikifunctions updates will soon be hosted on Wikifunctions
itself, moving off from Meta. Meta was a great place to host us, thank you,
but it makes sense to actually host the updates ourselves on Wikifunctions.
For now, we do not plan on moving all of the archives, but we will point to
them.
Recent Changes to the software
Our main focus last quarter was on better support for types (T343469
<https://phabricator.wikimedia.org/T343469>); with the list support that
shipped last week, our work in this area will now focus on custom,
user-defined types (like number, or datetime, or GPS position, or… whatever
the community wants!). This and other bigger ideas will be part of our
team's planning over the next few weeks – more to report soon!
We've changed the way the main error type that users will see works.
Previously, Z507/Error in evaluation
<https://www.wikifunctions.org/wiki/Z507> would reply with the request
wrapped in a Z99/Quote <https://www.wikifunctions.org/wiki/Z99>, and an
error for what went wrong, also wrapped in a Quote. Now, we only wrap the
first one, which should make for some much more understandable errors in
these cases (T349026 <https://phabricator.wikimedia.org/T349026>).
Outside of the big-ticket items, we worked on a community-inspired
simplification and re-design of the Function page (coming next week!), some
improvements to our documentation and installation guides. We also had to
make some emergency fixes so that our code kept working following some
upstream changes in MediaWiki.
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2024-01-03
Due to the images it might be easier to read this on-wiki.
--Introducing our first new type: Lists
When Wikifunctions was made publicly available last year, we supported only
two types: Strings and Booleans.
<https://meta.wikimedia.org/wiki/File:Composition_with_or.png>Composition
with several "or"s.
We are happy to announce that the first new type is now available for use
by the community: Lists.
What is a list? A list is a value that consists of a number of elements,
usually of the same type. It doesn’t matter how many elements there are in
the list; you don’t have to know in advance. You can add more elements to a
list, you can remove elements from a list, you can pick up parts of a list,
or look at an individual element inside the list. There are many more
things you can do with lists.
There are five pre-defined functions that are available right away for
lists:
- Add an element <https://www.wikifunctions.org/wiki/Z810> to the
beginning of a list
- Look at the first element <https://www.wikifunctions.org/wiki/Z811> of
a list
- Return a list without its first element
<https://www.wikifunctions.org/wiki/Z812>
- Check whether a list is empty <https://www.wikifunctions.org/wiki/Z813>
- Check if two lists are the same
<https://www.wikifunctions.org/wiki/Z889>
Lists make a number of compositions much easier to read and write.
This is useful for natural language functions. For example, we have a
function that checks whether a Breton word needs to mutate
<https://www.wikifunctions.org/wiki/Z11595>. The composition
<https://www.wikifunctions.org/wiki/Z11601> features a series of “ors
<https://www.wikifunctions.org/wiki/Z10184>”: does the word start with a *k*,
or does the word start with a *t*, or does the word start with a *p*, or
does the word start with a *g*, etc.
Instead, we can now create an implementation that asks “does the word start
with one of the strings from this list?
<https://wikifunctions.beta.wmflabs.org/view/en/Z11538>”, which becomes
both easier to write, read, and maintain than the previous implementation.
<https://meta.wikimedia.org/wiki/File:Composition_with_list.png>Composition
using a list.
Lists can be used in four different forms for now:
- You can have a list of Strings, as we have seen in the example above;
- You can have a list of Booleans;
- You can have a list of Objects, which allows you to mix the types of
the elements in the list (because everything is an Object); or
- You can have a list of lists.
This is possible because we implemented lists as so-called generic types:
instead of having four distinct types on Wikifunctions, there is a function
that takes an element type as input, and returns a list type
<https://www.wikifunctions.org/wiki/Z881>.
This also means that when we introduce the next type, you will
automatically be able to use lists with elements of that type, as one will
be able to specify the new type as the argument of the typed list function.
What’s next with types?
Although there are a few more generic types that we want to support —and,
eventually, allow you to create new generic types— we will now focus on a
few simpler types first. It will likely be a while before we come back to
generic types. The next type we plan to support is for natural numbers, and
we expect that to be sooner than later.
What’s next this year?
The team will soon have its planning session for the new half-year period.
We will then announce our plans, and let you know what to expect.
Recent changes in the software
Since the last update
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-12-13>,
mostly the team has been off celebrating the various end-of-Western-year /
solstice-related holidays.
Our focus has been on supporting custom Types (T343469
<https://phabricator.wikimedia.org/T343469>), discussed above. We landed
the new "mode selector" front-end control which lets you specify a "type"
as a function call, such as the one to make a list. It lets you specify the
type used in a list, either as a referenced type such as string
<https://www.wikifunctions.org/wiki/Z6> or a function call that produces a
type, such as typed list <https://www.wikifunctions.org/wiki/Z881>. New
entries in the list are automatically added in the specified type; if the
specified type is just the fallback "object", you can over-ride it to be
more specific for each entry. Happy hacking!
We fixed the label in the function editor for outputs, which was calling it
an 'input' instead (T348363 <https://phabricator.wikimedia.org/T348363>);
this was the first commit for Wikifunctions from community member
User:Punith.nyk <https://www.wikifunctions.org/wiki/User:Punith.nyk>, thank
you! We also fixed a couple of bugs that made lists not work correctly in
some circumstances. Over the holiday period, several dozen more software
labels were translated into new languages, meaning we're currently 90%+
translated
<https://translatewiki.net/wiki/Special:MessageGroupStats?group=ext-wikilamb…>
in
nine languages beyond English (Hebrew, Interlingua, Dutch, Slovenian,
German, French, Polish, Turkish, and Swedish); our thanks as always go to
the TranslateWiki.net community.
On the back-end, we've been working to simplify our test suites and
benchmarking code that has grown to be quite elaborate over the years, plus
continuing our back-end support work for custom types, and improving the
quality of errors the system can return in different situations.
Dear Wikipedia community,
we are happy to announce that Markus Krötzsch will be joining us in Paderborn for the keynote talk at SMWCon 2023!
In this talk, Markus will provide a personal perspective on the origins and principles of semantic wikis, and some of the key challenges that lie ahead in managing knowledge in the age of A!
Don't miss it - you can still attend in person, or follow the conference via YouTube or Zoom.
https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_2023
I you have not already participated in our __community survey__, please do so until this Wednesday. It is important for us to learn more about your SMW usage. Results will be presented next Tuesday.
We thank this year's conference sponsors:
ArchiXL: http://www.archixl.nl/
Specialists in enterprise architecture, knowledge management, and semantics
Hallo Welt!: https://bluespice.com/
The company behind BlueSpice, the open-source enterprise wiki software
MyWikis Europe: https://mywikis.eu/
GDPR compliant (Semantic) MediaWiki hosting from the heart of Europe.
Wikibase Solutions: https://wikibase-solutions.com/
Specialist in business solutions with MediaWiki
As well as the conference organizers
MediaWiki Stakeholders' Group: https://mwstake.org/
Advocating the needs of MediaWiki users outside the Wikimedia Foundation
KM-A Knowledge Management Associates: https://km-a.net/
KM-A educates and advises Knowledge Managers and connects the KM Community in Austria and the world.
Paderborn University: https://www.uni-paderborn.de/en/
While always keeping society's needs in mind, scientists at Paderborn University are working on the technologies of the future.
Juggel: https://www.juggel.com/
AI supported Knowledge Management based on MediaWiki and Semantic MediaWiki
Best regards,
Bernhard, Ad and Tobias
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-11-30
--
Welcome, Grace and Miguel!
This week we are happy to welcome two new members on the Wikifunctions team.
Grace Choi is joining us and the Foundation as a senior software engineer,
to work on all of the pieces of software Wikifunctions runs on. Here we
have Grace saying hello in her own words:
*As a classical violinist communicating in the small world of music (yes,
much smaller world than the small world as we know it), I long wanted to
learn how to communicate in the world of technology that was seemingly
dictating our very lives. After completing my masters, I carefully laid my
violin down for a rest and plunged into a bootcamp that introduced me to
web development. I then dabbled in some full-stack engineering followed by
nearly three years in backend development with Ruby; one could say the rest
is history.*
*I am thrilled to join the spectacular team at Abstract Wikipedia to
scrutinize yet another way to 'communicate' by way of functions and
linguistics to actualize an extraordinary project together. I don't have
many hobbies but does learning German on Duolingo count? For the winter
season, I have pulled out the Lord of the Rings books and am in search of
some delicious local (Seattle) hot cocoa to melt into my oat milk.*
Miguel Castro is joining us and the Foundation as our engineering manager,
to coordinate the engineers on the team, ensure their growth, and also
coordinate their work with the rest of the organization. Here we have
Miguel saying hello in his own words:
*I am excited and humbled to be part of the Abstract Wikipedia team! When I
was around 8, my mom bought me an old, worn-out encyclopedia set that
became a doorway to reading about everything that intrigued and fascinated
me. When I was around 10, I discovered computers and how they could grant
me access to knowledge. I began in the 90s, using my modem to connect to
local computers to download text files on subjects ranging from philosophy
to programming.*
*By the time I was 14, I was coding every day. After I graduated, I began
my first job in IT at 18 and have been working in the industry ever since.
I eventually graduated college with a degree in MIS and Computer Science.*
*I worked as a software engineer for most of my career, primarily with
open-source technologies. Today, my focus is engineering leadership and how
to bring people together to create things we are proud of and that have a
positive impact. Lastly, my work here with the Abstract Wikipedia team is a
full-circle moment as it connects me to why I started this journey.*
Please join me in welcoming Grace and Miguel to the team!
Functioneer rights control handed over to community
<https://meta.wikimedia.org/wiki/File:Wikifunctions_Functioneer_Icon.svg>Wikifunctions
Functioneers logo
The community has in the last few weeks successfully worked on and agreed
on a policy for Functioneer rights
<https://www.wikifunctions.org/wiki/Wikifunctions:Functioneers#How_to_request>.
Thank you all! Given that this is in place, we are happy to relinquish the
responsibility of giving out Functioneer rights, and allow the community to
implement the new rights.
All Functioneer rights that have been handed out by us so far have been
temporary and will expire by themselves, but the community is free to
manually speed that up if they so want. It is all up to you.
With that, we will archive the current request page
<https://www.wikifunctions.org/wiki/Wikifunctions:Apply_for_editing>. Thank
you for working with us on this step!
Recent changes to Wikifunctions software
Our main focus right now continues to be on better support of types (T343469
<https://phabricator.wikimedia.org/T343469>). To this end, the biggest
visible change this week is our continued work on the front-end to allow
the use of lists as inputs and outputs of functions (T326301
<https://phabricator.wikimedia.org/T326301> mainly, plus T351276
<https://phabricator.wikimedia.org/T351276> and others). This feature is
not quite ready yet for us to encourage use, as we're working on their uses
in compositions (T351272 <https://phabricator.wikimedia.org/T351272>) and
other places – expect progress in the next few weeks!
On the back-end, we've struggled through several related issues that were
blocking progress or even deployment (T350700
<https://phabricator.wikimedia.org/T350700> and T352328
<https://phabricator.wikimedia.org/T352328>, alongside several others), and
we've now finally updated the evaluator service to our latest work. Our
thanks to colleagues in SRE for their help on how we could hunt down
evidence to debug our issues. Beyond being a little faster now, the main
user-visible change from the newly-updated services is that reports of
duration less than 50 µs, previously reported as 0 (as the OS tries to
mitigate timing attack security concerns), will now show "< 50 µs".
We've changed the mechanism we use to pass your function calls into the
WebAssembler environment after we found our original approach caused some
hard-to-debug errors in some cases (T349385
<https://phabricator.wikimedia.org/T349385>). Additionally, a number of
rare error-states will now return proper, translatable errors, rather than
hard-coding a message in English, such as for timeouts (T327275
<https://phabricator.wikimedia.org/T327275>), recursion limits (T350608
<https://phabricator.wikimedia.org/T350608>), and invalid responses (T349785
<https://phabricator.wikimedia.org/T349785>).
For minor issues, we fixed and released early a bug that hid the button to
trigger test runs on implementations (T351121
<https://phabricator.wikimedia.org/T351121>), and we altered the "Function
explorer" box on Implementation and Test case pages to show now just the
ZID but the label when inputs are function-call types like a list (T351274
<https://phabricator.wikimedia.org/T351274>)
You can browse the full list of deployed changes
<https://www.mediawiki.org/wiki/MediaWiki_1.42/wmf.7#WikiLambda> for the
MediaWiki front-end for Wikifunctions. We didn't deploy any back-end
service changes this week.
Volunteers’ Corner on Monday, 4th of December 2023
We will have our monthly Volunteers’ Corner on the upcoming Monday 4
December 2023 at 18:30 UTC <https://zonestamp.toolforge.org/1701714600>.
You can join the meeting in the usual Google Meets room
<https://meet.google.com/xuy-njxh-rkw>.
We will give a quick update on the development and upcoming changes, offer
space for questions, and, if time permits, collaboratively create a new
function. Bring ideas for a new function along!
