This week's newsletter can also be found online here:
https://design.wikimedia.org/blog/2022/03/14/wikifunctions.html
--
Designing Wikifunctions, brick by brick.
*By Aishwarya Vardhana <https://www.aishwaryavardhana.com/>*
What is Wikifunctions
Wikifunctions will be a collaboratively edited multilingual library of
functions. It will be the first platform The Wikimedia Foundation launches
since Wikidata <https://en.wikipedia.org/wiki/Wikidata> in 2012, and part
of a bigger vision towards an Abstract Wikipedia, which you can read about
at Abstract Wikipedia on meta-wiki
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia>.
A function and its parts
If you’re not a programmer you might not be familiar with what a function
needs to be usable, or to use technical-speak “to run”. A function takes an
input (e.g. the population of a city) and provides an output (e.g. the
acres of watershed needed to supply residents with water). It can do this
calculation over and over again for different cases.
A function has three parts. The function definition (part 1), the
implementation (part 2), and a test (part 3) to validate. These three
components – the definition, implementation, and tests – are what a
function needs to perform and be usable.
A definition defines the functions behavior. What type of inputs does it
accept? How many? What will it output? An implementation is the set of
instructions that transforms an input into an output. A test is what
validates the implementation and ensures it performs correctly.
[image: Function model]What can functions do?
For starters, you might use a function to wake up in the morning. Let’s say
you want to sleep 8 hours tonight, you could ask a function, *“If I go to
sleep at 10 P.M. what time should I wake up to get 8 hours of sleep?”*
Examples of general questions that could be answered by functions:
- What is the distance from my house in northern Sao Paulo, Brazil to
the nearest beach?
- If I took out a $240,000 loan and my loan term is 30-year fixed with a
4% interest rate, what is my monthly mortgage?
- If the omicron variant has a transmission rate of 25% among vaccinated
people, how quickly could it spread in a city with 2,059 people per square
kilometer, 95% of whom are vaccinated?
- When in the next ten years will my birthday fall on a Saturday if I
was born March 28, 1954?
Examples of Wikimedia-related questions that could be answered by functions:
- Given the dates of birth and death of a person, how old did the person
get?
- Given a population and an area, what is the population density?
- Given an image on Commons, crop it and run it through a filter
- Given two images on Commons, which one is more red?
- Based on Wikidata, how are Queen Elizabeth II and Barack Obama related
to each other?
Functions are ubiquitous in modern life, from the apps we use daily (e.g.
maps and search) to the analyses made by scientists, doctors, journalists,
and financial advisors, functions enable our activities by efficiently and
precisely making meaning through calculations.
Functions will provide the code that translates language-independent
articles from Abstract Wikipedia into natural language Wikipedia articles.
This will theoretically allow more people to share more knowledge in more
languages!
This blog post is about how to create, view, and edit functions in
Wikifunctions, and the design process behind making this happen.
Our audience
Similar to other Wikimedia projects, our hope and aim is for Wikifunctions
to be inviting, accessible to, and available to all people across the
world. This is why we are designing a multilingual platform that is mobile
friendly.
There are several roles for people within the Wikifunctions ecosystem.
There are “answer seekers”, or people who simply want to learn about
functions and use them. A level deeper are those who edit or materially
contribute to Wikifunctions in some capacity. They could be function
patrollers, programmers who write implementations, translators, admins who
approve of proposed implementations and tests, or test writers who QA
implementations. These folks will maintain functions and documentation.
Among these editors there will be technical and nontechnical folks. Leading
up to launch and post-launch, our team aims to increase the agency of the
nontechnical folks and “junior” techies by creating pathways for them to
contribute meaningfully.
Part of creating these pathways is simplifying a complicated process.
Functions are not simple. They are complex in that they require
computational thinking and well-written documentation. When designing the
creation experience, our objective was to design an editor interface that
looks and feels friendly to nontechnical people and radically simplifies a
complex process.
Our approach
We explored different mental models for Wikifunctions such as collaborative
coding, a structured Q&A forum, a homework helper, infrastructure for
Abstract Wikipedia, and a ‘generic library of functions’. We settled on the
structured Q&A model, where we conceive of functions as tools which can
answer structured questions.
[image: Our approach]
WMF’s mission is to unlock knowledge and help all people share in it,
freely. The Abstract Wikipedia team believes that functions can play a key
role in unlocking knowledge, and sees Wikifunctions as a crucial player in
moving all of us towards this mission.
It is a source of power to know how to write and use functions. It gives
the person the power of knowledge creation. Our objective when designing
the function creation experience was to make it as friendly, intuitive, and
simple as possible so that more people can have access to this power.
Our design principles
1. Conceptual simplicity: Given the complex nature of this project, our
team aims to radically simplify the concepts involved so that we might be
inclusive of all people, especially those who are less familiar with
functions, code, natural language processing, or lexicographic knowledge.
2. Intuitive interface: We want to build a system with a shallow onramp,
one that anyone can figure out and engage with in meaningful ways. We wish
for there to be little gap between a newcomer’s existing knowledge (i.e.
current knowledge) and the knowledge they need to operate the product (i.e.
target knowledge). Our interface will provide community members with an
uninterrupted experience, where they are well-supported to complete tasks
with relative ease and confidence.
3. Multilingual: In a world where all people can share in the sum of all
knowledge, it is critical that Wikimedia products are designed as
multilingual interfaces and experiences. From the beginning, the Abstract
Wikipedia team will prioritize designing for and testing in the native
languages of our community-members. Our objective is to move at the speed
of trust with all populations, but especially with small Wikis and
underrepresented groups.
4. Balance: We wish to acknowledge that many of our community members
will be veterans of Wikipedia, Wikidata, or any of the movement’s other
projects. Many of them will be highly-skilled at navigating technical
interfaces and desire customizability similar to Wikipedia or other open
source projects. The challenge before us is prioritizing the needs of
non-programmers and those with little-to-no coding experience while keeping
in mind the expectations of veterans. This will require flexibility and
openness in our designs, and filtering design decisions through multiple
lenses in order to ensure that we are not locked into any one particular
user experience.
We have implemented these principles by doing the following:
- Intentionally minimizing the use of technical jargon and providing a
glossary of terms
- Providing tooltips and informative microcopy on every interface
- Focused attention on information hierarchy so as not to overwhelm
users with information
- User testing with nontechnical people and iterating to meet their
needs and expectations
- User testing with technical people to ensure we are not causing
irritation or creating unnecessary roadblocks to their progress
-Multilingual design and usability testing (testing is in the planning
process)
How to use WikifunctionsCreating a function
Let’s say we wanted to create a function that determined if two words
rhymed. The “two words” are two inputs of type string
<https://en.wikipedia.org/wiki/String_(computer_science)>, and the output
is a single “yes or no”, which is a type Boolean
<https://en.wikipedia.org/wiki/Boolean_data_type>. To create a function
that could do this, we would need to tell the system, “Hey this
function *accepts
two inputs*, both have to be *strings*, and the function’s output will be a
*Boolean*.”
This process is known as “defining” the function, and it is the first step
in creating a function.
Our team believes that the UI and UX for defining a function must be as
inclusive as possible because, technically, anyone can do it. It requires
no coding or composition writing. Defining a function can be thought of as
a request: *“Hey I want a function that does X”* translates to *“the
function accepts this input, returns this output”*. We also provide a way
to add documentation so that the user can further explain what they’re
looking for.
Design challenge 1: The function editor
We initially planned to design a “function editor” that provided an
end-to-end experience, meaning users could define, implement, and test in
one environment. Since we were designing for technical and nontechnical
people, we hypothesized that unifying the experience would:
- Simplify the concept of what a function needs
- Provide a contextual experience which would be helpful for
nontechnical users
- Keep related tasks together
- Allow more senior editors to move in and out of the process as they
pleased
- Not slow down the technical audience from using the editor as they
wanted, while providing guardrails for nontechnical users
Early drafts and variations of an end-to-end function editor
[image: Early draft 1][image: Early draft 2][image: Early draft 3]
The UX was inspired by Miro <https://dribbble.com/tags/miro>, Figma
<https://dribbble.com/figma>, Wikidata query builder
<https://wd-query-builder.toolforge.org/>, and Typeform
<https://dribbble.com/tags/typeform>.
Pivoting – we tested these mockups with technical non-Wikimedians and
Wikimedians, and the results were overwhelmingly positive. However, as our
developers were building this design, we realized that the underlying
design of the system was incompatible with this approach. Function
definitions, implementations, and tests are all separate objects in
Wikifunctions, and each object can refer to other objects within it, such
as data types (e.g. strings, Booleans, or other functions). Due to the
complexity allowed for each object, we decided it’s best to keep each
creation experience separate. Objects also have their own pages, so any
object you create automatically publishes as a Page.
Today the function editor is a place only for defining the function, in
other words part one of creating a function.
[image: Creating a function][image: Steps in creating a function]
The function editor as it currently exists today
[image: function editor]Design challenge 2: A function page
When you publish a function that you’ve defined, it is saved and published
as a function page. This is the page where all parts of a function live. We
framed this page as an entry point for the community to learn about, use,
and/or edit a function. The goal of this page is to help people
progressively edit a function or simply engage with one. The challenge of
this page, and a Wikifunctions motif, is that the page must appeal to both
non-technical and technical people.
Our short term goals were to
- Help all users UNDERSTAND the function so that they can edit or use it
- ENCOURAGE users with coding backgrounds to make technical contributions
- ENCOURAGE users who are not coders to make non-technical contributions
- Usability test a Figma prototype of the product solution with people
who have a strong coding background
Our hopes and dreams were to
- Encourage everyone that views a function to try using it
- Support coders who want to make technical contributions
- Encourage users who don’t code to make non-technical contributions
because functions need names, aliases, translations of the name/input
labels, and documentation
We aimed to avoid
- Confusing, discouraging, overwhelming, or intimidating non-coders
- Unintentionally signaling to non-coders that this isn’t a space for
them
- Irritating coders by slowing them down or hiding important actions
The function page as it currently exists today. A user lands on a function
page’s ‘About’ tab and can toggle between ‘About’ and ‘Details’.
[image: function page]
A user can learn more about the technical details of the function on the
‘Details’ tab, such as how many available implementations and verified
testers it has, and what the definition is.
[image: function details]How to get involved with Wikifunctions
- Subscribe to our mailing list
<https://lists.wikimedia.org/postorius/lists/abstract-wikipedia.lists.wikime…>
to
get weekly updates from our product lead Denny Vrandecic. Read past
newsletters on Meta-wiki
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates>!
- Join our Telegram community <https://t.me/Wikifunctions> or our IRC
channel
<https://kiwiirc.com/nextclient/irc.libera.chat/#wikipedia-abstract>
(mirrored)
for in-depth conversations on Wikifunctions
- Follow us on Twitter <https://twitter.com/Wikifunctions> for team and
product updates
- Email me (avardhana(a)wikimedia.org) if you’re interested in product
testing
*Thank you to Nicolas Ayoub, Amin Al Hazwani, Sudhanshu Gautam, Genoveva
Galarza Heredero, David Martin, Nick Wilson, and Denny Vrandečić for their
contributions!*
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-03-14
--
How would we generate a text in Abstract Wikipedia such as the first
sentence of the English Wikipedia article about Mariya Zerova
<https://en.wikipedia.org/wiki/Mariya_Zerova>?
*“Mariya Yakovlevna Zerova, alternately Marija Jakovlevna Zerova, (April 7,
1902 - July 21, 1994) was a Ukrainian biologist and taxonomist known for
her work in mycology.”*
There are plenty of interesting questions regarding generating this short
sentence - the name, the biographical dates, the description. Today, let’s
just focus on the name.
Given that Zerova was Ukrainian, was born in and has lived in Ukraine, her
name was written using the Cyrillic alphabet, Марія Яківна Зерова. In her
English Wikipedia article, her name in the Cyrllic alphabet is given in the
Wikipedia infobox, but not in the text of the article. There are several
ways to transliterate the name from the Cyrillyc alphabet to the Latin
alphabet. Particularly, the letter я
<https://en.wikipedia.org/wiki/Ya_(Cyrillic)> can be transliterated as *Ya*
or *Ja* in English, which leads to the variation given in the English
Wikipedia article.
Her Wikidata item <https://www.wikidata.org/wiki/Q12106673> states that her
first name is Marija <https://www.wikidata.org/wiki/Q18603722>, and not
Maria <https://www.wikidata.org/wiki/Q325872>, Mariya
<https://www.wikidata.org/wiki/Q39897333>, or Mariia
<https://www.wikidata.org/wiki/Q56433356> (all these three names are
written as Марія in Ukrainian). Names are a difficult mess, and so it is
not surprising that Wikidata is having trouble representing them. A big
thanks and shoutout to the hard work by the Wikiproject Names on Wikidata
<https://www.wikidata.org/wiki/Wikidata:WikiProject_Names>, which aims to
sort out this kind of issue. You should join them if you are interested in
helping.
So, how would we get her name for Abstract Wikipedia for the different
languages? Do we need Lexemes for every first name in every language? Such
as the Lexeme Maria <https://www.wikidata.org/wiki/Lexeme:L414214> in
English? And then how would we link the given name in Wikidata to the given
name, and in turn the Lexemes link to that given name?
What about Yakovlevna, her patronym? Or Zerova, her family name? Both names
are rarer than Mariya. Would we expect Lexemes for each of these names in
Wikidata too, for each language individually? That seems like a lot of work.
In such cases I hope that the answer is no, and that we can figure out a
way to avoid that. But what could that look like? As usual, I expect that
as a community we will come up with a better solution than what I could
come up with. Together we are smarter than any one of us. So think of this
as a first, rough draft.
My first thought would be to have functions in Wikifunctions that take a
name such as *“Yakovlevna”* as a string and can generate all necessary
forms based on regular morphological functions
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-09-10>.
Names that have irregular forms would still be Lexemes, but if a function
can create the necessary forms, we should be able to use that directly
based on a string. So if we need the genitive form of Yakovlevna’s name (as
in this very sentence), a function would just generate it.
The same mechanism to generate the necessary forms may be helpful for many
place names and other proper names. In addition, we will likely need
functions that can transliterate between different alphabets, which is a
hornets' nest in itself. Transliterations can differ from target language
to target language: the transliteration of Зерова into German would be
Serowa, not Zerova, as it is in English.
<https://meta.wikimedia.org/wiki/File:Wikipedia-logo-for-Ukraine.png>
<https://meta.wikimedia.org/wiki/File:Wikipedia-logo-for-Ukraine.png>
Ukrainian Wikipedia logo
But that’s not all. The astute reader might have already noticed that
*Yakovlevna* is not a direct transliteration of Яківна. That would be
*Yakivna* (or *Jakivna*). What happened here?
In addition to the name being *transliterated* (i.e. where we map from one
script to another) the name was also *translated*, or backformed, in the
way it would be formed in Russian. The English form *Yakovlevna* is based
on the Russian form Яковлевна, and indeed, if we look in the Russian
Wikipedia, the Russian name for the biologist is Мария Яковлевна Зерова
<https://ru.wikipedia.org/wiki/%D0%97%D0%B5%D1%80%D0%BE%D0%B2%D0%B0,_%D0%9C%…>
-
a version of the name that is never mentioned on her native Ukrainian
Wikipedia article
<https://uk.wikipedia.org/wiki/%D0%97%D0%B5%D1%80%D0%BE%D0%B2%D0%B0_%D0%9C%D…>
.
By the way, if you are surprised to find that names can be translated,
enjoy seeing the names of Pope John Paul II
<https://www.wikidata.org/wiki/Q989> in different languages on Wikidata by
clicking on “All entered languages”.
How would Abstract Wikipedia ever figure out that it should first translate
Яківна to Russian and then transliterate it? Is this even the right thing
to do? To be honest, I am entirely stumped here. Should Ukrainian names in
general first be translated to Russian variants, and then be
transliterated? Let’s take two other Ukrainians, who both have the same
name: the President of Ukraine, and the brother of the Mayor of Kyiv, are
both named Володимир, but English Wikipedia refers to the President as
Volodymyr <https://en.wikipedia.org/wiki/Volodymyr_Zelenskyy> (a direct
transliteration) and to the other as Wladimir
<https://en.wikipedia.org/wiki/Wladimir_Klitschko>. In Ukrainian, they have
the same name!
I guess in many of those cases the best we can do is to rely on Wikidata,
and use the labels on the items as string input and the structured data
around given and family names. This allows us to enter and fix the data
manually, item by item, where there is evidence that an individual used a
different form. Only if Wikidata does not offer the necessary data, would
we need to use fallback functions. And the fallback functions could be
different from language to language, so that in Russian, Zerova can be
Яковлевна and in Ukrainian Яківна.
And maybe, just maybe, having to encode that explicitly will make us more
aware of how names of people and places flow through our knowledge
ecosystem, how they reflect power and inequity.
So many interesting things about just the first few words of this one
sentence, and we haven’t even talked yet about whether her birth date is
stated in the Gregorian, Julian, or another calendar!
Wondering how the community will be allowed to interop with function
definitions in their languages of choice...even extending functions with
other languages.
I didn't know about NAPI-RS <https://napi.rs/> existed until watching a
podcast today (33:00) <https://youtu.be/Aeij24E-jHI?t=1980> where `#[napi]`
macro was mentioned as an interesting PR
<https://github.com/napi-rs/napi-rs/pull/696> and thus bringing about a
very powerful feature of building pre-compiled Node.js add-ons in Rust.
And now even ThreadSafe functions
<https://napi.rs/docs/concepts/threadsafe-function>.
How this might be utilized in Wikifunctions later on is an interesting
thought experiment... imagine someone wanting to use a Node library that
already affords a great implementation but also needs to be adapted or
expanded for special fit and purpose... and in Wikifunctions by writing a
Node.js addon in Rust language.
Does anyone feel that the idea of extending functions with other languages
(even native) might be too edgy a task on Function Composition? Or will
this eventually be expected to be supported in whole or partially with
the Launch
requirements
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Launch_requirements>
,one of which is:
- "Switching from one language to the other"
Thoughts?
Thad
https://www.linkedin.com/in/thadguidry/https://calendly.com/thadguidry/
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-03-04
--
You may have noticed that we didn’t send out our updates for the last few
weeks. Given world events, we thought them trivial, and decided to stay
silent. We have kept working on Wikifunctions in the meantime. For this
week, here is a short update on our activities. We are planning to resume
the normal weekly essays soon.
The development work is moving forward. The current phase η
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases#Phase_%CE%B7_(eta…>
turned
out to be our longest phase by far, but we have finally started to see the
finish line. This had several reasons, one of them being that we actually
grew the team during that time considerably, and as we know from the Mythical
Man-Month <https://en.wikipedia.org/wiki/The_Mythical_Man-Month>, absorbing
more capacity also takes time—but is now expected to lead to an increased
development speed moving forward. You can follow the progress on the Phase
η Phabricator work board
<https://phabricator.wikimedia.org/project/view/4960/>.
We are also working to plan out the next phase
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases#Phase_%CE%B8_(the…>
in
a way that will hopefully avoid a prolonged period as the one that is now
closing. You can see our initial work to break this down in the top-level
task in Phabricator <https://phabricator.wikimedia.org/T299176>.
The design work is gaining speed, and we have in the last few days done a
number of user-tests with some of you on the design for how viewing a
function will work and feel. We will very soon publish the UX research
results, the designs, and more background on that work. Stay tuned! As
always, if you're interested in participating in future user-tests, please
email Aishwarya <https://meta.wikimedia.org/wiki/User:AVardhana_(WMF)> and
let her know the natural languages you are literate in and a self-estimate
of your programming skills, if any (and yes, we in particular look for
people with no programming skills).
One thing we noticed is that we really need to improve our code coverage in
unit tests, and introduce automatic end-to-end tests. We are working on a
plan to improve the situation, which will allow us to evolve the code more
confidently in the future. We now have automatic publication of code
coverage for three of our five codebases, and we're working on the other
two: WikiLambda PHP
<https://doc.wikimedia.org/cover-extensions/WikiLambda/includes/index.html>
and Vue <https://doc.wikimedia.org/cover/WikiLambda/lcov-report/>, and the
schemata, utility, and definition
<https://doc.wikimedia.org/cover/mediawiki-services-function-schemata/lcov-r…>
code
bases have coverage, and the orchestrator and evaluator back-end services
should be coming soon. One major issue is that the Wikimedia continuous
integration and testing environment is not really set up for an
architecture as ours, where a back-end service for evaluation of functions
and the wiki are developed in parallel and both have moving parts. As said,
we are working on a plan towards getting a better grip on this situation.
For now, we have published our first draft of our manual smoke testing
protocol <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Smoke_tests>,
which also gives an idea of the capabilities that are currently available.
