The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-02-23
----Thank you, Adesoji!
Adesoji Temitope
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-10-08> joined
the team in October 2021 as a consultant through This Dot Labs, lending his
experience in Vue.js and front-end development to the team. As Adesoji
concludes his work with This Dot, we see Adesoji thus wrapping up his work
on Wikifunctions. We are sad to see him go!
Here are Adesoji’s own words:
It's been an incredible journey to work with some of the best of the minds
I have known in my professional career. I got to work on a very interesting
and unique project. Thanks for the opportunity, Sadly, I am leaving with a
warm heart. Thank you, everyone, and goodbye.
Adesoji has been working on the Wikifunctions front-end, developing the
interface for functions which will allow the contributors and users of
Wikifunctions to interact with them. Thank you Adesoji for your work, and
we wish you best of luck in your future endeavors!
Welcome, Allan Jeremy!
This Dot is on-boarding another engineer to support us in the development
of Wikifunctions, Allan Jeremy.
Here are Allan’s own words:
I am a self-taught software engineer from Nairobi, Kenya. I started
learning how to code at 14, owing to a desire to create video games like
the ones I loved playing when I was younger. When I am not coding, I am
either exploring clear beaches, recording videos, or nerding out about AI.
I am also interested in history, philosophy and psychology.
Welcome Allan to the team! We are looking forward to working with you.
A new engineering role is open
As Julia Kieserman
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-11-18>
transfers
to a different role within the Foundation during the next couple months, we
are hiring for a new full stack software engineer, with a need to ensure
continued delivery on the Vue.js front-end. The role is posted here:
Senior Software Engineer, Abstract Wikipedia
<https://boards.greenhouse.io/wikimedia/jobs/4868029?gh_src=4e048f911us>
Please consider whether you or someone you know might be the right person
for this opportunity to work on Abstract Wikipedia and Wikifunctions in an
engineering capacity.
Conversation with Trustees
Abstract Wikipedia and Wikifunctions will join this round of the *Conversation
with Trustees**,* organized by the Community Affairs Committee of the Board
of Trustees of the Wikimedia Foundation. You can join the meeting on February
23, 2023, at 19:00 UTC <https://zonestamp.toolforge.org/1677178854> as per
the following page. A recording will be made available.
- Conversation with Trustees
<https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Community_Affairs_Comm…>
Recordings from this week’s presentations
As announced last week, we had two recorded meetings.
Stef Dunlap is talking about our end to end testing environment. The
presentation and recording is available here, as well as a transcript:
- people.wikimedia.org/~kindrobot/duct/
Maria Keet is talking about the design choices for abstract representations
we had so far. We are trying to upload the video on Commons as well, and
will let you know when it is there, but for those who cannot wait, here is
a preliminary place where you can access the video for now:
- Temporary download location on Google Drive
<https://drive.google.com/file/d/1br0ldYvMz1dUzTZr0c-pWSBbJmDRWd_D/view?usp=…>
Development update (February 17, 2023)
Everyone’s working on large patchsets, which will hopefully land soon.
- *Goal 2* (efficient and correct evaluation) is seeing the split up of
the evaluator in separate evaluators per language
- *Goal 3* (meta-data) is working on the reordering of implementations
- *Goal 5* (default view) is working on typed lists
- *Goal 6* (stable and secure system) is seeing an effective system for
rights checking being implemented
The on-wiki version of this week's newsletter is available here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-02-17
----
Wikifunctions: a Compendium of Calculations
<https://meta.wikimedia.org/wiki/File:1983_CPA_5426_(1).png>
<https://meta.wikimedia.org/wiki/File:1983_CPA_5426_(1).png>
A stamp issued 6 September 1983 in the Soviet Union, commemorating
al-Khwārizmī's (approximate) 1200th birthday.
Twelve hundred years ago, the polymath Muhammad ibn Musa al-Khwarizmi
<https://en.wikipedia.org/wiki/Muhammad_ibn_Musa_al-Khwarizmi> wrote a
book, *The Compendious Book on Calculation by Completion and Balancing*
<https://en.wikipedia.org/wiki/The_Compendious_Book_on_Calculation_by_Comple…>.
He was born in Khwarazm <https://en.wikipedia.org/wiki/Khwarazm> in what is
today Uzbekistan and wrote the book in Baghdad
<https://en.wikipedia.org/wiki/Baghdad> in what is today Iraq. The goal of
the book was to help people find answers to a certain form of questions:
for example, how to calculate inheritances (*“a man leaves behind two sons
and a daughter”*), the area of a yard (*“assume the yard has the shape of a
rhombus”*), prices for mercantile transactions (*“ten for eight, so how
much for four?”*), and many other such practical questions for everyday
life. Al-Khwarizmi drew on knowledge and practices from Indian numbers,
Babylonian astronomy, and Greek mathematics (which had largely been lost in
Greece and the West by that time). The book remained a major textbook on
the topic for more than half a millennium, widely influencing mathematics
and its notation, and answered questions for many people, whether it was on
dividing their legacies, figuring out the size of their land, or buying and
selling merchandise.
The book was in fact so influential that it gave us two words that are in
widespread use today, with cognates in many languages: the word *algebra*,
which is derived from the the Arabic word جبر
<https://en.wiktionary.org/wiki/%D8%AC%D8%A8%D8%B1#Arabic> (*al-jabr*),
comes from the title of the book, and *algorithm* is derived from
al-Khwarizmi, the name of the author. Even today, al-Khwarizmi's image
graces the cover of some math textbooks in Ecuador, and the books seem to
contain excerpts of his Compendious Book, too!
<https://meta.wikimedia.org/wiki/File:Image-Al-Kit%C4%81b_al-mu%E1%B8%ABta%E…>
<https://meta.wikimedia.org/wiki/File:Image-Al-Kit%C4%81b_al-mu%E1%B8%ABta%E…>
A page from al-Khwārizmī's *Algebra*
The importance and relevance of that book indicates one of the legacies
Wikifunctions can aspire to: a compendium of methods on how to answer
questions, including questions like the ones above, and many others too.
Just as Wikipedia follows the spirit of Western enlightenment
<https://en.wikipedia.org/wiki/Age_of_Enlightenment> embodied in Denis
Diderot’s <https://en.wikipedia.org/wiki/Denis_Diderot> and Jean le Rond
d'Alembert’s <https://en.wikipedia.org/wiki/Jean_le_Rond_d%27Alembert>
*Encyclopédie,
or a Systematic Dictionary of the Sciences, Arts, and Crafts*
<https://en.wikipedia.org/wiki/Encyclop%C3%A9die>, Wikifunctions follows
the *Compendium* as an achievement of the Islamic Golden Age
<https://en.wikipedia.org/wiki/Islamic_Golden_Age>, also sometimes called
the Islamic Enlightenment.
It will be possible for functions in Wikifunctions to look like pure
mathematics (*“what is 25% of 1.5?”*) or to look like questions that we
might encounter in everyday life (*“given a land parcel of 1.5 acres, how
large would be one part out of four equal parts?”*). It is known that
translating from the latter to the former is not always trivial: a cousin
of mine was struggling with fractions as a child. But once I rephrased the
same questions to be about money instead of pure numbers, she answered each
of the questions, and more, without any hesitation or error.
Both types of function can live in Wikifunctions side by side, and the
everyday functions can use the pure mathematical ones in their
implementations. Functions can form layers on top of other functions, and
functions useful to calculate the amount of ingredients for recipes can
build on top of functions for multiplication and unit conversion.
In Wikifunctions, functions will be accompanied by documentation explaining
them. This documentation can guide users, and help them find the right
function to answer their questions. How a function is named, how its
arguments are named, the documentation surrounding functions – all of these
can be major factors in making Wikifunctions more accessible and find a
wider audience.
This puts a responsibility on the community to select accessible and
understandable names for functions and arguments, and to offer
documentation that makes a function relevant to the users of Wikifunctions.
However, just as many of the examples in al-Khwarizmi’s book had a limited
relevance to many people in many places in Europe, as they didn’t
follow Islamic
rules of inheritance
<https://en.wikipedia.org/wiki/Islamic_inheritance_jurisprudence>, we have
to be careful to draw on examples and naming sources beyond those rooted in
the Western world. Even though the underlying mathematics are universal,
the examples and use cases of the calculations often are not. I hope that
we as a community will be mindful of our responsibility.
You can find a scan of an English copy of al-Khwarizmi’s compendium on
Archive.org: archive.org/details/algebraofmohamme00khuwrich/
Interestingly, the translation also contains a translation into
mathematical notation, which for some modern readers might be easier to
read than the actual English translation. This would have been entirely
inconceivable in al-Khwarizmi’s time.
By the way, my cousin? She has grown up, and received a degree in economics
a few years ago.
DUCT: Doing Unique Ci Things
Stef Dunlap, a software engineer in Test Engineering embedded in the
Abstract Wikipedia team, is working on running end-to-end tests on
per-patch ephemeral test environments. The tests are orchestrated using
GitLab CI (Continuous Integration) on Wikimedia's Gitlab
<https://www.mediawiki.org/wiki/GitLab> instance. The ephemeral test
environments are run on a small kubernetes cluster on Cloud VPS
<https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS>. A small bridging
layer written in Rust running on Toolforge keeps these environments in sync
with Gerrit patches.
This novel approach, with the working title DUCT (Doing Unique Ci Things)
aspires to establish a pattern that other Wikimedia projects with auxiliary
services can use to run reliable end-to-end tests and manually test
features in single-tenant test environments (as opposed to the multi-tenant
Beta Cluster) while leveraging existing Wikimedia Foundation tools and
adopting new technologies that are being invested across the Wikimedia
movement.
Stef will present her work next Tuesday, February 21, 2023, at 17:00 UTC
<https://zonestamp.toolforge.org/1676998828>. The talk will be recorded and
made available afterwards. Interested parties are invited to join on Google
Meet, and there will be some opportunity for questions. The link to the
talk is: meet.google.com/tpn-fafq-ekj
Public NLG meeting on February 21, 2023
Next Tuesday, February 21, 2023, at 16:30 UTC
<https://zonestamp.toolforge.org/1676997047>, we will have our second
public NLG (Natural Language Generation) workstream meeting. The meeting
will be on Jitsi and can be joined here: meet.jit.si/AWVolunteersCorner
This meeting will feature a presentation by Maria Keet about the design of
the *"abstract content"* language for writing *"constructors"*, which are
those pieces of structured information that are positioned between Wikidata
and Wikifunctions as a source on the one side of the pipeline and the
machinery for rendering that content into natural language sentences or
paragraphs of text on the other side in the pipeline. We hope to take a
step forward from its discussion document
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Wikidata_Abstract_Repres…>
of
November 2022 towards more and more concrete, required features for the
data structure specification.
Apologies for the two events overlapping.
Development updates (as of February 10)
- In order to leave *Goal 2* (efficient and correct evaluation) in a
good state, work has started on replacing the current evaluator, which
covers all native programming languages we support, with individual
evaluators for each programming language. This will allow for easier
introduction of more programming languages in the future, and, more
importantly, to switch support for individual programming languages on and
off in case a need arises, e.g. to buy time in response to critical
vulnerabilities in a specific language runtime. This is a requirement from
the security review.
- In *Goal 6* (stable and secure system) many user rights and user
groups were defined and created. These are needed to define specific
actions that only certain users should be allowed to do, which are unique
to the Wikifunctions environment. That includes creating new types,
activating and deactivating implementations and testers, etc.
- In *Goal 5* (default view), work on the list component is ongoing,
which is one of the two major components remaining in this goal. At the
same time, designs for Goal 5 and for *Goal 9* (updating the other
components) have progressed or even finalized (including the selection of
icons for different modes, and a beautiful design for implementations).
- For *Goal 3* (meta-data), the last major patch for reordering
implementations has been sent to review, bringing us close to closing this
goal. This will allow us to choose efficient, instead of the current
approach of random, implementations to run for a given function, which will
make the runtime of the system more predictable and use fewer resources.
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-02-08
----
More thoughts on the Evaluation by the Google.org fellows
During the fellowship, the Google.org fellows gained detailed insight into
the Wikifunctions and Abstract Wikipedia project. With the goal to point
out potential issues and to discuss potential alternatives to some of the
project’s approaches, they wrote a detailed evaluation of the Wikifunctions
and Abstract Wikipedia projects. The team read through the evaluation and
wrote a detailed answer. Back in December, we published
<https://meta.wikimedia.org/wiki/Special:MyLanguage/Abstract_Wikipedia/Updat…>
the full documents:
-
Wikifunctions and Abstract Wikipedia: An Evaluation
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evalu…>
-
Answer to Wikifunctions and Abstract Wikipedia: An Evaluation
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evalu…>
The Signpost
<https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2023-01-01/Techn…>
reported on the evaluation, and an episode of the podcast Between the
Brackets <https://betweenthebrackets.libsyn.com/episode-130-ariel-gutman>
discussed it, among other things. Today, we want to provide a condensed and
not too technical summary of the documents. The documents were both long
and technical. We understand that a two page summary of a forty page
discussion will be necessarily simplifying and glance over a lot of nuance,
but for everyone who wants to dive into the details, the full documents are
available.
The evaluation describes many proposals and suggestions to change the
project, and culminates in eight sweeping recommendations. We have accepted
a large number of the suggestions the evaluation made. Some have already
been implemented, others are tasks on our project task board and are
awaiting implementation.
The main area of disagreement lies in the final eight recommendations. In
summary, the recommendations suggest that the Foundation should treat
Wikifunctions and Abstract Wikipedia as separate projects. Further, they
recommend dramatically narrowing the scope of both projects
For Wikifunctions, the evaluation recommends dropping the function model,
on which our multilingual user interface rests. They further recommend
cutting features that enable non-developers to contribute functions, and
focusing on a single programming language, Lua, instead of allowing for
several languages.
For Abstract Wikipedia, the evaluation recommends dropping the idea of
co-creating one or more natural language generation solutions with the
community, suggesting to select either an existing natural language
generation system, or to adopt the one that one of the fellows initiated
during the fellowship. Instead of creating an all-purpose function
repository that would allow the community to experiment and develop a
solution, the recommendation is for the project team to decide on and
hard-code a single solution to be a system external to the wiki integrating
the contributions of the editors.
This is similar to early critiques of Wikidata, which faced a very similar
suggestion: numerous voices declared that the decision to allow the
community to create and maintain the ontology of Wikidata, instead of
either choosing an existing ontology or hiring a team of ontologists, would
doom Wikidata. We lost some major funding because of sticking with that
decision.
The development team of Abstract Wikipedia and Wikifunctions rejected both
sets of recommendations.
One simple reason for some of the rejections was that the evaluation indeed
evaluated the technical plan, and not the current state of development. For
example, one recommendation is to focus solely on Lua instead of allowing
different programming languages, as the latter would take a long time to
implement. But given that we have already implemented that feature,
adopting the recommendation would slow us down and have us discard work
that had already been done.
Some of the rejections for the recommendations were based on Wikimedia
values. We think it crucial to have a project that people can contribute to
and use in their own natural language. We think it important to enable
non-programmers to contribute, and not just for the sake of diversity but
also simply because the pool of potential contributors with the relevant
skill set is very limited in many language communities. We need people to
create and maintain natural language generation functions in many
languages. Allowing for a wider skill set to contribute reduces this
potential bottleneck. This is why Abstract Wikipedia needs Wikifunctions,
and at the same time Wikifunctions benefits from Abstract Wikipedia as a
use case. It is a symbiotic relation.
There is currently no symbolic natural language generation system that
covers 300 languages, as the Wikimedia movement already does. We can not
confidently choose an existing solution, nor the novel one developed during
the fellowship. In fact, Prof. Maria Keet of the University of Cape Town,
South Africa, joined the Abstract Wikipedia development as a volunteer. She
criticized both the existing popular natural language generation systems as
well as the one created during the fellowship, because they wouldn’t work
well with Niger Congo B languages. The fellow’s proposal was updated to
better support the needs of such languages. We were able to do this because
our architecture is not tightly coupled to an external natural language
generation system.
We do not, and mostly cannot know whether the languages that have not yet
been implemented have features that would become a major problem for a
given system. In a general purpose, flexible system, the community will
have the power to fix any such issue. By selecting any single solution, we
would considerably increase the barrier to fixing problems. Instead of
contributing to a wiki, we would need developers contributing elsewhere,
e.g. in the case of SimpleNLG, to an upstream Java project through git, or
in the case of Grammatical Framework, to a project written in Haskell. And
worst of all, the communities that will most likely be adversely affected
by such a decision would be the ones that are already under-represented.
We are not against using an existing natural language generation system. We
will and already are supporting the community in leveraging existing
solutions (we are closely working with Grammatical Framework, which is
strongly recommended by the fellows). But at this point we will not commit
exclusively to any one of the existing solutions. Instead the plan was,
from the beginning, to allow the community to build, maintain, and own
their approaches, to enable the community to use and combine available
solutions, and to co-create Abstract Wikipedia with the community together.
We hope this summary of the eight recommendations and our decision helps in
understanding what we are building and why we made the decision.
What’s a good name for a process to improve?
Python has the PEP <https://peps.python.org/> (Python Enhancement
Proposal), Java the JSR <https://jcp.org/en/jsr/overview> (Java
Specification Request), and many wikis have the RfC
<https://www.wikidata.org/wiki/Q4654740> (Request for Comment, inspired by
IETF) or similar terms. Wikifunctions will be improving well after its
launch, and we will need a process to discuss these improvements, either on
wiki, on the mailing list, or in Phabricator. This process needs a name!
We won’t make a big vote or anything, but we would like to ask for ideas
for the name, and whether you have preferences or concerns. Some internal
suggestions have been
-
Community System Change (CSC)
-
Little Lambda chat (LLC)
-
Food for thought (FFT)
-
Wikifunctions Lambda Delta (WILD)
-
Wikifunctions System Consultation (WSC)
-
Functional Model Proposal (FMP)
Also if you have suggestions on how to run and structure the process of
changes to the formal Wikifunctions model initially, let us know. We expect
that this will change considerably over time, but there is likely plenty of
experience we can work from.
Weekly development update for February 3, 2023
Last week we had our two-monthly fix-it week, which is a meeting-light week
focused on fixing bugs and paying down technical debt. Some examples of the
work that was done include:
-
Fixing small bugs, e.g. fixWidth issues
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/886004>
or removing unnecessary checks
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/886048>
-
Improving test coverage, e.g. in serializer
<https://gerrit.wikimedia.org/r/c/mediawiki/services/function-schemata/+/885…>
or on the string component
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/760999>,
or re-enable temporarily disabled tests, e.g. for mapping empty lists
<https://gerrit.wikimedia.org/r/c/mediawiki/services/function-orchestrator/+…>
-
Renaming components and functions to make them clearer and more
consistent, e.g. rename ‘generic error’ to ‘unspecified error’
<https://gerrit.wikimedia.org/r/c/mediawiki/services/function-schemata/+/885…>
or add a prefix to components
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/884990>
-
Refactoring, e.g. restructure whole directories
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/885393>,
consolidate metadata dialogs into a single component
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/885846>
or generalize existing functions
<https://gerrit.wikimedia.org/r/c/mediawiki/services/function-schemata/+/885…>
Replacing components with more modern components, e.g. our own 'chip' with
the one from Codex
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/886143>
Hello everyone!
This is a friendly reminder that next Wikifunctions & Abstract
Wikipedia Developers-Volunteers' Corner will be held *Monday 6
February, at 18:30 UTC* (https://zonestamp.toolforge.org/1675708253)
The link to the meeting is the same as last time:
https://meet.jit.si/AWVolunteersCorner
If you have any questions or concerns, or if want to show your
progress on something you're working on, or if you just want to hang
around and get to know the project better, you're more than welcome to
join us!
Hope to see you there!
--
Luca Martinelli [Sannita] (he/him)
Community Relations Specialist
This week's newsletter can be found online here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-02-01
----
Goal 4 design update and closure
With goal 4, we improved the function definition experience. For example,
we optimized the layout, spacing, and information architecture of editing
the function definition on mobile. Previously, it was difficult to scan the
page top to bottom. Now, thanks to a more attentive use of white space and
type hierarchy, we are able to visually group related content.
Prior to this update, it was not possible to delete a function input on
desktop, while it was possible to do exactly that on mobile. We made sure
that no matter the device of your choice, the same features are available.
We also clarified which of the fields are required, and which are optional.
We added samples as placeholder texts, and included helper texts below each
field in order to provide better clarity about the expected type of content.
Last, but not least, we designed a bespoke publishing experience. We moved
the “publishing block” from the bottom of the function definition view to a
dialog, for a stronger separation of concerns.
With the implementation of these and many other designs, we have closed
Goal 4 of Phase θ in the development of Wikifunctions. The goal was to
update the Functions view in Wikifunctions, and to make the definition of
Functions as simple as possible.
Wikifunctions will have two main entry points that we hope can be used by a
wide number of people: using Functions from Wikifunctions, and defining
Functions for Wikifunctions. We understand that implementing a function may
be outside of what many contributors might feel comfortable with. But we
hope that we have designed an experience that will allow for a wide variety
of people to define a function, and thus to express their needs. As with
every wiki, we hope that people with different skill levels will be able to
work together, and that these definitions will then be picked up and
implemented by people with a skill set that makes them feel comfortable
with contributing implementations.
Goal 10 of Phase θ will then see us work on the UX for using Functions.
Before we get to that, we will work through Goal 5, where we will work on
the default component, which will be the main building block of our
UI, and Goal
9, where we use the default component in a number of core types.
ZObject Decoder
Stef Dunlap <https://en.wikipedia.org/wiki/User:KindRowboat> has developed
an online tool that, given a ZObject in JSON, augments the input with
helpful annotations in order to make them easier to read. Whereas we hope
that most users will never encounter a raw ZObject in the wild, for those
poor lost souls who do, this tool can come in mighty handy: find the ZObject
Decoder <https://zobject-decoder.toolforge.org/> at
https://zobject-decoder.toolforge.org/
Volunteer’s corner on Monday
This upcoming Monday will see our monthly Volunteer’s corner. The meeting
will be on Monday, February 6, 2023, 18:00-18:30 UTC
<https://zonestamp.toolforge.org/1675708202> and you can join on Jitsi on
the following link: https://meet.jit.si/AWVolunteersCorner
Bring your questions, your ideas, or even just your curiosity, and we will
find and help with places you can contribute.
Development update as of January 27, 2023
-
Goal 5 (Implementing the default view) is moving with good progress. The
base components, strings and references, have landed, as well as booleans
and monolingual text. Work has started on the two most complex components,
lists and function calls.
-
In preparation for Goal 9 (updating the other components), we have done
some awesome work identifying which components have to be reimplemented
with the current features to replace the current UI
-
Work for Goal 2 (efficient and correct evaluation) and Goal 6 (stable
and secure system) has been assessed, and we decided to not work on the two
goals in parallel. We will focus first on Goal 6.
-
In Goal 3 (meta-data) we are working on reordering implementations based
on the cached test runs.
-
This week we have our quarterly fix-it week, which is a meeting-light
week focused on fixing bugs and paying down technical debt
<https://en.wikipedia.org/wiki/Technical_debt>
.png){kind=link}
