The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-11-30
--
Welcome, Grace and Miguel!
This week we are happy to welcome two new members on the Wikifunctions team.
Grace Choi is joining us and the Foundation as a senior software engineer,
to work on all of the pieces of software Wikifunctions runs on. Here we
have Grace saying hello in her own words:
*As a classical violinist communicating in the small world of music (yes,
much smaller world than the small world as we know it), I long wanted to
learn how to communicate in the world of technology that was seemingly
dictating our very lives. After completing my masters, I carefully laid my
violin down for a rest and plunged into a bootcamp that introduced me to
web development. I then dabbled in some full-stack engineering followed by
nearly three years in backend development with Ruby; one could say the rest
is history.*
*I am thrilled to join the spectacular team at Abstract Wikipedia to
scrutinize yet another way to 'communicate' by way of functions and
linguistics to actualize an extraordinary project together. I don't have
many hobbies but does learning German on Duolingo count? For the winter
season, I have pulled out the Lord of the Rings books and am in search of
some delicious local (Seattle) hot cocoa to melt into my oat milk.*
Miguel Castro is joining us and the Foundation as our engineering manager,
to coordinate the engineers on the team, ensure their growth, and also
coordinate their work with the rest of the organization. Here we have
Miguel saying hello in his own words:
*I am excited and humbled to be part of the Abstract Wikipedia team! When I
was around 8, my mom bought me an old, worn-out encyclopedia set that
became a doorway to reading about everything that intrigued and fascinated
me. When I was around 10, I discovered computers and how they could grant
me access to knowledge. I began in the 90s, using my modem to connect to
local computers to download text files on subjects ranging from philosophy
to programming.*
*By the time I was 14, I was coding every day. After I graduated, I began
my first job in IT at 18 and have been working in the industry ever since.
I eventually graduated college with a degree in MIS and Computer Science.*
*I worked as a software engineer for most of my career, primarily with
open-source technologies. Today, my focus is engineering leadership and how
to bring people together to create things we are proud of and that have a
positive impact. Lastly, my work here with the Abstract Wikipedia team is a
full-circle moment as it connects me to why I started this journey.*
Please join me in welcoming Grace and Miguel to the team!
Functioneer rights control handed over to community
<https://meta.wikimedia.org/wiki/File:Wikifunctions_Functioneer_Icon.svg>Wikifunctions
Functioneers logo
The community has in the last few weeks successfully worked on and agreed
on a policy for Functioneer rights
<https://www.wikifunctions.org/wiki/Wikifunctions:Functioneers#How_to_request>.
Thank you all! Given that this is in place, we are happy to relinquish the
responsibility of giving out Functioneer rights, and allow the community to
implement the new rights.
All Functioneer rights that have been handed out by us so far have been
temporary and will expire by themselves, but the community is free to
manually speed that up if they so want. It is all up to you.
With that, we will archive the current request page
<https://www.wikifunctions.org/wiki/Wikifunctions:Apply_for_editing>. Thank
you for working with us on this step!
Recent changes to Wikifunctions software
Our main focus right now continues to be on better support of types (T343469
<https://phabricator.wikimedia.org/T343469>). To this end, the biggest
visible change this week is our continued work on the front-end to allow
the use of lists as inputs and outputs of functions (T326301
<https://phabricator.wikimedia.org/T326301> mainly, plus T351276
<https://phabricator.wikimedia.org/T351276> and others). This feature is
not quite ready yet for us to encourage use, as we're working on their uses
in compositions (T351272 <https://phabricator.wikimedia.org/T351272>) and
other places – expect progress in the next few weeks!
On the back-end, we've struggled through several related issues that were
blocking progress or even deployment (T350700
<https://phabricator.wikimedia.org/T350700> and T352328
<https://phabricator.wikimedia.org/T352328>, alongside several others), and
we've now finally updated the evaluator service to our latest work. Our
thanks to colleagues in SRE for their help on how we could hunt down
evidence to debug our issues. Beyond being a little faster now, the main
user-visible change from the newly-updated services is that reports of
duration less than 50 µs, previously reported as 0 (as the OS tries to
mitigate timing attack security concerns), will now show "< 50 µs".
We've changed the mechanism we use to pass your function calls into the
WebAssembler environment after we found our original approach caused some
hard-to-debug errors in some cases (T349385
<https://phabricator.wikimedia.org/T349385>). Additionally, a number of
rare error-states will now return proper, translatable errors, rather than
hard-coding a message in English, such as for timeouts (T327275
<https://phabricator.wikimedia.org/T327275>), recursion limits (T350608
<https://phabricator.wikimedia.org/T350608>), and invalid responses (T349785
<https://phabricator.wikimedia.org/T349785>).
For minor issues, we fixed and released early a bug that hid the button to
trigger test runs on implementations (T351121
<https://phabricator.wikimedia.org/T351121>), and we altered the "Function
explorer" box on Implementation and Test case pages to show now just the
ZID but the label when inputs are function-call types like a list (T351274
<https://phabricator.wikimedia.org/T351274>)
You can browse the full list of deployed changes
<https://www.mediawiki.org/wiki/MediaWiki_1.42/wmf.7#WikiLambda> for the
MediaWiki front-end for Wikifunctions. We didn't deploy any back-end
service changes this week.
Volunteers’ Corner on Monday, 4th of December 2023
We will have our monthly Volunteers’ Corner on the upcoming Monday 4
December 2023 at 18:30 UTC <https://zonestamp.toolforge.org/1701714600>.
You can join the meeting in the usual Google Meets room
<https://meet.google.com/xuy-njxh-rkw>.
We will give a quick update on the development and upcoming changes, offer
space for questions, and, if time permits, collaboratively create a new
function. Bring ideas for a new function along!
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-11-16
--
What types next?
We currently support two types: strings and Booleans. Our next big step for
expanding the capabilities of Wikifunctions is by introducing more types.
We are working hard to get all the pieces in place: we have previously
talked about serialization
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-27> to
make types work seamlessly with programming languages, and renderers and
parsers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20> to
improve the experience when using them.
<https://meta.wikimedia.org/wiki/File:Data_types_-_en.svg>Data can take
many different types for many different domains
The first new type we expect to open for the community is a type for lists.
This will not only allow for many new functions that are not possible yet
(such as breaking a sentence into words), but also to simplify existing
implementations such as this test for mutation in Breton
<https://www.wikifunctions.org/view/en/Z11601> or this (overly simple) check
if a letter is a vowel <https://www.wikifunctions.org/view/en/Z11894>,
implementing a function we created collaboratively in this week’s Volunteer
Corner.
But, as asked by Cool314 on the Project Chat
<https://www.wikifunctions.org/wiki/Wikifunctions:Project_chat#Once_we_get_n…>,
what should be the next types that the community would like to see? They
also suggest starting with the list type, which, as said, is on the way.
They say to follow with integers, and if that means non-negative integers
or counting numbers, I would be totally on-board. But what would be next?
The list of suggested functions requiring new types
<https://www.wikifunctions.org/wiki/Wikifunctions:Suggest_a_function#Propose…>
also
strongly suggests counting numbers. After that we find suggestions talking
about more complicated numbers, such as negative numbers, floating-point
numbers, fractions; then also bytes and specific-length vectors of bytes;
colors; years, months, and dates; and others.
One thing we need to take into account is to go for simpler types first:
for example, a calendar day could be built from a counting number and a
month, or it could be built from two counting numbers – but one way or the
other, we would need counting numbers first.
Another question is whether we prefer to have our types to be built from
simpler elements, or whether we prefer more complex types. To give a very
simple example, there are many different ways to represent integers, two of
which would be:
1. An integer could be represented by an object with a single key, a
single string that starts with an optional “-” and ends with a whole
number, i.e. a list of digits with no leading zeros
2. An integer could be represented by an object with two keys, one being
the sign (which is one of negative, positive, or none), and the other a
counting number, which we would have previously defined as a type
Then there is also the question of which string would represent which
value, a question that needs to be answered for each individual type. For
numbers one straightforward solution is to take the string representation
of the numbers in Hindu-Arabic numerals, but one could also consider
binary, hexadecimal, or even base64 encodings, potentially reducing storage
space. I think that the potential to more easily understand a JSON
representation would beat the small storage gain.
Finally, what limits, if any, should we put on these values? Frequently,
programming languages have a numerical limit for their 'number' type of the
range from 0 to 4.2 billion (2^^32) or 0 to 18 qunitillion
<https://en.wiktionary.org/wiki/quintillion> (2^^64). Should we add a limit
like this as well, or should we expect function writers to work with any
possible input?
Note that this does not mean that we only will be able to deal with
Hindu-Arabic numerals: with parsers and renderers
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-09-20> we
will be able to display numbers appropriately for a given language,
remaining fully translated for whichever language and locale users are
using. We might not necessarily have that feature available immediately
when introducing new types, but we will be working on enabling them soon.
Let us hear and discuss what you think (or even come up with a process for
us to follow, based on the types you are suggesting). What are the types
you are looking forward to?
Recent changes to Wikifunctions software
From now on, we will try to give a quick summary with each update of what
work you can see as it rolls out to Wikifunctions.
In terms of big items, following the completion of "General Availability"
last week
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-11-07>,
we've been primarily working on the software to better support types (as
discussed above; T343469 <https://phabricator.wikimedia.org/T343469>). On
the back-end this has meant infrastructure support for applying custom code
to convert from Wikifunction types to native types and back (T297509
<https://phabricator.wikimedia.org/T297509>), and on the front-end we've
been working on using lists as inputs and outputs of functions, coming soon
(T326301 <https://phabricator.wikimedia.org/T326301>). We've also been
working with several volunteers to understand how we can improve the
on-boarding experience (T285509 <https://phabricator.wikimedia.org/T285509>);
thank you to everyone who has taken part.
Our general improvements have included progress on being more consistent in
using proper, specific, translatable errors consistently throughout the
system (T321113 <https://phabricator.wikimedia.org/T321113>), and
researching if it's possible to make any further simple improvements to
picking a type or object (T345547
<https://phabricator.wikimedia.org/T345547>).
As minor fixes, we now use a different title on the creation page depending
on whether you're making a function, implementation, test case, or type,
rather than just say 'object' (T350673
<https://phabricator.wikimedia.org/T350673> and T341847
<https://phabricator.wikimedia.org/T341847>). We also fixed the view of
aliases to be one bulleted list of several items, not several lists of just
one (T345404 <https://phabricator.wikimedia.org/T345404>).
You can browse the full list of deployed changes
<https://www.mediawiki.org/wiki/MediaWiki_1.42/wmf.5#WikiLambda> for the
MediaWiki front-end for Wikifunctions. We didn't deploy any back-end
service changes this week.
No newsletter next week
Due to holidays, we will be skipping next week’s newsletter. Expect to hear
from us again after Thanksgiving
<https://en.wikipedia.org/wiki/Thanksgiving>!
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-11-07
--
Wikifunctions, the library of functions that anyone can use and edit
As of a few days ago, Wikifunctions can be used by anyone.
<https://meta.wikimedia.org/wiki/File:Anyone_can_edit_Wikifunctions.png>Wikifunctions
can now be edited by anyone!
That means that everyone visiting Wikifunctions is able to run functions.
Until now, this feature was limited to logged-in users only.
The last two times
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-10-25> we
talked about re-implementing our backend to run on WebAssembly
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-11-03>.
Since then, we have monitored the system and deployed a number of further
security features. We have improved the monitoring of the system to notice
issues sooner. We have also moved Wikifunctions.org to be the first
Wikimedia project to entirely run on Kubernetes
<https://en.wikipedia.org/wiki/Kubernetes>.
All of these steps gave us the confidence to drop the requirement to be
logged-in in order to run approved functions on Wikifunctions. We will be
monitoring the system, and in case we notice more load than we can handle,
we might be limiting function calls again. This might be a bit of a bumpy
ride, and we will see in the coming weeks and months how this will develop.
Thank you for your patience so far, and thank you for continued patience in
the future!
Furthermore, we have also considerably opened up editing rights. From now
on, all logged-in users can propose and improve draft functions, tests, and
implementations, rather than just special users with the Functioneer status.
Functioneers retain their role with the ability to connect and disconnect
tests and implementations on functions, which makes the function "live" so
that people can use it. The current set of Functioneers were all granted
their rights for a limited amount of time (for a few more months). We are
asking the community to set up a process to assign Functioneer rights
<https://www.wikifunctions.org/wiki/Wikifunctions:Project_chat#Your_input_ne…>
to
users, and keep a healthy number of Functioneers around.
For now, we will still not assign any function maintainer rights either.
Function maintainers will be able to do very wide-ranging, potentially
damaging edits, e.g. changing the definition of a type, or editing
connected implementations. There is no proper support for these workflows
yet, which is why we will not give out those rights for now.
With these changes, we are also dropping the word “soon” from the tagline
on the Wikifunctions main page: "Wikifunctions is a free library of
functions that (soon) anyone can edit.” We consider Wikifunctions now to
have reached general availability.
There is a lot more work to do. One of our next goals, which I'll post more
about next week, is to support more types beyond strings and Booleans, and
thus to allow many more functions to be created and made available.
Thanks so much to James
<https://meta.wikimedia.org/wiki/User:Jdforrester_(WMF)> for leading this
effort! And thanks to all the other team members who have worked on their
part, either within the Abstract Wikipedia team, within Security, or within
SRE (Site Reliability Engineering). We are excited to keep an eye on how
things develop from here on.
The on-wiki version of this newsletter can be found here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-11-03
--
Running Python on WebAssembly
<https://meta.wikimedia.org/wiki/File:Wikifunctions_-_object_selector_improv…>A
screenshot of the improved selector, finding types related to the input as
labels or aliases in different languages. The input, "cha", matches the
Polish "Checha" for Key, the French "Chaîne" for String, the English alias
"Unreachable" for Nothing, and the English alias "Character" for Code point.
As reported last week
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-10-25>, we
had switched our runtime for JavaScript to use a WebAssembly-based stack.
We still had to do the same for the Python language.
As of Wednesday, Python code is now also being executed in a
WebAssembly-based runtime. This means that now all user-written code will
be executed on a WebAssembly runtime, whether it is in JavaScript or Python.
The change turned out to be more difficult than anticipated. We tried
several different ways, and discovered entirely novel ways our testing
infrastructure could run into interesting issues. This has uncovered enough
work for weeks! Finally, we ended up using Wasmtime
<https://wasmtime.dev/> with
a locally-compiled RustPython <https://rustpython.github.io/> module. There
are a number of improvements and simplifications we want to work on in the
coming weeks and months, but for now we are happy that the system seems to
run in production. The WebAssembly-based Python runtime we have now seems
to run a bit slower than the previous one, but it is likely that you won’t
notice a difference.
With this change, we have completed adding an additional layer of security
to Wikifunctions. This way we are really close to actually opening up
editing. We will continue monitoring the systems, and if everything looks
good, we will soon open for wider editing.
Thanks to Cory and James for integrating and deploying the WebAssembly
runtime!
Improvement in the object selector
<https://meta.wikimedia.org/wiki/File:Wikifunctions_-_object_selector_improv…>Language
search
Another set of improvements were deployed to the object selector, the
widget that allows you to select a function or other object. You use this
every time you create a function definition, and often when calling them,
so it's important that we make it helpful for key parts of the workflow. It
should be much more descriptive now because it shows the right label and
type. It is also better at finding the right object because it’s taking
aliases and languages into account. For finding languages, now also BCP 47
<https://en.wikipedia.org/wiki/IETF_language_tag> (and when different, also
their MediaWiki) language codes are taken into account.
It used to be that you would sometimes need to type in raw ZIDs to find the
right object. This should now be needed much more rarely, and work more
fluently when you do need it. Let us know if you ever run into such a
situation, so that we have input for further improving the widget.
Thanks to Geno and Amin for improving and redesigning the object selector!
Also thanks to the community members who suggested improvements, including
GZWDer and egezort.
Volunteer’s Corner on November 13, 2023
<https://meta.wikimedia.org/wiki/File:Wikifunctions_-_object_selector_improv…>Searching
using the ZID
The next volunteer’s corner will be on November 13, 2023, at 18:30 UTC
<https://zonestamp.toolforge.org/1699900200>. We will meet in
meet.google.com/xuy-njxh-rkw. Bring your questions, and if time permits, we
will build a new function together.
Hello (Semantic) MediaWiki users, maintainers, software developers, consultants, researchers!
The SMWCon in fall 2023 https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_2023 will be held on location in Paderborn, Germany from December 11-13.
On three days there will be talks, tutorials and hackathons.
This conference addressed everybody interested in wikis and open knowledge, especially in Semantic MediaWiki, e.g. users, developers, consultants, business or government representatives, and researchers.
This conference aimed to:
. inspire/onboard new users,
. inform on where and how MediaWiki is used, . convey and consolidate best practices, . initiate/foster/integrate application and development and . strengthen the community of stakeholders and its service portfolio.
Learn how to "do" MediaWiki in order to assume your responsibilities regarding your organization's knowledge management.
Please not that early-bird ticket sale ends today!
Call for Contributions
----------------------------
We are looking for use cases and best practices that provide insight in issues like
* How does AI change the way we use MediaWiki
* How do semantic wikis fit in and be combined with AI tools
* How can we use Semantic MediaWiki in research and organizations
* How do we develop and deploy MediaWiki and extensions
Your experience is valuable for all of us! So please share and propose a talk, tutorial or other contribution.
Go to the Conference Page (https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_20223)
and hit the 'Propose a talk here' button.
Please propose a contribution if you plan to have one, even if you don't have the details yet. For us it is important to know what we can expect.
We look forward to your contribution!
Sponsoring
----------------
Thank you to the sponsors of SMWCon 2023!
* http://www.archixl.nl/ - Specialists in enterprise architecture, knowledge management, and semantics
* https://bluespice.com - The company behind BlueSpice, the open-source enterprise wiki software
* https://mywikis.eu - GDPR compliant (Semantic) MediaWiki hosting from the heart of Europe.
* https://wikibase-solutions.com/ - Specialist in business solutions with MediaWiki
Organization
------------------
The organizers of SMWCon 2023 and https://mwstake.org
* Bernhard Krabina, https://km-a.net (General Chair)
* Ad Strack van Schijndel, https://www.juggel.com (Program Chair)
* Tobias Oetterer, https://www.uni-paderborn.de/en/ (Local Chair)
