The on-wiki version of this week's newsletter is available here: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-06-30
--
When we started the development effort towards the Wikifunctions site, we subdivided the work leading up to the launch of Wikifunctions into eleven phases https://meta.wikimedia.org/wiki/Special:MyLanguage/Abstract_Wikipedia/Phases, named after the first eleven letters of the Greek alphabet.
- With Phase α (alpha) completed, it became possible to create instances of the system-provided Types in the wiki. - With Phase β (beta), it became possible to create new Types on-wiki and to create instances of these Types. - With Phase γ (gamma), all the main Types of the pre-generic function model https://meta.wikimedia.org/wiki/Special:MyLanguage/Abstract_Wikipedia/Pre-generic_function_model were available. - With Phase δ (delta). It became possible to evaluate built-in implementations https://meta.wikimedia.org/wiki/Special:MyLanguage/Abstract_Wikipedia/Updates/2021-05-11 . - This week, we completed Phase ε (epsilon).
The goal of Phase ε has been to provide the capability to evaluate contributor-written implementations in a programming language.
What does this mean? Every function in Wikifunctions can have several implementations. There are three different ways to express an implementation:
1. As a *built-in* function, written by the development team: this means that the implementation is handled by the evaluator as a black box. 2. As *code* in a programming language, created by the contributors of Wikifunctions: the implementation of a function can be given in any programming language that Wikifunctions supports. Eventually we aim to support a large number of programming languages, but we will start small. 3. As a *composition* of other functions: this means that contributors can use existing functions as building blocks in order to implement new capabilities.
In Phase ε, we extended the infrastructure for evaluating functions to allow for the running of contributed code in addition to the built-ins from Phase δ. For now, we support two programming languages: JavaScript https://en.wikipedia.org/wiki/JavaScript and Python https://en.wikipedia.org/wiki/Python_(programming_language). We plan to add more programming languages (most familiar to the Wikimedia community, Lua https://en.wikipedia.org/wiki/Lua_(programming_language)) and we will document a process for requesting the support of additional programming languages. We implemented the planned architecture https://www.mediawiki.org/wiki/Extension:WikiLambda#Architectural_concept: https://meta.wikimedia.org/wiki/File:Wikifunctions_-_Top-level_architectural_model.svg https://meta.wikimedia.org/wiki/File:Wikifunctions_-_Top-level_architectural_model.svg Wikifunctions top-level architectural model
We have now a system where the orchestrator receives the function call to be evaluated, gathers all necessary data from Wikifunctions and potential other resources, and then chooses the corresponding evaluators that can run the given programming language. Since this is contributor-created code, we are very careful about where and how to run the code, and which capabilities to give to the virtual machine that runs the code. For example, no network access and no persistence layer is allowed, in order to reduce the potential for security issues.
A security review and a separate performance review of our architecture and implementation are currently ongoing. Once we have dealt with the most pressing issues that are uncovered by the reviews, we plan to provide a demonstration system. This will probably be in the next Phase.
The next screenshot shows us an implementation of the Function “Concatenate”. A concatenate https://en.wikipedia.org/wiki/Concatenation function takes two strings as its arguments and returns a single string consisting of the two input strings joined end-to-end. Our implementation uses JavaScript’s native concat method, which is provided by JavaScript’s string object. https://meta.wikimedia.org/wiki/File:Wikilambda_Phase_%CE%B5_Code.png https://meta.wikimedia.org/wiki/File:Wikilambda_Phase_%CE%B5_Code.png Screenshot of an implementation in JavaScript
The next screenshot shows the function being called using the arguments “Wiki“ and “functions”, resulting in the string “Wikifunctions”. https://meta.wikimedia.org/wiki/File:Wikilambda_Phase_%CE%B5_Function_Call.png https://meta.wikimedia.org/wiki/File:Wikilambda_Phase_%CE%B5_Function_Call.png Screenshot of a function call
We are now moving on to Phase ζ (zeta). The goal of this Phase will be to allow for the third type of implementations: the composition of functions in order to build new capabilities. This will also be the first Phase to really highlight the advantages of our system for contributing implementations in non-English languages. We have published a few examples of composed implementations https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Examples/Function_composition. The example implementation of common Boolean functions might be particularly instructive. Phase ζ is the last of the trilogy of Phases dealing with the different ways to create an implementation.
During this Phase and subsequent ones, we will also spend some time to reduce technical debt that we accumulated in rushed development during the last two phases in order to be ready for the security and performance reviews. We also expect to begin early changes based on our user research and design development, replacing the current bare-bones user experience.
Structurally, Phase ζ aims to be the turning point of the development towards Wikifunctions. It will set us up with a system that is powerful enough to allow for its own refactoring in order to support generic types and functions in Phase η (eta), and then to implement monitoring, UX, security, and documentation. All the core technical capabilities should be in place by then, and then we will need to add the necessary supporting systems that will allow us to launch Wikifunctions. ------------------------------
Last week saw the Arctic Knot conference https://meta.wikimedia.org/wiki/Arctic_Knot_Conference_2021, which was about the future of indigenous and underrepresented languages and their presence and use on the Wikimedia projects. I want to point to a few talks that are potentially particularly interesting to the Abstract Wikipedia and Wikifunctions community:
- Mahir talked on preparing languages for natural language generation using Wikidata lexicographical data https://www.youtube.com/watch?v=f13c3lCghtE&t=27291s - Denny gave an introductory presentation on Abstract Wikipedia https://www.youtube.com/watch?v=f13c3lCghtE&t=30581s and suggestions on what tasks we want to tackle first - Sadik and Mohammed talked in Dagbani Wikipedia Bachinima https://www.youtube.com/watch?v=ee1TaLK3dJE about challenges and successes of growing the project, including the Spell4Wiki app
Thanks everyone for attending, presenting at, and organizing the event!
We close with a reminder that there is an invitation to attend for free https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-06-24 the Grammatical Framework Summer School http://school.grammaticalframework.org/2021// from 26 July to 6 August 2021.
The language committee has approved the creation of the Dagbani language Wikipedia https://phabricator.wikimedia.org/T284450. Dagbani is one of our focus languages, and has been the only one of the five that has been in Incubator. Congratulations to the Dagbani community!
Next week there will be no newsletter.