- Abstract-Wikipedia - lists.wikimedia.org

Newsletter #109: Selecting the right implementation, Reflections by Maria Keet, Abstract Wikipedia in Swedish
by Denny Vrandečić 20 Apr '23

20 Apr '23

The on-wiki version of this newsletter can be found here: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-04-19 -- Selecting the right implementation Functions in Wikifunctions can have more than one implementation. For example, if we have a function that capitalizes the first letter <https://wikifunctions.beta.wmflabs.org/wiki/Z10577> of a word, we can have several implementations, e.g. one <https://wikifunctions.beta.wmflabs.org/wiki/Z10711> or two in Python <https://wikifunctions.beta.wmflabs.org/wiki/Z10713>, one in JavaScript <https://wikifunctions.beta.wmflabs.org/wiki/Z10712>, and one using composition <https://wikifunctions.beta.wmflabs.org/wiki/Z10579>. You might find some of the implementations surprising. We previously discussed why we made the design choice to allow multiple implementations <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-06-17> for a single function. Until recently, Wikifunctions selected an implementation at random. Meaning, whenever someone was calling a function and there were multiple implementations available, Wikifunctions would select the implementation to be used randomly. Implementations of the same function can have wildly different runtime behavior. Some can be very slow, and others can be very fast: sorting a list of 100,000 random numbers using bubble sort <https://en.wikipedia.org/wiki/Bubble_sort> can take a minute on a current processor, but with quicksort <https://en.wikipedia.org/wiki/Quicksort> the same list of numbers can be sorted in less than two hundredth of a second - faster than the blink of an eye. Much faster. In Wikifunctions, functions should be accompanied by testers. The capitalization function we talked about earlier has only one tester <https://wikifunctions.beta.wmflabs.org/wiki/Z10578> as this is being written, that checks that capitalizing the word “test” returns “Test”. If all goes well, Wikifunctions will run each tester on each implementation. The results of these tests are stored: does the implementation pass, how many resources does it require, and other meta-data. This run-time information is also shown to the user in a pop-up on request, for people interested in the back-end details. Wikifunctions now ranks the implementations based on this meta-data, and updates the internal order of the implementations. Test failures result in downgrades, and quick results lead to a better ranking. And so, for the last few weeks, instead of selecting an implementation at random, we now select the first implementation based on that ranking. Here is an example of that reordering <https://wikifunctions.beta.wmflabs.org/w/index.php?title=Z10577&diff=prev&o…> working in practice (but alas, diffs are not implemented yet). This should lead to a considerable reduction in used resources, and to a more consistent behavior of Wikifunctions. Function calls should produce timeouts less often. This should also relieve the Wikifunctions community from worrying about inefficient implementations and whether we should accept them or not. Often, algorithms which are simpler are easier to read and verify, but are slower: bubble sort is a good example of this, compared with quicksort. Bubble sort is generally regarded to be much easier to explain and understand than quicksort. Having both allows for the results of the simpler implementation to be compared to results of the more complex implementation, with both passing the same suite of testers, and thus increase our confidence in the overall system. At the same time, we can in practice use the more efficient implementation and thus reduce overall resource usage. With this, the first version of a major element that will work behind the scenes of Wikifunctions has been put into place, and we have delivered another goal of the current phase. Maria Keet’s reflection on Abstract Wikipedia so far Maria Keet <http://www.meteck.org/> has been an active and central part of the Natural Language Generation Workstream. She is a professor at the University of Cape Town, South Africa, and her collaboration with Ariel Gutman <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-12-19> on the template language <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Template_Language_for_Wi…> and her arguments have been mentioned in the fellows’ evaluation <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evalu…> and the answer <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evalu…>. Maria has now written down her own reflections and published them on her blog: keet.wordpress.com/2023/03/14/some-reflections-on-designing-abstract-wikipe… The text is very accessible, gives context, and explains some of the issues that low resource languages face, and makes suggestions on how to proceed. Maria also describes some of the frustrating challenges she encountered in having her voice heard and recognized. That part makes for a painful read, and points to necessary changes. To repeat her closing words: The mountain we’ll keep climbing, be it with or without the Abstract Wikipedia project. If Abstract Wikipedia is to become a reality and flourish for many languages soon, it needs to allow for molehills, anthills, dykes, dunes, and hills as well, and with whatever flowers available to set it up and make it grow. We are thankful to Maria for her ongoing contributions. We hope that we can achieve a more inclusive space, with the goal to have contributing become a more wholesome experience. Talk about Abstract Wikipedia in Sweden Professor Aarne Ranta <https://www.cse.chalmers.se/~aarne/> will give a talk on Natural Language Generation and Abstract Wikipedia on Thursday, April 20th, 2023 at 17:30 local time, in the Maritime Museum and Aquarium <https://sv.wikipedia.org/wiki/Sj%C3%B6fartsmuseet_Akvariet> in Göteborg, Sweden. The in-person event is free for the public. The talk will be given in Swedish. You can find more information about the talk in Swedish here: https://www.vetenskapsfestivalen.se/for-alla/kunskap-utan-granser-abstract-…

2 2

My Wikipedia
by Shuvo Matubbar64 10 Apr '23

10 Apr '23

Sir, I am shuvo matubbar, My Wikimedia name shuvo matubbar. I am always government service and special person.

1 0

No updates for the next two weeks
by Denny Vrandečić 31 Mar '23

31 Mar '23

Hi all, Just a heads up that this week and the two following weeks will be without the usual updates. It's not that nothing update worthy is happening - on the contrary - but I will be taking vacations, and will not be working during that time, and my colleagues are busy enough that I decided not to delegate this task this time around. We will in the future set up a system that does not rely on me, but given that we are not launched yet, this has not been our priority so far. Volunteer's corner on April 3rd will proceed as planned. Enjoy the time, and see you again in the third week of April. All the best, Denny

2 1

Newsletter #108: Abstract Wikipedia and Grammatical Framework
by Denny Vrandečić 23 Mar '23

23 Mar '23

The on-wiki version of this newsletter can be found here: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-03-23 --Abstract Wikipedia and Grammatical Framework Aarne Ranta <https://www.cse.chalmers.se/~aarne/>, a member of the Abstract Wikipedia NLG Special Interest Group, and professor at the University of Gothenburg, Sweden <https://en.wikipedia.org/wiki/University_of_Gothenburg>, has written an article titled “Multilingual Text Generation for Abstract Wikipedia in Grammatical Framework: Prospects and Challenges”. Here is the abstract of the article: Abstract Wikipedia is an initiative to produce Wikipedia articles from abstract knowledge representations with multilingual natural language generation (NLG) algorithms. Its goal is to make encyclopaedic content available with equal coverage in the languages of the world. This paper discusses the issues related to the project in terms of an experimental implementation in Grammatical Framework (GF). It shows how multilingual NLG can be organized into different abstraction levels that enable the sharing of code across languages and the division of labour between programmers and authors with different skill requirements. The plan is to start with a simple but functional multilingual NLG system and to proceed towards more and more sophisticated language and wider coverage of topics, also allowing a human in the loop to create content via a Controlled Natural Language (CNL). The paper can be read here: https://link.springer.com/chapter/10.1007/978-3-031-21780-7_6 It can also be accessed as a preprint free of charge from the following link: https://www.grammaticalframework.org/~aarne/preprint-AAM-textgen.pdf

1 0

Newsletter #107: Quo Vadis, Abstract Wikipedia?
by Denny Vrandečić 21 Mar '23

21 Mar '23

The online version of this newsletter can be found here: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-03-15 Quo vadis, Abstract Wikipedia? Abstract Wikipedia will allow more people to contribute their voices to a baseline of knowledge, whilst working in an interface in their language. This shared baseline of knowledge will then be made available in many languages. This will be achieved by creating, storing, and maintaining the baseline of knowledge, per individual article, in a notation that is independent of natural language. We call content written in that notation "abstract content". This abstract content will then be turned into text in a specific natural language, with the help of the library of functions from Wikifunctions. Thus, Abstract Wikipedia will allow a speaker of any language to contribute content for readers of many different languages. This will allow more people to read more content in their language. Example Assume that we want to create a new Wikipedia article for the planet Jupiter, and the first version of this article shall be the following (these are the first two sentences of the Simple English Wikipedia <https://simple.wikipedia.org/wiki/Jupiter> article): *Jupiter is the largest planet in the Solar System. It is the fifth planet from the Sun.* The abstract content representing this natural text could look like this: Article( text: [ Superlative( subject: Jupiter, quality: large, class: planet, location constraint: Solar System), Definition( subject: Jupiter, definition: Rank( rank: Positive integer( value: 5), object: planet, by: Relational noun( noun: distance, to: Sun)))], categories: [Jupiter, planet, Solar System]) Wikifunctions would have types for Article, Superlative, Definition, Rank, *etc.*, which are used here as the abstract notation for the content of these two sentences. This abstract content is shown using labels in English here for our convenience, whereas in fact they would all be ZIDs (from Wikifunctions) and QIDs (from Wikidata). There will be software components to provide for the viewing, creation, and editing of abstract content. Wikifunctions will then also provide functions that take this object as an argument and generate natural language text such as the above. One question that needs to be answered is where these objects would be stored, and how to associate the above object with the Wikidata item for Jupiter, Q319 <https://www.wikidata.org/wiki/Q319>. We were originally planning to have this conversation and decision before the launch of Wikifunctions, but looking at the complexity of the system and the fact that it is very difficult to imagine given that so little of it is tangible so far, we decided not to open this question for discussion now, but to wait until after the launch of Wikifunctions, when we will all have a better understanding of how that part of the ecosystem works. Below, we outline a few options that came up in the discussion between folks on the Abstract Wikipedia team and the Wikidata team at Wikimedia Deutschland. It also ties to some of the questions Lydia Pintscher and I were answering in an interview on the Wikimove podcast episode <https://meta.wikimedia.org/wiki/WIKIMOVE/Podcast> that was released today and that we invite you to listen to. Thanks to Nicole Ebber and Nikki Zeuner for the interview! We are genuinely undecided about the best answer, and we would benefit from a wider discussion of the options, and potentially other options as well. Please also ask questions - these can often clarify and shine light on points that are muddy to us as well. We currently are focusing on the following five options: 1. A new tab on items in Wikidata 2. Create a new data type for objects on Wikidata 3. Objects on Wikifunctions 4. Objects on a new Wikipedia language edition 5. Unattached namespace on an existing project Let’s discuss these five options in the following. Option 1: A new tab on items in Wikidata We could add a tab, leading to a new namespace on Wikidata with a new content type, where the abstract content would live. This namespace would be attached to the item namespace in Wikidata. This way, every abstract content would have a natural place to store its associated abstract content. Given the little use of the item talk pages on Wikidata, it seems an additional talk page may not be valuable, so we might want to redirect people to the item's main talk page for any discussions. One big question would be where to store content for Wikivoyage and other projects about the same items (*e.g.* the abstract content we might want to write about Q90/Paris from the perspective of Wikipedia would be different from that for Wikivoyage). Would that need yet another namespace? Adding a new associated namespace would be a technical challenge; adding several would be a complex task, and we hope that not to happen often. Option 2: Create a new data type for Wikifunction objects We could create a new data type on Wikidata for Wikifunctions objects. Then the community could create a property on Wikidata that stores the abstract content on a given item as a literal. This would have the added flexibility that the community could add more abstract contents to an item for specific use cases, e.g. to represent content from Wikivoyage, or to represent the history of an item, the etymology of a word, *etc.* We will need such a datatype and the UX for it anyway, given our planned support for abstract descriptions <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-07-29>. In fact, this might be the simplest way to support abstract description. However, the UX of Wikidata doesn't lend itself to this easily, and adapting to this model would be challenging given the constraints of an item page. Existing properties are edited as one or a small number of simple, short text boxes, often with auto-complete; abstract content would instead be a larger text area, with helpers and possibly a toolbar, a preview control, *etc.*. One option could in principle be a modal dialog for editing, but these come with their own inherent UX downsides and are usually more complex to implement than the same functionality in its own dedicated environment. Also, this would break the current design patterns of an item page, and may not be aligned with the patterns that might be planned for its future. Further, while abstract content is somewhat structured and machine-readable, it is less (or differently) so than an Item, and its structure would probably not be queryable with SPARQL. This option comes with two additional challenges: some Wikidata items, already nearing the maximum size, would grow still further, and we would need a solution to allow that, and second how to deal with the visualization, editing, and diffing of potentially very large and complex values. Option 3: Objects on Wikifunctions Instead of having objects live in Wikidata as the values of statements or on an additional tab, Wikidata could merely store a pointer to an object on Wikifunctions. We still would create a new data type, but that data type would be just the ZID of an object stored in Wikifunctions. This would solve all the challenges of the previous option, and retain many of its advantages. It would have the additional advantage that several items could refer to the same object on Wikifunctions. Whereas this sounds rarely useful for the abstract content of Wikipedia articles, this might prove very useful for abstract descriptions of Wikidata items and the abstract glosses of lexemes. With the creation of abstract content, types, and natural language generation functions on the same platform, collaboration between people who focus on one of these areas would be more direct. This option could be a challenge for the Wikifunctions community. The scope would expand to cover content as well as functions. This could make it difficult for the smaller community, and need more community patrollers like those already active on Wikidata. Option 4: Objects on a new Wikipedia language edition We could launch a new Wikipedia language edition in which the main namespace is abstract content. This could be called e.g. the multi-lingual Wikipedia (mul.wikipedia.org) language edition, or the abstract Wikipedia language edition (abstract.wikipedia.org). Like all other language editions, the pages are connected to the items via sitelinks in Wikidata. If Wikivoyage wanted to use the same approach, they would need to copy the setup and create a multi-lingual Wikivoyage edition (or, as Option 4B, perhaps these could be a different namespace on a single shared ‘abstract’ content wiki?). This would give a very clear distinction of what is Wikipedia content and what is not, and give the abstract content a distinct visibility, which would otherwise be somewhat hidden between Wikifunctions and Wikidata. On the other hand, it would splinter the communities further, and mix in a "new" concept of a wiki that isn't really a Wikipedia but is labelled as such. Option 5: Unattached namespace on existing project We could introduce a new namespace to an existing project where the abstract content for Wikipedia would effectively live. Here are the most likely options: 1. Wikidata (as a separate namespace, not attached to the items) 2. Commons 3. Meta 4. English Wikipedia (not attached to the articles) Whereas technically all these options would be the same, they would be extremely different from a social and community perspective. We will refrain here from discussing these differences for now, unless this starts becoming a more likely option. Rant: One Wikimedia movement, or many projects? Conway’s law <https://en.wikipedia.org/wiki/Conway%27s_law> states that software mirrors organization, or, as it was put, if you put three teams to work on a compiler, you get a three-pass compiler. The opposite is also true, as previously observed <https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org…> by Guillaume Paumier <https://meta.wikimedia.org/wiki/User:Guillom>: a software system determines the community structure that evolves around the system. Within the Wikimedia projects, we often see this effect in the dynamics between the different Wikipedia language communities, the Wikidata community, the Commons community, Meta, *etc.* The stories of local protection of media files on Commons, of wikis opting-out from global anti-abuse tools, or of short descriptions in English Wikipedia should be warning enough. The question we ask today is not only hard because there are genuine technical challenges that we have to overcome, and we have to make a tradeoff. It is additionally so much harder because we can anticipate that there will be fracture lines between the different projects, and maybe even anticipate how these fracture lines would shape out. The whole story would be so much easier if we would, in general, regard ourselves as being part of one common movement, as one large community. But I doubt that we will see much progress on this trajectory before we have to resolve the above question. Until then we only can remain mindful about the possible solutions, their effects, and that we should be careful to design for the world as it is and as it likely will be, and not for the world we wish to be. Public NLG workstream on Tuesday On Tuesday, March 21, there will be the third public NLG workstream meeting on JITSI. Feel free to reach out to me and suggest presentations beforehand if you want. We have a bit already planned, but there is still space. The meeting is 16:30-17:30 UTC <https://zonestamp.toolforge.org/1679416242>, which is an hour off for US friends.

5 9

Newsletter #106: Decolonizing Functions
by Denny Vrandečić 17 Mar '23

17 Mar '23

An on-wiki version of this newsletter can be found here: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-03-02 ---- Decolonizing Functions In the newsletter two weeks ago <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-02-17>, we discussed the history of the idea of a compendium of functions, tracing it back to al-Khwarizmi <https://en.wikipedia.org/wiki/Muhammad_ibn_Musa_al-Khwarizmi> and his influential book on Algebra <https://en.wikipedia.org/wiki/The_Compendious_Book_on_Calculation_by_Comple…>. I noticed an uptick in reactions to that piece from people of cultural backgrounds that relate to al-Khwarizmi. And that should come as no surprise. If one identifies, even roughly, with white, Western, Christian, heterosexual, cis-gendered men, it is easy to miss how many of the narratives we encounter are tuned to that demographic. If one doesn’t identify with that demographic, having a figure or narrative that reverberates more with one's own identity or heritage can feel empowering and inspiring. It can offer an example of, “look, there’s someone like me, and look at that great thing they did!” For a few decades now, we have seen a refreshing trend of diversifying the protagonists in the stories being told in books, movies, and in newspapers in the Western world. Unfortunately, these 'novel' protagonists are often met with pushback and resistance, as if the world of stories and narratives was a limited space, as if by having more narratives that are tuned to under-represented demographics we thus reduce the narratives that are tuned to the most prominent demographic. But the cultural space is not a zero sum game; the space of narratives is infinite. Similarly, Wikipedia, by being online, does not have to think about page limitations in the way a printed encyclopedia does. Writing two paragraphs more on al-Khwarizmi in a Wikipedia version does not mean I have to write two paragraphs fewer on Pascal. I do not have to balance the space I dedicate to Ada Lovelace with the space dedicated to Charles Baggage. Writing more about the history of the Dagomba people does not mean I need to cut down on the history of Rome. Each Wikipedia has to (and does) struggle with the effect of its policies and guidelines on how we are biasing the encyclopedia towards certain narratives, but that is a different story, to be told by someone else in a different place. A few weeks ago, *Nature <https://en.wikipedia.org/wiki/Nature_(journal)>*, one of the leading science journals in the world, published two articles in tandem: one on making mathematics truly universal <https://www.nature.com/articles/d41586-023-00223-w> through the program of decolonization, and the other on why the idea of decolonizing mathematics is no cause for alarm <https://www.nature.com/articles/d41586-023-00240-9>. These articles are part of a much-needed series on decolonizing science <https://www.nature.com/collections/giaahdbacj> *Nature* is running. Decolonizing mathematics is not a novel concept nor phrase the term goes back at least a good quarter century <https://www.jstor.org/stable/41674951>, and also this newsletter wrote about it previously <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-12-09>. Predictably, the articles in *Nature* have caused alarm and have been misunderstood, pretty much in the way the articles themselves predicted. The way I understand the program of decolonizing mathematics is to follow two principles: - first, to recognize the contributions of people with diverse backgrounds, in order to offer more protagonists who can inspire and with whom more people can identify - second, to provide examples and motivations that are relatable to under-represented backgrounds and identities, in order to reach and be more immediately helpful to more people <https://meta.wikimedia.org/wiki/File:Yanghui_triangle.PNG> <https://meta.wikimedia.org/wiki/File:Yanghui_triangle.PNG> Yanghui's triangle, published 1303 The first principle relates more to Wikipedia than to Wikifunctions, and even though there is room for improvement, Wikipedia is already pretty good at reflecting a comprehensive and multi-faceted history (see, for example, the history of Pascal’s triangle <https://en.wikipedia.org/wiki/Pascal%27s_triangle#History> on English Wikipedia), especially across different language editions (compare to Yang Hui’s triangle <https://zh.wikipedia.org/wiki/%E6%9D%A8%E8%BE%89%E4%B8%89%E8%A7%92%E5%BD%A2> on Chinese Wikipedia). I hope that with Abstract Wikipedia we will see an even tighter integration of different narratives, and see their wider distribution in many languages. The second principle in particular can and should also be applied to Wikifunctions. We should make space for examples which are rooted in the individual backgrounds of under-represented users of Wikifunctions, to highlight how many different people can benefit from Wikifunctions. This was exemplified by al-Khwarizmi’s book and its focus on Muslim inheritance law, but also how relatable examples in university courses lead to much better results, as described by Jessica Nordell <https://en.wikipedia.org/wiki/Jessica_Nordell> in her book *The End of Bias: A Beginning* <https://en.wikipedia.org/wiki/Special:BookSources/978-1-250-18618-8>. I very much hope that Wikifunctions will consciously provide the space for relatable and diverse examples from many different areas. Recordings The recording of Maria Keet’s presentation on abstract representations is now available on Wikimedia Commons. Maria Keet talks about the design of the "abstract content" language for writing "constructors", which are those pieces of structured information that are positioned between Wikidata and Wikifunctions as a source on the one side of the pipeline and the machinery for rendering that content into natural language sentences or paragraphs of text on the other side in the pipeline. The recording can now be watched on Commons here: commons.wikimedia.org/wiki/File:Abstract_Wikipedia_Natural_language_generat… The regular *Conversation with Trustees* is an opportunity for community members to speak directly with the Wikimedia Foundation's Trustees about their work. The Board of Trustees is a volunteer body of movement leaders and external experts in charge of guiding the Wikimedia Foundation and ensuring its accountability. The 23 February 2023 conversation included a short update on Abstract Wikipedia and Wikifunctions, and answered some community questions that were asked. A recording of the conversation is available on YouTube for now, and will also be available on Wikimedia Commons eventually <https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Community_Affairs_Comm…> : www.youtube.com/watch?v=cGqHUrpU2Rc Volunteers’ corner on 6 March 2023 This upcoming Monday will see our monthly Volunteer’s corner. The meeting will be on Monday, 6 March 2023, at 18:30-19:00 UTC <https://zonestamp.toolforge.org/1678127424> and you can join on Jitsi on the following link: https://meet.jit.si/AWVolunteersCorner Bring your questions, your ideas, or even just your curiosity, and we will find and help with places you can contribute. Development update - The large patchsets we have been working on are landing or close to landing - *Goal 2* (efficient and correct evaluation) has seen the patch land that splits the one big evaluator into individual language evaluators. We are working now on propagating these changes to the beta. - *Goal 3* (meta-data) has seen the patch land that reorders implementations. We are now working on enabling that on the beta. - *Goal 5* (meta-data) the work on typed lists is ongoing, and work on function calls has been picked up and a first version has landed - *Goal 6* (stable and secure system) has seen the rights system land, and now requires deployment and testing - *QTE* has presented the work on e2e testing that will lead to integration testing become part of CI - *Design* has reached a state where we have caught up with the state of the implementation, and are have prepared most of *Goal 9* and have starting now to prepare for documentation in the next phase

3 2

Newsletter #105: Thank you, Adesoji! Welcome Allan! A new position for an engineer, and videos and another meeting opportunity
by Denny Vrandečić 23 Feb '23

23 Feb '23

The on-wiki version of this newsletter can be found here: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-02-23 ----Thank you, Adesoji! Adesoji Temitope <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-10-08> joined the team in October 2021 as a consultant through This Dot Labs, lending his experience in Vue.js and front-end development to the team. As Adesoji concludes his work with This Dot, we see Adesoji thus wrapping up his work on Wikifunctions. We are sad to see him go! Here are Adesoji’s own words: It's been an incredible journey to work with some of the best of the minds I have known in my professional career. I got to work on a very interesting and unique project. Thanks for the opportunity, Sadly, I am leaving with a warm heart. Thank you, everyone, and goodbye. Adesoji has been working on the Wikifunctions front-end, developing the interface for functions which will allow the contributors and users of Wikifunctions to interact with them. Thank you Adesoji for your work, and we wish you best of luck in your future endeavors! Welcome, Allan Jeremy! This Dot is on-boarding another engineer to support us in the development of Wikifunctions, Allan Jeremy. Here are Allan’s own words: I am a self-taught software engineer from Nairobi, Kenya. I started learning how to code at 14, owing to a desire to create video games like the ones I loved playing when I was younger. When I am not coding, I am either exploring clear beaches, recording videos, or nerding out about AI. I am also interested in history, philosophy and psychology. Welcome Allan to the team! We are looking forward to working with you. A new engineering role is open As Julia Kieserman <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-11-18> transfers to a different role within the Foundation during the next couple months, we are hiring for a new full stack software engineer, with a need to ensure continued delivery on the Vue.js front-end. The role is posted here: Senior Software Engineer, Abstract Wikipedia <https://boards.greenhouse.io/wikimedia/jobs/4868029?gh_src=4e048f911us> Please consider whether you or someone you know might be the right person for this opportunity to work on Abstract Wikipedia and Wikifunctions in an engineering capacity. Conversation with Trustees Abstract Wikipedia and Wikifunctions will join this round of the *Conversation with Trustees**,* organized by the Community Affairs Committee of the Board of Trustees of the Wikimedia Foundation. You can join the meeting on February 23, 2023, at 19:00 UTC <https://zonestamp.toolforge.org/1677178854> as per the following page. A recording will be made available. - Conversation with Trustees <https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Community_Affairs_Comm…> Recordings from this week’s presentations As announced last week, we had two recorded meetings. Stef Dunlap is talking about our end to end testing environment. The presentation and recording is available here, as well as a transcript: - people.wikimedia.org/~kindrobot/duct/ Maria Keet is talking about the design choices for abstract representations we had so far. We are trying to upload the video on Commons as well, and will let you know when it is there, but for those who cannot wait, here is a preliminary place where you can access the video for now: - Temporary download location on Google Drive <https://drive.google.com/file/d/1br0ldYvMz1dUzTZr0c-pWSBbJmDRWd_D/view?usp=…> Development update (February 17, 2023) Everyone’s working on large patchsets, which will hopefully land soon. - *Goal 2* (efficient and correct evaluation) is seeing the split up of the evaluator in separate evaluators per language - *Goal 3* (meta-data) is working on the reordering of implementations - *Goal 5* (default view) is working on typed lists - *Goal 6* (stable and secure system) is seeing an effective system for rights checking being implemented

1 0

Newsletter #104: Wikifunctions: a Compendium of Calculations; DUCT and Abstract Representation presentations
by Denny Vrandečić 17 Feb '23

17 Feb '23

The on-wiki version of this week's newsletter is available here: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-02-17 ---- Wikifunctions: a Compendium of Calculations <https://meta.wikimedia.org/wiki/File:1983_CPA_5426_(1).png> <https://meta.wikimedia.org/wiki/File:1983_CPA_5426_(1).png> A stamp issued 6 September 1983 in the Soviet Union, commemorating al-Khwārizmī's (approximate) 1200th birthday. Twelve hundred years ago, the polymath Muhammad ibn Musa al-Khwarizmi <https://en.wikipedia.org/wiki/Muhammad_ibn_Musa_al-Khwarizmi> wrote a book, *The Compendious Book on Calculation by Completion and Balancing* <https://en.wikipedia.org/wiki/The_Compendious_Book_on_Calculation_by_Comple…>. He was born in Khwarazm <https://en.wikipedia.org/wiki/Khwarazm> in what is today Uzbekistan and wrote the book in Baghdad <https://en.wikipedia.org/wiki/Baghdad> in what is today Iraq. The goal of the book was to help people find answers to a certain form of questions: for example, how to calculate inheritances (*“a man leaves behind two sons and a daughter”*), the area of a yard (*“assume the yard has the shape of a rhombus”*), prices for mercantile transactions (*“ten for eight, so how much for four?”*), and many other such practical questions for everyday life. Al-Khwarizmi drew on knowledge and practices from Indian numbers, Babylonian astronomy, and Greek mathematics (which had largely been lost in Greece and the West by that time). The book remained a major textbook on the topic for more than half a millennium, widely influencing mathematics and its notation, and answered questions for many people, whether it was on dividing their legacies, figuring out the size of their land, or buying and selling merchandise. The book was in fact so influential that it gave us two words that are in widespread use today, with cognates in many languages: the word *algebra*, which is derived from the the Arabic word جبر <https://en.wiktionary.org/wiki/%D8%AC%D8%A8%D8%B1#Arabic> (*al-jabr*), comes from the title of the book, and *algorithm* is derived from al-Khwarizmi, the name of the author. Even today, al-Khwarizmi's image graces the cover of some math textbooks in Ecuador, and the books seem to contain excerpts of his Compendious Book, too! <https://meta.wikimedia.org/wiki/File:Image-Al-Kit%C4%81b_al-mu%E1%B8%ABta%E…> <https://meta.wikimedia.org/wiki/File:Image-Al-Kit%C4%81b_al-mu%E1%B8%ABta%E…> A page from al-Khwārizmī's *Algebra* The importance and relevance of that book indicates one of the legacies Wikifunctions can aspire to: a compendium of methods on how to answer questions, including questions like the ones above, and many others too. Just as Wikipedia follows the spirit of Western enlightenment <https://en.wikipedia.org/wiki/Age_of_Enlightenment> embodied in Denis Diderot’s <https://en.wikipedia.org/wiki/Denis_Diderot> and Jean le Rond d'Alembert’s <https://en.wikipedia.org/wiki/Jean_le_Rond_d%27Alembert> *Encyclopédie, or a Systematic Dictionary of the Sciences, Arts, and Crafts* <https://en.wikipedia.org/wiki/Encyclop%C3%A9die>, Wikifunctions follows the *Compendium* as an achievement of the Islamic Golden Age <https://en.wikipedia.org/wiki/Islamic_Golden_Age>, also sometimes called the Islamic Enlightenment. It will be possible for functions in Wikifunctions to look like pure mathematics (*“what is 25% of 1.5?”*) or to look like questions that we might encounter in everyday life (*“given a land parcel of 1.5 acres, how large would be one part out of four equal parts?”*). It is known that translating from the latter to the former is not always trivial: a cousin of mine was struggling with fractions as a child. But once I rephrased the same questions to be about money instead of pure numbers, she answered each of the questions, and more, without any hesitation or error. Both types of function can live in Wikifunctions side by side, and the everyday functions can use the pure mathematical ones in their implementations. Functions can form layers on top of other functions, and functions useful to calculate the amount of ingredients for recipes can build on top of functions for multiplication and unit conversion. In Wikifunctions, functions will be accompanied by documentation explaining them. This documentation can guide users, and help them find the right function to answer their questions. How a function is named, how its arguments are named, the documentation surrounding functions – all of these can be major factors in making Wikifunctions more accessible and find a wider audience. This puts a responsibility on the community to select accessible and understandable names for functions and arguments, and to offer documentation that makes a function relevant to the users of Wikifunctions. However, just as many of the examples in al-Khwarizmi’s book had a limited relevance to many people in many places in Europe, as they didn’t follow Islamic rules of inheritance <https://en.wikipedia.org/wiki/Islamic_inheritance_jurisprudence>, we have to be careful to draw on examples and naming sources beyond those rooted in the Western world. Even though the underlying mathematics are universal, the examples and use cases of the calculations often are not. I hope that we as a community will be mindful of our responsibility. You can find a scan of an English copy of al-Khwarizmi’s compendium on Archive.org: archive.org/details/algebraofmohamme00khuwrich/ Interestingly, the translation also contains a translation into mathematical notation, which for some modern readers might be easier to read than the actual English translation. This would have been entirely inconceivable in al-Khwarizmi’s time. By the way, my cousin? She has grown up, and received a degree in economics a few years ago. DUCT: Doing Unique Ci Things Stef Dunlap, a software engineer in Test Engineering embedded in the Abstract Wikipedia team, is working on running end-to-end tests on per-patch ephemeral test environments. The tests are orchestrated using GitLab CI (Continuous Integration) on Wikimedia's Gitlab <https://www.mediawiki.org/wiki/GitLab> instance. The ephemeral test environments are run on a small kubernetes cluster on Cloud VPS <https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS>. A small bridging layer written in Rust running on Toolforge keeps these environments in sync with Gerrit patches. This novel approach, with the working title DUCT (Doing Unique Ci Things) aspires to establish a pattern that other Wikimedia projects with auxiliary services can use to run reliable end-to-end tests and manually test features in single-tenant test environments (as opposed to the multi-tenant Beta Cluster) while leveraging existing Wikimedia Foundation tools and adopting new technologies that are being invested across the Wikimedia movement. Stef will present her work next Tuesday, February 21, 2023, at 17:00 UTC <https://zonestamp.toolforge.org/1676998828>. The talk will be recorded and made available afterwards. Interested parties are invited to join on Google Meet, and there will be some opportunity for questions. The link to the talk is: meet.google.com/tpn-fafq-ekj Public NLG meeting on February 21, 2023 Next Tuesday, February 21, 2023, at 16:30 UTC <https://zonestamp.toolforge.org/1676997047>, we will have our second public NLG (Natural Language Generation) workstream meeting. The meeting will be on Jitsi and can be joined here: meet.jit.si/AWVolunteersCorner This meeting will feature a presentation by Maria Keet about the design of the *"abstract content"* language for writing *"constructors"*, which are those pieces of structured information that are positioned between Wikidata and Wikifunctions as a source on the one side of the pipeline and the machinery for rendering that content into natural language sentences or paragraphs of text on the other side in the pipeline. We hope to take a step forward from its discussion document <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Wikidata_Abstract_Repres…> of November 2022 towards more and more concrete, required features for the data structure specification. Apologies for the two events overlapping. Development updates (as of February 10) - In order to leave *Goal 2* (efficient and correct evaluation) in a good state, work has started on replacing the current evaluator, which covers all native programming languages we support, with individual evaluators for each programming language. This will allow for easier introduction of more programming languages in the future, and, more importantly, to switch support for individual programming languages on and off in case a need arises, e.g. to buy time in response to critical vulnerabilities in a specific language runtime. This is a requirement from the security review. - In *Goal 6* (stable and secure system) many user rights and user groups were defined and created. These are needed to define specific actions that only certain users should be allowed to do, which are unique to the Wikifunctions environment. That includes creating new types, activating and deactivating implementations and testers, etc. - In *Goal 5* (default view), work on the list component is ongoing, which is one of the two major components remaining in this goal. At the same time, designs for Goal 5 and for *Goal 9* (updating the other components) have progressed or even finalized (including the selection of icons for different modes, and a beautiful design for implementations). - For *Goal 3* (meta-data), the last major patch for reordering implementations has been sent to review, bringing us close to closing this goal. This will allow us to choose efficient, instead of the current approach of random, implementations to run for a given function, which will make the runtime of the system more predictable and use fewer resources.

1 0

Newsletter #103: More thoughts on the fellows’ evaluation; Name for improvement process
by Denny Vrandečić 09 Feb '23

09 Feb '23

The on-wiki version of this newsletter can be found here: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2023-02-08 ---- More thoughts on the Evaluation by the Google.org fellows During the fellowship, the Google.org fellows gained detailed insight into the Wikifunctions and Abstract Wikipedia project. With the goal to point out potential issues and to discuss potential alternatives to some of the project’s approaches, they wrote a detailed evaluation of the Wikifunctions and Abstract Wikipedia projects. The team read through the evaluation and wrote a detailed answer. Back in December, we published <https://meta.wikimedia.org/wiki/Special:MyLanguage/Abstract_Wikipedia/Updat…> the full documents: - Wikifunctions and Abstract Wikipedia: An Evaluation <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evalu…> - Answer to Wikifunctions and Abstract Wikipedia: An Evaluation <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Google.org_Fellows_evalu…> The Signpost <https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2023-01-01/Techn…> reported on the evaluation, and an episode of the podcast Between the Brackets <https://betweenthebrackets.libsyn.com/episode-130-ariel-gutman> discussed it, among other things. Today, we want to provide a condensed and not too technical summary of the documents. The documents were both long and technical. We understand that a two page summary of a forty page discussion will be necessarily simplifying and glance over a lot of nuance, but for everyone who wants to dive into the details, the full documents are available. The evaluation describes many proposals and suggestions to change the project, and culminates in eight sweeping recommendations. We have accepted a large number of the suggestions the evaluation made. Some have already been implemented, others are tasks on our project task board and are awaiting implementation. The main area of disagreement lies in the final eight recommendations. In summary, the recommendations suggest that the Foundation should treat Wikifunctions and Abstract Wikipedia as separate projects. Further, they recommend dramatically narrowing the scope of both projects For Wikifunctions, the evaluation recommends dropping the function model, on which our multilingual user interface rests. They further recommend cutting features that enable non-developers to contribute functions, and focusing on a single programming language, Lua, instead of allowing for several languages. For Abstract Wikipedia, the evaluation recommends dropping the idea of co-creating one or more natural language generation solutions with the community, suggesting to select either an existing natural language generation system, or to adopt the one that one of the fellows initiated during the fellowship. Instead of creating an all-purpose function repository that would allow the community to experiment and develop a solution, the recommendation is for the project team to decide on and hard-code a single solution to be a system external to the wiki integrating the contributions of the editors. This is similar to early critiques of Wikidata, which faced a very similar suggestion: numerous voices declared that the decision to allow the community to create and maintain the ontology of Wikidata, instead of either choosing an existing ontology or hiring a team of ontologists, would doom Wikidata. We lost some major funding because of sticking with that decision. The development team of Abstract Wikipedia and Wikifunctions rejected both sets of recommendations. One simple reason for some of the rejections was that the evaluation indeed evaluated the technical plan, and not the current state of development. For example, one recommendation is to focus solely on Lua instead of allowing different programming languages, as the latter would take a long time to implement. But given that we have already implemented that feature, adopting the recommendation would slow us down and have us discard work that had already been done. Some of the rejections for the recommendations were based on Wikimedia values. We think it crucial to have a project that people can contribute to and use in their own natural language. We think it important to enable non-programmers to contribute, and not just for the sake of diversity but also simply because the pool of potential contributors with the relevant skill set is very limited in many language communities. We need people to create and maintain natural language generation functions in many languages. Allowing for a wider skill set to contribute reduces this potential bottleneck. This is why Abstract Wikipedia needs Wikifunctions, and at the same time Wikifunctions benefits from Abstract Wikipedia as a use case. It is a symbiotic relation. There is currently no symbolic natural language generation system that covers 300 languages, as the Wikimedia movement already does. We can not confidently choose an existing solution, nor the novel one developed during the fellowship. In fact, Prof. Maria Keet of the University of Cape Town, South Africa, joined the Abstract Wikipedia development as a volunteer. She criticized both the existing popular natural language generation systems as well as the one created during the fellowship, because they wouldn’t work well with Niger Congo B languages. The fellow’s proposal was updated to better support the needs of such languages. We were able to do this because our architecture is not tightly coupled to an external natural language generation system. We do not, and mostly cannot know whether the languages that have not yet been implemented have features that would become a major problem for a given system. In a general purpose, flexible system, the community will have the power to fix any such issue. By selecting any single solution, we would considerably increase the barrier to fixing problems. Instead of contributing to a wiki, we would need developers contributing elsewhere, e.g. in the case of SimpleNLG, to an upstream Java project through git, or in the case of Grammatical Framework, to a project written in Haskell. And worst of all, the communities that will most likely be adversely affected by such a decision would be the ones that are already under-represented. We are not against using an existing natural language generation system. We will and already are supporting the community in leveraging existing solutions (we are closely working with Grammatical Framework, which is strongly recommended by the fellows). But at this point we will not commit exclusively to any one of the existing solutions. Instead the plan was, from the beginning, to allow the community to build, maintain, and own their approaches, to enable the community to use and combine available solutions, and to co-create Abstract Wikipedia with the community together. We hope this summary of the eight recommendations and our decision helps in understanding what we are building and why we made the decision. What’s a good name for a process to improve? Python has the PEP <https://peps.python.org/> (Python Enhancement Proposal), Java the JSR <https://jcp.org/en/jsr/overview> (Java Specification Request), and many wikis have the RfC <https://www.wikidata.org/wiki/Q4654740> (Request for Comment, inspired by IETF) or similar terms. Wikifunctions will be improving well after its launch, and we will need a process to discuss these improvements, either on wiki, on the mailing list, or in Phabricator. This process needs a name! We won’t make a big vote or anything, but we would like to ask for ideas for the name, and whether you have preferences or concerns. Some internal suggestions have been - Community System Change (CSC) - Little Lambda chat (LLC) - Food for thought (FFT) - Wikifunctions Lambda Delta (WILD) - Wikifunctions System Consultation (WSC) - Functional Model Proposal (FMP) Also if you have suggestions on how to run and structure the process of changes to the formal Wikifunctions model initially, let us know. We expect that this will change considerably over time, but there is likely plenty of experience we can work from. Weekly development update for February 3, 2023 Last week we had our two-monthly fix-it week, which is a meeting-light week focused on fixing bugs and paying down technical debt. Some examples of the work that was done include: - Fixing small bugs, e.g. fixWidth issues <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/886004> or removing unnecessary checks <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/886048> - Improving test coverage, e.g. in serializer <https://gerrit.wikimedia.org/r/c/mediawiki/services/function-schemata/+/885…> or on the string component <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/760999>, or re-enable temporarily disabled tests, e.g. for mapping empty lists <https://gerrit.wikimedia.org/r/c/mediawiki/services/function-orchestrator/+…> - Renaming components and functions to make them clearer and more consistent, e.g. rename ‘generic error’ to ‘unspecified error’ <https://gerrit.wikimedia.org/r/c/mediawiki/services/function-schemata/+/885…> or add a prefix to components <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/884990> - Refactoring, e.g. restructure whole directories <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/885393>, consolidate metadata dialogs into a single component <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/885846> or generalize existing functions <https://gerrit.wikimedia.org/r/c/mediawiki/services/function-schemata/+/885…> Replacing components with more modern components, e.g. our own 'chip' with the one from Codex <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/886143>

2 2

Reminder: Next Wikifunctions & Abstract Wikipedia Developers-Volunteers' Corner
by Luca Martinelli [Sannita＠WMF] 02 Feb '23

02 Feb '23

Hello everyone! This is a friendly reminder that next Wikifunctions & Abstract Wikipedia Developers-Volunteers' Corner will be held *Monday 6 February, at 18:30 UTC* (https://zonestamp.toolforge.org/1675708253) The link to the meeting is the same as last time: https://meet.jit.si/AWVolunteersCorner If you have any questions or concerns, or if want to show your progress on something you're working on, or if you just want to hang around and get to know the project better, you're more than welcome to join us! Hope to see you there! -- Luca Martinelli [Sannita] (he/him) Community Relations Specialist

1 0

2024

2023

2022

2021

2020

Abstract-Wikipedia