Great News Denny!
The team should be very proud!
1 question that has been in the back of my mind has been that around
using full capability of available hardware for implementations.
In particular, SSE4.2 String Compare intrinsic functions. Intel®
Intrinsics Guide
<https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#techs=SSE4_2&cats=String%20Compare>
Do you know if the servers that will provide compute operations offer
SSE4.2 String Compare functions to allow implementations to take advantage
of faster Vector processing of Strings?
Java has been getting lots of love for this in JEP 417: Vector API
(Third Incubator) (
) <https://openjdk.java.net/jeps/417>
But in particular for Abstract Wikipedia a considerable performance
improvement will be with usage of the SSE4.2 String Compare functions.
Thad
On Fri, Apr 8, 2022 at 5:38 PM Denny Vrandečić <dvrandecic(a)wikimedia.org>
wrote:
The on-wiki version is available here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-04-08
--
When we started the development effort towards the Wikifunctions site,
we sub-divided the work leading up to the launch of Wikifunctions into eleven
phases
<https://meta.wikimedia.org/wiki/Special:MyLanguage/Abstract_Wikipedia/Phases>,
named after the first eleven letters of the Greek alphabet.
- With Phase α (alpha) completed, it became possible to create
instances of the system-provided Types in the wiki.
- With Phase β (beta), it became possible to create new Types
on-wiki and to create instances of these Types.
- With Phase γ (gamma), all the main Types of the pre-generic
function model were available.
- With Phase δ (delta), it became possible to evaluate built-in
implementations.
- With Phase ε (epsilon), it became possible to evaluate
contributor-written implementations in any of our supported programming
languages.
- With Phase ζ (zeta), it became possible to evaluate
implementations composed of other functions.
- This week we declared Phase η (eta) as complete, in which we
planned to add support for generic types and functions.
We did that, but we also did so much more:
- We re-designed and re-built the error system entirely
- We moved from the preliminary function model
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Pre-generic_function_model>
to
the full function model
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Function_model>
- We implemented validators running on user-implemented types
- We designed the viewing and editing of functions
<https://design.wikimedia.org/blog/2022/03/14/wikifunctions.html>
- We moved the frontend to use Codex
<https://github.com/wikimedia/design-codex>, Wikimedia’s new
front-end toolkit
- We announced the logo
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-10-27> for
Wikifunctions
- We decided with the community on the licensing of Wikifunctions
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-12-21>
- We on-boarded quite a number of people since the start of the
phase: Cai
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-10-08>
, Adesoji
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-10-08>
, David
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-11-04>
, Julia
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-11-18>
, Mariya
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-12-16>,
and Luca
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-01-21>
We have learned a lot during this phase. Most notably, it took much
longer than anticipated: it took seven full months for this phase, whereas
previous phases took around two months. We held a retrospective on this,
and we identified a number of issues that we are aiming to considerably
improve. The scope creep, as witnessed by the number of things we have
accomplished, is one such issue. A lesson I certainly learned is the
real-world complexity of generic type processing, which presumably is why
so many programming languages only added support for generics later, and
did not have it from the start. And, particularly towards the end of the
phase, we were noticing an unsustainable and exhaustive working mode. We
will change our approach in the upcoming phase, by focusing on smaller,
more self-contained workstreams, and focus on these one by one.
Today, we have kicked off Phase θ. Originally called “thawing and
freezing”, the theme of this phase is to allow for the community and
technical processes on-wiki that allow the community to collaboratively
work on a library of functions in a stable and secure manner. This includes
deciding on and implementing relevant user-rights, features for
understanding edits done by others, having the system choose the right
implementation, and much more. The description in the phases page on
Meta <https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases> still
needs to be updated, but here are the workstreams that we will work on in
this phase:
- Decide and implement canonical form for typed lists
<https://meta.wikimedia.org/wiki/Special:MyLanguage/Abstract_Wikipedia/Updates/2022-04-01>
- Give users an intuitive user experience for functions by
implementing the designs for viewing and editing functions
- Allow the system to run function evaluations correctly and
efficiently by deciding which implementations to select
Provide users with the meta-data about individual function runs on the
wiki (e.g. how long it took, how many resources were used, etc.)
- Allow the system to run and evolve in a stable and secure manner,
by deciding on and implementing user rights, rate limiting and caches, and
by displaying edit diffs between revisions
- Ensure users have a comprehensible view experience for
non-function objects, as we redesign and reimplement texts with language
fallbacks, references, strings, and lists
- Help to start some related early community discussions about
topics such as user-group-rights, code of conduct, staff collaboration on
functions, and other new-wiki-preparation efforts
- Get ready for future phases, by designing for multilingual
documentation of objects and instrumenting the frontend
- Participate in the 2022 Wikimedia Hackathon
<https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022>
You can see that even within the concept of stability and security, we
have a lot of things planned for this phase, but each of the workstreams
are much more self-contained than the big and somewhat open-ended goals of
the previous phase. This should also allow for more visibility into our
progress, and we will keep you updated in the weeklies on progress.
Once this phase is complete, we are getting very close to the finish
line: in Phase ι (iota) we plan to add multilingual documentation of
objects. Then, we have set Phase κ (kappa) for last minute clean up tasks,
before launching in Phase λ (lambda).
A huge thank you to the whole team, this is a major milestone. I am
proud of all the things achieved, and I am very excited to see the work in
the upcoming phase, which will be crucial to allow for Wikifunctions to not
just be a platform for running functions, but to allow it to grow as a
community.
_______________________________________________
Abstract-Wikipedia mailing list --
abstract-wikipedia(a)lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/abstract-wikipedia.lists.wikime…
_______________________________________________
Abstract-Wikipedia mailing list -- abstract-wikipedia(a)lists.wikimedia.org
List information: