With respect to the early discussion topic [1] of the “data and evaluation model for the code repository (Wikilambda)”, I would like to broach, for discussion, that the V8 engine [9] and/or WebAssembly [10] can be of use for the Wikilambda project.
While reading the plan [2] and tasks [3] documents, I observed tasks: P1.12 [4], P1.15 [5], O6 [6], O16 [7] and O29 [8]. Task P1.12 [4] indicates JavaScript-based implementations. Task P1.15 [5] indicates Lua implementations. Lua can be compiled to WebAssembly [12]. Task O6 [6] indicates Python implementations. Python can be compiled to WebAssembly [13]. There is a lengthy list of programming languages which already can or are planned to soon be able to compile to WebAssembly [11]. Task O29 [8] indicates compilation for significant speedup.
The use of WebAssembly would facilitate task O16 [7], browser-based evaluation scenarios. WebAssembly is broadly supported by Web browsers.
The V8 engine is utilized by Chromium-based Web browsers such as Brave, Chrome, Edge, Opera, and Vivaldi [9]. “The V8 engine can be used in a browser or integrated into independent projects” [9]. The V8 engine is utilized by the popular Node.js runtime environment [9]. The V8 engine includes a WebAssembly implementation.
With respect to processing JavaScript, the V8 engine “first generates an abstract syntax tree with its own parser. Then, Ignition, the V8 interpreter, generates bytecode from this syntax tree using the internal V8 bytecode format [15]. TurboFan is the V8 optimizing compiler, it takes this bytecode and generates machine code from it. In other words, V8 compiles JavaScript directly to native machine code using just-in-time compilation before executing it. The compiled code is additionally optimized (and re-optimized) dynamically at runtime, based on heuristics of the code's execution profile. Optimization techniques used include inlining, elision of expensive runtime properties, and inline caching” [9]. The V8 engine “can compile to x86, ARM or MIPS instruction set architectures in both their 32- and 64-bit editions; as well, it has been ported to PowerPC and IBM s390 for use in servers” [9].
The V8 engine’s Ignition virtual machine [14] is “a register machine with an accumulator register” [15].
The WebAssembly virtual machine is “a portable virtual stack machine” [10].
In what ways can the V8 engine and/or WebAssembly be of use for the Wikilambda project? Could the V8 engine’s Ignition virtual machine [14][15] and/or the WebAssembly virtual machine [10] be of use as evaluation models or compilation targets for Wikilambda?
Best regards, Adam Sobieski
[1] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Early_discussion_topics [2] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan [3] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks [4] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.12:_JavaScr... [5] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.15:_Lua-bas... [6] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O6:_Python-base... [7] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O16:_Browser-ba... [8] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_O29:_Implement... [9] https://en.wikipedia.org/wiki/V8_(JavaScript_engine) [10] https://en.wikipedia.org/wiki/WebAssembly [11] https://github.com/appcypher/awesome-wasm-langs [12] https://github.com/appcypher/awesome-wasm-langs#lua [13] https://github.com/appcypher/awesome-wasm-langs#python [14] https://v8.dev/docs/ignition [15] https://medium.com/dailyjs/understanding-v8s-bytecode-317d46c94775
Hi Adam,
First time poster to the list, and I'm one of the core team members.
We'll definitely be looking into a Node.js server runtime - we do run Node.js services in production, and I think what you're getting at is in addition to simply running programs there may be some opportunities for optimization in a build (and runtime) system, is that right?
For the client side of things, there's a fair amount of R&D to be done, although compilation for WebAssembly, particularly for portability of high performance code and for countenancing sandboxed execution for non-JS languages, is intriguing. We'll need to take WebKit into consideration of course, where there is also WebAssembly support.
Thanks for raising this! -Adam
On Sat, Jul 11, 2020 at 3:39 AM Adam Sobieski adamsobieski@hotmail.com wrote:
With respect to the early discussion topic [1] of the “data and evaluation model for the code repository (Wikilambda)”, I would like to broach, for discussion, that the V8 engine [9] and/or WebAssembly [10] can be of use for the Wikilambda project.
While reading the plan [2] and tasks [3] documents, I observed tasks: P1.12 [4], P1.15 [5], O6 [6], O16 [7] and O29 [8]. Task P1.12 [4] indicates JavaScript-based implementations. Task P1.15 [5] indicates Lua implementations. Lua can be compiled to WebAssembly [12]. Task O6 [6] indicates Python implementations. Python can be compiled to WebAssembly [13]. There is a lengthy list of programming languages which already can or are planned to soon be able to compile to WebAssembly [11]. Task O29 [8] indicates compilation for significant speedup.
The use of WebAssembly would facilitate task O16 [7], browser-based evaluation scenarios. WebAssembly is broadly supported by Web browsers.
The V8 engine is utilized by Chromium-based Web browsers such as Brave, Chrome, Edge, Opera, and Vivaldi [9]. “The V8 engine can be used in a browser or integrated into independent projects” [9]. The V8 engine is utilized by the popular Node.js runtime environment [9]. The V8 engine includes a WebAssembly implementation.
With respect to processing JavaScript, the V8 engine “first generates an abstract syntax tree with its own parser. Then, Ignition, the V8 interpreter, generates bytecode from this syntax tree using the internal V8 bytecode format [15]. TurboFan is the V8 optimizing compiler, it takes this bytecode and generates machine code from it. In other words, V8 compiles JavaScript directly to native machine code using just-in-time compilation before executing it. The compiled code is additionally optimized (and re-optimized) dynamically at runtime, based on heuristics of the code's execution profile. Optimization techniques used include inlining, elision of expensive runtime properties, and inline caching” [9]. The V8 engine “can compile to x86, ARM or MIPS instruction set architectures in both their 32- and 64-bit editions; as well, it has been ported to PowerPC and IBM s390 for use in servers” [9].
The V8 engine’s Ignition virtual machine [14] is “a register machine with an accumulator register” [15].
The WebAssembly virtual machine is “a portable virtual stack machine” [10].
In what ways can the V8 engine and/or WebAssembly be of use for the Wikilambda project? Could the V8 engine’s Ignition virtual machine [14][15] and/or the WebAssembly virtual machine [10] be of use as evaluation models or compilation targets for Wikilambda?
Best regards,
Adam Sobieski
[1] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Early_discussion_topics
[2] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan
[3] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks
[4] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.12:_JavaScr...
[5] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.15:_Lua-bas...
[6] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O6:_Python-base...
[7] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O16:_Browser-ba...
[8] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_O29:_Implement...
[9] https://en.wikipedia.org/wiki/V8_(JavaScript_engine)
[10] https://en.wikipedia.org/wiki/WebAssembly
[11] https://github.com/appcypher/awesome-wasm-langs
[12] https://github.com/appcypher/awesome-wasm-langs#lua
[13] https://github.com/appcypher/awesome-wasm-langs#python
[14] https://v8.dev/docs/ignition
[15] https://medium.com/dailyjs/understanding-v8s-bytecode-317d46c94775
Abstract-Wikipedia mailing list Abstract-Wikipedia@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/abstract-wikipedia
Adam,
Excellent. Thank you for the information. Yes, I am also thinking about optimizing performance in build and runtime systems.
Best regards, Adam
From: Adam Basomailto:abaso@wikimedia.org Sent: Monday, July 13, 2020 7:22 AM To: General public mailing list for the discussion of Abstract Wikipedia (aka Wikilambda)mailto:abstract-wikipedia@lists.wikimedia.org Subject: Re: [Abstract-wikipedia] How Can the V8 Engine and/or WebAssembly Be of Use for the Wikilambda Project?
Hi Adam,
First time poster to the list, and I'm one of the core team members.
We'll definitely be looking into a Node.js server runtime - we do run Node.js services in production, and I think what you're getting at is in addition to simply running programs there may be some opportunities for optimization in a build (and runtime) system, is that right?
For the client side of things, there's a fair amount of R&D to be done, although compilation for WebAssembly, particularly for portability of high performance code and for countenancing sandboxed execution for non-JS languages, is intriguing. We'll need to take WebKit into consideration of course, where there is also WebAssembly support.
Thanks for raising this! -Adam
On Sat, Jul 11, 2020 at 3:39 AM Adam Sobieski <adamsobieski@hotmail.commailto:adamsobieski@hotmail.com> wrote: With respect to the early discussion topic [1] of the “data and evaluation model for the code repository (Wikilambda)”, I would like to broach, for discussion, that the V8 engine [9] and/or WebAssembly [10] can be of use for the Wikilambda project.
While reading the plan [2] and tasks [3] documents, I observed tasks: P1.12 [4], P1.15 [5], O6 [6], O16 [7] and O29 [8]. Task P1.12 [4] indicates JavaScript-based implementations. Task P1.15 [5] indicates Lua implementations. Lua can be compiled to WebAssembly [12]. Task O6 [6] indicates Python implementations. Python can be compiled to WebAssembly [13]. There is a lengthy list of programming languages which already can or are planned to soon be able to compile to WebAssembly [11]. Task O29 [8] indicates compilation for significant speedup.
The use of WebAssembly would facilitate task O16 [7], browser-based evaluation scenarios. WebAssembly is broadly supported by Web browsers.
The V8 engine is utilized by Chromium-based Web browsers such as Brave, Chrome, Edge, Opera, and Vivaldi [9]. “The V8 engine can be used in a browser or integrated into independent projects” [9]. The V8 engine is utilized by the popular Node.js runtime environment [9]. The V8 engine includes a WebAssembly implementation.
With respect to processing JavaScript, the V8 engine “first generates an abstract syntax tree with its own parser. Then, Ignition, the V8 interpreter, generates bytecode from this syntax tree using the internal V8 bytecode format [15]. TurboFan is the V8 optimizing compiler, it takes this bytecode and generates machine code from it. In other words, V8 compiles JavaScript directly to native machine code using just-in-time compilation before executing it. The compiled code is additionally optimized (and re-optimized) dynamically at runtime, based on heuristics of the code's execution profile. Optimization techniques used include inlining, elision of expensive runtime properties, and inline caching” [9]. The V8 engine “can compile to x86, ARM or MIPS instruction set architectures in both their 32- and 64-bit editions; as well, it has been ported to PowerPC and IBM s390 for use in servers” [9].
The V8 engine’s Ignition virtual machine [14] is “a register machine with an accumulator register” [15].
The WebAssembly virtual machine is “a portable virtual stack machine” [10].
In what ways can the V8 engine and/or WebAssembly be of use for the Wikilambda project? Could the V8 engine’s Ignition virtual machine [14][15] and/or the WebAssembly virtual machine [10] be of use as evaluation models or compilation targets for Wikilambda?
Best regards, Adam Sobieski
[1] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Early_discussion_topics [2] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan [3] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks [4] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.12:_JavaScr... [5] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.15:_Lua-bas... [6] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O6:_Python-base... [7] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O16:_Browser-ba... [8] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_O29:_Implement... [9] https://en.wikipedia.org/wiki/V8_(JavaScript_engine) [10] https://en.wikipedia.org/wiki/WebAssembly [11] https://github.com/appcypher/awesome-wasm-langs [12] https://github.com/appcypher/awesome-wasm-langs#lua [13] https://github.com/appcypher/awesome-wasm-langs#python [14] https://v8.dev/docs/ignition [15] https://medium.com/dailyjs/understanding-v8s-bytecode-317d46c94775
_______________________________________________ Abstract-Wikipedia mailing list Abstract-Wikipedia@lists.wikimedia.orgmailto:Abstract-Wikipedia@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/abstract-wikipedia
We'll definitely be looking at both JavaScript and WebAssembly runtimes, though the devil is always in the details!
The major web engines including V8 support both JavaScript and WebAssembly and good interoperation between them. This is potentially very useful, and gives an additional sandboxing layer on top of the Lua or Python runtimes if those end up being used for user-authored tasks.
This also means we can straightforwardly run evaluator systems browser-side and in native mobile apps, both of which can embed JS and Wasm using system libraries (V8 on Chrome and Android, WebKit's JSC for Safari, iOS and iPadOS, and SpiderMonkey for Firefox).
So it's a pretty strong contender, but we'll have to do some more comparisons.
Probably the biggest technical oddity to deal with is that it's hard in the JS world to turn async tasks into synchronous tasks without a native code helper that does some thread magic. So that might affect how internal APIs are designed inside JS (or Lua, or Python), or we might go ahead and model sync functions for some "may take a while" tasks like fetching data from the server. These could take longer when evaluated from a remote system such as an end-user device running edits/previews and needing to query the remote database, versus something rendering server-side where the wikidata servers are quicker to reach.
-- brion
On Sat, Jul 11, 2020 at 1:39 AM Adam Sobieski adamsobieski@hotmail.com wrote:
With respect to the early discussion topic [1] of the “data and evaluation model for the code repository (Wikilambda)”, I would like to broach, for discussion, that the V8 engine [9] and/or WebAssembly [10] can be of use for the Wikilambda project.
While reading the plan [2] and tasks [3] documents, I observed tasks: P1.12 [4], P1.15 [5], O6 [6], O16 [7] and O29 [8]. Task P1.12 [4] indicates JavaScript-based implementations. Task P1.15 [5] indicates Lua implementations. Lua can be compiled to WebAssembly [12]. Task O6 [6] indicates Python implementations. Python can be compiled to WebAssembly [13]. There is a lengthy list of programming languages which already can or are planned to soon be able to compile to WebAssembly [11]. Task O29 [8] indicates compilation for significant speedup.
The use of WebAssembly would facilitate task O16 [7], browser-based evaluation scenarios. WebAssembly is broadly supported by Web browsers.
The V8 engine is utilized by Chromium-based Web browsers such as Brave, Chrome, Edge, Opera, and Vivaldi [9]. “The V8 engine can be used in a browser or integrated into independent projects” [9]. The V8 engine is utilized by the popular Node.js runtime environment [9]. The V8 engine includes a WebAssembly implementation.
With respect to processing JavaScript, the V8 engine “first generates an abstract syntax tree with its own parser. Then, Ignition, the V8 interpreter, generates bytecode from this syntax tree using the internal V8 bytecode format [15]. TurboFan is the V8 optimizing compiler, it takes this bytecode and generates machine code from it. In other words, V8 compiles JavaScript directly to native machine code using just-in-time compilation before executing it. The compiled code is additionally optimized (and re-optimized) dynamically at runtime, based on heuristics of the code's execution profile. Optimization techniques used include inlining, elision of expensive runtime properties, and inline caching” [9]. The V8 engine “can compile to x86, ARM or MIPS instruction set architectures in both their 32- and 64-bit editions; as well, it has been ported to PowerPC and IBM s390 for use in servers” [9].
The V8 engine’s Ignition virtual machine [14] is “a register machine with an accumulator register” [15].
The WebAssembly virtual machine is “a portable virtual stack machine” [10].
In what ways can the V8 engine and/or WebAssembly be of use for the Wikilambda project? Could the V8 engine’s Ignition virtual machine [14][15] and/or the WebAssembly virtual machine [10] be of use as evaluation models or compilation targets for Wikilambda?
Best regards,
Adam Sobieski
[1] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Early_discussion_topics
[2] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan
[3] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks
[4] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.12:_JavaScr...
[5] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.15:_Lua-bas...
[6] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O6:_Python-base...
[7] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O16:_Browser-ba...
[8] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_O29:_Implement...
[9] https://en.wikipedia.org/wiki/V8_(JavaScript_engine)
[10] https://en.wikipedia.org/wiki/WebAssembly
[11] https://github.com/appcypher/awesome-wasm-langs
[12] https://github.com/appcypher/awesome-wasm-langs#lua
[13] https://github.com/appcypher/awesome-wasm-langs#python
[14] https://v8.dev/docs/ignition
[15] https://medium.com/dailyjs/understanding-v8s-bytecode-317d46c94775
Abstract-Wikipedia mailing list Abstract-Wikipedia@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/abstract-wikipedia
Le mar. 14 juil. 2020 à 19:19, Brion Vibber bvibber@wikimedia.org a écrit :
We'll definitely be looking at both JavaScript and WebAssembly runtimes, though the devil is always in the details!
The major web engines including V8 support both JavaScript and WebAssembly and good interoperation between them. This is potentially very useful, and gives an additional sandboxing layer on top of the Lua or Python runtimes if those end up being used for user-authored tasks.
It will be easier on targeting JavaScript because it is the only VM that wants to be sandboxed. I tried to work with webassembly using Python or Scheme. Using Python, there are two approaches one is micropython, but it does not support asynchronous code (or with a lot of pain) or wapy (https://github.com/pmp-p/wapy) which is still a work in progress. Using Scheme there is Chibi Scheme that has a wasm target via emscripten.
At the end of the day, I think the required API for Abstract Wikilambda should be expressed with something similar to Web IDL (https://en.wikipedia.org/wiki/Web_IDL) that people can implement in their favorite language.
I really think web assembly is the future, but we are not there yet, in particular, because host interop is not ready yet, read https://hacks.mozilla.org/2019/08/webassembly-interface-types/
Maybe graalvm has sandboxing support in which case it will be a good path.
Otherwise, I read a lot of companies are running untrusted code in their public clouds with things like serverless, so they might be a way to run untrusted code in a sandbox without diving into compilation et al. I think they rely on containers.
This also means we can straightforwardly run evaluator systems browser-side and in native mobile apps, both of which can embed JS and Wasm using system libraries (V8 on Chrome and Android, WebKit's JSC for Safari, iOS and iPadOS, and SpiderMonkey for Firefox).
So it's a pretty strong contender, but we'll have to do some more comparisons.
Probably the biggest technical oddity to deal with is that it's hard in the JS world to turn async tasks into synchronous tasks without a native code helper that does some thread magic. So that might affect how internal APIs are designed inside JS (or Lua, or Python), or we might go ahead and model sync functions for some "may take a while" tasks like fetching data from the server. These could take longer when evaluated from a remote system such as an end-user device running edits/previews and needing to query the remote database, versus something rendering server-side where the wikidata servers are quicker to reach.
-- brion
On Sat, Jul 11, 2020 at 1:39 AM Adam Sobieski adamsobieski@hotmail.com wrote:
With respect to the early discussion topic [1] of the “data and evaluation model for the code repository (Wikilambda)”, I would like to broach, for discussion, that the V8 engine [9] and/or WebAssembly [10] can be of use for the Wikilambda project.
While reading the plan [2] and tasks [3] documents, I observed tasks: P1.12 [4], P1.15 [5], O6 [6], O16 [7] and O29 [8]. Task P1.12 [4] indicates JavaScript-based implementations. Task P1.15 [5] indicates Lua implementations. Lua can be compiled to WebAssembly [12]. Task O6 [6] indicates Python implementations. Python can be compiled to WebAssembly [13]. There is a lengthy list of programming languages which already can or are planned to soon be able to compile to WebAssembly [11]. Task O29 [8] indicates compilation for significant speedup.
The use of WebAssembly would facilitate task O16 [7], browser-based evaluation scenarios. WebAssembly is broadly supported by Web browsers.
The V8 engine is utilized by Chromium-based Web browsers such as Brave, Chrome, Edge, Opera, and Vivaldi [9]. “The V8 engine can be used in a browser or integrated into independent projects” [9]. The V8 engine is utilized by the popular Node.js runtime environment [9]. The V8 engine includes a WebAssembly implementation.
With respect to processing JavaScript, the V8 engine “first generates an abstract syntax tree with its own parser. Then, Ignition, the V8 interpreter, generates bytecode from this syntax tree using the internal V8 bytecode format [15]. TurboFan is the V8 optimizing compiler, it takes this bytecode and generates machine code from it. In other words, V8 compiles JavaScript directly to native machine code using just-in-time compilation before executing it. The compiled code is additionally optimized (and re-optimized) dynamically at runtime, based on heuristics of the code's execution profile. Optimization techniques used include inlining, elision of expensive runtime properties, and inline caching” [9]. The V8 engine “can compile to x86, ARM or MIPS instruction set architectures in both their 32- and 64-bit editions; as well, it has been ported to PowerPC and IBM s390 for use in servers” [9].
The V8 engine’s Ignition virtual machine [14] is “a register machine with an accumulator register” [15].
The WebAssembly virtual machine is “a portable virtual stack machine” [10].
In what ways can the V8 engine and/or WebAssembly be of use for the Wikilambda project? Could the V8 engine’s Ignition virtual machine [14][15] and/or the WebAssembly virtual machine [10] be of use as evaluation models or compilation targets for Wikilambda?
Best regards,
Adam Sobieski
[1] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Early_discussion_topics
[2] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan
[3] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks
[4] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.12:_JavaScr...
[5] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_P1.15:_Lua-bas...
[6] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O6:_Python-base...
[7] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Plan#Task_O16:_Browser-ba...
[8] https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Tasks#Task_O29:_Implement...
[9] https://en.wikipedia.org/wiki/V8_(JavaScript_engine)
[10] https://en.wikipedia.org/wiki/WebAssembly
[11] https://github.com/appcypher/awesome-wasm-langs
[12] https://github.com/appcypher/awesome-wasm-langs#lua
[13] https://github.com/appcypher/awesome-wasm-langs#python
[14] https://v8.dev/docs/ignition
[15] https://medium.com/dailyjs/understanding-v8s-bytecode-317d46c94775
abstract-wikipedia@lists.wikimedia.org