Some interesting updates on current developments

Open Assistant is open source ChatGPT clone with crowdsourced fine-tuning
- https://open-assistant.io

Redpajama is project for reproducing LLaMA and releasing the model under open source licence. Current status is that they have released the pre-training data
- https://www.together.xyz/blog/redpajama

Free-dolly is CC-BY-SA licenced fine-tuning dataset. 
- https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm

 LLM chat which runs in web browser (non-open source Vicuna-7B)  
- https://simonwillison.net/2023/Apr/16/web-llm/

Afaik all of these are clear steps towards full open source LLM stack. Especially Open Assistant is especially interesting as it is focusing on crowdsourcing.

Br,
-- Kimmo Virtanen, Zache

On Mon, Apr 3, 2023 at 8:43 PM Samuel Klein <meta.sj@gmail.com> wrote:
At this point I guess I would recommend adding five or so
g2.cores8.ram36.disk20 flavor VPSs to WMCS, with between one and three
RTX A6000 GPUs each, plus a 1TB SSD each, which should cost under
$60k. That should allow for very widely multilingual models somewhere
between GPT-3.5 and 4 performance with current training rates.

Having part of the cluster for this makes sense, even as what it is used for changes over time.
 
These models can be quantized into int4 weights which run on cell
phones: https://github.com/rupeshs/alpaca.cpp/tree/linux-android-build-support
It seems inevitable that we will someday include such LLMs with
Internet-in-a-Box, and, why not also the primary mobile apps

Eventually, yes. A good reason to renew attention to mobile as a canonical wiki experience.
 

_______________________________________________
Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/R2KSXD3VE4MPGSIIYCPUMFXFPIY7D5CH/
To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org