Some interesting updates on current developments
Open Assistant is open source ChatGPT clone with crowdsourced fine-tuning - https://open-assistant.io
Redpajama is project for reproducing LLaMA and releasing the model under open source licence. Current status is that they have released the pre-training data - https://www.together.xyz/blog/redpajama
Free-dolly is CC-BY-SA licenced fine-tuning dataset. - https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-via...
LLM chat which runs in web browser (non-open source Vicuna-7B) - https://simonwillison.net/2023/Apr/16/web-llm/
Afaik all of these are clear steps towards full open source LLM stack. Especially Open Assistant is especially interesting as it is focusing on crowdsourcing.
Br, -- Kimmo Virtanen, Zache
On Mon, Apr 3, 2023 at 8:43 PM Samuel Klein meta.sj@gmail.com wrote:
At this point I guess I would recommend adding five or so
g2.cores8.ram36.disk20 flavor VPSs to WMCS, with between one and three RTX A6000 GPUs each, plus a 1TB SSD each, which should cost under $60k. That should allow for very widely multilingual models somewhere between GPT-3.5 and 4 performance with current training rates.
Having part of the cluster for this makes sense, even as what it is used for changes over time.
These models can be quantized into int4 weights which run on cell phones: https://github.com/rupeshs/alpaca.cpp/tree/linux-android-build-support It seems inevitable that we will someday include such LLMs with Internet-in-a-Box, and, why not also the primary mobile apps
Eventually, yes. A good reason to renew attention to mobile as a canonical wiki experience.
Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org