Some interesting updates on current developments
Open Assistant is open source ChatGPT clone with crowdsourced fine-tuning
-
https://open-assistant.io
Redpajama is project for reproducing LLaMA and releasing the model under
open source licence. Current status is that they have released the
pre-training data
-
https://www.together.xyz/blog/redpajama
Free-dolly is CC-BY-SA licenced fine-tuning dataset.
-
https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-vi…
LLM chat which runs in web browser (non-open source Vicuna-7B)
-
https://simonwillison.net/2023/Apr/16/web-llm/
Afaik all of these are clear steps towards full open source LLM stack.
Especially Open Assistant is especially interesting as it is focusing on
crowdsourcing.
Br,
-- Kimmo Virtanen, Zache
On Mon, Apr 3, 2023 at 8:43 PM Samuel Klein <meta.sj(a)gmail.com> wrote:
At this point I guess I would recommend adding five or
so
g2.cores8.ram36.disk20 flavor VPSs to WMCS, with
between one and three
RTX A6000 GPUs each, plus a 1TB SSD each, which should cost under
$60k. That should allow for very widely multilingual models somewhere
between GPT-3.5 and 4 performance with current training rates.
Having part of the cluster for this makes sense, even as what it is used
for changes over time.
These models can be quantized into int4 weights
which run on cell
phones:
https://github.com/rupeshs/alpaca.cpp/tree/linux-android-build-support
It seems inevitable that we will someday include such LLMs with
Internet-in-a-Box, and, why not also the primary mobile apps
Eventually, yes. A good reason to renew attention to mobile as a canonical
wiki experience.
_______________________________________________
Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines
at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
https://meta.wikimedia.org/wiki/Wikimedia-l
Public archives at
https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org…
To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org