Hi all,
Are there any LLMs available on Cloud services, or are there any plans for them? I think there are many possible use cases. Even free, lightweight LLMs (like LLaMa) could be helpful, e.g. in bots that review edits, categorize pages, etc.
Thanks, Huji
On 1/10/25 05:21, Huji Lee wrote:
Hi all,
Are there any LLMs available on Cloud services, or are there any plans for them? I think there are many possible use cases. Even free, lightweight LLMs (like LLaMa) could be helpful, e.g. in bots that review edits, categorize pages, etc.
Hi Huji,
No, there is none at this point in time.
But this has been in my radar for some time now, and it would be useful to know particular use cases and needs before we can work on any implementation.
This email is already helpful (to know there is at least one person interested). A phabricator ticket would also be useful, to help inform future actions. Would you mind creating one?
regards.
A phabricator ticket would also be useful, to help inform future actions. Would you mind creating one?
We have https://phabricator.wikimedia.org/T336905 that seems to cover the same topic.
Since it's now possible in Toolforge to expose services at custom ports[0], I think it should already be possible for someone to host an LLM service for other tools to use.
I could be wrong though, as LLMs might also require significant memory/CPU resources and/or system software not available to tools.
[0]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework#Configurin...
On Fri, 10 Jan 2025 at 16:31, Francesco Negri fnegri@wikimedia.org wrote:
A phabricator ticket would also be useful, to help inform future
actions. Would
you mind creating one?
We have https://phabricator.wikimedia.org/T336905 that seems to cover the same topic.
-- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation _______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
Huji wrote:
Even free, lightweight LLMs (like LLaMa) could be helpful
LLaMa itself is not under a free license. Let's call it an "almost-free license". So, I'm not sure if it would be acceptable to run it, given the requisite that
All code in the Tools project must be published under an OSI approved
open source license
I think the debate would be whether the models (in this case LLaMa) is "code" or "data". It might be considered both ways.
Separatedly, regarding the resources point mentioned by Siddhart:
LLMs might also require significant memory/CPU resources and/or system
software not available to tools
LLMs are very memory-hungry, but what they would benefit most would be from GPU memory. Of which we probably don't have any in cloud. The ideal setup would probably be a specific host in cloud providing a LLM service and offering that to tools and VMs.
(*) It's still possible to run LLMs without GPU and get acceptable results, although the time required and amount of requests that can be fulfilled (as well as number of models loaded) would be much more limited.
Hello,
it would be useful to know particular use cases and needs before we can
work on any implementation.
I have an example in mind which I have been thinking about for a while now: I have a bot task that creates category trees and adds the leaf categories to pages on arywiki. Currently, I have to create the tree of categories by hand as a json file, which is fed to that script, but I was thinking of generating the tree programmatically as well. It would be an interesting use case of LLMs. It could be done by translating the category tree from another language (mostly English) or generating a new tree by analogy to another.
Le ven. 10 janv. 2025 à 11:02, Arturo Borrero Gonzalez < aborrero@wikimedia.org> a écrit :
On 1/10/25 05:21, Huji Lee wrote:
Hi all,
Are there any LLMs available on Cloud services, or are there any plans
for them?
I think there are many possible use cases. Even free, lightweight LLMs
(like
LLaMa) could be helpful, e.g. in bots that review edits, categorize
pages, etc.
Hi Huji,
No, there is none at this point in time.
But this has been in my radar for some time now, and it would be useful to know particular use cases and needs before we can work on any implementation.
This email is already helpful (to know there is at least one person interested). A phabricator ticket would also be useful, to help inform future actions. Would you mind creating one?
regards. _______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/