What useful tasks are possible with an LLM with only 3B parameters?

talklittle
Tue, 26 Mar 2024 19:24:37 +0000

Playing with Llama 7B and 13B, I found that the 13B model was capable of doing a simple task, rewriting titles in sentence case for Tildes submissions. The 7B model doesn't appear capable of the same task, out of the box.

I heard about Android's new AICore available on a couple of new devices. But it sounds like Gemini Nano, which runs on-device, can only handle 2B or 3B parameters.

Is this size of model useful for real tasks? Does it only become useful after training on a specific domain? I'm a novice and wanting to learn a little bit about it. On-device AI is an appealing concept to me.

Comments URL: https://tildes.net/~comp/1f7a/what_useful_tasks_are_possible_with_an_llm_with_only_3b_parameters

Votes: 12

Comments: 2