• Activity
  • Votes
  • Comments
  • New
  • All activity
    • Showing only topics in ~tech with the tag "language models.large". Back to normal view / Search all groups

    11. Let's talk Local LLMs - So many questions

      Ask
      Hello there (oh god, I am opening my first thread here - so exciting) I'd love to ask the people here about local LLMs. To be honest, I got interested in this topic, but am leaving reddit, where a...

      Hello there
      (oh god, I am opening my first thread here - so exciting)

      I'd love to ask the people here about local LLMs.
      To be honest, I got interested in this topic, but am leaving reddit, where a sub r/locallama exists.
      I don't want to interact with that site anymore, so I am taking this here.

      My questions, to start us off:

      • Models are available on huggingface (among other places), but where do I get the underlying software? I read "oogabooga" somewhere, but honestly, I am lost.
      • If I only want to USE a local model, what are the requirements, and how do I judge if I can use something from the values of "4bit / 8 bit" and "30B, 7B"??
      • If I get crazy and want to TRAIN a LorA ... what then?
      • Good resources / wiki pages, tutorials, etc?
      21 votes