Check out /r/localllama. Preferably you need a Nvidia you with >= 24 GB VRAM but it also works with a cpu and loads of normal RAM, if you can wait a minute or two for a lengthy answer. Loads of models to choose from, many with no censorship at all. Won’t be as good as chatgptv4, but many are close to gpt3.
I understand why a graphics card and a lot of VRAM would be important for AI like stable diffusion, why does this spec matter for language models too that don’t use graphics?
They have a lot of fast memory and are great at doing things in parallel. Most AI are just operations on matrixes, which essentially is what a GPU is built for.
Check out /r/localllama. Preferably you need a Nvidia you with >= 24 GB VRAM but it also works with a cpu and loads of normal RAM, if you can wait a minute or two for a lengthy answer. Loads of models to choose from, many with no censorship at all. Won’t be as good as chatgptv4, but many are close to gpt3.
I understand why a graphics card and a lot of VRAM would be important for AI like stable diffusion, why does this spec matter for language models too that don’t use graphics?
They have a lot of fast memory and are great at doing things in parallel. Most AI are just operations on matrixes, which essentially is what a GPU is built for.
GPUs are great for parallel tasks. Computing answers requires a lot of parallel tasks. CPUs are amazing for doing one thing at a time.
Just played with it the other week, they have some models that run on less extreme hardware too https://ollama.ai/