I understand why a graphics card and a lot of VRAM would be important for AI like stable diffusion, why does this spec matter for language models too that don’t use graphics?
They have a lot of fast memory and are great at doing things in parallel. Most AI are just operations on matrixes, which essentially is what a GPU is built for.
I understand why a graphics card and a lot of VRAM would be important for AI like stable diffusion, why does this spec matter for language models too that don’t use graphics?
They have a lot of fast memory and are great at doing things in parallel. Most AI are just operations on matrixes, which essentially is what a GPU is built for.
GPUs are great for parallel tasks. Computing answers requires a lot of parallel tasks. CPUs are amazing for doing one thing at a time.