Yeah, this has to be pretty limited compared to actual ChatGPT. RAM required for the scale of models for GPT3/4 are well beyond consumer hardware. 13B parameter model takes about 8gb, and GPT3/4 use around 165B as I recall.
I think the way to get something comparable would be using an approach like Bloom where you have a torrent style distributed system effectively creating a crowd sourced supercomputer.
Yeah, this has to be pretty limited compared to actual ChatGPT. RAM required for the scale of models for GPT3/4 are well beyond consumer hardware. 13B parameter model takes about 8gb, and GPT3/4 use around 165B as I recall.
I think the way to get something comparable would be using an approach like Bloom where you have a torrent style distributed system effectively creating a crowd sourced supercomputer.
indeed. i just learned about Petals today (and reposted it here - i see you also posted it months ago)
The more posts about it the better. :)