Ryzen 5900X, 64 gig DDR4-3200, 2tb ssd,10tb hdd and an RTX2070. Hosting Stable Diffusion, various llama.cpp instances with python bindings, jellyfin, sonarr, multiple modded minecraft servers, and a network file share.
You must log in or # to comment.
Certainly blows my collection of old junk out of the water.
Has running Stable Diffusion had a big impact on the other stuff running? I’m thinking the Minecraft servers in particular.
SD mostly uses the GPU, so it’s pretty light on everything else. The largest process is probably web-chat-ui with Wizard-30b model running.
Nice, you must be into deep learning with such a setup, any particular reason the deep deep learning models and GPU run in your server rather than in a powerful desktop system? Maybe you’re actively offering AI services to the outside world?
It is a powerful desktop system as in it’s housed in a desktop case and uses a “non-server” CPU.
Which like any computer can be a server. Being a server is more of a role than a form factor even if there are form factors specifically aimed at servers (rack mounted).
Yup, mostly running pretrained models for text embedding and some generative stuff. No real fine tuning.
Very nice ! +1 for not showing an amazon tier server rack and calling it your modest homelab.
Do you have some links or resources for the hosted llama.cpp instances ?
I’m very ignorant about it. I wonder if I can buy a cheap computer and turn it into a server that I can only use by keeping it on all the time?
@Alpagu @behohippy Sure, that’s how we all started. An old computer, throw proxmox in there, a few VMs and/or a docker host and you have yourself a pretty sweet homelab.
Thank you for answer
Yup, typically we get into it after upgrading an older PC or something and instead of selling the parts, just turn it into a server. You can also find all sorts of cheap/good stuff on ebay from office off-lease.
Thanks for answer
