The original post: /r/homelab by /u/cpjet64 on 2025-03-13 14:01:51.
I have a cluster of R730s at home running proxmox and want to self host textgenwebui and a few models. What are people using? I want to host a single 32B model at a time so my thought was to get a pair of these, assemble them and then install the latest driver and send it. I figure 32gb of vram for <600 wasnt too bad of a deal. Does anyone have any other suggestions for a decent solution? I should add two of my 3 servers already have 1x M4000 gpus in them running GRID passing through to my windows vms.
You must log in or # to comment.