You shouldn’t use your computer to use local models out of reach of the Hyperscalers? You shouldn’t use your single GPU with 16 GB Vram because hyperscalers are building datacenters that can’t be used for anything else, using evaporative cooling because republican politicians just ignore their constituents, increasing everyones power bills while erecting gas turbines poisoning the area for a model you don’t have any control over?
Please try to separate local models and the hyperscalers bullshit. Local models stay under local control, don’t need racks full of blackwells or vera rubin, don’t need datacenters at all. All they need is a consumer grade GPU with enough Vram, which would be cheap as fuck if the hyperscalers wouldnt buy up every single last bit of silicon. They are cooled by 1-2 fans and an aluminium block, or if you are getting fancy using a closed loop watercooling solution.
Local LLM’s are not comparable in resource expension at all - they are a small blip, but that blip is at least under control of the person who decided to be independent from all this shit.
You shouldn’t use your computer to use local models out of reach of the Hyperscalers? You shouldn’t use your single GPU with 16 GB Vram because hyperscalers are building datacenters that can’t be used for anything else, using evaporative cooling because republican politicians just ignore their constituents, increasing everyones power bills while erecting gas turbines poisoning the area for a model you don’t have any control over?
Please try to separate local models and the hyperscalers bullshit. Local models stay under local control, don’t need racks full of blackwells or vera rubin, don’t need datacenters at all. All they need is a consumer grade GPU with enough Vram, which would be cheap as fuck if the hyperscalers wouldnt buy up every single last bit of silicon. They are cooled by 1-2 fans and an aluminium block, or if you are getting fancy using a closed loop watercooling solution.
Local LLM’s are not comparable in resource expension at all - they are a small blip, but that blip is at least under control of the person who decided to be independent from all this shit.