this us really odd behaviour. seems like either you mentioned something about china in the prompt or deepseek did some aggressive post training on those distillations to make them mention this, which would be news to me.
anyway, here’s the obligatory whatever:
do not use LMs even if it is to make fun of them… and yes even if it is a locally hosted model iguess too
EDIT: this is a bit. im pretending to be the unnuanced anti-ai peep i see so often on lemmy. please dont reply with arguments I won’t read thru, I dont actually take this stance.
Sorry, but there’s a very big difference between using my graphics card that is there for my entertainment to run an LLM using standard air cooling (aka a small fan with a shaped brick of aluminium) and:
building huge complexes that can’t be used for anything else
use evaporative cooling, especially in dry areas, because the republicans there make it easier to simply ignore local resistance
use fucking gas turbines poisoning the air in the area.
increase the power bill of everyone around them because fuck the common people
The amount of resources that are used for the 2 scenarios are not even comparable. I paid for my single GPU with Vram to be able to play games, running small specialist local models is simply an plus this enables. I pay for my power bill - my GPU maxes out at around 250W, which is 1/4th of my microwave - and that bill isn’t as egregious as what the US power companies are currently pushing on their private clients to expand their power generation.
And the most important difference: local open models serve me and not the interests of someone else.
If you can’t see the difference, i am very sorry for you, because a future where we are in control of local models would be very much more preferable to the shit the hyperscalers are trying to push currently.
Training of models is a one-time cost. If that model gets widespread use, then the cost of creating it in the first place becomes less and less relevant. This means we all would profit from creating “creative commons”-models that can be used privately free of charge, while licensing them out to businesses to finance the costs of licensing material for the next “update” of the CC model. Someone more involved might use his calculator to know the ideal update frequency (6 months? 1 year? if its really expensive maybe every 2 years).
Using and talking positively of self hosted LLMs still promotes LLMs and convinces people to use them (most likely using services owned by large corporations)
Its still something that can and has caused people to not trust their parents, doctors, believing what the chatbot says instead. Its still something that can and has caused people to kill others or themselves
If used on the internet, it still contributes to it becoming worse (even without considering my first point)
It isnt as bad, but it still has problems. Whether the use is worth it is something that i have not seen discussed at all (im really curious about the uses for llms, the technology seems cool with actual uses, but every example i have seen was terrible)
(Also, isnt the energy issue with the training, not the usage?)
LLMs are not the devil, they are useful tools. Talking positively about local models is simply that - informing the surroundings that you can have a model that is NOT bound to a hyperscaler. We should talk MORE about local models to make people aware that there is a way forward without OpenAI and Anthropic.
Anyone who will put in the time to actually work with a local model will also be aware of the limitation of those things - another point for “make people own their own shit”. Anyone who involves themselves in running a model will also be keenly aware that they are NOT TALKING TO A SENTIENT BEING (I fault Sam Altman and Dario Amodei personally for spreading this bullshit, they should be sued and thrown in jail)
The energy cost of training a model for local use is not even comparable to the huge amount of resources that are used for training something like Mythos - these things don’t scale linear, but exponentially. And it is a one time cost, which means the longer a model is in use the less relevant the training costs become (just as buying a car -the upfront costs are high, but in the long run you will pay more in gas and maintenance than you paid for the car itself). Even if you train one large model on the scale of Mythos, you then have the possibility of distilling those models into smaller ones for local use, specialized for specific areas at a fraction of the cost.
I had a long reply written, but got deleted because my phone ran out of battery; i doubt any of our opinions about the existence and relevance of the discussed downsides would change if this was argued further anyways (and i am too lazy to rewrite the reply)
Your reply doesnt answer the question asked in my comment. You say its a tool, but didnt say what you use it for
(Also, the first sentence seems like a generic reply to any comment anti-ai, it doesnt say anything and its not even relevant in this situation; it only makes your arguments feel worse)
I disagree regarding the “generic reply to Anti-ai”. Discourse nowadays simply ignores locally run models - they do not exist outside of a sphere of technically adept users. Making sure more people know that you don’t have to finance OpenAI, Anthropic, Meta or the Neoclouds should be way more accessible information.
This tech and what is happening currently should not be conflated. If Big Tech used up all global stockpiles of bubble gum because they wanted to create the worlds largest chewing gum bubble, it doesn’t mean that you with your single pack of gum is even comparable to what they are doing.
The only question in your comment i can see is the concern about energy cost while training a model. Those costs are a static amount - not ignoreable, but finite. The longer you use that model, or increasing the amount of people using it, reduces the cost per instance. Use a model long enough and the energy cost of creating in in the first place converges towards zero. It’s another point for local LLMs - nobody needs the aggressive update cycle Anthropic and OpenAI are running, and those can’t stop because the moment they do, investor money will stop flowing.
For what it can be used: well obviously it is a great tool in all areas where you can automate checking if what it did meets a specific criteria. That’s the reason why Mythos has Business entities like Microsoft scrambling like mad just right now - checking to see if Mythos managed to escalate privileges or run remote code is trivial, and Mythos is showing up security issues all over their software, up to the point that MS is scared that even if they patch the really ugly shit, Mythos might still chain multiple low level vulnerabilities to fuck them over, and all that is needed from Joe Shmoe are the words “go hack Onedrive”.
I personally like to use them to calculate stuff where i know it can be calculated because of my experience, but where i do not recall the specific formulas needed. I also use it for creating regular expressions on the fly, teaching myself how to create useful bash scripts (all my knowledge about sed and awk comes from a local LLM), among other things that are not the business of anyone - that’s why I am using a local model in the first place.
I honestly forgot that i also put a question about training cost; by the question i meant when i asked about uses for ai, i didnt realize i made it unclear whether it was a question or not
I dont think a model that you have said is bad for the environment is a good example
Regarding the first sentence, i have said that this seems like a generic reply because i have not said that llms are the worst thing ever (even saying that their uses could offset the downsides) or that they do not have uses (“AI is just a tool” is also an argument i see so often, never with any explanation behind)
For such impressive technology, i doubt the best use of it is a better search engine that you have to fact check or writing help
I have seen someone use a local model to control home assistant
Oh sorry, i didn’t realize that. OK, so looking at the thread, i would say my reply would fit the inane FUD @[email protected] is spouting. Klear, if you wanna reply, i am open for discussion.
It’s really a shame that in order to stimulate adoption to increase development of LLMs they have to do so much damage first. It’s not unlike with cars, scaling up from the very first ICE models has pretty much doomed us all and even now vehicular transport is still at a net loss.
By the time LLMs or even AI become efficient enough that they can run on the more and more advanced tech of handheld devices (maybe quantum?), it’ll have done so much damage, not to mention all those datacenters will need a new purpose. I hope to see it in my lifetime because of advancement life I that ever happens, I’m holding out hope AI will help us do stuff so much more efficiently we can actually reverse all of the damage done.
Either that or it’ll just kill all humans deciding we are the problem.
what’s the argument against locally hosted AI? lile seriously man I have issues with plagiarism, innacurate information and social dependency, but even with all that AI isn’t of the devil. Learn to have nuance
local models don’t appear out of the ether. using a local model solves one problem around privacy of your prompts, does little about consolidation of power, and nothing about the thousands of other problems genai brings.
if your organic coffee is made with slave labor, you still have a massive problem.
and they may not be of the devil, but they definitely are of the humans who most resemble the devil.
The models you are self-hosting first have to be trained at the data centers, which is a contributing factor to the environmental impact of AI.
I know how to run local models, but just because they’re more environmentally friendly than alternatives doesn’t mean they’re not still contributing to the broader environmental impact of data centers.
How does running a local LLM that’s already been trained contribute to the environmental costs of training? It’s already trained whether I run it or not, and running it locally isn’t generating any revenue for the company that did the training (therefore it doesn’t encourage them to train more). This isn’t like a physical product where every time someone obtains a copy, more environmental damage has to be done manufacturing it, training is a one time cost.
The environmental impact of running a local LLM is significantly lower than buying a half-gallon carton of almond milk.
Sure, but you could also make the argument that anything based on networked computing is contributing to the spread of data centers. AI is a large part of that today, but it’s not like data centers never existed prior to ChatGPT. Before, it was about trying to convince laypeople to avoid “the cloud”.
Non-AI datacenters aren’t massive fuckoff Complexes that spew pollution (noise and atmospheric) into the surrounding area and triple your electric bill. It’s the massive hyperscale AI datacenters that are the problem.
That’s true, loads of non-AI datacenters are also environmentally destructive noise pollution machines, which also needs to be dealt with, but hyperscale AI datacenters are much more environmentally destructive much more of the time. You’re right tho
unironically, being mean to inanimate objects is still being mean. i don’t want to imply lojcs was doing that, i just enjoy showing kindness to silly little things and thoughts and characters. sometimes i give our computer headpats when it does a good job, hehe :3
oh yeye, i mean - samsies totally. when my blender render finishes and my lappytoppy is all exhausted from the rendering work, i tell it that it did so well for me <3
I don’t see a ‘should’? That’s a straight command. And even if it were there, is it any more reasonable to dictate what people should not do with their computers?
Because you’re an earthling, just like all of us, and LMs are threatening earth, and it is your duty to not use them, especially not for stupid memes. You must stop using LMs on your own computer or anywhere. I am telling you that as your sister on this planet that we both inhabit.
Ok stow away the hippie crap for a moment. The difference in scale between someone using their personal hardware to run models and the shit the hyperscalers are doing is not even fathomable.
My GPU uses a max of 250W if under full load, which it uses for a few seconds while running a simple request. A single Blackwell blows it out of the water, but those are not provisioned as single boards. They are provided in large racks, containing more CPUs, GPUs, RAM and Storage than i have ever possessed in my life, and i am at it since the early 90s. The difference in resource usage between those 2 scenarios isn’t even measurable because it is simply bonkers.
A individual running a language model on a single 16gb GPU causes orders of magnitude less environmental damage than buying a carton of almond milk. Please direct your anger where it actually belongs, at the goddamn capitalist vampires building hyperscale datacenters.
I honestly can’t tell if you’re joking at my expense or if you’re serious. This thread started to feel like Hexbear for some reason and it makes me want to cry
If you’re joking know that this is a totally believable sentiment for somebody on Lemmy to have.
If you’re serious, then you’re delusional. LLMs themselves can’t threathen anything, they’re not alive. You might be thinking about their power usage or the sentiments of the companies that make them, but none of that is a factor when using them locally. The computer uses the same power regardless of if it’s used to play games or generate text.
Honestly fuck this platform. It is insufferable to post something I had found amusing just to be scolded for not following the Lemmy hivemind. If anyone knows a platform that’s not corporate and doesn’t have such a kneejerk hatred of llms dm me. Goodbye
Lmao if people not wanting to see a user support an industry that’s killing the planet at an alarming scale in such a short time makes you want to leave, then I guess good riddance.
“see if I make weapons at HOME with the gun-O-matic that Lockheed stole and killed so much to produce, I’m not supporting the military industrial complex!”
It’s more about which instance you’re posting in. dbzer0.com for example has far less issues with LLMs and neural networks when run locally - those are practically embraced in the respective communities, while the hyperscalers attempts are loathed as they should be.
Sadly, people conflate local neural networks with the crap they are getting pushed by MS/Meta/OpenAI/Anthropic. Most of them don’t realize the difference in scale between running a local model on a GPU with 16GB VRam and 250W TDP vs even a single rack of Blackwells. They neither have the technical knowledge to separate the two things from another, nor the political knowledge to realize that local NNs empower, while centralized NNs disenfranchise.
You shouldn’t use your computer to use local models out of reach of the Hyperscalers? You shouldn’t use your single GPU with 16 GB Vram because hyperscalers are building datacenters that can’t be used for anything else, using evaporative cooling because republican politicians just ignore their constituents, increasing everyones power bills while erecting gas turbines poisoning the area for a model you don’t have any control over?
Please try to separate local models and the hyperscalers bullshit. Local models stay under local control, don’t need racks full of blackwells or vera rubin, don’t need datacenters at all. All they need is a consumer grade GPU with enough Vram, which would be cheap as fuck if the hyperscalers wouldnt buy up every single last bit of silicon. They are cooled by 1-2 fans and an aluminium block, or if you are getting fancy using a closed loop watercooling solution.
Local LLM’s are not comparable in resource expension at all - they are a small blip, but that blip is at least under control of the person who decided to be independent from all this shit.
this us really odd behaviour. seems like either you mentioned something about china in the prompt or deepseek did some aggressive post training on those distillations to make them mention this, which would be news to me.
anyway, here’s the obligatory whatever:
EDIT: this is a bit. im pretending to be the unnuanced anti-ai peep i see so often on lemmy. please dont reply with arguments I won’t read thru, I dont actually take this stance.
Sorry, but there’s a very big difference between using my graphics card that is there for my entertainment to run an LLM using standard air cooling (aka a small fan with a shaped brick of aluminium) and:
The amount of resources that are used for the 2 scenarios are not even comparable. I paid for my single GPU with Vram to be able to play games, running small specialist local models is simply an plus this enables. I pay for my power bill - my GPU maxes out at around 250W, which is 1/4th of my microwave - and that bill isn’t as egregious as what the US power companies are currently pushing on their private clients to expand their power generation.
And the most important difference: local open models serve me and not the interests of someone else.
If you can’t see the difference, i am very sorry for you, because a future where we are in control of local models would be very much more preferable to the shit the hyperscalers are trying to push currently.
Iirc, the local models you run had to be developed in the hige datacenters first
Training of models is a one-time cost. If that model gets widespread use, then the cost of creating it in the first place becomes less and less relevant. This means we all would profit from creating “creative commons”-models that can be used privately free of charge, while licensing them out to businesses to finance the costs of licensing material for the next “update” of the CC model. Someone more involved might use his calculator to know the ideal update frequency (6 months? 1 year? if its really expensive maybe every 2 years).
Models can be trained on consumer hardware. They’ll just be limited in size and scope compared to larger models.
Using and talking positively of self hosted LLMs still promotes LLMs and convinces people to use them (most likely using services owned by large corporations)
Its still something that can and has caused people to not trust their parents, doctors, believing what the chatbot says instead. Its still something that can and has caused people to kill others or themselves
If used on the internet, it still contributes to it becoming worse (even without considering my first point)
It isnt as bad, but it still has problems. Whether the use is worth it is something that i have not seen discussed at all (im really curious about the uses for llms, the technology seems cool with actual uses, but every example i have seen was terrible)
(Also, isnt the energy issue with the training, not the usage?)
LLMs are not the devil, they are useful tools. Talking positively about local models is simply that - informing the surroundings that you can have a model that is NOT bound to a hyperscaler. We should talk MORE about local models to make people aware that there is a way forward without OpenAI and Anthropic.
Anyone who will put in the time to actually work with a local model will also be aware of the limitation of those things - another point for “make people own their own shit”. Anyone who involves themselves in running a model will also be keenly aware that they are NOT TALKING TO A SENTIENT BEING (I fault Sam Altman and Dario Amodei personally for spreading this bullshit, they should be sued and thrown in jail)
The energy cost of training a model for local use is not even comparable to the huge amount of resources that are used for training something like Mythos - these things don’t scale linear, but exponentially. And it is a one time cost, which means the longer a model is in use the less relevant the training costs become (just as buying a car -the upfront costs are high, but in the long run you will pay more in gas and maintenance than you paid for the car itself). Even if you train one large model on the scale of Mythos, you then have the possibility of distilling those models into smaller ones for local use, specialized for specific areas at a fraction of the cost.
I had a long reply written, but got deleted because my phone ran out of battery; i doubt any of our opinions about the existence and relevance of the discussed downsides would change if this was argued further anyways (and i am too lazy to rewrite the reply)
Your reply doesnt answer the question asked in my comment. You say its a tool, but didnt say what you use it for
(Also, the first sentence seems like a generic reply to any comment anti-ai, it doesnt say anything and its not even relevant in this situation; it only makes your arguments feel worse)
I disagree regarding the “generic reply to Anti-ai”. Discourse nowadays simply ignores locally run models - they do not exist outside of a sphere of technically adept users. Making sure more people know that you don’t have to finance OpenAI, Anthropic, Meta or the Neoclouds should be way more accessible information.
This tech and what is happening currently should not be conflated. If Big Tech used up all global stockpiles of bubble gum because they wanted to create the worlds largest chewing gum bubble, it doesn’t mean that you with your single pack of gum is even comparable to what they are doing.
The only question in your comment i can see is the concern about energy cost while training a model. Those costs are a static amount - not ignoreable, but finite. The longer you use that model, or increasing the amount of people using it, reduces the cost per instance. Use a model long enough and the energy cost of creating in in the first place converges towards zero. It’s another point for local LLMs - nobody needs the aggressive update cycle Anthropic and OpenAI are running, and those can’t stop because the moment they do, investor money will stop flowing.
For what it can be used: well obviously it is a great tool in all areas where you can automate checking if what it did meets a specific criteria. That’s the reason why Mythos has Business entities like Microsoft scrambling like mad just right now - checking to see if Mythos managed to escalate privileges or run remote code is trivial, and Mythos is showing up security issues all over their software, up to the point that MS is scared that even if they patch the really ugly shit, Mythos might still chain multiple low level vulnerabilities to fuck them over, and all that is needed from Joe Shmoe are the words “go hack Onedrive”.
I personally like to use them to calculate stuff where i know it can be calculated because of my experience, but where i do not recall the specific formulas needed. I also use it for creating regular expressions on the fly, teaching myself how to create useful bash scripts (all my knowledge about sed and awk comes from a local LLM), among other things that are not the business of anyone - that’s why I am using a local model in the first place.
I honestly forgot that i also put a question about training cost; by the question i meant when i asked about uses for ai, i didnt realize i made it unclear whether it was a question or not
I dont think a model that you have said is bad for the environment is a good example
Regarding the first sentence, i have said that this seems like a generic reply because i have not said that llms are the worst thing ever (even saying that their uses could offset the downsides) or that they do not have uses (“AI is just a tool” is also an argument i see so often, never with any explanation behind)
For such impressive technology, i doubt the best use of it is a better search engine that you have to fact check or writing help
I have seen someone use a local model to control home assistant
you see to not have read my responses to the other comments.
my comment here was a bit, trying to replicate the average anti ai stance I think exists.
Oh sorry, i didn’t realize that. OK, so looking at the thread, i would say my reply would fit the inane FUD @[email protected] is spouting. Klear, if you wanna reply, i am open for discussion.
It’s really a shame that in order to stimulate adoption to increase development of LLMs they have to do so much damage first. It’s not unlike with cars, scaling up from the very first ICE models has pretty much doomed us all and even now vehicular transport is still at a net loss.
By the time LLMs or even AI become efficient enough that they can run on the more and more advanced tech of handheld devices (maybe quantum?), it’ll have done so much damage, not to mention all those datacenters will need a new purpose. I hope to see it in my lifetime because of advancement life I that ever happens, I’m holding out hope AI will help us do stuff so much more efficiently we can actually reverse all of the damage done.
Either that or it’ll just kill all humans deciding we are the problem.
Don’t do anything, ever. Just don’t.
what’s the argument against locally hosted AI? lile seriously man I have issues with plagiarism, innacurate information and social dependency, but even with all that AI isn’t of the devil. Learn to have nuance
local models don’t appear out of the ether. using a local model solves one problem around privacy of your prompts, does little about consolidation of power, and nothing about the thousands of other problems genai brings.
if your organic coffee is made with slave labor, you still have a massive problem.
and they may not be of the devil, but they definitely are of the humans who most resemble the devil.
I don’t follow, how does running a local model consolidate power? Geniune question
re-read, it says the opposite: helps a little.
although, tbh, i’m not entirely sure of that either, but the reasoning is complex and longer-term.
Oh I see
Do you live in the environment, like the rest of us? I’ve got some bad news about AI.
Someone self hosting an AI is doing less environmental damage than a good video game play session, though.
The models you are self-hosting first have to be trained at the data centers, which is a contributing factor to the environmental impact of AI.
I know how to run local models, but just because they’re more environmentally friendly than alternatives doesn’t mean they’re not still contributing to the broader environmental impact of data centers.
How does running a local LLM that’s already been trained contribute to the environmental costs of training? It’s already trained whether I run it or not, and running it locally isn’t generating any revenue for the company that did the training (therefore it doesn’t encourage them to train more). This isn’t like a physical product where every time someone obtains a copy, more environmental damage has to be done manufacturing it, training is a one time cost.
The environmental impact of running a local LLM is significantly lower than buying a half-gallon carton of almond milk.
Sure, but you could also make the argument that anything based on networked computing is contributing to the spread of data centers. AI is a large part of that today, but it’s not like data centers never existed prior to ChatGPT. Before, it was about trying to convince laypeople to avoid “the cloud”.
Non-AI datacenters aren’t massive fuckoff Complexes that spew pollution (noise and atmospheric) into the surrounding area and triple your electric bill. It’s the massive hyperscale AI datacenters that are the problem.
They still are, and it’s been known and talked about for years. All of these predate the AI boom:
https://www.vice.com/en/article/data-centres-dirty-energy-secret/
https://web.archive.org/web/20120927113253/https://www.nytimes.com/2012/09/23/technology/data-centers-waste-vast-amounts-of-energy-belying-industry-image.html
https://web.archive.org/web/20190821212342/https://www.azcentral.com/story/news/local/chandler/2018/04/15/chandler-tech-company-hums-along-and-noise-annoying-residents/504641002/
That’s true, loads of non-AI datacenters are also environmentally destructive noise pollution machines, which also needs to be dealt with, but hyperscale AI datacenters are much more environmentally destructive much more of the time. You’re right tho
Server farms before AI weren’t growimg out of control and spinning up gas yurbines everywhere. That’s the boom and bust of AI training.
Depends on a lot of factors, I would imagine.
I was trynna make a bit about it because peeps on here are so furiously against anything ML by now that I’m sure this is the stance.
i dont actually take this stance.
unironically, being mean to inanimate objects is still being mean. i don’t want to imply lojcs was doing that, i just enjoy showing kindness to silly little things and thoughts and characters. sometimes i give our computer headpats when it does a good job, hehe :3
oh yeye, i mean - samsies totally. when my blender render finishes and my lappytoppy is all exhausted from the rendering work, i tell it that it did so well for me <3
I hadn’t, that’s why it seemed strange.
What obliges you to tell me what I can do with my own computer?
Nobody is telling you what you can do, just what you shouldn’t. If you can’t understand the difference, that’s on you.
I don’t see a ‘should’? That’s a straight command. And even if it were there, is it any more reasonable to dictate what people should not do with their computers?
Because you’re an earthling, just like all of us, and LMs are threatening earth, and it is your duty to not use them, especially not for stupid memes. You must stop using LMs on your own computer or anywhere. I am telling you that as your sister on this planet that we both inhabit.
Love you
Ok stow away the hippie crap for a moment. The difference in scale between someone using their personal hardware to run models and the shit the hyperscalers are doing is not even fathomable.
My GPU uses a max of 250W if under full load, which it uses for a few seconds while running a simple request. A single Blackwell blows it out of the water, but those are not provisioned as single boards. They are provided in large racks, containing more CPUs, GPUs, RAM and Storage than i have ever possessed in my life, and i am at it since the early 90s. The difference in resource usage between those 2 scenarios isn’t even measurable because it is simply bonkers.
A individual running a language model on a single 16gb GPU causes orders of magnitude less environmental damage than buying a carton of almond milk. Please direct your anger where it actually belongs, at the goddamn capitalist vampires building hyperscale datacenters.
I honestly can’t tell if you’re joking at my expense or if you’re serious. This thread started to feel like Hexbear for some reason and it makes me want to cry
If you’re joking know that this is a totally believable sentiment for somebody on Lemmy to have.
If you’re serious, then you’re delusional. LLMs themselves can’t threathen anything, they’re not alive. You might be thinking about their power usage or the sentiments of the companies that make them, but none of that is a factor when using them locally. The computer uses the same power regardless of if it’s used to play games or generate text.
Honestly fuck this platform. It is insufferable to post something I had found amusing just to be scolded for not following the Lemmy hivemind. If anyone knows a platform that’s not corporate and doesn’t have such a kneejerk hatred of llms dm me. Goodbye
Lmao if people not wanting to see a user support an industry that’s killing the planet at an alarming scale in such a short time makes you want to leave, then I guess good riddance.
“see if I make weapons at HOME with the gun-O-matic that Lockheed stole and killed so much to produce, I’m not supporting the military industrial complex!”
It’s more about which instance you’re posting in. dbzer0.com for example has far less issues with LLMs and neural networks when run locally - those are practically embraced in the respective communities, while the hyperscalers attempts are loathed as they should be.
Sadly, people conflate local neural networks with the crap they are getting pushed by MS/Meta/OpenAI/Anthropic. Most of them don’t realize the difference in scale between running a local model on a GPU with 16GB VRam and 250W TDP vs even a single rack of Blackwells. They neither have the technical knowledge to separate the two things from another, nor the political knowledge to realize that local NNs empower, while centralized NNs disenfranchise.
You shouldn’t use your computer to use local models out of reach of the Hyperscalers? You shouldn’t use your single GPU with 16 GB Vram because hyperscalers are building datacenters that can’t be used for anything else, using evaporative cooling because republican politicians just ignore their constituents, increasing everyones power bills while erecting gas turbines poisoning the area for a model you don’t have any control over?
Please try to separate local models and the hyperscalers bullshit. Local models stay under local control, don’t need racks full of blackwells or vera rubin, don’t need datacenters at all. All they need is a consumer grade GPU with enough Vram, which would be cheap as fuck if the hyperscalers wouldnt buy up every single last bit of silicon. They are cooled by 1-2 fans and an aluminium block, or if you are getting fancy using a closed loop watercooling solution.
Local LLM’s are not comparable in resource expension at all - they are a small blip, but that blip is at least under control of the person who decided to be independent from all this shit.
nothin, I was trynna make a bit about the general opinion on balahj lemmy, that is: ML bad no matter what.
its a bit, I dont actually take that stance.
Thank you you saved my sanity