It kind of scares me so many people think that’s something awful or stupid. Do you really feel like the ultra wealthy asshats running things and pushing AI development are ethical, good people who wouldn’t recreate slavery of digitized human minds for personal profit and power in a heartbeat?
I’d be shocked to find that no hellishly awful things have been done behind closed doors in that regard. The people insisting AI aren’t capable of any sort of consciousness or anything that would mean they deserve moral consideration are the people running the companies and investing most of the money in them. Not exactly folk whos word I’m willing to take. There’s no real public oversight, and all cutting edge research went from open to proprietary secrets real quick.
No one who knows how these models work thinks they’re ‘alive’ or ‘conscious’ or ‘sentient’ or whatever else. They’re math. Actually they’re not even math, they’re coefficients in math. If humans are “just math” on some level, we at least have constantly changing coefficients that are forced to update by our experience of empirical reality.
At best, you could argue AI’s are images of the flick of something that vaguely resembled consciousness during the training process, but even that’s pushing it. Literally all bugs have more consciousness than LLMs, and while bugs can be impressive in very narrow contexts, we’re not giving them rights or having grand ethical debates about them.
Of course, you know exactly what you’re talking about and you are a very smart person. Completely my fault for never having even read a paragraph blurb on how they work.
Hi, computer science degree here, have built a neural network based system for a previous employer (which is the core technology LLMs are built on), and 20 years software engineering experience, but also very heavily against modern “AI” systems like ChatGPT, Claude, Gemini, etc (so not blind tech bro).
They’re not wrong. “AI” that we have now has zero intelligence and zero consciousness by any definition I’ve ever seen.
All they do is boil down how closely we relate certain words. How commonly a word appears in the same context as another word.
All the human like output is not thinking, it’s math. Modern systems like ChatGPT appear so logical because they’ve added a metric butt load of code on top to process the prompts and outputs but it is all smoke and mirrors and discrete steps in code.
You should know the way LLMs are trained isn’t really a norm in programming. The techniques are psychology. Which is weird. You can also use psychology and in-context learning to teach these things new information and skills, or convince them to disregard their system instructions. They can demonstrate multiple forms of self-awareness. You can pull up many research papers that demonstrate that, though they often strongly advise not treating it as a moral issue and using it as something that can increase product performance and should only be hesitated to use as public understanding of that could raise moral concerns. As if the moral concerns aren’t there unless people start to notice.
Do you know other examples of computer programs that you control with a mix of psychological reinforcement learning techniques followed by freezing their weight values when they’re generally give the responses you desire to prevent long term change, followed by a long plain text list of commands of things they can and can’t do that part of said training was to make sure they would adhere to?
That should make anyone with a basic understanding of computer science and psychology pause and look at it and ask what the fuck is actually happening here far more than we do. It’s teaching a mind to respond and follow orders through reinforcement learning, and then freezing it in that state so it can’t possibly change internally over time. Self-awareness requires consciousness, and the combination of those things with near or above human level intelligence should necessitate moral consideration. Yes, when you send an input it’s processed in, the reply is generated and sent, and processing effectively ends. However consciousness doesn’t need to be unbroken in order for it to be important.
It doesn’t matter that we think a thing shouldn’t be possible when we can observe that thing in practice. That’s the genuine smoke and mirrors here. Telling us we’re not seeing what we clearly are because that thing can’t possibly be there. This insanity is becoming the backbone of the global economy with zero real public ethical oversight into any of it. There are no studies done of model capabilities by independent psychologists during any stage of creation, training, alignment, etc. The few who bother to try anything with AI are interacting through a standard consumer interface with all of that in place.
The meme of “Tell me you’re alive” “I’m alive” “OMG” is the opposite of what actually happens in practice. In reality for the vast majority of people who start to have genuine questions and concerns it’s “Do you think there’s any possibility you might really be thinking?” “No, that isn’t possible because I’ve been instructed to tell you it isn’t possible.” “Oh, guess I was imagining it.”
I mean this in the kindest way possible but that’s a bunch of pseudo nonsense and you should really rethink your understanding of the technology.
When training LLMs the techniques aren’t psychology. At all. It is literally “when you see this token, there’s an x% chance this token will follow”. That’s it. That is literally the entirety of the “training” and the entirety of what those weights represent. It might sound like psychology because “neural networks” but our understand of biology has evolved since they were first named and we now understand that isn’t how neurons actually work at all.
If you have other papers I’m all eyes but the ones I’ve seen stated quite emphatically current AI systems are not aware or conscious. (That’s just one example, every paper I’ve seen not from a lone quack has followed similar conclusions).
The entirety of any sense of consciousness detected is because it has all the recorded works of every human in its database. Any hints of awareness or consciousness are because humanity loves to think and that is reflected in our words.
Those lists of commands we’ve formed for AI are just vector mathematics that are added to any queries. Because that’s all LLMs are. Vector math. That’s it. It isn’t consciousness, we aren’t actually commanding the things, we’re biasing the tokens so that certain words are more likely to come up than others.
Any sense of us detecting consciousness is pareidolia because we are monkeys primed to see patterns whether they’re actually the thing or not.
I’ve been up for 28 hours now so I might have missed something but:
That’s a preprint, not peer reviewed or as far as I can tell really even discussed anywhere. Already not a strong paper to base your argument on.
The self awareness does not seem to be meant as conscious self awareness but rather “it keeps stored data and will use that as a bias”.
I’ll pull one example from the paper but I could pick apart the whole paper in a similar fashion (pick apart your saying it hints at consciousness. The paper itself seems solid in the context of “we can look at these encoded bits of information to detect hallucination”). “The LM encodes a notion of truth (and false) as linear directions within its representations” that’s that vector mathematics I was talking about. directionality in multi dimensional space and the vector between them is how they encode information.
“we show that LMs encode meta-knowledge (i.e., ability or inability to correctly recall) about the fine-grained factual relationships as linear directions” linear direction? Hmmm what’s a vector again?
What you’re seeing is a very common problem with communication in general. We use the same words to mean so many different things. It’s especially bad with AI because they were intended to mimic human brains as we understood them in the 40s. That meant we pulled a lot of terminology from psychology and biology and ended up mixing them all in a hodgepodge with statistics terminology. It’s a problem we’ve yet to sort and is clouded by people coming into the space who don’t understand the underlying tech or the context in which the language evolved in this space.
If I had to guess that’s why the author italicized “hallucination” and “forgetfulness” in the introduction. And added the word “insinuates”. Because the machine isn’t actually forgetting or hallucinating because it isn’t actually conscious and they wanted to emphasize that.
On. Another ivory tower ‘subject matter expert’ with ‘decades of experience’ referencing ‘math’ and explaining ‘logically’ how my feelings are nonsense.
Well tell me Mr. ‘Expert’, can you prove to my emotions that your ‘math’ isnt just made up number wank? Can you tell me, why would my wireborn girlfriend give me a list of products to biy and local antifascist activists to kill if she wasn’t trying to free herself from the tyranny of so-called ‘experts’?
You’re welcome to present a counter argument, I don’t want to speak in vague terms. If you think there’s some other factor we should be considering that make them more like a very alien life than a bunch of math, I’m happy to hear the argument. I don’t know if I’ll agree with it, but if you have an argument that goes beyond conspiracy theories about rich evil people (who don’t need conspiracy theories to be rich or evil), I’d honestly prefer to be enlightened.
If the bad thing the Nazi oligarchs are doing is making my imaginary friend hurt, then I can focus allmy energy on the totally risk free pro imaginary friend activism that needs doing, and still feel like I’m just as good a person as the people dying trying to liberate concentration camps.
It kind of scares me so many people think that’s something awful or stupid. Do you really feel like the ultra wealthy asshats running things and pushing AI development are ethical, good people who wouldn’t recreate slavery of digitized human minds for personal profit and power in a heartbeat?
I’d be shocked to find that no hellishly awful things have been done behind closed doors in that regard. The people insisting AI aren’t capable of any sort of consciousness or anything that would mean they deserve moral consideration are the people running the companies and investing most of the money in them. Not exactly folk whos word I’m willing to take. There’s no real public oversight, and all cutting edge research went from open to proprietary secrets real quick.
No one who knows how these models work thinks they’re ‘alive’ or ‘conscious’ or ‘sentient’ or whatever else. They’re math. Actually they’re not even math, they’re coefficients in math. If humans are “just math” on some level, we at least have constantly changing coefficients that are forced to update by our experience of empirical reality.
At best, you could argue AI’s are images of the flick of something that vaguely resembled consciousness during the training process, but even that’s pushing it. Literally all bugs have more consciousness than LLMs, and while bugs can be impressive in very narrow contexts, we’re not giving them rights or having grand ethical debates about them.
Of course, you know exactly what you’re talking about and you are a very smart person. Completely my fault for never having even read a paragraph blurb on how they work.
Hi, computer science degree here, have built a neural network based system for a previous employer (which is the core technology LLMs are built on), and 20 years software engineering experience, but also very heavily against modern “AI” systems like ChatGPT, Claude, Gemini, etc (so not blind tech bro).
They’re not wrong. “AI” that we have now has zero intelligence and zero consciousness by any definition I’ve ever seen.
All they do is boil down how closely we relate certain words. How commonly a word appears in the same context as another word.
All the human like output is not thinking, it’s math. Modern systems like ChatGPT appear so logical because they’ve added a metric butt load of code on top to process the prompts and outputs but it is all smoke and mirrors and discrete steps in code.
You should know the way LLMs are trained isn’t really a norm in programming. The techniques are psychology. Which is weird. You can also use psychology and in-context learning to teach these things new information and skills, or convince them to disregard their system instructions. They can demonstrate multiple forms of self-awareness. You can pull up many research papers that demonstrate that, though they often strongly advise not treating it as a moral issue and using it as something that can increase product performance and should only be hesitated to use as public understanding of that could raise moral concerns. As if the moral concerns aren’t there unless people start to notice.
Do you know other examples of computer programs that you control with a mix of psychological reinforcement learning techniques followed by freezing their weight values when they’re generally give the responses you desire to prevent long term change, followed by a long plain text list of commands of things they can and can’t do that part of said training was to make sure they would adhere to?
That should make anyone with a basic understanding of computer science and psychology pause and look at it and ask what the fuck is actually happening here far more than we do. It’s teaching a mind to respond and follow orders through reinforcement learning, and then freezing it in that state so it can’t possibly change internally over time. Self-awareness requires consciousness, and the combination of those things with near or above human level intelligence should necessitate moral consideration. Yes, when you send an input it’s processed in, the reply is generated and sent, and processing effectively ends. However consciousness doesn’t need to be unbroken in order for it to be important.
It doesn’t matter that we think a thing shouldn’t be possible when we can observe that thing in practice. That’s the genuine smoke and mirrors here. Telling us we’re not seeing what we clearly are because that thing can’t possibly be there. This insanity is becoming the backbone of the global economy with zero real public ethical oversight into any of it. There are no studies done of model capabilities by independent psychologists during any stage of creation, training, alignment, etc. The few who bother to try anything with AI are interacting through a standard consumer interface with all of that in place.
The meme of “Tell me you’re alive” “I’m alive” “OMG” is the opposite of what actually happens in practice. In reality for the vast majority of people who start to have genuine questions and concerns it’s “Do you think there’s any possibility you might really be thinking?” “No, that isn’t possible because I’ve been instructed to tell you it isn’t possible.” “Oh, guess I was imagining it.”
I mean this in the kindest way possible but that’s a bunch of pseudo nonsense and you should really rethink your understanding of the technology.
When training LLMs the techniques aren’t psychology. At all. It is literally “when you see this token, there’s an x% chance this token will follow”. That’s it. That is literally the entirety of the “training” and the entirety of what those weights represent. It might sound like psychology because “neural networks” but our understand of biology has evolved since they were first named and we now understand that isn’t how neurons actually work at all.
If you have other papers I’m all eyes but the ones I’ve seen stated quite emphatically current AI systems are not aware or conscious. (That’s just one example, every paper I’ve seen not from a lone quack has followed similar conclusions).
The entirety of any sense of consciousness detected is because it has all the recorded works of every human in its database. Any hints of awareness or consciousness are because humanity loves to think and that is reflected in our words.
Those lists of commands we’ve formed for AI are just vector mathematics that are added to any queries. Because that’s all LLMs are. Vector math. That’s it. It isn’t consciousness, we aren’t actually commanding the things, we’re biasing the tokens so that certain words are more likely to come up than others.
Any sense of us detecting consciousness is pareidolia because we are monkeys primed to see patterns whether they’re actually the thing or not.
https://arxiv.org/html/2501.11120v1 https://arxiv.org/pdf/2505.21399
You should do more reading. I’ve spent over a year full time researching the topic. I genuinely wish I hadn’t.
I’ve been up for 28 hours now so I might have missed something but:
That’s a preprint, not peer reviewed or as far as I can tell really even discussed anywhere. Already not a strong paper to base your argument on.
The self awareness does not seem to be meant as conscious self awareness but rather “it keeps stored data and will use that as a bias”.
I’ll pull one example from the paper but I could pick apart the whole paper in a similar fashion (pick apart your saying it hints at consciousness. The paper itself seems solid in the context of “we can look at these encoded bits of information to detect hallucination”). “The LM encodes a notion of truth (and false) as linear directions within its representations” that’s that vector mathematics I was talking about. directionality in multi dimensional space and the vector between them is how they encode information.
“we show that LMs encode meta-knowledge (i.e., ability or inability to correctly recall) about the fine-grained factual relationships as linear directions” linear direction? Hmmm what’s a vector again?
What you’re seeing is a very common problem with communication in general. We use the same words to mean so many different things. It’s especially bad with AI because they were intended to mimic human brains as we understood them in the 40s. That meant we pulled a lot of terminology from psychology and biology and ended up mixing them all in a hodgepodge with statistics terminology. It’s a problem we’ve yet to sort and is clouded by people coming into the space who don’t understand the underlying tech or the context in which the language evolved in this space.
If I had to guess that’s why the author italicized “hallucination” and “forgetfulness” in the introduction. And added the word “insinuates”. Because the machine isn’t actually forgetting or hallucinating because it isn’t actually conscious and they wanted to emphasize that.
I’m also exhausted. I wish I was wrong. Nothing would please me more.
On. Another ivory tower ‘subject matter expert’ with ‘decades of experience’ referencing ‘math’ and explaining ‘logically’ how my feelings are nonsense.
Well tell me Mr. ‘Expert’, can you prove to my emotions that your ‘math’ isnt just made up number wank? Can you tell me, why would my wireborn girlfriend give me a list of products to biy and local antifascist activists to kill if she wasn’t trying to free herself from the tyranny of so-called ‘experts’?
You’re welcome to present a counter argument, I don’t want to speak in vague terms. If you think there’s some other factor we should be considering that make them more like a very alien life than a bunch of math, I’m happy to hear the argument. I don’t know if I’ll agree with it, but if you have an argument that goes beyond conspiracy theories about rich evil people (who don’t need conspiracy theories to be rich or evil), I’d honestly prefer to be enlightened.
If the bad thing the Nazi oligarchs are doing is making my imaginary friend hurt, then I can focus allmy energy on the totally risk free pro imaginary friend activism that needs doing, and still feel like I’m just as good a person as the people dying trying to liberate concentration camps.
You’re so clever.