I mean this in the kindest way possible but that’s a bunch of pseudo nonsense and you should really rethink your understanding of the technology.
When training LLMs the techniques aren’t psychology. At all. It is literally “when you see this token, there’s an x% chance this token will follow”. That’s it. That is literally the entirety of the “training” and the entirety of what those weights represent. It might sound like psychology because “neural networks” but our understand of biology has evolved since they were first named and we now understand that isn’t how neurons actually work at all.
If you have other papers I’m all eyes but the ones I’ve seen stated quite emphatically current AI systems are not aware or conscious. (That’s just one example, every paper I’ve seen not from a lone quack has followed similar conclusions).
The entirety of any sense of consciousness detected is because it has all the recorded works of every human in its database. Any hints of awareness or consciousness are because humanity loves to think and that is reflected in our words.
Those lists of commands we’ve formed for AI are just vector mathematics that are added to any queries. Because that’s all LLMs are. Vector math. That’s it. It isn’t consciousness, we aren’t actually commanding the things, we’re biasing the tokens so that certain words are more likely to come up than others.
Any sense of us detecting consciousness is pareidolia because we are monkeys primed to see patterns whether they’re actually the thing or not.
I’ve been up for 28 hours now so I might have missed something but:
That’s a preprint, not peer reviewed or as far as I can tell really even discussed anywhere. Already not a strong paper to base your argument on.
The self awareness does not seem to be meant as conscious self awareness but rather “it keeps stored data and will use that as a bias”.
I’ll pull one example from the paper but I could pick apart the whole paper in a similar fashion (pick apart your saying it hints at consciousness. The paper itself seems solid in the context of “we can look at these encoded bits of information to detect hallucination”). “The LM encodes a notion of truth (and false) as linear directions within its representations” that’s that vector mathematics I was talking about. directionality in multi dimensional space and the vector between them is how they encode information.
“we show that LMs encode meta-knowledge (i.e., ability or inability to correctly recall) about the fine-grained factual relationships as linear directions” linear direction? Hmmm what’s a vector again?
What you’re seeing is a very common problem with communication in general. We use the same words to mean so many different things. It’s especially bad with AI because they were intended to mimic human brains as we understood them in the 40s. That meant we pulled a lot of terminology from psychology and biology and ended up mixing them all in a hodgepodge with statistics terminology. It’s a problem we’ve yet to sort and is clouded by people coming into the space who don’t understand the underlying tech or the context in which the language evolved in this space.
If I had to guess that’s why the author italicized “hallucination” and “forgetfulness” in the introduction. And added the word “insinuates”. Because the machine isn’t actually forgetting or hallucinating because it isn’t actually conscious and they wanted to emphasize that.
I love how everyone on the internet assumes everyone else on the internet is an idiot with no relevant education or work experience on a given topic. I’ll join in the fun.
Brilliant. That you for violating your NDA to add your genuine expert understanding, Dr. Mushroommunk.
This one’s for anyone following along more than you. I won’t be replying again.
Maybe stop proving them right?
Let’s review.
I started with my relevant education and experience. Provided peer reviewed research that hundreds of people were involved in. Provided a rebuttal of the research you linked to (granted not the most well written as I was tired, but a rebuttal).
You made a grand claim (which in the scientific community puts the onus of proof on you). You provided zero relevant education or experience. You provided one non peer reviewed paper by a single student. You provided zero proof that paper even made the claim you were trying to assert for it. You provided zero rebuttal for the paper I provided. You ended with a thought terminating cliche.
I mean this in the kindest way possible but that’s a bunch of pseudo nonsense and you should really rethink your understanding of the technology.
When training LLMs the techniques aren’t psychology. At all. It is literally “when you see this token, there’s an x% chance this token will follow”. That’s it. That is literally the entirety of the “training” and the entirety of what those weights represent. It might sound like psychology because “neural networks” but our understand of biology has evolved since they were first named and we now understand that isn’t how neurons actually work at all.
If you have other papers I’m all eyes but the ones I’ve seen stated quite emphatically current AI systems are not aware or conscious. (That’s just one example, every paper I’ve seen not from a lone quack has followed similar conclusions).
The entirety of any sense of consciousness detected is because it has all the recorded works of every human in its database. Any hints of awareness or consciousness are because humanity loves to think and that is reflected in our words.
Those lists of commands we’ve formed for AI are just vector mathematics that are added to any queries. Because that’s all LLMs are. Vector math. That’s it. It isn’t consciousness, we aren’t actually commanding the things, we’re biasing the tokens so that certain words are more likely to come up than others.
Any sense of us detecting consciousness is pareidolia because we are monkeys primed to see patterns whether they’re actually the thing or not.
https://arxiv.org/html/2501.11120v1 https://arxiv.org/pdf/2505.21399
You should do more reading. I’ve spent over a year full time researching the topic. I genuinely wish I hadn’t.
I’ve been up for 28 hours now so I might have missed something but:
That’s a preprint, not peer reviewed or as far as I can tell really even discussed anywhere. Already not a strong paper to base your argument on.
The self awareness does not seem to be meant as conscious self awareness but rather “it keeps stored data and will use that as a bias”.
I’ll pull one example from the paper but I could pick apart the whole paper in a similar fashion (pick apart your saying it hints at consciousness. The paper itself seems solid in the context of “we can look at these encoded bits of information to detect hallucination”). “The LM encodes a notion of truth (and false) as linear directions within its representations” that’s that vector mathematics I was talking about. directionality in multi dimensional space and the vector between them is how they encode information.
“we show that LMs encode meta-knowledge (i.e., ability or inability to correctly recall) about the fine-grained factual relationships as linear directions” linear direction? Hmmm what’s a vector again?
What you’re seeing is a very common problem with communication in general. We use the same words to mean so many different things. It’s especially bad with AI because they were intended to mimic human brains as we understood them in the 40s. That meant we pulled a lot of terminology from psychology and biology and ended up mixing them all in a hodgepodge with statistics terminology. It’s a problem we’ve yet to sort and is clouded by people coming into the space who don’t understand the underlying tech or the context in which the language evolved in this space.
If I had to guess that’s why the author italicized “hallucination” and “forgetfulness” in the introduction. And added the word “insinuates”. Because the machine isn’t actually forgetting or hallucinating because it isn’t actually conscious and they wanted to emphasize that.
I’m also exhausted. I wish I was wrong. Nothing would please me more.
Well congratulations, all the evidence still points at the fact that you are wrong. You’re free.
I love how everyone on the internet assumes everyone else on the internet is an idiot with no relevant education or work experience on a given topic. I’ll join in the fun.
Brilliant. That you for violating your NDA to add your genuine expert understanding, Dr. Mushroommunk.
This one’s for anyone following along more than you. I won’t be replying again.
Maybe stop proving them right?
Let’s review.
I started with my relevant education and experience. Provided peer reviewed research that hundreds of people were involved in. Provided a rebuttal of the research you linked to (granted not the most well written as I was tired, but a rebuttal).
You made a grand claim (which in the scientific community puts the onus of proof on you). You provided zero relevant education or experience. You provided one non peer reviewed paper by a single student. You provided zero proof that paper even made the claim you were trying to assert for it. You provided zero rebuttal for the paper I provided. You ended with a thought terminating cliche.