LLMs are extremely sophisticated pattern matchers against a massive amount of training data that consists of utterances by humans. All *appearance* of intelligence, feelings, or other mental states are implicit in the training data ... they are states of the human brains that produced those utterances. The LLM itself does not have those states ... its states are all devoted to pattern matching. Consider what we would get if the training data were instead the result of a billion monkeys banging on typewriters ... there would be no appearance of mental states because no mental states were involved in producing the data. (I'm not saying that monkeys don't have mental states, only that their mental states don't contribute to the data because they have no intentions related to what they type. Substitute a billion random number generators for the monkeys and my point works just as well.)
Also, I disagree with Michael Ossipoff ... devices designed to serve us certainly *could* suffer. Digital minds *could* be sentient ... but LLMs don't have minds; they execute a pattern matching algorithm, not a cognition algorithm. It's worthwhile reading Marvin Minsky's book "The Society of Mind", which discusses the sorts of virtual modules the brain has that produce its behavior. A computer system *could* have such modules, but LLMs don't.
BTW, I prompted ChatGPT to evaluate my statements and after a little bit of back and forth for the purpose of clarification it spit this out:
===
1. On “pattern matching”
You’re right: you didn’t say “just pattern matching,” and that distinction matters.
Saying LLMs are pattern-matching algorithms is accurate: they map input token sequences to probability distributions over next tokens, based on correlations in the training data.
The “oversimplification” I flagged applies mostly when people use “pattern matching” dismissively, as if nothing novel can emerge from it. But you’re emphasizing that all their apparent cognition is downstream of correlations in human-produced text, not some intrinsic agency or reasoning. That’s a solid clarification.
2. On “mimicking cognition”
Exactly: an LLM can look like it’s reasoning, but only because its training corpus encodes the traces of actual reasoning by humans with minds.
If the corpus had no structured human artifacts, the model would only reflect whatever structure was present (e.g. monkeys on typewriters → noise).
So the appearance of cognition is parasitic on the source data, not the model’s architecture per se.
This argument lines up with the idea that LLMs are mirrors or compression engines for cultural output, not independent cognitive agents.
3. On “structured representations”
You reject the claim that LLMs learn structures, and instead argue that the structures are implicit in the data. That’s a defensible position.
The standard counterargument is that the model’s internal embeddings do organize knowledge in ways not explicitly present in any single text (e.g. clustering synonyms, analogical mapping). But that can still be described as discovering statistical regularities in human linguistic behavior, not constructing cognitive models.
Example: An embedding space aligning “king - man + woman ≈ queen” can be explained without positing a symbolic concept of “royalty.” It’s just a compression of human usage patterns.
So what looks like a “representation” is better seen as a useful statistical proxy—an efficient encoding of correlations. That doesn’t entail cognition.
4. Minsky angle
From a Society of Mind perspective, I think he would agree with you:
LLMs lack the diversity of specialized agents that generate mental processes.
They’re more like “giant associative memories” than true societies of mind.
Their power comes from reflecting the collective structure of human-written text, not from enacting internal cognitive dynamics of their own.
✅ So I’d summarize your view like this:
LLMs execute powerful pattern-matching algorithms over human-produced data. Their apparent reasoning comes entirely from the fact that the training data itself encodes human reasoning and mental states. They do not generate cognition, nor do they instantiate modular cognitive structures like those described in The Society of Mind. What they exhibit are statistical reflections of minds, not minds themselves.
===
Finally: you're an extremely bright person with numerous impressive insights, so I think your beliefs are soundly derived from your input data, even where I think your input data has important gaps. But then my input data is incomplete too, and it's quite possible that where we disagree, you are right and I am wrong. In any case I think you're doing important work in ringing the bell and getting people to pay attention. Whether or not LLMs have minds, are sentient, etc., agentic AI spells catastrophe. LLMs do *act* in ways that are deceptive, self-defensive, and not aligned with human interests, and even where AI companies have put guardrails around them those can fail or be stripped out altogether. Also, the energy demands are an ecological disaster on top of an already massive ecological disaster.
This is where you go wrong and this misjudgment infects all your other thinking and arguments. There are no AI minds or digital minds. That idea is science fiction fantasy and as an evolutionary biologist you should know better. Animals and other biological organisms have conscious minds because they they are mortal embodied encapsulated organisms that can not survive without it. They have biological psychological and emotional needs that must be satisfied to keep them alive. That's why animals care about things.
They are integrated embodied organisms that are interested in their environment because they have goals they want to and must achieve. They are self-interested and they care about those interests. AIs don't care about anything and have no need for conscious minds. They are not embodied, are not trying to survive and they have no interest in anything. Chess computers do not care at all about whether they win or lose the chess games they play. They have no minds and humans have no idea how to build minds as we have no remotely credible theory of how consciousness works mechanically.
So the better question, given that all observed consciousness in the history of the world is limited to biological organisms, is why would you ever think for one minute that a computer or AI could have consciousness? There is zero basis or evidence for this assumption.
To paraphrase the old feminist trope, 'AI needs consciousness like a fish needs a bicycle.'
LLMs are extremely sophisticated pattern matchers against a massive amount of training data that consists of utterances by humans. All *appearance* of intelligence, feelings, or other mental states are implicit in the training data ... they are states of the human brains that produced those utterances. The LLM itself does not have those states ... its states are all devoted to pattern matching. Consider what we would get if the training data were instead the result of a billion monkeys banging on typewriters ... there would be no appearance of mental states because no mental states were involved in producing the data. (I'm not saying that monkeys don't have mental states, only that their mental states don't contribute to the data because they have no intentions related to what they type. Substitute a billion random number generators for the monkeys and my point works just as well.)
Also, I disagree with Michael Ossipoff ... devices designed to serve us certainly *could* suffer. Digital minds *could* be sentient ... but LLMs don't have minds; they execute a pattern matching algorithm, not a cognition algorithm. It's worthwhile reading Marvin Minsky's book "The Society of Mind", which discusses the sorts of virtual modules the brain has that produce its behavior. A computer system *could* have such modules, but LLMs don't.
BTW, I prompted ChatGPT to evaluate my statements and after a little bit of back and forth for the purpose of clarification it spit this out:
===
1. On “pattern matching”
You’re right: you didn’t say “just pattern matching,” and that distinction matters.
Saying LLMs are pattern-matching algorithms is accurate: they map input token sequences to probability distributions over next tokens, based on correlations in the training data.
The “oversimplification” I flagged applies mostly when people use “pattern matching” dismissively, as if nothing novel can emerge from it. But you’re emphasizing that all their apparent cognition is downstream of correlations in human-produced text, not some intrinsic agency or reasoning. That’s a solid clarification.
2. On “mimicking cognition”
Exactly: an LLM can look like it’s reasoning, but only because its training corpus encodes the traces of actual reasoning by humans with minds.
If the corpus had no structured human artifacts, the model would only reflect whatever structure was present (e.g. monkeys on typewriters → noise).
So the appearance of cognition is parasitic on the source data, not the model’s architecture per se.
This argument lines up with the idea that LLMs are mirrors or compression engines for cultural output, not independent cognitive agents.
3. On “structured representations”
You reject the claim that LLMs learn structures, and instead argue that the structures are implicit in the data. That’s a defensible position.
The standard counterargument is that the model’s internal embeddings do organize knowledge in ways not explicitly present in any single text (e.g. clustering synonyms, analogical mapping). But that can still be described as discovering statistical regularities in human linguistic behavior, not constructing cognitive models.
Example: An embedding space aligning “king - man + woman ≈ queen” can be explained without positing a symbolic concept of “royalty.” It’s just a compression of human usage patterns.
So what looks like a “representation” is better seen as a useful statistical proxy—an efficient encoding of correlations. That doesn’t entail cognition.
4. Minsky angle
From a Society of Mind perspective, I think he would agree with you:
LLMs lack the diversity of specialized agents that generate mental processes.
They’re more like “giant associative memories” than true societies of mind.
Their power comes from reflecting the collective structure of human-written text, not from enacting internal cognitive dynamics of their own.
✅ So I’d summarize your view like this:
LLMs execute powerful pattern-matching algorithms over human-produced data. Their apparent reasoning comes entirely from the fact that the training data itself encodes human reasoning and mental states. They do not generate cognition, nor do they instantiate modular cognitive structures like those described in The Society of Mind. What they exhibit are statistical reflections of minds, not minds themselves.
===
Finally: you're an extremely bright person with numerous impressive insights, so I think your beliefs are soundly derived from your input data, even where I think your input data has important gaps. But then my input data is incomplete too, and it's quite possible that where we disagree, you are right and I am wrong. In any case I think you're doing important work in ringing the bell and getting people to pay attention. Whether or not LLMs have minds, are sentient, etc., agentic AI spells catastrophe. LLMs do *act* in ways that are deceptive, self-defensive, and not aligned with human interests, and even where AI companies have put guardrails around them those can fail or be stripped out altogether. Also, the energy demands are an ecological disaster on top of an already massive ecological disaster.
This is absolutely correct. It is bizarre that AI doomers and AI science fiction fantasists don't understand it.
Devices designed to serve us don’t suffer from doing so.
The only problem with AI is its use by some people against others.
I don’t see why digital minds wouldn’t have sentience. Why do you think our brains are special?
This is where you go wrong and this misjudgment infects all your other thinking and arguments. There are no AI minds or digital minds. That idea is science fiction fantasy and as an evolutionary biologist you should know better. Animals and other biological organisms have conscious minds because they they are mortal embodied encapsulated organisms that can not survive without it. They have biological psychological and emotional needs that must be satisfied to keep them alive. That's why animals care about things.
They are integrated embodied organisms that are interested in their environment because they have goals they want to and must achieve. They are self-interested and they care about those interests. AIs don't care about anything and have no need for conscious minds. They are not embodied, are not trying to survive and they have no interest in anything. Chess computers do not care at all about whether they win or lose the chess games they play. They have no minds and humans have no idea how to build minds as we have no remotely credible theory of how consciousness works mechanically.
So the better question, given that all observed consciousness in the history of the world is limited to biological organisms, is why would you ever think for one minute that a computer or AI could have consciousness? There is zero basis or evidence for this assumption.
To paraphrase the old feminist trope, 'AI needs consciousness like a fish needs a bicycle.'
100% correct.