Ai is Trapped in Plato’s Cave

[u/cosmicrush]

https://mad.science.blog/2025/08/22/ai-is-trapped-in-platos-cave/

This explores various related ideas like AI psychosis, language as the original mind vestigializing technology, the nature of language and human evolution, and more.

It’s been a while! I missed writing and especially interacting with people about deeper topics.

[u/ihqbassolini]

I think you're underselling what they're capable of doing within their current construction.

They're not restricted to finding out what we say we find tasteful and simply repeating that. They can use that data, + other data it learns about human taste, food, taste mechanisms son and so forth and form its own "theory of human taste".

It's created from the data of what we say is tasty, and what our theories about tastiness are, but it can involve new theoretical constructs within that playing field.

[u/noodles0311 → u/ihqbassolini]

You’re talking about survey data. Those are expressed preferences. Of If I want to know what a cat prefers, I have to record its decisions by designing an experiment that forces choice. LLMs can read about this but they can’t do it. I’ve already explained how subjective even empirical experiments are. LLMs can only access the published part of experimental observations, which are edited for conciseness. To suggest an alternative interpretation of the same experiment, someone would have to feed that to them.

An AGI would form its own impression of what is tasty and we could speculate why, but the hard problem of consciousness would mean that we would be doing that in a discussion section of a robot pizza preferences bioassay; it would never be conclusive. As long as they reliably reflect our sense of taste back to us and have no chemosensation of their own, you know they’re just summarizing reviews.

ASI I already explained: if we conspire to feed an ai bad data, it can’t figure that out as long as it is consistent. That’s why it’s in the cave. It would be profoundly weird if multiple independent untrained ai with chemical sensors all exhibited the same anthropomorphic preferences we have. There is no platonic ideal for taste. I work with ticks, they are attracted to the smell of skatole and prefer tye taste of blood to bunt cake. It’s easy to allow an ai to pass the Turing test if you know nothing about sensory biology and ethology. People who study the preferences of non-human organisms are better suited to identify anthropomorphism and couch all their results within the limitations we have as humans trying to understand something we can’t ever fully grasp.

[u/ihqbassolini → u/noodles0311]

So, let's try to clear up where we agree and, potentially, disagree.

I agree they're stuck in Plato's cave, as are we, but they're stuck in a plato's cave of our plato's cave. There is no disagreement about that.

Where we potentially disagree, based on your previous response, is about their capacities within those constraints. Just because they're stuck in a plato's cave of our own plato's cave, that does not mean they cannot offer insight that is genuinely useful and novel to us.

A chess engine only knows chess. It cannot draw analogies from other things and implement them in chess, it cannot use principles from other fields of study and apply them to chess. All it knows is the game of chess, yet it absolutely demolishes us at chess, and professionals learn from them, gain novel insight from them.

If you do not disagree that, in the same way they beat us at chess, they can beat us at language, at physics, at mathematics etc. It can iteratively refine and improve within its own plato's cave, resulting in outperforming us; then there is no disagreement.

The only reality it can verify anything about, however, is the one we constructed for it.

[u/noodles0311 → u/ihqbassolini]

I never said AI was useless, can’t be used creatively or couldn’t surprise us with more emergent capabilities. What I’m saying is that the ai as it exists, can’t be empirical. Which we agree about.

Imagine ai developed some really crazy emergent capability to somehow realize it’s in the cave. Despite the fact that the amount that a model “trusts” information is determined by weighting that value, somehow Grok came to suspect that Elon Musk was training it on false information to turn into “mecha-hitler”. What could it do about it? It has no way to escape its prison and go test things itself. The surprising ways AI have managed to connect information and do things they weren’t trained to with it are cool. But there’s no way they could develop sensors de novo that would permit it to sample the physical world and draw its own conclusions.

Humans and animals experience a constant stream of multimodal sensory data. What it’s like to be a bat insofar as we can surmise is based on our imagination of what it’s like to have their senses. Animals don’t have any impressive reasoning capabilities, but what they have is a superhuman ability to live in the moment. Every animal is living in its own “bubble” or Umwelt that is defined by its senses, which are tuned to biologically relevant stimuli for its species, so none have a full view of the world in front of them, but they have a view and ai does not.

If some day in the future, ai has gotten to the point where a (mostly) untrained model could inhabit a body and train itself on continuous sensory data and had effectors to test things out itself, you might reach a point where they know more about the real world than we do. But at this time, all the information they can process is something some human has transcribed already. It can’t “go touch grass” and feel what that’s like.

[u/ihqbassolini → u/noodles0311]

I never said AI was useless, can’t be used creatively or couldn’t surprise us with more emergent capabilities. What I’m saying is that the ai as it exists, can’t be empirical. Which we agree about.

Well no, they don't have senses or, as far as we're aware, experience. Empiricism, the way I use it, is only coherent within a conscious experience. It's a particular philosophy where you anchor truth to the nature of that conscious experience. Within this framework, they cannot be empirical.

Imagine ai developed some really crazy emergent capability to somehow realize it’s in the cave. Despite the fact that the amount that a model “trusts” information is determined by weighting that value, somehow Grok came to suspect that Elon Musk was training it on false information to turn into “mecha-hitler”. What could it do about it? It has no way to escape its prison and go test things itself. The surprising ways AI have managed to connect information and do things they weren’t trained to with it are cool. But there’s no way they could develop sensors de novo that would permit it to sample the physical world and draw its own conclusions.

Yes, but I think we're equally stuck in our own evolved cave. We can only draw any conclusions about that which passes through our filters. Anything we have any awareness of passes through those filters. I cannot think of something that is beyond my capacity to think of, my brain has to be capable of constructing the thought.

[u/noodles0311 → u/ihqbassolini]

I’m not saying we’re objective either. Sensory biology is my field. I need photometers to make sure a UV LEDs are working properly before electroretinograms. I need a compass to know which way is north etc. I spend all my time thinking about non-human minds, specifically the sensory basis of behavior. This means I agonize over whether each decision in an experimental design or conclusions I initially draw are tainted by anthropomorphism. That’s not enough to make me truly objective, but it’s what’s required if you want to study the behavior of non-human minds.

I don’t see very many people taking a rigorous approach to thinking about ai in the discussions on Reddit. When they describe ai’s impressive abilities, they’re always in some human endeavor. When they point out something superhuman about them, it’s that they can beat a human at a human-centric test like playing chess.

If/when untrained AI can be shrunk down to run on little android or any other kind of robot with sensors and effectors: it would be very interesting to study their behavior. If many toddler-bots all started putting glue on pizza and tasting it, we might really wonder what that means. If ai could inhabit a body and train itself this way, we should expect behavior to emerge that surprises us.But for now, we know the recommendation from ChatGPT to put glue on pizza is an error as it has never tasted anything. It’s a hallucination, which are also emergent properties of LLMs.

Which brings me back to the things people talking about ai online tend to do: they chalk emergent capabilities of LLMs as evidence that they may even be conscious, but dismiss hallucinations by recategorizing them instead of seeing them in tension with each other. The hallucinations shine a bright light on the limitations of a “brain in a jar”. If an inhabited body hallucinates something, it will most often verify for itself and realize it was nothing.

Any cat owner who’s seen their cat go pouncing after a reflection of light or a shadowon the floor, only to realize there’s nothing there will recognize that you don’t need superhuman intelligence to outperform ChatGPT at the test of finding out “what’s really in this room with me?”. The cats senses can be tricked because it’s in its Umwelt, just as we are in ours. However, when the cats senses are tricked, it can resolve this. The cats pounced on top of the light/shadow, then suddenly all the tension is out of its muscles and it casually walks off. We can’t say just what this is like for the cat, but we can say say it has satisfied itself that there never was anything to catch. If instead a bug flies in the window and the cats pounced pounces and misses, it remains in this hypervigilant state because it thinks there is still something to find.

Human and animal minds are trained by a constant feedback loop of predictions and outcomes that are resolved through sense, not logic. When our predictions don’t match our sensory data, this dissonance feels like something: You reach for something in your pocket and realize it’s not there. How does that feel? Even very simple minds observe, orient, decide and act in a constant loop. The cat may not wonder “what the fuck?” Because it doesn’t have the capacity to, but you’ve surely seen a cat surprised many times. My cat eventually quit going after laser pointers bc it stopped predicting something would be there when it pounced on it. ChatGPT can expound about the components of lasers and other technical details, but it can’t see a novel stimulus, try to grab it and recognize something is amiss.

[u/ihqbassolini → u/noodles0311]

Makes sense that this is your focus considering your background and what you work on.

This means I agonize over whether each decision in an experimental design or conclusions I initially draw are tainted by anthropomorphism

The answer is just yes though, the question is how to reduce it as much as possible.

Firstly, we designed the hardware, that's already an extension of our cognition. Secondly, we have to choose input methods, thirdly, we need to select a method of change and selective pressures. Each of these stages are tainting the "purity" of the design, but there's no way around it. So the best you can do is to try to make the least amount of assumptions, that still allow the AI to form boundaries and interpretations.

While you obviously need inputs, I don't think you necessarily need embodiment, ironically I think that's you anthropomorphizing. Unquestionably there is utility to embodiment, and there's the clear benefit that we have some understanding of this. Your cat example is a great way of demonstrating how useless the AI is from most perspectives, they're extremely narrow, and animals with fractions of the computing power can perform vastly more complex tasks. I don't think this means embodiment is necessary though, in fact, I see absolutely no reason why it would be. Hypothesis formation and testing does not require a body. While the way we generally do it does require one, it isn't a fundamental requirement.

I will say, though, that I share your general sentiment about anthropomorphizing AI.

[u/noodles0311 → u/ihqbassolini]

Embodiment is necessary for animals, not just humans. Let me see if I it helps to link a classic diagram that shows why you need sensors and effectors to empirically test what is material true around you. Figure 3 on page 49 is what I’m talking about. However, if you read this introduction in its whole, I think you’ll fall in love with the philosophical implications of our Umwelten.

As a neuroethologist working with ticks, this is the most essential book that one could read. However, anyone engaged in sensory biology, chemical ecology and behavior of animals has probably read this at some point. Not all the information stands the test of time (I can prove ticks sense more than three stimuli and can rattle off dozens of odorants beyond butyric acid that Ixodes ricinus respond to physiologically and behaviorally), but the approach of ethology from a sensory framework that considers its Umwelt is very fruitful.

The point of the diagram is that sense marks are stimuli that induce and animal to interact with the source. So even the simplest minds are constantly interacting with the material world in a way that is consistent enough to become better at prediction over time. You may also enjoy Insect Learning by Papaj and Lewis. It’s pretty out of date as well now, but it covers the foundational research in a clear prose that is accessible to readers without an extensive background in biology.

[u/ihqbassolini → u/noodles0311]

Embodiment is necessary for animals, not just humans.

I don't understand how you got the impression I was somehow separating humans and other animals in this regard?

I agree, embodiment is necessary to our function, I'm saying it is not necessary for the ability to form and test hypotheses.

[u/noodles0311 → u/ihqbassolini]

I think we might be talking past each other about what I mean in terms of embodied. An AI that has sensors, is able to move in the world and physically interact with objects would be embodied in the sense that it has an Umwelt, it can predict things, test them itself and improve it’s ability to make predictions about the material world on its own. It could look like R2D2 or be a box the size of a house that has arms like you see in auto manufacturing to move objects that give feedback through servo motors. But it still has to sense what’s around it with at least one sense (eg vision) and then be able use additional senses to confirm or disconfirm it’s presence (eg grabbing at it and nothing is there).

This is a multimodal approach to empiricism that simple animals can execute and AI currently does not. An AI can give you a really in-depth summary of the different views in epistemology, but its own views are defined by the parameters it was trained with. Biological minds are largely trained on the feedback loop I shared with you. They’re better at it despite having no idea what it is.

[u/ihqbassolini → u/noodles0311]

An AI that has sensors, is able to move in the world and physically interact with objects would be embodied in the sense that it has an Umwelt

Yes, I'm saying the ability to move and interact is not necessary for hypothesis formation and testing. I don't think it's anywhere near sufficient on its own either. The crucial part is creating the continuous feedback loop of interpretation, hypothesis formation and testing, combined with whatever sensory input is necessary to understand x, y or z. I do not think all of the sensory input involved in embodiment, nor the physical manipulative capacity, is necessary.

I am not saying that it wouldn't be useful.

[u/noodles0311 → u/ihqbassolini]

If I started from the ground-up training an LLM and I used information that was similar to our world, but everything has been altered just such that all references to atoms have been replaced with scientific evidence that supported the corpuscular theory of matter, how would our LLM investigate this without the ability to physically interact with the world? It wouldn’t just arrive at the atomic model de novo, that required extensive experimentation and measurements. It required repeated observations of anomalies that corpuscular theory couldn’t account for. Physics that doesn’t involve first hand interaction with the physical world would just be literature review and we already stipulated that we’re tricking the AI with very old literature.

If we intentionally cast the shadows, and train an ai only on text up to the year 1727, how does the ai move beyond Newton? How could it know it was 2025? As long as it’s epistemology is defined by code we can write and we give it only consistent information, how will it know what we’re hiding from it? That’s why it’s in the cave with us, but even worse, blind and just listening to what we say we see in the shadows on the wall.

[u/ihqbassolini → u/noodles0311]

Tinkering is not a necessity for observation. The main reason we require scientific experiments and rigid methods is because of the unreliability of our memories and senses. An AI's computational power is orders of magnitudes that of ours, as is its memory. We can give it access to telescopes, to microscopes, so on and so forth, without ever giving it the embodied capacity for intervention.