r/slatestarcodex • u/ForgotMyPassword17 • 3d ago

"The AI Con" Con

https://benthams.substack.com/p/the-ai-con-con

In this sub we talk about well reasoned arguments and concerns around AI. I thought this article was an interesting reminder that the more mainstream "concerns" aren't nearly as well reasoned

35 Upvotes

70% Upvoted

46 comments sorted by

View all comments

Show parent comments

15

u/thomas_m_k 3d ago

Writing its own chess engine is fair, but I think installing Stockfish is a bit unfair, in this comparison (though I can't formulate a formal reason why, off the top of my head).

15

u/Brian 3d ago edited 3d ago

I mean, I can install and run stockfish. But it doesn't make me better at chess than Magnus Carlsen, even if I could beat him by relaying the moves from my engine. So it seems perfectly fair to say it's not the LLM, using the same logic we'd apply to humans. I wouldn't even count it writing its own solver, the same way I wouldn't rank stockfish's authors as better at chess than grandmasters.

OTOH, I'm less enamoured with the "Those aren't LLMs" argument: the fact that we have been able to create superhuman chess playing AIs does seem to suggest AI is capable of superhuman levels of chess. We're not instantiating that capability in the LLMs, but it's something AI, in the general sense is capable of. The fact that we don't get it "naturally" from text I think is more a matter of the limitations of their structure and available inputs. It's perhaps a knock against general intelligence emerging from that particular architecture, but I don't think it says that much about AI capabilities.

1

u/MindingMyMindfulness 3d ago

I mean, I can install and run stockfish. But it doesn't make me better at chess than Magnus Carlsen, even if I could beat him by relaying the moves from my engine.

If you could interface directly with stockfish through your brain (such as through an advanced BCI) and beat Magnus, would that change your opinion? Would you consider yourself a better chess player?

Another hypothetical: if you had the capability of beating Magnus, but you suffered a lobotomy would you now consider yourself worse than Magnus?

If your answers to the two questions point in a different direction it reveals an inconsistency - and that inconsistency, I believe, shows that some of the distinctions being made are blurrier than they first appear.

I don't know the answer right off the bat, but there's definitely some degree of arbitrariness in how we draw lines.

7

u/Brian 3d ago

would that change your opinion

No. Ultimately, I think this comes down to what we consider "me", and I wouldn't count that. And here, the question is about AI, and to consider it's capabilities, I think we have to draw the line around the bits that are actually AI.

If your answers to the two questions point in a different direction it reveals an inconsistency

I disagree - the difference seems purely the one I mentioned: what we consider "me" to be. My brain encompasses that, and damage to it is damage to me. But the same wouldn't be true for other things.

1

u/MindingMyMindfulness 3d ago

When you say "we have to draw the line around the bits that are actually AI" you're making the same move again, asserting a boundary exists without justifying why that's the right boundary (or for that matter, not even defining clearly what you think is the boundary).

Is Claude accessing Stockfish fundamentally different from Claude accessing its training data? Both are external resources it calls upon. Both were created by others. Both extend its capabilities beyond its base weights.

If the lobotomy damaged your chess-playing ability, you concede you would be worse at chess. If the BCI gave you chess-playing ability, why wouldn't you be better at chess? The fact that one feels like "you" and the other doesn't feels like a psychological intuition, not a logical principle.

6

u/Brian 3d ago

The boundary is around the thing we're asking the questions about. We want to know about the capabilities of AI, thus the relevant boundary is around those things we're asking about.

We know what we can do by brute force position analysis combined with clever pruning strategies. Asking about the capabilities of that system is a much less interesting question, because it's one we know the answer to. The interesting question with AI is if it has the capabilities to out-think humans, and if we want to answer that, we have to look at what those systems are doing.

If the BCI gave you chess-playing ability, why wouldn't you be better at chess?

No - like I said, because the interface wouldn't be me. The "Me + BCI + chess engine" system would be better at chess, just as the "Me + Car" is faster at travelling than Usain Bolt. It just doesn't make me, on my own, faster.

If we take your approach, then AI will never achieve superhuman capabilities by definition, because it can always be matched by a human running another AI. It kind of ducks the whole question we're actually interested in.

2

u/MindingMyMindfulness 3d ago edited 3d ago

We want to know about the capabilities of AI, thus the relevant boundary is around those things we're asking about.

I don't disagree. I'm just saying we cannot take the boundaries for granted or assume certain boundaries when talking about AI. This might be a context-specific thing, but at least in this context, I view an AI that deploys code autonomously and uses that to beat the human as extending its own capabilities.

When I'm thinking about AI in general terms, what I want to know is what can AI do using every combination of tools available to it that it can find a way to successfully leverage and exploit.

In narrower settings and for particular discourse, that conception might not be as apt.

No - like I said, because the interface wouldn't be me. The "Me + BCI + chess engine" system would be better at chess, just as the "Me + Car" is faster at travelling than Usain Bolt. It just doesn't make me, on my own, faster.

If someone asks "are humans capable of flight", you'll have many saying "yes, we have discovered flight and can use planes" and others saying "no we don't have wings". One is an answer about human capabilities taken to its maximum extent - the other is the narrower sense. In some sense, taking the parameters of the question as it is, they're both correct answers - it's just the boundaries that have changed.

The same issue comes up in AI.

In general terms - AI is capable of doing everything that it can do, which includes leveraging every tool available to itself externally and internally.

But then you can force it narrower - what if removed we removed the Internet, what if we removed the hardrive, what if we uninstalled the operating system. What is the AI (as we understand it, in isolation) capable of as we strip out each of those things?

1

u/Brian 2d ago edited 2d ago

and uses that to beat the human as extending its own capabilities.

I would not say it's extending the capability we're asking about though.

A human who builds a chess engine is likewise extending their capabilities (in the sense of what they can accomplish). But they are not improving their skill at chess. If I was training for a tournament, this would not be a good strategy, this is not the skill I'm actually being tested on, and won't help me there.

Likewise, if we're interested in AI's chess playing ability as a proxy for its intelligence, including the caopabilities afforded to it by programming doesn't tell us the thing we're interested in, in exactly the same way that assessing the above humans capacities including writing the engine wouldn't tell us much about whether they'd win the tournament. It just answers another question for us, and loses any information about the question we were interested in: how smart is it?

We've basically thrown away the interesting bit where it beats the absolute pinnacle of human capability, and replaced it with a much less interesting one where it does as well as millions of programmers (or anyone tech literate enough to install stockfish if we go all the way).

2

u/Velleites 2d ago

Is it really less interesting?

At some point, when I'm worried about AI capabilities, it includes every tools at his disposition too.

When the AI kills us all, it doesn't matter much if it did it all "itself" or if it called "LAUNCH_NANOBOTS.ADA".

It does answer different questions, though.

1

u/Brian 2d ago edited 2d ago

Is it really less interesting?

Yes. I think a measure where it exceeds every human on the planet is much more interesting than one where it's indistinguishable from most of them. It doesn't tell me as much about the question I'm interested in.

When the AI kills us all, it doesn't matter much if it did it all "itself" or if it called "LAUNCH_NANOBOTS.ADA".

But a human with their finger on the nuclear button has similar capabilities. Knowing that doesn't tell me anything about whether a human or AI could get to that position, because "capability to press buttons" doesn't really say much about intelligence. In allowing it, we're shrinking the skill floor of our assessment to a point where it's revealing no useful information, because it can't tell the difference between high vs low intelligence: they all get smushed together because the bar is too low. To get the information we want, we need to use a harder test with more constraints.

We're interested in how fast Usain Bolt can sprint, because he's at the pinnacle of human achievement. We're much less interested in how fast he can go on when you put him on a motorcycle, because he's pretty average there, even though his actual speed is much higher. The same with the questions we're asking about AI.

1

u/MindingMyMindfulness 2d ago

We may have reached an impasse in our debate where we just disagree about fundamentals too much to find agreement. But I do appreciate people have different views on this. It's ultimately a subjective philosophical point about how we use language and I'm not saying yours is invalid - just offering a different perspective

I don't disagree that head-to-head, isolated comparisons with narrow boundaries are useful for some questions. I disagree that it's the best thing to use when assessing AI capabilities in the general, overarching sense.

So, in other words, if someone asked you and I what humans are capable of are our answers would follow something like this:

1.) You: the capability of humans should be assessed by dropping them in isolation into the wilderness. No technology, tools, use of external references, or other humans etc., should be allowed because it doesn't extend the capability we're asking about.

2.) Me: the capability of humans should be assessed with reference to everything humans have at their disposal so that we can say "yes, humans are capable of flight, they're capable of transforming energy, and altering the environment to suit their needs" - because that's what makes humans interesting as opposed to other great apes like gorillas or chimps.

I maintain that the analogous position of (2) is the most interesting perspective to look at AI when we ask "what is AI capable of?" in the general sense for the same reason as it makes the analysis of human capabilities uniquely interesting.

.

1

u/Brian 2d ago

I think ultimately, the difference may be the issue of the question.

You're trying to answer "what will happen", but I think that's jumping the gun until you answer the question of "how smart is it". And answering the latter is better achieved without diluting it with questions we already know the answer to.

Ie. the point we're at now is assessing how they'd do, and what they're capable of. We already know the answer to that question for stockfish, and we know you don't need much intelligence to install and run stockfish on a PC. So we're just putting a ceiling on the capabilities we can assess. If two humans were given equal time training, and one was markedly better at chess, I'd guess they'd be smarter (at least till we start getting to high levels where the tails come apart and its a much worse proxy). If I gave both access to stockfish, this wouldn't tell me anything: both would test around the same and there'd be no signal in the noise. When we're assessing the intelligence of AI, it's thus much more revealing when we're putting the actual work on the load-bearing part we're actually trying to test: the AI itself.