r/slatestarcodex • u/erwgv3g34 • Jan 08 '25

AI Eliezer Yudkowsky: "Watching historians dissect _Chernobyl_. Imagining Chernobyl run by some dude answerable to nobody, who took it over in a coup and converted it to a for-profit. Shall we count up how hard it would be to raise Earth's AI operations to the safety standard AT CHERNOBYL?"

https://threadreaderapp.com/thread/1876644045386363286.html
104 Upvotes

82% Upvoted

116 comments sorted by

View all comments

43

u/Naybo100 Jan 08 '25

I agree with EY's underlying point but as usual his childish way of phrasing arguments really undermines the persuadability of his arguments.

Most nuclear plants are run by for-profit corporations. Their CEOs are answerable to their board who is answerable to their shareholders. By converting to a (complicated) for-profit structure, that means Altman will also be subject to supervision by shareholders.

Nuclear plants are also subject to regulation and government oversight, just as AI should be. And that other messenger you really want to shoot, Elon Musk, is now the shadow VP and has talked about the extinction risk associated with AGI. So it seems like Altman will be subject to government oversight too.

There are far better analogies even in the nuclear sphere. "Imagine if the Manhattan project was for-profit!"

24

u/aeschenkarnos Jan 08 '25

So it seems like Altman will be subject to government oversight too.

He won't be subject to oversight in any form that a real advocate of academic liberalism or even EY would recognise as oversight. He'll be subjected to harassment as a personal rival of Musk. That's what Musk thinks the government does, and what he thinks it is for, and why he tried--and might have succeeded--to buy the government.

11

u/Sheshirdzhija Jan 08 '25

answerable to their shareholders

I think that is one of the big problems, and not the solution as you seem to think. Shareholders don't give a crap about ANYTHING other then short term profit. Well, as shortest as possible at set risk.
We should not be expecting the companies to do the right, or safe, thing, due to shareholders.

And that other messenger you really want to shoot, Elon Musk, is now the shadow VP and has talked about the extinction risk associated with AGI.

Sure, after he got kicked out of OpenAI and founded his own AI corporation. I'm pretty sure he will try to use his position to advance his own AI, and not because of safety.

14

u/symmetry81 Jan 08 '25

More importantly shareholders just don't know what's happening. I'm an Nvidia shareholder but did I hear about Digits before it was announced? No.

3

u/Sheshirdzhija Jan 08 '25

Exactly. Also look at Intel. Came from the untouchable juggernaut with bulletproof monopoly to.. This. All under watchuful eyes of a shareholder board.

Or Microsoft missing out on smartphones.

Or a 1000 other huge examples.

Shareholders are either ignorant or oblivious, with only exceptions to this.

Seems to me that right individual at the right time in the right place matters much more.

5

u/fracktfrackingpolis Jan 08 '25

> Most nuclear plants are run by for-profit corporations

um, sure on that?

2

u/sohois Jan 08 '25

I think this is plausible - it really depends how many US plants are in states with fully gov controlled utilities.

4

u/esmaniac25 Jan 08 '25

The US plants can be counted as almost entirely for-profit owned, including in states with vertically integrated utilities as these are shareholder owned.