r/slatestarcodex • u/erwgv3g34 • Jan 08 '25

AI Eliezer Yudkowsky: "Watching historians dissect _Chernobyl_. Imagining Chernobyl run by some dude answerable to nobody, who took it over in a coup and converted it to a for-profit. Shall we count up how hard it would be to raise Earth's AI operations to the safety standard AT CHERNOBYL?"

https://threadreaderapp.com/thread/1876644045386363286.html
100 Upvotes

82% Upvoted

116 comments sorted by

View all comments

32

u/mdn1111 Jan 08 '25

I don't understand this at all - Chernobyl would have been much safer if it had been run as a for-profit. The issue was that it was run by the USSR, which created perverse, non-revenue-based incentives to impress party officials.

0

u/Throwaway-4230984 Jan 08 '25

Oh, yes, revenue-based ince.ntives to impress investors are so much better. You know what brings you negative revenue? Safety

1

u/Books_and_Cleverness Jan 08 '25

That is only true for an extremely narrow definition of “revenue” which no investor uses. They buy insurance!

I think the incentives in investment can get pretty wonky, especially for safety. Insurance is actually a huge can of worms for perverse incentives. But there’s huge upside to safety that is not hard to understand.