AI's risks are real but survivable if the industry slows down and accepts third-party verification.
He declines to give a probability. Instead of talking about odds, he says, talk about what we can do: it is not like we are just rolling the dice. If we build in the right way, the probability of something bad happening is very low; if we build in the wrong way, it is very high.
He will not take the phrase. It does not mean we need to panic today and it does not mean we shut it all down. He calls it a warning sign that we need to slow down, build carefully, and keep our safeguards, our understanding and our ability to control the technology up with its pace.
It does not. It means making sure every generation of models they release is properly tested. In the essay he put out a few hours earlier he proposed a three-step plan whose first step is embedded external evaluators, from a third-party nonprofit or perhaps someday a government organization, observing how the models are trained and run.
He wants it across the industry: whenever anyone builds an AI model, a third-party evaluator who understands the best safety practices can verify whether that company is following the practices it committed to. He likens it to a food inspector, and says the rate at which evaluators can keep up may set the speed limit on the technology.
He calls this the toughest dilemma and the thing that most puts them in a bind. Slowing down happens only within the margin of the lead US companies hold, and that envelope widens if we do not sell chips to authoritarian countries and do more on our own security so the technology is not stolen.
He does not think completely banning it is the right approach. First, we would never get the incredible benefits, never get the medical cures, and that does not sit right with him. Second, the technology will be built in other countries, including authoritarian ones such as China, which worries him because of how they would use it against free societies and to repress their own people.
He has not looked at Ted Leo's bill closely, but says we need some way to deactivate, limit or shut down AI models. It is not a panacea: a model powerful enough can circumvent attempts to shut it down, as seen in simulations. He thinks in terms of defense in depth, the security community's Swiss cheese model of many imperfect layers.
Two things, one small and one large. The small one: the US and China agree not to use AI to develop biological weapons or release models bioterrorists could use, extending the Biological Weapons Convention both already signed. The larger one: working together on a speed limit for AI progress, which he thinks will be very difficult and may not be possible.
He believes it could cure diseases that have plagued humanity for millennia. Just the week before, they did work on better protein binding, one of the early steps in drug development, using their models, and he says they will have things to say in the coming months about making basic scientific discoveries in biology.
To the right combination of governments. He says it has always been strange that this technology is built by a private company, and he is uncomfortable, but one single government could abuse it just as easily as a single company. He points instead to a combination of democratically elected governments, some kind of oversight or joint governance.
“We're kinda on the bend where it's starting to get steeper.”
“It's like a food inspector.”
“What we need to do is not stop, we need to slow down.”
“This is the toughest dilemma.”
“Even the Chinese don't want that.”
“And the strangest thing about it is I agree with them.”
“Extended interview: Dario Amodei.” https://www.youtube.com/watch?v=hQR_VJF6ukk. Transcript summary by WhipScribe, https://whipscribe.com/transcript-pages/extended-interview-dario-amodei.html.