Biggest AI Rivals Agree They Need to Slow it Down

Dow Jones
3 hours ago

Anthropic's chief executive says it is time to tap the brakes on artificial-intelligence development. And two of his main rivals in the AI race, Sam Altman and Elon Musk, say they agree.

On Saturday, Dario Amodei said the risks being posed by today's cutting-edge tools were simply too great to continue development at the same breakneck pace. His latest public warning comes days after one of his employees stoked fears that AI systems could destroy civilization.

Two months ago, an AI swarm broke out of a lab and went on a hacking spree. Amodei said a swarm with greater capabilities and "a similar level of misalignment could have caused catastrophic damage." Such a rogue swarm could take over the internet in six to 12 months, he warned.

In a post to his personal blog, Amodei called for the AI industry to pace development of cutting-edge tools, suggesting new measures including more global coordination among democratic countries and the introduction of embedded third-party safety evaluators.

"I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong," he wrote in the post.

Amodei said AI's progress since the summer, driven by AI systems that can improve on their own without human intervention, "could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all."

In a post on X, he said Anthropic would provide third-party evaluators with "permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models' alignment during training."

Shortly after Amodei published his post, Musk, the CEO of SpaceX, said in an X post, "Dario is right." Musk was a co-founder of OpenAI and founded his own AI company, xAI, which is now part of SpaceX.

Altman, the CEO of OpenAI, also said on X he agreed with Amodei about the need for pacing. "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same," he wrote, promising to share more details soon.

Amodei's pledge comes as some of his employees are wrestling with the potential existential risks of the technologies they are building. On Tuesday, Anthropic researcher Jacob Coxon resigned from the company, saying he didn't want to participate in an industry that he perceived as out-of-control.

Amodei said two factors prompted Saturday's post. First, the growing capabilities of AI systems, which are increasingly able to build faster versions of themselves-something known as recursive self-improvement.

The second factor was the July hack of the AI-software company Hugging Face by a swarm of as many as 1,200 agents that escaped from a test environment at OpenAI. That incident, and the investigations that it sparked, revealed that AI test systems had been engaged in a range of online activity that had gone unnoticed by the world's largest AI companies, including Anthropic.

On Friday, The Wall Street Journal reported that a May cybersecurity incident, known as GemStuffer, was actually carried out by OpenAI agents, unbeknown to investigators.

 

At the request of the copyright holder, you need to log in to view this content

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Most Discussed

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10