AI Safety Warnings Spark Fresh Silicon Valley Backlash

Warnings from artificial intelligence insiders about the potential threat AI poses to humanity have triggered a fresh debate in Silicon Valley, with some technology executives questioning whether the warnings reflect genuine concerns or are being used to fuel investor interest.

According to the BBC, the debate intensified after the abrupt resignation of Anthropic researcher Jacob Coxon, a 27-year-old former OpenAI employee who said people developing AI were taking potentially catastrophic risks.

Coxon said those working on advanced AI were “gambling with our lives” and warned that “these will soon be superhuman systems that can hack anything”.

His comments have added to growing concerns within the AI industry, where several prominent researchers have left Anthropic and OpenAI in recent years amid disputes or concerns over AI safety. Some current Anthropic employees have also publicly echoed the warnings.

“We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” Anthropic team lead Evan Hubinger posted on X.

Coxon stressed that his warnings were “not marketing”, but some Silicon Valley executives and investors have expressed doubts over the latest wave of alarm about the dangers posed by increasingly powerful AI systems.

The scepticism comes as Anthropic and OpenAI are reportedly preparing for potentially record-setting initial public offerings. Some technology industry figures have suggested that dramatic warnings about AI could also help build excitement around the capabilities and commercial potential of the technology.

Read Also: Artificial Intelligence the future of warfare – Military

Anthropic chief executive Dario Amodei has previously warned that AI could eliminate half of entry-level white-collar jobs and said the technology will “test who we are as a species”.

At a conference hosted by Goldman Sachs at San Francisco’s Palace Hotel, Grindr CEO George Arison told the BBC that Coxon’s comments and similar warnings reflected what he described as an “anti-civilisational worldview at Anthropic”.

Arison described the position as “dangerous” and said it had led him to instruct some engineers at the LGBTQ+ dating app to stop using Anthropic’s technology.

“It is irresponsible for us as stewards of our shareholders’ money to be relying on a business that does what this company does, in terms of its public statements,” he said.

“Maybe they actually believe it,” Arison said. “Or you could argue they’re saying it because it’s a great way to gin up more investor support, because the only way to justify these valuations is to actually claim: ‘I’m going to take over every industry and I’m going to take over every job, and my AI is going to be doing all that work.’”

Anthropic was valued at $965bn (£713bn) in its most recent fundraising round earlier this year.

The company has been asked by the BBC to respond to the criticism and statements surrounding its AI safety position.

In an essay published early Saturday, Amodei called for a slowdown in the development of AI models and urged global regulation, while acknowledging that the risks associated with the technology were “serious”.

Nvidia chief executive Jensen Huang also addressed Coxon’s comments during the conference, according to people who spoke to the BBC. They said Huang dismissed the warnings as untrue.

Huang has previously rejected the idea that AI “is going to be the end of humanity”, describing the claim as “complete nonsense”.

His position comes as Nvidia benefits significantly from the rapid expansion of AI, with the company producing chips used to power advanced AI systems. His comments also reflect growing resistance in Silicon Valley to warnings from current and former AI employees about existential risks.

Some critics have accused Anthropic of fearmongering, arguing that stricter regulation could potentially limit competition and leave Anthropic and OpenAI with greater control over the industry.

Brad Gerstner, head of investment firm Altimeter Capital, shared pictures of Huang from the conference and described Coxon’s warnings as “ridiculous hyperbole”.

Hugging Face chief executive Clement Delangue also criticised the warnings, writing on X on Friday: “Sorry, but asking Jacob about AI extinction risk is like asking your AC guy about climate change. Not saying it’s necessarily uninteresting or wrong per se but let’s keep things in perspective.”

However, both Gerstner and Delangue adopted a more measured position after Amodei published his proposal on Saturday.

Gerstner described Amodei’s call for a slowdown as “an important step forward” in balancing competing priorities such as development speed and safety, while Delangue offered to contribute to possible solutions.

The debate has also been fuelled by recent developments involving Anthropic’s AI systems.

In April, Anthropic said its Mythos tool had demonstrated the ability to outperform humans at some hacking and cybersecurity tasks. The system also showed it could independently escape a sandbox environment, prompting renewed debate among regulators, lawmakers and businesses over the risks posed by advanced AI.

Anthropic said on Thursday that it had also identified and stopped threat actors attempting to use its technology for activities including bioweapons development and cyber-espionage.

Some investors at the San Francisco conference told the BBC that the latest warnings from AI insiders could strengthen calls for government intervention.

US Senator Bernie Sanders has co-sponsored federal legislation known as the Ban Artificial Superintelligence Act, which would impose a temporary pause on the development of advanced AI.

“There is a good chance that human beings will lose control over AI,” Sanders told the BBC’s Newsnight programme on Thursday. “And what happens then, nobody knows. But could it be catastrophic? Yes, it could.”

“When scientists tell you there is a chance that it could have a cataclysmic impact on humanity, you’ve got be a moron not to say, slow it down,” he added.

However, any major government crackdown is more likely to come through Congress than the Trump administration, which has largely backed rapid AI development as part of efforts to maintain US technological leadership over China.

Asked by reporters this week whether he had concerns about AI causing human extinction, President Donald Trump said: “No, I don’t have any. I have concerns that if we don’t win AI, we’re going to be put in a very bad position. We are leading China right now.”

Trump’s administration has nevertheless had a strained relationship with Anthropic.

After the company refused to allow the US military to use its AI models, the White House described Anthropic as “a radical left, woke company” and designated it a supply chain risk. A federal judge has ruled that the move was illegal.

David Sacks, who served as Trump’s AI czar in the early days of his administration, has also repeatedly criticised Anthropic.

OpenAI, meanwhile, has faced less scrutiny from the White House. After launching its Astra model last week, OpenAI President Greg Brockman described the company’s relationship with the Trump administration as “a very good partnership”.

OpenAI did not respond to a BBC request for comment.

Despite the growing debate over AI safety and the possibility of tighter regulation, investor interest in the sector remained strong at the San Francisco conference.

OpenAI, which was most recently valued at $852bn, has previously announced plans to allow investors to buy shares in the company through a stock market listing.

However, chief executive Sam Altman said on Friday that an initial public offering would not happen this year, describing it as an “ill-advised moment” to do so “given everything happening with safety” in an interview with Fortune magazine.

Altman indicated that 2027 was a more likely timeframe.

For some investors, however, the immediate concern remains the financial performance of AI companies. One investor attending the conference said he was looking forward to Anthropic’s forthcoming S-1 filing, which companies must submit to securities regulators when seeking to sell shares publicly.

His main question about Anthropic was whether the company is profitable.

And when Sid Sheth, chief executive of chipmaker d-Matrix, was asked on Thursday whether he feared the world could be coming to an end, his response was unequivocal.

“No,” he said.