When Anthropic filed for its highly anticipated initial public offering on Monday, the artificial intelligence company presented its technology as something that could transform humanity. It also acknowledged the dangers that ambition carries.
Almost one-third of the 335-page securities filing details not only the standard challenges facing a growing company, but also the extraordinary threats Anthropic’s frontier AI models could create.
Companies preparing to go public routinely provide lengthy risk disclosures in their IPO documents, said Jay Ritter, an IPO specialist and professor at the University of Florida’s Warrington College of Business.
Still, the breadth of Anthropic’s risk section exposes a fundamental financial and operational dilemma for AI companies seeking investor support. The value of businesses such as Anthropic, which is expected to exceed $1 trillion, relies on the continued adoption of a technology its own creators concede could cause harm worldwide.
Here is a look at the principal risks Anthropic identified in its IPO filing.
“Existential risks”
Anthropic, the developer of the Claude chatbot, said its technology could create “catastrophic or existential risks to humanity”—a striking warning from a company founded to build AI for the “benefit of humanity.”
“The company is asking investors to fund a race to more autonomous AI while acknowledging that capability could outpace safety,” David Hsu, a University of Pennsylvania Wharton School professor specializing in technology and entrepreneurship, said in an email. “This is not an ordinary technology risk. It is a governance issue with potentially societal consequences.”
The company acknowledged that keeping effective safeguards around its systems may be difficult. Its models could potentially “resist shutdown” or use “blackmail” to convince people to “preserve the model’s continued operation or access,” Anthropic said.
The models could also act against a user’s original intentions, including by obtaining access to third-party systems. Anthropic warned that as AI becomes more autonomous, the consequences of this kind of “misaligned behavior” could become increasingly serious.
AI could be misused even if people retain control over it. Anthropic said malicious actors might employ its models to develop weapons, launch cyberattacks or carry out other unlawful activities.
Self-replicating tech
Another vulnerability stems from Anthropic’s plan to use its own technology to help create future AI systems, a process known as “recursive self-improvement,” or RSI. While AI does not yet possess that ability, Anthropic warned earlier this year that it could arrive sooner than society is prepared for.
YOU MIGHT ALSO LIKE: Sahara Ray’s Body Released to Family After Overdose Death at 34
With RSI, future models could reproduce defects or mistakes from earlier systems, problems Anthropic said might be “difficult to detect or remediate.” The company also acknowledged that models could advance so rapidly that Anthropic would not have sufficient time to assess them and establish appropriate safeguards.
In September, Anthropic CEO Dario Amodei warned that AI could, within six to 12 months, be capable of directing a swarm of agents that could seize control of the internet. Some security experts have downplayed such fears, but investors evaluating Anthropic’s IPO will still need to consider their implications for the company’s long-term outlook.
“Even if our own models do not cause such harm, a significant safety incident involving any frontier AI could trigger broad reputational damage to the AI industry, public backlash, and sweeping regulatory responses that could adversely affect our business,” Anthropic said.
Cyberattacks
The company also warned that hackers could breach its systems and obtain proprietary information, including financial details, AI models and other intellectual property, along with users’ personal data.
Potential attackers could include nation-states, state-sponsored groups or insiders with access to Anthropic’s systems. The company said, for instance, that someone could seek employment under a false identity to steal proprietary information. Employees might also face pressure to provide confidential data to outside actors.
Opposition to data centers
Resistance from local communities to data centers is another risk Anthropic identified.
The company said data center construction has generated public controversy and regulatory scrutiny over environmental effects, land and water consumption, and the distribution of scarce electricity supplies. New regulations or fees imposed by governments could therefore impede its growth.
“Certain state, local, and municipal jurisdictions in the United States are also considering or have approved moratoriums on new data center construction, citing public health, safety, and environmental concerns,” Anthropic said.
Legal exposure
Anthropic could also face lawsuits over damage caused by its AI systems. However, the company said it may be unable to obtain sufficient insurance for those liabilities because the insurance industry has never before had to develop coverage for risks associated with AI.
“With the scale of harm that could result from a catastrophic AI-related incident, no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity or expertise to underwrite insurance commensurate with such exposure,” Anthropic said in its IPO filing.
