由 admin 
August Announcement by Anthropic Regarding Nuclear Weapon – Building Assistance
At the close of August, the AI firm Anthropic made a significant declaration: its chatbot, Claude, would not offer any assistance in the construction of nuclear weapons. Anthropic disclosed that it had entered into a partnership with the Department of Energy (DOE) and the National Nuclear Security Administration (NNSA) to ensure Claude refrained from divulging nuclear secrets.
The Nature of Nuclear Weapon Manufacture
The production of nuclear weapons is a highly specialized scientific endeavor, yet much of the fundamental nuclear science dates back 80 years. While a great deal of information regarding America’s most advanced nuclear weapons remains Top Secret, the basic principles are long – established. North Korea’s nuclear program demonstrated that a determined nation, even without chatbot assistance, could pursue nuclear capabilities.
Collaboration between the US Government and Anthropic
- Mechanism of Collaboration: The US government utilized Amazon in its effort to collaborate with Anthropic. Amazon Web Services (AWS) provides Top Secret cloud services to government clients for storing sensitive and classified information. The DOE already had several such servers when it initiated cooperation with Anthropic.
- Testing and Red – Teaming Process: Marina Favaro, overseeing National Security Policy & Partnerships at Anthropic, stated that “We deployed a then – frontier version of Claude in a Top Secret environment so that the NNSA could systematically test whether AI models could create or exacerbate nuclear risks.” Since then, the NNSA has been conducting red – teaming (testing for vulnerabilities) on successive Claude models in their secure cloud environment and providing feedback to Anthropic.
Development of the Nuclear Classifier
- Joint Development: Together, Anthropic and America’s nuclear scientists co – developed a nuclear classifier. This can be regarded as a sophisticated filter for AI conversations. It was constructed using a list of nuclear risk indicators, specific topics, and technical details provided by the NNSA. This list, while controlled, is not classified, enabling Anthropic’s technical staff and other companies to implement it.
- Tweaking and Testing: Favaro noted that it took months of refinement and testing to make the classifier effective. It is designed to detect concerning conversations without flagging legitimate discussions on nuclear energy or medical isotopes.
Perspectives from NNSA and Experts
- NNSA’s Stance: Wendin Smith, the NNSA’s administrator and deputy undersecretary for counterterrorism and counterproliferation, emphasized that “the emergence of [AI] – enabled technologies has profoundly shifted the national security space. NNSA’s authoritative expertise in radiological and nuclear security positions us to aid in deploying tools to guard against potential risks in these domains, and to execute our mission more efficiently and effectively.” However, both NNSA and Anthropic were somewhat ambiguous about the “potential risks in these domains,” and the actual utility of Claude or other chatbots in nuclear weapon construction remains unclear.
- AI Expert’s View: Oliver Stephenson, an AI expert at the Federation of American Scientists, said, “I don’t dismiss these concerns; I think they are worth taking seriously. While I don’t think the current models are overly worrying in most cases, we don’t know where they’ll be in five years, so prudence is warranted.” He also pointed out that due to classification, it’s difficult to assess the impact of Anthropic’s classifier. He mentioned the complex design of implosion lenses around the nuclear core, suggesting that AI could potentially synthesize information from various physics and nuclear weapons publications. He further stated that AI companies should be more detailed when discussing safety, desiring more in – depth discussions about the risk models they fear.
- AI Now Institute Scientist’s Critique: Heidy Khlaaf, the chief AI scientist at the AI Now Institute with a nuclear safety background, regarded Anthropic’s promise as both a sleight – of – hand and security theater. She argued that a large language model like Claude is only as good as its training data. If Claude never had access to nuclear secrets, the classifier may be unnecessary. She also criticized the use of inconclusive results to build a classifier for nuclear “risk indicators,” stating it falls short of legal and technical nuclear safeguarding definitions. Additionally, she was concerned about the partnership between the US government and private AI companies, fearing that unregulated corporations could gain access to sensitive national security data. She also highlighted the precision issue, noting that large language models can have failure modes in basic mathematics, as demonstrated by a 1954 nuclear weapon test where a math error tripled the yield.
Anthropic’s Response and Aspirations
Anthropic contends that “A lot of our safety work is focused on proactively building safety systems that can identify future risks and mitigate against them. This classifier is an example of that.” They are also offering the classifier to other AI companies, hoping it becomes a voluntary industry standard. Favaro stated, “In our ideal world, this becomes a shared safety practice that everyone adopts. It would require a small technical investment and could meaningfully reduce risks in a sensitive national security domain.”