Anthropic proposes a judicial framework to block risky models

Anthropic is circulating a draft artificial intelligence safety bill that the company wants the US Congress to consider, TechCrunch reports, citing Politico. An Anthropic spokesperson confirmed the text's existence to Politico. It is a company-backed proposal, with no adoption established at this stage.

The framework would target a foreseeable and imminent risk that could cause the deaths of more than 50 people or property damage exceeding one billion dollars. The scenarios covered would involve a biological weapon, a model escaping its developer's controls or a cyberattack.

Independent evaluators registered with the US Department of Commerce would examine models. Upon identifying an imminent catastrophic risk, they would have 72 hours to notify the department. The department would then have to ask a judge in Washington to suspend distribution of the model. The draft would prohibit federal agencies from deciding on such a block on their own.

A question remains about the financial independence of evaluations. Labs could pay evaluators, provided that payment did not depend on the outcome. This rule would regulate compensation but is not enough to demonstrate the absence of influence.

The proposed structure would therefore favor judicial intervention following an evaluation. Its effectiveness would depend in particular on the ability to characterize the risk quickly and on the evaluators' actual independence, two points that the available account does not resolve.