The Pentagon has reportedly asked AI companies to relax restrictions on how their technology is used in warfare, particularly by reducing the need for human oversight in autonomous weapons. According to a report by The Intercept, Anthropic, an AI company, refused this request and consequently lost a major government contract. In contrast, OpenAI agreed to the Pentagon’s more flexible standards. The agreement included language stating that the Department of Defense would receive "OpenAI models that are designed for national security use cases and have minimal refusal rates." This wording suggests the Pentagon is seeking AI systems that rarely, if ever, refuse to carry out orders. A Justice Department lawyer confirmed this language before retracting it a few hours later.
The military operates under a strict hierarchy, where orders are generally followed without question, especially in high-pressure situations. However, soldiers are legally required to refuse orders that violate international law, as outlined in the Department of Defense's Law of War Manual. This obligation, known as the "duty to disobey," was established after the Nuremberg Trials, where the court ruled that individuals cannot claim innocence simply because they followed orders. The tribunal emphasized that moral and legal responsibility for illegal actions cannot be transferred to a higher authority.
Rebecca Crootof, a professor at the University of Richmond School of Law, explained that while the language in the OpenAI-Pentagon agreement might encourage compliance with unlawful orders, "minimal refusal doesn't mean no refusal." She noted that determining whether a military order is illegal in real time can be extremely challenging, even for trained soldiers. Some legal experts argue that issuing illegal orders is a form of abuse, as subordinates may not have the full legal training to identify such directives and could face consequences for making the wrong decision.
Supporters of AI in military applications argue that machines might be better at interpreting and applying the law than human soldiers, as they can be trained on vast amounts of legal data and battlefield scenarios. However, Crootof pointed out that legal interpretation is not a simple matter of right or wrong but is highly dependent on context. The legality of an action often hinges on subtle distinctions that are difficult to encode into AI systems.
Crootof suggested that instead of programming AI to refuse commands it deems illegal, systems could be designed to identify ambiguous situations and flag them for human review or higher authority. This approach would maintain a check on military command while avoiding the delegation of judgment over the use of force to machines. It is unclear whether OpenAI has implemented such safeguards in its "minimal refusal" models. As AI becomes increasingly involved in military decision-making, experts urge the public to demand greater transparency and clarity on how these systems are being used.
Pentagon Pushes for AI with Minimal Refusal in Military Use, Sparks Legal and Ethical Concerns
AI-rewritten from original reportingHow it works
aimilitaryautonomous-weaponsethicspentaonlaw



