A dispute between a leading artificial intelligence (AI) company and the US government highlights an issue that has been building in the tech industry for some time: who decides how powerful technologies are used, and, just as importantly, how they are not?

That dispute is now playing out between Anthropic and the US Department of Defence. The Pentagon has asked Anthropic – and other leading AI firms – to make its systems available for ‘all lawful purposes’. Anthropic has resisted strongly, arguing that legality alone is insufficient as a safeguard. In particular, it has raised concerns about the potential use of its models in autonomous weapons systems and mass domestic surveillance.

Anthropic is not operating at arm’s length. It is already deeply embedded within the US national security network, with its Claude model used in intelligence analysis, operational planning and other high-security applications. This is not simply a question about access or usage. It’s about who has the authority to decide the limits of a powerful technology.

This may sound abstract, but it’s not. Claude is not a future system confined to a lab. Colleagues who have had access to it describe it as a game changer: a system that can synthesise complex information, structure arguments and engage in sustained reasoning, rather than simply execute tasks. It is already finding a place in the everyday workflow of knowledge work. Yet the same system now sits at the centre of a dispute about military use.

AI is moving rapidly from a supporting role to something more consequential. It no longer simply processes information; it influences how decisions are made. In some contexts, those decisions may carry real-world consequences. Military applications are the clearest and most uncomfortable example, but they are not unique. Similar questions are beginning to emerge in healthcare, business processes and other areas where judgement and responsibility matter. This is where the Anthropic dispute becomes more than a contractual disagreement. It highlights a deeper shift in the role of tech companies.

Traditionally, the division was relatively clear, even if it evolved over time. Governments and regulators set the rules, often catching up as technologies advanced, while companies built the tools. Infrastructure providers, from tele coms networks and utilities to cloud platforms, largely supplied capability without questioning how it was used. AI does not fit this model. Its development is outpacing regulation. Systems such as Claude are not passive infrastructure; they are built with guardrails, constraints and embedded choices about what they will and will not do. Those guardrails and choices are still evolving and are far from settled. In resisting certain uses, Anthropic is making explicit something that is becoming harder to ignore: the companies building these systems are not just supplying technology – they are exercising judgement about its boundaries.

Different parts of the AI industry are taking different approaches to this problem. At the recent Nvidia developer conference, CEO Jensen Huang was a study in neutrality. He made the case for AI as foundational infra structure: a general-purpose technology that will underpin the next phase of economic growth. The message was clear: build, scale and deploy at industrial scale. Questions about how AI should be used, and where its limits should lie, were deliberately left to others.

Even among the model developers, there is variation. OpenAI, for example, has reached an agreement with the Pentagon while arguing that the red lines – around surveillance and autonomous weapons – are preserved in its contracts. The difference appears to lie, at least in part, in how those principles are enforced: through legal terms, technical architecture or a combination of both.

This is not just a US policy dispute. It cuts across geographies and industries and will shape what comes next. As AI becomes more embedded in the economy and society, the question is no longer simply what it can do, but who decides what it should not do.

There is no clear answer. A lot of thought has gone into how AI should be governed, across jurisdictions and institutions, but the guardrails that exist remain incomplete and are now being tested at their limits. The Anthropic dispute will be resolved one way or another. The underlying question it exposes will not.

By Sasha Planting