The 300-Year-Old Philosophical Fight Now Shaping AI Policy
Team Ilm Ora · 13 September 2026
Two philosophers who never met a computer, arguing over two centuries ago about what makes an action right, are now shaping how self-driving cars are programmed to decide who lives in a crash they can't avoid.
Judging Actions by Their Outcomes
The first major position in this fight is called utilitarianism, and its core idea is simple to state: an action is right if it produces the greatest overall good — usually framed as happiness or wellbeing — for the greatest number of people affected by it. The consequences of an action are what matter; the action itself has no inherent moral weight beyond the outcomes it causes.
The clearest early formulation of this idea traces back further than most people expect. In 1725, Scottish philosopher Francis Hutcheson wrote that "that action is best, which procures the greatest happiness for the greatest numbers" — a phrase later picked up and popularized by Jeremy Bentham, who formalized utilitarianism as a systematic ethical theory in his 1789 work An Introduction to the Principles of Morals and Legislation. Bentham even attempted to build something like a mathematical "hedonic calculus" for weighing pleasure against pain across a population. John Stuart Mill later refined and defended the theory in his own 1861 book, Utilitarianism, addressing objections that raw pleasure-maximization ignored the difference between higher and lower forms of wellbeing.
Judging Actions by Duty, Not Outcome
The second major position is deontology, most closely associated with the German philosopher Immanuel Kant, particularly through his 1785 work Groundwork of the Metaphysics of Morals. Kant's argument runs directly counter to utilitarianism: what makes an action right or wrong isn't its consequences at all, but whether the action itself follows correct moral duties and rules, regardless of outcome.
Kant's central test for this is called the categorical imperative, which asks whether the principle behind your action could be consistently made into a universal law that everyone follows. A second, related formulation of the same principle states that you should never treat another person merely as a means to an end, but always also as an end in themselves — meaning people can't simply be used as tools for producing the best overall outcome, even if doing so would genuinely help more people. Under strict Kantian deontology, some actions are wrong regardless of how much good they might produce, because they violate a duty or use a person merely instrumentally.
Why This Isn't Just Academic Anymore
For centuries, this argument stayed mostly inside philosophy departments. That changed once engineers had to write actual code determining how autonomous systems behave in situations involving genuine risk to human life — most visibly with self-driving cars.
The clearest real-world version of this problem is a modernized twist on the classic "trolley problem" — a decades-old philosophical thought experiment about whether it's right to divert harm from many people onto fewer people. Applied to a self-driving car: if a vehicle's brakes fail and it must choose between continuing straight into a group of pedestrians or swerving into a barrier that would badly injure its own passenger, what should it be programmed to do? A strict utilitarian approach says minimize total harm — swerve, if that produces fewer casualties, even at direct cost to the passenger. A strict deontological approach raises a serious objection: deliberately programming a car to sacrifice its own passenger treats that passenger as a means to protect others, which some readings of Kantian ethics would flatly rule out, regardless of the math.
What Millions of People Actually Think
Rather than leave this purely theoretical, researchers at MIT built something called the Moral Machine — an online platform presenting people worldwide with trolley-problem-style scenarios involving self-driving cars, asking them to choose who an autonomous vehicle should prioritize in an unavoidable-harm situation. The project collected roughly 40 million individual decisions from people across more than 200 countries and territories, published as a major study in the journal Nature in 2018.
The results showed some broadly consistent preferences across most regions studied — respondents tended to favor sparing more lives over fewer, and sparing human lives over animals — but also revealed significant regional variation in other judgments, such as how much weight to give a person's age or social status in these scenarios. In other words: there isn't a single universal moral intuition waiting to be coded into these systems. Different populations, in aggregate, actually weigh these dilemmas somewhat differently, which is precisely the kind of finding that makes writing a single, universally acceptable ethical rule into autonomous vehicle software a genuinely unresolved policy and engineering problem, not just a philosophical one.
Why Neither Side Has Simply Won
Both frameworks run into real, documented problems when engineers actually try to implement them directly. Pure utilitarian systems require a way to quantify and compare fundamentally different kinds of harm and benefit — a genuinely difficult, ethically fraught calculation to automate. Pure deontological systems, built from fixed rules, struggle with the fact that rigid rules often can't anticipate every real-world situation, and rules that seem clearly right in the abstract can produce results that feel deeply wrong in a specific, concrete case computers are notoriously bad at recognizing as an exception.
Most researchers working on machine ethics today don't fully adopt either framework in its pure form — they build hybrid approaches, or restrict automated systems to narrower, better-defined domains where the ethical stakes are more manageable. But the underlying philosophical disagreement between weighing outcomes and following fixed duties hasn't been resolved. It's simply moved from philosophy seminars into engineering meetings, software specifications, and now, national conversations about how autonomous systems should be regulated.
Forty million people, from more than 200 countries, were asked how a self-driving car should decide who to save in a crash it can't avoid — and the results didn't converge on one universal answer. Two philosophers, arguing centuries before the first car existed, could have predicted exactly why.