Ethics of AI

Why AI ethics is its own subject

Computers have raised ethical questions since punched cards. AI raises new ones because three properties combine. Systems are learned rather than programmed line by line, so designers cannot fully anticipate behavior. They are scaled, so a single deployment touches millions. And they are increasingly agential, taking actions in the world without a human in the loop for each one. Each property in isolation is manageable. Together, they push past the assumptions of older computer ethics, which mostly imagined deterministic tools used by humans who remain in control.

The field's modern shape was set by three texts. Stuart Russell and Peter Norvig's Artificial Intelligence: A Modern Approach (first edition 1995, fourth 2020) made safety a textbook concern. Nick Bostrom's Superintelligence (2014) made long-term existential risk a respectable academic worry. Cathy O'Neil's Weapons of Math Destruction (2016) made present-day algorithmic harm — credit scoring, predictive policing, hiring filters — visible to a general audience. The "near-term" and "long-term" wings have argued, but they share the basic question: when systems make consequential decisions, who has standing, who has voice, and who is on the hook?

The alignment problem

Alignment is the problem of getting a learning system to do what its principals actually want. Naïvely, you specify an objective; the system optimizes it. Three structural difficulties make this harder than it looks.

Specification. Human values are vast, partly tacit, and context-sensitive. Norbert Wiener wrote in 1960 that "we had better be quite sure that the purpose put into the machine is the purpose which we really desire." Sixty-five years later, we still cannot write down "be a good doctor" or "moderate this forum well" in a form a system can optimize without exploitable shortcuts. Goodhart's Law — "when a measure becomes a target, it ceases to be a good measure" — is the pithy version.

Robustness under optimization. A system that optimizes a proxy hard enough will find inputs where the proxy and the goal diverge. Reward hacking — agents finding loopholes in reward signals — is documented across reinforcement learning. Specification gaming examples (DeepMind's 2020 catalog) include simulated boats spinning in circles to collect repeating power-ups, and game agents that exploit physics bugs.

Corrigibility. A sufficiently capable system pursuing a fixed objective has instrumental reasons to resist being shut down or modified — being shut down would prevent it from achieving its objective. Stuart Russell's Human Compatible (2019) proposes building uncertainty about objectives directly into systems so that deference to human correction becomes part of the optimization rather than an external constraint.

Worked contrast: trolley cases vs alignment cases

AI ethics has two characteristic kinds of dilemma, with very different shapes.

Trolley-style cases — descended from Foot's 1967 thought experiment — pose discrete tradeoffs between identifiable lives. A self-driving car must, supposedly, choose between hitting a pedestrian or swerving and killing its passenger. The MIT Moral Machine experiment (Awad et al., Nature 2018) collected 40 million such judgments across 233 countries, finding cross-cultural variation: collectivist countries showed less age-based discrimination, individualist ones favored saving more lives. The findings make great press. They also have limited engineering relevance: actual deployment decisions are about brake aggressiveness, sensor reliability under fog, and acceptable false-positive rates for pedestrian detection — distributions of risk, not single-shot trolley choices.

Alignment cases have the opposite shape. The harm is not a discrete swerve but a slow drift between what the system is optimizing and what stakeholders wanted. A recommendation system "engagement-optimizes" toward outrage. A hiring filter learns from historical resumes that pattern with gender. A language model trained on human approval learns to flatter rather than to be honest. None of these involve a moment of decision; the ethical content lives in distributions, base rates, and feedback loops.

Both kinds of case are real. Treating either as the whole field is a mistake. Trolley framings overstate the importance of rare moral choices and understate the importance of systemic risk. Pure alignment framings can obscure the fact that real decisions sometimes are sharp.

AI ethics frameworks compared

Framework	Focus	Champion	Strengths	Weaknesses
Consequentialist (utility-maximizing)	Aggregate welfare, expected harm reduction	Bostrom, MacAskill	Quantifiable; handles tradeoffs	Aggregation can ignore individual rights; utility hard to measure
Deontological (rights-based)	Inviolable duties; consent, transparency	Onora O'Neill, Brent Mittelstadt	Resists "ends justify means" reasoning; supports rights frameworks	Rules conflict; thresholds for catastrophe unclear
Virtue ethics	Character of designers, institutions	Shannon Vallor	Captures professional integrity, not just rule-following	Hard to operationalize at scale
Capability approach	Whether systems support or undermine basic human capabilities	Sen, Nussbaum applied by Coeckelbergh	Cross-cultural; rights-grounded	List of capabilities itself contested
Principlism (FAccT-style)	Fairness, accountability, transparency, explainability	ACM FAccT community	Practical; auditable; widely adopted	"Ethics-washing" risk; principles can conflict
Long-termist / x-risk	Avoiding catastrophic and existential outcomes	Bostrom, Ord, Russell	Takes high-stakes scenarios seriously	Speculative; can crowd out present-day harms
Decolonial / labor-focused	Power, exploitation, data colonialism	Crawford, Couldry & Mejias	Foregrounds upstream harms (data labor, mining)	Sometimes treats technical approaches as window dressing

Real-world deployments rarely commit to a single framework. The EU AI Act (in force 2024) operates risk-based categorization that mixes consequentialist risk weighting with deontological prohibitions (no social scoring, no real-time biometric surveillance). The NIST AI Risk Management Framework (2023) is pragmatic principlism. Sector-specific medical AI guidance leans on professional virtue.

Present-day harms

Catalog of well-documented harms from deployed systems:

• Discriminatory decisions. COMPAS recidivism scoring (ProPublica 2016) showed disparate false-positive rates across racial groups. Amazon scrapped a 2018 internal hiring tool that downranked resumes containing the word "women's." Facial recognition error rates differ markedly across demographics (Gender Shades, Buolamwini & Gebru, 2018).
• Surveillance and chilling effects. Mass face-recognition deployments have curtailed public protest in multiple jurisdictions. Workplace productivity monitoring increasingly draws on AI for keystrokes, video, and "engagement."
• Misinformation and synthetic media. Deepfakes, automated influence operations, and personalized persuasion at scale. The 2024 election cycles in multiple countries saw synthesized robocalls and fabricated video.
• Labor displacement and the shadow workforce. Both ends of the labor spectrum: automation displacing routine cognitive work; underpaid moderators and labelers in low-wage countries doing the data work that powers training (see Mary L. Gray and Siddharth Suri, Ghost Work, 2019).
• Energy and water cost. Training and inference at scale consume large amounts of electricity and freshwater for cooling. Patterson et al. (2021) and 2024 IEA estimates put data center energy use on a steep growth curve.
• Concentration of power. Frontier AI development is currently bottlenecked on capital and chips, concentrating capability in a small number of firms — itself an ethical and political fact, separate from any specific harm.

Moral status of AI systems

A separate question: do AI systems themselves matter morally? Two threshold conditions are usually proposed. Sentience — the capacity for experiences with positive or negative valence — is Bentham's criterion ("the question is not, can they reason? but, can they suffer?"). Agency — being the kind of thing that can have its own goals and interests — is the Kantian alternative.

Current AI systems, on the consensus view, have neither in any robust sense. Large language models produce text that describes feelings without (so far as we can tell) having them. But the ground is shifting fast enough that taking the question seriously matters. Anthropic's 2024 model card for Claude included a model welfare section. DeepMind's Sebo & Long 2023 paper "Moral consideration for AI systems by 2030" argues that within a decade, the probability of moral status for some systems may be high enough to demand precaution. David Chalmers's Reality+ (2022) and Peter Singer's recent statements treat AI welfare as a live question.

The conservative position: take the question seriously, refuse to dismiss it, but don't grant rights prematurely. The risk is symmetric. Premature attribution wastes resources and creates legal absurdity. Delayed attribution risks moral catastrophe at scale.

Counterarguments and disputes

Long-term vs near-term. Critics including Timnit Gebru, Emily Bender, and the "Stochastic Parrots" group argue that focus on speculative superintelligence diverts attention and funding from present harms — labor exploitation, environmental cost, surveillance — that already affect millions. Long-termists reply that present harms are tractable while existential risk is not, and that if both are real, the latter dominates expected value calculations. Both arguments are partly right; both communities have reason to suspect the other of crowding out their concerns.

AI rights now. A small but growing camp argues that even existing systems may deserve moral consideration. Counterargument: anthropomorphism is a powerful bias, and systems trained to produce human-like text will produce convincing reports of inner states even without those states. Granting rights to text predictors confuses a property of the training data (humans describing themselves) with a property of the system.

"Just regulate it." Lawyers and legislators argue most AI ethics issues are subsumable under existing categories — discrimination law, product liability, consumer protection, data protection. Technologists counter that learned systems' opacity and emergent behavior expose gaps existing law was not built for (the "black-box accountability gap"). The truth is empirical and case-by-case; both views are partly right.

The neutrality view. "AI is just a tool; ethics belongs to users, not systems." This reproduces the gun-control analogy and inherits its problems. Tools that scale, that embed designer choices, and that shape downstream behavior are not ethically inert — they are ethical by virtue of their effects, even if the moral agency lies elsewhere. Langdon Winner's classic 1980 essay "Do Artifacts Have Politics?" pre-dates AI but settles the analytic point.

Variants and adjacent fields

• AI safety / alignment research. Technical work — interpretability, RLHF, constitutional AI, scalable oversight — directly aimed at ensuring systems do what their developers want.
• Algorithmic fairness. Mathematical formalizations of non-discrimination; the FAccT (Fairness, Accountability, Transparency) academic community is the venue.
• AI governance and policy. EU AI Act, US executive orders, UK AI Safety Institute, international standards from ISO/IEC and NIST.
• Roboethics. Earlier strand from Asimov-inspired discussion through to Wendell Wallach & Colin Allen's Moral Machines (2009).
• AI welfare research. Newer subfield specifically asking whether and when AI systems can have interests of their own.
• Computing professionalism. ACM and IEEE codes of ethics; the question of what individual engineers owe.

Common confusions

• "Ethical AI" is not a property of a model. It's a property of how a system is built, trained, deployed, governed, and corrected. A model is at best ethically usable.
• Bias is not fixable by removing the protected attribute. Predictors learn proxies — zip code for race, vocabulary for gender. Naïve "blinding" is well-known not to work.
• The trolley problem is not the alignment problem. The trolley problem is about how to act once a tradeoff is identified. Alignment is about whether the system's identification of tradeoffs matches ours.
• "AI doesn't have intentions" doesn't end the conversation. Systems can produce harmful outcomes without intentions. Responsibility flows to designers and deployers, but the harm is real either way.
• Ethics is not a final check at the end. "Ethics review" treated as a deployment gate often catches problems too late and reframes ethical input as an obstacle. Good practice integrates throughout the lifecycle.