The Responsibility Gap: Who Is Liable When AI Causes Harm?

On the agency of artificial intelligence, the legal weight of what it does, and the human share in that equation…

A legal question asked in 1546

Among the notes taken down at Martin Luther’s table in 1546, there is a case.

A miller’s donkey gets out of the yard and wanders down to the river. Looking for water, it steps into a fisherman’s boat that nobody has tied up. The current catches the boat, and the donkey goes with it.

The miller has lost a donkey. The fisherman has lost a boat.

The miller says the fisherman failed to tie up his boat. The fisherman says the miller failed to keep his donkey at home.

In the middle of the text sits a question in Latin: Nunc sequitur quid sit juris? So what does the law say?

Did the donkey take the boat, or the boat the donkey?

Luther’s ruling is two words. Ambo peccaverunt. Both were at fault.

That paragraph was written five hundred years ago, and it still sits at the very centre of the hardest legal problem we have. The motion that produced the loss started on its own, at the point where two omissions met. Nobody acted in bad faith. Nobody, strictly speaking, set the thing going. There was only something left untied, and something else set loose.

The whole law of artificial intelligence seems to fit inside that picture.

There is a concept called the responsibility gap

In 2004 a thinker named Andreas Matthias published a paper that few people paid attention to at the time, and gave a name to something: the responsibility gap.

His argument is simple and uncomfortable. In learning systems, it is not merely difficult but in principle impossible for the manufacturer or the operator to predict how the machine will behave in future. And it is unjust to be held morally responsible for what you could not foresee. So there is harm, and there is nobody who can fairly be held responsible for it.

Twenty years on, that sentence has stopped being an academic exercise. Artificial intelligence now refuses credit, screens job applicants, reads scans and proposes treatment, turns the wheel of the car we are sitting in, and forms contracts in the purchases we make.

And in every incident of harm, the same three sentences follow one another in order.

The manufacturer: we built the model, but you chose how to use it. The operator: we simply trusted the system. The system: the system says nothing, because a system is not spoken by. It is spoken about.

Which brings us to the real question. Is this gap a genuine metaphysical void, or a very useful vagueness?

This piece argues for the second. And the evidence for that answer lies in the law’s own history.

Rome’s solution: acting without being an agent

The law did not first meet the problem of “a thing that is not a person but acts on its own” through artificial intelligence. Two thousand years ago it was living inside that problem.

Under Roman law, a slave was not a person. In legal terms, a slave was property. And yet that property bought, sold, contracted on the master’s behalf, incurred debts and ran businesses. An entity with no legal personality was performing acts with full legal consequences.

Setting artificial intelligence and slavery side by side is not a comparison most readers arrive at willingly. It is worth staying with the discomfort, because the mechanism is the point.

Rome did not resolve the contradiction with metaphysics. It resolved it with plumbing.

The thing they called peculium was a separate fund that the slave managed in practice. The slave did business inside that purse. When harm arose, the master’s liability was as a rule capped at the size of the purse. Responsibility was neither dumped entirely on the master nor left hanging in the air. It was tied to a fund.

English law arrived at its own version of the same instinct, and its fate is instructive. Under the common law of the deodand, any object that had directly caused a person’s death — a cart, a horse, a piece of machinery — was forfeited to the Crown. Nobody asked whether the cart had intended anything. The harm was attached to the thing, and the thing was attached to a value.

The deodand was abolished in 1846. The timing is not incidental. Railway deaths had become routine, and coroners’ juries had started using deodands to fine railway companies; after the Sonning crash of 1841 one jury set the figure at a thousand pounds. A remedy that had survived for centuries while it concerned carts and horses was struck down within a decade of becoming expensive for industry.

Lawyers are discussing the older technique again. Ugo Pagallo’s proposal of a “digital peculium” runs along the same vein. If you are releasing an autonomous system into the market, you must also show a separate pool of assets standing behind it.

The lesson that comes out of this is elegant and unsettling at the same time.

The elegant part: agency is not discovered. It is assigned.

Roman law never asked whether the slave had a will, because it had no need to. What the law required was not an intention. It was a counterparty.

The unsettling part: this solution was developed inside one of the most repellent institutions in human history.

There is a moral price to likening a tool to a slave. The comparison does not elevate the tool; it normalises what was done to human beings. What can be taken from it is not the institution but the technique. The law can bind the harm caused by a non-person actor without ever declaring that actor a person.

The same technique goes by other names today. Corporate legal personality. An employer’s liability for the acts of an employee. Strict liability in hazardous activities. Every one of these is a legal construct built around the same purpose. Where there is harm, the harm has to be attached to a pocket, without anyone having to look inside a mind.

What courts are actually doing: three cases, one pattern

Artificial intelligence is very new in our lives. Which makes it worth leaving the theory aside and looking at the decisions of the last two years. Because the courts stopped asking the question philosophers are still debating some time ago.

In February 2024, in Canada, a man named Jake Moffatt wanted to buy an Air Canada ticket after his grandmother died. He asked the chatbot on the airline’s website whether he could get a bereavement fare. The assistant told him the discount could be claimed after the fact. This was wrong. The company’s actual policy said no such thing.

In front of the tribunal, the airline made a remarkable argument: the bot is a separate entity, responsible for its own statements.

The tribunal rejected it. The bot is part of the company’s own website, and the company is responsible for all the information on that website, whether it comes from a static page or a chat window. The award was small and symbolic, around six hundred and fifty Canadian dollars plus interest and fees. The principle is not small. You are answerable for whatever comes out of the mouth of your own corporate structure.

In August 2025, a Miami jury delivered a verdict over a crash that had happened in the Florida Keys in 2019. A Tesla with Autopilot engaged failed to stop at a junction and struck a parked car. One person died and another was gravely injured. The driver admitted that he had dropped his phone and was distracted at that moment.

This time the court split the fault: sixty-seven per cent to the driver, thirty-three per cent to the manufacturer. Total damages exceeded two hundred and forty million dollars, most of it punitive. The trial judge later refused to throw the verdict out, and the appeal is still running.

Notice what the court did not ask. It did not ask whether Autopilot was an agent. It divided the fault into percentages.

Luther’s ambo peccaverunt came back into our lives in that courtroom, five centuries later, in digital form.

The third example is not a court case but a contract. Mercedes has stated that inside the defined operating envelope of its Drive Pilot system — particular motorways, particular speeds, particular weather — it takes on legal responsibility for what happens while the system is in control. The envelope is narrow, and the carve-outs for the driver’s own duty of care are real.

What is happening there is very clear all the same. Responsibility was not discovered. It was installed as an undertaking. The manufacturer took on agency inside a defined envelope.

Three events, one pattern. The law is not asking whether this thing has a will.

It is asking who was in a position to prevent the harm, and who stood to gain or lose from it.

The gap is not closing, because nobody wants it closed

This is where the political side of the story starts, and the view is not pretty.

The European Union had a draft on its desk for years: the AI Liability Directive. It did two things.

The first was a rebuttable presumption of causation in defined circumstances. The injured party would not have to prove exactly what had happened inside the box; in certain cases the burden of disproving the causal link would shift to the producer.

The second was a duty to disclose evidence in high-risk systems. A court could order a company to hand over its records.

Those two provisions were aimed squarely at the heart of the responsibility gap. Because the real name of the gap, most of the time, is impossibility of proof. The person harmed has no model, no training data, no logs, no expert. Which makes proving anything impossible.

The Commission announced the withdrawal in its work programme of February 2025, and the withdrawal was formalised later that year. The grounds given were an absence of agreement among stakeholders and the demand for simplification in the digital field.

Now read these two sentences one after the other.

Matthias, 2004: in learning systems, attributing responsibility fairly is structurally difficult.

Brussels, 2025: the rules of proof that would have eased that difficulty were withdrawn in the name of digital simplification.

The first is a philosophical observation. The second is a political choice. And when the two are blurred together, something very useful appears: an exemption that looks like a law of nature.

The responsibility gap is not a discovery. It is a door left open.

And like any open door, somebody can walk through it and come out ahead.

The human share, first form: the new edition of “I was following orders”

Which brings us to the human side of the equation, where the real matter lies.

Consider the first war crimes trial in history. In the last fourteen months of the American Civil War, roughly forty-five thousand Union soldiers were held in a camp where malnutrition, disease and squalid conditions killed some thirteen thousand of them. After the war ended, the camp’s commandant, a Swiss-born Confederate captain named Henry Wirz, was arrested. At the Andersonville trial of 1865, his defence came down to a single line.

I was following orders.

The doctrine that trial established is still standing. A human being makes an internal judgment about the order he obeys; therefore, if an order offends his humanity deeply enough, he can disobey it. The sentence put to the defendant was this: a military superior is not a moral superior.

Today’s artificial intelligence edition speaks differently. The system recommended it. The score was low. The model flagged it.

And the proposed remedy is ready to hand. Put a human in the loop. Let the machine propose the decision and let a person approve it.

It sounds good. It sounds rather less good once you look at the data.

In Michigan’s unemployment system, an audit found that ninety-three per cent of the fraud determinations it reviewed were wrong. The comparison is the interesting part. Determinations that had passed through some level of human review still carried an error rate of forty-four per cent. Human oversight roughly halved the damage and still got close to half the cases wrong. On its own, it is not a safety mechanism.

The reason is psychological and familiar. If a system shows you an eighty-nine per cent confidence score, disagreeing with it costs more than time. It costs nerve. Approving is free; objecting is expensive. The officer who confirms the system gets promoted. The officer who stops the system has to explain himself.

A human in the loop who has neither real authority nor real time is not positioned as a safeguard. He is an absorber. When the harm lands, the blame stops there and travels no further up.

This has to be said plainly. For an organisation that does not want to carry responsibility, the ideal arrangement is one where the decision sits in the system and the blame sits in a person.

The human share, second form: the gap is not empty

The phrase “responsibility gap” summons the image of outer space. A place with nobody in it.

There are people in there. They are simply not visible in all that vacuum.

So-called autonomous systems rest on a global layer of labour that labels data, moderates content and cleans up output. The pay is low, the visibility is nil, the legal protection is thin.

The sharpest detail comes from a delivery driver. The monitoring system checks that the seatbelt is fastened. The quota penalises stopping. So drivers fasten the belt behind their backs. The system records a belt in place, and the human drives unbelted.

That small scene is the summary of an enormous structure. Up top there is measurement, the measurement looks correct, the record is clean. Down below there is risk, and the risk is not transferred to anyone. It simply flows downhill.

The Dutch childcare benefits case matters precisely because it is the exception. An automated fraud detection system wrongly accused tens of thousands of families, drove them into debt, and the affair ran all the way to the collective resignation of the government in January 2021. It is one of the rare instances in which an algorithmic harm was answered for at the very top.

Its rarity tells you what the rule is.

So it is not that nobody stands at the mouth of the cannon in the responsibility gap. It is that whoever is least powerful is available to be treated as the culprit.

The Eastern answer: action was never a one-person affair

Western law is built on making responsibility singular. It finds an agent, measures the will, establishes the fault.

Artificial intelligence strains that machinery, because there is no single agent to find.

In classical Chinese thought this was never a problem, because there action was collective from the start.

Think about riding a horse. This is neither the rider’s act nor the horse’s. The terrain, the training, the tack, the feed, the weather — all of it takes part in the act. Classical Chinese texts describe action in exactly this way: not a single will striking the world, but a consistency into which many things converge.

Confucian role ethics continues from the same place. A person is constituted at the intersection of relationships, and responsibility attaches not to individual will but to the role. A physician’s responsibility comes from being a physician, not from the intention held in that particular moment.

Translated into the present, this yields something specific. Instead of asking whether the model is an agent, ask who took on the role. Who made the recommendation, who distributed it, who sold it, who used it, whose job was it to check. Where there is a role, there is responsibility, and no one has to measure a will.

That produces an unexpectedly practical question for machine harm. When a system causes damage, what is the end of the process? Punishment, repair, correction, shutdown?

There is no such thing as forgiving a model. There is such a thing as absolving a company, and most of the time the sentence that covers it over is: we have updated the faulty model.

---

The question of scale: Hobbes and the “fruitful crime”

One more thing has to be added, because this is what separates artificial intelligence harm from ordinary harm.

In Leviathan, Thomas Hobbes drew a distinction while weighing the gravity of crimes. An act that injures only the present is not of the same weight as an act that injures the future by way of example. The first he called barren, the second fruitful. A fruitful crime multiplies and harms many.

Hobbes also held that the standing of the offender changes the weight of the offence. The same act committed by someone in authority is graver than when it is committed by an ordinary person.

Both criteria can be applied to the law of artificial intelligence today.

One person’s prejudice stays inside one room. A model’s prejudice repeats across millions of decisions, and pushes in the same direction every single time.

A model is, by definition, a fruitful machine for steering. In good and in harm alike.

In his judgment on Justinian in The History of the Decline and Fall of the Roman Empire, Edward Gibbon observed that the Romans were oppressed at once by the multiplicity of their laws and by the arbitrary will of their master.

Rule inflation and arbitrariness, in other words, are not antidotes to one another. They live together quite comfortably. Today’s compliance documents, ethical principles and transparency reports are enormous by page count. What the injured party actually holds in his hand is still very small.

So what should be done

The thesis of this piece fits into a single sentence.

Agency is not something to be discovered. It is something to be assigned.

And when it is not assigned, it falls of its own accord to the weakest link.

Four things follow from that.

1. The question of whether this thing is an agent has gone unanswered for two thousand years and will not be answered in the next ten.

The answerable question is a different one: who was in a position to prevent it, who profited, who carried the loss.

Every successful solution the law has produced was built on that question. From the slave’s peculium to the employer’s liability, from corporate personality to a Miami jury’s sixty-seven / thirty-three split.

2. Show the security behind whatever you set loose, and an agentic workflow counts.

Rome’s purse can take other names today: compulsory insurance, a segregated fund, a guarantee with a defined ceiling, or something not yet named.

The principle has to stay the same. If you are putting an untied boat into the water, it must be settled in advance who pays when it drifts. This does not stop innovation. It prices the uncertainty. Insurers and reinsurers may well build an economy of their own on top of it.

3. A human in the loop who has no authority cannot carry responsibility either.

An official whose objection could cost him his promotion cannot be a reviewer and cannot be the final decision-maker.

Real oversight demands three things: the power to override the system, the time to do it, and protection when you do.

Without those, the human is not in the loop. He is only a buffer.

4. The gap must not be treated as a law of nature.

If we allow rules of proof, record-keeping duties and evidence disclosure to be removed, whether deliberately or in ignorance, the responsibility gap widens.

In Luther’s donkey and boat, both parties were in the right and both were at fault at the same time. A ruling handed down at a table five hundred years ago said that in an incident where nobody is entirely guilty, nobody is entirely innocent either.

Today’s danger is the exact inverse of that ruling.

It lies in the fact that concluding nobody was at fault is very much easier than apportioning fault.

If the donkey is untied and so is the boat, the river will do the rest. And we should not tell ourselves that there is no point being angry at the river.