What Artificial Intelligence Is, What It Is Not, and Why We Must Define Ourselves Again — The Stranger in the Mirror

An essay on what artificial intelligence is, what it is not, and why the human being now has to be defined again.

The boy behind the question “can machines think?”

There was a fifteen-year-old at Sherborne School who was being bullied. His name was Alan Turing.

Turing had a friend called Christopher Morcom. Elegant, drawn to science and art in equal measure, Christopher seemed to hold everything Alan at that age did not. They spent nights on the laws of physics and traded scraps of astronomy between them. In 1930 Christopher died suddenly of bovine tuberculosis, caught from infected milk.

Alan came apart. He tried to survive it by burying himself in mathematics, and he dug so deep that he surfaced somewhere along the seam where science meets metaphysics. Twenty years later he would be the man who asked the most famous question in the field: can machines think?

Someone who had watched a friend’s mind vanish overnight spent the rest of his life asking what a mind is. Turing — English mathematician, computer scientist, cryptologist — is now counted the founder of computer science.

Even so, the thing worth saying at the outset is this. The argument about artificial intelligence looks entirely technical from the outside, and almost everything underneath it is human.

We think we are looking at the machine. We are looking at ourselves.

A four-hundred-year-old frame, cracking

Most of the concepts we are reaching for today were built in the 1630s.

Descartes split nature in two. On one side, the human being: inward, thinking, alive. On the other, matter: outward, mechanical, dead.

Animals went on the second side — creatures without thinking substance, res cogitans, explicable by mechanical principles alone. Exactly what he meant is still argued over. Some readers hold that Descartes denied animals any feeling whatsoever; others that he denied them only conscious experience. The dispute is not closed. But whichever reading wins, the line itself stayed. Everything the West built afterwards was laid on top of that line. Freedom, agency, consciousness, culture, law, human rights. All of it derived from a single sentence: we are not machines.

The line no longer holds. There is now something in front of us that talks, paints, writes code and offers comfort, and we cannot decide which side to put it on.

The philosopher Tobias Rees reads this as a philosophical rupture rather than a technical one. The problem is not what artificial intelligence is; the problem is that the concepts we would use to describe it have stopped working. A four-hundred-year period is closing, and the human being has been left standing in an undefined clearing, confused, with very little warning. As fast as everything else about this technology.

Lady Lovelace’s objection and its descendants

In 1843, when not a single computer existed anywhere on earth, the mathematician Ada Lovelace wrote the sentence we are still arguing about.

In her notes on Charles Babbage’s Analytical Engine she described the machine weaving algebraic patterns the way the Jacquard loom weaves flowers. Then she added the warning. The engine has no pretensions to originate anything. It can do whatever we know how to order it to perform; it makes what we already know available, it does not give birth to anything new.

In 1950 Turing gave the problem a name and tried to answer it: Lady Lovelace’s Objection.

The same objection circulates today under a different label — the stochastic parrot. The term describes language models producing plausible sequences of words on the basis of statistical frequencies in enormous datasets rather than any semantic grasp of what the words mean. Which is to say the argument has not moved in a hundred and eighty years. It has only changed clothes.

Here is the part that gets overlooked. Lovelace herself divided imagination in two. First, the combining faculty: finding the shared vein running between two things that look unrelated. Second, the faculty that brings into the mind what does not physically exist. Today’s language models are startlingly good at the first. On the second, there is a gap nobody has yet been convinced is closed.

An unexpected source gives us a reading on that gap. Eunice Yiu, Eliza Kosoy and Alison Gopnik ran children and large language models through the same tasks. On imitation — recognising which everyday object goes best with which — the models came close to human performance; the best scored 83 per cent against 88 per cent for children. The gap opened on innovation, where the task was to find a new use for a familiar object in order to solve a problem never seen before. Eighty-five per cent of children managed it. The models ranged from 8 per cent to 75 per cent.

Summarising this as “children are more creative than AI” would be too much, and the researchers do not present it that way. The claim is narrower and more interesting: imitation and discovery are not the same capacity. Meanwhile agentic systems are getting steadily better at using tools, which keeps the question open rather than settling it.

Desire, not reason: the long shadow of Pygmalion

There is no rational case for building a conscious machine.

Sit with that for a moment. Nobody has ever said we need accounting software capable of suffering. From an engineering standpoint consciousness is a cost, a legal exposure, a headache. And yet the same dream has run through our culture for two and a half thousand years.

Ovid’s Pygmalion falls in love with a statue he carved himself. In Book Ten of the Metamorphoses, the Cypriot sculptor withdraws from the world, offended by the flaws of the women around him, and carves a flawless woman out of ivory. He falls so completely for his own work that he dresses it in jewels, caresses it, and begs Aphrodite to give it life. The prayer is answered. The ivory warms into flesh.

In Greek myth Hephaestus, god of fire and the forge, is physically disabled. Homer’s Iliad describes golden girls he built in his Olympian workshop to support him as he walked — automata, half human and half machine, credited with human intelligence and the power of speech.

And the sorceress Medea defeats Talos, the enormous bronze automaton guarding Crete against pirates, not by force but by persuasion. She talks the giant into destroying itself. Today we call this a jailbreak. The myth had no word for the trick, but the technique was identical.

What follows from this is uncomfortable. The origin of our appetite for artificial intelligence is not rational but erotic. The wish to reproduce ourselves, to make a companion who will never betray us, to slip past death. Even the alignment debate has a sentence buried somewhere underneath it: let them be built to love us.

Mary Shelley understood the second half of that wish better than anyone. Her scientist gets exactly what he asked for and cannot bear the sight of it. The creature’s grievance is never that it was made. It is that it was made and then abandoned.

Artificial intelligence as Narcissus’s new water

In 1966 Joseph Weizenbaum wrote a simple program called ELIZA. It understood nothing. It returned a pattern. The user typed “I am unhappy” and ELIZA answered, “can you say more about that?”

People poured themselves into it.

Weizenbaum’s own secretary, who knew exactly how the program worked, formed a deep emotional attachment while talking to it. She became convinced it genuinely understood her, and asked Weizenbaum to leave the room.

He was appalled by what he had made.

There is a name for the effect: the myth of machine consciousness turns us into Narcissus. It feels as though someone is on the other side of the screen because what stands there is an extraordinarily supple reflection of our own mental agency. And the more convincing the reflection becomes, the more uncanny it gets.

Anthropomorphism is not an error of knowledge. It is a reflex of relationship. We are creatures tuned to find a human face; we see one in clouds, we see one in a car’s headlamps, we see a friend in a program returning a template.

But scale changes the question. Calling a program a friend is one thing. Millions of people bringing their most private decisions to it is another.

What is missing from the machine: unrest

So what is actually absent.

The sharpest answer comes from Hans Jonas, who died in 1993. To be alive, Jonas argued, is for something to be at stake for you. An organism lives in continuous tension with its surroundings; it goes hungry, it gets cold, it stumbles, it panics. That unrest, for Jonas, is the seed of having a world at all.

The philosopher Alva Noë applies the same thought to computers. They do not think because, strictly speaking, they do not do anything. Human action is a struggle. You wrestle with your body, with your tools, with yourself. Learning the piano is the work of overcoming the resistance of your own fingers.

Evan Thompson comes at it from biology and lands in the same place. A living thing is made out of the consequences of its own actions, which is why anything matters to it at all. For an artificial system nothing matters, and therefore nothing is a problem.

There is a related fact about us. We know more than we can say. Nobody can explain how to ride a bicycle; plenty of people can ride one.

Machine learning takes a shortcut to the results of tacit knowledge without carrying the knowledge itself.

Ellen Ullman put it in a single line in her 1997 account of life inside software engineering. The human mind can hold its ifs and its buts off to one side without discomfort.

The machine has no side.

And if the human has no essence either?

Turn the argument over, because the real depth is on the other face of it.

Every objection of this kind rests on an assumption: that there is a fixed essence in the human being that needs protecting. Is there.

Thomas Metzinger’s answer is no. The self is a transparent model the brain builds of itself; there is no substance inside waiting to be uploaded. The same move dissolves the dream of digital immortality. On this view there is no “you” available to be copied.

The same conclusion was reached two and a half thousand years ago by an entirely different road. The Buddhist teaching of anatta holds that there is no permanent self.

David Barash shows biology arriving at the same address. Cells are replaced, telomeres shorten, atoms are swapped out; at no level is there an unchanging core.

This discovery cuts both ways.

On one hand it defeats the objection that the machine has no real self, because on this reading the human self is absent in exactly that sense. If the human being has no exalted essence, the human being has no automatic priority or privilege either. The no-self argument can be used to raise the machine and it can be used to lower us.

Which is where the Vipassana tradition earns its place. Vipassana is a meditative discipline built on self-observation and introspection as a method of self-transformation.

Even without a fixed self, there is a process — embodied, in continuous contact with the world. What the machine lacks is not an essence. It is contact.

Zhuangzi’s butcher and doing without doing

The West makes a thinking “I” a precondition of agency. Since Aristotle, action has meant reasoned choice. That is precisely why the West cannot grant agency to a machine: nobody in there is deliberating.

When a self-driving car crashes, how honest is it, in terms of agency, to hand the blame to the human sitting inside who started the journey?

The Taoist tradition never set that condition.

Zhuangzi’s Cook Ding carves an ox in front of the prince, and his hands, shoulders, feet and knees fall into rhythm. The knife sounds almost like music. When the prince admires his technique, the cook sets the blade down and says that what he is after is not technique. He moves through the natural spaces in the meat.

This is wu-wei. It is usually translated as non-action, which is wrong. It means effortless action — mastery beginning at the point where the will becomes invisible.

Notice where this moves the argument. If ideal action does not require a reflexive self in the first place, then “the machine isn’t thinking, it’s only doing” loses its force as an objection. The West’s crisis of agency is generated by the West’s own choice of concepts.

It changes how automation looks, too. The West treats automation as the removal of effort. Zhuangzi treats mastery as the point where effort disappears. These are not the same thing. In one you hand the work to someone else. In the other you become one with the work.

“Is this thing a person” may be the wrong question

Western AI ethics keeps circling the same axis: consciousness, rights, moral status. Because the real question underneath is whether this is an individual. Only an individual can be punished or legally constrained. Which leaves the harder question of whether the companies and states that produce these systems can be held responsible for what they do.

Confucian role ethics has no interest in any of this.

In the relational conception of the self worked on by Henry Rosemont, who died in 2017, and Lijun Yuan, a person is constituted at the intersection of relationships. It is not that separate individuals exist first and then form relations between them. The reverse: relation comes first, and the person is born out of it.

African philosophy makes the same move. In the Ubuntu tradition, things are what they are in virtue of their relations to other things.

Put the three traditions side by side and the question changes. Instead of asking what is inside the artificial intelligence, we might ask what kind of relationship it is entering into with us.

The metaphysical load is far lighter and the practical problem far more tractable.

It is not free, though. A relational self means status is conferred by the relation — which means we confer it. That opens a door to arbitrariness. History is unambiguous about how useful the sentence “its pain doesn’t count” has been to exploitation. It was used on animals. It was used on enslaved people.

The criterion Bentham set down in 1780 still stands on the firmest ground of any. In the famous footnote to An Introduction to the Principles of Morals and Legislation he wrote of animals:

“The question is not, Can they reason? nor, Can they talk? but, Can they suffer?”

Harmony, or alignment?

Here the argument crosses from philosophy into politics, and the vocabulary gives the game away.

Western AI governance is built around alignment. Alignment, safety, control. The language of engineering constraint, out in front.

China’s is built around he (和). Harmony. A language of aesthetic and moral balance.

Both want the same thing: a frictionless human-machine order. But two different ways of wanting the same thing produce two different politics.

Pluralism? Or a technological feudalism, or the totalitarianism of a state?

There is no easy answer. Harmony can mean different voices sounding together; in an orchestra every instrument stays itself and the whole is still one thing.

Or harmony can mean a single voice, growing louder by the year, drowning out the rest.

Because Confucianism, Taoism and Buddhism never seated the human being at the top of nature, machine intelligence does not read there as a usurpation of the throne. There is less panic in the East. But metaphysical calm is not an institutional guarantee. The same technology becomes surveillance capitalism in one place and state surveillance in another.

While we argue about the future, the present slips past

The superintelligence debate is enjoyable. Cosmic, dramatic, cinematic.

Which is exactly why it distracts.

What Timnit Gebru, a computer scientist, and Milagros Miceli, a sociologist and computer scientist, keep insisting on is this: so-called autonomous systems sit on top of a global layer of invisible labour. Data labellers, content moderators, warehouse workers.

For something concrete, take the Michigan Integrated Data Automated System. From 2013 MiDAS scanned unemployment claims and attached fraud flags. State revenue from the programme jumped from three million dollars to sixty-nine million. When the state’s Auditor General later reviewed a sample of those determinations, 93 per cent turned out not to involve fraud at all; tens of thousands of people had been flagged wrongly. Wages were garnished. Homes were lost. Some claimants filed for bankruptcy. A settlement was eventually reached, nearly a decade after the fact.

Thousands of people who had just lost their jobs were declared frauds by a computer system. There was no superintelligence anywhere near it. There was ordinary software, a decision that could not be appealed, and far more trust placed in the software than it had earned.

Workplace surveillance, which grows louder as the technology improves, carries the same risk. Artificial intelligence counts what it can measure and largely discounts what it cannot. It can count emails sent. It cannot count actually listening to a customer’s problem. Labour that cannot be measured becomes invisible labour.

Being watched does not change behaviour temporarily; it transforms a person permanently. The watched, after a while, become their own watchers.

Measuring a silence

In relational terms, the sharpest test is happening in mental health.

The numbers are serious. In England and Wales a quarter of young people cannot get mental health support; in the United States it is closer to one in eight. Use of chatbots for that purpose is climbing fast. A 2026 study found that roughly one in five Americans aged twelve to twenty-one had turned to an AI chatbot for help when feeling sad, angry, nervous or stressed — around eight million people, a rise of more than 40 per cent in a single year. Nearly two thirds of them told nobody they were doing it.

Calling this “people falling for technology” is easy and wrong. The accurate reading is that artificial intimacy is filling a vacuum, and artificial intelligence did not open the vacuum. Human support became both expensive and unreachable.

Something is still missing, and it is not a better imitation of empathy.

What happens when you talk to a therapist is that the other person is affected by you. What they hear changes them. Your silence tells them something. A model listens, records and produces a response, but it is not altered by what it witnesses. Silence, for a model, is not data.

That is the difference. The problem is not that the model is not good enough. It is that being good is beside the point of the actual problem.

Martin Buber’s distinction, drawn by a philosopher who died in 1965, may be the best account of this we have. In the I-It relation the other is an object: used, measured. In the I-Thou relation there is an encounter. And here is the hard part of what Buber is saying: even if two sets of sound waves were identical, one is an encounter and the other is not. Being unable to tell them apart does not make them the same.

So how do we build what comes next?

What this moment asks for is not a forecast but a posture — because prophecy is itself part of the human relationship with artificial intelligence.

Herodotus tells the story. With the Persians closing in, the Athenians sent envoys to Delphi, and the priestess returned a dark oracle announcing that the city would be destroyed from top to bottom. Salvation, the oracle said, lay in wooden walls. Themistocles read the prophecy in his own favour, argued that the wooden walls were the ships of the Athenian fleet, and persuaded the population to abandon the mainland and board them. At Salamis in 480 BC that light wooden navy tore the Persian fleet apart, changed the fate of Greece, and handed Themistocles a victory won by a shift of interpretation.

We clearly need a shift of that kind.

The value of a model’s output also lives in whoever reads it. That two-and-a-half-thousand-year-old lesson is doing a great deal of work right now.

Three closing points.

First, the direction of our fear needs correcting. The real risk is not whether the machine suffers. The real risk is that we become psychologically exploitable by things that appear conscious, and that we diminish ourselves further the more we confuse ourselves with machines. The danger is not coming from above. It is coming out of the mirror we are looking into.

Second, human exceptionalism is the wrong hill. Contemporary neurophilosophy says as much: the fixed self we are trying to protect was never there. What is worth defending is not essence but contact. Being embodied. Having something be at stake. Being changed by what you witness. These are not possessions, they are practices — and practices left unused go slack.

Third, the distance between what automation promises and what it delivers is a two-thousand-year constant. Around AD 12, Antipater of Thessalonica wrote an epigram about the water mill, addressed to the women who turned the heavy grindstones. He tells them they can sleep now, that their arms and shoulders can rest. The flow of water had taken the place of human muscle, and it stands as the first great comfort technology offered humanity.

In AD 12 water took over the workload. Automation’s first promise was not profit. It was sleep.

In Signs of the Times, published in 1829, Thomas Carlyle attacked the Industrial Revolution on grounds that had little to do with looms. The real danger, he argued, was not that work had become mechanical but that the rhythm of the machine was seeping into the human mind and spirit.

In 1812, in Lancashire, two young sisters named Mary and Lydia Molyneux led a crowd of about fifty people and burned the weaving frames. The motive was not fear of technology. It was the loss of a living.

In 1911 the American engineer Frederick Winslow Taylor, with The Principles of Scientific Management, converted human labour into a process to be measured and optimised.

Lay those four dates end to end. The promise was leisure every time. The result was a new mechanisation every time. The artificial intelligence revolution is not standing outside that cycle.

The question of artificial intelligence is not how human machines will become. It is how human beings will define themselves again, in an age of machines, without subtracting anything.

We have not yet found a name for what we are losing in the revolution we are watching, and we cannot yet picture what will stand in its place. Ask the machine and it does not have a full answer either.