The Sauron
Problem.
When intelligence stops serving judgment and starts turning control into order.
Most people remember Sauron as an eye.
It is an understandable mistake. The eye is memorable. It watches from a tower, burns over a ruined landscape and gives evil a logo simple enough to put on a T-shirt.
But the eye is only the interface. The real threat is the system behind it.
Sauron is not frightening because he wants to destroy the world. That is too crude. He wants the world to work. To be legible, coordinated, efficient and finally obedient to one intelligence. He does not merely want subjects. He wants a world in which the question of disobedience no longer occurs.
The Wrong Villain
Morgoth is hunger without limits: possession, ruin, the desire to bend creation until it reflects his own resentment. Sauron is something more recognisable. He is administration after it has forgotten why it exists.
Tolkien’s own account matters here. Sauron did not begin as a caricature of evil. He was drawn to order, coordination and the removal of wasteful friction. At first, Tolkien writes, he even considered the material well-being of those living in the world he wished to arrange. Then the plan became its own justification. What began as organisation hardened into the conviction that organisation required a single will.
That is the Sauron problem: perfectionism becomes dangerous when it cannot tolerate another person’s imperfect freedom.
Rousseau’s Test
Rousseau understood something every technology company, government and institution should keep close: force does not become legitimate merely because it is effective.
In The Social Contract, he writes that the strongest cannot remain master unless strength is transformed into right and obedience into duty. The sentence is older than machine learning, but it describes the modern default perfectly. Power lasts when people stop experiencing it as power.
A recommendation becomes a ranking. The ranking becomes a workflow. The workflow becomes policy. Policy becomes “how the system works.” At each stage, a human choice becomes harder to see — and therefore harder to challenge.
That is not a conspiracy. It is how systems acquire authority. They make the contingent appear inevitable.
Where AI Actually Sits
We still talk about AI as if it were one layer: the model, the chatbot, the answer in the box. That is the layer people can see. Power accumulates in the layers underneath.
Summaries, drafts, suggestions and a human decision still visibly follows.
It researches, routes, writes, changes files or carries a task across several steps.
Ranking, recommendation, eligibility and workflow decide which options are easiest to see or take.
Its output is treated as neutral, inevitable or beyond the competence of ordinary people to contest.
The first layer is a tool. The fourth is a political condition.
We are not living under an AI sovereign. That language would be melodrama. But we are building systems that move steadily from assistance to delegation, and from delegation to defaults. The risk is not that one model wakes up and takes the throne. The risk is that thousands of ordinary decisions quietly lose their human author.
Why Efficiency Makes Such a Good Alibi
Every concentration of power arrives with a useful argument. It is faster. It is cheaper. It reduces mistakes. It removes friction.
Often, it does all of those things. That is why the argument works.
Efficiency is not the enemy. Waste matters. Bad processes exhaust people. A capable system can give a small team the leverage once reserved for an institution. The question is what kind of friction is being removed. Administrative repetition is not the same as dissent. A bad handover is not the same as an appeal. An unnecessary form is not the same as a person asking, “Why was I denied?”
Camus wrote about the temptation to turn an idea into an absolute. The danger is not conviction; the danger is the moment conviction no longer has to answer to the people it touches. AI gives that temptation a new interface: smooth, useful and increasingly difficult to refuse without appearing irrational.
A dangerous system removes the moment in which someone can say no, ask why or choose another path.
Both may call themselves efficient. Only one preserves agency.
The One Ring Test
The ring was not powerful because it made its wearer louder. It was powerful because it promised one point of control over many wills. That is the fantasy behind every total system: one dashboard, one source of truth, one layer of intelligence that makes the rest of the world manageable.
No serious AI strategy should accept that fantasy without a test.
2. Can the affected person challenge it and receive a meaningful review?
3. Can the organisation leave the system without losing its records, capability or memory?
4. Is the default serving the user’s interest, or the system owner’s incentive?
5. Who signs their name under the outcome when it goes wrong?
These are not anti-technology questions. They are the questions that allow technology to remain a tool rather than become a quiet constitution.
The Human Reserve
There is a reason the best systems still need a person with the authority to stop them. Judgment cannot be fully outsourced because judgment includes responsibility for the exception — the case that does not fit the rule, the context no score captured, the harm no dashboard counted.
That reserve is expensive. It takes trained people, time, independent scrutiny and the courage to reverse a decision that looked efficient on paper. It is also the cost of remaining free.
Sauron’s error was not intelligence. It was the belief that intelligence entitled him to decide for everyone else. The error becomes familiar whenever a system’s designers confuse their ability to optimise a world with the right to define it.
The Verdict
AI should make us more capable, not more governable.
Its best use is to expand the range of work a person, a small team or a public institution can do with clarity and speed. Its worst use is to make human judgment look like a delay in a process designed by someone else.
We do not need to fear an eye above a tower. We need to watch the quieter moment: the day a recommendation becomes a rule, a rule becomes a default and a default becomes the only language a system understands.
Nothing further. Intelligence remains a gift only while people retain the freedom to question the order it creates.