Humans Are the Ghosts in the Machine

In 1949, the philosopher Gilbert Ryle coined a phrase to mock an idea he thought was incoherent. Descartes had argued that the mind is a non-physical substance somehow piloting the physical body, and Ryle called this "the dogma of the ghost in the machine." He did not mean it as a compliment. He meant that the picture was confused: that if you found yourself needing to posit an invisible operator inside the mechanism to explain the mechanism's behavior, you had misunderstood the mechanism.

Three quarters of a century later, enterprises have built machines that genuinely do contain invisible operators. Not metaphorically. Literally. There are people in there.

They are the annotators who labeled the training data. The raters who taught the model which answers were acceptable. The engineer who wrote the retry logic that catches the failure mode no one documented. The analyst who receives the model's output, quietly corrects the two things it got wrong, and forwards it to the executive committee. The operations lead who knows that the forecast runs hot in Q4 and adjusts. The manager who decided which of the model's recommendations to act on and which to ignore, and whose judgment is the actual reason the program shows a return.

None of these people appear in the architecture diagram. Most do not appear in the business case. Several do not appear in any job description. And when the quarterly review asks what produced the improvement, the answer given is: the AI.

This is the most expensive accounting error in enterprise technology today. It is not expensive because it is unfair to the humans involved, though it is that. It is expensive because you cannot manage, measure, price, staff, govern, or scale a contribution you have made invisible. Every one of those functions requires a name. Ghosts do not have names, which is precisely what makes them ghosts.

This paper is an argument for a specific and slightly unglamorous discipline: finding the human judgment inside your automated systems and resolving it into named, accountable, budgeted, career-tracked work. Not removing it. Resolving it. The goal is not exorcism. The goal is that the ghost stops being a ghost.

The argument runs in twelve movements, and it is worth saying up front where it goes.

We begin with history, because this is a pattern rather than a novelty, and the pattern predicts what will happen next in your organization. Then a taxonomy of the ghosts themselves, described concretely enough that you can go looking for yours this week — eight recognisable shapes, most of which are probably in your building. Then the consequences, in ascending order of how much they cost.

First measurement: why invisible judgment makes every number you collect about an AI-mediated process a measurement of something other than what you think, and how that single error explains most of the pilot-to-production gap that has frustrated boards for three years running. Then the economics, worked through arithmetically on an ordinary use case, because the abstraction conceals how quickly the distortion flips a decision. Then governance, which is where the stakes stop being financial and start being legal, as regulators across jurisdictions converge on requirements that a ghost cannot satisfy.

Then a part that is uncomfortable and, we think, the most important in the paper: why human-in-the-loop oversight tends to degrade into rubber-stamping, why this gets worse as your models get better, and what to do about it. Most organizations that believe they have a control have a log, and the human factors literature explains why with some precision.

Then talent, including a species of key-person risk that most enterprises are carrying unpriced. Then the case against the obvious reflex — automating the ghost away — and why it usually relocates the problem somewhere harder to see. Then three composites drawn from real engagements, because patterns are easier to recognise in the particular. Then the method itself, step by step, most of which requires no hiring at all. Then the serious objections, including the one about our own commercial interest, which deserves a straight answer. And finally what changes for each chair around the table.

There is an appendix at the end containing the audit instrument, with the actual questions. If you read nothing else, read that, and ask the last question in it.

If you need an invisible operator inside the machine to explain the machine's results, you have not built an autonomous system. You have built a system with an undocumented dependency on a person.

Part I — Where the Ghosts Came From

The pattern is older than computing, and it is worth understanding as a pattern rather than as a novelty, because the pattern tells you what to expect next.

In 1770, Wolfgang von Kempelen unveiled an automaton to the Habsburg court: a mechanical figure seated at a cabinet, dressed in Ottoman clothing, that played chess. It beat aristocrats. It beat, reportedly, Napoleon. It toured Europe and America for decades and was examined by the curious and the skeptical alike. It was, of course, a cabinet with a person in it. A skilled human chess player, concealed in a sliding compartment, operated the figure's arm through a pantograph linkage.

The interesting thing about the Mechanical Turk is not that it was a fraud. It is that the fraud worked for sixty-eight years, on sophisticated audiences, in an era when nobody had any reason to believe a machine could play chess. The illusion was not really about the cabinetry. The illusion worked because the audience wanted to believe it, and because the human contribution was arranged to be structurally invisible — not hidden behind a lie so much as hidden behind a frame. You were told to look at the automaton. So you looked at the automaton.

Amazon named its crowd-work platform Mechanical Turk deliberately, and with more candor than the industry usually manages. The platform's original purpose was to route tasks that computers could not perform to humans who could, in volumes and at granularities that made the human labor disappear into the API response. Jeff Bezos described it as "artificial artificial intelligence." The joke is accurate. The frame says machine. Inside the frame, there are people.

Between Kempelen and Bezos, the pattern repeats with remarkable consistency.

The word "computer" referred to a person until the middle of the twentieth century. It was a job. The people who held it were disproportionately women, and their work — ballistics tables, wind tunnel data reduction, orbital mechanics — was genuine mathematical labor performed under time pressure with real consequences. When electronic machines took the name, the humans who had held it largely vanished from the account of what had been accomplished. It took decades and a popular film for the general public to learn that NASA's early orbital calculations were substantially the work of named individuals rather than of "the computers."

Telephone switching was performed by operators, whose skill included not just connecting circuits but exercising judgment about urgency, discretion about content, and improvisation during emergencies. Automatic switching replaced the circuit-connecting. The judgment, discretion, and improvisation were not replaced; they were redistributed, mostly downward and outward, to callers and to a smaller number of specialists, and the improvement was recorded as an unambiguous automation win.

Industrial automation followed the same arc. The assembly robot does not eliminate human judgment about tolerance, wear, calibration, and exception. It relocates that judgment to a maintenance technician and a process engineer, and the resulting headcount shows up in a different cost center where nobody attributes it to the robot.

In each case, three things happen at once. First, a mechanized system genuinely does displace a large volume of routine human effort — this is real, and it is the source of the productivity gain. Second, a residue of human judgment remains, usually the hardest and least codifiable part, and it becomes more concentrated and more consequential per person than it was before. Third, the accounting frame credits the machine and loses the residue.

The third part is the problem. Not the second. There is nothing wrong with a system that needs skilled human judgment at particular points; that describes almost every valuable system humans have ever built, including the airline industry, the practice of medicine, and the capital markets. The problem is a system that needs skilled human judgment at particular points and has no record of that fact.

What is different about the current wave is the ratio and the ambiguity. Previous automation displaced physical or clerical effort and left judgment behind, which made the boundary reasonably legible: the robot welds, the technician decides when the weld is wrong. Large language models produce artifacts that look like judgment. They generate the memo, the analysis, the recommendation, the code. The output arrives in the shape of a finished decision. This makes the remaining human contribution far harder to see, because it is no longer "the machine did the labor and the person decided." It is "the machine produced something decision-shaped and a person decided how much of it to believe."

That second thing is much more valuable, much more difficult, and much easier to erase.

Part II — The Anatomy of a Ghost

Abstraction is the enemy here. If you want to find the invisible human judgment in your own organization, you need a taxonomy specific enough to hunt with. What follows is drawn from the shapes we see repeatedly in enterprises that have deployed AI at any real scale. Most organizations running a serious program have at least five of these. Very few have any of them on an org chart under a name that describes what the person actually does.

The Annotator

Someone decided what "good" looks like. If you fine-tuned, someone produced the examples. If you evaluated, someone wrote the rubric and applied it. If you used a foundation model as-is, someone at the vendor did this, and your system inherited their definition of acceptable — including the parts of that definition that do not match your industry, your risk posture, or your regulator.

This is the most consequential invisible role in the stack, because it sets the objective. It is also the one most often treated as a procurement line item or a one-time project rather than an ongoing function. The annotator's judgment is baked into behavior and then forgotten, which means that when behavior drifts from what the business needs, nobody knows where to go to change it.

The Prompt Author

Somewhere in your codebase, or in a spreadsheet, or in a Confluence page, or in one person's head, there is a body of instructions that determines how the model behaves in your context. It encodes domain knowledge, tone, escalation rules, formatting requirements, and a long tail of hard-won corrections. It was probably written iteratively by one or two people who discovered through failure what needed to be said.

That artifact is institutional knowledge of real value, and in most organizations it is versioned worse than the marketing copy. The prompt author is a ghost because the work reads as configuration rather than as expertise. It is expertise. Ask what happens to your system's behavior when that person leaves.

The Last-Mile Editor

This is the most common ghost and the easiest to verify. Find someone whose workflow includes receiving model output and doing something to it before it goes anywhere consequential. Ask them what they change. You will get a precise, detailed, and slightly weary answer, because they have been doing it for months.

They rewrite the opening because the model's register is wrong for the audience. They delete the confident sentence about the thing the model cannot know. They fix the number that is right in general and wrong for this client. They add the caveat that legal requires and the model omits.

Every one of those edits is a captured requirement that nobody has captured. The editor is running a quality function that does not exist in any process document, absorbing the difference between the model's output and the organization's standard, and doing it invisibly enough that leadership's mental model of the system is "the AI writes our first drafts" rather than "the AI and Priya write our first drafts, and Priya is the reason they are usable."

The Exception Handler

Automated processes have distributions, and distributions have tails. Somebody works the tail. In a claims process, it is the adjuster who gets the cases the model routes to manual review plus the cases where the model was confidently wrong. In a procurement flow, it is the buyer who handles the vendors the system cannot classify.

Exception handling is where the difficult cognition in an automated process concentrates. As automation improves, the volume of exceptions falls and their average difficulty rises. This is a well-understood dynamic and it has an uncomfortable implication: successful automation makes the remaining human work harder, more specialized, and more consequential per case, while simultaneously making it look less important because there is less of it. Organizations frequently respond to a falling exception volume by reducing the seniority of the people who handle exceptions. This is precisely backwards.

The Escalation Absorber

Related, but distinct, and almost never named. When an automated system produces a bad outcome for a customer, an employee, or a regulator, a person absorbs that. They apologize, they explain, they make it right, they rebuild the relationship. Frequently they cannot explain what happened, because the system is opaque to them too, so they absorb the failure using their own credibility as the currency.

This is real labor with a real cost, and it is charged against an account nobody monitors: the trust the organization has accumulated with its counterparties. It shows up in attrition among the people doing the absorbing long before it shows up in a metric anyone reviews.

The Integrator

Somebody makes the model's output actually reach the system where work happens. Often this involves an unglamorous mesh of scripts, transformations, retries, and reconciliation logic that encodes dozens of small decisions about what to do when things do not line up. The integrator's judgment is embedded in code and therefore feels like engineering rather than judgment, which is exactly why it is invisible. Ask what the fallback behavior is when the model times out mid-batch. Somebody chose that. Do you know who, or why?

The Taste-Setter

In domains where quality is real but not fully specifiable — brand voice, design, advisory judgment, clinical reasoning, deal instinct — someone holds the standard. They may never touch the system directly. Their function is to look at outputs and say "no, not like that," and to be right often enough that the organization defers.

Taste is the hardest contribution to make visible because it resists articulation, and the hardest to replace because it is generally acquired over a career rather than trained in a quarter. It is also the contribution most likely to be dismissed as subjective by people building systems that optimize measurable proxies. When the taste-setter leaves, quality does not fall off a cliff. It erodes at a rate slow enough that no single quarter shows it and no one is held accountable.

The Trust Broker

Finally, and most underrated: someone's personal credibility is why colleagues use the system at all. Adoption in a large organization is not primarily a function of interface quality. It is a function of whether a respected person in the relevant domain has vouched for the thing. The trust broker's contribution is that the tool is used. Remove them and usage decays, which will be recorded as a change management problem rather than as the loss of a specific person's endorsement.

Read that list again with your own program in mind. The exercise is not academic. In most enterprises, naming those eight roles takes about two hours of conversation and produces a list of between four and twenty people whose judgment the system depends on and whose contribution is not documented anywhere. That list is the most useful artifact you will produce this quarter.

The org chart says the process is automated. The people who make it work know better, and they have never been asked.

Part III — The Measurement Illusion

Now the consequences, starting with the one that damages decision-making most directly.

If human judgment inside an AI-mediated process is invisible, then every measurement you take of that process is measuring something other than what you think it is measuring. This is not a rounding error. It systematically distorts four things at once.

It inflates capability. A pilot that succeeds with a senior analyst reviewing every output is evidence that the model plus that analyst produces good work. It is not evidence that the model produces good work. Organizations routinely conduct the first experiment and record the second conclusion, then scale to a context where the analyst is absent or junior, and are surprised when quality collapses. The pilot was never wrong; the attribution was.

This is the mechanism behind a statistic that ought to be more alarming than it is. Boston Consulting Group's research finds that only about 26% of companies have moved beyond pilots to generate meaningful value from AI. Most of the industry reads that as a scaling problem — an issue of infrastructure, integration, or change management. Some of it is. But a substantial portion is an attribution problem that only becomes visible at scale. The pilot's results were produced by a configuration that included a specific, skilled, motivated, closely-attending human. The rollout replicated the model and not the human. Then the results did not replicate, and the program was recorded as failing to scale.

It hides cost. If the last-mile editor spends ninety minutes a day fixing output, that is a real operating cost of the system, and it is charged to a headcount line unconnected to the AI program's budget. The program's reported cost is the license, the compute, and the implementation. Its actual cost includes a fraction of several salaries that nobody has apportioned. Business cases built this way are not conservative or aggressive; they are simply not measuring the thing.

The same distortion runs in reverse when organizations do try to account for it and find the numbers uncomfortable. BCG's finding that leading companies direct roughly 70% of AI resources to people and process, and only 30% to technology and algorithms, is often read as advice about how to allocate a budget. It is better read as a description of where the cost actually is, whether or not you have chosen to see it. The 70% is being spent in most organizations. In the unsuccessful ones it is being spent unintentionally, by people who were hired to do something else.

It corrupts the improvement loop. When the editor fixes the same class of error every week, that is signal. It is the highest-quality evaluation data the organization possesses: a real expert, on real work, identifying real defects, continuously. In an organization where the editing is invisible, that signal is discarded daily. The model never learns what Priya knows, because nobody has built a path from Priya's corrections back to the system, and nobody has built that path because nobody has acknowledged that the corrections are happening.

This is the most maddening part of the pattern, because the fix is cheap and the value is high. An organization that captures last-mile edits as structured evaluation data converts an invisible cost into a compounding asset. Most do not, and then commission a project to acquire evaluation data.

It rewards the wrong things. Measurement drives attention, and attention drives careers. When results are attributed to the model, the people who advance are the people associated with the model: the technologists who built and deployed it. The people whose judgment made the output usable are, at best, invisible, and at worst, a friction line in a slide about adoption. Over two or three cycles, this reliably teaches the organization that being a ghost is a bad career decision, and the people capable of that work go do something else. The capability degrades for reasons that appear entirely unrelated to AI.

Put these four together and you get the shape that shows up in survey after survey: enormous investment, widespread pilot activity, persistent inability to convert, and a rising rate of abandonment — S&P Global found the share of companies scrapping most of their AI projects jumped to 42% in a single year, up from 17%. Some of those projects deserved to die. But a meaningful share were killed because they were measured against a fiction and could not meet it.

The fix is not more measurement. It is measuring the system as it actually exists, which means measuring the humans in it as first-class components rather than as unrecorded overhead.

Part IV — The Economics of an Invisible Line Item

It is worth working the arithmetic, because the abstraction hides how large the distortion gets and how quickly it flips a decision.

Take a document-generation use case of the kind almost every enterprise now runs: client-facing summaries, produced at volume, where quality matters because the audience is external. Say the process produces four hundred documents a month. Before AI, an analyst drafted each one in ninety minutes. After deployment, the model drafts and an analyst reviews, and review takes twenty-five minutes.

The business case writes itself and gets approved in one meeting. Ninety minutes to twenty-five is a seventy-two percent reduction in analyst time. Against a fully loaded analyst cost, the savings dwarf the license and compute. The program is recorded as a success and becomes the reference case for the next three deployments.

Now add the ghosts, which were present the entire time.

The twenty-five minutes of review was measured during the pilot, and the pilot used two senior analysts who had been involved in the tool's design. At scale, review is performed by a mixed population, and for the less experienced half it takes forty minutes, because they are less certain what to change and slower to find it. That is not a failure of the model. It is the pilot's human configuration failing to replicate — and it moves the average review time from twenty-five to roughly thirty-three minutes before anything else is counted.

Then there is the person who maintains the prompt. Call it four hours a month of genuine work — investigating a new failure mode, adjusting instructions, re-testing. It is done by a senior person, in the evenings, and charged nowhere.

Then the escalation absorber. Roughly two percent of documents reach a client with an error the reviewer missed. Eight documents a month, each requiring an apology, a correction, and a relationship repair conducted by a client-facing lead. Call it two hours each. Sixteen hours a month of senior client-facing time, charged to account management, invisible to the AI program.

Then the integrator, who spends a few hours most months on the pipeline: a schema change upstream, a timeout, a batch that half-completed and needed reconciling.

Add it up and the true operating cost of the process is materially higher than the reported cost — and, more importantly, its composition has changed in a way that matters strategically. The savings are real but roughly a third smaller than claimed. And the residual cost has migrated from a large pool of interchangeable junior drafting time into a small pool of senior, hard-to-replace, single-threaded attention.

That second fact is the one that should change your decisions, and it is completely absent from the business case. You have not merely reduced cost. You have concentrated your dependency. The process is now cheaper and considerably more fragile, and the fragility sits on four named people nobody has identified as critical.

Notice also what happens to the improvement path when the numbers are recorded honestly. If the dominant remaining cost is thirty-three minutes of review across four hundred documents, then the highest-value engineering work in the entire program is reducing review time — which means understanding what reviewers change and eliminating the codifiable half of it. That work is invisible and unprioritized in the fictional version of the accounts, and obvious in the honest version. Organizations with honest accounts find these improvements. Organizations without them go looking for a better model.

None of this argues against the deployment. The deployment was correct. It argues that the deployment was cheaper, more fragile, and more improvable than anyone recorded, and that all three facts were knowable.

You did not remove the cost. You moved it somewhere nobody is counting, and concentrated it on people nobody has named.

Part V — The Governance Vacuum

The second consequence is that you cannot govern a ghost, and this is where the stakes stop being economic and start being legal.

Accountability is a relation between an outcome and a person. It is not a property of a process, a policy, or a system. When a decision goes wrong and the question is "who was responsible," the answer must resolve to a human being with a name, a role, and — critically — the authority and information required to have decided otherwise. Every serious accountability regime, in every domain, works this way. Aviation assigns a pilot in command. Medicine assigns an attending physician. Banking assigns a model owner. Corporate law assigns fiduciary duty to identified officers and directors.

An organization with invisible human judgment inside its automated decisions has broken this relation in a specific and dangerous way. There is a human whose judgment shaped the outcome. That human is simply not the one the accountability framework names. The framework names a business owner who has never seen the prompt, or a technology owner who does not understand the domain, while the actual determination was made by an analyst exercising undocumented discretion at 4pm on a Thursday under time pressure with no guidance.

When such an outcome goes badly, three failures compound. The organization cannot explain what happened, because the decisive judgment was never recorded. It cannot demonstrate that reasonable care was taken, because the care that was taken was informal. And it frequently discovers that the person who made the call had neither the authority nor the mandate to make it, which converts an operational error into a governance finding.

This matters more each year, because "a human was involved" is becoming a legal load-bearing claim rather than a reassuring aside.

The EU AI Act requires, for high-risk systems, that human oversight be genuine — that the people assigned to oversee a system be able to understand its capabilities and limits, remain aware of automation bias, correctly interpret output, and decide not to use it or to override it. Read that list as a specification and it becomes obvious that it cannot be satisfied by a ghost. It requires a named person, with training, with authority, with time allocated to the task, and with a record.

Financial regulators arrived here earlier. The Federal Reserve and OCC's model risk management guidance has for over a decade required identified model owners, documented limitations, effective challenge by parties with the competence and standing to challenge, and independent validation. The phrase "competence, influence, and incentives" appears in that guidance precisely because supervisors understood that oversight without authority is theater. An organization that has not named its human contributors cannot produce the artifacts that regime requires. It does not have them.

The pattern generalizes. Wherever regulation touches automated decisions — credit, employment, insurance, clinical care, safety-critical systems — the requirement converges on the same thing: an identified human with real authority and a record of their reasoning. Every jurisdiction is, in effect, legislating against ghosts.

There is a quieter internal version of this problem that costs organizations more in aggregate than any regulatory finding. Undocumented human workarounds accumulate risk. The analyst who has learned to correct a systematic model error is, from a risk perspective, a control. That control is undocumented, untested, unmonitored, single-threaded, and dependent on one person's continued attention and employment. No risk function would knowingly accept a control with those properties. Many organizations have dozens.

A control that depends on one undocumented person paying attention is not a control. It is a hope with a payroll number.

Part VI — Why Loops Rubber-Stamp

There is an uncomfortable finding buried in the human factors literature that every organization designing oversight needs to confront, because it determines whether the loop you build will function or merely exist.

People are bad at supervising automation that is usually right.

This is not a character flaw and it is not solved by training or by exhortation. It is a well-documented feature of human attention, studied since the 1940s, when researchers examining radar operators found that vigilance degrades sharply within about half an hour of monitoring for rare signals. The effect has been replicated across domains for eighty years. Humans are excellent at active problem-solving and poor at sustained detection of infrequent anomalies in a stream that is mostly fine.

Layered on top of this is automation bias: the tendency to over-trust automated output and to under-search for disconfirming evidence when a system has provided an answer. Aviation research has documented both directions of the failure — errors of commission, where a crew follows an automated recommendation despite contrary evidence available to them, and errors of omission, where they fail to notice a problem the automation did not flag. Crucially, the effect strengthens as the system's reliability increases. A system that is right 99% of the time produces far more automation bias than one that is right 80% of the time, because the reviewer has been trained by experience that checking is a waste of effort. They are, in a narrow sense, responding rationally.

Put these together and you get the central design problem of human-in-the-loop oversight: the better your model gets, the less effective your human reviewer becomes. Reliability and oversight quality move in opposite directions. Organizations that improve their model and keep their oversight design constant are quietly converting a functioning control into a ceremonial one, and the transition is invisible because the metric they watch — error rate — is improving throughout.

This is why "we have a human in the loop" is not a governance answer. It is the beginning of a question. A loop staffed by someone reviewing four hundred nearly-identical outputs a month, ninety-eight percent of which are fine, under time pressure, with no consequence for approving and friction for rejecting, will approve almost everything. You have built a log, not a control, and you have built it while satisfying every box on the checklist.

Designing a loop that actually holds requires working against these dynamics rather than ignoring them, and there are five moves that help.

Sample rather than review everything. Deep scrutiny of a random ten percent detects systematic problems far better than shallow scrutiny of one hundred percent, because it preserves the attention budget for cases where it can function. Full review is usually a false economy that produces the appearance of control at the cost of its substance.

Route by consequence, not by volume. Reserve human-in-the-loop for the decisions where a single error is expensive or irreversible. Put everything else on the loop. This is what makes attention affordable where it matters.

Make disagreement cheap and visible. If overriding requires a justification memo and approving requires a click, you have priced disagreement out of the market. Override should be one action, logged, with no penalty and no friction. The override rate is your most valuable telemetry; do not tax it.

Give the reviewer something to check against. "Does this look right" is a task humans perform badly. "Does this figure reconcile with the source, is the disclaimer present, does the recommendation follow from the stated evidence" is a task humans perform well. Specific checks defeat vigilance decay; general impressions do not.

Measure the loop itself. Inject known-bad cases periodically and see whether they are caught. This feels adversarial and is standard practice in every mature safety-critical domain, because it is the only way to know whether oversight is functioning. An organization that has never tested its reviewers does not know whether it has a control.

The broader point is that a human in the loop is not a safety property you acquire by assigning a person. It is an engineering outcome that depends on how you have arranged the person's attention, authority, and incentives — and it can be designed well or badly, tested or assumed. Most organizations assume.

Reliability and oversight pull against each other. The better the model gets, the harder you have to work to keep the human real.

Part VII — What It Costs You in Talent

The third consequence is the one that eventually determines whether any of this is fixable, because it operates on the supply of people capable of doing the work.

Roles that do not exist cannot be hired for. This sounds tautological, and it is the practical crux. If the function of "person who owns the quality of AI-mediated output in this domain" is real but unnamed, then it is not in a requisition, not in a compensation band, not in a competency framework, and not in anyone's development plan. The organization fills it accidentally, by discovering that a particular person is good at it, and then quietly relying on them.

Four things follow, and they compound.

Key-person risk that nobody has priced. The person who holds the prompt, the taste, or the exception knowledge is load-bearing. Their departure degrades a process that leadership believes is automated, which means the degradation is not anticipated, not detected quickly, and not correctly diagnosed when it is. We have watched organizations spend two quarters investigating a quality decline that was fully explained by one resignation, because nobody had modeled that person as a dependency. In a documented process, that resignation triggers a handover. In a ghost process, it triggers a mystery.

Compensation drift, in the direction that loses you the person. Invisible contributions are hard to reward, because reward systems run on documented scope. Meanwhile the market for people who can genuinely operate at the seam between AI systems and business consequence has tightened considerably. The result is predictable: your most valuable ghost is underpaid relative to their market value in a role that does not appear on their résumé — until they write it there themselves, at which point they discover their own price. Organizations routinely lose these people for fifteen percent, having extracted a contribution worth many multiples of that.

Career logic that pushes capability away. Ambitious people read the promotion signal accurately. If the visible AI wins accrue to the technology organization, then the message to the domain expert who is quietly making those wins possible is that this work is a detour. They will optimize accordingly — either by moving toward the visible work, or by leaving. Either way the organization loses the specific combination of domain depth and system fluency that is hardest to hire and most necessary to have.

A hiring profile you cannot write. When the role finally becomes visible enough to recruit for, the organization has no vocabulary for it, because it never described the work while it was being done. Job descriptions come out as a shopping list — machine learning plus domain expertise plus change management plus stakeholder management plus governance — that describes four people and attracts candidates who are strong in the most legible dimension rather than the most necessary one.

This last point is where we spend most of our professional time, so let us be concrete about what actually distinguishes candidates for this work, since it is rarely what the requisition emphasizes.

The scarce quality is not technical depth in machine learning. It is calibrated judgment about when to trust a system, held by someone with enough domain authority that their judgment is deferred to and enough technical fluency that they can articulate why. That combination is uncommon, and it is frequently found in people whose backgrounds do not read as AI backgrounds at all: operators who have run a function and know where its decisions actually get made, analysts who have spent years being accountable for numbers other people acted on, program leaders who have shipped things across organizational boundaries.

Hiring for the model and hoping judgment shows up is the standard error. Hiring for judgment and building technical fluency around it is slower to feel modern and substantially more likely to work.

Part VIII — Why Exorcism Is the Wrong Goal

At this point a particular reflex kicks in, and it is worth naming because it is the most common wrong turn.

The reflex is: if invisible human dependency is the problem, remove the human. Automate the annotation. Automate the editing. Use a model to check the model. Get to a genuinely autonomous system and the ghost problem dissolves.

This is appealing, it is occasionally correct, and as a general strategy it fails for four reasons.

The judgment is often the product. In advisory work, clinical reasoning, credit decisions, and executive communication, the discrimination the human is applying is the value being delivered. Automating it does not remove overhead; it removes the thing the customer is paying for. Organizations that have done this discover it through churn rather than through metrics, and later than they would like.

Autonomy is less available than the discourse suggests. Deloitte's research has found only about 4% of enterprises pursuing AI actively implementing agentic systems, and Gartner estimates that agentic deployments can cost two to five times more than traditional machine learning projects. Those figures describe a capability that is real, advancing, and considerably narrower and more expensive than the roadmaps assume. Planning your governance model around the arrival of autonomy is planning around a date you do not control.

Automating oversight relocates the ghost rather than removing it. If a model now checks the model, someone specified the checking criteria, someone decides what happens on disagreement, and someone monitors whether the checker is still calibrated. You have not eliminated human judgment. You have moved it one level up, made it more abstract, made it apply to a much larger volume of decisions, and — because it now looks like infrastructure — made it considerably harder to see. This is a worse position than the one you started in, arrived at while feeling like progress.

It optimizes away your evaluation signal. The human corrections in your process are your best source of truth about where the system is wrong. Automate them and you get efficiency now in exchange for blindness later. You will notice the blindness during your first serious incident.

The alternative is not to keep humans in the loop indefinitely out of caution. It is to make the human contribution explicit, and then make deliberate, decision-by-decision choices about which posture each decision has earned — human in the loop, human on the loop, or human out of the loop. Those choices are legitimate and they differ across a single workflow. What is illegitimate is arriving at the third posture by drift, which is how most organizations actually arrive there.

Resolve the ghost. Then decide. In that order.

Part IX — Three Composites

Patterns are easier to recognize in the particular. The following are composites — assembled from engagements, with details changed — chosen because each shows a different way the ghost problem presents and a different fix.

The summariser that got worse for six months

A specialty insurer deployed a model to summarise loss-run histories for underwriters. Adoption was strong, cycle time improved, and the program was held up internally as the template.

Fourteen months in, quality complaints began arriving from the field. Not a spike — a drift. Underwriters reported that summaries were "missing things," without agreeing on what. Engineering investigated the model, confirmed no change in version, weights, or configuration, and found no measurable degradation on the evaluation set. The evaluation set had been assembled at launch.

What had changed was a person. The senior underwriter who had spent the first year reviewing summaries before they entered the workflow — and who had, without telling anyone, been correcting a systematic weakness in how the model handled non-contiguous coverage gaps — had moved to a different division. The review step had been formally described as a temporary quality gate during rollout, so his departure was not treated as a change to the system. Nobody replaced the function because on paper the function had ended.

The fix took a quarter and had three parts. His corrections were reconstructed from email and turned into fourteen evaluation cases, which the model failed. Half the failures were resolved by explicit prompt instructions. The remainder were genuinely irreducible, and the review step was reinstated as a permanent, named, staffed seat rather than a rollout artifact.

The instructive detail is that the organization spent two quarters investigating a technology problem that was fully explained by one person changing jobs — and that the information required to diagnose it had existed in his sent-mail folder the whole time.

The pilot that could not be repeated

A healthcare services firm ran a six-week pilot using a model to draft prior-authorisation appeal letters. Results were excellent: approval rates on appealed cases rose sharply, and the pilot team wrote a confident business case for enterprise rollout.

The rollout underperformed badly enough that the program was nearly cancelled.

The pilot had been run by two clinicians with unusual profiles: both had prior payer-side experience, and both therefore knew which clinical arguments a specific reviewer population found persuasive. They were not editing the model's grammar. They were substantively restructuring the argument on roughly two thirds of letters, drawing on knowledge neither had ever been asked to document, because nobody had realised they were doing anything other than proofreading.

The rollout replaced them with a larger pool of clinically competent staff who lacked payer-side experience. The model was identical. The letters were fluent, accurate, and considerably less persuasive.

The recovery was not technical. The two clinicians spent three weeks being interviewed about how they decided what to argue, which produced a structured argument taxonomy that went into the prompt and into a training module. Approval rates recovered to roughly eighty percent of pilot levels. The remaining twenty percent was, on honest assessment, irreducible: it lived in two people's accumulated judgment, and the organization made a deliberate decision to route the highest-value appeals through them.

This is the 26% statistic in miniature. The pilot did not fail to scale for infrastructure reasons. It scaled the model and not the humans, because nobody had noticed the humans were the intervention.

The control that was a hope

A mid-size bank had deployed a model to assist in commercial credit memo preparation, with a documented human review step and a named model owner — a properly governed program by most standards.

An internal audit reviewed the control and asked a question the program had not anticipated: in the previous twelve months, how many times had the reviewer materially changed the model's output, and what did those changes concern?

Nobody knew. The reviewer's changes had never been logged, because the review happened in the document rather than in the system. Reconstruction from version history showed an override rate that had fallen from about eleven percent in the first two months to under one percent by month nine. The reviewer, interviewed, was candid: the model was almost always fine, the queue was long, and he had stopped reading closely some months earlier.

The model had not degraded. The control had. And the metric the program monitored — error rate — had looked good throughout, because the errors that got through were not being detected by anyone.

The remediation was a redesign along the lines described in Part VI: sampling instead of full review, consequence-based routing, one-click logged override, a specific checklist replacing general impression, and periodic injection of known-bad cases to test whether the reviewer was functioning. Override rate became a monitored metric with an alerting floor, on the reasoning that an override rate near zero is not evidence of a good model but evidence of an absent reviewer.

Three different presentations, one shared structure. In each case a person's judgment was load-bearing, undocumented, and treated as either temporary or clerical. In each case the failure was diagnosed initially as a technology problem. And in each case the information required to prevent it was already in the building, held by someone nobody had interviewed.

Part X — The Method: How to Name the Ghosts

Everything above is diagnosis. This is the part that changes anything. What follows is the sequence we use, and it is deliberately unexotic: most of the difficulty is in doing it at all rather than in doing it cleverly.

1. Run the ghost audit

Take one AI-mediated process that matters. Follow it end to end and ask, at every step, "whose judgment shapes what happens here?" Not "who is assigned to this step" — whose judgment.

Interview the people actually doing the work, not their managers, and ask specific questions. What do you change about the output before you pass it on? How often? What made you learn to do that? What would happen if you were on leave for three weeks? What does the system get wrong that you have stopped reporting because nothing happened last time?

That final question is the highest-yield question in this entire exercise. The answer is always yes, there is something, and it is always informative.

Budget a week. Expect to produce a list of four to twenty named individuals and a description of what each contributes. Expect at least one genuine surprise about where the load actually sits.

2. Classify each contribution

For each contribution, decide which of three things it is. This determines what you do about it.

Codifiable — a rule that can be written down and enforced in the system. The model omits a required disclaimer; add it to the template. Much of the last-mile editing load resolves here, and this is where the fastest returns are. Organizations regularly find that a third to a half of manual correction is a handful of missing rules.

Signal — a correction that reflects genuine expert judgment and should be captured as evaluation data rather than eliminated. Build the path from the correction back into your evaluation set. This converts an invisible cost into a compounding asset and is the single highest-leverage move available to most programs.

Irreducible — judgment that genuinely requires a person, because it depends on context the system cannot have, accountability the system cannot hold, or standards nobody can fully specify. This does not get automated. It gets named, staffed, resourced, and put on the org chart.

The discipline is in the third category. The temptation is to classify everything as codifiable eventually, which converts an org design decision into an indefinite roadmap item and leaves the ghost in place while you wait.

3. Name the seat

Every irreducible contribution gets an owner with a name and a defined scope. Not a committee. A person, who can be identified in a sentence: "Maria owns the quality and escalation of AI-assisted underwriting summaries for commercial lines."

The seat requires four things to be real, and the fourth is where most organizations fail. Explicit scope, so it can be handed over. Time allocated, so it is not unpaid work on top of a full job. Authority to stop, because a reviewer who cannot halt a rollout is not a control. And standing, meaning the organization actually defers to them — which is a function of seniority and reputation, not of a title.

Assigning a junior person to oversee a system used by executives satisfies the audit and accomplishes nothing.

4. Choose the loop posture, decision by decision

For each consequential decision class in the process, choose explicitly: in the loop, on the loop, or out of the loop. Write it down. Put a review date on it.

The point is not that the strictest posture is always right. It plainly is not; applying human-in-the-loop review to millions of low-stakes reversible decisions produces rubber-stamping, which is worse than an honest human-on-the-loop design because it manufactures a false record of oversight. The point is that the choice is made, recorded, and revisited — rather than inherited from whatever the implementation happened to do.

5. Instrument the human contribution

Measure what the humans in the system are doing. Override rate and its direction. Time spent per item on correction. Categories of correction, trended. Exception volume and exception difficulty. Escalation load and who absorbs it.

These metrics do two jobs. They make the real cost of the system visible, which lets you make honest investment decisions. And they surface degradation early: a rising override rate is the earliest available signal of model drift, and it is free, and almost nobody collects it.

6. Establish the cadence

Judgment needs a rhythm or it decays into habit. Daily escalation path for anomalies. Weekly review of overrides and corrections, attended by someone with authority to change the system. Monthly recalibration of what is codifiable, what is signal, and what remains irreducible. Quarterly re-examination of loop postures.

Without cadence, the audit you just ran becomes a document. With cadence, it becomes a function.

7. Close the talent loop

Now that the roles exist, treat them as roles. Write the job description. Put it in a compensation band that reflects that the person is load-bearing. Build a development path so the work is a career rather than a detour. Establish succession, because you now know these seats are single points of failure. Hire deliberately for the combination of domain authority and system fluency, and look for it in operators and analysts rather than only in people whose titles already say AI.

The order matters. Steps one through six make the work visible. Step seven is what makes it survivable.

Part XI — Objections

The argument above attracts four serious objections. They deserve straight answers rather than dismissal, and two of them are partly right.

"This is just a bureaucratic tax on a working program."

It would be, if it stopped at documentation. A ghost audit that produces a binder is a waste of a week. The audit is only worth running because of what follows it: the codifiable third that gets fixed and removes real cost, the correction signal that becomes evaluation data and compounds, and the key-person risk that gets identified before it fires rather than after.

The test is whether the exercise changes anything within a quarter. If it produces a taxonomy and no prompt changes, no evaluation cases, no named seats, and no altered pay band, then the objection is correct and the exercise failed. Judge it on that.

"Naming these roles ossifies work that should be temporary. You are institutionalising a stopgap."

This is the strongest objection, and it is right about the risk. There is a real danger in giving a permanent title to work that a better model would eliminate in nine months, because roles defend themselves once they exist.

The answer is the classification step, done honestly, with review dates. Codifiable work is explicitly marked for elimination and someone owns eliminating it. Signal work is expected to shrink as the evaluation loop closes. Only irreducible work gets a permanent seat, and every loop posture carries a date on which the question gets reopened.

What is not defensible is the alternative that this objection implies: leaving the work unnamed on the theory that it will go away. That does not keep the organization flexible. It keeps the organization ignorant, and ignorant organizations cannot tell the difference between work that is about to become unnecessary and work that is about to become critical.

"Our models are good enough now that this is a legacy concern."

Part VI is the response. Model quality and oversight quality move in opposite directions. A better model does not reduce the importance of getting the loop right; it raises it, because it increases automation bias, lowers override rates, and makes the residual errors rarer and therefore harder to catch and more likely to be consequential when they escape.

There is also a compositional point. As models improve, organizations extend them to higher-stakes decisions. The judgment burden per decision goes up even as the error rate goes down. The ghost does not shrink with capability. It moves upmarket.

"You are a search firm. Of course you think the answer is people."

Fair, and worth stating plainly: we have a commercial interest in the conclusion that human capability matters. Read the argument with that in mind.

Two things partly answer it. The first is that a meaningful share of the method above involves no hiring at all — the audit, the classification, the codification of the codifiable third, the evaluation loop, the loop-posture design, the override instrumentation. If you did only those, you would capture most of the value and we would earn nothing from it. We would still recommend it.

The second is that the argument cuts against our interest in one important respect. The most common version of the conclusion we are supposedly motivated to reach is "hire a Chief AI Officer" — a large, visible, expensive search. What we are actually arguing is that the binding constraint is usually a small number of correctly-defined seats filled by people who frequently already work for you, in operations and analysis, and who need authority and a band rather than a new title from outside. That is a smaller commercial opportunity than the one the hype cycle offers, and it is what the evidence supports.

Part XII — What Changes for Leaders

Different chairs, different implications.

For the CEO and the board. Stop accepting "AI did it" in any report you receive. The question to ask in a review is not "what did the model achieve" but "who was in the loop, what did they change, and what did their changes tell us." When capability is described, ask what human configuration produced the evidence, and whether that configuration exists at the scale being proposed. This one habit will do more for the quality of your AI reporting than any dashboard.

Also: ask which people the program depends on, by name, and what happens if each leaves. If nobody can answer, you have found an unpriced risk.

For the CIO and CTO. The engineering discipline here is treating human judgment as a component of the system rather than as an external actor. That means logging overrides and corrections as first-class events, building the return path from correction to evaluation set, defining and documenting fallback behavior, and versioning prompts and rubrics with the seriousness applied to code. If your architecture diagram has no humans on it, it does not describe your system.

For the COO and functional leaders. The ghosts are almost entirely in your organization, and you are paying for them out of headcount already. The upside of making them visible accrues to you: you get accurate unit economics, an early-warning signal on quality, and the ability to staff the work intentionally instead of hoping the right person keeps volunteering.

For the CHRO. This is an org design and job architecture problem before it is a training problem. The work is naming seats that do not exist, banding them against a market that is repricing them quickly, building competency definitions for judgment under system-mediated uncertainty, and identifying internal candidates whose profiles do not read as technical. Your analysts and operators are the largest available supply of the scarcest capability in the enterprise, and most of them have not been asked.

For risk, audit, and legal. Every undocumented human control is a finding waiting to happen and, more usefully, a real exposure. The ghost audit produces exactly the artifact your obligations increasingly require: named oversight, documented authority, recorded reasoning, and evidence of effective challenge. Run it before someone runs it for you.

Conclusion — Give the Ghost a Name

The confusion Ryle was attacking was the belief that you need an invisible operator inside a mechanism to explain the mechanism. His objection was that the invisible operator is a symptom of a bad description.

He was right, and the lesson transfers exactly. If you cannot explain your AI system's results without appealing to something you have not documented, the problem is your description. There is no mystery in the machine. There is a person in there, doing difficult and valuable work, and your account of the system has failed to mention them.

Fixing the description is unglamorous and it is most of the job. It does not require a new model, a larger budget, or a wait for autonomy to mature. It requires walking a process, asking people what they actually do, writing it down, deciding what is a rule and what is expertise, putting names on the expertise, giving those names authority and a pay band, and reviewing all of it on a rhythm.

Organizations that do this get several things at once, and they compound. Honest unit economics, because the real cost is on the books. A functioning improvement loop, because expert corrections become training signal instead of evaporating. Governance that survives contact with a regulator, because oversight resolves to a person with a mandate. Retention of the people who are actually creating the value, because those people can finally see themselves in the organization. And the ability to move a decision from human-in-the-loop to human-on-the-loop as a considered choice supported by evidence, rather than as a drift nobody authorized.

Organizations that do not will keep running the experiment that produces the 26% figure: real capability, real investment, real pilots, and a persistent inability to convert, diagnosed as a technology problem and treated with more technology.

The machine is not haunted. It is staffed. Find out by whom.

Appendix — The Ghost Audit

A working instrument. Take one AI-mediated process, walk it end to end, and ask these questions of the people doing the work rather than of their managers. Expect a week.

Questions for anyone who touches model output

• What do you change before this goes anywhere? Walk me through the last three.
• How often do you change something? What fraction goes through untouched?
• How did you learn that it needed changing? Were you told, or did you find out?
• What does it get wrong that you have stopped reporting, because nothing happened the last time you did?
• If you were on leave for three weeks, what would go wrong, and who would notice first?
• Is there anything you check that is not written down anywhere?
• What would you need in order to stop doing this manually?

Questions for the process owner

• Which decisions in this process are irreversible or externally visible?
• For each, is a human in the loop, on the loop, or out of the loop? When was that chosen, and by whom?
• Who has the authority to halt this process? Have they ever used it?
• What is the override rate, and what direction has it moved in over the last six months?
• What happens when the model is unavailable mid-batch? Who decided that?

Questions for the technology owner

• Where do the prompts and rubrics live, and who has changed them in the last quarter?
• When did the evaluation set last change, and what triggered the change?
• Are human corrections captured anywhere in a form an engineer could use?
• What is logged when a reviewer overrides? Anything?

What to produce

Four artifacts, none of them long.

1. A named list of every person whose judgment shapes an outcome, with one sentence each on what they contribute.
2. Each contribution classified as codifiable, signal, or irreducible — with an owner for the first two categories and a date.
3. A loop-posture decision for every consequential decision class, written down, with a review date.
4. A key-person register: for each irreducible contributor, what breaks if they leave and what the handover requires.

The one question that matters most

If you run out of time, ask only this, of everyone: "What does it get wrong that you have stopped reporting?"

The answer is never nothing. It is frequently the most important thing anyone will tell you this quarter, and the reason you have not heard it before is that no one asked.

---

Search & Strategy runs the Orchestration Practice: retained search for applied AI, data, and platform leadership, and embedded operators who make the loop real. If you want a clear-eyed conversation about the ghosts in your own systems, start a conversation.