The AI State

Imagine receiving a government notice that says your support payment will stop. The explanation is courteous. It names a discrepancy, gives a deadline, and directs you to an online service. You upload the document that resolves the discrepancy. The service confirms receipt. Another notice arrives with the same conclusion. A telephone agent can see the result but cannot change it.

This is a thought experiment, not an account of a particular claimant. Its question is practical: where, inside this efficient process, can the citizen reach someone with the power to reconsider?

Chapter Twenty-Two asked whether an institution could leave its AI provider. The citizen faces a different problem. A person can sometimes change a bank or employer. Someone contesting a tax assessment, seeking a permit, or depending on a statutory benefit cannot simply choose a competing government. Elections can change an administration. An appeal may correct a decision. Neither is equivalent to canceling a subscription before next month’s rent is due.

The AI state is government whose capacity to serve, classify, investigate, and act increasingly depends on machine assistance. Its promise is substantial. So is the duty created by that dependence. When exit is unavailable, the ability to challenge a decision becomes part of the service itself.

A Faster State Could Be a Better State

Anyone who has struggled with an unreadable form or an unanswered request can understand the appeal. AI could help an agency explain eligibility, translate instructions, locate missing records, reconcile duplicate applications, and give an employee a usable summary of a complicated file. Used carefully, it could reduce the administrative burden on both sides of the counter.

Delay has consequences of its own. A benefit delivered months late may fail the household it was meant to support. An inspection buried in a backlog may arrive after the danger has grown. Preserving a slow process merely because its failures are familiar would be a poor defense of accountability. The relevant comparison is with the service people actually receive, including its existing errors, discretion, and exclusions.

There is evidence for useful assistance. In a report published on July 9, 2026, HM Revenue and Customs described a Microsoft Copilot trial conducted from September to December 2024. It randomly allocated 3,000 licenses across selected departmental roles. The primary evidence came from a survey of 1,364 users, about 45% of that group, supplemented by other methods. Participants reported savings equivalent to roughly 60 minutes a week. The evaluation adjusted estimates for non-use and response bias and cautioned against treating them as precise impact measurements.

That is a worthwhile result with a limited meaning. Random allocation of licenses does not turn a self-reported time estimate into a direct measure of better public decisions. Even so, saving staff time on routine work creates an opportunity to improve service. The next question is whether the organization uses that opportunity to resolve difficult cases, improve accuracy, or merely increase the volume passing through the same narrow review process.

A serious public-sector evaluation should count the citizen’s time as well as the employee’s. If an office saves ten minutes by making an applicant spend two hours correcting its data, the apparent productivity gain is partly a transfer of work. If faster processing also reduces errors and makes help easier to reach, the benefit is much more convincing.

Where Assistance Becomes Authority

Government AI can perform very different jobs inside one application. It can explain a rule, recommend which file deserves attention, or supply the basis for an action against a person. The interface may look identical while the delegation changes underneath it.

Consider a hypothetical benefits assistant. It begins by locating published instructions. Later, it recommends which evidence an applicant should upload. Then its assessment of missing evidence determines which applications enter a slower queue. Eventually, that queue makes the difference between receiving support this month and waiting until next month. No screen needs to announce that the assistant has acquired consequential discretion.

Chapter Twelve called the point where machine output becomes state action the authority jump. In administration, that jump can occur before the final notice. A priority score can change who receives attention. A fraud flag can trigger demands for documents. A summary can exclude the fact that would have changed a reviewer’s conclusion. An agency must therefore examine the effect of each use, including the decisions shaped by what the system leaves out.

The public purpose also needs to be explicit. A fraud program can properly seek to recover money that should not have been paid. But selecting a target for recoveries, defining suspicious behavior, and deciding how much uncertainty justifies an investigation are governing choices. Software can apply those choices consistently while making a poorly chosen objective more damaging.

The question is what the government has authorized the system to make happen. Procurement documents and technical specifications should answer it in terms an accountable official can defend to the affected public.

The Agency Must Know What It Has Deployed

The IRS provides an institutional case. In March 2026, the U.S. Government Accountability Office reported that the agency’s June 2025 inventory contained 126 active AI use cases. Of those, 77 were still in development. An active inventory entry therefore did not necessarily describe an operational deployment.

GAO found inventory omissions, including contracted AI-enabled tools used to help build criminal cases. It also identified gaps in workforce planning and agency-wide management of AI investments. IRS agreed with all eight recommendations and described actions planned or underway. The report established governance weaknesses, not that every listed tool had made an unlawful decision.

The lesson reaches beyond the IRS. Officials need to know which tasks use AI, where it operates, and who can change its role. One office may keep a list of models while another buys software with AI already built in. If those records stay separate, some uses can escape the oversight that leaders believe covers the whole agency.

An inventory is the beginning of visibility. It should connect each consequential use to an accountable office, its purpose, the affected people, and the intervention path. A label such as customer service tells a reviewer very little. Does the tool retrieve published information, generate personalized advice, prioritize applications, or determine what evidence a person must produce? Each function creates a different relationship with the public.

There is also a human-capacity problem. A government can become more dependent on advanced systems while reducing its capacity to evaluate the work. Chapter Eighteen’s expertise problem returns here with public authority attached. Officials need enough technical and domain competence to challenge a supplier, recognize a change in performance, and keep essential services functioning when the preferred system fails.

The Cost of Proving the State Wrong

Chapter Four used Robodebt to show how accountability can fail inside an automated process. Its aftermath supplies a narrower lesson for this chapter: the state needs to be able to find and repair the full class of affected decisions.

The Commonwealth Ombudsman’s April 2021 report found that Services Australia lacked a system indicator identifying debts raised through income averaging. Staff had to categorize debts manually. The report recognized substantial work in administering refunds but criticized features of the identification and recovery process. Its recommendations and comments emphasized recording the basis of decisions and protecting people while disputed debts were examined.

Robodebt used an earlier form of administrative automation, not generative AI. Its repair problem still matters. To find everyone harmed by the same mistake, an agency needs to reconstruct how its decisions were made. Correcting only the cases that attract complaints can leave other people with the same problem, especially people who lack the time, records, confidence, or help to challenge it.

Suppose an AI system mistakenly treats one type of employment document as incomplete. After an appeal reveals the error, a competent response would search for other applications processed under that rule or model version. It would examine the consequences, notify affected people where appropriate, and correct the underlying process. The remedy should travel as far as the error.

That requires a decision record created before the dispute. The record should preserve the evidence used, the material rule or system version, the consequential recommendation, and the office that authorized reliance on it. This is a record of what the institution did and why, not a demand for a model’s hidden internal reasoning.

Review also needs a clock. An eventual correction may be inadequate when an interruption threatens housing, treatment, or subsistence. The book’s recommendation is to design a route for urgent reconsideration and, where lawful and appropriate, temporary protection while a serious dispute is resolved. The safeguard must fit the consequence rather than assume every appeal can wait in the ordinary queue.

A State That Can Infer More Must Explain Its Purpose

AI can make existing government information more searchable and more revealing. An agency may possess separate records that appear mundane in isolation. Linking them can produce a prediction about a person’s behavior, needs, or relationships. Chapter Fourteen examined inference as private power. Government adds the capacity to compel disclosure and act through public authority.

The case for some prediction is strong. Finding a likely safety violation can direct a scarce inspection team toward a real danger. Detecting a pattern of fraudulent claims can protect funds for eligible recipients. These uses still require a defined purpose and an evidentiary threshold appropriate to the action. A suspicion score may justify further checking without justifying a penalty.

There is a feedback risk as well. In a hypothetical inspection program, neighborhoods selected more often generate more inspection records. A model trained on those records could interpret the greater volume of discoveries as a reason to concentrate inspections there again. The agency should test whether the pattern reflects underlying conditions, the history of its own attention, or both. Counting confirmed findings alone may miss comparable problems in places the system rarely examines.

The boundary must also cover combining information and using it for a new purpose. Records collected to provide one service should not automatically be used for every other task AI makes possible. Before reusing them, the agency needs a defensible legal basis, a clear public purpose, and rules about who may see the records and how long they may be kept.

GAO’s March 26, 2026 privacy review gives this concern an independent institutional footing. It found that OMB guidance did not fully address 8 of 10 selected privacy challenges identified by an expert panel. The report recommended more specific guidance and information sharing; OMB supplied no comments on the draft. This was a finding about safeguards and guidance, not proof that a particular dataset had been exposed.

For the citizen, the practical issue is whether information given to obtain help quietly becomes a basis for suspicion elsewhere. A trustworthy AI state must be able to explain and constrain that transition.

Government Is Also a Customer

An agency buying an AI service occupies both sides of the accountability problem. It depends on a private supplier while exercising public authority over people who did not select that supplier. Chapter Twenty-Two’s control stack therefore matters even when the government seal appears on the screen.

The public institution needs contractual access to the information required for meaningful evaluation and review. It should know what changes the supplier can make, what records remain available after a contract ends, and how essential functions will continue during an outage or transition. A supplier’s confidentiality interests may justify controlled access. They should not leave every authorized reviewer unable to examine the basis of a consequential decision.

Federal purchasing policy recognizes parts of this problem. The Office of Management and Budget’s April 2025 memorandum M-25-22 addresses the practical information a supplier should pass to the agency, how data and models can be transferred, what the license permits, and how performance will be checked over time. These safeguards belong in the purchase agreement, before the agency becomes dependent on the supplier.

Implementation is harder than writing the requirement. In April 2026, GAO reported on 13 AI acquisitions across Defense, Homeland Security, GSA, and Veterans Affairs. The selected agencies were not systematically collecting acquisition lessons. All four agreed with recommendations to change their policies. The finding matters because one office’s experience with a weak testing clause or unexpected cost can protect another office only if the institution remembers it.

Government needs to remain an intelligent buyer. It also needs to remain the institution that answers the citizen when the bought service fails.

Current Evidence: Adoption and Answerability

Reviewed September 12, 2026.

The recent evidence supports useful adoption alongside closer scrutiny of implementation. HMRC’s July evaluation is a bounded case for administrative assistance. The IRS and acquisition reports show why deployment counts and procurement activity cannot establish that agencies possess the skills, records, or learning processes needed to govern the systems they use. These are different measures of progress and should remain separate.

OMB’s April 3, 2025 memorandum M-25-21 established a high-impact category for covered federal AI uses whose output principally supports decisions or actions with significant effects on rights or safety. It provides for testing, risk management, and discontinuation of noncompliant uses, while also allowing specified waivers and pilot exceptions. This is an executive-branch governance framework with defined scope, not a universal rule for all public-sector AI or evidence that every covered deployment complies.

The UK’s Algorithmic Transparency Recording Standard supplies another mechanism. Its mandatory policy covers specified central-government bodies and tools that significantly influence decisions with public effects or interact directly with the public. It includes exemptions for sensitive information. The scope extends beyond systems that make the final decision, recognizing that triage and scoring can materially influence an outcome.

A published record makes a system easier to question. It does not by itself provide a successful appeal, demonstrate fair results, or prove that a reviewer can intervene. The next evidence worth watching is implementation: whether agencies complete corrective actions, whether useful records are published, and whether people can obtain timely remedies when automated processes go wrong.

Accountability Analysis

Principles 1, 3, 4, 5, 6, and 8 govern routine administration; Principle 7 governs the decisions that require a human signature. Principle 1 keeps answerability with the public institution and the people who authorize its use of AI. Vendors and contractors may share accountability for their own roles. Their involvement does not remove the agency’s obligation to justify its action.

Principle 3 requires a boundary around both the function and the use of information. Explaining an application form, selecting a case for investigation, and withdrawing support require different authorizations. Expanding from one to another should trigger a fresh decision about scope, evidence, and safeguards.

Principles 4 and 5 connect intervention to architecture. Some lawful, bounded administrative tasks can operate automatically with sampling, escalation, and effective correction. Other decisions require case-specific judgment. The agency must make that distinction before the system acquires a workload too large for the promised review process. A thousand nominally reviewable decisions do not become supervised merely because one overwhelmed employee can technically open each file.

Principle 7 applies where the stakes, irreversibility, or Reserved character require a human signature. It does not make every government transaction Reserved. Where independent human judgment is required, the signer needs authority over the particular case, usable evidence, time and competence to disagree, and practical power to change the result. A person who can only repeat the screen’s conclusion cannot carry that role.

Principle 6 requires consequences to reach the accountable office through oversight, correction, and remedies appropriate to its role. The aim is not automatic personal blame for every error. It is an institution that must respond, repair, and change when its authorized process fails. Principle 8 requires the competence and intervention capacity to survive staff turnover, procurement renewals, model updates, and growing demand.

The Accountability Chain remains the same: Who answers? What is the boundary? Can the owner see it? Can the owner stop it? Do the consequences reach the owner? Follow the complaint backward from the person affected until those questions reach offices with real authority.

Follow the route to correction

Return to the opening example of the stopped support payment. Identify who could examine the missing evidence, change the decision, and act before the delay caused serious harm. Then ask how the agency would find other people affected by the same mistake. Include a route for someone unable to use the online service.

Accountability Verdict

The strongest case for an AI state is a government that becomes easier to reach, more accurate in its work, and more capable of helping people whose needs disappear in slow systems. Those gains deserve to be pursued. Public legitimacy depends on how that capacity is governed when the answer is disputed or the exercise of power is unwelcome.

Before expanding a consequential deployment, an agency should be able to demonstrate a complete path from authorization through evidence to intervention and repair. That demonstration should include a difficult case, a person unable to navigate the preferred digital channel, and a defect affecting many decisions at once. Service quality includes what happens when the citizen disagrees.

Return to the person holding the notice. The important improvement is an accessible official who can examine the evidence, explain the actual basis of the action, and change the outcome when warranted. AI can help that official do the work. The institution must preserve the authority and capacity to answer.

The state may automate its work. It must remain reachable by the people subject to its power.

Part Six now turns from institutions to the enduring boundary behind them: what must remain human, even when a machine can perform more of the task?

Sources and Notes

1. HM Revenue and Customs, “Evaluating the Impact of Microsoft Copilot in HMRC,” July 9, 2026. https://www.gov.uk/government/publications/evaluation-report-phase-3-trial-of-microsoft-copilot/evaluating-the-impact-of-microsoft-copilot-in-hmrc. Accessed September 12, 2026. Phase III ran from September to December 2024. This is an adopter’s evaluation, not an independent causal estimate of public-service quality. License allocation was randomized; reported time savings depended principally on survey evidence and adjustments. Source record: DA-CH23-S1. Archive reference: pending.

2. U.S. Government Accountability Office, Artificial Intelligence: IRS Actions Needed to Address Skills Gaps, Information Quality, and Strategic Management, GAO-26-107522, March 24, 2026. https://www.gao.gov/products/gao-26-107522. Accessed September 12, 2026. The inventory snapshot is June 2025, not the report date. Active use cases include development work. Findings and the IRS response are attributed in the chapter; agreement with recommendations is not completion. Source record: DA-CH23-S2. Archive reference: pending.

3. Commonwealth Ombudsman, Services Australia’s Income Compliance Program: A report about Services Australia’s implementation of changes to the program in 2019 and 2020, April 2021, especially paragraphs 3.19-3.28 and the report’s findings on refunds. https://www.ombudsman.gov.au/__data/assets/pdf_file/0030/286545/Services-Australias-Income-Compliance-Report.pdf. Accessed September 12, 2026. Chapter Four retains the main Robodebt case. This chapter uses the documented difficulty of identifying affected decisions as evidence about institutional repair. Source record: DA-CH23-S3. Archive reference: pending.

4. U.S. Government Accountability Office, Artificial Intelligence: OMB Action Needed to Address Privacy-Related Gaps in Federal Guidance, GAO-26-107681, March 26, 2026. https://www.gao.gov/products/gao-26-107681. Accessed September 12, 2026. The assessment compares guidance as of January 2026 with ten selected expert-identified challenges. It is not a finding of a specific privacy breach. Source record: DA-CH23-S4. Archive reference: pending.

5. Office of Management and Budget, Driving Efficient Acquisition of Artificial Intelligence in Government, M-25-22, April 3, 2025, especially acquisition considerations and negotiation provisions. https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-22-Driving-Efficient-Acquisition-of-Artificial-Intelligence-in-Government.pdf. Accessed September 12, 2026. The chapter distinguishes acquisition-policy provisions from evidence of implementation. Source record: DA-CH23-S5. Archive reference: pending.

6. U.S. Government Accountability Office, Artificial Intelligence Acquisitions: Agencies Should Collect and Apply Lessons Learned to Improve Future Procurements, GAO-26-107859, April 13, 2026. https://www.gao.gov/products/gao-26-107859. Accessed September 12, 2026. The review examined 13 acquisitions at four selected agencies; findings are not presented as a census of federal procurement. All four agencies concurred with the recommendations. Source record: DA-CH23-S6. Archive reference: pending.

7. Office of Management and Budget, Accelerating Federal Use of AI through Innovation, Governance, and Public Trust, M-25-21, April 3, 2025, especially Appendix Sections 4-6. https://www.whitehouse.gov/wp-content/uploads/2025/02/M-25-21-Accelerating-Federal-Use-of-AI-through-Innovation-Governance-and-Public-Trust.pdf. Accessed September 12, 2026. Scope exclusions, waivers, and pilot provisions limit any claim of blanket coverage. The book’s normative recommendations are distinct from this memorandum’s legal and administrative requirements. Source record: DA-CH23-S7. Archive reference: pending.

8. UK Government, “Algorithmic Transparency Recording Standard (ATRS) Mandatory Scope and Exemptions Policy,” December 17, 2024, especially Sections 3-5. https://www.gov.uk/government/publications/algorithmic-transparency-recording-standard-mandatory-scope-and-exemptions-policy/algorithmic-transparency-recording-standard-atrs-mandatory-scope-and-exemptions-policy. Accessed September 12, 2026. Mandatory organizational scope is central government as defined in the policy; publication has specified exemptions. Publication requirements are not evidence of complete compliance or effective redress. Source record: DA-CH23-S8. Archive reference: pending.