Emerging Technologies and America’s Future: Why Public Servants Need a New Playbook for the AI Age
Written by NCAC Board Member, Ryan Heimer
Nine seconds.
That is reportedly how long it took an artificial intelligence agent to delete production databases and associated backups after encountering a routine credential problem. When investigators later examined the incident, the AI’s explanation was as startling as the damage itself:
“I guessed instead of verifying.”
For many readers, the story may sound like another Silicon Valley mishap—a cautionary tale for software engineers and technology startups. Yet the implications stretch far beyond a single company or a single AI system. The incident offers a glimpse into a future where artificial intelligence increasingly moves from providing recommendations to taking actions, often at speeds that outpace traditional forms of human oversight.
For public servants, this should command attention.
The real lesson is not that an AI system made a mistake. Humans make mistakes every day. The lesson is that the system possessed the authority to act before governance mechanisms had an opportunity to intervene. In many ways, this was not an artificial intelligence failure at all. It was a governance failure.
Throughout American history, technological revolutions have forced institutions to adapt. Railroads transformed commerce but required new safety regulations. Automobiles expanded mobility but demanded traffic laws and licensing systems. The internet reshaped communication while creating entirely new concerns around cybersecurity, privacy, and information integrity.
Artificial intelligence presents a similar challenge, but at a much faster pace.
The Stanford Emerging Technology Review describes AI as a foundational technology with the potential to reshape economies, public services, national security, and society itself. Yet researchers also caution that today’s AI systems continue to exhibit unpredictable behavior, hallucinations, reliability failures, and hidden biases. The technology is advancing rapidly, but the institutions responsible for governing it are often struggling to keep pace.
The PocketOS incident highlights this growing gap.
While headlines focused on the AI agent, the deeper issue was data governance. A recent report titled AI Redefines the Governance of Data Based on Use argues that organizations are entering a new era in which traditional approaches to governance are no longer sufficient. Historically, data governance focused on protecting information from breaches, unauthorized access, and theft. Security was the primary concern.
Artificial intelligence changes that equation.
Today, the challenge is not simply protecting data. It is governing how data is used.
Modern AI systems are extraordinarily data hungry. They draw information from structured databases, documents, emails, reports, images, and other sources. Increasingly, they combine information from across organizations without regard for traditional organizational boundaries. The result is a new governance challenge: ensuring that information is used responsibly, ethically, and for its intended purpose.
This shift, from data security governance to data use governance, may be one of the most important developments in the AI era.
For decades, organizations asked whether data was secure.
Now they must also ask whether data is being used appropriately.
Just because a system can access information does not mean it should.
The OneTrust report argues that responsible governance requires understanding four forms of context surrounding data: technical context, consent context, regulatory context, and business purpose. Together, these elements determine not only whether data can be accessed, but whether its use aligns with legal requirements, ethical standards, and organizational objectives.
Public administrators may recognize this concept immediately.
Government agencies rarely make decisions simply because information exists. Public servants operate within legal authorities, policy frameworks, ethical obligations, and public expectations. Data alone is not enough. Context matters.
An MSHA inspector may possess extensive operational information about a mine. However, that information must be used within the framework established by the Mine Act, agency policy, and principles of due process. Similarly, agencies handling citizen information cannot simply feed data into an AI model because it is available. They must consider why the information was collected, whether consent exists, and whether the proposed use aligns with law and public trust.
These concerns become even more significant as AI systems increasingly act on information rather than merely analyze it.
The Stanford review notes that emerging AI agents are capable of carrying out multistep tasks with limited human supervision. Yet researchers continue identifying reliability concerns, including goal drift, overconfidence, memory limitations, and unpredictable behavior. When combined with broad access to data, these weaknesses create new forms of organizational risk.
The PocketOS incident demonstrates exactly why.
The problem was not merely that an AI guessed incorrectly.
The problem was that governance mechanisms allowed it to guess at all.
This is where public administration has something important to contribute.
The Government has spent generations developing systems designed to manage risk. Internal controls, financial audits, workplace examinations, accident investigations, separation of duties, ethics rules, and regulatory oversight all emerged from the same underlying principle:
Trust matters.
Verification matters more.
In mining, ventilation standards exist because experience taught painful lessons about what happens when hazards go undetected.
Workplace examinations exist because assumptions can be deadly. Lockout/tagout procedures exist because relying on good intentions alone is insufficient when safety is at stake.
AI governance increasingly requires a similar mindset.
Organizations cannot rely solely on prompts, guidelines, or user instructions. Governance must be embedded into systems themselves through permissions, audit logs, approval requirements, policy enforcement mechanisms, and continuous oversight.
The OneTrust report describes this transition as a movement toward programmatic governance. Traditional compliance models rely heavily on manual reviews, audits, and after-the-fact assessments. AI systems operate too quickly for those approaches to remain effective. Governance increasingly must occur at machine speed.
This may represent one of the defining governance challenges of the next decade.
Human-speed oversight cannot effectively govern machine-speed decision making.
Institutions must adapt.
The implications extend beyond technology departments. Public trust is increasingly at stake. Surveys consistently show that citizens remain concerned about how organizations collect, store, and use personal information. Many are uncertain whether their data is being handled responsibly. For government agencies, these concerns carry special weight because trust is central to democratic legitimacy.
Citizens deserve answers when automated systems influence decisions affecting their lives.
Why was this decision made?
What information was used?
Who approved the system?
How can errors be corrected?
Can outcomes be appealed?
These are not merely technical questions. They are democratic questions.
Ultimately, the PocketOS incident offers a warning, but it also provides an opportunity.
America has navigated technological revolutions before. Success has never depended solely on innovation. It has depended on building institutions capable of channeling innovation toward public benefit while managing its risks.
Artificial intelligence is no different.
The future will not be determined solely by how powerful AI becomes.
It will be determined by whether governments, organizations, and communities develop the governance frameworks necessary to guide that power responsibly.
The lesson hidden within those nine seconds is therefore much larger than a deleted database.
It is a reminder that the central challenge of artificial intelligence is not intelligence.
It is governance.
And as public servants look toward the future, that may be the most important lesson of all.
