From AI Pilots to Agent Fleets: Governance Becomes the Platform

AI adoption has entered a new phase. The near-term constraint is no longer model access or prompt quality, it is operational control. As organizations deploy more autonomous, tool-using agents across codebases and business workflows, the limiting factor becomes governance: identity, policy, auditability, and runtime oversight.

Concrete production use cases are already pushing teams there. DoorDash described using a multi-agent LLM system to clean up more than 60,000 feature flags across 623 repositories, combining experimentation data (via MCP) with human review to close the loop on safety and correctness (InfoQ). That story matters less for feature flags than for the operating model: multiple agents, real system access, measurable outcomes, and a workflow that has to be controlled like any other production automation.

Vendors are responding to the same pressure. WSO2’s Agent Manager positions “agent sprawl” as the new shadow IT problem, offering centralized governance, identity management, security controls, and operational oversight for enterprise agents (InfoQ). MIT CISR is publishing directly into the same gap, arguing for organizational guardrails to prevent redundant investment and unmanaged risk (“AI Roller Coaster”), then translating that into architecture-level guardrails to help business units apply AI faster with less exposure (MIT CISR, MIT CISR). Public discourse is also sharpening around AI risk and critical infrastructure protection, which increases regulatory and board attention even for teams that only want “internal productivity” wins (BBC, BBC).

CTOs should treat agent governance as a platform capability, not a procurement checkbox. The core design question is: who is allowed to run automation, against which systems, under what permissions, with what logging, and with what rollback path. Agent identity and authorization need first-class treatment (service accounts, scoped credentials, just-in-time access), and agent observability needs to look like production observability (structured traces of tool calls, data access logs, policy evaluation results, cost and latency budgets). Short, punchy rule: no invisible agents.

A practical way to start is to standardize an “agent runtime contract” before agent count explodes: (1) a single gateway for tool access (internal APIs, data stores, ticketing, CI), (2) policy-as-code for allowable actions and data classes, (3) mandatory human-in-the-loop checkpoints for destructive operations, and (4) an incident playbook that assumes the agent can be wrong in novel ways. Teams that already invested in internal developer platforms can extend the same patterns, golden paths, paved roads, and guardrails, to AI agents.

Action items for the next 30 days: inventory existing agents and automations (including “hidden” scripts and copilots with elevated access), define tiers of agent capability (read-only, write-with-approval, autonomous), and pick a minimal set of telemetry you will require everywhere. The next wave of AI wins will come from controlled autonomy, not maximum autonomy.

Sources

1. https://www.infoq.com/news/2026/09/doordash-feature-flag-cleanup/
2. https://www.infoq.com/news/2026/09/ws02-agent-manager/
3. https://cisr.mit.edu/publication/2026_0901_AIRollerCoaster_ThorogoodReynoldsGill
4. https://cisr.mit.edu/publication/2026_0917_TalkingPoints_AIGuardrails_ThorogoodReynoldsGill
5. https://www.bbc.co.uk/news/articles/c65ymj7njvl7o
6. https://www.bbc.co.uk/news/articles/cm2dw1w9d3yjo