From Prompting to Operating AI: Self-Hosted Copilots and the Agentic Enterprise Become a Platform Decision

AI adoption is entering a new phase. The question for CTOs is no longer whether teams can get value from LLMs, it is where AI runs, how it is governed, and who owns the operational surface area. Product teams want “native AI” features, security teams want control, and platform teams are being asked to make the whole thing repeatable.

A noticeable signal is the re-platforming of AI capabilities around enterprise deployment constraints. GitLab’s expansion of GitLab Duo Self-Hosted to support models deployed through Microsoft Foundry positions AI assistance as something that can be brought to the customer’s model endpoint, inside the customer’s Azure setup, with customer-selected models and controls (InfoQ: GitLab Duo + Microsoft Foundry). That move reflects a broader architectural preference: keep code and data in-place, bring the AI to the perimeter you already defend.

Another signal is the shift in language from “using AI” to “employing agents”. Snowflake’s blueprint for the “agentic enterprise” frames the work as hiring deliberately, preparing a data handbook, and managing actively (Snowflake Blog). The important CTO implication is operational: agentic workflows behave less like a feature toggle and more like a distributed system with policies, failure modes, audit requirements, and lifecycle management. Put plainly, agents create a new class of production workload.

Engineering leadership content is converging on the same point from the org angle. Refactoring.fm’s discussion of “native AI products” and team development emphasizes that AI features are becoming part of core product strategy, not a bolt-on experiment (Refactoring.fm). Once AI is core, the organization needs a stable internal contract for evaluation, prompt and tool changes, model upgrades, and incident response. Ad hoc experimentation does not survive contact with uptime targets and compliance reviews.

CTOs should treat the “self-hosted copilot plus agentic workflows” shift as a platform decision with three practical consequences. First, plan for a control plane: identity, authorization, policy, evaluation, and audit logs across AI interactions. Second, decide where inference and tool execution live (vendor SaaS, your cloud, VPC, on-prem) based on data sensitivity and latency, not developer convenience. Third, assign an operating model: who on-calls agent failures, who approves model changes, and what counts as a rollback when behavior changes.

Actionable next steps: inventory the highest-value workflows for copilots and agents, then map each workflow to data access patterns and required controls. Stand up a minimal “AI platform” slice (model endpoint, evaluation harness, policy gateway, logging) before scaling use cases. Finally, require every AI-enabled system to ship with an owner, an SLO, and a change management path for prompts, tools, and models. AI is becoming production infrastructure, and production infrastructure needs operators.

Sources

1. https://www.infoq.com/news/2026/09/gitlab-microsoft-foundry/
2. https://www.snowflake.com/en/blog/stop-prompting-start-employing-agentic-enterprise/
3. https://refactoring.fm/p/growing-juniors-native-ai-products