An internal AI platform is the set of shared components your engineering teams build AI systems on, such as model access, prompt versions, evaluation and project templates, run by one team as a product. It is not the AI assistant employees chat with. Our test: a platform is what the second team does not have to build. Everything else is documentation with a logo.
Platforms pay off, with a bill attached. The 2024 Accelerate State of DevOps Report found that 89% of respondents use an internal developer platform, and that its users had "8% higher levels of individual productivity and 10% higher levels of team performance," while throughput fell by 8% and change stability by 14%. DORA studied internal developer platforms for software delivery, not AI, using survey-based estimates. Our reading for an internal AI platform: it is a product with a price, and it pays back only if the right components go in and teams adopt it by choice.
What an Internal AI Platform Is, and What It Is Not
An internal AI platform is the shared, product-owned set of components and templates that teams use to build, evaluate and ship AI systems, so that each new team adds only the logic specific to its problem instead of rebuilding access to models, prompts, evaluation and deployment.
The phrase also names internal AI assistants, chat tools rolled out to employees. That is a different product with a different buyer.
The CNCF Platforms White Paper defines a platform as "an integrated collection of capabilities defined and presented according to the needs of the platform's users." Applying that cloud native definition to AI is our interpretation, and AI adds components a DevOps list lacks, such as a model gateway with per-team keys.
This article covers the shared components AI teams build on: which ones belong in the platform, what interface each one exposes, who owns it, and how teams come to use it by choice. It does not cover the layers and contracts these components sit on, the organizational shape that decides who runs the platform, or the budget that pays for it.
Why Every AI Team Rebuilds the Same Things
Duplication is the default state. Google's TFX paper (KDD 2017) describes ML orchestration built from "glue code and custom scripts developed by individual teams for specific use cases, leading to duplicated effort and fragile systems." One integrated platform cut time to production "from the order of months to weeks," a 2017 result for classic ML at Google scale, not a general law.
When Uber introduced Michelangelo in 2017, it reported that "there was no established path to deploying a model into production," and engineering teams built "bespoke one-off systems."
Spotify's 2020 post on golden paths describes the same problem: without a shared path, engineers learned how to build something by asking a colleague, a habit the company called rumor-driven development.
Generative AI repeats the pattern. AWS Prescriptive Guidance for generative AI platforms includes a layer of repeatable application patterns, offering "standardized templates and best practices for common generative AI use cases" to help "prevent duplication of effort."
What Earns a Place in the Internal AI Platform
The hard question is which reusable AI components to promote from one team into the shared layer. In our delivery work, a component has to meet four conditions:
- A second team needs it in the same form. Promote on demand, not on forecast.
- It has a stable interface: an API, a template or a file format that teams use without knowing its internals.
- It has a named owner who answers for versions, support and retirement.
- It can be retired, because it carries a version whose end can be announced ahead of time.
Logic specific to one problem, one product's prompts and unrepeated experiments stay out. CNCF tells platforms to prioritize "the most common use cases across product teams" over capabilities "only used by a single team." Our shorthand: a component used by one team is a project, and a component used by three teams without a meeting is a platform.
The Component Catalog: What Teams Stop Building
Each entry answers one question: what does a team stop building? For the model gateway, a team gets a key, a quota and one endpoint instead of its own provider integration. The AI gateway in Azure API Management lists "Enable self-service for developer teams," and the open source LiteLLM proxy issues virtual keys with rate limits and usage tracking per team.
The prompt registry moves prompts out of application code. MLflow Prompt Registry lets teams "version, track, and reuse prompts across your organization," with immutable versions and aliases for rollback, while Langfuse prompt management is built to "Decouple Prompt Updates from Code Deployment."
Evaluation harness: a shared runner, dataset format and report, while each team keeps its own dataset and thresholds. The ingestion template loads documents with permissions attached, and a guardrail policy pack applies default filters and logging from the first deployment.
Retries, state and tool routing recur in every team that chains model calls, so a shared orchestration runtime belongs in the catalog rather than in each repository.
Backstage Software Templates "load skeletons of code, template in some variables, and then publish," and the Backstage Software Catalog keeps track of "ownership and metadata for all the software in your ecosystem," the record the named-owner condition depends on.
The last three columns are our catalog, not a standard.
Golden Paths for AI: Templates With an Exit
Spotify defines a golden path as "the 'opinionated and supported' path to 'build something'" and runs separate ones for data science and machine learning. Kubeflow frames its mission as "well-lit paths across the AI lifecycle." DORA 2024 calls golden paths "highly-automated, self-service workflows" and notes that once a capability is in the platform, teams "essentially get it for free through adoption of the platform." For a golden path for AI, that means evaluation and logging live in the template.
Every path needs an exit. The CNCF paper lists "Optional and composable" as a platform attribute, so that teams can "provide and manage their own capabilities outside of the platform's offerings when necessary." DORA 2024 estimated a 6% decrease in throughput where respondents had to "exclusively use the platform," a finding about internal developer platforms. A golden path that cannot be left is a gate, and teams route around gates.
Our template for a new retrieval service holds a repository skeleton, a gateway key, a catalog entry, an evaluation harness with an empty dataset, and default logging. A team with an unusual need swaps one part.
Run the Internal AI Platform as a Product, Measured by Pull
Whether a platform should exist is an operating-model question. This section assumes it does.
Start thin. Team Topologies defines a platform team as providing "a compelling internal product to accelerate delivery by Stream-aligned teams" and warns that most internal platforms "become bloated monstrosities." The DORA guidance on platform engineering recommends a minimum viable platform: find the golden path for the most common workflow and build only enough to make it clearly better.
DORA 2024's full account for internal developer platforms reads 8% higher individual productivity, 10% higher team performance and 6% better organizational delivery and operations performance, against 8% lower throughput and 14% lower change stability. User independence, finishing tasks without an enabling team, was linked to 5% higher productivity, and a dedicated platform team to 6% higher team-level productivity with a negligible effect for individuals. Our reading for an internal AI platform team: independence is the lever you control, and a ticket queue erases it.
Measure adoption by pull. The CNCF Platform Engineering Maturity Model describes adoption moving from "extrinsic push" to "intrinsic pull." The CNCF paper suggests active users, retention and "latency to build and deploy a brand new service into production." In AI platform engineering, we track the time from a new team's first request to its first evaluated release.
DORA's platform engineering page also summarizes 2025 research on AI-assisted software development, meaning coding assistants rather than building AI systems: 90% of organizations use an internal developer platform and 76% have dedicated platform teams.
Give every component an announced end of life, or the platform becomes a catalog of dead components.
DORA names three anti-patterns: "Build it and they will come," "Ivory tower" and "Ticket-ops," where the platform team becomes a "vending machine" for requests. We add two: a full catalog built before any team needs it, and a gateway owned by everyone, which breaks at the first provider API change. The check: would a new AI team get a model key from a form in minutes, or from a ticket in days?
At DS Stream we start with the components a second team is already asking for, and we hand the platform over with its backlog and named owners. See how we build reusable components into an AI factory, or talk to our platform engineers about what your teams keep rebuilding.
Frequently Asked Questions
What is an internal AI platform?
An internal AI platform is the shared set of components that engineering teams use to build, evaluate and ship AI systems, such as a model gateway, a prompt registry and an evaluation harness, owned by one team as a product. It is not an AI assistant for employees.
What should an internal AI platform include first?
Start with what a second team already asks for, usually model access with a per-team key and quota, plus a project template with logging. The evaluation harness and prompt registry come next. That order is our recommendation.
Do you need a dedicated platform team for AI?
Usually yes, once several teams depend on it. DORA 2024 estimated that a dedicated platform team brings 6% higher productivity at the team level with a negligible effect for individuals, a finding about internal developer platforms. The team also has to collect feedback and measure adoption, or it turns into a ticket queue.
Should teams be required to use the internal AI platform?
Not as a rule. CNCF says platforms should be optional and composable, and DORA 2024 estimated a 6% decrease in throughput where teams had to use an internal developer platform exclusively. Adoption by choice is the better signal: teams take the golden path because it is faster.
