You cannot measure agent ROI in a human-driven workflow

Every company investing in coding agents eventually asks the same question: what return are we getting? The available answers come from individual products. Agent vendors count tokens, sessions, suggestions, and generated code. GitHub counts pull requests and merges. CI systems count successful builds. Delivery platforms count deployments. Each measure accurately describes activity inside its own boundary, but none establishes whether that activity produced an acceptable software change or what the complete effort cost.

The useful economic unit is not the model invocation, suggestion, pull request, or pipeline run. It is the accepted software change and the complete effort required to produce it:

The arithmetic is easy. The boundary is not. Jira owns the request, GitHub owns the source activity and review, Jenkins owns the build, Harness owns the deployment, and ServiceNow owns the approval. MCP can make every system callable and orchestration can automate movement between them, but no system owns the complete change whose success and cost we are trying to measure.

Humans have always supplied the missing continuity by recognizing that an issue key, pull request, build number, and deployment belong to the same work even when the relationship was never recorded as data. An agent has to search, correlate, and infer that boundary before it can perform the work. An organization cannot measure agent ROI when the production unit generating that return has no system of record.

Multiple systems of record fragment the ROI calculation

A software change may begin as a customer request in Salesforce, become an initiative in Productboard, acquire context in Confluence, turn into tasks in Jira, be implemented through an agent, reviewed in GitHub, tested in Jenkins, deployed through Harness, approved in ServiceNow, and associated with an incident in PagerDuty. Every system accurately records one part of the lifecycle. None can independently determine which collection of records constitutes the complete change.

The conventional answer is integration. Put the Jira key in the pull request. Attach the commit SHA to the build. Associate the build with the deployment. Connect the deployment to the ServiceNow request. Export all of it into a warehouse and join the records later.

That produces a chain of identifiers, but the chain inherits the assumptions of a human workflow. One ticket can produce several pull requests. One pull request can satisfy several tickets. A build can contain many changes, and a deployment can contain several builds. Some agent attempts are abandoned. Others produce knowledge reused by later work. An incident may implicate a dependency introduced several releases earlier. Rebase and squash can replace the source identifiers downstream systems relied on.

An integration can establish that two records share a string or occurred near each other. It cannot establish that one was causally necessary to the other. More importantly, people still decide when the change began, which agent sessions belong to it, which patches satisfy it, which evidence verifies it, and when the result became acceptable.

MCP improves access to these systems. Orchestration automates transitions between them. A data warehouse aggregates their events. None changes the data primitives underneath them.

The workflow can only calculate over the boundary it is given. It cannot prove that the boundary describes the whole change. Any ROI calculation inherits that uncertainty.

But what if the entire lifecycle is already in GitLab?

GitLab presents the strongest counterargument. Unlike a toolchain assembled from several vendors, GitLab can hold planning, source control, review, CI/CD, security findings, and deployments in one platform. GitLab describes projects, releases, and code as sharing one data plane so teams and agents can work from a single source of truth.

That matters. Consolidation removes many integration boundaries and gives GitLab considerably more lifecycle context than a standalone coding assistant or source host. If the problem were only that records live in different applications, GitLab would be the answer.

The problem persists because co-location does not redefine the change.

A GitLab issue remains a planning object. Git remains the record of source history. A merge request remains a container for review. A pipeline remains an execution record. A deployment remains an environment event. These objects live closer together, but their boundaries were still designed around activities performed and coordinated by people.

The creation of GitLab Orbit makes the distinction visible. GitLab describes Orbit as a context graph for AI agents and its repository describes a unified context API for AI systems and human users. Orbit builds and queries a graph across lifecycle information so an agent can recover relationships that are difficult to use directly from the underlying objects.

Orbit may become a powerful context layer. Its necessity is also evidence that putting human-native records in one application did not make them one agent-native change. The lifecycle still has to be extracted, indexed, and projected into another representation before an agent can traverse it coherently.

An index over systems of record is not the same as changing the system of record. The index tells an agent how existing objects appear to relate. The underlying objects still determine what is durable, what owns identity, and where the boundaries of planning, implementation, review, verification, and deployment begin and end.

GitLab put much of the human software factory in one application. Orbit exists because one application still did not make it one change.

A change means something different to an agent

For a human, a software change can be represented as a diff because the human supplies everything the diff omits. A developer remembers the ticket, understands the conversation, recognizes the relevant architecture, infers why particular files matter, and evaluates whether the result satisfies the request. The diff only needs to show what text changed because people reconstruct what the change means.

To an agent, the added and removed lines are merely the physical effect of the work. They do not establish why the work exists, what success means, which constraints apply, what the implementation depends on, what evidence verifies it, or what context should survive for the next agent.

This is the primitive we need to redefine.

An agent-native change is a transition from intent to verifiable state. It has a durable identity that begins before the patch and accumulates the structure required to understand, execute, and judge the work. Intent records why the work exists. Acceptance criteria define what must be true when it is complete. Tasks establish scope and authority. Patches record the causal transformations. Evidence records what was observed. Provenance records the agents, models, tools, sessions, and humans involved. Review applies a judgment to an exact state rather than a floating collection of files.

The semantic and causal dimensions of the change must remain distinct. Semantic relationships explain what the work means by connecting intent, criteria, tasks, code entities, memory, evidence, and review. Causal dependencies explain which patches produced the state and what that state depends on. Together they give an agent both a coherent context boundary and a verifiable result.

Once the change becomes that primitive, workflow can become agent-native. Without it, agents remain consumers of a human workflow, reconstructing the meaning people once carried between tools.

Workflow is downstream of the data model

We often describe agentic software development as a workflow problem. Give an agent a task, connect it to tools, add review gates, and orchestrate the path to production. This focuses on what the agent should do without first establishing what the agent is acting upon.

A workflow engine can enforce a sequence. It cannot create causal relationships the underlying systems never recorded. A context layer can improve retrieval. It cannot make an inferred relationship part of the durable change. An agent can search a ticket, repository, build history, and deployment log. Access to those documents does not tell it which facts define the current work or which must survive after the session ends.

The data model determines what the workflow can know. If intent, implementation, evidence, review, and provenance have separate identities, every agent run begins with a reconstruction problem. If they are durable parts of the change, the workflow can reason over the closure directly.

This is the difference between indexing context around a change and making context part of the change.

ROI becomes measurable when the change has a boundary

Atomic starts from this change primitive. The semantic change graph connects intent, acceptance criteria, tasks, code intelligence, memory, evidence, and review. The causal change graph records patches and their dependencies. Provenance records how the work happened. A view resolves those records into a state whose closure can be queried and independently verified.

The workflow does not create these relationships after the work is complete. It operates over relationships recorded while the change is produced.

That architectural difference shows up in software outcomes. In our directional pilot across 90 coding runs on real Drizzle ORM issues, a frontier model using Git and an LLM produced accepted changes in 60 percent of runs at $0.77 per accepted change. The same frontier model working from self-authored Atomic intents, with no code-query access, reached 90 percent acceptance at $0.76 per accepted change.

The strongest frontier configuration did not win because it retrieved more code. It knew why the change existed, what success meant, what was in and out of scope, and what it had to prove. Giving the model a more coherent definition of the work increased the probability that its activity became accepted software.

The pilot was directional: ten tasks, nine configurations, and one run per cell. The changes were independently blind-scored, but the matrix still requires replication and human semantic audit before the differences should be treated as statistical confirmation. The narrower finding is already useful. Model capability alone did not determine change success or cost. The system around the model changed both.

A separate live Atomic view shows what becomes measurable once the change owns its boundary. The view contains four AI-authored patches produced across 59 model steps and 23,069 uncached tokens, with $0.4319403 in captured agent cost. Its four intents conform, all five acceptance criteria connect to verification evidence, and three post-initial human interventions belong to the same history.

The cost is intentionally narrow. It does not include all human labor, infrastructure, or downstream business value. Its importance is that the spending is attached to the software state it produced. The same closure can establish whether the agent satisfied the intent, remained within authorized scope, accumulated sufficient evidence, and received independent review.

In this case, every criterion was satisfied and every product resource remained within authorized scope, yet promotion was correctly blocked because independent review was missing. Implementation completeness, cost, exposure, and assurance are distinct projections of the same change.

That is what a system of record makes possible. Agent ROI is no longer inferred from tokens, suggestions, or pull requests. It is calculated from the cost and probability of producing accepted software change.

Agent ROI starts with the change

Agent ROI is where this architectural failure becomes measurable. If the platform cannot identify the complete change, it cannot determine which activity produced it, whether it satisfied its intent, how much human intervention it required, or what the result cost.

The future agent software factory will not be created by placing autonomous workers on top of Jira tickets, Git branches, pull requests, and pipeline runs. Those records were designed to help people coordinate activities across the software lifecycle. Making them callable does not make them coherent.

Nor is consolidation sufficient. A single platform can remove application boundaries while preserving the human definitions embedded in its issues, commits, merge requests, pipelines, and deployments. Building a context index over those objects can help an agent navigate them, but it leaves the software change itself distributed across the underlying records.

Agent-native workflow requires agent-native data primitives. The software change must become the system of record and the boundary for context, execution, verification, governance, and economics. It must carry the intent that authorized it, the causal transformations that produced it, the evidence that supports it, the provenance that explains it, and the verifiable state that resulted.

Only then can an agent software factory know what its agents are building, why they are building it, what the result depends on, whether it should be accepted, and what producing it actually cost. Change success and cost per accepted change become computable properties of the same closure rather than estimates assembled from product telemetry.

Until that primitive changes, we are not measuring the economics of agent-produced software. We are measuring agent activity inside human-driven tools.