# The Two-Year Migration Plan Is Now the Risk: Why agentic delivery changes the economics of data platform migration

URL: https://dcrewe.com/article/the-two-year-migration-plan-is-now-the-risk  
Author: Donovan Crewe  
Published: 2026-10-26

Almost every enterprise AI roadmap I see depends on a data platform migration. The plan is usually measured in quarters or years, with AI agents, copilots and decision support systems sitting behind it, waiting for the data to become accessible.

I think we need to reconsider that sequencing.

A data warehouse migration has always been difficult to justify. The work is expensive, the timelines are long and the value often arrives at the very end. Thousands of tables need to be understood and converted. Stored procedures nobody has touched in a decade need to be reverse-engineered. Jobs run at 2 am for reasons nobody can fully explain, and dashboards need to keep producing the same numbers throughout the transition.

The standard response has been to add people, extend the timeline and run both platforms in parallel until somebody is comfortable enough to switch off the old one. It is a sensible response to a difficult engineering problem, but it is also a delivery model built around the assumption that most of the work has to be done manually.

We recently worked with a regulated global asset manager that had planned to migrate four legacy source systems onto Databricks over approximately two years, using a team of around twenty people. The estate had accumulated over a century of business history. It contained thousands of tables and data structures that nobody in the organisation fully understood.

We delivered the migration in three months with three people: one data engineer and two agentic engineers. The agents executed the conversion workflows in three to four days.

That is a significant difference in delivery time, but the interesting part is how we achieved it. The team did not simply ask an AI model to rewrite thousands of SQL objects and hope the results were correct. We built an engineering system that allowed agents to apply tested conversion logic across the estate, with the controls needed to operate in a regulated environment.

This is the distinction I think matters most as we move towards AI-native engineering.

In [The Meat Proxy Paradigm](https://www.linkedin.com/pulse/meat-proxy-paradigm-lumenalta-venmc/), I wrote about the problem of producing sophisticated outputs that nobody can properly explain. Data migration is a particularly unforgiving example. A pipeline can run successfully and still produce incorrect results. A table can look structurally correct while the meaning of its data has changed. In financial services, that is not something you discover after the old platform has been switched off.

The way we approached the problem was to move the engineering judgment into the system itself.

Engineers built and validated the conversion workflows in advance. The agents mapped the estate, identified the appropriate workflow for each object and executed the conversion. The logic was tested, the outputs were traceable and the same input followed the same defined process. Rather than relying on a model to improvise a solution thousands of times, we established the rules once and applied them consistently.

That changes the nature of the work. Engineers spend less time implementing repetitive conversions and more time designing the system, validating its behaviour and resolving the exceptions that require judgment. Agents provide the execution capacity, while the engineering team remains responsible for the logic governing that execution.

The client's team was involved before the conversion began. Agents mapped the tables, jobs, dependencies and scripts, then presented a plain-language migration plan that included naming conventions and mapping changes. The client reviewed and approved the plan. Our specialists worked alongside their engineers throughout delivery, addressing the issues that required intervention.

Verification was continuous rather than something left until the end. The old and new environments ran side by side, with a nightly agent reconciling data and record counts. A live view showed which objects had passed validation and which needed attention. The new platform started carrying real workloads before final cutover, giving the team evidence that the migration was working while the existing systems remained available.

All of this ran inside a locked-down environment using the client's approved AI tools locally. For a regulated organisation, that is a fundamental part of the design. An approach that only works when proprietary data and code can be sent to an external AI service is not an approach that every enterprise can use.

The point is that speed came from the engineering model, not from removing engineering controls. The workflows were deterministic, the client retained control over the plan, and the results were continuously reconciled. Those decisions allowed a small team to coordinate work at a scale that would otherwise have required substantially more manual effort.

I would not interpret this as evidence that agents can independently redesign any regulated data estate. That is a different problem, with a different risk profile. What this project demonstrates is that complex engineering work can be executed at scale when the rules are well defined, the decisions are reviewable and the outcomes can be verified. Autonomy should expand as the evidence supports it, not simply because the technology makes it possible.

There is a wider implication here for how we design AI-native delivery systems. We should be looking at which parts of engineering require human judgment, which decisions can be encoded into reusable workflows and how agents can execute those workflows without losing the controls that make the result trustworthy. That is a more useful starting point than asking how much code a model can generate.

It also changes how I think about platform migrations themselves.

A lot of migration programmes begin with the assumption that everything should move from one platform to another. But many organisations running Snowflake, for example, are perfectly happy with the warehouse experience. Their analysts know the platform, their dashboards work and the existing governance model is established. The pressure often comes from elsewhere, particularly when large engineering workloads become expensive or machine learning and AI require data to be copied into separate environments.

Moving every workload is not necessarily the right answer. In many cases, the better architecture is to move data engineering, machine learning and AI workloads onto Databricks while retaining the governed warehouse layer where it continues to serve analysts. The right decision depends on the workload, the operating model and the economics of running it.

We work with Databricks extensively, but I do not think every migration should be treated as a competition between platforms. The more useful question is where each workload belongs and what the organisation needs its data platform to support over the next several years. When the cost of migration falls, that becomes a much easier architectural decision to make on its merits.

The same applies to organisations that want a straightforward lift-and-shift because they are not ready to take on a larger transformation. That is a reasonable place to start. The problem is when the migration becomes a reason to postpone the engineering improvements that would make the new platform valuable.

One option is to build the orchestration and re-architecture alongside the migration itself. The client can retain the scope and timeline they have approved while preparing the next stage of work. It takes deliberate planning, but it avoids turning every improvement into a separate programme that begins only after the original migration has finished.

This is where the economics become important.

When a migration takes two years, it becomes a prerequisite for the AI strategy. Teams spend that time moving data and validating pipelines while the use cases that justified the investment wait for the new platform. When the same work can be completed in a quarter, the migration can become the first delivery milestone of the AI strategy itself.

That matters because AI initiatives are already struggling with the quality and accessibility of the data they depend on. [Gartner has forecast](https://www.gartner.com/en/newsroom/press-releases/2025-02-26-lack-of-ai-ready-data-puts-ai-projects-at-risk) that through 2026, organisations will abandon 60% of AI projects that are not supported by AI-ready data. A migration is not the only way to address that problem, but it can be a major dependency. Every additional quarter spent on delivery is time in which the organisation cannot fully realise the value of its investment.

I think technology leaders need to look more critically at the assumptions behind their migration plans. The complexity of the estate still matters. So do data quality, security, business continuity and the consequences of a failed cutover. But a timeline based on traditional staffing levels and manual conversion effort should not be treated as inevitable when the delivery model itself has changed.

There is one outcome from the asset manager engagement that I find particularly valuable.

The workflows did not leave with us. The client retained the tested conversion library, mapping rules and reconciliation logic. The next source system or platform change can start from that foundation rather than repeating the same discovery and implementation work.

This connects directly to the argument in [Why Every CTO Needs an Intelligence Strategy](https://www.linkedin.com/pulse/why-every-cto-needs-intelligence-strategy-lumenalta-phhzc/). I believe engineering organisations need to become better at retaining and reusing the knowledge they develop through delivery. When that knowledge is encoded into tested systems, it becomes something the organisation can apply repeatedly, rather than expertise that exists only in the heads of the people who completed the last project.

That is the foundation of Engineering Intelligence. The value of AI-native engineering is not just that a team can deliver a migration faster. It is that the team can leave behind a system that makes the next migration easier, more consistent and less dependent on starting from scratch.

For me, that is a much more interesting outcome than productivity alone. The organisation has a modernised platform, but it also has a reusable engineering capability that can compound over time.

If I were reviewing a migration roadmap today, I would want to understand how the work will actually be performed. Which workloads need to move? Are agents selecting from tested workflows or generating conversions independently? How are plans approved and results reconciled? Can the delivery model operate within the organisation's security boundaries? And what engineering capability will remain when the project is complete?

Those answers tell you more about the quality of a migration plan than the timeline alone.

A two-year migration may still be justified by the complexity of a particular environment. But it should be the result of an engineering assessment, not the default assumption. If agents can apply tested engineering judgment across thousands of objects, with the right controls in place, technology leaders need to reconsider how much time and effort the work should require.

The long plan used to be the safe plan. I am increasingly convinced that continuing to treat it as the default is becoming the greater risk.
