The GitLab Direct Transfer Failure That a Groups-First Migration Fixed

Short version, for anyone who found this page mid-migration: if a GitLab Direct Transfer run of subgroups and their projects fails without telling you why, stop debugging permissions and change the order instead. Create the group structure first and verify it is complete — including the CI/CD variables and settings, which Direct Transfer itself does not carry and the orchestration’s post-migration tasks lay in — then migrate the projects into groups that already exist. In our migration, that ordering change eliminated the failure, and it went on to move over a thousand repositories onto a single GitLab instance.

The standard combined-migration path failed without a usable error code

The first tooling approach on that migration was the obvious one. The migration ran on Congregate — GitLab Professional Services’ orchestration tooling, driving GitLab’s Direct Transfer underneath — which supports moving a subgroup and the projects inside it as a single combined operation. That combined path is the one most teams reach for: it is fewer steps, it is the shape the tooling offers up front, and with dozens of waves ahead of it, fewer steps is a real argument. (Congregate’s own documentation, read closely, recommends migrating groups before or alongside projects and running them sequentially — advice this story ends up vindicating.)

It ran for hours and came back failed. What it did not come back with was an error code anyone could act on.

That distinction is the whole story. A migration that fails loudly costs you a morning: you read the code, you fix the thing it named, you run it again. A migration that fails quietly, after hours of work, is a different class of problem, because every hypothesis you want to test costs another multi-hour run. You get maybe two or three attempts a day, and you have to spend them well.

Four checks came back clean

The obvious suspects were eliminated in roughly this order, and every one of them is worth checking first on your own migration, because usually one of them is the answer.

The PAT’s scopes. The access token driving the migration was checked against what Direct Transfer actually needs. Nothing missing.

The token user’s permissions on the target top-level group. A token can carry every scope in the list and still belong to a user who does not hold the role required on the destination group. That was checked directly against the target top-level group. Also clean.

A name collision with an existing group. A destination path that is already taken is a classic cause of a migration that reports failure in an unhelpful way, so the target namespace was checked for an existing group of the same name. Nothing there either.

The target subgroup itself. The last environmental hypothesis: something specific to that destination subgroup. A trial migration was run into a different subgroup entirely — and it failed in exactly the same way.

Four checks, four clean results, and the migration still failing. That is not a wasted afternoon, even though it feels like one at the time. Four clean checks is the evidence that the problem is not in the category everybody assumes it is in, and until you have that evidence you cannot justify trying something structural.

The fix was to migrate groups first, then projects into them

The call was to stop treating it as one operation and split it in two:

1. Migrate the groups on their own. Create the target group structure and let it come up completely — settings applied, and the CI/CD variables (which Direct Transfer itself does not carry) laid in by the post-migration tasks — before anything else is asked of it.
2. Then migrate the projects into it. The destination groups already exist and are already configured, so the project migration has nothing to create; it only has to land.

It worked without issue.

The lesson is about ordering, not about tooling

There is no root cause at the end of this story, and it is worth being honest about that rather than inventing one. What came out of it was a repeatable ordering rule, which on a migration with many waves still to run was worth considerably more than a satisfying explanation would have been.

The transferable version generalizes past GitLab, and past migrations:

When one operation is asked to create a container and populate it in the same run, and it fails without telling you which half failed, split the operation. Create the container, verify it is exactly what you wanted, then fill it. You lose a step and gain a boundary — and the boundary is the point, because a two-phase run that fails tells you which phase failed even when it tells you nothing else.

The corollary is about how you spend attempts. When each run costs hours, the cheap checks (scopes, roles, collisions) are worth doing first precisely because they are cheap — but once they are exhausted, the next move should be a change to the shape of the operation, not a fifth variation on permissions. Structural failures do not respond to credential debugging, and the only signal that you are looking at one is a run of clean checks.

We do structure first as a default now

Group structure, settings and variables get created and verified as their own step, and projects move into a destination that is already standing. It costs one extra wave-planning line and it removes an entire class of failure that reports itself badly.

That consolidation finished: over 1,000 repositories from five platforms — GitHub, GitHub Enterprise, GitLab CE, GitLab EE on Google Cloud and Azure DevOps — spread over more than five instances, onto one GitLab instance, in waves of 200 to 300, with near-zero impact on business-as-usual delivery. The full write-up is GitLab consolidation: five source systems into one instance.

Migrating or running self-managed GitLab, and building the pipelines that sit on top of it, is the service described on DevOps as a Service. If you are staring at a failed Direct Transfer right now, send us what it did — an engineer will read it.