Tutorials Developer

Migrating a Media Archive Without Losing Anything

Professional · ~20 min

Overview

Every asset system is eventually replaced, and the cost of that replacement is set by decisions made when it was implemented. A migration has to move three things (the files, the metadata, and the relationships between them) and the third is where migrations quietly fail, because a pile of files with a spreadsheet of fields is not an archive. This guide covers how to plan a migration, how to run it without a hard cutover, and how to prove afterwards that nothing was lost.

What You Need

A full export from the source system, including metadata and relationships
Checksums for every asset, generated before the migration begins
A field mapping document agreed before anything moves
Enough storage to hold both copies during the transition
A rollback plan that does not depend on the new system working
Time: migrations run long, and rushing them is how data is lost

Steps

Inventory and checksum before you touch anything

Produce a complete manifest of what exists: every asset, its size, its checksum, and its metadata. This is both your migration plan and your proof of completeness afterwards. Without a pre-migration manifest you cannot demonstrate that the destination matches the source, only that it contains a lot of files.

Map the fields explicitly, including what will not survive

Go field by field from source schema to destination schema and write the mapping down. Some fields will have no equivalent: decide deliberately whether to drop them, concatenate them into a notes field, or extend the destination schema. Undocumented field loss is the most common migration regret.

Migrate the relationships, not just the assets

Version history, parent-child links between masters and derivatives, project groupings, and collection membership are what make an archive navigable. They are also frequently expressed as internal database identifiers with no meaning outside the source system, which is what makes them the hardest part to move. Establish how these are represented in the destination before starting.

Run a pilot on a representative slice

Migrate a few hundred assets chosen to cover your awkward cases. The oldest, the largest, ones with unusual characters in their names, ones with rich metadata, ones with derivatives. Verify them fully. A pilot surfaces mapping errors while they still cost hours to fix rather than weeks.

Run both systems in parallel before cutting over

Keep the source system readable while the destination becomes authoritative. A hard cutover means any discovery of missing data is an emergency. Parallel running means it is a correction. Set an explicit date when the source is decommissioned, or it will run indefinitely.

Verify by checksum and by count, then spot-check by hand

Compare the destination against the pre-migration manifest: every asset present, every checksum matching, counts agreeing per collection. Then have a human open a random sample and confirm the metadata and relationships look right, automated verification confirms the bytes arrived, not that the archive is usable.

Pro Tips

Never delete from the source until verification is complete and signed off. Storage is cheaper than a lost archive.
Migrate a copy, never the original. A migration that modifies the source has no rollback.
Expect character encoding problems in older metadata. Accented characters and smart quotes are where they surface.
Budget several times longer than the raw transfer time. Verification and correction dominate the schedule.
Write down what you could not migrate. Future you will want to know whether a field is missing or never existed.

Knowledge Base

What You'll Learn

Migrations fail on relationships and on proof, rather than on moving bytes. Below: why relationships are the hard part, and what verification actually has to demonstrate.

Relationships Are the Hard Part

Moving files is a solved problem. Moving metadata is mostly a mapping exercise. Moving the relationships between assets is where migrations fail, and it is the part least often planned for.

An asset system encodes a great deal in links: which proxy belongs to which master, which version supersedes which, which assets form a project or collection, which delivery came from which source. None of that is visible in the files themselves. It lives in database rows joining internal identifiers.

When those identifiers are proprietary and meaningless outside the source system, an export produces a pile of files and a table nobody can reconstruct. The archive technically arrives and is functionally destroyed, because the thing that made it navigable did not come with it.

This is why the question to ask a vendor before purchase is not "can we export" but "can we export assets, metadata, and relationships in a documented open format, without your assistance". A migration is where that answer is finally tested, and by then it is too late to choose differently.

What Verification Has to Demonstrate

A migration is not finished when the copy completes. It is finished when you can demonstrate three things.

Completeness: every asset in the pre-migration manifest exists in the destination. This requires the manifest to have been made before the migration. A count taken afterwards proves nothing about what was there to begin with.

Integrity: every file's checksum matches its recorded value. This catches silent corruption during transfer, which is rare per file and near-certain across a large archive.

Usability: the archive still works as an archive, search returns sensible results, proxies link to their masters, collections are intact, rights information is attached to the right assets. This part cannot be automated meaningfully and requires a human opening a random sample and using it.

Teams routinely do the first two and skip the third, then discover months later that the metadata arrived but the relationships did not, at which point the source system has been decommissioned and the information is gone.

Where This Fits

This guide covers one specific part of media asset management. The wider picture, metadata schemas, naming conventions, proxies and storage tiers, governance, and avoiding vendor lock-in, is in Media Asset Management (MAM) Explained, which frames the discipline as a whole and links out to the detailed guides underneath it, including this one. If you are starting from scratch rather than solving a specific problem, read that first and come back here.

FAQ

Q: How long should a media archive migration take?
A: Considerably longer than the raw transfer time: verification, correction, and parallel running dominate the schedule. Budget for a pilot, a full migration, a verification pass, and a period where both systems are readable. Rushing to a hard cutover is the most reliable way to lose something irreversibly.

Q: What is most likely to be lost in a migration?
A: Relationships between assets (master-to-proxy links, version history, collection membership) because they are usually stored as internal identifiers with no meaning outside the source system. Files and flat metadata fields transfer relatively easily. The structure that made the archive navigable is what disappears.

Q: How do I prove nothing was lost?
A: Generate a complete manifest with checksums before the migration begins, then verify the destination against it. Every asset present, every checksum matching, counts agreeing per collection. Then have a person open a random sample and confirm it is actually usable, since automated checks prove the bytes arrived and not that the archive works.

Q: Should I delete the old system once migration completes?
A: Not until verification is complete, signed off, and the new system has been in real use for a meaningful period. Run both in parallel with the source readable, and set an explicit decommissioning date so parallel running does not become permanent. Storage costs far less than an unrecoverable archive.