At a previous job, our company acquired a competitor and inherited their entire product catalog: over 100,000 SKUs in a data model that agreed with ours about almost nothing. The initial plan was the default plan everywhere — export both catalogs to spreadsheets and let a team reconcile them by hand. I ended up building the pipeline that replaced that plan, and the merge finished roughly 80% faster than the manual track. The speed wasn't the interesting part. The interesting part was which decisions made the pipeline survivable.
## Map the model before the data
The instinct in a catalog merge is to start moving rows on day one, because rows are visible progress. We spent the first stretch moving nothing — instead writing an explicit mapping between the two product models: which of their fields meant the same thing as ours, which looked the same but weren't (their 'weight' included packaging; ours didn't), and which had no counterpart at all. Every one of those semantic mismatches was cheap to resolve as a mapping decision and brutal to resolve after 40,000 rows had already flowed through a wrong assumption.
## Validation as a first-class citizen
We built the validators before the transforms, as reusable components with the same standards used for supplier onboarding — not as a QA step bolted on the end. Every record either passed the full rule set or landed in a quarantine table with a machine-readable reason. That decision changed the politics of the merge: instead of arguing about whether the data was 'good enough,' we had a dashboard of exactly which records failed which rules, and business owners could triage by category. Rejected-data rates that would have been discovered in production got discovered in week two.
## Idempotency is the whole game
An acquisition-era pipeline never runs once. Source systems keep changing while the deal closes, mappings get corrected, and someone will always find a category that was transformed wrong after the fact. Every stage of ours was rerunnable: deterministic keys, upserts instead of inserts, and transforms that produced the same output for the same input no matter how many times they ran. 'Fix the mapping and rerun category 7' was a ten-minute operation, not a restore-from-backup event. If your pipeline can only be run forward once, you don't have a pipeline — you have a stunt.
## Report from the pipeline, not around it
While the merge ran, Purchasing, Sales, and Operations still needed numbers — and the worst outcome would have been each team building private spreadsheets against half-merged data. We pointed 20+ automated reports at the pipeline's own outputs, quarantine tables included, so everyone watched the same consolidation state. The reports that started as merge monitoring saved an estimated 12 hours a week of manual reporting and outlived the acquisition itself.
## Takeaways
If a catalog merge is headed your way: spend unglamorous time on the model mapping before any data moves, make validation a gate with a quarantine lane rather than a report at the end, build every stage to rerun safely, and give the business visibility from inside the pipeline. Acquisitions are chaotic by nature — the pipeline is the one part you can make boring, and boring is what survives.