Change Data Capture: Streaming Database Changes

From the System Design Interview Roadmap Series - Part II: Data Storage

You're running an e-commerce platform handling millions of transactions daily. Every time a customer places an order, updates their profile, or modifies their cart, your database changes. Now imagine if you could instantly know about every single change as it happens, without constantly polling your database or writing complex triggers. This is the power of Change Data Capture (CDC) – a technique that has quietly become the backbone of modern distributed systems.

What Change Data Capture Really Is

Change Data Capture is like having a vigilant observer sitting beside your database, meticulously recording every insert, update, and delete operation as it occurs. Think of it as a detailed journal that captures not just what changed, but when it changed and often what it looked like before and after the change.

Unlike traditional batch processing where you might export data every few hours, CDC operates in real-time, streaming changes as they happen. It's the difference between receiving a daily newspaper and having a live news ticker – both inform you, but the immediacy changes everything about how you can respond.