Core Code
Core Code Modern Dev Handbook
DATABASE-ADVANCED Contents
Storage & Index Internals
DB Page Layouts B-Tree Internals Hash, GIN, and GiST Indexes Selectivity & Cardinality Index Maintenance & Bloat Covering & Partial Indexes WAL & Write Amplification MVCC and HOT Updates
Query Planner & Execution
Query Lifecycle Cost-Based Optimizer Basics Reading EXPLAIN ANALYZE Join Strategies Index Scan vs Seq Scan Statistics & Misestimation Plan Regressions Query Optimization Checklist
Transactions & Isolation
ACID Realities Isolation Levels Deep Dive Phantoms, Lost Updates, and Anomalies Deadlocks: Detection and Retries Locking Mechanisms Long-Running Transactions Optimistic vs Pessimistic Concurrency Transaction Safety Checklist
Replication & High Availability
Replication Types Postgres Replication Model MySQL Replication Model Replica Lag and Read-After-Write Failover Strategies Backup Strategies Point-in-Time Recovery HA Architecture Patterns
Sharding & Horizontal Scale
Vertical vs Horizontal Scaling Sharding Patterns Shard Key Design Cross-Shard Queries Rebalancing and Resharding Multi-Region Data Architectures Global Consistency Tradeoffs Sharding Operational Checklist
Postgres vs MySQL Tradeoffs
Storage Engine Differences JSON Support Comparison Indexing Capability Differences Replication Differences Ecosystem and Extensions Interpreting Benchmarks Choosing Between Postgres and MySQL Migration Surface Area
Caching & Redis Strategies
Caching Patterns Redis Data Structures in Production Cache Invalidation Cache Stampede and Thundering Herd TTL Design Strategies Read-Heavy Optimization Cache Consistency Models Cache Observability Checklist
Data Modeling at Scale
Normalization vs Denormalization Soft Delete Patterns Event-Driven Modeling Time-Series Modeling Multi-Tenant Schema Design Audit Trail Design Schema Migration Strategies Data Modeling Checklist

Event-Driven Modeling

Events, projections, and schema evolution patterns that scale organizationally.
On this page

Event-Driven Modeling: Design for Replay, Not Just for Today

Event-driven modeling treats state changes as events. Instead of modeling only the current state, you model the sequence of changes that produced it. This enables auditability, rebuildable read models, and better integration boundaries, but introduces ordering and operational complexity.

Production rule: if you adopt events, you must design for replay and idempotency from day one.

State Tables vs Event Log

Traditional modeling stores current state:

  • orders table has current status

Event-driven modeling stores changes:

  • OrderCreated, OrderPaid, OrderShipped events

You can still keep a current-state table, but it becomes a projection derived from events.

Append-Only: Why It Matters

Events are typically append-only. You do not mutate old events; you add new events.

Benefits:

  • Audit trail for free
  • Rebuildable projections
  • Debugging via history

Costs:

  • Data volume grows continuously
  • Need retention/archival strategy
  • Queries on current state require projections

Outbox Pattern: Make Events Transactional

The hardest problem is ensuring you do not lose events or emit events for writes that never committed.

Outbox pattern:

  • Write domain state changes
  • Write corresponding event row into an outbox table in the same DB transaction
  • A publisher process reads outbox and publishes to the event bus

Production advantage: DB commit is the source of truth for whether an event exists.

Idempotency: Events Will Be Delivered More Than Once

Most event systems provide at-least-once delivery. Consumers must be idempotent.

Practical approach:

  • Each event has a unique id (event_id)
  • Consumer stores processed event_id (or last offset per partition)
  • Duplicate deliveries are ignored

Failure mode: consumers apply the same event twice (double charge, double increment).

Ordering: Define What Must Be Ordered

Global ordering is expensive or impossible. Instead, define ordering per entity:

  • All events for order_id must be processed in order
  • Events across different orders can be processed concurrently

Production rule: design event partitioning/routing to preserve per-entity order.

Exactly-Once Illusion

Teams often chase “exactly-once.” In production, it is usually achieved by combining:

  • At-least-once delivery
  • Idempotent consumers
  • Transactional boundaries where needed (outbox)

This is practical and reliable without requiring a mythical perfect transport.

Read Models (CQRS): Optimize for Queries

Events are great for integration, but most applications need fast reads. Build projections:

  • OrderSummary table for UI
  • UserBalance table derived from payment events
  • Search index derived from content events

Projection design rules:

  • Projections must be rebuildable from event history
  • Projection updates must be idempotent
  • Projection lag is a first-class metric

Replay and Backfill: The Real Superpower (and Risk)

Because you have events, you can replay to rebuild projections or fix bugs.

But replay is also dangerous:

  • Can overload downstream systems
  • Can re-trigger side effects if consumers are not properly separated

Production patterns:

  • Separate “projection consumers” from “side effect consumers”
  • Use replay-safe modes (no external calls during replay)
  • Throttle replay and monitor progress

Schema Evolution for Events

Events are contracts. Changing them breaks consumers.

Rules:

  • Prefer additive changes (new fields) over breaking changes
  • Version event schemas explicitly
  • Keep backward compatibility for a defined window

Failure Modes in Production

  • Lost events: emitting events outside the DB transaction.
  • Ghost events: event emitted but DB write rolled back.
  • Duplicate side effects: non-idempotent consumer.
  • Ordering bugs: out-of-order processing breaks invariants.
  • Projection lag: read model becomes stale and user experience breaks.
  • Replay disaster: replay triggers external calls and doubles actions.
  • Contract break: event schema change breaks downstream services.

Operational Checklist

  • Use outbox (or CDC) to make event emission transactional with DB writes.
  • Make every consumer idempotent; store processed event ids/offsets.
  • Define ordering scope (per-entity ordering) and enforce partitioning accordingly.
  • Build projections for hot queries; measure projection lag.
  • Ensure projections are rebuildable; rehearse rebuild in staging.
  • Design replay mode that avoids external side effects.
  • Implement throttling for replay/backfill to protect downstream systems.
  • Version event schemas; prefer additive evolution.
  • Monitor outbox backlog, consumer lag, error rate, and retry storms.
  • Document event contracts and ownership (who produces, who consumes).

Summary

Event-driven modeling improves auditability and scalability by representing state changes as append-only events. In production, it only works if you treat events as contracts, emit them transactionally (outbox), build idempotent consumers, define ordering scope, and design for replay/backfill without causing side effects. The operational model is as important as the schema.

Time-Series Modeling →