Hey everyone, welcome to the thirty-eighth issue of The Main Thread.
It is true that event sourcing rescue systems that needed full audit trails, regulatory compliance, and the ability to answer questions about the past that nobody thought to ask at the time. It is also true that sometimes event sourcing turns straightforward CRUD apps into multi-year archaeology projects. Both outcomes come from the same pattern, applied with different judgement.
The pattern is simple to describe: instead of storing the current state of an entity, we store the sequence of events that produced the state. The current state is then a function of the events - derivable, replayable, predictable into any view we need. If done well, it gives us a system with perfect history, natural audit, and remarkable flexibility in how we read our data. If done poorly, it gives us a system where adding a field requires understanding the entire history of how the model has evolved over the last four years.
The honest answer to “should we use event sourcing?“ is “almost certainly not, but if you need it, you really need it”. Usually, other architectural patterns don’t have this binary character, but event sourcing does. The cost of adoption is high enough that doing it for the wrong reasons is a serious mistake; the value when we have the right reasons is high enough that simulating it with traditional storage is also a serious mistake.
This is the pragmatic, opinionated guide. I will cover what event sourcing actually is, the patterns that make it work, the failure modes that make it painful, and most importantly, the criteria for deciding whether we should adopt it at all.
The Core Idea
Traditional storage records the current state of our entities. A bank account has a balance of $500. An order has a status of “SHIPPED”. A user has an email of “[email protected]“. When something changes, we overwrite the old value.
Event sourcing inverts this. The source of truth is an append-only log of events: facts about things that happened. The current state is a derived view, computed by folding the events together.
Traditional:
account_balance: $500
Event-sourced:
Event 1: AccountOpened(account_id=54, initial_deposit=$1000)
Event 2: MoneyWithdrawn(account_id=54, amount=$300)
Event 3: MoneyWithdrawn(account_id=54, amount=$200)
→ Derived state: balance = $500Three properties matter and define the rest of the design:
Events are immutable
Once written, an event cannot be modified or deleted. If we got it wrong, we write a corrective event. The log is append-only, hence the history is preserved.
Events are facts, not commands
In above example WithdrawMoney is a command - a request to do something. MoneyWithdrawn is an event - a record that it happened. This difference matters: commands can be rejected, events cannot. The naming conventions (past-tense verbs) are not aesthetic; it reflects this semantic difference.
State is derived, never stored as a source of truth
We may cache derived state for performance (snapshots, projections), but it is always reconstructable from the event log. If our derived state diverges from what the events would produce, the events win.
These properties make event sourcing useful. They are also what makes it hard. Every operational practice we have internalized for mutable databases - schema migrations, deletes, updates, the ability to “fix” data in production - has to be rethought.
Designing the Event Store
The event store is the durable, append-only log of events. It is the foundation everything else sits on, and most of the failure modes of event-sourced systems come from getting it subtly wrong.
A working event store has four required capabilities:
Append events atomically to a stream (the sequence of events for a single aggregate).
Read events from a stream in order.
Optimistic concurrency control: rejects appends that would conflict with concurrent writes.
Subscribe to all events in order across all streams (for projections and integrations).
Following the minimal schema in Postgres that captures the essentials:
CREATE TABLE events (
global_position BIGSERIAL PRIMARY KEY,
stream_id TEXT NOT NULL,
stream_version BIGINT NOT NULL,
event_type TEXT NOT NULL,
event_data JSONB NOT NULL,
metadata JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (stream_id, stream_version)
);
CREATE INDEX events_stream_idx ON events (stream_id, stream_version);The stream_id is the aggregate identifier. For example: account-54. The stream_version is the position within that stream, starting from 1 and incrementing by 1 with each event. The unique constraint on (stream_id, stream_version) is what gives us optimistic concurrency: if two writers try to append the same version, one of them gets a constraint violation and must retry.
Appending an event:
def append_event(
conn,
stream_id,
expected_version,
event_type,
event_data,
metadata
):
next_version = expected_version + 1
try:
conn.execute(
"""
INSERT INTO events (stream_id, stream_version, event_type, event_data, metadata)
VALUES (%s, %s, %s, %s, %s)
""",
(
stream_id,
next_version,
event_type,
event_data,
metadata
),
)
except UniqueViolation:
raise ConcurrencyConflict(
f"Stream {stream_id} was modified concurrently. "
f"Expected version {expected_version}, got conflict."
)Loading a stream and applying events:
def load_aggregate(conn, stream_id, aggregate_class):
rows = conn.execute("""
SELECT stream_version, event_type, event_data
FROM events
WHERE stream_id = %s
ORDER BY stream_version
""", (stream_id,)).fetchall()
aggregate = aggregate_class()
for version, event_type, event_data in rows:
aggregate.apply(event_type, event_data)
aggregate.version = rows[-1].stream_version if rows else 0
return aggregateThe aggregate (e.g., Account) implements an apply method per event that mutates its in-memory state. The aggregate is never persisted directly; only its events are.
A few production-grade considerations the minimal schema misses:
Don’t put projections in the same database as the event store. They have different access patterns and different operational characteristics. Co-locating them is convenient at first and painful later.
Be deliberate about what counts as “an event“. Coarse-grained events (e.g. OrderUpdated with the whole order) are easy to write but useless for analysis and brittle for evolution. Fine-grained events (e.g. OrderShippingAddressChanged, OrderItemAdded, OrderItemRemoved) cost more upfront but are what gives us the audit and replay benefits. If our events are just CRUD verbs, we have audit logging, not event sourcing.
Event metadata matters. We must always store: who triggered the event (user/service ID), when (server timestamp), correlation/causation IDs (which command produced this, which event caused this command). We will want that six months in.
Decide if we need multi-stream transactions. Most event-sourced systems require an event to belong to exactly one stream and forbid cross-stream transactions. This is a deliberate constraint because it makes the model scalable and forces us to design around saga patterns when we need cross-aggregate coordination. We must not fight this but design with it.
For real systems, we can build on Postgres (works well up to high tens of thousands of events/second), use a purpose-built event store like EventStoreDB, or use Kafka as an event store with caveats (more on that later). The schema above is the conceptual model regardless of the underlying technology.
Projections: Building Read Models
The event log is the source of truth, but it is a terrible thing to query directly. “How many active customers did we have last Tuesday?“ is not a question we answer by replaying the entire event log every time someone asks. We need read models.
A projection is a process that subscribes to the event stream and maintains a read-optimized view. The view can be an SQL table, a document store, a search index, an in-memory cache, whatever fits the query pattern.
class ActiveCustomersProjection:
def __init__(self, db):
self.db = db
def handle(self, event_type, event_data, position):
if event_type == "CustomerRegistered":
self.db.execute(
"INSERT INTO active_customers (id, registered_at) VALUES (%s, %s)",
(event_data["customer_id"], event_data["registered_at"])
)
elif event_type == "CustomerDeactivated":
self.db.execute(
"DELETE FROM active_customers WHERE id = %s",
(event_data["customer_id"],)
)
self.db.execute(
"UPDATE projection_state SET last_position = %s WHERE name = 'active_customers'",
(position,)
)The projection records its position so it can resume after a restart. We can have many projections, each tailored to a specific query pattern. A single event might update half a dozen of them.
Three properties of projections that we have to design around:
Projections are eventually consistent. When a command produces an event, the projection updates after the event is written. There is a window - usually milliseconds, sometimes second - where the read model lags. Our UI has to tolerate this, or we have to provide a “read your own writes“ guarantee through other means (returning the new state from the command handler, sticky reads, etc.).
Projections can be rebuilt. This is the superpower. If we discover a bug in a projection, or we want to add a new view, we re-run the projection from event 0. The event log is the truth; projections are disposable. In practice, this is harder than it sounds at scale - replaying billions of events takes hours or days. We should have rebuildability into our operational model from the start.
Projections fail in interesting ways. A projection that crashes mid-process can leave its underlying store in an inconsistent state. We should make projection updates idempotent (track the position alongside data, in the same transaction if possible) so we can replay safely. If our projection is updating an SQL table, this is straightforward. If it is updating an external system that doesn’t support idempotency natively, we need to think harder.
The other thing to internalize: projections and the event store are decoupled by design. We can change a projection’s schema, drop it, rebuild it, add a new one, without touching the event store. This is the operational flexibility that makes event sourcing valuable. It’s also why CQRS shows up next.
CQRS: The Natural Companion
Command Query Responsibility Segregation is the architectural pattern that pairs naturally with event sourcing.
The main idea is that the model we use to write data is different from the model we use to read it. Commands go through a write model that produces events. Queries go through the read models built by projections. The two sides have entirely different schemas, scale independently, and serve different purposes.
We can do CQRS without event sourcing (the read and write models can be derived from the same RDBMS through different paths), and we can do event sourcing without CQRS (we can query the projection that holds aggregate state). But the two together are mutually reinforcing:
Event sourcing gives CQRS a clean way to build read models: the event log is the natural integration point.
CQRS gives event sourcing a clean way to express that the write side is small and structured (aggregates) while the read side is varied and use-case-specific (many projections).
Once we are committed to this split, several things that were hard become easy:
Different read models for different needs. The mobile app’s “order summary” view, the admin dashboard’s “all orders by status“ view, and the analytics team’s “orders by region by day” view are three different projections. We don’t contort to one schema and serve all three; we maintain three projections, each ideal for its query.
Polyglot persistence. The write model lives in Postgres. The user-facing read model lives in Redis. The search read model lives in Elasticsearch. The analytics read model lives in BigQuery. Each is built from the same event stream by a different projection. Nothing about the write side changes.
Independent scaling. Reads typically vastly outnumber writes. With CQRS, we scale the read side horizontally without touching the write side. Some read models can be cached aggressively; the write side can stay strongly consistent.
The trade-off is operational complexity. We now have a write store, multiple read stores, and a fleet of projection processes that have to stay running. The aggregate boundary becomes load-bearing - get it wrong, and we can’t easily fix it later because we have baked it into our event schema. Many teams that adopt CQRS without event sourcing get most of the benefits with less operational pain; many teams that adopt event sourcing without CQRS end up reinventing CQRS badly.
Snapshotting
Loading an aggregate by replaying every event from the beginning works fine when the aggregate has 50 events. It doesn’t work when it has 500k events.
Snapshotting is the standard answer: periodically save the aggregate’s state at a known stream version, and on load, start from the most recent snapshot and replay only events newer than it.
CREATE TABLE snapshots (
stream_id TEXT PRIMARY KEY,
stream_version BIGINT NOT NULL,
aggregate_data JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);def load_aggregate_with_snapshot(
conn,
stream_id,
aggregate_class
):
snapshot = conn.execute(
"SELECT stream_version, aggregate_data FROM snapshots WHERE stream_id = %s",
(stream_id,)
).fetchone()
if snapshot:
aggregate = aggregate_class.from_snapshot(snapshot.aggregate_data)
from_version = snapshot.stream_version
else:
aggregate = aggregate_class()
from_version = 0
rows = conn.execute("""
SELECT stream_version, event_type, event_data
FROM events
WHERE stream_id = %s AND stream_version > %s
ORDER BY stream_version
""", (stream_id, from_version)).fetchall()
for version, event_type, event_data in rows:
aggregate.apply(event_type, event_data)
aggregate.version = rows[-1].stream_version if rows else from_version
return aggregateA few decisions we will have to make:
When to snapshot. Common policies: every N events, every T minutes since the last snapshot, and on-demand when load latency exceeds a threshold. Most systems are fine with “every 100 events”. There’s rarely a reason to snapshot synchronously inside a command path; we do it in a background process that watches the event stream.
How many snapshots to keep. At minimum the latest. In practice, we should keep a few historical snapshots so we can recover if a recent snapshot is corrupted (e.g., the aggregate code had a bug at the time the snapshot was taken).
Snapshots are not source of truth. This bears repeating because many engineers forget it. A snapshot is a cache. If the aggregate code changes, our old snapshots are wrong. We should either rebuild them or delete them. We mustn’t trust a snapshot taken by an old version of the code.
The last point is the connection to schema evolution.
Schema Evolution: The Hardest Part
The naive view is: events are immutable, so we can never change them. The reality is: our business changes, our understanding of the domain changes, and we will absolutely need to evolve our schema. The question is how.
Event sourcing forces us to confront a problem most systems avoid. In a CRUD system, when we add a column or rename a field, we migrate the database and move on. The historical state in those rows is lost, so there’s nothing to evolve. In an event sourced system, our historical events are still there. We will be reading them for the lifetime of the system. They have to remain interpretable forever.
There are four practical strategies, listed roughly from simplest to most powerful.
Strategy 1: Additive Changes Only
New fields are optional. New event types are introduced for new behaviour; old event types are never modified. Code that reads old events ignores fields it doesn’t recognize. This works for the vast majority of changes if we discipline ourselves.
Strategy 2: Versioned Event Types
When a meaningful change is needed, we should introduce a new event type. OrderShipped becomes OrderShippedV2. Both versions exist in the log forever. Read code handles both. The downside is that our code accumulates handlers for old versions. The upside is each version is statically clear and self contained.
class OrderProjection:
def handle(self, event_type, data):
if event_type in ("OrderShipped", "OrderShipped_v2"):
self.handle_shipped(data)
def handle_shipped(self, data):
# v2 added 'carrier' field; v1 events won't have it
carrier = data.get("carrier", "unknown")
...Strategy 3: Upcasters
A pipeline that transforms old event versions into the latest version on read. The aggregate code only ever sees the current version; the upcaster does the translation. This keeps domain code clean at the cost of an extra layer.
def upcast(event_type, data, version):
if event_type == "OrderShipped" and version == 1:
# v1 didn't have carrier; default it
data = {**data, "carrier": "unknown"}
version = 2
return event_type, data, versionStrategy 4: Weak Schema
We should use a flexible serialization format (JSON, Avro with schema evolution rules) and code defensively. Read code uses field defaults, treats missing fields as optional, and never relies on field order. This is the lowest ceremony approach and works well for many systems.
What we cannot do - at least, not without serious operational effort - is delete historical events or change their semantics. If we ever realize an event was wrong, we write a compensating event, we don’t rewrite history. There are exceptions (GDPR right-to-be-forgotten requirements may force us to actually erase personal data from old events), and they are uniformly painful. Crypto-shredding (encrypting personal data with per-subject key, and deleting key on request) is the least bad workaround, but it has to be designed in from the start.
The lesson here is: our event schema is a public API. We should treat it with same care. Once an event is in the log, we live with it forever. This is freeing in some ways (we can never lose data) and constraining in others (we can never quite take a design decision back).
When NOT to Use Event Sourcing
Most articles about event sourcing are written by people for whom it worked. Survivorship bias is severe. The honest version is that event sourcing is the wrong choice for the majority of systems, and adopting it for the wrong reasons is one of the more expensive architectural mistakes you can make.
Don’t use event sourcing if:
Your domain is fundamentally CRUD
A user profile editor, a content management system, a configuration store - these don’t have a meaningful “history of events” that’s interesting beyond “what’s the current state?” The events you would write would be UserUpdated, ContentEdited, SettingsChanged. That’s not event sourcing; that’s CRUD with extra steps.
You don’t need an audit trail
If "who did what when" doesn't matter for compliance, debugging, or business analysis, you are paying the cost of event sourcing for benefits you don't use. A traditional database with an audit table covers most "occasional history lookup" needs at a fraction of the complexity.
Your team doesn't have event-sourcing experience.
The patterns are subtle. Aggregate boundaries are hard to get right and expensive to change later. Schema evolution requires discipline that takes time to develop. A team encountering all this for the first time on a production system will get hurt. If you are going to adopt it, do it on a non-critical system first.
You need ad-hoc analytical queries on current state.
Event sourcing is terrible for "give me all customers in California with active subscriptions and recent activity." You would need a projection that pre-computes that view, or you would have to extract data into a traditional database for querying. If most of your data access is exploratory analytics, the projection overhead is not worth it.
Your aggregates are unbounded in size.
A "shopping cart" aggregate is bounded: most carts have a few items. A "user activity log" aggregate is unbounded: it grows forever. Unbounded aggregates lead to slow loads, expensive snapshots, and memory pressure. You can mitigate with snapshotting and stream truncation, but at that point you have half-built a traditional database.
You think it will be easier to scale.
It usually won't. The write side of an event-sourced system is constrained by the same things any append-only log is constrained by; the read side is constrained by your projection infrastructure. Both can scale, but neither is automatically better than well-designed traditional storage.
You are using Kafka and calling it event sourcing.
Kafka is a great event log for inter-service communication. Using it as an aggregate event store has serious problems: no per-stream optimistic concurrency, retention windows that conflict with "events are forever," partition-level ordering that doesn't match aggregate boundaries. There are patterns for making it work (one-aggregate-per-partition, infinite retention, custom concurrency control), but you are fighting the tool. Use a real event store or a relational database for the event store, and use Kafka for downstream propagation.
The legitimate use cases - and they exist - share characteristics:
Regulatory or compliance reasons require complete audit history (banking, healthcare, legal).
The business genuinely cares about temporal questions ("what did we know about this customer two weeks ago?", "reconstruct the state at the time of this dispute").
Multiple read views over the same data justify the projection overhead.
Domain modeling benefits from an event-driven worldview (the business already thinks in events: trades, shipments, claims, transactions).
The team has the experience and operational maturity to handle the long-term complexity.
If two or three of these apply to you, event sourcing might be the right call. If none do, you are chasing an architecture pattern just for the sake of it.
The Takeaway
Event sourcing is one of the most powerful patterns in distributed systems, and one of the most over-prescribed. It rewards systems that have a real reason for it and punishes systems that adopted it because it was on a conference slide.
The good version of event sourcing gives you something traditional storage simply cannot: a complete, immutable record of how your system arrived at its current state, queryable from any angle you eventually realize you need. For systems where the past matters such as financial systems, regulated industries, complex business processes, this is enormous. The complexity tax is real, but the alternative (faking history with audit tables and after-the-fact reconstructions) costs more.
The bad version of event sourcing gives you everything you would get from a normal database, plus eventual consistency you didn't need, plus operational complexity you can't afford, plus a schema you can never quite change. This is the version that ends up rewritten, three years in, by a different team that wonders aloud what the original architects were thinking.
Is event sourcing a good pattern? It is. But is the past valuable enough in this system to justify treating events as the source of truth? If the answer is yes, commit fully - the half-measures will hurt you. If the answer is no, build the boring CRUD system and add an audit table. Reserving the complexity for where it pays off is most of what good architecture is.
Store events when you need history. Store state when you don't. Knowing the difference is the whole skill.
Have you adopted event sourcing - or de-adopted it? I am collecting stories from both sides: the systems where it was the right call and the ones where it became the long unwinding. The patterns and the post-mortems are both educational.
Hit reply. I read everything.
Namaste!
— Anirudh
If this clicked, forward it to your team. Event sourcing is one of those patterns that everyone has opinions about and few people have shipped at scale. And if you want more deep dives like this, subscribe to The Main Thread - practical distributed systems engineering, one essay per week.
