Decomposing Aurora DSQL
Decomposing Aurora DSQL
Earlier today, Alex Miller wrote an excellent blog post titled Decomposing Transaction Systems. It’s one of the best things I’ve read about transactions this year, maybe the best. You should read it now.
In the post, Alex breaks transactions down like this:
Every transactional system does four things:
- It executes transactions.
- It orders transactions.
- It validates transactions.
- It persists transactions.
then describes how these steps map to traditional OCC and PCC systems, research designs like Calvin, and real-world systems like FoundationDB.
How do these steps map to Aurora DSQL?
An overview of DSQL’s architecture may be useful if you haven’t been following along so far:
Now, an hour of video and 20 minutes of reading another post later, we’re back. Let’s dive in.
Executing a transaction means evaluating the body of the transaction to produce the intended reads and writes.
Aurora DSQL executes transactions on a horizontally scalable fleet of Firecracker-isolated, PostgreSQL-powered, query processors. At this stage there’s no coordination at all. MVCC is used for reads during execution, and no writes escape the query processor running that particular transaction. See this post for more on this step.
Ordering a transaction means assigning the transaction some notion of a time at which it occurred. Validating a transaction means enforcing concurrency control, or more rarely, domain-specific semantics.
In DSQL, ordering and validating happen in parallel, done by the adjudicators involved in the transaction. Each adjudicator provides a range of possible orderings (not after, not before), and one final adjudicator makes the final ordering decision. Similarly, each involved adjudicator weighs in on validation, with one final adjudicator checking that everybody says yes. Strictly, the final-final order is chosen only after validation completes, but the ordering and validation protocols run in parallel. See this post for more.
Persisting a transaction makes making it durable, generally to disk.
The transaction is persisted by writing it to a replication log (the Journal). At this point it is durable to stable storage in multiple availability zones (or multiple regions).
DSQL then goes through another step, which is applying that persisted change to the storage nodes to make it available to reads. This happens after the transaction is committed, and isn’t actually required for durability, just visibility. This slightly breaks Miller’s model, but it can be rolled into persisting without too much stretching of the truth.
