Database Systems
From the relational model to learned indexes — the data structures, transaction protocols, and storage architectures that organise the world's information.
Minimum viable reading path
The 3 papers that give you most of the field's mental model, in reading order.
01 A Relational Model of Data for Large Shared Data Banks MVRP
Proposes the relational model: data as relations (tables), with set-theoretic operators for querying.
The single founding document of modern databases. Read it to see SQL's entire intellectual scaffolding in 11 pages.
Set theory, basic logic
Separate the logical structure of data from how it's physically stored.
02 System R: Relational Approach to Database Management
Reports on the first full implementation of a relational system, including SQL, optimisation, and transactions.
The blueprint of every commercial RDBMS that followed.
Codd's relational model
Almost every modern RDBMS architectural choice was made first in System R.
03 The Notions of Consistency and Predicate Locks
Defines serialisability and lays out the locking protocols for transactions.
The transaction-theory paper underlying every concurrency-control implementation.
Concurrency basics
Two-phase locking + serialisability is the recipe for transactional correctness.
04 ARIES: A Transaction Recovery Method
Write-ahead logging with redo, undo, and physiological logging — the basis of every modern crash-recovery system.
The reference for how databases survive crashes; foundational for storage engineers.
Two-phase locking, B-trees
Write-ahead logging plus carefully ordered redo/undo gives durability without giving up performance.
05 Hekaton: SQL Server's Memory-Optimized OLTP Engine
A latch-free, lock-free in-memory OLTP engine that uses MVCC and compiled query plans.
A clean, modern industry account of in-memory transaction processing.
ARIES, concurrency control
When everything fits in RAM, the entire database design changes.
06 C-Store: A Column-Oriented DBMS
Argues that storing data by column, not row, is the right design for analytical workloads.
Read this against System R to see how analytical workloads pulled DB design in a new direction.
System R
OLTP and analytics want fundamentally different storage layouts.
07 The End of an Architectural Era (H-Store)
A polemic against general-purpose RDBMSs, arguing for specialised in-memory engines.
A bracing critique that anticipates Hekaton, MemSQL, and most modern OLTP engines.
System R, basic OLTP
General-purpose database architectures leave too much performance on the table.
08 The Vertica Analytic Database: C-Store 7 Years Later
A retrospective on what worked and didn't in commercialising C-Store.
A rare, candid post-mortem on translating a research prototype into a production system.
C-Store
Real workloads break clean research designs in interesting and instructive ways.
09 Bigtable: A Distributed Storage System for Structured Data MVRP
A sparse, distributed, persistent multi-dimensional sorted map built atop GFS and Chubby.
A foundational system; deserves to be read once for distributed systems and once for databases.
GFS, LSM-trees
A simple sorted-key data model plus an LSM scales to huge datasets.
10 Dynamo: Amazon's Highly Available Key-value Store
A masterless, eventually-consistent KV store that prioritises availability over consistency.
The conceptual ancestor of every AP-side NoSQL store.
Distributed systems basics, consistent hashing
Tuneable consistency at the operation level beats a global setting.
11 PNUTS: Yahoo!'s Hosted Data Serving Platform
A geo-replicated database with per-record timeline consistency between strong and eventual.
A pragmatic middle-ground design between Dynamo and Spanner — clarifies the consistency spectrum.
Distributed systems basics
Real applications usually want a consistency knob, not a fixed setting.
12 The Log-Structured Merge-Tree (LSM-Tree)
A write-optimised data structure that batches updates in memory and merges them to disk in sorted runs.
The foundational data structure of every modern KV store: RocksDB, LevelDB, Cassandra.
B-trees, basic algorithms
Trade some read amplification for huge wins on write throughput.
13 The Case for Learned Index Structures
Replaces B-trees and Bloom filters with neural networks that learn the data's CDF.
A speculative-but-influential paper redefining what a data structure can be.
B-trees, basic ML
A data structure is just a function; you might as well learn it.
14 Spanner: Google's Globally-Distributed Database MVRP
A globally-distributed SQL database with externally-consistent transactions powered by TrueTime.
The watershed paper that made global, strongly-consistent transactions practical.
Bigtable, Paxos
Bound your clock skew, wait it out, and you can serialise transactions across continents.
15 F1: A Distributed SQL Database That Scales
The SQL layer atop Spanner that runs Google's AdWords business.
Companion piece to Spanner; shows the SQL engineering needed on top of a distributed KV layer.
Spanner, SQL internals
Most apps want SQL even when they need a distributed KV — so build the SQL layer well.
16 The Snowflake Elastic Data Warehouse
Decouples compute and storage in the cloud, with elastic, billed-by-the-second virtual warehouses.
A clean modern reference on the cloud-native warehouse architecture now industry-standard.
C-Store, cloud architecture basics
Decoupled compute and storage is the right shape for the cloud.
17 CockroachDB: The Resilient Geo-Distributed SQL Database
An open-source SQL database that approximates Spanner's guarantees without atomic clocks.
A clear case study in re-implementing a research system as production-grade open source.
Spanner, Raft
Hybrid logical clocks and careful engineering can substitute for TrueTime.
18 DuckDB: An Embeddable Analytical Database
An in-process columnar SQL engine designed for analytical queries on a single machine.
The "SQLite of analytics" — represents a real movement back toward single-node power.
C-Store
Most analytical workloads fit on one big modern machine, and shouldn't pay distributed overhead.
19 Photon: A Fast Query Engine for Lakehouse Systems
A vectorised C++ execution engine for Apache Spark that beats hand-tuned warehouses on TPC-DS.
A modern reference on vectorised execution and the lakehouse architecture.
Spark, vectorised execution
You can replace a slow JVM execution layer without breaking the rest of the stack.
20 Self-Driving Database Management Systems
Argues for autonomous DBMSs that tune themselves using ML over their workloads.
A useful manifesto for where database operations is heading.
Database internals
The DBMS itself is a control system that should learn its own configuration.
21 FAISS: A Library for Efficient Similarity Search
GPU-accelerated billion-scale nearest-neighbour search via inverted indexes and product quantisation.
The reference work for vector search; foundation of every "vector database".
Linear algebra, basic IR
Approximate nearest-neighbour search is mostly an indexing and quantisation problem.
22 Lakehouse: A New Generation of Open Platforms
Argues for unified data architectures combining the openness of lakes with the management of warehouses.
Frames the data-platform debate of the early 2020s.
Snowflake, data lakes basics
Schema, transactions, and governance can live atop open file formats.
23 RocksDB: Evolution of Development Priorities in a Key-Value Store
A retrospective on a decade of running RocksDB as a foundational storage engine inside Meta.
Honest reflections on what really matters in a long-lived storage engine.
LSM-tree
Real-world priorities (efficiency, then features, then performance) shift dramatically over time.
24 Vectorized Query Execution at Microsoft (SQL Server Batch Mode)
Adds batch-mode columnstore execution into SQL Server, mixing column and row processing in one engine.
Underrated production retrofit paper; useful contrast with the column-only DuckDB and Photon approaches.
C-Store, query execution basics
You don't have to ditch your row engine — batch mode and column stores can coexist.