The Curious Engineer

All blogs / Distributed systems / cloud

Marc Brooker's Blog

brooker.co.za/blog

Distributed systems / cloudDatabasesPerformance / optimizationAI / MLEngineering management / careers

’10’15’20’25
Posts
167 from 15 Dec 2008 to 23 Oct 2026
Posting
monthly (23 in the last year)
Kind
personal blog
Feed
https://brooker.co.za/blog/atom.xml

2026 18

  1. Animations for System Builders: Metastability and Lost Writes
  2. Strands Decider: Why Not an Encoder?
  3. Small Decisions: Engineering a Leading Model
  4. Lorenz and Little: How Much Does Your Tail Cost?
  5. Aurora DSQL: Scalable, Multi-Region OLTP
  6. Meet Alice. Alice is impatient.
  7. Is this blog written by AI?
  8. Agentic software development hypothesis
  9. What's Easy Now? What's Hard Now?
  10. It's time to be right.
  11. Spec Driven Development isn't Waterfall
  12. What about juniors?
  13. My heuristics are wrong. What now?
  14. Music To Build Agents By
  15. SFQ: Simple, Stateless, Stochastic Fairness
  16. You Are Here
  17. Pass@k is Mostly Bunk
  18. Agent Safety is a Box

2025 18

  1. On the success of 'natural language programming'
  2. What Does a Database for SSDs Look Like?
  3. What Now? Handling Errors in Large Systems
  4. Why Strong Consistency?
  5. DSQL: Simplifying Architectures
  6. Fixing UUIDv7 (for database use-cases)
  7. Is Systems Research Really Just About Making Numbers Bigger?
  8. Locality, and Temporal-Spatial Hypothesis
  9. Seven Years of Firecracker
  10. Dynamo, DynamoDB, and Aurora DSQL
  11. LLMs as Parts of Systems
  12. Career advice, or something like it
  13. Systems Fun at HotOS
  14. Good Performance for Bad Days
  15. Decomposing Aurora DSQL
  16. One or Two? How Many Queues?
  17. What Fekete's Anomaly Can Teach Us About Isolation
  18. Versioning versus Coordination

2024 19

  1. Snapshot Isolation vs Serializability
  2. DSQL Vignette: Wait! Isn't That Impossible?
  3. DSQL Vignette: Transactions and Durability
  4. DSQL Vignette: Reads and Compute
  5. DSQL Vignette: Aurora DSQL, and A Personal Story
  6. Ten Years of AWS Lambda
  7. Garbage Collection and Metastability
  8. Resource Management in Aurora Serverless
  9. Let's Consign CAP to the Cabinet of Curiosities
  10. Not Just Scale
  11. It's always TCP_NODELAY. Every damn time.
  12. MemoryDB: Speed, Durability, and Composition.
  13. Formal Methods: Just Good Engineering Practice?
  14. Finding Needles in a Haystack with Best-of-K
  15. The Builder's Guide to Better Mousetraps
  16. Better Benchmarks Through Graphs
  17. How Do You Spend Your Time?
  18. Pat's Big Deal, and Transaction Coordination
  19. What is Scalability Anyway?

2023 18

  1. Why Aren't We SIEVE-ing?
  2. It's About Time!
  3. Optimism vs Pessimism in Distributed Systems
  4. Writing For Somebody
  5. Exponential Value at Linear Cost
  6. On The Acoustics of Cocktail Parties
  7. Invariants: A Better Debugger?
  8. My Favorite Bits of OSDI/ATC'23
  9. Bélády's Anomaly Doesn't Happen Often
  10. What is a container?
  11. Container Loading in AWS Lambda
  12. Open and Closed, Omission and Collapse
  13. The Four Hobbies, and Apparent Expertise
  14. Surprising Scalability of Multitenancy
  15. False Sharing versus Perfect Placement
  16. Hot Keys, Scalability, and the Zipf Distribution
  17. NoSQL: The Baby and the Bathwater
  18. Erasure Coding versus Tail Latency

2022 17

  1. Under My Thumb: Insight Behind the Rules
  2. Lambda Snapstart, and snapshots as a tool for system builders
  3. Amazon's Distributed Computing Manifesto
  4. Writing Is Magic
  5. Give Your Tail a Nudge
  6. Atomic Commitment: The Unscalability Protocol
  7. Histogram vs eCDF
  8. What is Backoff For?
  9. Getting into formal specification, and getting my team into it too
  10. The DynamoDB paper
  11. Formal Methods Only Solve Half My Problems
  12. What is a simple system?
  13. Simple Simulations for System Builders
  14. Fixing retries with token buckets and circuit breakers
  15. Will circuit breakers solve my problems?
  16. Software Deployment, Speed, and Safety
  17. DynamoDB's Best Feature: Predictability

2021 12

  1. The Bug in Paxos Made Simple
  2. Serial, Parallel, and Quorum Latencies
  3. Caches, Modes, and Unstable Systems
  4. My Proposal for Arecibo: Drones
  5. Latency Sneaks Up On You
  6. Metastability and Distributed Systems
  7. Tail Latency Might Matter More Than You Think
  8. Redundant against what?
  9. What You Can Learn From Old Hard Drive Adverts
  10. Incident Response Isn't Enough
  11. The Fundamental Mechanism of Scaling
  12. Quorum Availability

2020 12

  1. Getting Big Things Done
  2. Consensus is Harder Than It Looks
  3. Focus on the Good Parts
  4. Surprising Economics of Load-Balanced Systems
  5. A Story About a Fish
  6. Code Only Says What it Does
  7. Some Virtualization Papers Worth Reading
  8. Reading Research: A Guide for Software Engineers
  9. Two Years With Rust
  10. Firecracker: Lightweight Virtualization for Serverless Applications
  11. Physalia: Millions of Tiny Databases
  12. Why do we need distributed systems?

2019 6

  1. Kindness, Wickedness and Safety
  2. When Redundancy Actually Helps
  3. Is Anatoly Dyatlov to blame?
  4. Some risks of coordinating only sometimes
  5. Learning to build distributed systems
  6. Control Planes vs Data Planes

2018 3

  1. Telling Stories About Little's Law
  2. Availability and availability
  3. Balls Into Bins In Distributed Systems

2017 1

  1. Is the Mean Really Useless?

2016 1

  1. Why Must Systems Be Operated?

2015 10

  1. Heuristic Traps for Systems Operators
  2. Is there a CAP theorem for Durability?
  3. CALISDO: Threat Modeling for Distributed Designs
  4. Sodium Carbonate, and Ramenized Pasta
  5. The Zero, One, Infinity Disease
  6. How Amazon Web Services Uses Formal Methods
  7. Jitter: Making Things Better With Randomness
  8. Electoral Trouble in Sybilania
  9. Does Bitcoin Solve Byzantine Consensus?
  10. A Quiet Defense of Patterns

2014 17

  1. Make Your Program Slower With Threads
  2. Two Farmers and Common Knowledge
  3. Exactly-Once Delivery May Not Be What You Want
  4. Ice Cream and Distributed Systems
  5. Harvest and Yield: Not A Natural Cure for Tradeoff Confusion
  6. The Essential Barbara Liskov
  7. The Space Between Theory and Practice in Distributed Systems
  8. Use of Formal Methods at Amazon Web Services
  9. CAP and PACELC: Thinking More Clearly About Consistency
  10. Two traps in iostat: %util and svctm
  11. The Operations Gradient: Improving Safety in Complex Systems
  12. Viewstamped Replication: The Less-Famous Consensus Protocol
  13. The Essential Nancy Lynch
  14. Failure Detectors, and Non-Blocking Atomic Commit
  15. The Essential Leslie Lamport
  16. Snark, Chord, and Trust in Algorithms
  17. Distributed Consensus: Beating Impossibility with Probability One

2013 6

  1. Restricted Transactional Memory on Haswell
  2. Hardware Lock Elision on Haswell
  3. Beyond iostat: Storage performance analysis with blktrace
  4. Some Patterns of Engineering Design Meetings
  5. Exploring TLA+ with two-phase commit
  6. C++11's atomic and volatile, under the hood on x86

2012 8

  1. Java's Atomic and volatile, under the hood on x86
  2. Are volatile reads really free?
  3. Highly contended and fair locking in Java
  4. Expect Less, Get More?
  5. Latency lags bandwidth
  6. The properties of crash-only software
  7. The power of two random choices
  8. The benefits of having data

2008 1

  1. My Publications and Videos