The Curious Engineer

All blogs / AI / ML

ezyang's blog

blog.ezyang.com

Recent content on ezyang's blog

AI / MLPythonHaskellCompilers

’10’15’20’25
Posts
457 from 28 Dec 2009 to 17 Apr 2026
Posting
monthly (19 in the last year)
Kind
personal blog
Feed
https://blog.ezyang.com/feed.xml

2026 16

  1. OSS code review, in the era of LLMs
  2. 2026
  3. Autograd and Mutation
  4. Read Less, Steer More
  5. Parallel Agents ❤️ Sapling
  6. AI-assisted programming for spmd_types
  7. Silent BC Breaking Changes
  8. Vibe Coding Design Study: tlparse
  9. Replicate Forwards, Partial Backwards
  10. DTensor erasure
  11. The JAX sharding type system
  12. Global vs Local SPMD
  13. Megatron via shard_map
  14. Computing sharding with einsum
  15. Hugo Migration
  16. The gap between a Helpful Assistant and a Senior Engineer

2025 11

  1. Code review as human alignment, in the era of LLMs
  2. 2025
  3. Learning to love mesh-oriented sharding
  4. Draw high dimensional tensors as a matrix of matrices
  5. So you want to control flow in PT2
  6. The Parallelism Mesh Zoo
  7. You could have invented CuTe hierarchical layout (but maybe not the rest of it?)
  8. State of torch.compile for training (August 2025)
  9. Vibe coding case study: ScubaDuck
  10. Why you should maintain a personal LLM coding benchmark
  11. New Years resolutions for PyTorch in 2025

2024 5

  1. Ways to use torch.export
  2. 2024
  3. Ways to use torch.compile
  4. Tensor programming for databases, with first class dimensions
  5. What's different this time? LLM edition

2021 5

  1. Interactive scraping with Jupyter and Puppeteer
  2. 2021
  3. PyTorch Developer Podcast
  4. Rage bug reporting
  5. The PyTorch open source process

2020 7

  1. The hidden problem(?) with basic block procedures in SSA
  2. 2020
  3. Idiomatic algebraic data types in Python with dataclasses and Union
  4. Let's talk about the PyTorch dispatcher
  5. Dynamic scoping is an effect, implicit parameters are a coeffect
  6. A brief taxonomy of PyTorch operators by shape behavior
  7. vmap in Haskell

2019 4

  1. PyTorch internals
  2. 2019
  3. A short note about functional linear maps
  4. Microsoft Surface Book 2

2018 6

  1. HIW'18: Let’s Go Mainstream with Eta!
  2. 2018
  3. A year into Backpack
  4. A compile-time debugger that helps you write tensor shape checks
  5. Online/offline continuous integration
  6. Semantic Import Versioning in the wild

2017 12

  1. Systems ML workshop panel
  2. Accelerating Persistent Neural Networks at Datacenter Scale (Daniel Lo)
  3. MOCHA: Federated Multi-Tasks Learning (Virginia Smith)
  4. A Machine Learning Approach to Database Indexes (Alex Beutel)
  5. Ray: A Distributed Execution Framework for Emerging AI Applications (Ion Stoica)
  6. 2017
  7. Backpack for deep learning
  8. Proposal: Suggest explicit type application for Foldable length and friends
  9. Prio: Private, Robust, and Scalable Computation of Aggregate Statistics
  10. Designing the Backpack signature ecosystem
  11. How to integrate GHC API programs with Cabal
  12. Try Backpack: Cabal packages

2016 18

  1. A tale of backwards compatibility in ASTs
  2. 2016
  3. Backpack and the PVP
  4. Left-recursive parsing of Haskell imports and declarations
  5. The problem of reusable and composable specifications
  6. Thoughts about Spec-ulation (Rich Hickey)
  7. Try Backpack: ghc --backpack
  8. Seize the Means of Production (of APIs)
  9. The Base of a String Theory for Haskell
  10. The Edit-Recompile Manager
  11. Backpack and separate compilation
  12. cabal new-build is a package manager
  13. Optimizing incremental compilation
  14. What Template Haskell gets wrong and Racket gets right
  15. Debugging tcIfaceGlobal errors in GHC: a study in interpreting trace output
  16. Announcing cabal new-build: Nix-style local builds
  17. Hindley-Milner with top-level existentials
  18. ghc-shake: Reimplementing ghc -​-make

2015 8

  1. The convergence of compilers, build systems and package managers
  2. 2015
  3. What is Stateless User Interface?
  4. Is no-reinstall Cabal coming to GHC 8.0?
  5. Help us beta test "no-reinstall Cabal"
  6. Ubuntu Vivid upgrade (Xmonad)
  7. Width-adaptive XMonad layout
  8. An Eq instance for non de Bruijn terms

2014 23

  1. Unintended consequences: Bound threads and unsafe FFI calls
  2. 2014
  3. Ubuntu Utopic upgrade (Xmonad)
  4. Tomatoes are a subtype of vegetables
  5. Haskell Implementor's Workshop '14
  6. Open type families are not modular
  7. A taste of Cabalized Backpack
  8. The fundamental problem of programming language package management
  9. What's a module system good for anyway?
  10. New theme!
  11. Type classes: confluence, coherence and global uniqueness
  12. Parsec: "try a <|> b" considered harmful
  13. GHC and mutable arrays: a DIRTY little secret
  14. Elimination with a Motive (in Coq)
  15. The cost of weak pointers and finalizers in GHC
  16. Calculating Shanten in Mahjong
  17. Haskell for Coq programmers
  18. Equality, roughly speaking
  19. How to maintain a pristine copy of your configuration files
  20. PEPM'14: The HERMIT in the Stream
  21. Ott ⇔ PLT Redex
  22. When a lock is better than an MVar
  23. So you want to add a new concurrency primitive to GHC...

2013 29

  1. Two bugs in the borrow checker every Rust developer should know about
  2. 2013
  3. Visualizing a block allocator
  4. Xmonad and media keys on Saucy
  5. If you're using lift, you're doing it wrong (probably)
  6. Of Monadic Fixpoints and Heap Offsets
  7. Induction and logical relations
  8. Cost semantics for STG in modern GHC
  9. Blame Trees
  10. OPLSS lecture notes
  11. No grammar? No problem!
  12. HoTT exercises in Coq (in progress)
  13. (Homotopy) Type Theory: Chapter One
  14. The AST Typing Problem
  15. Anatomy of an MVar operation
  16. HotOS "Unconference" report: Verifying Systems
  17. Category theory for loop optimizations
  18. The Difference between Recursion & Induction
  19. Kindle is not good for textbooks
  20. A Zerocoin puzzle
  21. A classical logic fairy tale
  22. Ndseg
  23. Resource limits for Haskell
  24. The single export pattern
  25. The duality of weak maps and private symbols
  26. What is a membrane?
  27. Kindle Paperwhite notes
  28. The GHC scheduler
  29. NLP: the missing framework

2012 55

  1. Google Nexus 7 setup notes
  2. 2012
  3. Metro Maps of the News
  4. Maildir synchronizing Sup
  5. Why can't I just be a little lazy?
  6. Functional Encryption
  7. Extremist Programming
  8. Plan 9 mounts and dependency injection
  9. hp/D3.js: an interactive heap profile viewer
  10. Ubuntu Quantal upgrade (Thinkpad/Xmonad)
  11. ACM XRDS: Jeff Dean profile
  12. Duality for Haskellers
  13. Hails: Protecting Data Privacy in Untrusted Web Applications
  14. Visualizing satisfiability, validity & entailment
  15. GET /browser.exe
  16. Generalizing the programmable semicolon
  17. Template project for GHC plugins
  18. "This is really the End."
  19. Unintuitive facts about Safe Haskell
  20. The Y Combinator and strict positivity
  21. So you want to hack on IMAP...
  22. OfflineIMAP sucks
  23. How OfflineIMAP works
  24. The Monad Reader: Issue 20
  25. Applicative functors
  26. Practical Foundations for Programming Languages (first impressions)
  27. Is Haskell liberal or conservative?
  28. Two ways of representing perfect binary trees
  29. Polymorphic variants in Ur/Web
  30. Managing the server/client split in Ur/Web
  31. Secure multiparty Bitcoin anonymization
  32. Why verification results in higher quality code
  33. Thoughts on gamifying textbooks
  34. An Interactive Tutorial of the Sequent Calculus
  35. Ubuntu Precise upgrade (Thinkpad/Xmonad)
  36. What happens when you mix three research programming languages together
  37. Some thoughts about literature review
  38. How Ur/Web records work and what it might mean for Haskell
  39. Use the source, don't read it
  40. Reduce Ubuntu latency by disabling mDNS
  41. Visit month: Princeton
  42. Is it better to teach formalism or intuition?
  43. Visit month: University of Pennsylvania
  44. You could have invented fractional cascading
  45. Visualizing range trees
  46. Anatomy of "You could have invented..."
  47. Transcript of "Inventing on Principle"
  48. Travel: Spring 2012 Edition
  49. How to build DRM you can trust
  50. Popl
  51. Modelling IO: MonadIO and beyond
  52. monad-control is tricky
  53. Mystery Hunt and the Scientific Endeavour
  54. Problem Set: The Codensity Transformation
  55. Why iteratees are hard to understand

2011 97

  1. Bugs and Battleships
  2. 2011
  3. How to build i686 glibc on Ubuntu
  4. Interactive Demo of Zero-Knowledge Proofs
  5. Accessing lazy structures from C
  6. Transparent xmobar
  7. Ubuntu Oneiric upgrade (Thinkpad/Xmonad)
  8. How to read Haskell like Python
  9. The new Reflections on Trusting Trust
  10. Obviously Correct
  11. Polyglot programming
  12. Why you shouldn't do a PhD in systems
  13. Let's play a game
  14. 8 ways to report errors in Haskell revisited
  15. Joseph and the Amazing Technicolor Box
  16. Diskless Paxos crash recovery
  17. First impressions of module programming
  18. In-program GC stats redux
  19. Changes to IntMap
  20. Food-related functional humor
  21. BlockedIndefinitelyOnMVar
  22. From data type definitions to code
  23. Variant types and GADTs
  24. In-program GC stats for GHC
  25. Synthetic Git merges
  26. Parallelism to plug space leaks
  27. Facebook support for BarnOwl
  28. Grad School, Oh My
  29. Cambridge retrospective: History and Philosophy of Science
  30. A new vision for Mendeley
  31. IVar leaks
  32. Scheduling IVars
  33. The IVar monad
  34. Debugging compilers with optimization fuel
  35. Multi-monitor xmobar placement on Gnome
  36. A Year of Notebooking (Part 2)
  37. A Year of Notebooking (Part 1)
  38. A pattern for increasing sharing
  39. On type synonyms
  40. A Taxonomy of Socialization on the Internet
  41. Pinpointing space leaks in big programs
  42. Measurement, quantification and reduction
  43. What Philosophy of Science Can Say for Software Engineers
  44. The Cryptography of Bitcoin
  45. Bitcoin is not decentralized
  46. Chain Rule + Dynamic Programming = Neural Networks
  47. An insufficiently lazy map
  48. If it has lots of comments, it's probably buggy
  49. Tail recursion makes your loops cleaner
  50. Computing function composition
  51. Anatomy of a thunk leak
  52. Space leak zoo
  53. Reified laziness
  54. Calling all space leaks
  55. Bindings and CAFs on the Haskell Heap
  56. How Aristotle got it Wrong On the importance of conceptual frameworks
  57. Unraveling the mystery of the IO monad
  58. Haskell: Not pure enough?
  59. How the Grinch stole the Haskell Heap
  60. Functions produce the Haskell Heap
  61. Implementing the Haskell Heap in Python, v1
  62. IO evaluates the Haskell Heap
  63. Evaluation on the Haskell Heap
  64. The Haskell Heap
  65. Mailbox: Advice for a sup first-timer
  66. Tracing the compilation of Hello Factorial!
  67. Bugs from using the wrong variable
  68. Hoopl: Dataflow analysis
  69. Type Kata: Local data type
  70. Hoopl: Dataflow lattices
  71. Hoopl guided tour: Base system
  72. GHC migrating to Git
  73. HTML Purifier 4.3.0 released
  74. Ely Cycles
  75. Interleaving stream processors
  76. The return of Hellenistic reasoning
  77. Spring Reading: 2011 edition
  78. On expressivity
  79. Someone is wrong on the Internet
  80. Many-valued logics and bottom
  81. Killer mutants attack (mutation gone wrong)
  82. Type Technology Tree
  83. Petri net concurrency
  84. The creation of a statically-typed functional programmer
  85. On cargo culting and hacking
  86. Multi-day debugging
  87. Ad hoc approximations
  88. Semi-automatic testing
  89. Two short tips for FFI bindings
  90. On checked exceptions and proof obligations
  91. Picturing Hoopl transfer/rewrite functions
  92. Picturing binomial coefficient identities
  93. Android 2.x Sensor Simulator
  94. A suggestion for indent/php.vim
  95. Lav'net is watching you
  96. OCaml gotchas
  97. All about MVars

2010 158

  1. A year of blogging
  2. 2010
  3. Greetings from Switzerland
  4. No one expects the Scott induction!
  5. Thriller: Doing it for the thrills
  6. Travel advisory
  7. Generalization and vagueness in specifications
  8. Tourist by day, Blogger by night
  9. ω: I’m lubbin’ it
  10. Getting a fix on fixpoints
  11. Errata for gin and monotonic
  12. Gin and monotonic
  13. Hussling Haskell types into Hasse diagrams
  14. Talk Friday
  15. How I Learned to Stop Worrying and Love the ⊥
  16. My first proof in Isabelle
  17. Reflexivity. Qed.
  18. Integer sequences every computer scientist should know?
  19. Is multiply-carry strongly universal?
  20. Cambridge potpourri
  21. Haskell.org committee
  22. It's just a longjmp to the left
  23. You are in a maze of twisty little passages, all alike... (a GHC hacking post)
  24. mod_fcgid 2.3 is broken (fixed in 2.3.6)
  25. Medieval medicine and computers
  26. DP Zoo Tour
  27. Dead Edward Day
  28. DP Zoo
  29. Intelligence is the ability to make finer distinctions: Another Haskell Advocacy Post
  30. Blog name changed...
  31. OCaml for Haskellers
  32. Don't Repeat Yourself is context dependent
  33. Purpose of proof: semi-formal methods
  34. Rapidly prototyping scripts in Haskell
  35. Existential type-curry
  36. Quote Day
  37. Why being a sysadmin will help you do Science!
  38. The HTML purification manifesto
  39. Abstraction without a concrete concept
  40. Why I am in Cambridge
  41. Tips for running a hackathon
  42. How to get answers from busy maintainers
  43. So long America!
  44. High performance monads
  45. Data is Code
  46. Tension of patch and tree version control
  47. Session types, subtyping and dependent types
  48. Evolution of a Shared Web Host
  49. Keyword arguments in Haskell
  50. Towards platform-agnostic interruptibility
  51. pthread_cancel on Windows
  52. Embracing Windows
  53. Annotating slides
  54. My type signature overfloweth
  55. Defining “Haskelly”
  56. Interrupting GHC
  57. abcBridge: Functional interfaces for AIGs and SAT solving
  58. Type Kata: Distinguishing different data with the same underlying representation
  59. Day in the life of a Galois intern
  60. Type kata: Controlled sharing of references
  61. What high school Algebra quizzes and NP-complete problems have in common
  62. Generalizing APIs
  63. A radical Hackage social experiment
  64. Paper Monday
  65. The Gateway Drug to Type Programming
  66. Tour of a distributed Erlang application
  67. Buffered streams and iteratees
  68. How to pick your string library in Haskell
  69. Suggestion box
  70. Delaying implicit parameter binding
  71. Reader monad and implicit parameters
  72. Managing foreign pointers effectively
  73. Pipelines and continuations
  74. System.Posix.Redirect
  75. Maximum matching deadlock solution
  76. Flipping arrows in coBurger King
  77. Graphs not grids: How caches are corrupting young algorithms designers and how to fix it
  78. Safety first: FFI and threading
  79. Groom: human readable Show for Haskell
  80. Little’s law
  81. MVC and Purity
  82. Call and fun: marshalling redux
  83. Thinking about talk
  84. Marshalling with get and set
  85. First steps in c2hs
  86. The secret to successful autogenerated docs
  87. Principles of FFI API design
  88. Well-founded recursion in Agda
  89. Setting up Cabal, the FFI and c2hs
  90. The Haskell Preprocessor Hierarchy
  91. Static Analysis for everyday (not-PhD) man
  92. AP Physics: Stuck in the concrete
  93. Databases are categories
  94. Bug boogie: Git and symlinks
  95. Punt the Prelude
  96. Use The Monoid: A worked example
  97. Bananas, Lenses, Envelopes and Barbed Wire A Translation Guide
  98. Lazy exceptions and IO
  99. Upgrading to Ubuntu Lucid
  100. Class Reflections
  101. I Hate Patches: Confessions of an Open-Source Developer
  102. Spring 2010: A Random Walk
  103. Refactoring Haskell code?
  104. Nested Data Parallelism versus Creative Catamorphisms
  105. Omnipresent Cabal
  106. Name conflicts on Hackage
  107. Design Patterns in Haskell
  108. Art. Code. Math. (And mit-scheme)
  109. Inessential guide to fclabels
  110. The Problem with xUnit
  111. Creative catamorphisms
  112. Association maps in mit-scheme
  113. Thoughts on discussion
  114. Dudamel visits MIT
  115. Inessential Guide to data-accessor
  116. Later Impressions of the VX-8R
  117. Diagramming in Xournal and Gimp
  118. You could have invented zippers
  119. Cup of FP with a Java twist
  120. Summer internship at Galois
  121. More fun with Futamura projections
  122. The case of the Hash Array Mapped Trie
  123. Ad hoc wireless
  124. Hunting for abstractions in mathematics
  125. Mutation sleuthing in Python
  126. Haskell, The Hard Sell
  127. Straitjacket programming
  128. Five tips for maintainable shell scripts
  129. Replacing small C programs with Haskell
  130. Being an expert considered harmful
  131. How to use Vim's textwidth like a pro
  132. Third-party unattended upgrades in three steps
  133. Writing generator friendly code
  134. History as documentation
  135. The Art of Posing a Problem
  136. Comonads and Convolutions
  137. Type manipulation: Tricks of the trade
  138. Why we stay up late
  139. Hunting for constraints
  140. Scheming environments
  141. Sources of music
  142. Nested loops and exceptions (Oleg Kiselyov)
  143. Nested loops and continuations
  144. Classes begin
  145. Cute macro tricks in the kernel
  146. Workflows in Git: Single-user style
  147. Arcadia Rising posters
  148. To the right! Autocompletable names
  149. Hacking git-rerere
  150. Too many leftovers!
  151. Five advanced Git merge techniques
  152. Typeclasses matter
  153. Sup: Mail for Nerds
  154. First impressions of the VX-8R
  155. Why Haskell? The big question
  156. rxvt-unicode for gnome-terminal refugees
  157. Audio connectors and radios
  158. Adventures in Three Monads

2009 3

  1. 2010: A Roadmap
  2. 2009
  3. Iron Blogger