The Curious Engineer

All blogs / AI / ML

Transformer Circuits Thread

transformer-circuits.pub

Can we reverse engineer transformer language models into human-understandable computer programs?

AI / MLMath

’20’21’22’23’24’25’26
Posts
57 from 10 Mar 2020 to 29 Oct 2026
Posting
monthly (10 in the last year)
Kind
project news
Feed
https://transformer-circuits.pub/feed.xml

2026 8

  1. Circuits Updates — October 2026
  2. Characterizing interference weights in a tiny language model
  3. Verbalizable Representations Form a Global Workspace in Language Models
  4. Circuits Updates — June 2026
  5. Circuits Updates — May 2026
  6. Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations
  7. HeadVis
  8. Emotion Concepts and their Function in a Large Language Model

2025 19

  1. Circuits Cross-Post — Activation Oracles
  2. Circuits Updates — November 2025
  3. Emergent Introspective Awareness in Large Language Models
  4. Circuits Updates — October 2025
  5. When Models Manipulate Manifolds: The Geometry of a Counting Task
  6. Circuits Updates — September 2025
  7. Circuits Updates — August 2025
  8. A Toy Model of Mechanistic (Un)Faithfulness
  9. Tracing Attention Computation Through Feature Interactions
  10. A Toy Model of Interference Weights
  11. Sparse mixtures of linear transforms
  12. Circuits Updates — July 2025
  13. Automated Auditing
  14. Circuits Updates — April 2025
  15. Progress on Attention
  16. On the Biology of a Large Language Model
  17. Circuit Tracing: Revealing Computational Graphs in Language Models
  18. Insights on Crosscoder Model Diffing
  19. Circuits Updates — January 2025

2024 13

  1. Stage-Wise Model Diffing
  2. Sparse Crosscoders for Cross-Layer Features and Model Diffing
  3. Using Dictionary Learning Features as Classifiers
  4. Circuits Updates — September 2024
  5. Circuits Updates — August 2024
  6. Circuits Updates — July 2024
  7. Circuits Updates — June 2024
  8. Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
  9. Circuits Updates — April 2024
  10. Circuits Updates — March 2024
  11. Reflections on Qualitative Research
  12. Circuits Updates — February 2024
  13. Circuits Updates — January 2024

2023 7

  1. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
  2. Circuits Updates — July 2023
  3. Circuits Updates — May 2023
  4. Interpretability Dreams
  5. Distributed Representations: Composition & Superposition
  6. Privileged Bases in the Transformer Residual Stream
  7. Superposition, Memorization, and Double Descent

2022 4

  1. Toy Models of Superposition
  2. Softmax Linear Units
  3. Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases
  4. In-Context Learning and Induction Heads

2021 5

  1. PySvelte
  2. A Mathematical Framework for Transformer Circuits
  3. Garcon
  4. Exercises
  5. Videos

2020 1

  1. Original Distill Circuits Thread