The Curious Engineer

All blogs / AI / ML

EleutherAI

blog.eleuther.ai

Research notes, announcements, and technical essays from EleutherAI.

AI / ML

’21’22’23’24’25’26
Posts
54 from 20 Apr 2021 to 25 Oct 2026
Posting
now and then (5 in the last year)
Kind
project news

2026 5

  1. Read post →
  2. Read post →
  3. Read post →
  4. A Dynamical Model of AI Governability
  5. Early Indicators of Reward Hacking via Reasoning Interpolation

2025 7

  1. Reward Hacking Research Update
  2. Pretraining Data Filtering for Open-Weight AI Safety
  3. Attention Probes
  4. Research Update: Applications of Local Volume Measurement
  5. Studying inductive biases of random networks via local volumes
  6. The Common Pile v0.1
  7. Product Key Memory Sparse Coders

2024 14

  1. SAEs trained on the same data don’t learn the same features
  2. Partially rewriting an LLM in natural language
  3. Third-party evaluation to identify risks in LLMs’ training data
  4. Mechanistic Anomaly Detection Research Update 2
  5. RLHF and RLAIF in GPT-NeoX
  6. The Practitioner's Guide to the Maximal Update Parameterization
  7. Mechanistic Anomaly Detection Research Update | EleutherAI Blog
  8. Open Source Automated Interpretability for Sparse Autoencoder Features | EleutherAI Blog
  9. Experiments in Weak-to-Strong Generalization | EleutherAI Blog
  10. Free Form Least-Squares Concept Erasure Without Oracle Concept Labels | EleutherAI Blog
  11. VINC-S: Closed-form Optionally-supervised Knowledge Elicitation with Paraphrase Invariance | EleutherAI Blog
  12. Pile-T5 | EleutherAI Blog
  13. Yi-34B, Llama 2, and common practices in LLM training: a fact check of the New York Times | EleutherAI Blog
  14. The Foundation Model Development Cheatsheet | EleutherAI Blog

2023 17

  1. Least-Squares Concept Erasure with Oracle Concept Labels | EleutherAI Blog
  2. Diff-in-Means Concept Editing is Worst-Case Optimal | EleutherAI Blog
  3. The third New England RLHF Hackers Hackathon | EleutherAI Blog
  4. Extending the RoPE | EleutherAI Blog
  5. How the Foundation Model Transparency Index Distorts Transparency | EleutherAI Blog
  6. Llemma: An Open Language Model For Mathematics | EleutherAI Blog
  7. The second New England RLHF Hackers Hackathon | EleutherAI Blog
  8. Contributor Spotlight: Mohammad Aflah Khan | EleutherAI Blog
  9. The first New England RLHF Hackers Hackathon | EleutherAI Blog
  10. EleutherAI's Thoughts on the EU AI Act | EleutherAI Blog
  11. Minetester: A fully open RL environment built on Minetest | EleutherAI Blog
  12. 🐶Safetensors audited as really safe and becoming the default | EleutherAI Blog
  13. Alignment Research @ EleutherAI | EleutherAI Blog
  14. Transformer Math 101 | EleutherAI Blog
  15. Exploratory Analysis of TRLX RLHF Transformers with TransformerLens | EleutherAI Blog
  16. EleutherAI Second Retrospective: The long version | EleutherAI Blog
  17. The View from 30,000 Feet: Preface to the Second EleutherAI Retrospective | EleutherAI Blog

2022 1

  1. Announcing GPT-NeoX-20B | EleutherAI Blog

2021 10

  1. A Preliminary Exploration into Factored Cognition with Language Models | EleutherAI Blog
  2. Multiple Choice Normalization in LM Evaluation | EleutherAI Blog
  3. Downstream Evaluations of Rotary Position Embeddings | EleutherAI Blog
  4. What A Long, Strange Trip It's Been: EleutherAI One Year Retrospective | EleutherAI Blog
  5. Why Release a Large Language Model? | EleutherAI Blog
  6. On the Sizes of OpenAI API Models | EleutherAI Blog
  7. Evaluating Different Fewshot Description Prompts on GPT-3 | EleutherAI Blog
  8. Finetuning Models on Downstream Tasks | EleutherAI Blog
  9. Activation Function Ablation | EleutherAI Blog
  10. Rotary Embeddings: A Relative Revolution | EleutherAI Blog