The Curious Engineer

All blogs / AI / ML

Beren's Blog

beren.io

Scattered Thoughts

AI / MLEssays / personalScience

’17’18’19’20’21’22’23’24’25’26
Posts
185 from 9 Jan 2017 to 22 Oct 2026
Posting
weekly (39 in the last year)
Kind
personal blog
Feed
https://www.beren.io/feed.xml

2026 31

  1. Three Phases of Capabilities and Alignment
  2. History as Information Dynamics
  3. Algorithmic Progress is not Exogenous
  4. Can Individuality Survive in the Limit?
  5. Many Benefits Of AGI Could Still Be Realized In A Pause
  6. Math Capabilities could be Differentially Good for Alignment
  7. Page2
  8. Whence the Fractals in our Stars?
  9. Architecture Research as Addressing Constraints to Scaling
  10. Mitigating Reward Hacking as Institutional Design
  11. Scaling is subtler than it seems
  12. Maximum Entropy Morality, Metaplastic Constitutionalism, and the Dynamic Virtues
  13. Page3
  14. The Probe
  15. How can LLM RL Work Despite Information-Theoretic Inefficiency
  16. Is Endogenous Exploration Sufficient?
  17. Autarkic Agency Will Likely Migrate Upwards
  18. Baudrillard and Interiority
  19. Page4
  20. Taking the Singularity Seriously
  21. Understanding Functional Decision Theory as Program Search
  22. Preliminary Notes on Colonizing the Universe
  23. Distillation Ain't What It Used To Be
  24. Thermopylae
  25. Page5
  26. Thoughts on Claude Mythos
  27. When Does Competition Lead To Recognizable Values
  28. AI Monotheism vs AI Polytheism
  29. Two Mechanisms of Decadence
  30. Intellectual Progress in 2025
  31. Page6

2025 24

  1. Initial Quick Thoughts on Singular Learning Theory
  2. Addendum to Fertility, Inheritance, and the Concentration of Wealth
  3. The Biosingularity Alignment Problem Seems Harder than AI Alignment
  4. Gradual Disempowerment Might Not Be So Bad
  5. Space Warfare Seems Mostly Defense Dominant
  6. Whence Human Talents Neurobiologically?
  7. Page7
  8. Continual learning explains some interesting phenomena in human memory
  9. Thoughts on (AI) consciousness
  10. Millennials as the Forever Generation
  11. The Limit of Prediction is not Omniscience
  12. Page8
  13. Do We Want Obedience or Alignment?
  14. Most Algorithmic Progress is Data Progress
  15. Should we be behaviorist about an AI's values?
  16. Preliminary Thoughts on Reward Hacking
  17. Current neural networks are not overparametrized
  18. The Scaling Laws Are In Our Stars, Not Ourselves
  19. Why Not Sparse Hierarchical Graph Learning
  20. Page9
  21. Maintaining Alignment during RSI as a Feedback Control Problem
  22. Review of Mind Children By Hans Moravec (1988)
  23. Intellectual Progress in 2024
  24. Page10

2024 14

  1. A Retrospective on Active Inference
  2. Right to Left (R2L) Integer Tokenization
  3. Alignment likely generalizes further than capabilities.
  4. Capital Ownership Will Not Prevent Human Disempowerment
  5. The Unconditioned Distribution of Current Open LLMs
  6. Page11
  7. Alignment In The Age Of Synthetic Data
  8. Integer tokenization is now much less insane
  9. Does scaffolding help humans?
  10. Addendum to the Surprising Parameter Efficiency of Vision Models
  11. Fertility, Inheritance, and the Concentration of Wealth
  12. Page12
  13. My Preliminary Thoughts on AI Safety Regulation
  14. Linear Attention as Iterated Hopfield Networks

2023 39

  1. Learning Linear Representations through Implicit Subspace Selection
  2. Intellectual Progress in 2023
  3. Open source AI has been vital for alignment
  4. Page13
  5. Addendum to Grokking Grokking
  6. Strong infohazard norms lead to predictable failure modes
  7. Preference Aggregation as Bayesian Inference
  8. Thoughts on loss landscapes and why deep learning works
  9. Entrepreneurs as Catalysts
  10. My path to prosaic alignment and open questions
  11. Page14
  12. Private Equity as Legibility Arbitrage
  13. My rough categorization of AI risk types
  14. The case for removing alignment and ML research from the training data
  15. BCIs and the ecosystem of modular minds
  16. Eigenvector successor representations
  17. Hedonic loops and taming RL
  18. Predictive coding networks can perform causal and counterfactual inference
  19. Scaffolded LLMs as natural language computers
  20. Why GOFAI failed
  21. The singularity as cognitive decoupling
  22. GPUs vs Brains. Hardware and Architecture
  23. My PhD experience
  24. Deep learning models are secretly (almost) linear
  25. The surprising parameter-efficiency of vision models
  26. The computational anatomy of human values
  27. CEM as inference
  28. Thoughts on the future of Predictive Coding
  29. The Paperclip King, by GPT4
  30. LLMs confabulate not hallucinate
  31. Orthogonality is expensive
  32. Against ubiquitous alignment taxes
  33. Fingerprinting LLMs with their unconditioned distribution
  34. The solution to alignment is many not one
  35. Validator models. A simple approach to detecting and counteracting goodhearting
  36. Boxing might work but we won't use it
  37. Intellectual progress in 2022
  38. Integer tokenization is insane
  39. Gradient Hacking is extremely difficult.

2022 20

  1. Creating worlds where iterative alignment succeeds
  2. An ML interpretation of Shard Theory
  3. Preventing Goodheart with homeostatic reward functions.
  4. Don't argmax; Distribution match
  5. The development of human sexuality as example of alignment
  6. AGI will have learnt reward models.
  7. Why not just stop FOOM?
  8. Deconfusing direct vs amortized optimization
  9. Intellectual progress in 2021
  10. Scaling laws vs individual differences
  11. Alignment needs empirical evidence
  12. Probabilities multiply in our favour for AGI containment
  13. Towards concrete threat models for AGI
  14. Empathy as a natural consequence of learnt reward models
  15. The ultimate limits to alignment determine the shape of the long term future
  16. How to evolve a brain
  17. The Scale of the Brain vs Machine Learning
  18. Understanding Overparametrized Generalization
  19. Grokking 'grokking'
  20. Clarifying Value Alignment in Predictive Processing

2021 4

  1. Self-Hosting beren.io
  2. Selection and the Just World Fallacy
  3. Intellectual Progress in 2020
  4. How the Tails Come Apart

2020 11

  1. Walkthrough of
  2. Thoughts on the Falsifiability of the Free Energy Principle
  3. Predictive Coding as Backprop and Natural Gradients
  4. The Schelling Point Theory of Money
  5. The AI AI safety problem
  6. Safer Value Learning through uncertainty
  7. Reflections on the Bitter Lesson
  8. Priors on the 10x Programmer
  9. Review of Certain to Win by Chet Richards
  10. Intellectual Progress in 2019
  11. Happy New Year

2019 6

  1. Visualizing The Beta Distribution
  2. Initial Thoughts on Boyd and OODA Loops
  3. Thoughts on the Election
  4. Long time no see
  5. Regularising Utility Functions
  6. Test first post

2017 16

  1. What if the UFOs were Aliens
  2. A Predictive Processing Model of Autism
  3. Friedrich List: Addendums
  4. Thoughts on Recursive Self Improvement
  5. On Mathematics and Morality
  6. Thoughts on Metaethics
  7. Book Review: Cardinal Richeliu and the Making of Modern France
  8. Some Thoughts on Space War and Space Strategy
  9. Bayesian Optimization and Importance Sampling
  10. Book Review: Russian Embassies to the Georgian Kings 1589-1605
  11. Book Review: Catherine the Great by Robert K Massie
  12. Book Review: The High Frontier
  13. Reflections on Friedrich List's National System of Political Economy
  14. The High Frontier: Addendums
  15. Book Review: Who Paid the Pipers by Frances Stonor Sanders
  16. Contra David Lewis on the Superfluity of Worlds

Undated 20

  1. Sequences
  2. Misc
  3. Page15
  4. Page16
  5. Page17
  6. Page18
  7. Page19
  8. Page20
  9. Page21
  10. Page22
  11. Page23
  12. Page24
  13. Page25
  14. Page26
  15. Page27
  16. Page28
  17. Page29
  18. Page30
  19. Page31
  20. CV.pdf