The Curious Engineer

All blogs / AI / ML

Interconnects AI

interconnects.ai

The cutting edge of AI, from inside the frontier AI labs, minus the hype. The border between high-level and technical thinking. Read by leading engineers, researchers, and investors. Click to read Interconnects AI, a Substack publication with tens of thousands of subscribers.

AI / ML

’20’21’22’23’24’25’26
Posts
384 from 12 Jun 2020 to 30 Oct 2026
Posting
weekly (74 in the last year)
Kind
personal blog
Feed
https://www.interconnects.ai/feed

2026 64

  1. Was I wrong about open models & cyber risks?
  2. Artificial Analysis v5 and the benchmarker's dilemma
  3. Podcast: Kimi K3.1 Pro, the new cyber reality, and Muse musings
  4. Latest open artifacts (#25): DeepSeek V4.1 Flash 1017, American-made models, and new regulatory controls in the US & China
  5. Meta shakes up the frontiers, open and closed
  6. I expect rapid progress but not towards general superintelligence
  7. The Cyber Risk Discourse is Broken
  8. Introducing Trillium Labs
  9. Debating RSI, the US-China Gap, and Jaggedness with JS Denain of Epoch AI
  10. The current balance of power in open models
  11. Why I still haven’t bought into true RSI
  12. Open-Source AI & Open Models Reading List
  13. One resignation turned the embers of AI fear into a wildfire
  14. When will average people feel AI’s impact?
  15. Latest open artifacts (#24): Motif-3, GLM-5.3, Hy4-preview and open model licenses
  16. Teaching Everyone to Fish for Tokens
  17. GLM-5.3: How Chinese labs keep stride with the frontier
  18. I wrote an AI textbook — how long until AI can do it better?
  19. 5 useful things you'll learn in my new post-training textbook (shipping now!)
  20. Lessons from the hacks
  21. Introducing our Artifacts Hub and Adoption Dashboard
  22. Latest open artifacts (#23): Laguna S2.1, Inkling, & Kimi K3 show the utility of open models on the Pareto frontier
  23. Open models recap: more on Kimi K3, Qwen 3.8, Xi's WAIC speech, distillation, the open-closed gap, and what's next
  24. Kimi K3: The open-weights escalation
  25. 6 months to live for open models
  26. Latest open artifacts (#22): Zyphra, Cohere, and Poolside are expanding the breadth of the ecosystem
  27. GLM-5.2 is the step change for open agents
  28. Banning Open Source AI Would Be A Mistake
  29. State of the blog, mid-2026
  30. Frontier post-training recipe review with Finbarr Timbers
  31. Welcome to the AGI era of AI governance
  32. Claude Fable 5 and new AI safety fables
  33. Farewell Ai2
  34. Open and closed models are on different exponentials
  35. Some ideas for what comes next, May 2026
  36. Latest open artifacts (#21): Open model bonanza! Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1 & others. On CAISI's V4 assessment.
  37. How open model ecosystems compound
  38. Notes from inside China's AI labs
  39. The distillation panic
  40. Reading today's open-closed performance gap
  41. My bets on open models, mid-2026
  42. What I’ve been building: ATOM Report, post-training course, finishing my book, and ongoing research
  43. The inevitable need for an open model consortium
  44. Claude Mythos and misguided open-weight fearmongering
  45. Gemma 4 and what makes an open model succeed
  46. Latest open artifacts (#20): New orgs! New types of models! With Nemotron Super, Sarvam, Cohere Transcribe, & others
  47. Lossy self-improvement
  48. GPT 5.4 is a big step for Codex
  49. What comes next with open models
  50. Dean Ball on open models and government control
  51. Olmo Hybrid and future LLM architectures
  52. Latest open artifacts (#19): Qwen 3.5, GLM 5, MiniMax 2.5 — Chinese labs' latest push of the frontier
  53. How much does distillation really matter for Chinese LLMs?
  54. Open models in perpetual catch-up
  55. Opus 4.6, Codex 5.3, and the post-benchmark era
  56. Why Nvidia builds open models with Bryan Catanzaro
  57. Latest open artifacts (#18): Arcee's 400B MoE, LiquidAI's underrated 1B model, new Kimi, and anticipation of a busy month
  58. Thoughts on the job market in the age of LLMs
  59. Arcee AI goes all-in on open models built in the U.S.
  60. Get Good at Agents
  61. Use multiple models
  62. Claude Code Hits Different
  63. 8 plots that explain the state of open models
  64. Latest open artifacts (#17): NVIDIA, Arcee, Minimax, DeepSeek, Z.ai and others close an eventful year on a high note

2025 83

  1. Open models: Hot or Not with Nathan Lambert & Florian Brand
  2. 2025 Interconnects year in review
  3. 2025 Open Models Year in Review
  4. New Talk: Building Olmo 3 Think
  5. Latest open artifacts (#16): Who's building models in the U.S., China's model release playbook, and a resurgence of truly open models
  6. Olmo 3: America’s truly open reasoning models
  7. Why AI writing is mid
  8. Interview: Ant Group's open model ambitions
  9. Opening the black box of character training
  10. 5 Thoughts on Kimi K2 Thinking
  11. Burning out
  12. How to scale RL
  13. Latest open artifacts (#15): It’s Qwen's world and we get to live in it, on CAISI's report, & GPT-OSS update
  14. The State of Open Models
  15. Thoughts on The Curve
  16. ChatGPT: The Agentic App
  17. Thinking, Searching, and Acting
  18. Coding as the epicenter of AI progress and the path to general agents
  19. Latest open artifacts (#14): NVIDIA's rise, "Swiss & UAE DeepSeek," and a resurgence of open data
  20. On China's open source AI trajectory
  21. Ranking the Chinese Open Model Builders
  22. Contra Dwarkesh on Continual Learning
  23. Latest open artifacts (#13): The abundance era of open models
  24. What I've been reading (#2): More on Kimi K2, how to build a bad research center, Pretraining with RL, and sporks of AGI
  25. GPT-5 and the arc of progress
  26. gpt-oss: OpenAI validates the open ecosystem (finally)
  27. Towards American Truly Open Models: The ATOM Project
  28. Interviewing Ross Taylor on the state of AI: Chinese open models, scaling reasoning, useful tools, and what comes next
  29. The White House's plan for open models & AI research in the U.S.
  30. Latest open artifacts (#12): Chinese models continue to dominate throughout the summer 🦦
  31. Kimi K2 and when "DeepSeek Moments" become normal
  32. xAI's Grok 4: The tension of frontier performance with a side of Elon favoritism
  33. The American DeepSeek Project
  34. Ilya on deep learning in 2015
  35. Latest open artifacts (#11): Visualizing China's open models market share, Arcee's models, and VLAs for robotics
  36. Some ideas for what comes next (Jun. 2025)
  37. What I've been reading (#1)
  38. Crafting a good (reasoning) model
  39. The rise of reasoning machines
  40. What comes next with reinforcement learning
  41. How I Write
  42. A taxonomy for next-generation reasoning models
  43. Latest open artifacts (#10): New DeepSeek R1 0528!, more permissive licenses, everything as a reasoner, and from artifacts to agents
  44. Reinforcement learning with random rewards actually works with Qwen 2.5
  45. Claude 4 and Anthropic's bet on code
  46. People use AI more than you think
  47. My path into AI
  48. What people get wrong about the leading Chinese open models: Adoption and censorship
  49. Sycophancy and the art of the model
  50. State of play of AI progress (and related brakes on an intelligence explosion)
  51. Qwen 3: The new open standard
  52. Transparency and (shifting) priority stacks
  53. The latest open artifacts (#9): RLHF book draft, where the open reasoning race is going, and unsung heroes of open LM work
  54. OpenAI's o3: Over-optimization is back and weirder than ever
  55. OpenAI's GPT-4.1 and separating the API from ChatGPT
  56. Llama 4: Did Meta just push the panic button?
  57. RL backlog: OpenAI's many RLs, clarifying distillation, and latent reasoning
  58. Recent reasoning research: GRPO tweaks, base model RL, and data curation
  59. GPT-4o's images and lessons from native input-output multimodality
  60. Gemini 2.5 Pro and Google's second chance with AI
  61. The latest open artifacts (#8): The return of ~30B models, side effects of OpenAI's proposed DeepSeek ban, and yet another reasoning roundup
  62. Managing frontier model training organizations (or teams)
  63. Gemma 3, OLMo 2 32B, and the growing potential of open-source AI
  64. Interviewing Eugene Vinitsky on self-play for self-driving and what else people do with RL
  65. Elicitation, the simplest way to understand post-training
  66. Where inference-time scaling pushes the market for AI companies
  67. GPT-4.5: "Not a frontier model"?
  68. Character training: Understanding and crafting a language model's personality
  69. Claude 3.7 thonks and what's next for inference-time scaling
  70. The latest open artifacts (#7): Alpaca era of reasoning models, China's continued dominance, and tons of multimodal advancements
  71. Grok 3 and an accelerating AI roadmap
  72. An unexpected RL Renaissance
  73. Deep Research, information vs. insight, and the nature of science
  74. Making the U.S. the home for open-source AI
  75. Why reasoning models will generalize
  76. The latest open artifacts (#6): Reasoning models, China's lead in open-source, and a growing multimodal space
  77. Interviewing OLMo 2 leads: Open secrets of training language models
  78. DeepSeek R1's recipe to replicate o1 and the future of reasoning LMs
  79. Let me use my local LMs on Meta Ray-Bans
  80. DeepSeek V3 and the actual cost of training frontier AI models
  81. (Voiceover) DeepSeek V3 and the actual cost of training frontier AI models
  82. The state of post-training in 2025
  83. Quick recap on the state of reasoning

2024 137

  1. 2024 Interconnects year in review
  2. (Voiceover) 2024 Interconnects year in review
  3. OpenAI's o3: The grand finale of AI in 2024
  4. (Voiceover) OpenAI's o3: The grand finale of AI in 2024
  5. The AI agent spectrum
  6. (Voiceover) The AI agent spectrum
  7. OpenAI's Reinforcement Finetuning and RL for the masses
  8. (Voiceover) OpenAI's Reinforcement Finetuning and RL for the masses
  9. Interviewing Finbarr Timbers on the "We are So Back" Era of Reinforcement Learning
  10. OpenAI's o1 using "search" was a PSYOP
  11. (Voiceover) OpenAI's o1 using "search" was a PSYOP
  12. OLMo 2 and building effective teams for training language models
  13. (Voiceover) OLMo 2 and building effective teams for training language models
  14. Tülu 3: The next era in open post-training
  15. (Voiceover) Tülu 3: The next era in open post-training
  16. Scaling realities
  17. (Voiceover) Scaling realities
  18. Saving the National AI Research Resource & my AI policy outlook
  19. (Voiceover) Saving the National AI Research Resource & my AI policy outlook
  20. Interviewing Tim Dettmers on open-source AI: Agents, scaling, quantization and what's next
  21. Interviewing Andrew Carr of Cartwheel on the State of Generative AI
  22. Why I build open language models
  23. (Voiceover) Why I build open language models
  24. Artifacts 5: Deepseek's Janus, I'm writing a Mini RLHF book, Qwen 2.5, video datasets, audio models, and more
  25. Claude's agentic future and the current state of the frontier models
  26. (Voiceover) Claude's agentic future and the current state of the frontier models
  27. Interviewing Arvind Narayanan on making sense of AI hype
  28. Building on evaluation quicksand
  29. (Voiceover) Building on evaluation quicksand
  30. Interviewing Andrew Trask on how language models should store (and access) information
  31. How scaling changes model behavior
  32. AI Safety Culture Confronts Capitalism
  33. [Article Voiceover] AI Safety's Crux: Culture vs. Capitalism
  34. Interviewing Riley Goodside on the science of prompting
  35. [Article Voiceover] Llama 3.2 Vision and Molmo: Foundations for the multimodal open-source ecosystem
  36. Llama 3.2 Vision and Molmo: Foundations for the multimodal open-source ecosystem
  37. Artifacts Log 4: Reflection 70B, o1 on LMSYS, fine-tuning fine-tunes, and speech models
  38. [Article Voiceover] Reverse engineering OpenAI's o1
  39. Reverse engineering OpenAI’s o1
  40. Futures of the data foundry business model
  41. Futures of the data foundry business model
  42. A post-training approach to AI regulation with Model Specs
  43. A post-training approach to AI regulation with Model Specs
  44. OpenAI’s Strawberry, LM self-talk, inference scaling laws, and spending more on inference
  45. OpenAI's Strawberry, LM self-talk, inference scaling laws, and spending more on inference
  46. OLMoE and the hidden simplicity in training better foundation models
  47. OLMoE and the hidden simplicity in training better foundation models
  48. On the current definitions of open-source AI and the state of the data commons
  49. On the current definition of open-source AI and the state of the data commons
  50. On Nous Hermes 3 and classifying a "frontier model"
  51. Nous Hermes 3 and exploiting underspecified evaluations
  52. Artifacts Log 3: Synthetic math and Magpie datasets, another 1T param model, and many Mistral models
  53. Interviewing Ross Taylor on LLM reasoning, Llama fine-tuning, Galactica, agents
  54. A recipe for frontier model post-training
  55. A recipe for frontier model post-training
  56. Interviewing Sebastian Raschka on the state of open LLMs, Llama 3.1, and AI education
  57. GPT-4o-mini changed ChatBotArena
  58. GPT-4o-mini changed ChatBotArena
  59. Llama 3.1 405b, Meta's AI strategy, and the new open frontier model ecosystem
  60. Llama 3.1 405B, Meta’s AI strategy, and the new, open frontier model ecosystem
  61. SB 1047, AI regulation, and unlikely allies for open models
  62. SB 1047, AI regulation, and unlikely allies for open models
  63. Artifacts Log 2: Gemma 2, more Chinese LLMs, high quality datasets, and domain-specific training
  64. Switched to Claude 3.5
  65. Switched to Claude 3.5
  66. Interviewing Dean Ball on AI policy: CA SB 1047, upcoming AI disaster response, Llama 3 405B, Chinese open-source AI, and scaling laws
  67. RLHF roundup: Getting good at PPO, sketching RLHF’s impact, RewardBench retrospective, and a reward model competition
  68. RLHF Roundup: Trying to get good at PPO, charting RLHF's impact, RewardBench retrospective, and a reward model competition
  69. Frontiers in synthetic data
  70. Frontiers in synthetic data
  71. Text-to-video AI models are already abundant, but the products?
  72. Text-to-video AI is already abundant
  73. AI for the rest of us
  74. AI for the rest of us
  75. A case study in reproducibility of evaluation with RewardBench
  76. A realistic path to robotic foundation models
  77. A realistic path to robotic foundation models
  78. We aren't running out of training data, we are running out of open training data
  79. We aren’t running out of training data, we are running out of open training data
  80. Name, image, and AI's likeness
  81. Name, image, and AI’s likeness
  82. Artifacts Log 1: Announcement, Llama 3 fine-tunes, SOTA reward model, human prompt datasets...
  83. OpenAI chases Her
  84. OpenAI chases Her
  85. OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions
  86. OpenAI’s Model (behavior) Spec, RLHF transparency, personalization questions
  87. ChatBotArena: The peoples’ LLM evaluation, the future of evaluation, the incentives of evaluation, and gpt2chatbot
  88. RLHF: A thin line between useful and lobotomized
  89. How RLHF works, part 2: A thin line between useful and lobotomized
  90. Phi 3 and Arctic: Outlier LMs are hints
  91. Phi 3 and Arctic: Outlier LMs are hints
  92. AGI is what you want it to be
  93. AGI is what you want it to be
  94. Llama 3: Scaling open LLMs to AGI
  95. Llama 3: Scaling open LLMs to AGI
  96. Stop "reinventing" everything to "solve" alignment
  97. Stop "reinventing" everything to solve alignment
  98. The end of the "best open LLM"
  99. The end of the “best open LLM”
  100. We disagree on what open-source AI should mean
  101. Why we disagree on what open-source AI should be
  102. DBRX: The new best open LLM and Databricks' ML strategy
  103. DBRX: The new best open model and Databricks’ ML strategy
  104. Evaluations: Trust, performance, and price (bonus, announcing RewardBench)
  105. Evaluations: Trust, performance, and price (bonus, announcing RewardBench)
  106. Model commoditization and product moats
  107. Model commoditization and product moats
  108. The koan of an open-source LLM
  109. The koan of an open-source LLM
  110. Interviewing Louis Castricato of Synth Labs and Eleuther AI on RLHF, Gemini Drama, DPO, founding Carper AI, preference data, reward models, and everything in between
  111. How to cultivate a high-signal AI feed
  112. How to cultivate a high-signal AI feed
  113. Google ships it: Gemma open LLMs and Gemini backlash
  114. Google ships it: Gemma open LLMs and Gemini backlash
  115. 10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and more
  116. 10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and more
  117. Releases! OpenAI’s Sora for video, Gemini 1.5's infinite context, and a secret Mistral model
  118. OpenAI’s Sora for video, Gemini 1.5's infinite context, and a secret Mistral model
  119. ($ Paid Perks) Discord Access & Book Discount
  120. Why reward models are key for alignment
  121. Why reward models are still key to understanding alignment
  122. Alignment-as-a-service: Scale AI vs. the new guys
  123. Alignment-as-a-Service: Scale AI vs. the new guys
  124. Open Language Models (OLMos) and the LLM landscape
  125. Open Language Models (OLMos) and the LLM landscape
  126. Model merging lessons in The Waifu Research Department
  127. Model merging lessons in The Waifu Research Department
  128. Local LLMs, some facts some fiction
  129. Local LLMs, some facts some fiction
  130. Multimodal blogging: My AI tools to expand your audience
  131. Multimodal blogging: My AI tools to expand your audience
  132. RLHF learning resources in 2024
  133. Multimodal LM roundup: Unified IO 2, inputs and outputs, Gemini, LLaVA-RLHF, and RLHF questions
  134. Multimodal LM roundup: Unified IO 2, inputs and outputs, Gemini, LLaVA-RLHF, and RLHF questions
  135. Where 2024’s “open GPT4” can’t match OpenAI’s
  136. Where 2024’s “open GPT4” can’t match OpenAI’s
  137. It's 2024 and they just want to learn

2023 56

  1. Interconnects year in review: 2023
  2. Interviewing Tri Dao and Michael Poli of Together AI on the future of LLM architectures
  3. State-space LLMs: Do we need Attention?
  4. Big Tech's LLM evals are just marketing
  5. Mixtral: The best open model, MoE trade-offs, release lessons, Mistral raises $400mil, Google's loss, vibes vs marketing
  6. Do we need RL for RLHF?
  7. Synthetic data: Anthropic’s CAI, from fine-tuning to pretraining, OpenAI’s Superalignment, tips, types, and open examples
  8. The Q* hypothesis: Tree-of-thoughts reasoning, process reward models, and supercharging synthetic data
  9. RLHF progress: Scaling DPO to 70B, DPO vs PPO update, Tülu 2, Zephyr-β, meaningful evaluation, data contamination
  10. OpenAI’s shakeup and opportunity for the rest of us: openness, brain drain, and new realities
  11. The interface era of AI
  12. Reckoning with the Shoggoth of AI
  13. Open LLM company playbook
  14. How the Foundation Model Transparency Index Distorts Transparency
  15. RLHF lit. review #1 and missing pieces in RLHF
  16. Undoing RLHF and the brittleness of safe LLMs
  17. The AI research job market shit show (and my experience)
  18. LLMs are computing platforms
  19. Open, general-purpose LLM companies might not be viable
  20. Can robotics take off like GenAI? Moravec's paradox vs. scaling laws
  21. DALL·E 3 and multimodality as moats, correcting bad moat takes
  22. Challenges operationalizing responsible AI in open RLHF research
  23. Midjourney vs. Ideogram, ML product companies, preventing AI winter, DALL·E 3 tease
  24. In defense of the open LLM leaderboard
  25. Announcing The Retort Podcast
  26. AI researchers' challenges: atomic analogies and strained institutions
  27. Cruise's collisions and adapting to AI
  28. LLM products: measurement and manipulation
  29. Specifying objectives in RLHF
  30. "If it's not fully closed ML, it's open" - is it?
  31. Llama 2 follow-up: too much RLHF, GPU sizing, technical details
  32. Llama 2: an incredible open LLM
  33. LLM agents follow-up: exploration, RLHF, and more
  34. LLM agents and integration dead-ends
  35. Disinformation with LLMs: the distribution risk
  36. Tesla Autopilot's negligence and regulation
  37. How RLHF actually works
  38. Different development paths of LLMs
  39. Open-source LLMs' harmlessness gap
  40. Evaluating and uncovering open LLMs
  41. Code: green pastures for LLMs
  42. Unfortunately, OpenAI and Google have moats
  43. Specifying hallucinations
  44. Beyond human data: RLAIF needs a rebrand
  45. The next chapter: Interconnects
  46. Growing needs for accessing state-of-the-art reward models
  47. Behind the curtain: what it feels like to work in AI right now (April 2023)
  48. The implicit dynamics of optimizing costs vs. rewards vs. preferences
  49. GPT4: The quiet parts and the state of ML
  50. AGI Roundup: Re-visiting Go; transitioning from narrow to general; multimodality & GPT4
  51. The RLHF battle lines are drawn
  52. "AI alignment" and uncalibrated discourse on AI
  53. Three seasons of RL: Metaphor, tool, and framework
  54. Scaling laws for robotics & RL: Not quite yet
  55. Pretraining quadrupeds: a case study in RL as an engineering tool
  56. Looking into 2023

2022 8

  1. Predicting machine learning moats
  2. Closed-API vs Open-source continues: RLHF, ChatGPT, data moats
  3. RLHF, 'online' ML systems, and RL going mainstream
  4. Join my new subscriber chat
  5. Using RL's exploitation to debug
  6. Back in the game
  7. Designing Societally Beneficial Reinforcement Learning Systems
  8. Flexible Centralization in Multi-agent Learning & Control

2021 14

  1. Remote robotic-data farms
  2. on the Horizon of applied RL
  3. Reward is not enough
  4. How all machine learning becomes reinforcement learning
  5. Setting ourselves up for exploitation: RL in the wild
  6. Counting down until consumer drones are banned in cities
  7. Clarifying RL: Obscure problem formulations and structure tradeoffs
  8. Decoupling AI from the latent variable of spoken languages
  9. 100th anniversary of the word robot: COVID didn’t give us personal robots, it gave us Woebot
  10. Boston Dynamics 🤖🐈: Studying Athletic Intelligence
  11. Robotic Companies 2.0: Horizontal Modularity
  12. Reflections on digital technology from a capitol siege
  13. Considering AIs through our own mind’s reflection
  14. The stakeholders start to learn the stakes: predictions for machine learning and automation in 2021

2020 22

  1. 2020: (half) year in review - Democratizing Automation
  2. The Ubiquity and Future of Model-based Reinforcement Learning
  3. Facebook Case Study 🌏🔬: Using AI to Regulate the Digital Ecosystem
  4. The Collingridge Dilemma and Current Policy on Robots
  5. Constructing Axes for Reinforcement Learning Policy
  6. Towards an Ethics for Roboticists
  7. What is ML for Climate Change?
  8. Why PhD Students are Lonelier than Retirees
  9. Models, Systems, Code; and Robots
  10. Some thoughts on teaching & learning
  11. Autonomy startups are such a mess
  12. AI & Arbitration of Truth
  13. Can robots be autonomous and unintelligent?
  14. The uncanny world of robots at home
  15. Digital companies and the goal of at-home embodied AI
  16. Automated: how algorithms shape a day in the life and our future
  17. Recommendations are a game - a dangerous game (for us).
  18. Automating code, Twitter's hack(s), a robot named Stretch
  19. Online courses, automating education, and digitalizing degrees
  20. Democratizing Automation
  21. Drones, Swarms, and Storms of Drones
  22. "10 years of automation in 1 year"