Surge AI
Our latest thoughts, notes, and insights from the post-training frontier.
- Posts
- 61 from 8 Mar 2021 to 8 Oct 2026
- Posting
- weekly (28 in the last year)
- Kind
- company blog
2026 24
- sudo L7: Can Coding Agents Act Like Staff Engineers? | Surge AI
- GDP.xlsx: Can Agents Understand the Spreadsheets That Run the World? | Surge AI
- What Does GDP.pdf Teach — Beyond PDFs? | Surge AI
- DAYJOB: Can Agents Survive a 9 to 5? | Surge AI
- Helping Anthropic Build Automated Alignment Researchers
- Hill-Climbing a SWE Agent: What 1,700 Coding Tasks Taught Kimi K2.7
- Fable 5.1, Muse Spark 1.3, and Gemini 3.8 Flash on the Tuesday Work Index
- DeepSeek V4 Pro Scores 59.7 on the Tuesday Work Index
- Qwen 3.8 Max: 58.7 on the Tuesday Work Index
- Tuesday Work Index: Can AI Get Through an Ordinary Day?
- We Trained a Model on Office Work. It Also Got Better at Coding.
- Chartography Benchmark: Can AI Read the Charts Professionals Use?
- OpenAI Cites Surge AI’s GDP.pdf in GPT-5.6 Release
- Training on ComplexConstraints: +10.1 on MultiChallenge, +8.5 on AdvancedIF
- HANDBOOK.md Benchmark: Can AI Agents Follow a 100-Page Company Policy?
- Anthropic cited GDP.pdf and Riemann-bench in their Fable 5 and Mythos 5 system card
- ComplexConstraints: A Benchmark for Entangled Instruction Following
- Microsoft used Surge human evaluations to benchmark MAI-Thinking-1
- Training on Long-Horizon Agent Tasks
- Antidote Leaderboard: Optimizing for You
- GDP.pdf Benchmark: Can Frontier Models Master the Documents that Run the World?
- Riemann-bench: A Benchmark for Moonshot Mathematics
- EnterpriseBench: CoreCraft – Measuring AI Agents in Chaotic, Enterprise RL Environments
- Hemingway-bench: The AI Writing Leaderboard Judged by Expert Writers
2025 11
- Building AdvancedIF: Evolving Instruction Following Beyond IFEval and “Avoid the Letter C”
- LMArena is a cancer on AI
- How do frontier models perform on real-world finance problems?
- RL Environments and the Hierarchy of Agentic Capabilities
- A Product Take on Sonnet 4.5
- Is Sonnet 4.5 the best coding model in the world?
- The Human Frontier: A Conversation with Bogdan Grechuk
- When Coding Agents Spiral Into 693 Lines of Hallucinations
- Benchmarks are broken. Here's why frontier labs treat them as PR.
- Unsexy AI Failures: The PDF That Broke ChatGPT
- Bringing light to the GPT-4o vs. GPT-5 personality controversy
2024 1
2023 1
2022 18
- Evaluating ChatGPT vs. Google on 500 Search Queries
- AI Red Teams for Adversarial Training: How to Make ChatGPT and LLMs Adversarially Robust
- HellaSwag or HellaBad? 36% of this popular LLM benchmark contains errors
- How TikTok is Evolving the Next Generation of Search
- Evaluating Generative AI: Did Astral Codex Ten Win His Bet on AI Progress?
- Why Instagram is Losing Gen Z: We Asked 100 Users to Compare TikTok vs. Reels
- The $250K Inverse Scaling Prize and Human-AI Alignment
- Search Engine Behind-the-Scenes: How Neeva Uses Human Evaluation to Take On Google
- Human Evaluation of Large Language Models: How Good is Hugging Face's BLOOM?
- 30% of Google's Emotions Dataset is Mislabeled
- AI Red Teams and Adversarial Data Labeling, with Redwood Research
- Humans vs. Gary Marcus vs. Slate Star Codex: When is an AI failure actually a failure?
- How Surge AI Built OpenAI's GSM8K Dataset of 8,500 Math Problems
- We asked 100 humans to draw the DALL·E prompts
- Google Search is Falling Behind
- Moving Beyond Engagement: Optimizing Facebook's Algorithms for Human Values
- Holy $#!t: Are popular toxicity models simply profanity detectors?
- Is Google Search Deteriorating? Measuring Google's Search Quality in 2022