The Curious Engineer

All blogs / AI / ML

Doubleword

blog.doubleword.ai

Notes on building AI systems

AI / MLPerformance / optimizationHardware

’25’26
Posts
63 from 28 May 2025 to 22 Sep 2026
Posting
weekly (46 in the last year)
Kind
company blog
Feed
https://blog.doubleword.ai/rss.xml

2026 40

  1. Unlocking parallel test-time scaling for long-horizon agents
  2. What is a Megawatt Worth?
  3. What happens when a GPU writes memory
  4. What happens when a GPU reads memory
  5. On-the-fly snapshot compression for elastic inference at scale
  6. The case for disaggregated LLM serving
  7. Gigatoken: making tokenisation care about the hardware
  8. You Could Have Come Up With Kimi Delta Attention
  9. Throughputmaxxing: DeepSeek-V4-Flash on Isambard-AI
  10. NVLink, NVSwitch, and all that
  11. Reverse-engineering NVIDIA's cuda-checkpoint for faster cold starts
  12. Width vs. depth: speculating on the margin
  13. The swarm that designs itself
  14. What happens when you run a CUDA kernel
  15. Prediction: A Frontier Open Source LLM Will Be Released On 3rd December 2026
  16. Anatomy of a Diffusion Language Model
  17. FlashOffload: 7x Cheaper Prefills with Offloading
  18. Adaptive speculative decoding: picking draft lengths at runtime
  19. InfiniBand, RoCE, and all that
  20. UCCL-EP: An expert parallel communications kernel without owning the NIC
  21. Anatomy of a high-performance EP kernel
  22. The economics of speculative decoding
  23. Bringing up DeepSeek-V4-Flash on AMD MI300X
  24. Pushing memory bound kernels beyond the speed of light with lossless decompression
  25. Inference when no one is waiting
  26. MoE expert co-activations: Reordering inputs yields easy throughput gains.
  27. Speculative KV coding: losslessly compressing KV cache by up to ~4× using a predictor model
  28. Tensor Network Attention
  29. In search of wasted bits: how much information do LLM weights carry?
  30. tANS: precomputing rANS
  31. Also-rANS: Asymmetric Numeral Systems for entropy coding
  32. Cloudburst: 70x faster cold(ish) starts for SGLang
  33. OCR and the Bitter Lesson
  34. Weighted random fallback flattens to uniform under high error rates
  35. QueueSpec: Drafting While You Wait
  36. Price Reduction for Qwen3-235B on Doubleword
  37. Scaling Curation with LLM Comparisons
  38. LLM powered data structures: A lock-free binary search tree
  39. ZeroDP: Just-In-Time Weight Offloading over NVLink for Data Parallelism
  40. Large-Scale Semantic Search Without Embeddings

2025 23

  1. Parallel Primitives for Multi-Agent Workflows
  2. $1 for a Year of Research Digests. That's Less Than a Coffee.
  3. Why Batch Inference Matters: Moving from AI Assistants to Autonomous Agents
  4. Behind the Stack, Ep 13: Faster Inference: Speculative Decoding for Batched Workloads
  5. Behind the Stack, Ep 12: Understanding Model Parallelism
  6. Behind the Stack, Ep 11: How Speculative Decoding Speeds Up Language Models
  7. Benchmarking the Doubleword Control Layer
  8. Chasing Cheap Tokens: 2x Cheaper Tokens Than H100s with Consumer Cards
  9. Choosing the Right Model for the Use Case
  10. Understanding Chargeback in the Context of Self-Hosted Systems
  11. LLM guided scheduling | Doubleword
  12. Scheduling in inference engines | Doubleword
  13. Should GPUs make Free Trade Agreements? | Doubleword
  14. Behind the Stack, Ep 10: Batched Endpoints | Doubleword
  15. Behind the Stack, Ep 9: How to Evaluate Open Source LLMs | Doubleword
  16. Behind the Stack, Ep 8: Choosing the Right Inference Engine for Your LLM Deployment | Doubleword
  17. Behind the Stack, Ep 7: Choosing the Right Quantization for Self-Hosted LLMs | Doubleword
  18. Behind the Stack, Ep 6: How to Speed up the Inference of AI Agents | Doubleword
  19. Behind the Stack, Ep 5: Making RAG Work for Multimodal Documents | Doubleword
  20. Behind the Stack, Ep 4: Making Your Load Balancer LLM-Aware | Doubleword
  21. Behind the Stack, Ep 3: How to Serve 100 Models on a Single GPU with No Cold Starts | Doubleword
  22. Behind the Stack, Ep 2: How Many Users Can My GPU Serve? | Doubleword
  23. Behind the Stack, Ep 1: What Should I Be Observing in my LLM Stack? | Doubleword