The Curious Engineer

All blogs / AI / ML

Baseten

baseten.co/blog

Stories, updates, and other resources from Baseten.

AI / MLPerformance / optimizationDevOps / SRE / infrastructure

’25’26
Posts
252 from 4 Apr 2025 to 30 Oct 2026
Posting
weekly (119 in the last year)
Kind
company blog

2026 111

  1. The fastest delta weight syncs in the world
  2. Autoscaling schedules: reduce cold-start delays for predictable traffic
  3. Baseten and Postman: Building reliable, cost-efficient agents for the enterprise
  4. Decisions for a penny: introducing joey-1, our fastest decision model
  5. The fastest, lowest-cost batch transcription with Qwen3-ASR
  6. Introducing Baseten’s newest engineering leaders
  7. BDN Sealed Volumes: publish once, mount anywhere
  8. Introducing the Baseten Environment Network
  9. The top ranked decision models in the world: mammoth-1 and elephant-1
  10. Beam: Efficient reasoning for coding and agentic workloads
  11. Muse Spark Open, an open-weight model for coding agents
  12. Announcing NVIDIA Nemotron 3.5 Super VL on Baseten
  13. Deploy Runway’s video models on Baseten
  14. Delivering real-time text-to-speech with Gradium
  15. Project Beacon: Bringing safety signals to open models at scale
  16. What makes a high-quality NVFP4 quantization?
  17. Agentic inference optimization: 50-90% faster engines
  18. Announcing our partnership with OpenAI
  19. Securing the open frontier with NVIDIA OpenShell and Blaxel sandboxes
  20. Sheila Vashee joins Baseten as CMO
  21. Fine-tune on your LangSmith traces with Baseten Loops
  22. NVIDIA Nemotron 3 Diarization: real-time speaker labels at a cent per audio hour
  23. Introducing Baseten Hosted Tools
  24. LangChain trains custom models for LangSmith Engine with Baseten Loops
  25. DeepSeek-V4.1-Flash: more efficient prefill for coding agents
  26. Blaxel is joining Baseten to build the future of agentic infrastructure
  27. How Baseten makes pyannote’s diarization models 9.6x faster
  28. Optimizing delta weight syncs for managed RL rollouts
  29. Baseten leads Coval’s voice AI benchmark
  30. New MCP and skill for coding agents to use Baseten
  31. Best open-source models for post-training
  32. The efficient frontier of LLM inference
  33. Agentic kernels in production
  34. GLM-5.3: Scaling with post-training, intuitively explained
  35. How to run any open model inside DeepSeek Harness
  36. How leading platforms ensure observability for LLM inference
  37. Inference engineering for DeepSeek V4 Pro 0813
  38. Baseten delivers open-source inference for You.com
  39. Introducing NVIDIA Nemotron 3.5 Lightning
  40. Introducing NVIDIA Nemotron 3.5 ASR Streaming
  41. Laguna S 2.1 goes Greek: a repository-scale game transformation
  42. Fine-tuning Qwen3-TTS for high-quality voice cloning
  43. 22,580: GPT-2 to Kimi K3, explained
  44. How to run Kimi K3 in any harness: routing with Baseten Switch
  45. Announcing Baseten for Model Labs
  46. Welcome, Mani Parkhe!
  47. Making Kimi K3 tokenization 18x faster for million-token agentic workloads
  48. How to build a day-0 API for Kimi K3
  49. How we built the new fastest API for GLM-5.2
  50. Introducing GLM-5.2 Fast
  51. How to choose an AI model: lessons from Notion and Gamma
  52. How to optimize LLM inference speed and reduce costs in production
  53. H100 vs. H200 GPUs
  54. GLM 5.2 with vision
  55. Real-time video generation inference on Baseten
  56. Fast, accurate retrieval with NVIDIA Nemotron 3 Embed
  57. Meet Inkling: Thinking Machines Lab's new customizable model
  58. Introducing Step 3.7 Flash: multimodal reasoning at scale
  59. Building with NVIDIA Nemotron 3 Ultra and LangChain Deep Agents Code on Baseten
  60. H100 vs. H200 vs. B200: which GPU should you use?
  61. How to run GLM-5.2 in any harness
  62. AI training vs. inference: what's the difference?
  63. Live draft model training for speculative decoding
  64. NVIDIA BioNeMo Agent Toolkit on Baseten
  65. How we built the world’s fastest API for GLM-5.2
  66. Announcing our Series F
  67. The best open-source large language models (LLMs)
  68. Rolling deployments for zero-downtime model updates
  69. Mercury 2, the first reasoning diffusion LLM, is now on Baseten
  70. Welcome, Gabe Stern!
  71. Introducing NVIDIA Nemotron 3 Ultra: The Nemotron 3.x family is here!
  72. MAI-Thinking-1 is coming to Baseten
  73. NVIDIA Cosmos 3: Robots finally take over
  74. Timestep distillation: 2.5x faster FLUX.2 image generation
  75. Powering Inference for the Continual Learning Era
  76. Sub-second image generation with Flux.2 and Qwen-Image
  77. Cost-efficient, high-performance TTS with Qwen3-TTS
  78. Baseten Loops SDK
  79. DFlash: 3x faster LLM inference
  80. Inference Infrastructure for AI Labs | Baseten
  81. NVIDIA Nemotron 3 Nano Omni: Build multimodal agents on Baseten
  82. How we built RBAC that scales for the enterprise
  83. Harnesses are everything. Here's how to optimize yours.
  84. How to train custom EAGLE-3 heads for speculative decoding
  85. How the Baseten Delivery Network (BDN) makes cold starts fast
  86. Sub-3 millisecond named entity recognition (NER) inference
  87. Open-source LLM training is a mess. Here is how it all works.
  88. Baseten Training: an autoresearch substrate
  89. I spent 31 hours on the math behind TurboQuant so you don't have to
  90. Welcome, Sameer Paranjpye!
  91. Introducing the Baseten Delivery Network: Fast cold starts for big models
  92. Secure your harness: how to run NVIDIA's NemoClaw with frontier open source models
  93. Welcome, Matt Slagle!
  94. NVIDIA Nemotron 3 Super for agentic AI in financial services
  95. How we built the fastest GLM 5 API
  96. Outperforming frontier models on emergency medicine documentation
  97. 4-Bit Quantization for Inference Optimization
  98. Announcing the acquihire of Inferless by Baseten
  99. MiniMax M2.5: Intelligence too cheap to meter, RL process rewards, real-world productivity
  100. How we built the fastest Kimi K2.5 on Artificial Analysis
  101. AI model performance metrics explained
  102. Fine-tuning models, AI and Hollywood: A conversation with Oxen’s founder Greg
  103. How to run LLM performance benchmarks (and why you should)
  104. OpenClaw + Kimi K2.5 on Baseten = frontier agent performance with OSS
  105. The Baseten Inference Stack at NVIDIA Dynamo Day
  106. Announcing Baseten's $300M Series E
  107. Open-sourcing Baseten’s suffix automaton MTP accelerator
  108. Wan 2.2 video generation in less than 60 seconds
  109. The fastest Whisper — with streaming and diarization
  110. Build AI agents in production (without writing code)
  111. A Q&A From Inference To Training: The Inside Story Of Baseten’s Newest Product

2025 46

  1. Baseten AI Wrapped: 3 trends to help you build better in 2026
  2. NVIDIA Nemotron 3 Nano: Build Agentic AI Applications on Baseten
  3. Parsed + Baseten: Building Models That Touch Grass
  4. DeepSeek V3.2's path to GPT-5-level performance: sparse attention, RL at scale, and context reuse
  5. Enterprise vision intelligence with Mistral AI and Baseten
  6. Kimi K2 Thinking at 140+ TPS on NVIDIA Blackwell
  7. Purpose-built LLMs for dental note-taking
  8. Tool Calling in Inference
  9. Train AI Models When You Want. Deploy on Ultra Performant Infrastructure. Baseten Training Is GA.
  10. High-performance agents for financial services with NVIDIA Nemotron on Baseten
  11. How we made the fastest GPT-OSS on NVIDIA GPUs 60% faster
  12. DeepSeek-OCR and the Unreasonable Usefulness of Compression
  13. Building production AI for regulated industries with a leading digital insurer
  14. 2x faster inference with KV cache-aware routing
  15. From Sketch to 3D Model: Building a flower card generator with open source AI
  16. Building AI agents, open code, and open source: A conversation with Dax
  17. Baseten brings AI video to life on Nebius
  18. Delivering GenAI solutions for healthcare with Baseten and Vultr
  19. Making Zed fast: A conversation with Richard Feldman
  20. Building the future of AI infrastructure: Q&A with Baseten Co-founder Amir Haghighat
  21. Announcing Baseten’s $150M Series D
  22. How Baseten MCM, our cloud ecosystem partners, and NVIDIA drive fast, reliable inference at scale
  23. HTTP vs. WebSockets vs. gRPC for AI model inference
  24. Welcoming Dannie Herzberg to Baseten
  25. How to fine-tune gpt-oss-120b with Baseten and Axolotl
  26. Fine-tuning small open-source LLMs to outperform large closed-source models by 60% on specialized tasks
  27. From prompt to production: Baseten inference in your IDE with Cline
  28. Joey Zwicker joins Baseten as Head of FDE
  29. Zero to real-time text-to-speech: The complete Orpheus + WebSockets tutorial
  30. How we run GPT OSS 120B at 500+ tokens per second on NVIDIA GPUs
  31. Run Qwen3 Embedding on NVIDIA Blackwell GPUs
  32. Zero to real-time transcription: The complete Whisper V3 streaming tutorial
  33. Voxtral vs. Whisper: Build a Voice-Controlled Smart Home App
  34. Kimi K2 Explained: The 1 Trillion Parameter Model
  35. Building reliable AI agents
  36. How we built Multi-cloud Capacity Management (MCM)
  37. Your client code matters: 12x higher embedding throughput with Python and Rust
  38. Forward deployed engineering on the frontier of AI
  39. How Baseten multi-cloud capacity management (MCM) unifies deployments
  40. Introducing Model APIs and Training
  41. Introducing our new brand
  42. Canopy Labs selects Baseten as preferred inference provider for Orpheus TTS models
  43. Day zero benchmarks for Qwen 3 with SGLang on Baseten
  44. Accelerating inference with NVIDIA B200 GPUs
  45. Building performant embedding workflows with Chroma and Baseten
  46. The best open-source embedding models

Undated 95

  1. How we built bei high throughput embedding inference
  2. Introducing baseten embeddings inference bei
  3. Announcing baseten 75m series c
  4. How multi node inference works llms deepseek r1
  5. Testing llama inference performance nvidia gh200 lambda cloud
  6. Baseten chains for production compound ai systems
  7. Private secure deepseek r1 in production in us eu data centers
  8. Driving model performance optimization 2024 highlights
  9. Activity logging llm metrics dashboard customization
  10. How we built production ready speculative decoding with tensorrt llm
  11. A quick introduction to speculative decoding
  12. Speculative decoding engine builder integration
  13. The fastest most accurate and cost efficient whisper transcription
  14. Deploy production model servers from docker images
  15. Custom environments for deployments on baseten
  16. Canary deployments on baseten
  17. Evaluating nvidia h200 gpus for llm inference
  18. Export model inference metrics to observability tools
  19. Baseten partners with google cloud to deliver high performance ai infra
  20. Baseten hybrid control and flexibility in your cloud and ours
  21. Building high performance compound ai applications with mongodb atlas and baseten
  22. How to build function calling and json mode for open source and fine tuned llms
  23. Function calling and structured output for llms
  24. The best open source image generation model
  25. How to double tokens per second for llama 3 with medusa
  26. Spc hackathon winners build with llama 3 1 on baseten
  27. Baseten self hosted
  28. Compound ai systems explained
  29. Automatic llm optimization with tensorrt llm engine builder
  30. Deploying custom comfyui workflows as apis
  31. Ten reasons to join baseten
  32. How to serve 10 000 fine tuned llms from a single gpu
  33. Using asynchronous inference in production
  34. Baseten chains explained
  35. Introducing baseten chains
  36. Comparing few step image generation models
  37. How latent consistency models work
  38. What i learned as a forward deployed engineer working at an ai startup
  39. Control plane vs workload plane in model serving infrastructure
  40. Comparing tokens per second across llms
  41. Ci cd for ai model deployments
  42. Streaming real time text to speech with xtts v2
  43. Continuous vs dynamic batching for ai inference
  44. Using fractional h100 gpus for efficient model serving
  45. Benchmarking fast mistral 7b inference
  46. 33 faster llm inference with fp8 quantization
  47. High performance ml inference with nvidia tensorrt
  48. Fp8 efficient model inference with 8 bit floating point numbers
  49. Announcing our series b
  50. The benefits of globally distributed infrastructure for ml model serving
  51. 40 faster stable diffusion xl inference with nvidia tensorrt
  52. Why gpu utilization matters for model inference
  53. Unlocking the full power of nvidia h100 gpus for ml inference with tensorrt
  54. Introduction to quantizing ml models
  55. How to benchmark image generation models like stable diffusion xl
  56. Understanding performance benchmarks for llm inference
  57. Faster mixtral inference with tensorrt llm and quantization
  58. Playground v2 vs stable diffusion xl 1 0 for text to image generation
  59. How to serve your comfyui model behind an api endpoint
  60. Nvidia a10 vs a10g for ml model inference
  61. Stable video diffusion
  62. Gpt vs mistral migrate to open source llms with minor code changes
  63. Open source alternatives for machine learning models
  64. A checklist for switching to open source ml models
  65. Llm transformer inference guide
  66. Pinning ml model revisions for compatibility and security
  67. Deployment and inference for open source text embedding models
  68. Nvidia a10 vs a100 gpus for llm and stable diffusion inference
  69. Sdxl inference in under 2 seconds the ultimate guide to stable diffusion optimiza
  70. Build your own open source chatgpt with llama 2 and chainlit
  71. Ai infrastructure build vs buy
  72. Build a chatbot with llama 2 and langchain
  73. Deploying and using stable diffusion xl 1 0
  74. Model autoscaling features on baseten
  75. Three techniques to adapt llms for any use case
  76. What i learned from my ai startups internal hackathon
  77. Understanding nvidias datacenter gpu line
  78. Comparing gpus across architectures and tiers
  79. Comparing nvidia gpus for ai t4 vs a10
  80. How to host an ai meetup
  81. Baseten announces hipaa compliance
  82. How we achieved soc 2 and hipaa compliance as an early stage company
  83. Soc 2 type 2
  84. Technical deep dive truss live reload
  85. Serving four million riffusion requests in two days
  86. 100x faster dev loops with draft models
  87. Openai whisper in five minutes
  88. Why we open sourced truss
  89. Four ml models that accelerate content creation
  90. Announcing our series a
  91. Soc 2 type 1
  92. Docs as code
  93. Designing parental leave at an early stage startup
  94. Working at an early stage company as an early stage engineer
  95. Introducing baseten