The Curious Engineer

All blogs / Software / engineering practice

Dagster Blog

dagster.io/blog

Read the latest from the Dagster team: insights, tutorials, and updates on data engineering, orchestration, and building better pipelines.

Software / engineering practicePythonDevOps / SRE / infrastructure

’21’22’23’24’25’26
Posts
176 from 16 Nov 2021 to 8 Sep 2026
Posting
weekly (42 in the last year)
Kind
company blog
Feed
https://dagster.io/blog/rss.xml

2026 33

  1. The Data Plane Is Yours: Dagster+ Hybrid on Kubernetes and Azure
  2. Orchestration is More than Scheduling: Declarative Automation in Dagster
  3. Classifying a Million Snowflake Columns in 9 Days, Solo, with Dagster
  4. Prefect is Acquiring Dagster
  5. Community Showcase Part 2
  6. Operationalizing Data Orchestration: Best Practices for DevOps, Infra, and Code Locations
  7. How to Make the Architectural Case for Dagster
  8. Community Showcase Part 1
  9. How Dagster Compass Powers Brooklyn Data’s Self-Service Analytics
  10. Snowflake Runs Your Data: Dagster Runs Everything Else
  11. We Tried ty for Performance. It Found Real Bugs
  12. The Dagster Almanack: From Complexity to Composability
  13. Announcing the Dagster+ Terraform Provider
  14. The Missing Half of the Enterprise Context Layer
  15. How to Orchestrate Across Multiple Databricks Workspaces Without Losing Your Mind
  16. Dagster 1.13: Octopus's Garden
  17. Monorepos, the hub-and-spoke model, and Copybara
  18. Making Dagster Easier to Contribute to in an AI-Driven World
  19. DataOps with Dagster: A Practical Guide to Building a Reliable Data Platform
  20. Unlocking the Full Value of Your Databricks
  21. Announcing AI Driven Data Engineering
  22. When to Move from Dagster OSS to Dagster+
  23. Sample-Level Versioning for ML Pipelines with Dagster and Metaxy
  24. Evaluating Skills
  25. Great Infrastructure Needs Great Stories: Designing our Children’s Book
  26. Closing the DataOps Loop: Why We Built Compass for Dagster+
  27. Pytest for Agent-Generated Code: Concrete Testing Strategies to Put Into Practice
  28. Dagster + Snowflake: Building Production AI Pipelines with Cortex
  29. Your GTM Data, Finally Untangled
  30. Dignified Python: 10 Rules to Improve your LLM Agents
  31. Evaluating Model Behavior Through Chess
  32. How to Enforce Data Quality at Every Stage: A Practical Guide to Catching Issues Before They Cost You
  33. When Sync Isn’t Enough

2025 45

  1. How to Build a Data Platform That Actually Scales
  2. Orchestrating Nanochat: Deploying the Model
  3. Data Ingestion Patterns: When to Use Push, Pull, and Poll (With Real Examples)
  4. Dagster + Atlan: Real-Time Asset Observability in Your Data Catalog
  5. Orchestrating Nanochat: Training the Models
  6. Orchestrating Nanochat: Building the Tokenizer
  7. When (and When Not) to Optimize Data Pipelines
  8. Your Data Team Shouldn't Be a Help Desk: Use Compass with Your Data
  9. Introducing Our New eBook: Scaling Data Teams
  10. Dagster 1.12: Monster Mash
  11. How Compass Turns Questions Into Queries
  12. Scaling Analysis Without Scaling the Team
  13. Bridging High-Code and Low-Code
  14. Building a Better Lakehouse: From Airflow to Dagster
  15. Designing User-Friendly Dagster Components
  16. Dagster+ Now Available in the EU
  17. Dagster Components are Generally Available (GA)
  18. Introducing Compass: Data-driven decisions right in Slack
  19. Introducing the new Dagster+ UI – Your Data Platform’s Command Center
  20. dbt Fusion Support Comes to Dagster
  21. What CoPilot Won’t Teach You About Python (Part 2)
  22. Software-Defined Assets Explained
  23. Untangling Python Packages Part 2
  24. Untangling Python Packages Part 1
  25. A Practical Guide to Dagster Resources
  26. How to Structure Your Dagster Project
  27. What CoPilot Won’t Teach You About Python (Part 1)
  28. Announcing ETL Course with Dagster
  29. Dagster 1.11: Build Me Up Buttercup
  30. DSLs to the Rescue
  31. Code Location Best Practices
  32. Beyond Point to Point
  33. Dagster Pipes: Now available for TypeScript, Rust, and Java
  34. Cut Through the Noise: Precision Data Management with Dagster's Asset Selection Framework
  35. Accelerate Data Pipeline Development with Dagster Components
  36. The Case for Dagster: Moving Beyond Airflow in the Modern Data Stack™
  37. Why we love uv
  38. Free Your Mind With Dagster
  39. MS Fabric vs. Dagster: Why Your Architecture Choices Matter
  40. Dagster University Presents: Testing with Dagster
  41. Observability That Matters with Dagster+ Alerts
  42. Building with Dagster vs Airflow
  43. Dagster 1.10: Mambo No 5
  44. AI Reference Architectures
  45. From Prototype to Production: Building AI Products That Scale with Dagster

2024 40

  1. Data Platform Week 2024
  2. Interactive Debugging With Dagster and Docker
  3. Announcing 4 New Integrations: Dagster + Your Favorite BI Tools
  4. Bridging Business Intelligence and Data Orchestration with Dagster + Sigma
  5. Dagster 1.9: Spooky
  6. AI's Long-Term Impact on Data Engineering Roles
  7. From Chaos to Control: How Dagster Unifies Orchestration and Data Cataloging
  8. 10 Reasons Why No-Code Solutions Almost Always Fail
  9. 5 Best Practices AI Engineers Should Learn From Data Engineering
  10. Dagster Deep Dive Recap: Orchestrating Flexible Compute for ML with Dagster and Modal
  11. The Rise of the Data Platform Engineer
  12. Dagster vs. Airflow
  13. What is Data Visibility?
  14. Dagster Deep Dive Recap: Building a True Data Platform
  15. Dagster Deep Dive Recap: Evolution of the Data Platform
  16. Dagster 1.8: Call Me Maybe
  17. Dagster Deep Dive Recap: Building Reliable Data Platforms
  18. A Look Inside the Dagster Labs Culture
  19. Enabling Data Quality with Dagster and Great Expectations
  20. The Rise of Medium Code
  21. Dagster’s Code Location Architecture
  22. What is Dagster: A Guide to the Data Orchestrator
  23. Building Cost Effective AI Pipelines with OpenAI, LangChain, and Dagster
  24. Unlocking Flexible Pipelines: Customizing the Asset Decorator
  25. Change Tracking Branch Deployments in Dagster+
  26. Dagster+ Catalog: A New Built-in Asset Library for All Practitioners
  27. Ensuring Reliable Data with Dagster+
  28. The Data Engineering Impedance Mismatch
  29. Announcing Dagster 1.7: Love Plus One
  30. Expanding the Dagster Embedded ELT Ecosystem with dltHub for Data Ingestion
  31. Sling Out Your ETL Provider with Embedded ELT
  32. Exploring The Data Engineering Lifecycle
  33. New Dagster Integration: Include OpenAI Calls Into Your Data Pipelines
  34. Breaking Packages in Python
  35. Balancing the Data Scales: Centralization vs. Decentralization
  36. Standardize Pipelines with Domain-Specific Languages
  37. Thinking in Assets When Building Data Pipelines
  38. What Dagster Believes About Data Platforms
  39. Announcing Dagster 1.6: Back to Black
  40. Retain.ai joins Dagster Labs

2023 38

  1. How Dagster Labs runs Dagster: Open-Sourcing our Own Pipelines
  2. Scaling Dagster’s DAG Visualization to Handle Tens of Thousands of Assets
  3. High-performance Python for Data Engineering
  4. Orchestrate Unstructured Data Pipelines with Dagster and dlt
  5. CI/CD and Data Pipeline Automation (with Git)
  6. Introducing Dagster External Assets
  7. Introducing Dagster Pipes
  8. Stop Reinventing Orchestration: Embedded ELT in the Orchestrator
  9. Improving the Dagster learning curve
  10. Improving visibility into data operations with Dagster Insights
  11. Introducing Dagster Asset Checks
  12. Write-Audit-Publish in data pipelines
  13. Escaping the Modern Data Trap
  14. Factory Patterns in Python
  15. Introducing Dagster Labs
  16. Building an Outbound Reporting Pipeline
  17. Type Hinting in Python
  18. Environment Variables in Python
  19. Podcast: Drill to Detail - Dagster, Orchestration and Software-Defined Assets
  20. Orchestrating dbt™ with Dagster
  21. Speeding up the dbt™ docs by 20x with React Server Components
  22. Announcing Dagster 1.4: Material Girl
  23. LLM training pipelines with Langchain, Airbyte, and Dagster
  24. The Dagster Master Plan
  25. Backfills in Data & Machine Learning: A Primer
  26. Dagster and the Decade of Data Engineering
  27. Elementl Raises $33 Million in Series B Funding to Accelerate Data Orchestration and Unleash Advanced Data Use Cases
  28. Introducing Dynamic Definitions for Flexible Asset Partitioning
  29. From Python Projects to Dagster Pipelines
  30. Community Memo: Pythonic Config and Resources
  31. Best Practices in Structuring Python Projects
  32. Partitions in Data Pipelines
  33. Tracking the Fake GitHub Star Black Market with Dagster, dbt and BigQuery
  34. Announcing Dagster 1.2: Formation
  35. How Dagster Deploys 5X Faster with Warm Docker Containers
  36. Python Packages: a Primer for Data People (part 1 of 2)
  37. Python Packages: a Primer for Data People (part 2 of 2)
  38. Build a GitHub Support Bot with GPT3, LangChain, and Python

2022 18

  1. Announcing Dagster 1.1: Thank U, Next
  2. Getting Stuff Done: a Guide to Productive Software Engineering
  3. My Path to Elementl - Part 2
  4. Pushing REST-API data to Google Sheets with Dagster
  5. Adding Types to a Large Python Codebase
  6. Orchestrating Machine Learning Pipelines with Dagster
  7. Build a poor man’s data lake from scratch with DuckDB
  8. The Unreasonable Effectiveness of Data Pipeline Smoke Tests
  9. Dagster at all 5 Steps of the Development Lifecycle
  10. A Dagster Crash Course
  11. Postgres: a Better Message Queue than Kafka?
  12. Spend Less Time Debugging with Dagster
  13. Launching Dagster Cloud to GA
  14. Introducing Dagster 1.0: Hello
  15. The Open Core Business Model
  16. Dagster Cloud goes SOC 2
  17. My Path to Elementl: Pete Hunt
  18. Rebundling the Data Platform

2021 2

  1. Introducing Dagster Cloud
  2. Why Elementl and Dagster: The Decade of Data