The Curious Engineer

All blogs / Databases

Azure Data Factory Blog

techcommunity.microsoft.com/category/azuredatabases/blog/azuredatafactoryblog

DatabasesDev tools / editors / IDEsMicrosoft blogsMicrosoft Tech Community

’19’20’21’22’23’24’25’26
Posts
197 from 25 Sep 2019 to 5 Jun 2025
Posting
now and then
Kind
company blog

2025 2

  1. Continued region expansion: Azure Data Factory is generally available in Mexico Central
  2. Announcing the new Databricks Job activity in ADF!

2024 5

  1. Integrate Microsoft Fabric with SAP data with USB4SAP [in live and cached mode]
  2. Optimizing ETL Workflows: A Guide to Azure Integration and Authentication with Batch and Storage
  3. Data Factory Increases Maximum Activities Per Pipeline to 80
  4. Action Required: Switch from Memory Optimized Data Flows in Azure Data Factory to General Purpose
  5. Continued region expansion: Azure Data Factory is generally available in two more regions

2023 27

  1. Continued region expansion: Azure Data Factory is generally available in Poland Central
  2. General Availability of Time to Live (TTL) for Managed Virtual Network in Azure Data Factory
  3. Metadata Driven Pipelines for Dynamic Full and Incremental Processing in Azure SQL
  4. Integer Type Available for Pipeline Variables
  5. Documentation search now embedded in Azure Data Factory
  6. Comment Out Part of Pipeline
  7. Continued region expansion: Azure Data Factory just became generally available in Sweden Central
  8. Securing outbound traffic with Azure Data Factory's outbound network rules
  9. Introducing optional Source settings for DelimitedText and JSON sources for top-level CDC resource
  10. Unroll multiple arrays in a single Flatten step in ADF
  11. ADF private DNS zone overrides ARM DNS resolution causing ‘Not found’ error.
  12. Trigger ADF pipeline using Storage event trigger over private network.
  13. Pipeline Logic 3: Error Handling and Try Catch
  14. Capture Changed Data From your Cosmos DB analytical store (Preview)
  15. Azure Data Factory February 2023 Monthly Update
  16. Process your data in seconds with new ADF real-time CDC
  17. Introducing Dark Mode for ADF Studio
  18. Best practices for Azure Data Factory Integration Runtime
  19. You can now customize the return value from your pipeline!
  20. Azure Data Factory January 2023 Monthly Update
  21. Directly Launch Pipeline Template Gallery through the Azure Portal
  22. Introducing 'Workflow Orchestration Manager' powered by Apache Airflow in Azure Data Factory
  23. Orchestrate and operationalize Synapse Notebooks and Spark Job Definitions from Azure Data Factory
  24. Pipeline Logic 2: OR (at least 1 activity succeeded or failed)
  25. Announcing the Public Preview of a new top-level CDC resource in ADF
  26. Pipeline Logic 1: Error Handling and Best Effort Step
  27. Connecting to Snowflake using Key-Pair Authentication in Azure Data Factory

2022 46

  1. Process Azure Event Hubs data using Azure Data Factory Mapping Data Flows
  2. Azure Data Factory November 2022 Monthly Update
  3. Azure Data Factory October 2022 Monthly Update
  4. Introducing the Learning Center to Azure Data Factory Studio
  5. Improved UI for Launching Azure Data Factory Studio
  6. Granular Billing for Azure Data Factory
  7. Configuring Private endpoint to Snowflake Private link using managed Virtual Network
  8. Introducing Azure Data Factory Community Templates
  9. General availability of SAP CDC capabilities for Azure Data Factory and Azure Synapse Analytics
  10. Data Integration News from Ignite 2022: SAP CDC, M365 Datasets, and More
  11. Azure Data Factory September 2022 Monthly Update
  12. Azure VNET vs Azure IR inside of Pipelines
  13. Azure Data Factory August 2022 Monthly Update
  14. Azure Data Factory Updated Pipeline Designer
  15. Further ADF Monitoring Improvements
  16. Copy Dataverse data from ADLS Gen2 to Azure SQL DB leveraging Azure Synapse Link
  17. CI/CD improvements related to pipeline triggers deployment
  18. Azure Data Factory Changing Default Pipeline Activity Timeout
  19. New Experience for Creating Data Factory within Seconds
  20. Cast transformation added to mapping data flows
  21. Azure Data Factory July 2022 Monthly Update
  22. 3 New Data Transformation Functions in ADF
  23. ADF Makes it Easy to Select Azure IR Size for Data Flows
  24. Map Data tool in Synapse Analytics now generally available
  25. Azure Data Factory June 2022 Monthly Update
  26. Coming soon to ADF: More Pipeline Expression Builder Ease-of-Use Enhancements!
  27. Introducing the Azure Data Factory Studio Preview Experience
  28. Announcing public preview of Time-To-Live (TTL) in managed virtual network
  29. CI/CD improvement using Global Parameters in Azure Data Factory
  30. Announcing Public Preview of the SAP CDC solution in Azure Data Factory and Azure Synapse Analytics
  31. Introducing User Defined Functions preview for mapping data flows in ADF and Synapse
  32. Transform data in ADF with Azure Cognitive Services
  33. Data flow sink supports user db schema for staging in Azure Synapse and PostgreSQL connectors
  34. Azure Functions available in ADF managed virtual network
  35. ADF Monitoring Improvements
  36. AND logic - Many Pre-processing Jobs - Part 1
  37. New Connectors Available in Mapping Data Flow
  38. Data Preview and Debug Improvements in Mapping Data Flows
  39. Flowlets and Change Feed now GA in Azure Data Factory
  40. Web activity response timeout improvement
  41. ADF Mapping Data Flows adds ScriptLines and Link Service parameter support
  42. New UI layout in Azure integration runtime creation and edit page
  43. Execute SQL statements using the new 'Script' activity in Azure Data Factory and Synapse Pipelines
  44. Introducing the 'Data Integration in a box' solution
  45. New Connector Available: Dynamics 365 are Supported as both Source and Sink in Mapping Data Flow
  46. Introducing Map Data preview for Synapse

2021 48

  1. 'IntelliSense' support in Expression Builder for more productive Pipeline authoring experiences
  2. Native change data capture (CDC) - CosmosDB change feed is supported in ADF now
  3. Introducing the Flowlets preview for ADF and Synapse
  4. Use ADF to Turn Hierarchical Structures into Relational Data
  5. Mapping Data Flow gets new native connectors
  6. Support for user-assigned managed identity in Azure Data Factory
  7. General availability (GA): Azure Data Factory managed virtual network
  8. Data Wrangling at Scale with ADF's Power Query Activity, now Generally Available
  9. Operationalize and Provide SLA for Data Pipelines
  10. CICD Improvements with GitHub support in Azure Government and Azure China
  11. Event Driven Architecture with Custom Event Trigger and Advanced Filtering
  12. Synapse Data Flows Enables Direct Workspace DB Connector
  13. Diagnostic tool for self-hosted integration runtime
  14. REST source and sink now available for data flows
  15. Share Your Best Practice about Data Integration
  16. Support setting custom metadata when writing to Blob/ADLS Gen2
  17. Get metadata driven data ingestion pipelines on ADF within 10 minutes
  18. The new and refreshing data factory home page
  19. SharePoint Online Multiple Files (Folder) Copy with Http Connector
  20. Send notification on a Microsoft Teams channel from a Data Factory pipeline
  21. Azure Data Factory Copy now supports Always Encrypted for both source and sink
  22. New Connectors Available in ADF: MongoDB and MongoDB Atlas are Supported as both Source and Sink
  23. Updated data flows monitoring UI in ADF & Synapse
  24. A Re-designed Copy Data Tool Experience
  25. New Data Flow Connector: SQL Server as Source and Sink
  26. New Connectors Available in ADF: Oracle Cloud Storage; Amazon S3 Compatible Storage
  27. How to access on premises data stores from ADF managed virtual network using Private Endpoint
  28. XML added to Parse transformation in ADF and Synapse Data Flows
  29. Understanding Pipeline Failures and Error Handling 2
  30. How to startup your data flows execution in less than 5 seconds! (Public Preview)
  31. Azure Data Factory Product April 2021 Updates - At a Glance
  32. ADF connector updates: Azure Data Explorer, SFTP, REST and HTTP
  33. ADF adds Azure Database for MySQL connector in Data Flow
  34. Improved support for Delta Lake added to data flows in ADF and Synapse
  35. Retrieve and Reference Trigger Metadata in Pipeline
  36. Azure Data Factory Ignite 2021 Updates
  37. [UI Only] Event Trigger Renamed to Storage Event Trigger
  38. Bring SSIS ETL lineage into Azure Purview Data Map
  39. Bootstrap ETL process by bringing Azure Purview assets into Azure Data Factory
  40. Analyze root cause and impact using ADF ETL lineage in Azure Purview
  41. Data Factory Azure Policy integration is live now
  42. Data Flows in Azure Data Factory Now Support Reserved Instance Pricing
  43. Automated publish improvement in ADF's CI/CD flow
  44. Azure Data Factory UI Getting Command Bar Updates
  45. How to use Profisee with Azure Data Factory (ADF)
  46. Azure Data Factory is HITRUST Certified
  47. Storage Event Trigger - Permission and RBAC setting
  48. Azure Data Factory Enables Data Wrangling at Scale with Power Query

2020 54

  1. Implement Fill Down in ADF and Synapse Data Flows
  2. ADF connector updates: Azure Database for PostgreSQL, REST and Excel
  3. ADF Data Flow Updates: CDM and Delta Lake Connectors Generally Available
  4. You can enable data consistency verification in copy activity
  5. Session log is now available in copy activity
  6. Azure Databricks activities now support Managed Identity authentication
  7. How to Handle SQL DB Row-level Errors in ADF Data Flows
  8. Data flow zoom improvements
  9. ADF Adds Cached Lookups to Data Flows
  10. Time Zone and Daylight Saving Support for Schedule Trigger
  11. Performance Tune ADF Data Flow Transformations
  12. Cancel and Rerun Tumbling Window Trigger Run
  13. New MongoDB Atlas connector available in ADF
  14. Performance Tuning ADF Data Flow Sources and Sinks
  15. Read and Write Complex Data Types in ADF
  16. Rank transformation and find function in mapping data flows
  17. Use Azure IR to Tune ADF and Synapse Data Flows
  18. Data Factory adding ORC and SQL MI (preview) for ADF Data Flows and Synapse Data Flows
  19. Mapping data flow derived column and expression builder updates
  20. ADF Data Flows Sneak Peek: Expression Builder UX Updates
  21. See your related resources in ADF
  22. ADF Data Flow Adds Support For No Quote Character in Data Flows (Coming Soon)
  23. Understanding Pipeline Failures and Error Handling
  24. ADF adds Snowflake support in Mapping Data Flow
  25. ADF debugging improvements
  26. Global Parameters generally available in Azure Data Factory
  27. ADF Release Notes 07/31/2020
  28. Best practices of how to use ADF copy activity to copy new files only
  29. Azure Data Factory adds support for XML format
  30. Azure Data Factory Managed Virtual Network
  31. ADF Adds Connectors for Delta Lake and Excel
  32. Data Flow activity can now run on cross-region integration runtimes
  33. ADF now supports data integration with Snowflake
  34. ADF Adds Support for Inline Datasets and Common Data Model to Data Flows
  35. Introducing the Management Hub to Azure Data Factory
  36. New ADF pipeline consumption report
  37. New data flow functions for dynamic, reusable patterns
  38. Introducing the Properties Pane to Azure Data Factory
  39. ADF adds new hierarchical data handling and new flexibility for complex joins
  40. How to Handle Evolving Database Schemas in your ETL with Azure Data Factory
  41. ADF Data Flows Script Intro: Copy, Paste, Snippets
  42. ADF adds Managed Identity & Service Principal to Data Flows Synapse staging
  43. How to Save Your Data Profiler Summary Stats in ADF Data Flows
  44. Transform complex JSON structures from CosmosDB to SQL DB with Azure Data Factory
  45. Coming soon to Azure Data Factory
  46. ETL in the Cloud is Made Easy Together with Azure Data Factory and Azure Databricks
  47. Azure Data Factory UI Design Update for Container Activities
  48. ADF Data Flow Expressions: Tips and Tricks
  49. Parquet format support added to Wrangling Data Flow in Azure Data Factory
  50. Basic and Cert Authentication support added to ADF Web & Webhook activities in GIT mode
  51. New data flow features: Import schema and test connection from debug cluster, custom sink ordering
  52. Azure Data Factory Adds Managed Identity Support to Data Flows
  53. Move petabytes of data with resilience – Azure Data Factory adds resume support!
  54. Azure Data Factory now supports Static IP address ranges

2019 15

  1. Evaluate Data Flow Expressions Inline with Strings in ADF
  2. Download Azure Data Factory support files
  3. Load different SQL database tables based on different file types in ADF
  4. Transform Data at Scale Easily with ADF Data Flows: Getting Started
  5. Azure Data Factory Wrangling Data Flows are now in public preview
  6. ADF Adds Avro and CosmosDB Data Flow Connectors
  7. Data Factory is now a 'Trusted Service' in Azure Storage and Azure Key Vault firewall
  8. Azure Data Factory Mapping Data Flows for U-SQL Developers
  9. ADF Mapping Data Flows for Databricks Notebook Developers
  10. Create Generic SCD Pattern in ADF Mapping Data Flows
  11. Support for Azure Databricks Instance pool for operationalizing Databricks workloads in Data Factory
  12. Popular links to learn ADF Mapping Data Flows
  13. ADF Adds Hierarchical & JSON Data Transformations to Mapping Data Flows
  14. ADF adds TTL to Azure IR to reduce Data Flow activity times
  15. Welcome to the new Azure Data Factory blog!