April 2025, Revisited: Airflow 3.0, the Biggest Release the Project Ever Shipped

This post is part of our Three-Year Retrospective series: thirty-six posts, one per month, looking back at what actually mattered in software engineering. This one covers April 2025.

On April 22, 2025, Apache Airflow 3.0 went GA — by wide agreement the biggest release in the project's history, and the first major version since 2.0 arrived in December 2020. Airflow had spent the 2.x era accumulating capability while carrying architectural decisions made when "data orchestration" meant nightly cron-shaped batch jobs. Airflow 3 was the project deciding, at last, what it wanted to be for the next decade: an orchestrator where tasks are decoupled from the scheduler's internals, pipelines are described in terms of the data they produce, and history is something the system actually remembers.

The Task SDK draws the line

The change with the longest consequences was the Task SDK. Airflow 3 introduced airflow.sdk as the official, stable surface for DAG authors — DAG, @dag, @task, Param, the context helpers — replacing years of DAG files importing from airflow.models, airflow.utils, and whatever internal module happened to work.

This wasn't namespace tidying. In Airflow 3, workers no longer connect to the metadata database; tasks execute against an execution API instead. That single architectural cut is what made the rest possible: workers that run in remote networks via the new Edge Executor, a path toward task code in languages other than Python, and a security posture where task code simply can't reach into the scheduler's state. The price was paid by every DAG that had quietly opened a database session or called internal APIs — that code broke, deliberately and permanently.

Assets, versioning, and a new process diagram

Three more changes defined the release. Datasets became Assets: more than a rename, the new @asset decorator inverted pipeline authorship — define the data product, and the task that materializes it comes along — while event-driven scheduling let external events trigger runs without sensor gymnastics. In code, the inversion was a single decorator from the new namespace:

from airflow.sdk import asset


@asset(uri="s3://warehouse/orders.parquet", schedule="@daily")
def orders():
    """Define the data product; the task that materializes it comes along."""
    ...

DAG versioning became automatic: Airflow 3 records which version of the DAG code each run executed, so the UI can show you last Tuesday's failure against last Tuesday's graph. Anyone who has debugged a modified DAG against old run history understands exactly how big a deal this was. And the runtime topology changed: the webserver gave way to an api-server hosting both the new React UI and the execution API workers talk to, with the dag processor promoted to a mandatory standalone component.

The removals were just as decisive. SubDAGs: gone entirely, with TaskGroups as the long-standing replacement. SLAs: gone, replaced by scheduler-evaluated Deadline Alerts — a real redesign for anyone with alerting built on sla_miss callbacks, not a rename. Airflow 3 was unusually willing, for a project of its install base, to delete its mistakes.

What April 2025 advice sounded like

Our guidance at GA time was patience with a plan: let the 3.0.x line stabilize, but start the prep work immediately, because all of it was useful on 2.x anyway. Upgrade to the latest 2.x bridge release; run the ruff-based upgrade checks in CI and fix imports toward airflow.sdk while still on Airflow 2; excise SubDAGs and SLA callbacks; audit task code for metadata-database access. Teams that did this made the eventual cutover an afternoon. Teams that treated 3.0 as a routine version bump discovered, in the worst way, that it wasn't one.

Looking back from June 2026

The release earned its billing. The 3.x line matured quickly, the migration wave through late 2025 and into this year went smoothly for prepared teams — we wrote up the detailed field guide in our Airflow 2-to-3 migration post — and the architectural bets are already paying out: remote execution is genuinely used, assets changed how new pipelines get designed, and DAG versioning is the feature nobody migrated for but everyone now refuses to live without. April 2025 reset what teams expect an orchestrator to be.

We teach the full Airflow 3 picture — Task SDK, assets, event-driven scheduling, the new architecture — hands-on in Apache Airflow Programming: Developing, Configuring, and Automating Workflows, and for platform teams running it on Kubernetes, Apache Airflow for Developers (Kubernetes) covers executors, topology, and deployment in depth.