Airflow catchup ran 400 DAG runs at once and killed our warehouse, how to prevent this?
Deployed new DAG yesterday with start_date=datetime(2025, 9, 1) and schedule='@daily' . Did not set catchup. Airflow scheduled ~400 runs for all past days, they start in parallel and warehouse queue full for 2 hours, other team jobs stuck. Very embarrassing. with DAG( dag_id='orders_daily_snapshot', start_date=datetime(2025, 9, 1), schedule='@daily', ) as dag: ... I paused the DAG and marked the runs failed…