Apache Airflow
OSSApache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows. It uses ...
Comparable Data Pipeline AI agents, ranked by popularity. Not sold on Hamilton? These are the closest options worth a look.
You're comparing against
Hamilton is a Python-based micro-orchestration framework designed to define and manage dataflows, such as feature engineering, model pipelines, and LLM workflows. It integrates with common tools like Jupyter, FastAPI, Spark, Ray, and Dask, while enforcing software engineering best practices. It includes a self-hostable UI for lineage tracking, execution telemetry, and cataloging.
Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows. It uses ...
Apache NiFi is an open-source data integration tool designed to automate the flow of data between systems. It provides a...
Apache Oozie is a workflow scheduler system designed to manage Hadoop jobs. It allows users to define complex workflows ...
Argo Workflows is a Kubernetes-native workflow engine designed to orchestrate parallel, containerized jobs. It enables u...
Couler provides a unified Python-based interface for defining and managing machine learning workflows across multiple wo...
Dagster is an open-source data orchestrator designed for building, testing, and running data pipelines. It focuses on de...
DBT (Data Build Tool) is an open-source tool that enables data analysts and engineers to transform data in their warehou...
Flyte is a cloud-native workflow automation platform designed for machine learning and data processing. It enables users...
Genie is a job orchestration engine designed to manage and trigger the execution of jobs in Hadoop-based systems. It pro...
Instill VDP is an open-source, self-hostable data pipeline tool designed to automate and manage end-to-end data processi...
Instructor is an open-source tool that simplifies extracting structured data (e.g., JSON) from large language models (LL...
Kedro is an open-source Python framework for creating reproducible, maintainable, and modular data pipelines. It provide...