Apache Airflow
OSSApache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows. It uses ...
Comparable Data Pipeline AI agents, ranked by popularity. Not sold on Pachyderm? These are the closest options worth a look.
You're comparing against
Pachyderm is an open-source data pipeline platform built on Kubernetes, designed to automate and manage machine learning workflows with version-controlled data and reproducible pipelines. It enables dynamic processing of large-scale datasets, ensuring traceability and consistency across ML experiments and deployments.
Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows. It uses ...
Apache NiFi is an open-source data integration tool designed to automate the flow of data between systems. It provides a...
Apache Oozie is a workflow scheduler system designed to manage Hadoop jobs. It allows users to define complex workflows ...
Argo Workflows is a Kubernetes-native workflow engine designed to orchestrate parallel, containerized jobs. It enables u...
Couler provides a unified Python-based interface for defining and managing machine learning workflows across multiple wo...
Dagster is an open-source data orchestrator designed for building, testing, and running data pipelines. It focuses on de...
DBT (Data Build Tool) is an open-source tool that enables data analysts and engineers to transform data in their warehou...
Flyte is a cloud-native workflow automation platform designed for machine learning and data processing. It enables users...
Genie is a job orchestration engine designed to manage and trigger the execution of jobs in Hadoop-based systems. It pro...
Hamilton is a Python-based micro-orchestration framework designed to define and manage dataflows, such as feature engine...
Instill VDP is an open-source, self-hostable data pipeline tool designed to automate and manage end-to-end data processi...
Instructor is an open-source tool that simplifies extracting structured data (e.g., JSON) from large language models (LL...