airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- 48k
- Python
- Apache-2.0
- 1 h ago
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Open-source data movement for ELT pipelines and AI agents β from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
π§ Build, run, and manage data pipelines for integrating and transforming data.
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Scalable and efficient data transformation framework - backwards compatible with dbt.
OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 431 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
π Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. π
Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
ReplicaDB is open source tool for database replication, designed for efficiently transferring bulk data between relational and non-relational databases
CLI tool for dbt users to simplify creation of staging models (yml and sql) files
Pluggable data replication with checkpointing, batching, and integrity events
PyAirbyte brings the power of Airbyte to every Python developer. Powers the Airbyte Cloud Replication MCP.
Move whole tables between databases fast β Postgres, MySQL, ClickHouse, BigQuery. Rust engine, one-line Python API, bounded memory.
Frontend & BFF (Backend for frontend) for Olake. This includes the UI code and backend code for storing the configuration of sync and orchestrating it.
The fast, config-driven way to move data in Rust β a native CLI and an embeddable Rust ETL library
Local-first ETL and data analysis desktop app. Describe a pipeline in plain English, review the plan, run it on DB or locally using DuckDB. Bring your own model or run local LLMs.