data-pipeline

Open repos and models their makers tagged data-pipeline, most starred first. The same for everyone.

Run my own servicesRepo

airbyte

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

  • 22k
  • Python
  • 1 d ago
airbytehqTake it
AI agentsRepo

dagu

Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.

  • 4.6k
  • Go
  • GPL-3.0
  • 5 h ago
dagucloudTake it
Data and analyticsRepo

whylogs

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

  • 2.8k
  • Jupyter Notebook
  • Apache-2.0
  • 23 mo ago
whylabsTake it
DatabasesRepo

olake-go

OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

  • 2.2k
  • Go
  • Apache-2.0
  • 16 h ago
datazip-incTake it
AI agentsRepo

duckle

Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 446 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.

  • 1.6k
  • Rust
  • Apache-2.0
  • 2 d ago
Search GitHub and Hugging Face for data-pipeline

Get , open it once and sign in. Then come back here and press the button again.

data-pipeline: open source repos and models on Take My Repo