etl

Open repos and models their makers tagged etl, most starred first. The same for everyone.

AutomationRepo

kestra

Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.

  • 30k
  • Java
  • Apache-2.0
  • 2 d ago
kestra-ioTake it
Run my own servicesRepo

airbyte

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

  • 22k
  • Python
  • 1 d ago
airbytehqTake it
Data and analyticsRepo

cloudquery

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

  • 6.6k
  • Go
  • MPL-2.0
  • 1 d ago
cloudqueryTake it
Data and analyticsRepo

aws-sdk-pandas

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

  • 4.1k
  • Python
  • Apache-2.0
  • 1 d ago
awsTake it
Data and analyticsRepo

sqlmesh

Scalable and efficient data transformation framework - backwards compatible with dbt.

  • 3.4k
  • Python
  • Apache-2.0
  • 1 d ago
SQLMeshTake it
Data and analyticsRepo

hamilton

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

  • 2.6k
  • Jupyter Notebook
  • Apache-2.0
  • 6 d ago
apacheTake it
Data and analyticsRepo

devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

  • 3.2k
  • Go
  • Apache-2.0
  • 3 d ago
apacheTake it
Data and analyticsRepo

proton

The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.

  • 2.3k
  • C++
  • Apache-2.0
  • 4 d ago
timeplus-ioTake it
Developer toolsRepo

instill-core

🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications

  • 2.3k
  • Python
  • 6 mo ago
instill-aiTake it
DatabasesRepo

olake-go

OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

  • 2.2k
  • Go
  • Apache-2.0
  • 18 h ago
datazip-incTake it
Data and analyticsRepo

dozer

Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.

  • 1.6k
  • Rust
  • AGPL-3.0
  • 2 y ago
getdozerTake it
Search GitHub and Hugging Face for etl

Get , open it once and sign in. Then come back here and press the button again.

etl: open source repos and models on Take My Repo