pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- 62k
- Python
- 11 h ago
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.
The RaftLib C++ library, streaming/dataflow concurrency via C++ iostream-like operators
High performance, streaming and concurrent functional programming in Haskell
The Refreshingly Simple Cross-Platform C++ Dataflow / Patching / Pipelining / Graph Processing / Stream Processing / Reactive Programming Framework
Documentation for Kestra — an event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
A helper tool to design CG Pipeline interactive diagrams and data flow documentation
A reactive Julia notebook built for ambitious work — distributed compute, agentic AI from day one, high-throughput low-latency streaming to the browser, an extension system that integrates anything, and beautiful rendering, publishing, and standalone exports.
A simple library that provides utilities to compose your application from function-based components
This project provides a complete Terraform setup to automate the deployment of a real-time Change Data Capture (CDC) pipeline on Google Cloud. It streams data from a Cloud SQL for MySQL source to BigQuery, using Google Cloud Storage (GCS) as a staging area and a Dataflow streaming job for processing and enrichment.