mega-data-factory
🏭 Mega Scale Multimodal DataPipeline for SOTA Foundation Models
- 382
- Python
- MIT
- 6 mo ago
🏭 Mega Scale Multimodal DataPipeline for SOTA Foundation Models
Tensorflow 2 Tutorials (use tensorflow and keras in a better way!)
Extract transform load CLI tool for extracting small and middle data volume from sources (databases, csv files, xls files, gspreadsheets) to target (databases, csv files, xls files, gspreadsheets) in free combination.
Материалы для курса Введение в Data Engineering: дата пайплайны
Data engineering skills with Python ETL/ELT projects and warehousing.
Dagster Tutorial to get you started with Dagster as an absolute Beginner. The tutorial covers various topics like Dagster Installation, Dagster Asset, Dagster Job, Dagster Scheduler, Dagster Ops, and more. It is completely free on YouTube and is beginner-friendly without any prerequisites.
Let's be honest - most data pipeline frameworks treat types as suggestions. Config files are strings. Schemas are "validated" at runtime. Data quality is an afterthought. So, let's do differently
Python event consumer boilerplate which takes advantages from combine of multiprocessing, multithreading and asynchronous programming