dataset-curation

Open repos and models their makers tagged dataset-curation, most starred first. The same for everyone.

AI on my own machineRepo

jev-curate

High-throughput synthetic and pretraining dataset sifter for TypeSafe Jev. Rust streaming core, Parquet and JSONL I/O, typed Choice/Score/Noul judgments, speculative fan-out, 24.0 rows/sec measured.

  • 115
  • Rust
  • MIT
  • 11 d ago
GitHubRepo

jev-account-filter

Account screening, channel screening and classification with Jev for dataset curation and research. Find sign language, news, piano/instrument performance and makeup tutorial creators from titles and thumbnails. 视频平台工具:批量筛选账号与频道,寻找手语账号、新闻频道、乐器演奏与美妆博主。Douyin, Kuaishou, YouTube, Bilibili.

  • 44
  • Python
  • MIT
  • 5 wk ago
caiweihanTake it
AI on my own machineRepo

TSDS

Implementation of TSDS: Data Selection for Task-Specific Model Finetuning. An optimal-transport framework for selecting domain-specific and task-specific training data to improve LLM finetuning and instruction tuning.

  • 19
  • Python
  • MIT
  • 23 mo ago
ZifanLTake it
AI on my own machineRepo

LLM_InSight

This is my personal home rig for serious LLM experimentation. I built it to test models head-to-head, create custom evaluation rubrics, automatically improve prompts based on the previous run’s results, and generate high-quality synthetic training data. Everything runs locally first (Ollama by default), with optional cloud support. logged locally.

  • 5
  • Python
  • MIT
  • 5 mo ago
yuvhaim-gifTake it
Search GitHub and Hugging Face for dataset-curation

Get , open it once and sign in. Then come back here and press the button again.