duplicate-detection
duplicut
Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)
- 984
- C++
- GPL-3.0
- 10 d ago
slopo
Embedding-based discovery of duplicated logic for projects too large or messy to maintain reliably
- 834
- Python
- AGPL-3.0
- 3 wk ago
dude
Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.
- 198
- Python
- 4 wk ago
fast-near-duplicate-image-search
Fast Near-Duplicate Image Search and Delete using pHash, t-SNE and KDTree.
- 163
- Python
- Apache-2.0
- 4 y ago
apollo
Advanced similarity and duplicate source code proof of concept for our research efforts.
- 51
- Python
- GPL-3.0
- 4 y ago
QtHashSum
File Checksum Integrity Verifier & Duplicate File Finder written in C++ Qt
- 48
- C++
- GPL-3.0
- 2 y ago
rekordbox-migration-helper
DJ library editor for Rekordbox and Serato. Inspect, organize, review duplicates, check file health and prepare safer migrations on Linux, macOS and Windows.
- 7
- Python
- MIT
- 1 d ago
aisets
🖼️ Local image library auditor — find duplicates, optimize sizes, search by meaning, and stage assets on an AI canvas.
- 6
- TypeScript
- MIT
- 3 mo ago
photo-renamer
Batch-rename and deduplicate photos and videos using EXIF/QuickTime metadata, perceptual hashing, and Apple Live Photo pairing.
- 2
- PHP
- MIT
- 21 h ago
Catfish
files indexing / disk cataloging tool, with duplicates finding, desktop gui, cathy-inspired and backwards-compatible for .caf files
- 2
- Python
- MIT
- 6 d ago
dupdig
CLI tool that walks a directory tree, hashes every file with SHA-256, and generates reports on duplicate files
- 2
- Go
- GPL-3.0
- 3 wk ago
codedupes
Detect duplicate & unused code (py, C, Rust, js/ts) via AST hashing, Jaccard sim, and semantic embeddings (ModernBERT, EmbeddingGemma). w hybrid synthesis
- 2
- Python
- Apache-2.0
- 8 wk ago
dedupe_requests
Automatic server-side de-duplication of inbound mutating Rails requests (POST / PUT / PATCH) — rejects duplicates with a 409 using a Redis-backed request fingerprint -- DOES NOT require client-side idempotency key
- 1
- Ruby
- MIT
- 5 mo ago








