trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
- 7.2k
- Python
- Apache-2.0
- 2 d ago
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
A command-line tool to turn web pages into readable PDF, EPUB, HTML, or Markdown docs.
Full-content web fetcher for AI agents — Chrome TLS fingerprinting, browser impersonation, multi-strategy article extraction, and web scraping.
A Python utility for moving bookmarks/reading lists between services
Analyze key metrics like number of words, readability, complexity, code duplication, … of any kind of text
Use LLMs via OpenRouter to simplify German institutional communication into Einfache or Leichte Sprache, with writing coaching and ZIX readability scoring.
Technical writing assistant, side-panel Markdown editor, Model Context Protocol (MCP) server, and Agent Skills suite for Antigravity: Gunning Fog readability, AI slop detection, Vale linting, and SEO/GEO.
Modern Android RSS reader for phones & tablets, featuring full-text reading, multi-source discovery, and bi-directional AI citation back to source.
Multilingual readability library for Python - 86 languages, 17 formulas, TeX-based syllable counting via Frank M. Liang algorithm.
Multilingual readability library for PHP - 86 languages, 17 formulas, TeX-based syllable counting via Frank M. Liang algorithm. Zero dependencies.
GoodJsCode™ guidebook. Learn the best coding practices and standards for clean, efficient and quality JavaScript code that lasts long! 💪
Multilingual readability engine for Node.js/TypeScript - 86 languages, 17 formulas, TeX (Frank M. Liang) syllable counting. Zero runtime deps, dual ESM+CJS. Byte-accurate port of the PHP ReadSight library.
Multilingual readability library for Rust - 86 languages, 17 formulas, TeX-based syllable counting via the Frank M. Liang algorithm. Byte-accurate port of PHP/Python ReadSight, zero network, embedded data.
Generating Summaries with Controllable Readability Levels (EMNLP 2023)
Fast, idiomatic Go MCP server for web search, image search, and page scraping to Markdown. Runs locally, no API keys, free.
A PHP-based reading list for web articles. Maintains a searchable archive and computes lots of stats.
CommonLit Readability Prize 2021. NLP competition hosted at Kaggle: https://www.kaggle.com/c/commonlitreadabilityprize/. Rate the complexity of literary passages for grades 3-12 classroom use
Code, models, and data for "Strategies for Arabic Readability Modelling". ArabicNLP 2024, ACL.
An anti-AI-slop writing standard and checker for AI-drafted text. Scores any document 0-100 against 43 rules on four axes: plain words, human voice, structure, information density. One JSON rule pack compiles to 18 targets - Claude Code skill, AGENTS.md, Cursor rule, system prompt, Vale style. Zero dependencies, runs offline.
Article title, authors, date and body extraction dataset.
AI-powered Chrome extension that makes any web article accessible — on-device LLM via WebGPU, zero cloud, zero backend. Built for dyslexia, ADHD, low vision, and everyone in between.
Agent skill that makes AI replies parseable in one pass — verdict first, prose over fake structure, claims that trace to evidence
HTML to clean Markdown optimized for LLMs. Replaces readability + turndown. One function: content extraction + conversion + token estimation.
OpenClaw web clipper skill: save any web page as clean local Markdown with frontmatter. Dual-engine extraction, Chinese-safe filenames, batch clipping with dedup, output lands in knowledge/clippings/ ready for kb-retriever indexing. 中文:OpenClaw 网页剪藏工具,双引擎正文提取、中文友好、批量去重,输出直通知识库建索引。
Detect complex and unreadable Mermaid diagrams using research-backed metrics and best practices.
Chrome extension that bolds the first few letters of every word so your eyes can skim. Manifest V3, zero dependencies, no build step, nothing tracked.