firecrawl
Knowledge infrastructure to make your agents smarter. Search, scrape and 550+ data providers through one API.
- 204k
- TypeScript
- AGPL-3.0
- 4 h ago
Knowledge infrastructure to make your agents smarter. Search, scrape and 550+ data providers through one API.
A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/网页爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
A Simple and Universal Swarm Intelligence Engine, Predicting Anything. 简洁通用的群体智能引擎,预测万物
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
The 30 Days of Python programming challenge is a step-by-step guide to learn the Python programming language in 30 days. This challenge may take more than 100 days. Follow your own pace. These videos may help too: https://www.youtube.com/channel/UC7PNRuno1rzYPb1xLa4yktw
Scrapy, a fast high-level web crawling & scraping framework for Python.
Rich is a Python library for rich text and beautiful formatting in the terminal.
一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、微信读书、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )
| Name | Stars | License | Language | Last change | In common | Compare |
|---|---|---|---|---|---|---|
| rieuse/learnPython | 648 | None declared | Python | 6 y ago | ||
| firecrawl/firecrawl | 204k | AGPL-3.0 | TypeScript | 4 h ago | crawler | Compare |
| sherlock-project/sherlock | 115k | MIT | Python | 1 d ago | python3 | Compare |
| NaiboWang/EasySpider | 45k | AGPL-3.0 | JavaScript | 4 d ago | crawler, spider | Compare |
| D4Vinci/Scrapling | 104k | BSD-3-Clause | Python | 3 d ago | crawler | Compare |
| 666ghj/MiroFish | 86k | AGPL-3.0 | Python | 5 wk ago | python3 | Compare |
| gocolly/colly | 26k | Apache-2.0 | Go | 5 wk ago | crawler, spider | Compare |
| unclecode/crawl4ai | 88k | Apache-2.0 | Python | 15 h ago | crawler | Compare |
| Asabeneh/30-Days-Of-Python | 79k | None declared | Python | 7 wk ago | python3 | Compare |
| jhao104/proxy_pool | 24k | MIT | Python | 5 mo ago | crawler, spider | Compare |
| scrapy/scrapy | 65k | BSD-3-Clause | Python | 2 d ago | crawler | Compare |
| Textualize/rich | 58k | MIT | Python | 7 d ago | python3 | Compare |
| shengqiangzhang/examples-of-web-crawlers | 15k | MIT | HTML | 17 mo ago | crawler, spider | Compare |