commoncrawl
Common Crawl provides an archive of webpages going back to 2007.
Built from commoncrawl's public GitHub profile. Not on Take My Repo yet, so there are no posts or updates here.
Statistics of Common Crawl monthly archives mined from URL index files
The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)
A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine
Crowd-sourced lists of urls to help Common Crawl crawl under-resourced languages. See https://github.com/commoncrawl/web-languages-code/ for the code
A library of examples showing how to use the Common Crawl corpus (2008-2012, ARC format)
Graph embedding built using Common Crawl's host-level hyperlink graph