Skip to content

Recursively crawls and scrapes a Blogger site to archive post content

Notifications You must be signed in to change notification settings

harr1424/scrape_blogger

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 

Repository files navigation

scrape_blogger

Usage: scrape_blogger [OPTIONS]

Options:
  -t, --threads <THREADS>  Sets the number of threads to use when scraping all post links [default: 4]
  -r, --recent-only        Scrapes only recent posts from the blog homepage without clicking 'Older Posts'
  -h, --help               Print help
  -V, --version            Print version

Recurisvely crawl and scrape a specific Blogger site in order to archive post content. This project may not generalize well to all Blogger sites. It is hardcoded to work with a specific site, but the source code may be modified to work with any English Blogger site where the site's homepage has a link to older posts.

About

Recursively crawls and scrapes a Blogger site to archive post content

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages