Skip to content

Two leading newspaper Prothom-Alo and Dailystar news scrubber

Notifications You must be signed in to change notification settings

mnpappo/newspaper-scrubber

Repository files navigation

prothomalo-scrubber.py

a newspaper scrubber for http://prothom-alo.com to collect the images and their corresponding caption.

dailystar-scrubber.py

a newspaper scrubber for http://www.thedailystar.net to collect news info.

installing dependencies

  1. just use pip install -r requirements.txt

parameters - for prothom-alo

  1. set start_date and end_date

parameters - for dailystar

  1. set category and page range

using script

  1. after installing dependencies and setting the parameters just run with python dailystar-scrubber.py or python prothomalo-scrubber.py