Search for urls of (sub)domains using the web archive database
This tool is mainly based on waybackurls by tomnomnom.
The biggest difference between this tool and the one on which it is based is the ability to filter results based on status code, Mime-Type and file extension, which allows obtaining more relevant results within the context of a specific target.
In the current version, you can query results from the Wayback Machine, AlienVault and IntelligenceX (needs an API key).
waybackurls.pl [option(s)] <domain_1> <domain_2> ... <domain_n>
Options:
-h, --help Show this help message and exit
-v, --version Show program version and exit
-s, --silent Do not show status messages
-j, --json Output the urls in JSON format
-e, --extensions Comma-separated list of extensions to filter by
-m, --mime-types Comma-separated list of mime-types to filter by
-c, --status-codes Comma-separated list of status codes to filter by
-i, --input-file Read list of domains from file
-d, --subdomains Also search for subdomain urls (default)
-o, --output-file Save output to a file, don't print to stdout
-E, --exclude-exts Comma-separated list of extensions to be ignored
-M, --exclude-types Comma-separated list of mime-types to be ignored
-C, --exclude-codes Comma-separated list of status codes to be ignored
-g, --images Include image links
-k, --credentials JSON file with credentials/API keys for sources
--no-subdomains Don't search for subdomain urls
--no-images Don't include image links (default)
user@host:~$ waybackurls.pl -o output.txt -E css,jpg,jpeg,js,pdf,doc,docx -C 404 targetsite.com
user@host:~$ waybackurls.pl --no-subdomains -e php,txt,bkp -c 200 --json -i targets.txt
user@host:~$ waybackurls.pl -o output.txt -C 404,403,500 -i - < targets.txt
user@host:~$ waybackurls.pl -M text/html,image/jpeg,text/css targetsite.com --json > output.txt
Lucas V. Araujo – [email protected]
Distributed under the GNU GPL license. See LICENSE for more information.
- Fork it (https://github.com/LvMalware/waybackurls.pl/fork)
- Create your feature branch (
git checkout -b feature/fooBar) - Commit your changes (
git commit -am 'Add some fooBar') - Push to the branch (
git push origin feature/fooBar) - Create a new Pull Request