Recon — my way.

Sahil Ahamad

·

Follow

10 min read

·

Jun 6, 2018

--

13

Listen

Share

A detailed blog post on my reconnaissance processes for web applications security testing. I always wanted to write about this subject being asked by many friends, community members, etc. but I hooked wasting a lot of hours on a Meme Channel & The Big Bang Theory TV Series.

UPDATE: I created a GitHub repository with tools from this post and personal installation guide.

yes, I did !! 🌋

Recently, some new members of the InfoSec community asked me to share my recon process. Hence, I decided to begin writing this blog and tried to include such tools and services which helps me a lot while testing and will help the readers too, for sure.

Summary

1. Introduction
2. A tool I modified.
3. Visual recon
4. More Assets — More findings — More win.
5. Data Storage Buckets.
6. Github for Recon.
7. Read Every JS
8. Archive
9. Continuous Recon
10. Extra points for recon.

1. Introduction

Whenever I get an invitation for a new program or I want to test a target, I start my recon process by using the Knockpy.

Why I use knockpy Initially? It provides me with a quick overlook of the subdomains with a response code.

Once, I found a subdomain takeover bug within 2 mins.

I ran knockpy on an old program’s in-scope asset with almost 150 bugs resolved on HackerOne. Quickly saw 404 page pointed to AWS S3 bucket and bucket were available to create. Hence, with no delay, I created the new AWS S3 bucket and uploaded a text file with the encoded filename and reported the bug and guess what? I got the bounty that too within 15 mins.

Lesson: knockpy = quick win

2. A Tool I Modified.

I use a customized tool for subdomain reconnaissance.

I used a resolver tool by Malvinsh and customized it.

Malvinsh resolver tool runs as.

Malvinsh tool is doing two simple processes.

• Getting the IP address of the domain/subdomain from the wordlist using HOST.
• Performing the Nmap scan.

Using its logic I created two scripts.

• Subdomain.rb
• Recon.rb

Caution: Do not use these scripts on programs where automatic testing is forbidden and also on the targets on which you are not allowed to.

Subdomain.rb

Subdomain.rb is a lightweight script to automate tools for subdomain finding and it’s damn flexible — more tools can be added easily.

Subfinder and sublist3r results sometime overlap. I tried to run both tools separately on a particular asset and got some distinct results — that’s why I kept both tools in this script.

The script is using the following tools to get the subdomains data.

• Subfinder

• Censys subdomain finder.

• Knockpy

• Sublist3r

• Aquatone

subdomain.rb gist.

Usage: ruby subdomain.rb domain.com

I, usually, create a file and add all subdomains from the above outputs. Then, I use the sort command to delete all duplicate/overlapped subdomains from the file.

sort wordlist | uniq

I pass the final unique subdomains file to the recon.rb

Resolve.rb

Recon.rb is another lightweight script and it is also flexible hence, more tools can be added easily.Tools included in recon.rb

• Host : Resolve the subdomain.
• Nmap : Perform Port scan on subdomain

• AWS CLI — In the script AWS, CLI used test subdomains are connected to AWS bucket or not also checks for list permission. (Can be customized to test for write permission files.)

• Dirsearch — Search for directories with default wordlist and all (*) extensions.

recon.rb gist.

Usage: ruby recon.rb wordlist

A tool with Above functionality in the organized fashion.

Lazyrecon

Interesting blog posts

3. Visual Recon

I use the previously generated wordlist from subdomian.rb for visual recon.

I generally use the following two tools.

• WebScreenshot

• Lazyshot

Interesting blog posts

4. More Assets — More findings — More win.

I go for this section after reporting 2–3 issues in a particular program and wait for the response, if I find program interesting then I try to collect as much information as I can about the target using the following services.

• Censys

Tool for Censys:

• Shodan

• ViewDNS — Reverse Whois Lookup.

Get the whois information of target using whois command or use an online tool.

whois domain.com

If the company is not using Domain Privacy Service,

you will find the host-masters email address then use that email to find other domains registered on same email address using Reverse Whois Lookup. Targets Registered legal name also can be used.

• IP range Crawl

• AltDNS

• Nmap Subdomain finding.
• Content-Security-Policy (CSP)

Tools

• More targets with Burp Suite by Jason Haddix

Jason Haddix Tweet about the method

• Domain Analyzer

• Domain Profiler

• VHost Scan

• ThreatCrowd

• Visual Site Mapper

• Certificate Transparency
• Google Transparency Report

• Certsspotter

https://certspotter.com/api/v0/certs?domain=hackerone.com

• CertDB

• Crt.sh —

https://crt.sh/?q=%25domain.com

• Facebook Certificate Transparency Monitoring Subscriptions.

Interesting blog posts & tools taken from

5. Data Storage Buckets.

Common Places to find Data Storage buckets.

• Github
• Javascript files
• CSP Headers
• Archive crawl
• Pastebin

Tip: If a bucket is returning access denied. Try to search it on Google. There are good chances that the team has recently changed the permissions of the bucket and specific files have been indexed by Google (which are available with read permission).

If the application has file uploading functionality, then try to capture the file uploading request and see where the file is being uploaded. Sometimes you will find AWS or other data storage buckets, which cannot be detected by any other method.

If you find the bucket like upload-user-content-target-prod — try to change the prod to dev, staging, sandbox, etc.

• AWS CLI — AWS CLI is useful for verifying or testing the permissions of the AWS S3 buckets, Creating Buckets and Read other buckets data. AWS Account needed to use CLI.

• Bucket Finder — Great Tool for finding buckets using subdomains wordlist, can be integrated into recon.rb script but I don’t always use it.

• LazyS3 — LazyS3 is an another tool which I use almost frequently to find the staging, sandboxed, dev and production buckets.

• Slurp: Slurp great tool for AWS Buckets Recon. (Deleted After Microsoft Purchased Github I guess). Weird!

https://github.com/bbb31/slurp

• S3 Bucket Finder — Another good tool for AWS S3 buckets.

Interesting blog post

6. Github For Recon.

Github is extremely helpful in finding Sensitive information regarding the targets. Access-keys, password, open endings, s3 buckets, backup files, etc. can be found on public GitHub repositories.

Interesting blog post

7. Read every JS.

Sometimes, Javascript files contain sensitive information including various secrets or hardcoded tokens. It’s always worth to examine JS files manually.

I have found the following things in Javascript.

• AWS or Other services Access keys
• AWS S3 buckets or other data storage buckets with read/write permissions.
• Open backup sql database endpoints
• Open endpoints of internal services.

The more Javascript files you read, the more chances will be of win.

Tools

I generally like to read the javascript code manually with the help of JSBeautifier.

The following tools are useful:

LinkFinder

JSParser — Another great tool by Behrouz Sadeghipour

Interesting blog post

8. Archive

Searching for the targets webpages in waybackmachine, the following things can be found.

• Old and abandoned JS files.
• Old API endpoints.
• Abandoned CDN’s Endpoints.
• Abandoned Subdomains.
• Dev & staging endpoint with juicy info in source code comments.

If you are getting 403 on a page, you can also search that 403 pages of targets in way back machine sometimes, you will find them open with helpful information.

Tool: Waybackurl

9. Continuous Recon.

• The most essential thing in continuous recon is handling of the recon data for the future uses, for this, SecurityEscape created a tool known as Swiftness. I use Swiftness to hold all of my recon data for every target.

• I use the reminder to revisiting the targets and perform recon regularly. (Every month)

10. Extra points for recon.

Note: I have provided some data in gist because I regularly update them and those updates will be automatically available here.

Best of luck for all of your future infosec things.

If you have questions and anything about the post you want to ask me, Please contact me via twitter/fb. I’ll have my DM open.

Feedbacks and edits are welcome

Twitter Facebook

Until next time!

If you like my blog posts and my work, Please consider checking out my “Buy me a coffee” page