This document provides an agenda for a presentation on going beyond automated testing for security assessments. The presentation covers testing methodologies, soft skills needed for manual testing like determination and focus, limitations of automated testing, techniques for finding unknown systems and content, exploiting vulnerabilities, reporting and automation best practices, and recommendations for useful training resources. The goal is to share experiences with external security assessments and how manual testing finds issues missed by automated tools.

1. Beyond Automated Testing

2. Build | Protect | Learn
Agenda
2
• ~$ whoami
• Overview
• Testing Methodologies
• Soft Skills
• Why Go Beyond Automated Testing
• Finding Unknown Stuff
• Vulnerability Exploitation
• Reporting and Automation
• Useful Training and Talks

3. Build | Protect | Learn
~$ whoami
3
• Security Geek
• Pentester @ BreakPoint Labs (0xcc_labs)
• Security Researcher at Primal Security (@PrimalSec)
• Certification Junkie (OSCE, OSCP, etc.)
• Loves Python, CTFs, Pentesting, and Hiking/Camping

4. Build | Protect | Learn
Quick Caveat
4
• I am not a master of all things….but I have acquired a
particular set of skills…

5. Build | Protect | Learn
Overview
5
• Goal: Share my experience with external security assessments
• Motivation: How many of you have heard this?
Is the scan done?
Can you scan us?
• Quick Note: Automated vs. Manual Testing in the context of this talk
Automated Testing: Kicking off a vulnerability scanner (Nessus, Burp,
Acunetix, Webinspect, etc.)
Manual Testing: Everything else you do beyond the scope of the scan

6. Build | Protect | Learn
Testing Methodologies
6
• Having an established testing methodology is an important first step
• You do not need to marry a methodology
• Several great methodologies out there:
Pentesting Execution Standard (PTES)
OWASP Testing Guide (OTG) 4.0
Web Application Hackers Handbook Task Checklist
• Good methodologies should include both Automated and Manual testing

7. Build | Protect | Learn
Soft Skills
7

8. Build | Protect | Learn 8
Determination
• Pentesting is all about failure….and not giving up

9. Build | Protect | Learn 9
Mindset
• Hunt for what automated tools miss

10. Build | Protect | Learn 10
Focus

11. Build | Protect | Learn 11
Confidence

12. Build | Protect | Learn
Automated Testing
12

13. Build | Protect | Learn 13
Automated Testing Will Miss Stuff
• The DHS National NCATS organization reported that 67% of high
impact vulnerabilities required manual testing to enumerate.

14. Build | Protect | Learn 14
Automated Testing Can Break Stuff

15. Build | Protect | Learn 15
Automated Testing Can Take a Long Time

16. Build | Protect | Learn 16
Automated Testing Can Have False Positives
• Burp: Right-Click -> [Send to Repeater] [Request in Browser]

17. Build | Protect | Learn 17
Okay, So Now What?
• So we see just running a scan is not the best solution
• The rest of this talk will demonstrate some things I do beyond
an automated tool to help me find cool stuff

18. Build | Protect | Learn 18
Some Things To Think About
• What technology is in use?
• Ensure you properly map the application
• Enumerate all technology features (File upload, Comments, etc.)
• Enumerate all areas of user input "Injection Points"
• Can you figure what is being done with your input?
• Is your input being presented on the screen? -> XSS
• Is your input calling on stored data? -> SQLi
• Does input generate an action to an external service? -> SSRF
• Does your input call on a local or remote file? -> File Inclusion
• Does your input end up on the file system? -> File Upload

19. Build | Protect | Learn 19
Think About How Input Is Being Used
• Given the following parameters, what would you attempt to do?
http://example-site.com/index.php?redirect=/contact/contact-us.php
http://example-site.com/index.php?file=/app/load.php
http://example-site.com/index.php?name=bob
http://example-site.com/index.php?search=starwars
http://example-site.com/index.php?sql=SELECT * FROM USERS

20. Build | Protect | Learn
Finding Unknown Stuff
20

21. Build | Protect | Learn 21
Finding Unknown Systems
• Companies are normally quite surprised about what is exposed to
the Internet.
• How do you tackle large /8’s, /16’s, how do you even build out this
footprint starting with a company name?
- Shodan + Censys.io
- Domain + IP Research
- Masscan + Nmap
- Whatweb + Wapalayzer
- Google, Bing, etc.
- OSINT: Company Mergers + Acquisitions

22. Build | Protect | Learn 22
Finding Unknown Systems: Recon-ng
• Great tool for helping to automate reconnaissance written by Tim Tomes
• Jason Haddix wrote a script to automate several modules: enumall.sh

23. Build | Protect | Learn 23
Quick Visual: EyeWitness
• EyeWitness is a tool that takes in URLs and creates a report with server
headers + Screen shot of the web GUI
• Extremely useful when facing a large scope

24. Build | Protect | Learn 24
Quick Visual: Masscan Web GUI
• Offensive Security’s Masscan Web GUI is a great way to get a quick
visual of the Masscan results

25. Build | Protect | Learn 25
Don’t Judge a System By It’s IP
• Requesting an application URL by IP might give back different content vs.
the domain
• Keep in mind you can have several applications living on the same IP
• Pointing an automated tool to “http://ip/” may miss a lot of stuff vs.
“http://ip/AppIsHere/”

26. Build | Protect | Learn 26
Can You Find Source Code?
• Can you deploy the technology in a VM and test?
• If you have access to the source code you can enumerate vulnerabilities
more efficiently

27. Build | Protect | Learn 27
Finding Unknown Content
• Unlinked content can be a gold mind of interesting functionality
• Ensure you test for unlinked directories, files, and parameters
• Useful wordlists for brute force content discovery:
- FuzzDB and Raft Lists
- Burp Suite’s Built-in Lists
- SecLists
- My Github
- Robots Disallowed
Tools: Dirbuster, Papator, Burp’s Intruder, Burp’s Content Discovery

28. Build | Protect | Learn
Vulnerability Exploitation
28

29. Build | Protect | Learn 29
Version Specific Vulnerabilities
• Enumerating the technology and version in use go a long way with finding
vulnerabilities (Google + Exploit-db)
• What do I know about the technology and how can I find more
information?

30. Build | Protect | Learn 30
Custom Fuzzing
• FuzzDB, Raft Lists, and SecLists provide great lists for customer fuzzing
• As you start to understand how your input is being leveraged you can
start your fuzzing
• Burp Suite Pro’s Intruder is my go to tool for web application fuzzing

31. Build | Protect | Learn 31
Weak Authentication Mechanism
• Very common finding with web application penetration testing
• Often combines several vulnerabilities:
- Username enumeration (Low) +
- Lack of Automation Controls (Low) +
- Lack of Password Complexity Requirements (Low) =
- Account Compromise (Critical)

32. Build | Protect | Learn 32
Weak Authentication: Username
Enumeration
• Password Reset Features “Email address not found”
• Login Error Messages “Invalid Username”
• Timing for login Attempts: Valid = 0.4 secs Invalid = 15 secs
• User Registration “Username already exists”
• Various error messages, and HTML source
• Contact Us Features “Which Admin do you want to contact?”
• Google Hacking and OSINT
• Document Metadata
• Sometimes the application tells you

33. Build | Protect | Learn 33
Weak Authentication: Automation Controls
• Pull the authentication request up in Burp’s Repeater and try
it a few times
• If you see no sign of automation controls send to Burp’s
Intruder for more aggressive testing
- No account lockout
- No/Weak CAPTCHA
- Main login is strong, but other resources are not
(Mobile Interface, API, etc.)

34. Build | Protect | Learn 34
Weak Authentication: Weak Passwords
• We as humans are bad at passwords…here are some tricks
that work for me:
- Password the same as username
- Variations of “password”: “p@ssw0rd”…
- Month+Year, Season+Year: winter2015…
- Company Name + year
- Keyboard Walks – PW Generator: “!QAZ2wsx”
• Lots of wordlists out there, consider making a targeted
wordlist using CeWL
• Research the targeted user’s interests and build lists around
those interests

35. Build | Protect | Learn 35
File Upload Abuse
• Automated tools do this poorly. They will commonly report
that the functionality is there as an Informational finding
• You should leverage a proxy and look at all the details
associated with the file upload:
- Can you determine where the file is stored?
- What controls are in place (File extension, file type…)
- Can you modify the content-type header?
- Can you add code to a legit file and have it run?
- Can you change the case to bypass filters?
• Insecure file upload -> RCE, Web Shells, XSS, etc.

36. Build | Protect | Learn 36
SMTP Injection
• Contact Us, and Feedback forms are commonly vulnerable
to SMTP Injection
• How excited would you be if the application allowed you to
send Emails as the CEO?

37. Build | Protect | Learn 37
SMTP Injection Cont.

38. Build | Protect | Learn 38
Social Engineering: Phishing
• Lots of ways to do this: XSS, Open Redirect, New Domain, etc.
Step 1: Register a domain that closely matches the target organization:
breakpoint-labs.com vs. breakpoint-lab.com
Step 2: Social Engineering Toolkit (SET) -> clone site
Step 3: Figure out payload(s): Credential Grabber, Unicorn PowerShell
HTA, Java, BeEF Hook, CSRF, Malicious Doc, etc.
Step 4: Send Email modeled off the companies previous communications
(HR, Taxes, Pay, etc.) - Check out Gophish open source phishing
framework.

39. Build | Protect | Learn 39
Social Engineering: Phishing Cont.

40. Build | Protect | Learn 40
File Inclusion to RCE
• File inclusion vulnerabilities can lead to code execution "php
include()".
• Sometimes they are just limited file inclusion "php echo()".
• File inclusion can lead to code execution via LFI or RFI.
• LFI's normally require you to get your input somewhere on
disk then include that resource.
• RFI's are normally easier to exploit as you can point them to
an external resource containing your code.

41. Build | Protect | Learn 41
File Inclusion to RCE: Step 1
• Enumerate an unlinked resource "debug.php" that gives an HTTP 200
OK and blank screen. This is where most automated tools stop.

42. Build | Protect | Learn 42
File Inclusion to RCE: Step 2
• Parameters are fuzzed in an attempt to enumerate inputs "page=" gives
back a different response "Failed opening 'test' for inclusion".

43. Build | Protect | Learn 43
File Inclusion to RCE: Step 3
• Attempt to point the page parameter to local and remote resources and
attempt to execute code on the server.

44. Build | Protect | Learn 44
File Inclusion to RCE: Step 4
• PHP was running as SYSTEM on the vulnerable application. An attacker
could dump password hashes and pivot throughout the organization

45. Build | Protect | Learn 45
Troubleshooting Exploits: Part 1
• Something might be vulnerable, but take some work to successfully
exploit
• Many try to exploit stuff and throw complex payloads. When I am working
on exploiting something, I try to use very simple payloads at first and then
work my way to something more complex
• Webshell Example: Some simple examples, more here

46. Build | Protect | Learn 46
Troubleshooting Exploits: Part 2 - WordPress
• Recent pentest I found a WordPress server
• Could access the login panel - Brute Force is in!
• wpscan found some stuff to play with:
- Avada Theme with Revslider
- Paid Memberships Pro Path Traversal (PHP echo())
- Photo Gallery <=1.2.7 - Unauthenticated Blind SQLi

47. Build | Protect | Learn 47
Troubleshooting Exploitation: Part 2 Cont.
• Brute Force: Had a valid username, but very slow :(

48. Build | Protect | Learn 48
Troubleshooting Exploitation: Part 2 Cont.
• LFI: Paid Memberships Pro Path Traversal: Worked! Sweet
LFI (Grabbed wp-config.php)
• Via PHP echo() so no code execution :(

49. Build | Protect | Learn 49
Troubleshooting Exploitation: Part 2 Cont.
• SQLi: Replicated WP/Plugin in lab environment
• MSF Module worked in lab, caused no impact to system
• Needed to have a valid Photo gallery ID, so I had to dig
around on the site to find it
• Attempted MSF module and it failed :(
• Dug through pcap and saw RSTs from server shortly after
SQLi string

50. Build | Protect | Learn 50
Troubleshooting Exploitation: Part 2 Cont.
• Figured out how to replicate with sqlmap:
• Attempted on the application and still blocked, hmmm....
• I added in a "--delay=4" which delayed my SQLi requests by
4 seconds (bypassing the WAF).

51. Build | Protect | Learn 51
Troubleshooting Exploitation: Part 2 Cont.
• 4 second delay per request, so I am back to this....

52. Build | Protect | Learn 52
Troubleshooting Exploitation: Part 2 Cont.
• Result? Took a long time, but led to full compromise.
• Lots of places I could have stopped. It is always easier to say
something isn't vulnerable.

53. Build | Protect | Learn 53
Remediation Testing
• I could upload a webshell using fileupload.php (client-side
MIME type check).
• Developer fixes the issue with server side checks and names
the resource: fileupload2.php.
• How would you go about testing this?

54. Build | Protect | Learn 54
Remediation Testing Cont.
• I thought "Hmm, I wonder if fileupload.php is still
accessible...."

55. Build | Protect | Learn
Reporting and Automation
55

56. Build | Protect | Learn 56
Reporting
• Markdown is awesome. Common Findings
Database - Check it out

57. Build | Protect | Learn 57
Start the Fire: Learn Python
• I find it a necessary part of my day to day
• Learning a scripting language is not that hard
• It is like starting a friction fire

58. Build | Protect | Learn 58
Python

59. Build | Protect | Learn 59
Python: Shodan

60. Build | Protect | Learn 60
Elasticsearch Python + Kibana (EPyK)

61. Build | Protect | Learn 61
Useful Resources
• CTFs: Vulnhub, Past CTF Writeups, Pentester Lab
• Training: Offensive Security
• Book: Web Application Hackers Handbook
• Book: Black Hat Python
• Talk: How to Shot Web - Jason Haddix
• Talk: How to be an InfoSec Geek - Primal Security
• Talk: File in the hole! - Soroush Dalili
• Talk: Exploiting Deserialization Vulnerabilities in Java -
Matthias Kaiser
• Talk: Polyglot Payloads in Practice - Marcus Niemietz
• Talk: Running Away From Security - Micah Hoffman

62. Build | Protect | Learn 62
Conclusion