Presentation at ISSA 10th Annual InfoSec Summit about using OSINT for attack and defense

1. OSINT Basics for Attack and
Defense
By Andrew McNicol
&
Matt Foreman

2. Matt Foreman
@s7foreman
• Security Consultant
• I have some certifications, they are made of
letters
• I do Penetration Testing, Security
Assessments, and sometimes what I call
research….

3. Andrew McNicol
• Security consultant
• Part-time beard developer
try:
I enjoy writing error-free Python with Google and
stackoverflow
except:pass
• I do both offensive and defensive stuff

4. We didn’t do it
• We are not lawyers or giving you legal advice
• We are not giving you permission or
authorizing you in any way to do anything
ever
• In fact don’t do anything ever

5. What is OSINT?
• OSINT has been formally defined this way…
Open-source intelligence (OSINT) is intelligence
collected from publicly available sources. In the
intelligence community (IC), the term "open" refers
to overt, publicly available sources (as opposed to
covert or clandestine sources); it is not related to
open-source software or public intelligence.
• Also check out the PTES , tons of great info
http://www.pentest-standard.org

6. This talk
• OSINT has been discussed from a high level to
very deep dives in past talks by others
• This talk might cover some offensive methods
of OSINT you might have seen before, but we
also want to cover some defensive uses
levering the same/similar OSINT tools that we
see mentioned less often

7. Shodan
• Allows users to search for publicly connected
internet devices that have been seen by Shodan
• Routers
• Servers
• Firewalls and other Security Devices
• SCADA or other Control Systems…
– This data can be searched for by IP/CIDR combo
– Open ports seen by Shodan
– Hostname, OS, Geo-Location, etc…
– Server Response

8. Shodan for Attackers
• So it’s fairly easy to see how this can be useful to attackers.
• This simple query will show everything seen by Shodan in the
US (MERICA!) with TCP 445 open to the internet…

9. Shodan for Attackers
• Hopefully this an uncommon thing you would
see on engagements but you get the idea
• Without sending a packet to the end
customer/target we can identify some of their
external infrastructure and at one point what
was there

10. Shodan for Defenders
• Understanding what information is available in Shodan
can help defenders too
• Shodan can be leveraged to fingerprint C2 servers
Attackers sometimes make mistakes in server responses
These unique strings could help enumerate additional
C2 servers
• Can be leveraged to see server responses without actually
making a request

11. Shodan for Defenders
• Example of searching for “Apach” and “202”:

12. Maltego by Paterva
• Commercially licensed
• Runs on multiple different OS
• Can integrate API’s from many different Sources
• Great for stalking people! <note> remove this its creepy </note>
• Uses various “transforms” to gather and hopefully correlate
data between various sources

13. Maltego for Attackers
• Here is a simple graph output of a Maltego search

14. Maltego for Attackers
• From this point we can start mapping out infrastructure,
people, known aliases, social media, etc..
• All can be valuable information for attackers depending on the
goal…..and the scope

15. Maltego for Attackers
• This doesn’t come with out false positives, but after enough
digging you could end out with a map like this….

16. Maltego for Attackers
• There are many add-ons to Maltego, including one for Shodan

17. Maltego for Defenders
• Maltego can be a great way to perform link
analysis with indicators of compromise
• Malformity adds a lot of malware functionality:

18. Maltego for Defenders
• Example of running various transforms and
enumerating more information from the hash
value (mutex, C2, other samples, etc.):

19. Have you seen this thing, Google?
• So we have all seen Google hacking before and probably the
most notable example is the Google Hacking Database or
GHDB – Originally created by Johnny Long
• And attackers obviously still use these methods today
• Here is a very simple Google search for Juniper’s SSL VPN
login page…I'm sure this was searched during the Heartbleed
craziness #heartbleedcyberAPT

20. Google for Attackers
• This search looks for a WordPress plugin that is vulnerable to an open
redirect. About 235 results came back with modifying the query much
• exploit-db/exploits/18350/

21. Google for Attackers
• This search looks for a search looks for open Cisco Routers, finding over 15
million results
• And here we see one of the results has an open command window
running with level 15 privileges

22. Google for Attackers
• People tend to reuse usernames, handles, etc...
• So if we can find some target IT personnel on a resource like
Linkedin, Facebook, or Twitter and do some searching for
common handles they like to use, sometimes you end up with
system administrators posting complete firewall
configurations onto public websites….

23. Google for Attackers
• A little more digging on the person who shall not be named
showed that his/her username was reused on multiple sites
and one tech-help forum, which had public profiles
• This included corporate email used to register, full name, and
location
• Some users of these forums include their corporate email
signature and tagline (giving us more terms to include in
targeted searches) “We are the leader in
CyberDongleWidgets, and we know it”
• Try a Google search for some of the popular tech forums…
site:http://www.tek-tips.com/viewthread.cfm? /etc/shadow

24. Google for Defenders
• Knowing your organizations exposure online can help you
defend
• Google searching indicators from malware can save you time:
• Hashes, Strings, Domains/IPs, persistence mechanisms, mutexes, etc.

25. Google for Defenders
• Humans lie, and humans are creatures of habit:
• Fake Domain Registration Information (Emails, Phone numbers,
Addresses, etc.)

26. Online Data Dumps
• Monitoring data dumps from the target or 2rd parties can be provide a
treasure trove of information for the attacker (Usernames, passwords,
etc.)
• From a defensive standpoint, monitoring these data dumps for your
organization can allow you to take appropriate action

27. Linkedin
• If Social Engineering or Phishing is in scope you can
use this data to find targets
• Existing personnel to enumerate technologies and
partner relationships or company updates listing new
projects or acquisitions
• New employees are often good targets
– Minimal Training
– Don’t know IT staff on a first name basis
– Sometimes have default AD credentials (changem3)

28. Additional Search Resources
• Don’t put all your operators in one basket try
multiple resources
• Yandex (Russian search engine, many
operators to filter out data)
• Bing (similar to google operators but has “ip:”
option)
• Nerdydata (Indexes Code snippets, meta tags,
HTML, and JavaScript)
• Searchdiggity & FOCA (Can use API’s)

29. Additional Search Items
• More things to search for…
o Business Partners
o Vendor Relationships
o Are certain functions outsourced? Like HR, the
helpdesk, etc…

30. Wireless Communications
• Openbmap.org
• wigle.net
Find previously discovered wireless in the area of your target

31. Researching IPs and Domains
• Link analysis between IPs, Domains, and Name Servers can
help map out additional hostile infrastructure:
• Robtex, iplist.net, nslist.net, pop.dnstree.com, webboar.com,
centralops.net, etc.

32. Researching IPs and Domains
• Given a hostile Domain/IP ask yourself:
• Any fake registration information?
• What other domains point to IP?
• What other domains leverage that name server?
• What domains point to IPs around the hostile?
• Additional subdomains (skills.cnndaily.com, jobs.cnndaily.com)
• Resolve back to non-routable IP space (Loopback, bogon)
• Domains that look right, but are slightly off:
• update.macfee.com
• mirosoft.supportca.com

33. Researching IPs/Domains
• Passive DNS can allow you to track changes to domains overtime:
•Virustotal, DNSDB, Edv-consulting
• Hostile infrastructure gets reused:
– Can help enumerate additional infrastructure
– Can assist with attribution

34. Automation
• Automating tasks is key – especially since you may have to do
something thousands of times
• Use Case: Whois automation with Team Cymru's Python whois
module – 1000s of lookups within seconds:

35. Automation
• Creating and parsing web requests via a scripting
language can save a lot of time
• Use Case: Looking up IPs via iplist.net with Python

36. OPSEC and OSINT
• As you start digging on the line be aware of the information you
are exposing about yourself or your organization
• Many ways to control what information you give to the Internet:
• Google Cache
• Firefox Plugins:
• Foxyproxy + ssh tunneling
• User Agent Switcher
• NoScript
• Refcontrol
• Tamperdata
• Tor, VPNs, Proxy services etc.
• Separate non-attrib ISP link

37. Recon-ng for Attackers
• Started by Tim Tomes (@LaNMaSteR53)
• Many contributors
• Menu feels similar to msfconsole
• Way too many great features to list today
• Can be a one-stop-shop to gather a ton of data
recon/hosts/gather/http/web/bing_domain

38. Recon-ng for Attackers
• This above example is querying searchdns.netcraft.com for additional
hosts.
• Also its worth looking at these for DNS info as well. These are querying an
DNS server of your choice instead of searching
recon/hosts/gather/dns/reverse_resolve
recon/hosts/gather/dns/brute_hosts

39. Recon-ng for Attackers
• Search xssed.com for past entries. Can be useful for the later phases of
attack. Keep in mind the dates on some of the entries

40. Recon-ng for Defense
• Malwaredomainlist.com Module:

41. Recon-ng for Defense
• Hostname Resolver Module:

42. Malware Sandboxes
• Many Internet resources exist to analyze malicious samples:
Virustotal
Malwr.com
ThreatExpert.com
CWSandbox
• These are very useful, but keep in mind that they often make some
of the data public
• Adversaries can monitor these online resources just like defenders
• Uploading a sample could let the adversary know you found their malware
• Cuckoo sandbox can be a free solution

43. Malware Sandboxes
• Cuckoo Sandbox is a free alternative to standup a
local malware sandbox:

44. Public doc’s and metadata
• Strings, Exiftool, etc..
• Pull down public documents (pdf, doc, ppt)
• The content itself could be as useful as
metadata
• Sometimes IT creates “how-to” guides
disclosing technology and settings used
• Metadata (What version of Office, Adobe,
etc…) When was it created and so on.

45. Metadata Defenders
• Can be used to extract useful strings for further research (C2,
language settings, timestamps, etc.):
– Strings, pescanner.py, Exiftool, CFF Explorer etc.
• Metadata can be used to link attacks together, and is
commonly used to name malware
• Pescanner.py:

46. In Summary
• OSINT is important and still gets overlooked by
attackers and defenders
• We hope that you found this talk useful
• This talk and the Python tools mentioned will
be available here shortly after the conference:
– www.primalsecurity.net