TLS/SSL is ubiquitous nowadays and is widely used to protect traffic on the Internet. Although TLS/SSL originated to protect web sites, it is now also being used for email, web services and VPNs among others. Yet, despite its widespread adoption, it is still difficult to keep track of the constantly emerging vulnerabilities that affect it. If not monitored, certificates can expire and new attacks are discovered even for mainstream ciphers and hashing algorithms. Add to that accidental server misconfigurations, unpatched systems, and flawed implementations, and the task of keeping systems current can become daunting. To make matters worse, commonly-used server configurations can be deprecated in major browsers/clients, breaking backwards-compatibility, providing a worse user experience for some, and leaving product maintainers scrambling to update certificates and settings quickly. For all these reasons, systems need constant vigilance to keep them up to date. This session will examine some steps which can be taken to keep up with the constantly changing landscape and demonstrate a few tools which can help manage and automate necessary updates to TLS/SSL endpoints.
This deck was originally presented at LASCON 2015: https://lascon2015.sched.org/event/4D3G/managing-certificates-and-tls-endpoints
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Managing Certificates and TLS Endpoints
1. Managing Certificates and TLS
Endpoints
Charles Neill & Henry Yamauchi
Security Engineering
Rackspace Hosting
2. • Charles Neill & Henry Yamauchi of the Security
Engineering team at Rackspace
• Tasked with managing a large number of TLS/SSL
endpoints
• Since Heartbleed, everyone's eyes are on TLS/SSL
• Many headaches later, we now have a tool called
TLSManager to help us out! (More on that later)
www.rackspace.com 2
What's this all about?
3. • Wire encryption protocol
• Provides authentication, integrity and confidentiality
• Based on Public Key Infrastructure (PKI)
• SSL originally designed at Netscape for eCommerce in the early-mid 1990’s
– SSLv1 was never released, leaving us with SSLv2 and v3 + TLSv1.0, 1.1, and 1.2 today
• Widely deployed on a variety of devices (smart phones, laptops, servers,
appliances, etc.)
• Deceptively simple to get started with (but can be much harder to maintain)
www.rackspace.com 3
What is TLS/SSL?
4. • Time is fast approaching where TLS/SSL will be mandatory
• Browsers beginning to warn users about sites not using TLS/SSL
Firefox
Chrome
• Encryption overhead, cost are no longer valid excuses
– Initiatives like Let's Encrypt (letsencrypt.org) are aiming to provide a free SSL certificate
authority very soon to provide free certificates for all
• Despite becoming easier/cheaper to set up, TLS/SSL can't just be turned on
and left alone in most cases, as we'll see…
www.rackspace.com 4
TLS adoption is increasing
5. •Email
•IRC
•APIs and web services
•VPN
•Appliances (load balancers, proxies, etc)
•Smart phone apps
www.rackspace.com 5
TLS/SSL not just for web servers
7. • Multiple protocol versions (SSLv2, v3, TLSv1, v1.1, v1.2)
• Certificates expire at different times
• Server misconfiguration, multiple servers for single domain
• Outdated systems
• Large number of endpoints to keep track of
www.rackspace.com 7
Issues managing TLS/SSL:
8. • Intermediate certs expire
• Server certificates can accidentally get checked into
version control and have to be revoked
• Lax or compromised CAs may have to reissue their certs
(e.g. Comodo attack)
• Use of self-signed certificates
www.rackspace.com 8
Also:
9. • New SSL vulnerabilities are discovered (e.g. POODLE)
• Bugs in encryption software like OpenSSL or LibreSSL (e.g.
Heartbleed)
• Regulation compliance (e.g. PCI-DSS 3.1 will disallow
TLSv1 in 2016)
• SHA1 in certs will be flagged insecure in Jan 1st, 2017
(browser warnings now)
www.rackspace.com 9
Oh, and don't forget:
12. • When first connecting, client and server engage in negotiation
• Client initially sends the cipher suites it supports
• Server then (ideally) selects the strongest cipher that both supports
• Server should be configured with only strong ciphers
• There's a catch: the client essentially gets to decide what cipher is used by
controlling the ciphers that they send to the server. In some man-in-the-middle
attacks, the attacker can potentially downgrade ciphers in this way (e.g.
LOGJAM).
www.rackspace.com 12
TLS cipher negotiation
13. • Combination of encryption, MAC and key exchange
algorithms
• Available suites depend on the SSL/TLS version
• TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
^ ^ ^ ^
| | | |
| | | MAC hash algo
| | Symmetric key attrs (algo, key size, mode)
| Authentication algo
Key exchange algo
www.rackspace.com 13
Cipher suites
16. • Avoid DES, RC4, MD5, 3DES key length < 128, export ciphers
• Prefer AES, GCM, SHA2, TLSv1.2, OpenSSL >= 1.0.1p / 1.0.2d
• Not all permutations of these are supported (see: cipher suites)
• Sometimes you are dependent on vendor support (e.g. hardware load
balancers)
• Some servers deliberately configured with weak ciphers to support old
browsers (IE6 still in use)
• Cert expiration is just one reason to replace them (other reasons: use of SHA1,
weak key sizes, cert revocation, CA compromises, etc)
www.rackspace.com 16
Server configuration
17. Example nginx configuration
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-
SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-
SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-
AES256-SHA;
ssl_prefer_server_ciphers on;
Example Apache configuration
SSLProtocol All -SSLv2 -SSLv3
SSLCipherSuite HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
More in-depth examples can be found on https://cipherli.st/
www.rackspace.com 17
Configuration examples
18. • Certificate ordered/installed has too short a key (< 1024 bits)
• Full certificate chain (CA bundle) not installed
• Preferred cipher not set correctly
• ‘Honor cipher order’ feature not turned on
• SSL engine/library outdated
• Server not sufficiently protected (so private key is not protected)
• Servers in load balance group configured inconsistently
www.rackspace.com 18
Other server configuration issues
19. • Protects past conversations even if server private key is
compromised
• Uses ephemeral keys in key agreement protocols
• ECDHE_ECDSA, DHE_RSA
www.rackspace.com 19
PFS (Perfect Forward Secrecy)
20. • Server configuration to only accept secure connections for this site
• Server initially sends Strict-Transport-Security header to browser
• Browser makes a note that site should only be accessed with HTTPS
• If an attempt is made to connect to port 80, force user to reconnect via
HTTPS (port 443 by default)
• Particularly useful if sites mix secure and unsecure resources
www.rackspace.com 20
STS / HSTS
22. • Heartbleed (leaks random memory in plain text. Fixed in OpenSSL 1.0.1g)
• Logjam (force export ciphers) / WeakDH (weak/reused Diffie-Hellman params)
• BEAST (browser issue but can be mitigated server side. TLSv1 and earlier)
• CRIME (TLSv1 compression)
• FREAK (export ciphers)
• POODLE (affects SSLv3)
www.rackspace.com 22
Some recent vulnerabilities
23. • Problem: MITM can force client and server to agree to use a lower protocol
version than what they actually support
• Note: The above is possible because SSL/TLS handshake is in cleartext
(session key has not been agreed to yet)
• Solution: Browser include a dummy cipher to tell the server to check whether its
highest protocol version is greater than ClientHello’s. If so, abort
communication.
• Fake ciphers (x00FF)
– EMPTY-RENEGOTIATION-INFO-SCSV
– TLS_FALLBACK_SCSV
* Signaling Cipher Suite Value
www.rackspace.com 23
TLS downgrade attack protection (SCSV*)
24. • Ciphers allowed by the US Government for the export market
• Devised in the 1990’s. Laws restricting encryption export are no longer
in force.
• TLS 1.2 no longer includes them
• Export ciphers use short keys (40-56 bit), making them insecure
• Identified by the presence of EXP or EXPORT in their name
www.rackspace.com 24
Export ciphers
26. • Just because an endpoint is acceptable today does not
mean it will remain so tomorrow since:
–Certificates expire
–Servers are reconfigured
–New vulnerabilities are uncovered
–Increasing computing power can make old
standards/assumptions obsolete (e.g. SHA1)
• This means you need to be constantly monitoring your
endpoints
www.rackspace.com 26
In summary
29. • OpenSSL's "s_client" is useful for debugging SSL. On most
Linux/UNIX systems, you can call it like so:
openssl s_client [-debug] –host host.com –port 443
• The "-debug" flag will give you more information about the
actual connection, showing a hex dump of the handshake
and negotiation process
• Doesn't tell you much about vulnerabilities, but gives you
the information you need if you already know what you're
looking for
www.rackspace.com 29
OpenSSL's s_client
30. From the testssl.sh site:
"testssl.sh is a free command line tool which checks a server's
service on any port for the support of TLS/SSL ciphers,
protocols as well as recent cryptographic flaws and more."
www.rackspace.com 30
testssl.sh
31. • Reports SSL/TLS issues such as:
– SSL/TLS version
– Weak protocols, ciphers and hashes
– Known vulnerabilities
• Uses an easy to understand letter rating system
• Scoring is calculated by assigning weights to:
– Protocol support (30%)
– Key exchange support (30%)
– Cipher strength (40%)
• Can be accessed through API
– Official API client here: https://github.com/ssllabs/ssllabs-scan
www.rackspace.com 31
Qualys SSL Labs
32. Score Meaning
A+ Domains with no warnings and use HSTS and TLS_FALLBACK_SCSV
A Domains with 2048 bit keys, at least TLS 1.2 support and no warnings
A- Domains with 2048 bit keys, at least TLS 1.2 support but with some warnings
B
Uses SSL 3.0, RC4, missing intermediate certificates, vulnerable to BEAST or
CRIME, NS >= 65
C Domain is vulnerable to POODLE, NS >= 50
D Numeric Score >= 35
E Numeric Score >= 20
M Domain and cert host mismatch
T Certificate is not trusted
F
Uses MD5, keys with length < 1024, or SSLv3 is the strongest protocol
supported
www.rackspace.com 32
Qualys letter rating system
34. • https://cipherli.st
• Collection of good configurations for TLS/SSL, covering
several different servers (Apache, nginx, HAProxy, Postfix,
OpenSSH …)
• Useful for making recommendations to product teams for
software you might not have configured before
• Simple copy-and-paste format takes headaches out of
implementers making decisions about certain ciphers or
other settings (though you should vet these configurations
based on what clients you want/need to support!)
www.rackspace.com 34
Cipherli.st
35. • Hosts several endpoints with broken SSL configurations
• Great resource for vetting SSL verification code / test
scripts
• Examples:
–https://expired.badssl.com/
–https://self-signed.badssl.com/
• Many, many more!
www.rackspace.com 35
badssl.com
38. • Automatically scans daily all endpoints in its database
• Uses Qualys Labs SSL tool (https://www.ssllabs.com/ssltest)
• Reports SSL vulnerabilities
• Proactively warns of any certificates about to expire
• Emails reports to interested parties
• Keeps a history of the results
www.rackspace.com 38
Features
41. • On Github: https://github.com/rackerlabs/TLSManager
OR http://bit.ly/1MUQiss
• License: Apache 2.0
• Pull requests most welcome!
www.rackspace.com 41
Go get it!
48. • SSL/TLS management is more involved than it looks at first
• Lots of moving parts
• A constantly changing landscape: a server rated A+ today can be
downgraded tomorrow if:
– Certs expire
– New cipher vulnerabilities are uncovered
– Bugs in SSL/TLS libraries are reported
– Server is misconfigured
– Cert or CA compromise
• It's helpful to monitor SSL endpoints in an automated way to keep from
missing any issues
www.rackspace.com 48
Closing thoughts
50. • Group that focuses on the security of one of the largest
open source projects in the world!
• IRC channel: #openstack-security on Freenode
• Weekly meetings: #openstack-meeting-alt @ 1700 UTC
Thursdays
• Mailing list: openstack-dev ML with [Security] tag
www.rackspace.com 50
OpenStack Security Project