In the Juno summit, Symantec presented it's perspective on securing Keystone. Security is really a mindset and process. We proposed a layered security approach starting with the process for securing Keystone architecture, followed by securing the environment where Keystone is deployed and configured. Since then we have been implementing those security measures in our production environment. In this talk, we will discuss exactly how we have made our Keystone deployment secure and what we have learnt along the way.
2. The Symantec Team
• Cloud Platform Engineering
– We are building a consolidated cloud platform that provides infrastructure and
platform services for next generation Symantec products and services
• Me
– In Security for over 6 years
– Symantec Insight - Reputation Based Security
– Symantec Data Analytics Platform
– OpenStack Engineer - Keystone
– OpenStack Security Group
– Cop Open Source
Secure Keystone Deployment
2
3. OpenStack Security Group
Secure Keystone Deployment
3
security
notes
Retrieved from http://www.openstack.orgRetrieved from http://docs.openstack.org
4. Secure Keystone Deployment
Why is Keystone security critical?
What is Keystone?
How is Authentication process implemented in
Keystone?
How is Authorization mechanism implemented in
OpenStack?
5. AuthN Overview
Secure Keystone Deployment
5
Cloud User
Cloud User
Identity
(SQL/LDAP)
Keystone
Token (SQL)
Identity
(SQL/LDAP)
Keystone
Token (SQL)
Request sent with
Username and Password
Verify username and
password (hash of
password)
Successful verification
Request metadata for user
tenant relationship
Assignment
(SQL)
Assignment
(SQL)
User tenant relationship
information
Request to generate new
token
Response with new token
Response with token
6. AuthZ Overview
Secure Keystone Deployment
6
Cloud User
Cloud User Keystone
OpenStack
Service
Keystone
OpenStack
Service
Request sent with session
token
Verify session token
Successful verification
Is this token correct?
Does it allow the service
usage?
Service executes
the request
Response with success
7. Secure Keystone Deployment
Why is Keystone security critical?
Does it store/transmit any sensitive information?
What kind of cloud asset does it store?
Is any type of attack possible on Keystone? Can it
bring down the entire cloud?
8. Keystone Security is Critical
Secure Keystone Deployment
8
• Gatekeeper
• Access to OpenStack Cloud
• Assets
• Users
• Passwords
• Tokens
• Roles
• Catalog
• Vulnerable to DoS
Retrieved from http://internet.phillipmartin.info
Retrieved from http://blogs.citypages.com
Retrieved from http://assets.nydailynews.com
9. What was our approach to identifying
key vulnerabilities?
Secure Keystone Deployment
9
10. Security Risks
Secure Keystone Deployment
10
• Global Security Office
Threat Model
Penetration Tests
Traceability Matrix
Retrieved from http://www.technetics.com.au
13. What kind of security deficiencies did
we discover?
Secure Keystone Deployment
13
14. Secure Keystone Deployment
14
Attack: Keystone user credential theft
Attack: Insecure file permissions on
Keystone.conf
Keystone.conf
Attack: Access to cloud admin
privileges for almost free
Attack: Leaking sensitive data in log
messages
Attack: DoS – Authentication chaining
- Havana
Attack: Unauthorized access to MySQL
database
Many more …
17. Mitigate: Secure Communication - SSL
Secure Keystone Deployment
17
Hardware Load Balancer Hardware Load Balancer
Keystone KeystoneKeystone
SSL Client
SSL Server
SSL Client
SSL Server
mod_ssl
35357/SSL 5000/SSL
mod_ssl
35357/SSL 5000/SSL
mod_ssl
35357/SSL 5000/SSL
Public API Admin API
18. Insecure file permissions on Keystone.conf
Secure Keystone Deployment
18
Mitigate:
• Restrict ownership to service user
- chown keystone:keystone /etc/keystone/keystone.conf
• Restrict to read and write by the owner
- chmod 640 /etc/keystone/keystone.conf
hostnameabc
hostnameabc
hostnameabcuser
user
user
19. Access to admin privileges is almost free
Secure Keystone Deployment
19
• Service Token
• Bootstrap Keystone
• Cloud admin privileges
• Register bad service/endpoints
20. Mitigate: Disable Service Token
• Comment out admin_token from /etc/keystone/keystone.conf:
admin_token=e2112effd3ff05b8c88ad14e096e6615
• Remove admin token auth middleware from
/etc/keystone/keystone-paste.ini:
[filter:admin_token_auth]
paste.filter_factory =
keystone.middleware:AdminTokenAuthMiddleware.factory
Secure Keystone Deployment
20
21. Who is the cloud admin now?
Secure Keystone Deployment
21
22. Create Cloud Admin
• Leveraging Keystone Domain
• Before disabling service token:
• Create a domain “cloud_admin_domain”
• Grant “admin” role to appropriate user “Bob Smith”
• Update keystone policy.json file:
• Replace:
"cloud_admin": [["rule:admin_required", "domain_id:admin_domain_id"]],
• With:
"cloud_admin”: [["rule:admin_required”,"domain_id:<cloud_admin_domain_id>"]],
Secure Keystone Deployment
22
23. Leaking Sensitive Information in Log Messages
• Debug mode include plaintext request logging
• Passwords
• Tokens
• Mitigate:
• Disable debug mode in keystone.conf with:
• With debug mode ON, upgrade keystone client:
• python-keystoneclient >= 0.10.1 (OSSN-0024)
Secure Keystone Deployment
23
[DEFAULT]
debug=False
24. Leaking Sensitive Information in Log Messages
Identity API V2 - INFO level logs contains auth tokens (OSSN-0023)
Mitigate:
• Set the log level to WARN in logging.conf:
Secure Keystone Deployment
24
[handler_file]
class = FileHandler
Level = WARN