Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Transparent Data Encryption in PostgreSQL and Integration with Key Management Service

1 905 vues

Publié le

Talk at FOSDEM PGDAY 2019

Publié dans : Logiciels
  • Soyez le premier à commenter

Transparent Data Encryption in PostgreSQL and Integration with Key Management Service

  1. 1. Copyright©2019 NTT Corp. All Rights Reserved. ,
  2. 2. Copyright©2019 NTT Corp. All Rights Reserved. • Database and Threats • Data at rest Encryption • Transparent data encryption • Transparent data encryption in PostgreSQL • Key rotation • Key Management • Integration of PostgreSQL with key management system • Conclusion
  3. 3. Copyright©2019 NTT Corp. All Rights Reserved.
  4. 4. Copyright©2019 NTT Corp. All Rights Reserved. • Database servers are often the primary target of the following attacks • Privilege abuse • Database SQL injections attacks • Storage media theft • Eavesdropping attacks between client and server • etc. 44 Eavesdropping attacks 4 4
  5. 5. Copyright©2019 NTT Corp. All Rights Reserved. • Reason to protect database • Databases store valuable and sensitive data, some leakage also causes a high risk • Essential data protection standards and regulations for businesses • PCI DSS, GDPR, HIPPA etc. Various data protection standards and regulations require data encryption
  6. 6. Copyright©2019 NTT Corp. All Rights Reserved.
  7. 7. Copyright©2019 NTT Corp. All Rights Reserved. • Data at rest • Backup files and database cluster files stored in physical storage • Measures against threats to data at rest in PostgreSQL • Data encryption using pgcrypto or Full disk encryption However, using pgcrypto or full disk encryption does not meet some requirements Data is secure ! Data leak Threat of theft occurred Database storage Database storage User data stored encrypted User data stored non-encrypted
  8. 8. Copyright©2019 NTT Corp. All Rights Reserved. • Minimize performance degradation • Using pgcrypto degrades application program performance • Platform-independent • Minimize application program development cost • Secure encryption key management • Periodic key rotation
  9. 9. Copyright©2019 NTT Corp. All Rights Reserved. • Minimize performance degradation • Platform-independent • Using full disk encryption depends on platforms • Minimize application program development cost • Secure encryption key management • Periodic key rotation
  10. 10. Copyright©2019 NTT Corp. All Rights Reserved. • Minimize performance degradation • Platform-independent • Minimize application program development cost • Using pgcrypto requires many application program modifications • Secure encryption key management • Periodic key rotation
  11. 11. Copyright©2019 NTT Corp. All Rights Reserved. • Many modifications of the application program source code for areas(Tables) where encryption is required • Regression testing required after source code modification • Encryption supports increase time and labor costs =# INSERT INTO card_info ( user_name, card_number) VALUES ( ‘MOON INSUNG’, ‘1234-2345-3456-4567’ ); =# INSERT INTO card_info ( user_name, card_number) VALUES ( ‘MOON INSUNG’, encrypt (‘1234-2345-3456-4567’, 'KEY_VALUE', ‘aes-cbc’) ); SQL statement before encryption SQL statement after encryption using pgcrypto
  12. 12. Copyright©2019 NTT Corp. All Rights Reserved. • Minimize performance degradation • Platform-independent • Minimize application program development cost • Secure encryption key management • pgcrypto doesn't provide secure encryption key management facilities out-of-the-box • Periodic key rotation
  13. 13. Copyright©2019 NTT Corp. All Rights Reserved. • If encryption key is leaked, the encrypted data at rest cannot be protected from threats of malicious access • Storing encrypted data and its key in the same place makes the data encryption meaningless
  14. 14. Copyright©2019 NTT Corp. All Rights Reserved. • Key management in a separate secure place for the encryption keys Database storage Encrypted data Threat of theft occurred Separate location Secure Place Data is secure
  15. 15. Copyright©2019 NTT Corp. All Rights Reserved. • Minimize performance degradation • Platform-independent • Minimize application program development cost • Secure encryption key management • Periodic key rotation • Reduced performance due to re-encryption when rotating keys Database storage Encrypted data Decryption current key Database storage Non- encrypted data Database storage Encrypted dataEncryption New key Performance degradation due to re-encryption and data unavailability
  16. 16. Copyright©2019 NTT Corp. All Rights Reserved. • Requirements related to data encryption • Minimize performance degradation • Platform-independent • Minimize application program development cost • Secure encryption key management • Periodic key rotation • Difficult to satisfy these requirements related to data encryption using pgcrypto and full disk encryption in PostgreSQL In the following slides, we describe the result of our efforts to address these challenges
  17. 17. Copyright©2019 NTT Corp. All Rights Reserved.
  18. 18. Copyright©2019 NTT Corp. All Rights Reserved. 1. Per tablespace encryption 2. Transparent encryption at a layer between PostgreSQL’s shared buffer and OS 3. 2-tier key architecture 4. WAL encryption 5. System catalogs encryption 6. Temporary files encryption - - -
  19. 19. Copyright©2019 NTT Corp. All Rights Reserved. • Create encryption-enabled tablespaces • Tables and indexes on the tablespace are transparently encrypted • Less modification of DDLs . =# CREATE TABLESPACE enc_tblsp LOCATION ... WITH (eyncryption_algorithm = ‘aes128’); =# CREATE TABLE card (...) TABLESPACE enc_tblsp; =# SET default_tablespace TO enc_tblsp; =# CREATE TABLE card2 (..);
  20. 20. Copyright©2019 NTT Corp. All Rights Reserved. • Two possible solutions for now • Using pgcrypto with views and triggers • Full disk encryption (FDE) • Our solution is that encryption and decryption are performed when writing/reading to/from disk
  21. 21. Copyright©2019 NTT Corp. All Rights Reserved. / / / postgres Shared Buffer Disk postgres postgres Page Cache (Kernel) raw block data
  22. 22. Copyright©2019 NTT Corp. All Rights Reserved. / / / postgres Disk postgres postgres Page Cache (Kernel) raw block data Shared Buffer Backend processes read pages from the shared buffers and modify them.
  23. 23. Copyright©2019 NTT Corp. All Rights Reserved. / / / postgres Disk postgres postgres Page Cache (Kernel) raw block data Shared Buffer bgwriter periodically writes the dirty pages out to the kernel page cache.
  24. 24. Copyright©2019 NTT Corp. All Rights Reserved. / / / postgres Disk postgres postgres raw block data Shared Buffer Page Cache (Kernel) Dirty pages are flushed to the disk by the checkpointer or the kernel.
  25. 25. Copyright©2019 NTT Corp. All Rights Reserved. - postgres Shared Buffer Disk Pros: • Protect data even on shared buffer Cons: • Encryption and decryption are needed whenever accessing buffers on shared buffer • Different backends encrypt/decrypt the same buffer postgres postgres Page Cache (Kernel) raw data encrypted data
  26. 26. Copyright©2019 NTT Corp. All Rights Reserved. - . postgres Shared Buffer Disk Pros: • Less execution of encryption and decryption Cons: • Platform dependence • Cannot protect data from peeking by logged-in OS user postgres postgres Page Cache (Kernel) raw data encrypted data
  27. 27. Copyright©2019 NTT Corp. All Rights Reserved. 3 ) ) ) -( 2 .- .3 . 32 .- postgres Shared Buffer Disk Pros: • Relatively less execution of encryption and decryption • Prevent peeking file on disk Cons: • Possibly more encryption and decryption are performed when database size > shared buffer postgres postgres Page Cache (Kernel) raw data encrypted data
  28. 28. Copyright©2019 NTT Corp. All Rights Reserved. Transparence Performance impacts Protection against threats 1. Using pgcrypto OK (using views and triggers) High * Disk thefts * Memory dump * Peeking at data on disk 2. Full disk encryption OK Low * Disk thefts 3. Buffer level encryption OK Middle * Disk thefts * Peeking at data on disk
  29. 29. Copyright©2019 NTT Corp. All Rights Reserved. • Wrote proof-of-concept code of per tablespaces TDE • Tablespace encryption • Transparent encryption • 2-tier key architecutre • Key rotation • Features the PoC code doesn’t support (for now): • WAL encryption • System catalog encryption • Temporary files encryption
  30. 30. Copyright©2019 NTT Corp. All Rights Reserved. • PostgreSQL 11.1 • Vanilla PostgreSQL • PostgreSQL with TDE PoC code • PostgreSQL with pgcrypto • 32GB RAM, 500GB HDD • 6GB shared buffers • Data sets • 5GB (< shared_buffers) • 15GB (> shared_buffers) • Observations • TPS • Response times • Duration: 5min
  31. 31. Copyright©2019 NTT Corp. All Rights Reserved. Latency (90%tile): vanilla: 1.98 ms, TDE: 2.01 ms, pgcrypto: 2.28 ms 6000 6500 7000 7500 8000 8500 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 TPS Duraiton(sec) TPS comparison (R:100,W:3) vanilla tde pgcrypto 8000 8500 9000 9500 10000 10500 11000 10 30 50 70 90 110 130 150 170 190 210 230 250 270 TPS Duration (sec) TPS comparison (R:100) vanilla tde pgcrypto Latency (90%tile): vanilla: 2.32 ms, TDE: 2.45 ms, pgcrypto: 2.66 ms DB size < shared buffers DB size > shared buffers
  32. 32. Copyright©2019 NTT Corp. All Rights Reserved. • Master Key and Data Encryption Key • The master key is separated from encrypted data • Stored outside of the database • The data encryption keys are managed by database • Faster key rotation ENCRYPTED DATA Master Key Data Encryption Key Encrypt/Decrypt Encrypt/ Decrypt plain
  33. 33. Copyright©2019 NTT Corp. All Rights Reserved. • Master Key and Data Encryption Key • The master key is separated from encrypted data • Stored outside of the database • The data encryption keys are managed by database • Faster key rotation ENCRYPTED DATA Master Key Data Encryption Key Encrypt/Decrypt Encrypt/ Decrypt encry pted plain New Master Key Encrypt/Decrypt
  34. 34. Copyright©2019 NTT Corp. All Rights Reserved. • Key rotation always requires re-encrypting data • The 2-tier key architecture requires only key rotation of data encryption keys • 16, 24, 32 bytes key for AES-128, AES-192, AES-256 • One symmetric key per tablespaces
  35. 35. Copyright©2019 NTT Corp. All Rights Reserved. • WAL is also the sensitive data • WAL of encrypted relations is encrypted when inserting to the WAL buffer
  36. 36. Copyright©2019 NTT Corp. All Rights Reserved. • Two system catalogs could have user sensitive data • pg_statistics • pg_statistics_ext . =# SELECT tablename, attname, histogram_bounds FROM pg_stats WHERE tablename = 'card'; -[ RECORD 1 ]----+----------------------------------------------- tablename | card attname | card_number histogram_bounds | {1102-6674-6045-5459,1606-6441-9374-1335,2507- 2573-1560-9962,3323-3000-4260-1336,4319-9183-6377-7031,6035-9617- 5940-2060,6682-5210-8901-2679,7304-3837-8200-8185,8391-3583-3888- 1725,9091-3895-2466-7845,9970-5910-3522-1423}
  37. 37. Copyright©2019 NTT Corp. All Rights Reserved. • Temporary files are written bypassing the shared buffers • base/pgsql_tmp/ • pg_replslots/ . postgres Shared Buffer Disk temp files
  38. 38. Copyright©2019 NTT Corp. All Rights Reserved. • Per tablespace, buffer-level transparent encryption • 2-tier key architecture • Encrypt WAL, system catalogs and temporary files • SRLU buffer and fork relations are not encrypted • Pros • Less DDL modification • Less performance impact • Fast key rotation • Cons • Cannot set per users • Cannot prevent attack by malicious super user
  39. 39. Copyright©2019 NTT Corp. All Rights Reserved.
  40. 40. Copyright©2019 NTT Corp. All Rights Reserved. • Services or systems that are dedicated to robustly manage keys • Usually support some kinds of protocols • KMIP • PKCS#11 • SafeNet KeySecure, Amazon KMS, Oracle KeyVault etc ( )( ( (
  41. 41. Copyright©2019 NTT Corp. All Rights Reserved. • Robust key management • User don’t need to worry about key life cycles ENCRYPTED DATA Master Key Data Encryption Keys Get the master key and decrypt/decrypt Encrypt/Decrypt Key Management System PostgreSQL Register the master key Remove an old master key
  42. 42. Copyright©2019 NTT Corp. All Rights Reserved. • KMSs support different interfaces and protocols • KMIP, PKCS#11, etc. • Our solution • Pluggable architecture to communicate with various KMSs • Add generic key management APIs • get key, register key, remove key etc.
  43. 43. Copyright©2019 NTT Corp. All Rights Reserved. • Encryption key is also important • Integration with KMS frees user from key management • Adding generic key management APIs enable us to communicate with various key management systems
  44. 44. Copyright©2019 NTT Corp. All Rights Reserved. • Per tablespace, buffer-level transparent data at rest encryption • Less performance overhead • Encrypt WAL, system catalogs and temporary files as well • 2-tier key architecture • Fast key rotation • Integration with KMSs • More flexible and robust key management
  45. 45. Copyright©2019 NTT Corp. All Rights Reserved. 1. Basic components of transparent data encryption • Per tablespace encryption* • 2-tier key architecture* • Key rotation* • System catalog encryption • Temporary file encryption • Fetching the master key by arbitrary commands* 2. WAL encryption 3. Integration with KMSs • Pluggable • Registering key, removing key ... PoC codes has *-marked features
  46. 46. Copyright©2019 NTT Corp. All Rights Reserved. !

×