SlideShare une entreprise Scribd logo

SQL Database Design For Developers at php[tek] 2024

•Télécharger en tant que PPTX, PDF•
•
S
Scott Keck-Warren

My talk about SQL Database Design For Developers given at php[tek] 2024

Technologie
1  sur  150
SQL Database Design For Developers Scott Keck-Warren php[tek] 2024 @scottKeckWarren@phpc.social
1.Never Worked With a DBA On a Project 2.Every Project Has At Least One SQL Database
1.Never Worked With a DBA On a Project 2.Every Project Has At Least One SQL Database
My (Early) Relationship to SQL Database: Less Than Ideal
How Did I Get Better?
Trial By Fire
Problems • “Random” slowness • Data inconsistencies • Weird Bugs
SQL Database Design For Developers
Scott Keck-Warren Director of Technology @ WeCare Connect
Scott Keck-Warren PHP Developer
Scott Keck-Warren Director of Technology @ WeCare Connect
Scott Keck-Warren Content Creator @php[architect] YouTube Community Corner Podcast
Scott’s Rules For Database Design
Never Worked With a DBA On a Project
Scott’s Rules For Database Design 1. Normalize Your Database For Data Deduplication 2. Use The Database Engine to Keep Data Clean 3. Proactively Add Indexes to Keep Queries Performant
Users Table
Users Table • Email address • Password • Active state • Hire Date • Listing of previous passwords • Office Name • Office City • Office Zip
Users Table • Email address (string) • Password (string) • Active state (string) • Hire Date (string) • Listing of previous passwords (string) • Office Name (string) • Office City (string) • Office Zip (string)
Users Table email password active hire_date previous_ password office_name office_phone office_city office_zip alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 avery@exa mple.com NULL 1 8/11/2024 hash2 hash7 hash8 main office 5555555555 Saginaw 48609 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555-5555 Saginaw 48609 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
Normalize Your Database For Data Deduplication
Normalize Your Database For Data Deduplication “[T]he process of structuring a relational database in accordance with a series of so- called normal forms in order to reduce data redundancy and improve data integrity.” -“Database normalization” on Wikipedia
Normalize Your Database For Data Deduplication • UNF: Unnormalized form • 1NF: First normal form • 2NF: Second normal form • 3NF: Third normal form • EKNF: Elementary key normal form • BCNF: Boyce–Codd normal form • 4NF: Fourth normal form • ETNF: Essential tuple normal form • 5NF: Fifth normal form • DKNF: Domain-key normal form • 6NF: Sixth normal form
Normalize Your Database For Data Deduplication • UNF: Unnormalized form • 1NF: First normal form • 2NF: Second normal form • 3NF: Third normal form • EKNF: Elementary key normal form • BCNF: Boyce–Codd normal form • 4NF: Fourth normal form • ETNF: Essential tuple normal form • 5NF: Fifth normal form • DKNF: Domain-key normal form • 6NF: Sixth normal form
Normalize Your Database For Data Deduplication • Boyce–Codd Normal Form: • X should be a superkey for every functional dependency (FD) X−>Y in a given relation.
Unnormalized Form
Unnormalized Form • A table doesn’t meet any of the conditions of normalization • Essentially a spreadsheet email password active hire_date previous_ password office_name office_phone office_city office_zip alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 avery@exa mple.com NULL 1 8/11/2024 hash2 hash7 hash8 main office 5555555555 Saginaw 48609 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555-5555 Saginaw 48609 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
First Normal Form (1NF)
First Normal Form (1NF) 1. The table contains a unique identifier, also called the primary key, that is used to identify the row. 2. Each column contains atomic values (values that can not be broken down)
1NF - users email password active hire_date previous_p assword office_ name office_phone office_city office_zip alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 avery@exa mple.com NULL 1 8/11/2024 hash2 hash7 Hash8 main office 5555555555 Saginaw 48609 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555-5555 Saginaw 48609 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
1NF - users • A unique identifier should be: • Auto-incrementing int • UUID
1NF - users id email password active hire_date previous_ password office_ name office_phone office_cit y office_zip 1 alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 2 avery@ex ample.com NULL 1 8/11/2024 hash2 hash7 Hash8 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555- 5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
1NF - users id email password active hire_date previous_ password office_ name office_phone office_cit y office_zip 1 alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 2 avery@ex ample.com NULL 1 8/11/2024 hash2 hash7 hash8 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555- 5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
1NF - user_password_histories
1NF - user_password_histories id user_id password
1NF - user_password_histories id user_id password 1 1 hash1 2 1 hash5 3 1 hash6 4 2 hash2 5 2 hash7 6 2 hash8 7 3 hash3 8 4 hash4
1NF - users id email password active hire_date previous_ password office_ name office_phone office_cit y office_zip 1 alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 2 avery@ex ample.com NULL 1 8/11/2024 hash2 hash7 Hash8 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555- 5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
1NF - users id email password active hire_date office_ name office_phone office_city office_zip 1 alice@exa mple.com hash1 1 1/1/2024 Main Office 555-555-5555 Saginaw 48609 2 avery@exa mple.com NULL 1 8/11/2024 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 Man office (555)555-5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday Main 555/555/5555 Saginaw 48609
Second Normal Form (2NF)
Second Normal Form (2NF) 1. Is already in 1NF 2. All the non-key columns are dependent on the primary key of the table
Second Normal Form (2NF) id email password active hire_date office_ name office_phone office_city office_zip 1 alice@exa mple.com hash1 1 1/1/2024 Main Office 555-555-5555 Saginaw 48609 2 avery@exa mple.com NULL 1 8/11/2024 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 Man office (555)555-5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday Main 555/555/5555 Saginaw 48609
2nd - offices id name phone city zip 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609 3 Man office (555)555-5555 Saginaw 48609 4 Main 555/555/5555 Saginaw 48609
2NF - users id email password active hire_date office_ name office_phone office_city office_zip 1 alice@exa mple.com hash1 1 1/1/2024 Main Office 555-555-5555 Saginaw 48609 2 avery@exa mple.com NULL 1 8/11/2024 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 Man office (555)555-5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday Main 555/555/5555 Saginaw 48609
2NF - users id email password active hire_date office_ name office_phone office_cit y office_zip office_id 1 alice@exa mple.com hash1 1 1/1/2024 Main Office 555-555-5555 Saginaw 48609 1 2 avery@ex ample.com NULL 1 8/11/2024 main office 5555555555 Saginaw 48609 2 3 scott@exa mple.com hash3 1 May 11th, 23 Man office (555)555- 5555 Saginaw 48609 3 4 scott@exa mple.com hash4 1 Tuesday Main 555/555/5555 Saginaw 48609 4
2NF - users id email password active hire_date office_id 1 alice@example.co m hash1 1 1/1/2024 1 2 avery@example.c om NULL 1 8/11/2024 2 3 scott@example.co m hash3 1 May 11th, 23 3 4 scott@example.co m hash4 1 Tuesday 4
Third Normal Form (3NF)
Third Normal Form (3NF) 1. Is already in 2NF 2. It contains columns that are non-transitively dependent on the primary key
3NF - offices id name phone city zip 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609 3 Man office (555)555-5555 Saginaw 48609 4 Main 555/555/5555 Saginaw 48609
3NF - zips id city 48609 Saginaw 48640 Midland 48642 Midland 48901 Lansing
3NF - zips id city state 48609 Saginaw MI 48640 Midland MI 48642 Midland MI 48901 Lansing MI
Old Table Structure
New Table Structure
Use The Database Engine to Keep Data Clean
Why?
Why?
mysql> insert into users (password) values (“just a password?"); Query OK, 1 row affected (0.01 sec)
mysql> insert into users (password) values (“just a password?"); Query OK, 1 row affected (0.01 sec)
Garbage In -> Garbage Out
Garbage Data -> Bugs
To Prevent Bugs: Make The Database Work For Us
id email password active hire_date office_id 1 alice@example.co m hash1 1 1/1/2024 1 2 avery@example.c om NULL 1 8/11/2024 2 3 scott@example.co m hash3 1 May 11th, 23 3 4 scott@example.co m hash4 1 Tuesday 4 5 Hash12 2 2024-04-01 1000
Use Correct Column Types
Use Correct Column Types id email password active hire_date office_id 1 alice@example.co m hash1 1 1/1/2024 1 2 avery@example.c om NULL 1 8/11/2024 2 3 scott@example.co m hash3 1 May 11th, 23 3 4 scott@example.co m hash4 1 Tuesday 4 5 Hash12 2 2024-04-01 1000
Use Correct Column Types • Numeric: INT, TINYINT, BIGINT, FLOAT, REAL, etc. • Date/Time: DATE, TIME, DATETIME, etc. • String: CHAR, VARCHAR, TEXT, etc. • Binary data types such as: BLOB, etc.
Use Correct Column Types
Use Correct Column Types
Use Correct Column Types mysql> insert into users (hire_date) values ("tuesday"); ERROR 1292 (22007): Incorrect date value: 'tuesday' for column 'hire_date' at row 1 mysql> insert into users (hire_date) values ("May 11th, 23"); ERROR 1292 (22007): Incorrect date value: 'May 11th, 23' for column 'hire_date' at row 1
Use NOT NULL for Required Fields
Use NOT NULL for Required Fields mysql> insert into users (password) values (“just a password?"); Query OK, 1 row affected (0.01 sec)
Use NOT NULL for Required Fields
Use NOT NULL for Required Fields
Use NOT NULL for Required Fields mysql> insert into users (password) values (“just a password?”); ERROR 1364 (HY000): Field ‘email' doesn't have a default value
Use NOT NULL for Required Fields mysql> insert into users (password) values (“just a password?”); ERROR 1364 (HY000): Field ‘email' doesn't have a default value
Use NOT NULL for Required Fields mysql> insert into users (email, password) values (“s@s”, "just a password?"); ERROR 1364 (HY000): Field 'active' doesn't have a default value
Use UNIQUE for Unique Values
Use UNIQUE for Unique Values mysql> insert into users (email) values ("scott@keck-warren.com"); Query OK, 1 row affected (0.01 sec) mysql> insert into users (email) values ("scott@keck-warren.com"); Query OK, 1 row affected (0.02 sec)
Use UNIQUE for Unique Values
Use UNIQUE for Unique Values
Use UNIQUE for Unique Values mysql> insert into users (email) values ("scott@keck-warren.com"); ERROR 1062 (23000): Duplicate entry 'scott@keck-warren.com' for key 'users.email'
Use UNIQUE for Unique Values
Use Foreign Keys For References To Other Tables
Use Foreign Keys For References To Other Tables id name phone city zip_id 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609 3 Man office (555)555-5555 Saginaw 48609 4 Main 555/555/5555 Saginaw 48609
Use Foreign Keys For References To Other Tables id name phone city zip_id 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609 3 Man office (555)555-5555 Saginaw 48609
Use Foreign Keys For References To Other Tables id name phone city zip_id 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609
Use Foreign Keys For References To Other Tables id name phone city zip_id 1 Main Office 555-555-5555 Saginaw 48609
Constraints - Foreign Keys
Orphaned Rows
mysql> insert into users (email, office_id) values ("s@kw.com", 1000000); Query OK, 1 row affected (0.03 sec)
mysql> insert into users (email, office_id) values ("s@kw.com", 1000000); Query OK, 1 row affected (0.03 sec)
Foreign Key Constraints
Foreign Key Constraints
Foreign Key Constraints
Foreign Key Constraints
Foreign Key Constraints
Foreign Key Constraints mysql> insert into users (email, office_id) values ("s@kw.com", 1000000); ERROR 1452 (23000): Cannot add or update a child row: a foreign key constraint fails (`databasetalk`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`office_id`) REFERENCES `offices` (`id`))
Foreign Key Constraints mysql> insert into users (email, office_id) values ("s@kw.com", 1000000); ERROR 1452 (23000): Cannot add or update a child row: a foreign key constraint fails (`databasetalk`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`office_id`) REFERENCES `offices` (`id`))
Foreign Key Constraints mysql> insert into users (email, office_id) values ("s@kw.com", 1); Query OK, 1 row affected (0.01 sec)
Foreign Key Constraints mysql> delete from offices where id = 1; ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`databasetalk`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`office_id`) REFERENCES `offices` (`id`))
Foreign Key Constraints mysql> delete from offices where id = 1; ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`databasetalk`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`office_id`) REFERENCES `offices` (`id`))
Foreign Key Constraints
Foreign Key Constraints
Foreign Key Constraints
Foreign Key Constraints mysql> insert into user_password_histories (user_id, password) values (2, "test1"); Query OK, 1 row affected (0.01 sec) mysql> insert into user_password_histories (user_id, password) values (2, "test2"); Query OK, 1 row affected (0.01 sec)
Foreign Key Constraints mysql> delete from users where id = 2; Query OK, 1 row affected (0.01 sec) mysql> select * from user_password_histories where user_id = 2; Empty set (0.00 sec)
Foreign Key Constraints mysql> delete from users where id = 2; Query OK, 1 row affected (0.01 sec) mysql> select * from user_password_histories where user_id = 2; Empty set (0.00 sec)
Foreign Key Constraints
Foreign Key Constraints
Downsides to Constraints
Performance
Use Triggers For Complex Requirements
Use Triggers For Complex Requirements • Triggers add additional power to DB • Operate based on create, update, or delete
Use Triggers For Complex Requirements id email password active hire_date office_id 1 alice@example.co m hash1 1 1/1/2024 1 2 avery@example.c om NULL 1 8/11/2024 2 3 scott@example.co m hash3 1 May 11th, 23 3 4 scott@example.co m hash4 1 Tuesday 4 5 Hash12 2 2024-04-01 1000
Use Triggers For Complex Requirements
Use Triggers For Complex Requirements
Use Triggers For Complex Requirements
Use Triggers For Complex Requirements
Use Triggers For Complex Requirements
Use Triggers For Complex Requirements
Use Triggers For Complex Requirements mysql> insert into users (active) values (2); ERROR 1644 (45000): active must be 0 or 1
Use Triggers For Complex Requirements mysql> insert into users (active) values (2); ERROR 1644 (45000): active must be 0 or 1
Proactively Add Indexes to Keep Queries Performant
Indexes in Databases
Indexes in Databases
Indexes in Databases
Indexes in Databases 2023-12-03 Index: hire_date 2023-12-04 2023-12-05 2023-12-06
Start Out Simple
What You Need to Know
What You Need to Know 1. Normalize Your Database For Data Deduplication 2. Use The Database Engine to Keep Data Clean 3. Proactively Add Indexes to Keep Queries Performant
What You Need to Know 1. The table contains a unique identifier, also called the primary key, that is used to identify the row. 2. Each column contains atomic values (values that can not be broken down) 3. All the non-key columns are dependent on the primary key of the table 4. It contains columns that are non-transitively dependent on the primary key
What You Need to Know • Make the DB Work With You • Correct Column Types • NOT NULL for Required Fields • UNIQUE for Unique Values • Foreign Keys For References To Other Tables • Triggers For Complex Requirements
What You Need to Know • Use indexes on commonly searched columns • Start simple • See recorded talks about how to add
Thank The Speakers
Thank The Organizers
Questions/Follow Me • Questions • Please rate the talk • <link> • @scottKeckWarren@phpc.social • @scottKeckWarren@twitter.com

Recommandé

MySQL Integral DB Design #JAB14
MySQL Integral DB Design #JAB14MySQL Integral DB Design #JAB14
MySQL Integral DB Design #JAB14
Eli Aschkenasy
 
Normalisation revision
Normalisation revisionNormalisation revision
Normalisation revision
Dr. Ramkumar Lakshminarayanan
 
Strayer cis-515-week-7-assignment-6-sql-concepts-and-database-design
Strayer cis-515-week-7-assignment-6-sql-concepts-and-database-designStrayer cis-515-week-7-assignment-6-sql-concepts-and-database-design
Strayer cis-515-week-7-assignment-6-sql-concepts-and-database-design
kxipvscsk02
 
Realtimestream and realtime fastcatsearch
Realtimestream and realtime fastcatsearchRealtimestream and realtime fastcatsearch
Realtimestream and realtime fastcatsearch
상욱 송
 
Rakuten Technology Conference 2017 A Distributed SQL Database For Data Analy...
Rakuten Technology Conference 2017 A Distributed SQL Database For Data Analy...Rakuten Technology Conference 2017 A Distributed SQL Database For Data Analy...
Rakuten Technology Conference 2017 A Distributed SQL Database For Data Analy...
Rakuten Group, Inc.
 
A Call for Sanity in NoSQL
A Call for Sanity in NoSQLA Call for Sanity in NoSQL
A Call for Sanity in NoSQL
C4Media
 
NoSQL Couchbase Lite & BigData HPCC Systems
NoSQL Couchbase Lite & BigData HPCC SystemsNoSQL Couchbase Lite & BigData HPCC Systems
NoSQL Couchbase Lite & BigData HPCC Systems
Fujio Turner
 
Sasi, cassandra on the full text search ride At Voxxed Day Belgrade 2016
Sasi, cassandra on the full text search ride At Voxxed Day Belgrade 2016Sasi, cassandra on the full text search ride At Voxxed Day Belgrade 2016
Sasi, cassandra on the full text search ride At Voxxed Day Belgrade 2016
Duyhai Doan
 

Recommandé

MySQL Integral DB Design #JAB14
MySQL Integral DB Design #JAB14MySQL Integral DB Design #JAB14
MySQL Integral DB Design #JAB14
Eli Aschkenasy
 
Normalisation revision
Normalisation revisionNormalisation revision
Normalisation revision
Dr. Ramkumar Lakshminarayanan
 
Strayer cis-515-week-7-assignment-6-sql-concepts-and-database-design
Strayer cis-515-week-7-assignment-6-sql-concepts-and-database-designStrayer cis-515-week-7-assignment-6-sql-concepts-and-database-design
Strayer cis-515-week-7-assignment-6-sql-concepts-and-database-design
kxipvscsk02
 
Realtimestream and realtime fastcatsearch
Realtimestream and realtime fastcatsearchRealtimestream and realtime fastcatsearch
Realtimestream and realtime fastcatsearch
상욱 송
 
Rakuten Technology Conference 2017 A Distributed SQL Database For Data Analy...
Rakuten Technology Conference 2017 A Distributed SQL Database For Data Analy...Rakuten Technology Conference 2017 A Distributed SQL Database For Data Analy...
Rakuten Technology Conference 2017 A Distributed SQL Database For Data Analy...
Rakuten Group, Inc.
 
A Call for Sanity in NoSQL
A Call for Sanity in NoSQLA Call for Sanity in NoSQL
A Call for Sanity in NoSQL
C4Media
 
NoSQL Couchbase Lite & BigData HPCC Systems
NoSQL Couchbase Lite & BigData HPCC SystemsNoSQL Couchbase Lite & BigData HPCC Systems
NoSQL Couchbase Lite & BigData HPCC Systems
Fujio Turner
 
Sasi, cassandra on the full text search ride At Voxxed Day Belgrade 2016
Sasi, cassandra on the full text search ride At Voxxed Day Belgrade 2016Sasi, cassandra on the full text search ride At Voxxed Day Belgrade 2016
Sasi, cassandra on the full text search ride At Voxxed Day Belgrade 2016
Duyhai Doan
 
Simple Nested Sets and some other DB optimizations
Simple Nested Sets and some other DB optimizationsSimple Nested Sets and some other DB optimizations
Simple Nested Sets and some other DB optimizations
Eli Aschkenasy
 
Strayer cis-515-entire-course-new
Strayer cis-515-entire-course-newStrayer cis-515-entire-course-new
Strayer cis-515-entire-course-new
kxipvscsk02
 
CIS 515 Entire Course NEW
CIS 515 Entire Course NEWCIS 515 Entire Course NEW
CIS 515 Entire Course NEW
shyamuopfive
 
Lecture 7 database
Lecture 7 databaseLecture 7 database
Lecture 7 database
moduledesign
 
Locality of Reference
Locality of ReferenceLocality of Reference
Locality of Reference
Kouji Matsui
 
JSSUG: SQL Sever Index Tuning
JSSUG: SQL Sever Index TuningJSSUG: SQL Sever Index Tuning
JSSUG: SQL Sever Index Tuning
Kenichiro Nakamura
 
Trouble-shooting Tips for Primo (2013)
Trouble-shooting Tips for Primo (2013)Trouble-shooting Tips for Primo (2013)
Trouble-shooting Tips for Primo (2013)
Alison Hitchens
 
OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
Databricks
 
Relational Database Design Bootcamp
Relational Database Design BootcampRelational Database Design Bootcamp
Relational Database Design Bootcamp
Mark Niebergall
 
ElasticSearch AJUG 2013
ElasticSearch AJUG 2013ElasticSearch AJUG 2013
ElasticSearch AJUG 2013
Roy Russo
 
Cassandra introduction apache con 2014 budapest
Cassandra introduction apache con 2014 budapestCassandra introduction apache con 2014 budapest
Cassandra introduction apache con 2014 budapest
Duyhai Doan
 
Scaling Machine Learning Feature Engineering in Apache Spark at Facebook
Scaling Machine Learning Feature Engineering in Apache Spark at FacebookScaling Machine Learning Feature Engineering in Apache Spark at Facebook
Scaling Machine Learning Feature Engineering in Apache Spark at Facebook
Databricks
 
Kumar lav
Kumar lavKumar lav
Kumar lav
kumar Lav
 
Dead Simple Scalability Patterns
Dead Simple Scalability PatternsDead Simple Scalability Patterns
Dead Simple Scalability Patterns
Vedang Manerikar
 
What You Need To Know About The Top Database Trends
What You Need To Know About The Top Database TrendsWhat You Need To Know About The Top Database Trends
What You Need To Know About The Top Database Trends
Dell World
 
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...
ITD Systems
 
Splunk bsides
Splunk bsidesSplunk bsides
Splunk bsides
Macy Cronkrite
 
Normalization | (1NF) |(2NF) (3NF)|BCNF| 4NF |5NF
Normalization | (1NF) |(2NF) (3NF)|BCNF| 4NF |5NFNormalization | (1NF) |(2NF) (3NF)|BCNF| 4NF |5NF
Normalization | (1NF) |(2NF) (3NF)|BCNF| 4NF |5NF
Biplap Bhattarai
 
Dev Analytics Aggregate DB Design Analysis
Dev Analytics Aggregate DB Design AnalysisDev Analytics Aggregate DB Design Analysis
Dev Analytics Aggregate DB Design Analysis
Sunita Shrivastava
 
Recursive Query Throwdown
Recursive Query ThrowdownRecursive Query Throwdown
Recursive Query Throwdown
Karwin Software Solutions LLC
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
Scott Keck-Warren
 
Getting Started with Test-Driven Development at Longhorn PHP 2023
Getting Started with Test-Driven Development at Longhorn PHP 2023Getting Started with Test-Driven Development at Longhorn PHP 2023
Getting Started with Test-Driven Development at Longhorn PHP 2023
Scott Keck-Warren
 

Contenu connexe

Similaire à SQL Database Design For Developers at php[tek] 2024

OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
Databricks
 
Scaling Machine Learning Feature Engineering in Apache Spark at Facebook
Scaling Machine Learning Feature Engineering in Apache Spark at FacebookScaling Machine Learning Feature Engineering in Apache Spark at Facebook
Scaling Machine Learning Feature Engineering in Apache Spark at Facebook
Databricks
 
Dev Analytics Aggregate DB Design Analysis
Dev Analytics Aggregate DB Design AnalysisDev Analytics Aggregate DB Design Analysis
Dev Analytics Aggregate DB Design Analysis
Sunita Shrivastava
 

Similaire à SQL Database Design For Developers at php[tek] 2024 (20)

Simple Nested Sets and some other DB optimizations
Simple Nested Sets and some other DB optimizationsSimple Nested Sets and some other DB optimizations
Simple Nested Sets and some other DB optimizations
 
Strayer cis-515-entire-course-new
Strayer cis-515-entire-course-newStrayer cis-515-entire-course-new
Strayer cis-515-entire-course-new
 
CIS 515 Entire Course NEW
CIS 515 Entire Course NEWCIS 515 Entire Course NEW
CIS 515 Entire Course NEW
 
Lecture 7 database
Lecture 7 databaseLecture 7 database
Lecture 7 database
 
Locality of Reference
Locality of ReferenceLocality of Reference
Locality of Reference
 
JSSUG: SQL Sever Index Tuning
JSSUG: SQL Sever Index TuningJSSUG: SQL Sever Index Tuning
JSSUG: SQL Sever Index Tuning
 
Trouble-shooting Tips for Primo (2013)
Trouble-shooting Tips for Primo (2013)Trouble-shooting Tips for Primo (2013)
Trouble-shooting Tips for Primo (2013)
 
OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...
 
Relational Database Design Bootcamp
Relational Database Design BootcampRelational Database Design Bootcamp
Relational Database Design Bootcamp
 
ElasticSearch AJUG 2013
ElasticSearch AJUG 2013ElasticSearch AJUG 2013
ElasticSearch AJUG 2013
 
Cassandra introduction apache con 2014 budapest
Cassandra introduction apache con 2014 budapestCassandra introduction apache con 2014 budapest
Cassandra introduction apache con 2014 budapest
 
Scaling Machine Learning Feature Engineering in Apache Spark at Facebook
Scaling Machine Learning Feature Engineering in Apache Spark at FacebookScaling Machine Learning Feature Engineering in Apache Spark at Facebook
Scaling Machine Learning Feature Engineering in Apache Spark at Facebook
 
Kumar lav
Kumar lavKumar lav
Kumar lav
 
Dead Simple Scalability Patterns
Dead Simple Scalability PatternsDead Simple Scalability Patterns
Dead Simple Scalability Patterns
 
What You Need To Know About The Top Database Trends
What You Need To Know About The Top Database TrendsWhat You Need To Know About The Top Database Trends
What You Need To Know About The Top Database Trends
 
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...
 
Splunk bsides
Splunk bsidesSplunk bsides
Splunk bsides
 
Normalization | (1NF) |(2NF) (3NF)|BCNF| 4NF |5NF
Normalization | (1NF) |(2NF) (3NF)|BCNF| 4NF |5NFNormalization | (1NF) |(2NF) (3NF)|BCNF| 4NF |5NF
Normalization | (1NF) |(2NF) (3NF)|BCNF| 4NF |5NF
 
Dev Analytics Aggregate DB Design Analysis
Dev Analytics Aggregate DB Design AnalysisDev Analytics Aggregate DB Design Analysis
Dev Analytics Aggregate DB Design Analysis
 
Recursive Query Throwdown
Recursive Query ThrowdownRecursive Query Throwdown
Recursive Query Throwdown
 

Plus de Scott Keck-Warren

Plus de Scott Keck-Warren (8)

Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Getting Started with Test-Driven Development at Longhorn PHP 2023
Getting Started with Test-Driven Development at Longhorn PHP 2023Getting Started with Test-Driven Development at Longhorn PHP 2023
Getting Started with Test-Driven Development at Longhorn PHP 2023
 
Getting Started with Test-Driven Development at Longhorn PHP 2023
Getting Started with Test-Driven Development at Longhorn PHP 2023Getting Started with Test-Driven Development at Longhorn PHP 2023
Getting Started with Test-Driven Development at Longhorn PHP 2023
 
Static Code Analysis PHP[tek] 2023
Static Code Analysis PHP[tek] 2023Static Code Analysis PHP[tek] 2023
Static Code Analysis PHP[tek] 2023
 
Getting Started with Test-Driven Development at PHPtek 2023
Getting Started with Test-Driven Development at PHPtek 2023Getting Started with Test-Driven Development at PHPtek 2023
Getting Started with Test-Driven Development at PHPtek 2023
 
Getting Started with Test-Driven Development at Midwest PHP 2021
Getting Started with Test-Driven Development at Midwest PHP 2021Getting Started with Test-Driven Development at Midwest PHP 2021
Getting Started with Test-Driven Development at Midwest PHP 2021
 
Developing a Culture of Quality Code at Givelify (Tech Talk)
Developing a Culture of Quality Code at Givelify (Tech Talk)Developing a Culture of Quality Code at Givelify (Tech Talk)
Developing a Culture of Quality Code at Givelify (Tech Talk)
 
Developing a Culture of Quality Code (Midwest PHP 2020)
Developing a Culture of Quality Code (Midwest PHP 2020)Developing a Culture of Quality Code (Midwest PHP 2020)
Developing a Culture of Quality Code (Midwest PHP 2020)
 

Dernier

From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Enterprise Knowledge
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
Delhi Call girls
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Drew Madelung
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
Enterprise Knowledge
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Delhi Call girls
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel Araújo
 

Dernier (20)

What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 

SQL Database Design For Developers at php[tek] 2024

  • 1. SQL Database Design For Developers Scott Keck-Warren php[tek] 2024 @scottKeckWarren@phpc.social
  • 2. 1.Never Worked With a DBA On a Project 2.Every Project Has At Least One SQL Database
  • 3. 1.Never Worked With a DBA On a Project 2.Every Project Has At Least One SQL Database
  • 4. My (Early) Relationship to SQL Database: Less Than Ideal
  • 5. How Did I Get Better?
  • 7.
  • 8. Problems • “Random” slowness • Data inconsistencies • Weird Bugs
  • 9.
  • 10.
  • 12. Scott Keck-Warren Director of Technology @ WeCare Connect
  • 14. Scott Keck-Warren Director of Technology @ WeCare Connect
  • 15. Scott Keck-Warren Content Creator @php[architect] YouTube Community Corner Podcast
  • 16. Scott’s Rules For Database Design
  • 17. Never Worked With a DBA On a Project
  • 18. Scott’s Rules For Database Design 1. Normalize Your Database For Data Deduplication 2. Use The Database Engine to Keep Data Clean 3. Proactively Add Indexes to Keep Queries Performant
  • 20. Users Table • Email address • Password • Active state • Hire Date • Listing of previous passwords • Office Name • Office City • Office Zip
  • 21. Users Table • Email address (string) • Password (string) • Active state (string) • Hire Date (string) • Listing of previous passwords (string) • Office Name (string) • Office City (string) • Office Zip (string)
  • 22. Users Table email password active hire_date previous_ password office_name office_phone office_city office_zip alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 avery@exa mple.com NULL 1 8/11/2024 hash2 hash7 hash8 main office 5555555555 Saginaw 48609 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555-5555 Saginaw 48609 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
  • 23. Normalize Your Database For Data Deduplication
  • 24. Normalize Your Database For Data Deduplication “[T]he process of structuring a relational database in accordance with a series of so- called normal forms in order to reduce data redundancy and improve data integrity.” -“Database normalization” on Wikipedia
  • 25. Normalize Your Database For Data Deduplication • UNF: Unnormalized form • 1NF: First normal form • 2NF: Second normal form • 3NF: Third normal form • EKNF: Elementary key normal form • BCNF: Boyce–Codd normal form • 4NF: Fourth normal form • ETNF: Essential tuple normal form • 5NF: Fifth normal form • DKNF: Domain-key normal form • 6NF: Sixth normal form
  • 26. Normalize Your Database For Data Deduplication • UNF: Unnormalized form • 1NF: First normal form • 2NF: Second normal form • 3NF: Third normal form • EKNF: Elementary key normal form • BCNF: Boyce–Codd normal form • 4NF: Fourth normal form • ETNF: Essential tuple normal form • 5NF: Fifth normal form • DKNF: Domain-key normal form • 6NF: Sixth normal form
  • 27. Normalize Your Database For Data Deduplication • Boyce–Codd Normal Form: • X should be a superkey for every functional dependency (FD) X−>Y in a given relation.
  • 29. Unnormalized Form • A table doesn’t meet any of the conditions of normalization • Essentially a spreadsheet email password active hire_date previous_ password office_name office_phone office_city office_zip alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 avery@exa mple.com NULL 1 8/11/2024 hash2 hash7 hash8 main office 5555555555 Saginaw 48609 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555-5555 Saginaw 48609 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
  • 31. First Normal Form (1NF) 1. The table contains a unique identifier, also called the primary key, that is used to identify the row. 2. Each column contains atomic values (values that can not be broken down)
  • 32. 1NF - users email password active hire_date previous_p assword office_ name office_phone office_city office_zip alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 avery@exa mple.com NULL 1 8/11/2024 hash2 hash7 Hash8 main office 5555555555 Saginaw 48609 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555-5555 Saginaw 48609 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
  • 33. 1NF - users • A unique identifier should be: • Auto-incrementing int • UUID
  • 34. 1NF - users id email password active hire_date previous_ password office_ name office_phone office_cit y office_zip 1 alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 2 avery@ex ample.com NULL 1 8/11/2024 hash2 hash7 Hash8 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555- 5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
  • 35. 1NF - users id email password active hire_date previous_ password office_ name office_phone office_cit y office_zip 1 alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 2 avery@ex ample.com NULL 1 8/11/2024 hash2 hash7 hash8 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555- 5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
  • 38. 1NF - user_password_histories id user_id password 1 1 hash1 2 1 hash5 3 1 hash6 4 2 hash2 5 2 hash7 6 2 hash8 7 3 hash3 8 4 hash4
  • 39. 1NF - users id email password active hire_date previous_ password office_ name office_phone office_cit y office_zip 1 alice@exa mple.com hash1 1 1/1/2024 hash1 hash5 hash6 Main Office 555-555-5555 Saginaw 48609 2 avery@ex ample.com NULL 1 8/11/2024 hash2 hash7 Hash8 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 hash3 Man office (555)555- 5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday hash4 Main 555/555/5555 Saginaw 48609
  • 40. 1NF - users id email password active hire_date office_ name office_phone office_city office_zip 1 alice@exa mple.com hash1 1 1/1/2024 Main Office 555-555-5555 Saginaw 48609 2 avery@exa mple.com NULL 1 8/11/2024 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 Man office (555)555-5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday Main 555/555/5555 Saginaw 48609
  • 42. Second Normal Form (2NF) 1. Is already in 1NF 2. All the non-key columns are dependent on the primary key of the table
  • 43. Second Normal Form (2NF) id email password active hire_date office_ name office_phone office_city office_zip 1 alice@exa mple.com hash1 1 1/1/2024 Main Office 555-555-5555 Saginaw 48609 2 avery@exa mple.com NULL 1 8/11/2024 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 Man office (555)555-5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday Main 555/555/5555 Saginaw 48609
  • 44. 2nd - offices id name phone city zip 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609 3 Man office (555)555-5555 Saginaw 48609 4 Main 555/555/5555 Saginaw 48609
  • 45. 2NF - users id email password active hire_date office_ name office_phone office_city office_zip 1 alice@exa mple.com hash1 1 1/1/2024 Main Office 555-555-5555 Saginaw 48609 2 avery@exa mple.com NULL 1 8/11/2024 main office 5555555555 Saginaw 48609 3 scott@exa mple.com hash3 1 May 11th, 23 Man office (555)555-5555 Saginaw 48609 4 scott@exa mple.com hash4 1 Tuesday Main 555/555/5555 Saginaw 48609
  • 46. 2NF - users id email password active hire_date office_ name office_phone office_cit y office_zip office_id 1 alice@exa mple.com hash1 1 1/1/2024 Main Office 555-555-5555 Saginaw 48609 1 2 avery@ex ample.com NULL 1 8/11/2024 main office 5555555555 Saginaw 48609 2 3 scott@exa mple.com hash3 1 May 11th, 23 Man office (555)555- 5555 Saginaw 48609 3 4 scott@exa mple.com hash4 1 Tuesday Main 555/555/5555 Saginaw 48609 4
  • 47. 2NF - users id email password active hire_date office_id 1 alice@example.co m hash1 1 1/1/2024 1 2 avery@example.c om NULL 1 8/11/2024 2 3 scott@example.co m hash3 1 May 11th, 23 3 4 scott@example.co m hash4 1 Tuesday 4
  • 49. Third Normal Form (3NF) 1. Is already in 2NF 2. It contains columns that are non-transitively dependent on the primary key
  • 50. 3NF - offices id name phone city zip 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609 3 Man office (555)555-5555 Saginaw 48609 4 Main 555/555/5555 Saginaw 48609
  • 51. 3NF - zips id city 48609 Saginaw 48640 Midland 48642 Midland 48901 Lansing
  • 52. 3NF - zips id city state 48609 Saginaw MI 48640 Midland MI 48642 Midland MI 48901 Lansing MI
  • 55. Use The Database Engine to Keep Data Clean
  • 56. Why?
  • 57. Why?
  • 58.
  • 59. mysql> insert into users (password) values (“just a password?"); Query OK, 1 row affected (0.01 sec)
  • 60. mysql> insert into users (password) values (“just a password?"); Query OK, 1 row affected (0.01 sec)
  • 61. Garbage In -> Garbage Out
  • 63. To Prevent Bugs: Make The Database Work For Us
  • 64. id email password active hire_date office_id 1 alice@example.co m hash1 1 1/1/2024 1 2 avery@example.c om NULL 1 8/11/2024 2 3 scott@example.co m hash3 1 May 11th, 23 3 4 scott@example.co m hash4 1 Tuesday 4 5 Hash12 2 2024-04-01 1000
  • 66. Use Correct Column Types id email password active hire_date office_id 1 alice@example.co m hash1 1 1/1/2024 1 2 avery@example.c om NULL 1 8/11/2024 2 3 scott@example.co m hash3 1 May 11th, 23 3 4 scott@example.co m hash4 1 Tuesday 4 5 Hash12 2 2024-04-01 1000
  • 67. Use Correct Column Types • Numeric: INT, TINYINT, BIGINT, FLOAT, REAL, etc. • Date/Time: DATE, TIME, DATETIME, etc. • String: CHAR, VARCHAR, TEXT, etc. • Binary data types such as: BLOB, etc.
  • 70. Use Correct Column Types mysql> insert into users (hire_date) values ("tuesday"); ERROR 1292 (22007): Incorrect date value: 'tuesday' for column 'hire_date' at row 1 mysql> insert into users (hire_date) values ("May 11th, 23"); ERROR 1292 (22007): Incorrect date value: 'May 11th, 23' for column 'hire_date' at row 1
  • 71. Use NOT NULL for Required Fields
  • 72. Use NOT NULL for Required Fields mysql> insert into users (password) values (“just a password?"); Query OK, 1 row affected (0.01 sec)
  • 73. Use NOT NULL for Required Fields
  • 74. Use NOT NULL for Required Fields
  • 75. Use NOT NULL for Required Fields mysql> insert into users (password) values (“just a password?”); ERROR 1364 (HY000): Field ‘email' doesn't have a default value
  • 76. Use NOT NULL for Required Fields mysql> insert into users (password) values (“just a password?”); ERROR 1364 (HY000): Field ‘email' doesn't have a default value
  • 77. Use NOT NULL for Required Fields mysql> insert into users (email, password) values (“s@s”, "just a password?"); ERROR 1364 (HY000): Field 'active' doesn't have a default value
  • 78. Use UNIQUE for Unique Values
  • 79. Use UNIQUE for Unique Values mysql> insert into users (email) values ("scott@keck-warren.com"); Query OK, 1 row affected (0.01 sec) mysql> insert into users (email) values ("scott@keck-warren.com"); Query OK, 1 row affected (0.02 sec)
  • 80. Use UNIQUE for Unique Values
  • 81. Use UNIQUE for Unique Values
  • 82. Use UNIQUE for Unique Values mysql> insert into users (email) values ("scott@keck-warren.com"); ERROR 1062 (23000): Duplicate entry 'scott@keck-warren.com' for key 'users.email'
  • 83. Use UNIQUE for Unique Values
  • 84. Use Foreign Keys For References To Other Tables
  • 85. Use Foreign Keys For References To Other Tables id name phone city zip_id 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609 3 Man office (555)555-5555 Saginaw 48609 4 Main 555/555/5555 Saginaw 48609
  • 86. Use Foreign Keys For References To Other Tables id name phone city zip_id 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609 3 Man office (555)555-5555 Saginaw 48609
  • 87. Use Foreign Keys For References To Other Tables id name phone city zip_id 1 Main Office 555-555-5555 Saginaw 48609 2 main office 5555555555 Saginaw 48609
  • 88. Use Foreign Keys For References To Other Tables id name phone city zip_id 1 Main Office 555-555-5555 Saginaw 48609
  • 90.
  • 91.
  • 93.
  • 94. mysql> insert into users (email, office_id) values ("s@kw.com", 1000000); Query OK, 1 row affected (0.03 sec)
  • 95. mysql> insert into users (email, office_id) values ("s@kw.com", 1000000); Query OK, 1 row affected (0.03 sec)
  • 101. Foreign Key Constraints mysql> insert into users (email, office_id) values ("s@kw.com", 1000000); ERROR 1452 (23000): Cannot add or update a child row: a foreign key constraint fails (`databasetalk`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`office_id`) REFERENCES `offices` (`id`))
  • 102. Foreign Key Constraints mysql> insert into users (email, office_id) values ("s@kw.com", 1000000); ERROR 1452 (23000): Cannot add or update a child row: a foreign key constraint fails (`databasetalk`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`office_id`) REFERENCES `offices` (`id`))
  • 103. Foreign Key Constraints mysql> insert into users (email, office_id) values ("s@kw.com", 1); Query OK, 1 row affected (0.01 sec)
  • 104. Foreign Key Constraints mysql> delete from offices where id = 1; ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`databasetalk`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`office_id`) REFERENCES `offices` (`id`))
  • 105. Foreign Key Constraints mysql> delete from offices where id = 1; ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`databasetalk`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`office_id`) REFERENCES `offices` (`id`))
  • 109. Foreign Key Constraints mysql> insert into user_password_histories (user_id, password) values (2, "test1"); Query OK, 1 row affected (0.01 sec) mysql> insert into user_password_histories (user_id, password) values (2, "test2"); Query OK, 1 row affected (0.01 sec)
  • 110. Foreign Key Constraints mysql> delete from users where id = 2; Query OK, 1 row affected (0.01 sec) mysql> select * from user_password_histories where user_id = 2; Empty set (0.00 sec)
  • 111. Foreign Key Constraints mysql> delete from users where id = 2; Query OK, 1 row affected (0.01 sec) mysql> select * from user_password_histories where user_id = 2; Empty set (0.00 sec)
  • 116. Use Triggers For Complex Requirements
  • 117. Use Triggers For Complex Requirements • Triggers add additional power to DB • Operate based on create, update, or delete
  • 118. Use Triggers For Complex Requirements id email password active hire_date office_id 1 alice@example.co m hash1 1 1/1/2024 1 2 avery@example.c om NULL 1 8/11/2024 2 3 scott@example.co m hash3 1 May 11th, 23 3 4 scott@example.co m hash4 1 Tuesday 4 5 Hash12 2 2024-04-01 1000
  • 119. Use Triggers For Complex Requirements
  • 120. Use Triggers For Complex Requirements
  • 121. Use Triggers For Complex Requirements
  • 122. Use Triggers For Complex Requirements
  • 123. Use Triggers For Complex Requirements
  • 124. Use Triggers For Complex Requirements
  • 125. Use Triggers For Complex Requirements mysql> insert into users (active) values (2); ERROR 1644 (45000): active must be 0 or 1
  • 126. Use Triggers For Complex Requirements mysql> insert into users (active) values (2); ERROR 1644 (45000): active must be 0 or 1
  • 127. Proactively Add Indexes to Keep Queries Performant
  • 128.
  • 129.
  • 130.
  • 131.
  • 133.
  • 136. Indexes in Databases 2023-12-03 Index: hire_date 2023-12-04 2023-12-05 2023-12-06
  • 137.
  • 138.
  • 139.
  • 140.
  • 142. What You Need to Know
  • 143. What You Need to Know 1. Normalize Your Database For Data Deduplication 2. Use The Database Engine to Keep Data Clean 3. Proactively Add Indexes to Keep Queries Performant
  • 144. What You Need to Know 1. The table contains a unique identifier, also called the primary key, that is used to identify the row. 2. Each column contains atomic values (values that can not be broken down) 3. All the non-key columns are dependent on the primary key of the table 4. It contains columns that are non-transitively dependent on the primary key
  • 145. What You Need to Know • Make the DB Work With You • Correct Column Types • NOT NULL for Required Fields • UNIQUE for Unique Values • Foreign Keys For References To Other Tables • Triggers For Complex Requirements
  • 146. What You Need to Know • Use indexes on commonly searched columns • Start simple • See recorded talks about how to add
  • 147.
  • 150. Questions/Follow Me • Questions • Please rate the talk • <link> • @scottKeckWarren@phpc.social • @scottKeckWarren@twitter.com

Notes de l'éditeur

  1. Ask people for photos Good morning all! 2 Shocking facts
  2. Anyone else in this boat Like to think I’m good at working with DBs Know that there’s always something to learn
  3. Early in my journey: just threw data into it and it spit it back Might have been magic Core piece of technology that I don’t understand
  4. <slide> Didn’t have a more senior level developer who could mentor So I had to figure it out Not necessarily a bad thing because that’s how I work best
  5. Push a new feature users are initially happy But as usage grows we start finding problems
  6. Angry people Customers My boss Not ideal
  7. Results in me fixing things under distress Night/weekends Once in a bathroom at a holiday inn
  8. My goal is to have you learn from my thrama
  9. For those of you who haven’t met me my name is …
  10. Professional PHP Developer for 16 years // team lead/CTO role for 11 of those 16
  11. Currently Director of Technology at WeCare Connect Use PHP and mysql for our backend Also …
  12. That being said My goal for today give you <slide> These are the rules I give new hires so they can understand our teams design
  13. So we have to figure it out ourselves
  14. All of these rules exist to prevent bugs or performance problem
  15. Like examples Today’s example is <slide> from a project
  16. Initial version of this database as it existed when we took over project
  17. Track everything using a string
  18. Only going to talk about the first four forms today as the others are hard to understand and demo 1 and 2 give us huge bang for our buck and we start looking a demising returns around 3
  19. A table doesn’t meet any of the conditions of normalization Essentially a spreadsheet
  20. The table contains a unique identifier, also called the primary key, that is used to identify the row.
  21. Make it auto incrementing a primary key so the database knows how to handle it
  22. Each column contains atomic values (values that can not be broken down) To solve this we need to create another table
  23. A lot of normalization is fixed with more tables
  24. Now in 1NF Still a lot of duplication and mismatched data
  25. Review users table Three sections Primary key Second section - all related to that Third section - not related First X columns are dependent
  26. Fix? It’s a new table Create offices
  27. Link our offices table to the users table
  28. Link our offices table to the users table
  29. Drop all the office columns 2nf
  30. When columns are transitively dependent one column's data relies on another column through a third column. For example, our offices' city column is dependent on the zip column which is dependent on the office's id.
  31. To fix this we'll split out the zip in a new table.
  32. To fix this we'll split out the zip in a new table.
  33. As many validation rules as possible <slide has a bunch>
  34. Not going to prevent lazy me Right to the DB This is just hiding future bugs want to prevent that
  35. Not going to prevent lazy me Right to the DB This is just hiding future bugs want to prevent that
  36. Not going to prevent lazy me Right to the DB This is just hiding future bugs want to prevent that
  37. <slide> Not the other way around Let’s start with one of the most basic constraints
  38. Looking back at our users table Still issues: Blank emails, Date problem , Duplicate emails , Deleted Sites Problem We can and should enforce rules at application level but …
  39. Next thing: weird dates Want dates in the “correct” format Right now if someone asks for all the employees hired in 2023 getting that information will be a challenge Especially the person who starts on Tuesday
  40. List of all the types in mysql SQL has a ton of types to best fit our needs
  41. Switch this column to a date Reformat a little and we get consistent values Now easy to find everyone who start in 2023
  42. Switch this column to a date Reformat a little and we get consistent values Now easy to find everyone who start in 2023
  43. Might have required for field but Show insert missing email
  44. Might have required for field but Show insert missing email
  45. Embrace NOT NULL for required columns
  46. Embrace NOT NULL for required columns
  47. Embrace NOT NULL for required columns
  48. Embrace NOT NULL for required columns
  49. Example insert 2 users with same email and password
  50. Example insert 2 users with same email and password
  51. Embrase unique constraints Allows us to specify this column is unique Good for thing we never ever want to see two of email is the best option
  52. Embrase unique constraints Allows us to specify this column is unique Good for thing we never ever want to see two of email is the best option
  53. Can specify multiple columns for uniqueness Example: multi-tenant database could support email address uniqueness per office
  54. Gave users access to clean up offices So they started deleting the duplicates
  55. Gave users access to clean up offices So they started deleting the duplicates
  56. Gave users access to clean up offices So they started deleting the duplicates
  57. Gave users access to clean up offices So they started deleting the duplicates
  58. Deleted locations so the values don’t match
  59. This table is using a join which is breaking the results User as assigned to locations that no longer exist
  60. Users that belong to non-existent offices
  61. Need some way to say what’s valid
  62. Need some way to say what’s valid
  63. Allow us to define the relationship of one column to another table
  64. Allow us to define the relationship of one column to another
  65. Allow us to define the relationship of one column to another
  66. Allow us to define the relationship of one column to another
  67. Allow us to define the relationship of one column to another
  68. Allow us to define the relationship of one column to another
  69. Allow us to define the relationship of one column to another
  70. Allow us to define the relationship of one column to another
  71. Allow us to define the relationship of one column to another
  72. Allow us to define the relationship of one column to another
  73. Allow us to define the relationship of one column to another
  74. Allow us to define the relationship of one column to another
  75. Allow us to define the relationship of one column to another
  76. Allow us to define the relationship of one column to another
  77. Allow us to define the relationship of one column to another
  78. Allow us to define the relationship of one column to another
  79. Allow us to define the relationship of one column to another
  80. Allow us to define the relationship of one column to another
  81. Performance
  82. Not enough I ever worry about but each FK requires looks
  83. “Magic” according to some developers
  84. Active column can accept any integer value
  85. I also like this for complex requirement that a standard column doesn’t cover Ex: if a row is one type different fields are not null
  86. Indexes In Life I love to cook Love to try new recipes
  87. Leftover food from recipe Now get a neural network to figure out But could use cookbooks
  88. Option 1 Go through every page looking for matches Slow as most don’t meet our criteria
  89. Option 2 Go to back of book to the index and look up ingreditants Use that to look up recipes Much faster
  90. Same
  91. Database is going to look at every row Fine when you have 100 users Slow when you have 10 million
  92. We’re going to use indexes to tell the database common things we’re going to query on <click>
  93. For example, I’m going to search commonly on email and active so that’s a prime candidate
  94. For example, I’m going to search commonly on email and active so that’s a prime candidate
  95. For example, I’m going to search commonly on email and active so that’s a prime candidate
  96. For example, I’m going to search commonly on email and active so that’s a prime candidate
  97. All of these rules exist to prevent bugs or performance problem
  98. Thank the sponsors