SlideShare une entreprise Scribd logo
1  sur  48
Télécharger pour lire hors ligne
SHARING DATA IN A
MULTITENANT ARCHITECTURE
DAN FEY
https://github.com/crowdskout/laravel-shared-multitenancy
https://joind.in/talk/71f10
About Dan Fey
Grew up in northern NJ 

Back-end engineer at Crowdskout in DC

Work on API and data layers in PHP using Laravel

Work with MySQL, Mongo, and Elasticsearch
Crowdskout
Data collection, analytics, and outreach SaaS
platform

Collects, normalizes, and matches data for
customers

Provides tools for segmenting audiences, building
dynamic charting, and acting on segments
In this talk
Foundations of Multitenancy

Data sharing with Multitenancy

Laravel/MySQL application demonstrating data
sharing with Multitenancy
Multitenant CRM/Contact List
Build a scalable, self-serve CRM application

View profiles/contacts

Allow customers to share subsets of profiles

Customers have multiple users that can view
their data
Multitenancy
Sharing resources among multiple tenants (i.e.
accounts/customers/users)

Can save on infrastructure, development,
maintenance

Great for scaling

Three general forms: separate databases,
separate schema, shared schema.
No Multitenancy
Separate Databases
Separate database servers per tenant

Application shared

Can extend/change database/schema per tenant

Adding tenants requires infrastructure

Each tenant has isolated database resources
Separate Databases
Separate Schemas
Same database server, different schemas per
tenant

Application shared

Can extend/change schema per tenant, but not
database type/configuration

Less resource isolation

Adding tenants may require operations/migrations
Separate Schemas
Shared Schema
Shared application, database server, and schema
for all tenants

Harder to extend/change schema or database per
tenant

Harder to isolate resources

Adding tenants requires minimal changes
Shared Schema
Shared Schema
Shape Food Plant Tenant
Sharing With Multitenancy
Multiple tenants share subsets of data

Sharing configurable and non permanent

Tenants share changes to data
Sharing With Multitenancy
Shape Food Plant Tenant
Multitenant CRM/Contact List
Build a scalable, self-serve CRM application

View profiles/contacts

Allow customers to share subsets of profiles

Customers have multiple users that can view
their data
Demo 1
Create Customer table and model

Create Profile, Name, Email tables and models

Populate Users, Customers, and Profiles 

Login to app
48d18621
Migrations
Models
Controller
Pivoting
Customers with Users
user_id name
1 Dan
2 Bob
3 Amy
customer_id user_id
Google 5 1 Dan
Apple 6 1 Dan
Apple 6 2 Bob
Google 5 3 Amy
customer_id name
5 Google
6 Apple
Demo 2
Add customer_user pivot table and model

Add belongs to many relationships

php artisan db:seed --class=CustomerUserSeeder
Migration
customers customer_user users
Sources Data Model
profile_id first last source_id
1 Kenyon Harris 2
2 Bette Kemmer 1
3 Oscar Gorczany 1
profile_id email source_id
1 kharris@test.com 1
1 kharris@example.com 2
2 bkemmer@test.com 1
3 ogorcz@test.com 1
source_id name
1 Imported
2 Web Form
Pivoting
Customers with Sources
customer_id source_id
Apple 6 1 Apple Imported
Apple 6 3 Public Data
Google 5 3 Public Data
customer_id name
5 Google
6 Apple
source_id name
1 Apple Imported
2 Google Forms
3 Public Data
Demo 3
Add sources table, add source_id to profile data
tables

Adding query scopes for access control

Show access filtering

Show adding/removing access
fd0eafa
Constraining sources
customer_user
Advantages of Sources
Data Model
Instantly share/remove large subsets of data

Updates to data points shared between customers

Data sources are isolated for change/cleaning/
deleting

Granular to row level

Model is scalable
Issues with Sources Data
Model
Duplicate data

Requires matching

Application code can get complex
Code tweaks for scalability
Cache requests to high use tables and queries

users

sources

client_users

client_sources

Add scrolling support with LIMIT and OFFSET

Use Query Builder instead of Eloquent or even raw queries
when necessary
Caching example
Extending - additional
profile tables
Examples: date of birth, gender, phone numbers

Add new table with new fields including profile id
and source id

Add new info to the views to display

Also works with activities like emailings, phone
calls, or page views
Additional profile tables
date_of_births
prf_profiles_id int
date_of_birth datetime
source_id int
phones
prf_profiles_id int
phone_number string
source_id int
genders
prf_profiles_id int
gender
enum('male',
'female', 'other')
source_id int
phone_calls
prf_profiles_id int
customer_id int
contact_datetime datetime
phone_number string
source_id int
QUESTIONS
https://github.com/crowdskout/laravel-shared-multitenancy
Get slides and leave feedback: https://joind.in/talk/71f10
Extending - add user
permissioning
Examples: some users can only see names, not
email addresses

Add new table for permissions, and a pivot table
for user_permissions

Check permissions before querying data to limit
access to certain tables
User Permissions
user_id permission_id
George 6 1 View Emails
George 6 2 View Names
Dan 5 2 View Names
user_id name
5 Dan
6 George
permission_id name
1 View Emails
2 View Names
Extending - adding search
capabilities
Examples: search for profiles by name, email, etc.

Possible on smaller scales, but full-text search is
difficult to scale with MySQL

Elasticsearch is great for this, but adds complexity

Data must be replicated into Elasticsearch and
kept in-sync

Does not use SQL for querying

Contenu connexe

Similaire à Sharing data in a multitenant architecture

Summer '16 Realease notes
Summer '16 Realease notesSummer '16 Realease notes
Summer '16 Realease notesaggopal1011
 
14.02.2017 Business model innovation slides
14.02.2017 Business model innovation slides14.02.2017 Business model innovation slides
14.02.2017 Business model innovation slidesGODAN Secretariat
 
How to govern and secure a Data Mesh?
How to govern and secure a Data Mesh?How to govern and secure a Data Mesh?
How to govern and secure a Data Mesh?confluent
 
Automating your tasks with microsoft flow
Automating your tasks with microsoft flowAutomating your tasks with microsoft flow
Automating your tasks with microsoft flowDipti Chhatrapati
 
Workshop on requirements and modeling at HAE 2015
Workshop on requirements and modeling at HAE 2015Workshop on requirements and modeling at HAE 2015
Workshop on requirements and modeling at HAE 2015Olivier Béghain
 
Real World API Design Using The Entity Framework Services
Real World API Design Using The Entity Framework ServicesReal World API Design Using The Entity Framework Services
Real World API Design Using The Entity Framework ServicesDavid McCarter
 
Paris Salesforce Developer Group - 16 09 2014 - Summer '14
Paris Salesforce Developer Group - 16 09 2014 - Summer '14Paris Salesforce Developer Group - 16 09 2014 - Summer '14
Paris Salesforce Developer Group - 16 09 2014 - Summer '14Paris Salesforce Developer Group
 
SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from At...
SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from At...SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from At...
SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from At...David J Rosenthal
 
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...DianaGray10
 
April 8th: Module Builder & Studio
April 8th: Module Builder & StudioApril 8th: Module Builder & Studio
April 8th: Module Builder & Studioticomixcrm
 
SugarCRM Power Hour with Jeff Bickart; Module Builder and Studio a developers...
SugarCRM Power Hour with Jeff Bickart; Module Builder and Studio a developers...SugarCRM Power Hour with Jeff Bickart; Module Builder and Studio a developers...
SugarCRM Power Hour with Jeff Bickart; Module Builder and Studio a developers...Jeff Bickart
 
MAX 2008: Build collaborative applications with Flex LCDS and Cairngorm
MAX 2008: Build collaborative applications with Flex LCDS and CairngormMAX 2008: Build collaborative applications with Flex LCDS and Cairngorm
MAX 2008: Build collaborative applications with Flex LCDS and CairngormXavier Agnetti
 
One Web (API?) – Alexandre Bertails - Ippevent 10 juin 2014
One Web (API?) – Alexandre Bertails - Ippevent 10 juin 2014One Web (API?) – Alexandre Bertails - Ippevent 10 juin 2014
One Web (API?) – Alexandre Bertails - Ippevent 10 juin 2014Ippon
 

Similaire à Sharing data in a multitenant architecture (20)

Summer '16 Realease notes
Summer '16 Realease notesSummer '16 Realease notes
Summer '16 Realease notes
 
14.02.2017 Business model innovation slides
14.02.2017 Business model innovation slides14.02.2017 Business model innovation slides
14.02.2017 Business model innovation slides
 
slides.pdf
slides.pdfslides.pdf
slides.pdf
 
How to govern and secure a Data Mesh?
How to govern and secure a Data Mesh?How to govern and secure a Data Mesh?
How to govern and secure a Data Mesh?
 
Automating your tasks with microsoft flow
Automating your tasks with microsoft flowAutomating your tasks with microsoft flow
Automating your tasks with microsoft flow
 
Workshop on requirements and modeling at HAE 2015
Workshop on requirements and modeling at HAE 2015Workshop on requirements and modeling at HAE 2015
Workshop on requirements and modeling at HAE 2015
 
Real World API Design Using The Entity Framework Services
Real World API Design Using The Entity Framework ServicesReal World API Design Using The Entity Framework Services
Real World API Design Using The Entity Framework Services
 
Paris Salesforce Developer Group - 16 09 2014 - Summer '14
Paris Salesforce Developer Group - 16 09 2014 - Summer '14Paris Salesforce Developer Group - 16 09 2014 - Summer '14
Paris Salesforce Developer Group - 16 09 2014 - Summer '14
 
SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from At...
SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from At...SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from At...
SQL Server 2014 Faster Insights from Any Data -Level 300 Presentation from At...
 
29.4 mb
29.4 mb29.4 mb
29.4 mb
 
29.4 Mb
29.4 Mb29.4 Mb
29.4 Mb
 
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...
 
April 8th: Module Builder & Studio
April 8th: Module Builder & StudioApril 8th: Module Builder & Studio
April 8th: Module Builder & Studio
 
SugarCRM Power Hour with Jeff Bickart; Module Builder and Studio a developers...
SugarCRM Power Hour with Jeff Bickart; Module Builder and Studio a developers...SugarCRM Power Hour with Jeff Bickart; Module Builder and Studio a developers...
SugarCRM Power Hour with Jeff Bickart; Module Builder and Studio a developers...
 
MAX 2008: Build collaborative applications with Flex LCDS and Cairngorm
MAX 2008: Build collaborative applications with Flex LCDS and CairngormMAX 2008: Build collaborative applications with Flex LCDS and Cairngorm
MAX 2008: Build collaborative applications with Flex LCDS and Cairngorm
 
Intro to GraphQL
Intro to GraphQLIntro to GraphQL
Intro to GraphQL
 
One Web (API?) – Alexandre Bertails - Ippevent 10 juin 2014
One Web (API?) – Alexandre Bertails - Ippevent 10 juin 2014One Web (API?) – Alexandre Bertails - Ippevent 10 juin 2014
One Web (API?) – Alexandre Bertails - Ippevent 10 juin 2014
 
ChandraSekhar CV
ChandraSekhar CVChandraSekhar CV
ChandraSekhar CV
 
Bring Your Data Model Alive with Automation - Data Modeling Zone Europe 2018
Bring Your Data Model Alive with Automation - Data Modeling Zone Europe 2018 Bring Your Data Model Alive with Automation - Data Modeling Zone Europe 2018
Bring Your Data Model Alive with Automation - Data Modeling Zone Europe 2018
 
Exploring the Salesforce REST API
Exploring the Salesforce REST APIExploring the Salesforce REST API
Exploring the Salesforce REST API
 

Dernier

Networking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxNetworking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxHimangsuNath
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our WorldEduminds Learning
 
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Boston Institute of Analytics
 
SMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxSMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxHaritikaChhatwal1
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfblazblazml
 
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxThe Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxTasha Penwell
 
Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Cathrine Wilhelmsen
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Seán Kennedy
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...Jack Cole
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxaleedritatuxx
 
Semantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxSemantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxMike Bennett
 
Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Seán Kennedy
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksdeepakthakur548787
 
Real-Time AI Streaming - AI Max Princeton
Real-Time AI  Streaming - AI Max PrincetonReal-Time AI  Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max PrincetonTimothy Spann
 
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Boston Institute of Analytics
 
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfWorld Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfsimulationsindia
 
Principles and Practices of Data Visualization
Principles and Practices of Data VisualizationPrinciples and Practices of Data Visualization
Principles and Practices of Data VisualizationKianJazayeri1
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelBoston Institute of Analytics
 

Dernier (20)

Networking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxNetworking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptx
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our World
 
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
 
SMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxSMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptx
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
 
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxThe Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
 
Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...
 
Data Analysis Project: Stroke Prediction
Data Analysis Project: Stroke PredictionData Analysis Project: Stroke Prediction
Data Analysis Project: Stroke Prediction
 
Insurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis ProjectInsurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis Project
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
 
Semantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxSemantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptx
 
Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing works
 
Real-Time AI Streaming - AI Max Princeton
Real-Time AI  Streaming - AI Max PrincetonReal-Time AI  Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max Princeton
 
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
 
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfWorld Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
 
Principles and Practices of Data Visualization
Principles and Practices of Data VisualizationPrinciples and Practices of Data Visualization
Principles and Practices of Data Visualization
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
 

Sharing data in a multitenant architecture

  • 1. SHARING DATA IN A MULTITENANT ARCHITECTURE DAN FEY https://github.com/crowdskout/laravel-shared-multitenancy https://joind.in/talk/71f10
  • 2. About Dan Fey Grew up in northern NJ Back-end engineer at Crowdskout in DC Work on API and data layers in PHP using Laravel Work with MySQL, Mongo, and Elasticsearch
  • 3. Crowdskout Data collection, analytics, and outreach SaaS platform Collects, normalizes, and matches data for customers Provides tools for segmenting audiences, building dynamic charting, and acting on segments
  • 4. In this talk Foundations of Multitenancy Data sharing with Multitenancy Laravel/MySQL application demonstrating data sharing with Multitenancy
  • 5. Multitenant CRM/Contact List Build a scalable, self-serve CRM application View profiles/contacts Allow customers to share subsets of profiles Customers have multiple users that can view their data
  • 6. Multitenancy Sharing resources among multiple tenants (i.e. accounts/customers/users) Can save on infrastructure, development, maintenance Great for scaling Three general forms: separate databases, separate schema, shared schema.
  • 8. Separate Databases Separate database servers per tenant Application shared Can extend/change database/schema per tenant Adding tenants requires infrastructure Each tenant has isolated database resources
  • 10. Separate Schemas Same database server, different schemas per tenant Application shared Can extend/change schema per tenant, but not database type/configuration Less resource isolation Adding tenants may require operations/migrations
  • 12. Shared Schema Shared application, database server, and schema for all tenants Harder to extend/change schema or database per tenant Harder to isolate resources Adding tenants requires minimal changes
  • 14. Shared Schema Shape Food Plant Tenant
  • 15.
  • 16. Sharing With Multitenancy Multiple tenants share subsets of data Sharing configurable and non permanent Tenants share changes to data
  • 17. Sharing With Multitenancy Shape Food Plant Tenant
  • 18. Multitenant CRM/Contact List Build a scalable, self-serve CRM application View profiles/contacts Allow customers to share subsets of profiles Customers have multiple users that can view their data
  • 19.
  • 20. Demo 1 Create Customer table and model Create Profile, Name, Email tables and models Populate Users, Customers, and Profiles Login to app 48d18621
  • 22.
  • 25.
  • 26. Pivoting Customers with Users user_id name 1 Dan 2 Bob 3 Amy customer_id user_id Google 5 1 Dan Apple 6 1 Dan Apple 6 2 Bob Google 5 3 Amy customer_id name 5 Google 6 Apple
  • 27. Demo 2 Add customer_user pivot table and model Add belongs to many relationships php artisan db:seed --class=CustomerUserSeeder
  • 29.
  • 30.
  • 31. Sources Data Model profile_id first last source_id 1 Kenyon Harris 2 2 Bette Kemmer 1 3 Oscar Gorczany 1 profile_id email source_id 1 kharris@test.com 1 1 kharris@example.com 2 2 bkemmer@test.com 1 3 ogorcz@test.com 1 source_id name 1 Imported 2 Web Form
  • 32. Pivoting Customers with Sources customer_id source_id Apple 6 1 Apple Imported Apple 6 3 Public Data Google 5 3 Public Data customer_id name 5 Google 6 Apple source_id name 1 Apple Imported 2 Google Forms 3 Public Data
  • 33. Demo 3 Add sources table, add source_id to profile data tables Adding query scopes for access control Show access filtering Show adding/removing access fd0eafa
  • 34.
  • 36.
  • 38.
  • 39. Advantages of Sources Data Model Instantly share/remove large subsets of data Updates to data points shared between customers Data sources are isolated for change/cleaning/ deleting Granular to row level Model is scalable
  • 40. Issues with Sources Data Model Duplicate data Requires matching Application code can get complex
  • 41. Code tweaks for scalability Cache requests to high use tables and queries users sources client_users client_sources Add scrolling support with LIMIT and OFFSET Use Query Builder instead of Eloquent or even raw queries when necessary
  • 43. Extending - additional profile tables Examples: date of birth, gender, phone numbers Add new table with new fields including profile id and source id Add new info to the views to display Also works with activities like emailings, phone calls, or page views
  • 44. Additional profile tables date_of_births prf_profiles_id int date_of_birth datetime source_id int phones prf_profiles_id int phone_number string source_id int genders prf_profiles_id int gender enum('male', 'female', 'other') source_id int phone_calls prf_profiles_id int customer_id int contact_datetime datetime phone_number string source_id int
  • 46. Extending - add user permissioning Examples: some users can only see names, not email addresses Add new table for permissions, and a pivot table for user_permissions Check permissions before querying data to limit access to certain tables
  • 47. User Permissions user_id permission_id George 6 1 View Emails George 6 2 View Names Dan 5 2 View Names user_id name 5 Dan 6 George permission_id name 1 View Emails 2 View Names
  • 48. Extending - adding search capabilities Examples: search for profiles by name, email, etc. Possible on smaller scales, but full-text search is difficult to scale with MySQL Elasticsearch is great for this, but adds complexity Data must be replicated into Elasticsearch and kept in-sync Does not use SQL for querying