With increasing levels of abstraction in frameworks, it is rare that web developers interact directly with databases. However, ORMs can slow seemingly simple queries down to a crawl. Instead of trying to maintain complex cache keys with the latest fashionable key-value store, let's revisit this 30 year old technology.
3. Megan Bowra-Dean !
• Rails/JS/Android/iOS/
kitchen sink developer
at Rabid Tech
• 2 years .NET
enterprise web dev
• 2 years C/C++
embedded dev
• Ruby NZ Committee
Member
11. Saves development time & easier to maintain
SELECT
*
FROM
books
INNER JOIN
libraries ON
books.library_id =
libraries.id
WHERE
libraries.name = 'Wellington
City Library'
Book
.joins('libraries')
.where(libraries: { name:
'Wellington City Library' })
VS
12. Security
• Most ORMs stop SQL injection attacks 💉
• Some restrict columns that can be updated by
user input (e.g. Rails 4’s strong_params)
15. Ultimately ORMs are an
Abstraction
• Simplified for specific use cases
• What happens when the relationships between
your models get more complex?
• What happens when you need data not tied to a
model’s fields? 📊
16. N+1 Problem
A book has an editor and magazines are a type of
book, how do we find all the editors belonging to
magazines to print them out?
📚'(
17. Naive Way
magazines = Book
.where(type: 'magazine')
magazines.each do |magazine|
puts magazine.editor
end
18. Resultant SQL
SELECT * FROM books WHERE type = 'magazine'
SELECT * FROM editors WHERE book_id = 1
SELECT * FROM editors WHERE book_id = 2
..
SELECT * FROM editors WHERE book_id = n
19. We end up with 1 + n queries, where n is the number
of magazines, hence the N+1 problem.
20. Optimised Way
magazines = Book
.includes('editor')
.where(type: 'magazine')
magazines.each do |magazine|
puts magazine.editor
end
21. Resultant SQL
SELECT * FROM books WHERE type = 'magazine'
SELECT * FROM editors WHERE (book_id IN (1,2,3..n))
22. • Not always as obvious as this
• Can have a lot of things happening between
fetching the parent model and the child models
• Still, (most) ORMs have the capability to help
24. • Page load times can slow down noticeably with
just a few thousand instances of models.
• May expect to run operations on hundreds of
thousands.
• Is it WEBSCALE?
25. We Usually Deal With This
by Over-engineering
• Adding extra caching layers
• Load balancing with horizontal scaling
• Progressive page loading
27. • We can wrest the raw database connection from
the ORM
• With this we can improve performance without
greatly increasing complexity
28. A real world example
• Web app for client that surveyed organisational
performance
• Produced an online report with several different
breakdowns of statistics from the survey
• Was surprisingly slow - hit web server timeout
30. • 100,532 calls to Class#new ‼
• A simple page was only creating ~900 objects
• One suspect was a function calculating the
average of responses to a group of questions (a
“domain”)
31. sum = 0.0
count = 0.0
domain.questions.each do |q|
response = q.response_for(respondent)
sum += response.value
count += 1.0
end
if count > 0
return sum / count
else
return 1.0
end
32. conn = ActiveRecord::Base.connection
result = conn.execute <<-SQL
SELECT
SUM(responses.value) as sum,
COUNT(*) as count
FROM
domains
INNER JOIN
questions ON questions.domain_id = domains.id
INNER JOIN
responses ON responses.question_id = questions.id
AND responses.respondent_id = #{respondent.id}
WHERE
domains.id = #{domain.id}
SQL
score = 1.0
sum = result[0]['sum'].to_f
count = result[0]['count'].to_f
score = sum / count if count > 0
33. • Reduced page load time by more than a half
• Reduced number of objects created by 30%
35. • Not as maintainable
• Need to keep an eye on security.
Never insert user provided values into raw SQL.
• Not as portable.
XKCD #327 https://xkcd.com/327/
36. sql = Cat.where(name: 'Ms Tibbles').to_sql
ActiveRecord::Base.connection.execute sql
Can sometimes get ORM
to help you defeat itself
40. To Summarise
• ORMs bring great benefits much of the time
• However being aware of what they’re doing is
essential
• If need be, it is possible to work around them.
• Databases are your friend.