Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

A Related Matter: Optimizing your webapp by using django-debug-toolbar, select_related() and prefetch_related()

Ad

A Related Matter: 
Optimizing your webapp by using django-debug-toolbar, 
select_related(), and prefetch_related() 
Christ...

Ad

Christopher Adams 
• Software Engineer at Venmo 
• Twitter/Github: @adamsc64 
• I’m not Chris Adams (@acdha), who works at...

Ad

Django is great 
But Django is really a set of tools

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Chargement dans…3
×

Consultez-les par la suite

1 sur 55 Publicité
1 sur 55 Publicité

A Related Matter: Optimizing your webapp by using django-debug-toolbar, select_related() and prefetch_related()

Télécharger pour lire hors ligne

This talk explains how to perform SQL query analysis and how to rewrite your views to reduce the number of queries Django uses in evaluating your model objects and their attributes. Special emphasis will be given to the powerful methods "select_related" and "prefetch_related." I will highlight the problem with a naive use of the ORM, how to target code for optimization, and the beneficial result.

Like any abstraction layer, the Django ORM hides the messy details about its underlying implementation. This is both the benefit and the risk. If used naively, any tool can cause unexpected or problematic outcomes. Likewise, the ORM can cause your application to interact with the database in an ugly and inefficient way, creating special challenges regarding scaling a quickly-prototyped webapp.

Many design patterns and best practices have been developed as a result to nudge developers to use the ORM more efficiently. The good news is, one of the easiest and most powerful patterns has been wrapped into Django itself, in the dual pairs of methods in the Django ORM's Queryset API, "select_related" and "prefetch_related." These methods instruct the Queryset, when evaluated, to perform two kinds of useful optimizations for you that can reduce the number of queries by orders of magnitude resulting from iterating over model objects and many-to-many relations.

This talk summarizes the problem these methods of the Queryset API try to solve, how to effectively use them, and the beneficial result. Mastering how to use Queryset methods efficiently and powerfully is a major step in moving from a beginner to intermediate Django developer.

Delivered at DjangoCon 2014.

This talk explains how to perform SQL query analysis and how to rewrite your views to reduce the number of queries Django uses in evaluating your model objects and their attributes. Special emphasis will be given to the powerful methods "select_related" and "prefetch_related." I will highlight the problem with a naive use of the ORM, how to target code for optimization, and the beneficial result.

Like any abstraction layer, the Django ORM hides the messy details about its underlying implementation. This is both the benefit and the risk. If used naively, any tool can cause unexpected or problematic outcomes. Likewise, the ORM can cause your application to interact with the database in an ugly and inefficient way, creating special challenges regarding scaling a quickly-prototyped webapp.

Many design patterns and best practices have been developed as a result to nudge developers to use the ORM more efficiently. The good news is, one of the easiest and most powerful patterns has been wrapped into Django itself, in the dual pairs of methods in the Django ORM's Queryset API, "select_related" and "prefetch_related." These methods instruct the Queryset, when evaluated, to perform two kinds of useful optimizations for you that can reduce the number of queries by orders of magnitude resulting from iterating over model objects and many-to-many relations.

This talk summarizes the problem these methods of the Queryset API try to solve, how to effectively use them, and the beneficial result. Mastering how to use Queryset methods efficiently and powerfully is a major step in moving from a beginner to intermediate Django developer.

Delivered at DjangoCon 2014.

Plus De Contenu Connexe

Les utilisateurs ont également aimé (20)

A Related Matter: Optimizing your webapp by using django-debug-toolbar, select_related() and prefetch_related()

  1. 1. A Related Matter: Optimizing your webapp by using django-debug-toolbar, select_related(), and prefetch_related() Christopher Adams DjangoCon 2014 https://github.com/adamsc64/a-­‐related-­‐matter
  2. 2. Christopher Adams • Software Engineer at Venmo • Twitter/Github: @adamsc64 • I’m not Chris Adams (@acdha), who works at Library of Congress • Neither of us are “The Gentleman” Chris Adams (90’s-era Professional Wrestler)
  3. 3. Django is great But Django is really a set of tools
  4. 4. Tools are great But tools can be used in good or bad ways
  5. 5. The Django ORM: A set of tools
  6. 6. Manage your own expectations for tools • Many people approach a new tool with broad set of expectations as to what the think it will do for them. • This may have little correlation with what the project actually has implemented.
  7. 7. As amazing as it would be if they did…
  8. 8. Unicorns don’t exist
  9. 9. The Django ORM: An abstraction layer
  10. 10. Abstraction layers • Great because they take us away from the messy details • Risky because they take us away from the messy details
  11. 11. Don’t forget You’re far from the ground
  12. 12. The QuerySet API
  13. 13. QuerySets are Lazy
  14. 14. QuerySets are Immutable
  15. 15. Lazy: Does not evaluate until it needs to
  16. 16. Immutable: Never itself changes
  17. 17. Each a new QuerySet, none hit the database • queryset = Model.objects.all() • queryset = queryset.filter(...) • queryset = queryset.values(...)
  18. 18. Hits the database (QuerySet is “evaluated”): • queryset = list(queryset) • queryset = queryset[:] • for model_object in queryset: do_something(...)
  19. 19. Our app: a blog
  20. 20. Models class Blog(models.Model):! submitter = models.ForeignKey('auth.User')! ! class Post(models.Model):! blog = models.ForeignKey('blog.Blog', related_name="posts")! likers = models.ManyToManyField('auth.User')! ! class PostComment(models.Model):! submitter = models.ForeignKey('auth.User')! post = models.ForeignKey('blog.Post', related_name="comments")!
  21. 21. List View def blog_list(request):! blogs = Blog.objects.all()! return render(request, "blog/blog_list.html", {! "blogs": blogs,! })!
  22. 22. List Template
  23. 23. Detail View def blog_detail(request, blog_id):! blog = get_object_or_404(Blog, id=blog_id)! posts = Post.objects.filter(blog=blog)! return render(request, "blog/blog_detail.html", {! "blog": blog,! "posts": posts,! })!
  24. 24. Detail Template
  25. 25. SQL Queries?
  26. 26. If you can’t measure it…
  27. 27. …you’d never know if there are problems.
  28. 28. First view: The blog list page
  29. 29. select_related() • select_related works by creating an SQL join and including the fields of the related object in the SELECT statement. • For this reason, select_related gets the related objects in the same database query. • However, to avoid the much larger result set that would result from joining across a ‘many’ relationship, select_related is limited to single-valued relationships - foreign key and one-to-one.
  30. 30. List View def blog_list(request):! blogs = Blog.objects.all()! blogs = blogs.select_related("submitter")! return render(request, "blog/blog_list.html", {! "blogs": blogs,! })!
  31. 31. Second view: The blog detail page
  32. 32. prefetch_related() • prefetch_related does a separate lookup for each relationship, and does the ’joining’ in Python. • This allows it to prefetch many-to-many and many-to-one objects … in addition to the foreign key and one-to-one relationships. • It also supports prefetching of GenericRelation and GenericForeignKey.
  33. 33. Detail View def blog_detail(request, blog_id):! blog = get_object_or_404(Blog, id=blog_id)! posts = Post.objects.filter(blog=blog)! posts = posts.prefetch_related(! “comments__submitter", "likers",! )! return render(request, "blog/blog_detail.html", {! "blog": blog,! "posts": posts,! })!
  34. 34. Summary • The QuerySet API methods select_related() and prefetch_related() automate some best practices to avoid extra queries in views/templates. • Use select_related() for one-to-many or one-to-one relations. • Use prefetch_related() for many-to-many or many-to-one (e.g. reverse foreign key) relations.
  35. 35. Thanks! Christopher Adams (@adamsc64) https://github.com/adamsc64/a-related-matter

×