This document provides an overview of composing queries with Ecto by: 1) Introducing Ecto models and basic queries. 2) Explaining that queries can be constructed separately from execution for flexibility. 3) Describing how queries can be composed by making one query the subject of another, allowing reusable query components to be defined and combined. 4) Framing queries as pipelines with sources, transformations, and sinks, noting pipelines can also be nested for more complex operations like pagination.

1. Composable Queries
with Ecto
Drew Olson
@drewolson

2. * Brief Ecto Intro
* Query Expressions
* Composition
* Query Pipelines

3. Please ask
questions

4. Intro

5. Patterns from real
applications

6. A few models for
examples

7. Intro - Models
defmodule MyApp.Post do
use Ecto.Model
import Ecto.Query
schema "posts" do
field :body, :string
field :published, :boolean
field :published_at, Ecto.Date
field :title, :string
has_many :comments, MyApp.Comment
end
end

8. Intro - Models
defmodule MyApp.Comment do
use Ecto.Model
import Ecto.Query
schema "comments" do
field :body, :string
field :commenter, :string
field :votes, :integer
belongs_to :post, MyApp.Post
end
end

9. Keyword query
syntax

10. Intro - Baby’s First Query
posts = MyApp.Repo.all(
from p in MyApp.Post,
where: p.published == true
)

11. This is how I started
writing Ecto queries
Intro

12. Separate construction
and execution
Intro

13. Intro - First Query Deconstructed
query = from p in MyApp.Post,
where: p.published == true
MyApp.Repo.all(query)

14. Intro - Fancy Query
query = from c in MyApp.Comment,
join: p in assoc(c, :post),
where: p.id == 1 and c.votes > 5
select: c
comments = MyApp.Repo.all(query)

15. Query Expressions

16. Query Expressions - Translation
query = from p in MyApp.Post,
where: p.published == true
query = where(MyApp.Post, [p], p.published == true)

17. Query Expressions - Fancy
query = from c in MyApp.Comment,
join: p in assoc(c, :post),
where: p.id == 1 and c.votes > 5
select: c
query = MyApp.Comment
|> join(:left, [c], p in assoc(c, :post))
|> where([_, p], p.id == 1)
|> where([c, _], c.votes > 5)
|> select([c, _], c)

18. Composition

19. Both query styles are
composable.
Queries can be the
“subject” of new
queries.
Composition

20. Composition
query1 = MyApp.Comment
query2 = from c in query1,
where: c.votes > 5
query3 = from c in query2,
join: p in assoc(c, :post),
where: p.id == 1,
select: c
MyApp.Repo.all(query3)

21. Composition
query1 = MyApp.Comment
query2 = query1
|> where([c], c.votes > 5)
query3 = query2
|> join(:left, [c], p in assoc(c, :post))
|> where([_, p], p.id == 1)
|> select([c, _], c)
MyApp.Repo.all(query3)

22. We can now extract
reusable components,
name them and compose
them.
Composition

23. Composition
defmodule MyApp.Comment do
...
def for_post(query, id) do
from c in query,
join: p in assoc(c, :post),
where: p.id == ^id,
select: c
end
def popular(query) do
from c in query,
where: c.votes > 5
end
end
alias MyApp.Comment
Comment
|> Comment.popular
|> Comment.for_post(1)
|> MyApp.Repo.all

24. Composition - Reads so nice :)
Comment
|> Comment.popular
|> Comment.for_post(1)
|> MyApp.Repo.all

25. Query Pipelines

26. Query Pipelines
Comment
|> Comment.popular
|> Comment.for_post(1)
|> MyApp.Repo.all

27. Query Pipelines
Comment
|> Comment.popular
|> Comment.for_post(1)
|> MyApp.Repo.all
<- source
<- transformation
<- transformation
<- sink

28. Query Pipelines - A note
Welcome to my typespecs,
where the types are all
made up and the points
don’t matter.

29. Query Pipelines - Source
A source is the starting
point for a query.
@spec source() :: Query.t

30. Query Pipelines - Source Examples
MyApp.Post
MyApp.Post.owned_by(user)

31. Query Pipelines - Transformation
A transformation expands
or constrains an
existing query.
@spec transformation(Query.t) :: Query.t

32. Query Pipelines - Transformation Examples
MyApp.Post.published(query)
MyApp.Comment.for_post(query, post)

33. Query Pipelines - Sink
A sink executes a query
and returns a result.
@spec sink(Query.t) :: Result.t

34. Query Pipelines - Sink Examples
MyApp.Repo.all(query)
MyApp.Repo.one(query)
MyApp.Repo.paginate(query)

35. Query Pipelines
Comment
|> Comment.popular
|> Comment.for_post(1)
|> MyApp.Repo.all

36. Query Pipelines
Pipelines are fractal

37. Query Pipelines - Fractal
MyApp.Repo.paginate(query)

38. Query Pipelines
Pagination acts as a
sink, but is really
several smaller
pipelines

39. defmodule MyApp.Repo do
def paginate(query, page_number 1) do
{entries(query, page_number), total_entries(query)}
end
defp entries(query, page_number) do
page_size = 10
offset = page_size * (page_number - 1)
query
|> limit([_], ^page_size)
|> offset([_], ^offset)
|> all
end
defp total_entries(query) do
query
|> exclude(:order_by)
|> exclude(:preload)
|> exclude(:select)
|> select([_], count("*"))
|> one
end
end

40. See Also
* blog.drewolson.org
* hex.pm/packages/scrivener

41. Fin
Thanks.
Questions?
@drewolson