Developers deal with SQL databases daily, yet very few speak SQL fluently. Even when an ORM hides most of SQL's details, SQL fluency will allow for increased performance and better troubleshooting capabilities. Given at UTOSC 2009

1. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
Fun With SQL INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
Other Useful
Joshua Tolley Operations
Subqueries
End Point Corporation Set Operations
Common Operations
Advanced
Operations
Common Table
October 5, 2009 Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

2. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
”The degree of normality in a database is inversely Self Joins
proportional to that of its DBA.” - Anon, twitter Other Useful
Operations
Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

3. Fun With SQL
Why Not Do Stuff in SQL
Joshua Tolley
End Point
Databases are harder to replicate, if you really need to Corporation
scale out
Why and Why Not
Often, one complex SQL query is more efficient than
Joins
several simple ones CROSS JOIN
Sometimes, indeed, it’s useful to reduce the load on the INNER JOIN
OUTER JOIN
database by moving logic into the application. Be NATURAL JOIN
careful doing this Self Joins
Other Useful
c.f. Premature Optimization Operations
Subqueries
More complex queries are harder to write and debug Set Operations
Common Operations
True. But so is more complex programming.
Advanced
Operations
More complex queries are harder for the next guy to Common Table
Expressions
maintain Window Functions
Also, good DBAs are often more expensive than good Real, Live Queries
Something Simple
programmers Something Fun
These are both true. But complex programming is also Key Points
hard for the next guy to maintain
Of all the reasons not to write fluent SQL, this is
probably the most widely applicable

4. Fun With SQL
Why do stuff in SQL?
Joshua Tolley
End Point
Corporation
Why and Why Not
The database is more efficient than your application for
Joins
processing big chunks of data CROSS JOIN
INNER JOIN
...especially if your code is in an interpreted language OUTER JOIN
NATURAL JOIN
The database is better tested than your application Self Joins
Applications trying to do what SQL should be doing Other Useful
Operations
often get big and complex quickly Subqueries
Set Operations
...and also buggy quickly Common Operations
That’s what the database is there for Advanced
Operations
SQL is designed to express relations and conditions on Common Table
Expressions
Window Functions
them. Your application’s language isn’t.
Real, Live Queries
A better understanding of SQL allows you to write Something Simple
Something Fun
queries that perform better Key Points

5. Fun With SQL
Why do stuff in SQL?
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
In short, the database exists to manage data, and your Other Useful
Operations
application exists to handle business logic. Write software Subqueries
accordingly. Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

6. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
So let’s get started... Other Useful
Operations
Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

7. Fun With SQL
Tables we’ll use
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
# SELECT * FROM a; # SELECT * FROM b; INNER JOIN
OUTER JOIN
NATURAL JOIN
id | value id | value Self Joins
----+------- ----+------- Other Useful
Operations
1 | a1 5 | b5 Subqueries
Set Operations
2 | a2 4 | b4 Common Operations
3 | a3 3 | b3 Advanced
Operations
4 | a4 6 | b6 Common Table
Expressions
(4 rows) (4 rows) Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

8. Fun With SQL
JOINs
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
If you want data from multiple tables, you probably Self Joins
Other Useful
want a join Operations
...but see also Subqueries, later on Subqueries
Set Operations
Common Operations
There are several different kinds of joins Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

9. Fun With SQL
JOINs
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
<table1> [alias1] OUTER JOIN
[ [ [NATURAL] [ [FULL | RIGHT | LEFT] [OUTER] | NATURAL JOIN
Self Joins
INNER] ] | CROSS ] JOIN Other Useful
Operations
<table2> [alias2] Subqueries
Set Operations
[USING (...) | Common Operations
ON (<value1> <op> <value2> Advanced
Operations
[,<value3> <op> <value4>...] ) ] Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

10. Fun With SQL
CROSS JOIN
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
SELECT <... >FROM table1 JOIN table2 CROSS JOIN
INNER JOIN
With no explicit join type and no join qualifiers (an ON OUTER JOIN
NATURAL JOIN
clause, WHERE clause involving both relations, etc.) Self Joins
this is a CROSS JOIN Other Useful
Operations
Equivalent to Subqueries
Set Operations
Common Operations
SELECT <... >FROM table1, table2
Advanced
SELECT <... >FROM table1 CROSS JOIN table2 Operations
Common Table
”Cartesian product” of the two relations Expressions
Window Functions
Combines every row of table1 with every row of table2 Real, Live Queries
Makes LOTS of rows, and can thus be very slow Something Simple
Something Fun
Key Points

11. Fun With SQL
CROSS JOIN
Joshua Tolley
End Point
Corporation
# SELECT * FROM a, b; Why and Why Not
id | value | id | value Joins
CROSS JOIN
----+-------+----+------- INNER JOIN
OUTER JOIN
1 | a1 | 5 | b5 NATURAL JOIN
Self Joins
1 | a1 | 4 | b4 Other Useful
<snip> Operations
Subqueries
3 | a3 | 3 | b3 Set Operations
Common Operations
3 | a3 | 6 | b6 Advanced
Operations
4 | a4 | 5 | b5 Common Table
Expressions
4 | a4 | 4 | b4 Window Functions
4 | a4 | 3 | b3 Real, Live Queries
Something Simple
4 | a4 | 6 | b6 Something Fun
(16 rows) Key Points

12. Fun With SQL
INNER JOIN
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
SELECT <...>FROM table1 INNER JOIN table2 ON Self Joins
Other Useful
(table1.field = table2.field ...) Operations
Subqueries
Only returns rows satisfying the ON condition Set Operations
Common Operations
Equivalent to a CROSS JOIN with a WHERE clause Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

13. Fun With SQL
INNER JOIN
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
# SELECT * FROM a INNER JOIN b USING (id); NATURAL JOIN
Self Joins
id | value | value
Other Useful
----+-------+------- Operations
Subqueries
3 | a3 | b3 Set Operations
Common Operations
4 | a4 | b4
Advanced
(2 rows) Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

14. Fun With SQL
OUTER JOIN
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
Return all rows from one or both relations CROSS JOIN
INNER JOIN
OUTER JOIN
LEFT: Return all rows from the relation on the left NATURAL JOIN
Self Joins
RIGHT: Return all rows from the relation on the right Other Useful
Operations
FULL: Return all rows from both relations Subqueries
Set Operations
Returns nulls for values from one relation when it Common Operations
Advanced
contains to match with the other relation Operations
Common Table
The OUTER keyword is redundant Expressions
Window Functions
Requires ON or USING clause Real, Live Queries
Something Simple
Something Fun
Key Points

15. Fun With SQL
LEFT JOIN
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
# SELECT * FROM a LEFT JOIN b USING (id); INNER JOIN
OUTER JOIN
id | value | value NATURAL JOIN
Self Joins
----+-------+------- Other Useful
1 | a1 | Operations
Subqueries
2 | a2 | Set Operations
Common Operations
3 | a3 | b3 Advanced
Operations
4 | a4 | b4 Common Table
Expressions
(4 rows) Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

16. Fun With SQL
RIGHT JOIN
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
# SELECT * FROM a RIGHT JOIN b USING (id); INNER JOIN
OUTER JOIN
id | value | value NATURAL JOIN
Self Joins
----+-------+------- Other Useful
3 | a3 | b3 Operations
Subqueries
4 | a4 | b4 Set Operations
Common Operations
5 | | b5 Advanced
Operations
6 | | b6 Common Table
Expressions
(4 rows) Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

17. Fun With SQL
FULL JOIN
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
# select * from a full join b using (id); CROSS JOIN
id | value | value INNER JOIN
OUTER JOIN
----+-------+------- NATURAL JOIN
Self Joins
1 | a1 | Other Useful
Operations
2 | a2 | Subqueries
Set Operations
3 | a3 | b3 Common Operations
4 | a4 | b4 Advanced
Operations
5 | | b5 Common Table
Expressions
6 | | b6 Window Functions
(6 rows) Real, Live Queries
Something Simple
Something Fun
Key Points

18. Fun With SQL
Applications
Joshua Tolley
End Point
Corporation
Why and Why Not
Find rows with no match in table b: Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
# SELECT * FROM a LEFT JOIN b USING (id) NATURAL JOIN
Self Joins
WHERE b.value IS NULL; Other Useful
id | value | value Operations
Subqueries
----+-------+------- Set Operations
Common Operations
1 | a1 | Advanced
Operations
2 | a2 | Common Table
Expressions
(2 rows) Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

19. Fun With SQL
NATURAL JOIN
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
Other Useful
NATURAL is syntactic sugar to match all columns with Operations
the same name Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

20. Fun With SQL
NATURAL JOIN
Joshua Tolley
End Point
Corporation
# SELECT * FROM a NATURAL FULL JOIN b;
id | value Why and Why Not
Joins
----+------- CROSS JOIN
1 | a1 INNER JOIN
OUTER JOIN
2 | a2 NATURAL JOIN
Self Joins
3 | a3 Other Useful
Operations
3 | b3 Subqueries
Set Operations
4 | a4 Common Operations
4 | b4 Advanced
Operations
5 | b5 Common Table
Expressions
6 | b6 Window Functions
(8 rows) Real, Live Queries
Something Simple
Something Fun
Key Points
This looked for matches in both the id and value columns,
so no rows matched. It returned all rows of both relations
because it’s a FULL JOIN.

21. Fun With SQL
Self Joins
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
”Self joins” are particularly counterintuitive Other Useful
Operations
Joins one table to itself Subqueries
Set Operations
It helps to give the table two different aliases Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

22. Fun With SQL
Self Joins
Joshua Tolley
Find all employees’ names, and each employee’s manager End Point
Corporation
SELECT Why and Why Not
e.first || ’ ’ || e.last, Joins
(SELECT CROSS JOIN
INNER JOIN
m.first || ’ ’ || m.last OUTER JOIN
NATURAL JOIN
FROM employee m Self Joins
Other Useful
WHERE m.id = e.manager); Operations
Subqueries
Set Operations
Common Operations
... will generally be much faster rewritten as ... Advanced
Operations
Common Table
SELECT Expressions
Window Functions
e.first || ’ ’ || e.last, Real, Live Queries
m.first || ’ ’ || m.last Something Simple
Something Fun
FROM Key Points
employee e
JOIN employee m ON (e.manager = m.id)

23. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
More useful operations... Other Useful
Operations
Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

24. Fun With SQL
Subqueries
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
Embeds one query within another CROSS JOIN
INNER JOIN
OUTER JOIN
Examples (some bad, some good) NATURAL JOIN
Self Joins
SELECT id FROM table WHERE field = (SELECT
Other Useful
MAX(field) FROM table) Operations
SELECT id, (SELECT COUNT(*) FROM table2 Subqueries
Set Operations
WHERE id = table1.id) FROM table1 Common Operations
SELECT a, b FROM (SELECT a, COUNT(*) AS c Advanced
Operations
FROM table1) t1 JOIN (SELECT b, COUNT(*) AS c Common Table
Expressions
FROM table2) t2 on (t1.c = t2.c) Window Functions
You can join subqueries just like you’d join tables Real, Live Queries
Something Simple
Something Fun
Key Points

25. Fun With SQL
Set Operations
Joshua Tolley
End Point
Corporation
INTERSECT Why and Why Not
Returns the intersection of two sets Joins
Doesn’t exist in MySQL CROSS JOIN
INNER JOIN
SELECT (SELECT a, b FROM table1) INTERSECT OUTER JOIN
NATURAL JOIN
(SELECT c, d FROM table2) Self Joins
UNION Other Useful
Operations
Appends one set of rows to another set with matching Subqueries
Set Operations
column types Common Operations
SELECT a FROM table1 UNION SELECT b FROM Advanced
Operations
table2 Common Table
Expressions
EXCEPT Window Functions
Real, Live Queries
Returns rows in one SELECT that aren’t in another Something Simple
SELECT Something Fun
SELECT a FROM table1 EXCEPT SELECT b FROM Key Points
table2

26. Fun With SQL
Common Operations
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
COALESCE(a, b) INNER JOIN
OUTER JOIN
If a is null, return b, else return a NATURAL JOIN
Self Joins
SELECT COALESCE(first, ’<NULL>’) FROM table
Other Useful
Oracle calls this NVL() Operations
Subqueries
CASE...WHEN Set Operations
Common Operations
Conditional operation
Advanced
SELECT CASE WHEN langused IN (’Lisp’, ’OCaml’, Operations
Common Table
’Haskell’) THEN ’Functional’ ELSE ’Imperative’ AS Expressions
Window Functions
langtype FROM software
Real, Live Queries
Something Simple
Something Fun
Key Points

27. Fun With SQL
Series Generation
Joshua Tolley
End Point
Corporation
generate series() in PostgreSQL; might be something
else in other databases Why and Why Not
Joins
Returns a series of numbers CROSS JOIN
INNER JOIN
Can be used like a for loop (example given later) OUTER JOIN
NATURAL JOIN
Self Joins
# SELECT * FROM generate_series(1, 5); Other Useful
Operations
generate_series Subqueries
Set Operations
----------------- Common Operations
Advanced
1 Operations
2 Common Table
Expressions
Window Functions
3
Real, Live Queries
4 Something Simple
Something Fun
5
Key Points
(5 rows)

28. Fun With SQL
Common Table Expressions
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
Abbreviated CTEs INNER JOIN
OUTER JOIN
Fairly advanced; not available in all databases NATURAL JOIN
Self Joins
Not in PostgreSQL before v. 8.4, or any version of Other Useful
Operations
MySQL Subqueries
Set Operations
It’s just like defining a one-time view for your query Common Operations
One major benefit: CTEs allow recursion Advanced
Operations
Recursing with CTEs is much more efficent than Common Table
Expressions
processing recursive data in your application Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

29. Fun With SQL
A Simple CTE Example
Joshua Tolley
End Point
Corporation
Why and Why Not
# SELECT * FROM GENERATE_SERIES(1,3)
Joins
CROSS JOIN CROSS JOIN
INNER JOIN
(SELECT * FROM GENERATE_SERIES(8,9)) AS f; OUTER JOIN
NATURAL JOIN
generate_series | generate_series Self Joins
-----------------+----------------- Other Useful
Operations
1 | 8 Subqueries
Set Operations
1 | 9 Common Operations
2 | 8 Advanced
Operations
2 | 9 Common Table
Expressions
3 | 8 Window Functions
Real, Live Queries
3 | 9 Something Simple
Something Fun
(6 rows)
Key Points

30. Fun With SQL
A Simple CTE Example
Joshua Tolley
End Point
Corporation
# WITH t AS (
Why and Why Not
SELECT * FROM GENERATE_SERIES(8,9) Joins
) CROSS JOIN
INNER JOIN
SELECT * FROM GENERATE_SERIES(1,3) OUTER JOIN
NATURAL JOIN
CROSS JOIN t; Self Joins
Other Useful
generate_series | generate_series Operations
-----------------+----------------- Subqueries
Set Operations
1 | 8 Common Operations
Advanced
1 | 9 Operations
Common Table
2 | 8 Expressions
Window Functions
2 | 9
Real, Live Queries
3 | 8 Something Simple
Something Fun
3 | 9 Key Points
(6 rows)

31. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
That last example was a bit cheesy, but the technique can Self Joins
Other Useful
be useful for complex queries in several parts Operations
Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

32. Fun With SQL
Recursion
Joshua Tolley
End Point
Corporation
Start with this: Why and Why Not
Joins
# SELECT * FROM employee; CROSS JOIN
INNER JOIN
first | last | id | manager OUTER JOIN
NATURAL JOIN
--------+----------+----+--------- Self Joins
john | doe | 1 | Other Useful
Operations
fred | rogers | 2 | 1 Subqueries
Set Operations
speedy | gonzales | 3 | 1 Common Operations
carly | fiorina | 4 | 1 Advanced
Operations
hans | reiser | 5 | 2 Common Table
Expressions
johnny | carson | 6 | 5 Window Functions
Real, Live Queries
martha | stewart | 7 | 3 Something Simple
(7 rows) Something Fun
Key Points

33. Fun With SQL
Recursion
Joshua Tolley
Recursive CTE to retrieve management hierarchy: End Point
Corporation
# WITH RECURSIVE t (id, managernames) AS ( Why and Why Not
SELECT e.id, first || ’ ’ || last Joins
AS managernames CROSS JOIN
INNER JOIN
FROM employee e WHERE manager IS NULL OUTER JOIN
NATURAL JOIN
UNION ALL Self Joins
Other Useful
SELECT e.id, Operations
Subqueries
first || ’ ’ || last || ’, ’ || managernames Set Operations
Common Operations
AS managernames
Advanced
FROM employee e Operations
Common Table
JOIN t ON (e.manager = t.id) Expressions
Window Functions
WHERE manager IS NOT NULL Real, Live Queries
) Something Simple
Something Fun
SELECT e.id, first || ’ ’ || last AS name, Key Points
managernames
FROM employee e JOIN t ON (e.id = t.id);

34. Fun With SQL
Recursion
Joshua Tolley
End Point
Corporation
...and get this...
Why and Why Not
id | name | managernames Joins
----+-----------------+-----------------------------CROSS JOIN
INNER JOIN
1 | john doe | john doe OUTER JOIN
NATURAL JOIN
2 | fred rogers | fred rogers, john doe Self Joins
3 | speedy gonzales | speedy gonzales, john doe Other Useful
Operations
4 | carly fiorina | carly fiorina, john doe Subqueries
Set Operations
5 | hans reiser | hans reiser, fred rogers, Common Operations
Advanced
| | john doe Operations
6 | johnny carson | johnny carson, hans reiser, Common Table
Expressions
Window Functions
| | fred rogers, john doe
Real, Live Queries
7 | martha stewart | martha stewart, speedy Something Simple
Something Fun
| | gonzales, john doe Key Points
(7 rows)

35. Fun With SQL
Fractals in SQL
Joshua Tolley
End Point
Corporation
WITH RECURSIVE x(i) AS
(VALUES(0) UNION ALL SELECT i + 1 FROM x WHERE i < 101),
Z(Ix, Iy, Cx, Cy, X, Y, I) Why and Why Not
AS (
SELECT Ix, Iy, X::float, Y::float, X::float, Y::float, 0 Joins
FROM (SELECT -2.2 + 0.031 * i, i FROM x) AS xgen(x,ix) CROSS JOIN
CROSS JOIN INNER JOIN
(SELECT -1.5 + 0.031 * i, i FROM x) AS ygen(y,iy) OUTER JOIN
UNION ALL NATURAL JOIN
Self Joins
SELECT
Ix, Iy, Cx, Cy, X * X - Y * Y + Cx AS X, Other Useful
Y * X * 2 + Cy, I + 1 Operations
FROM Z Subqueries
WHERE X * X + Y * Y < 16.0 AND I < 27), Set Operations
Zt (Ix, Iy, I) AS ( Common Operations
SELECT Ix, Iy, MAX(I) AS I
FROM Z GROUP BY Iy, Ix Advanced
ORDER BY Iy, Ix Operations
) Common Table
SELECT array_to_string( Expressions
array_agg( Window Functions
SUBSTRING(’ .,,,-----++++%%%%@@@@#### ’, Real, Live Queries
GREATEST(I,1), 1)
Something Simple
),’’ Something Fun
)
FROM Zt GROUP BY Iy ORDER BY Iy; Key Points
(yes, this query is SQL-spec compliant)

36. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
Other Useful
Operations
Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

37. Fun With SQL
Window Functions
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
Like CTEs, these are quite advanced NATURAL JOIN
Self Joins
Also unavailable in MySQL, and PostgreSQL before 8.4 Other Useful
Operations
Allow ranking, moving averages Subqueries
Set Operations
Like a set-returning aggregate function. Window Common Operations
Advanced
functions return results for each row based on a Operations
”window” of related rows Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

38. Fun With SQL
Window Functions
Joshua Tolley
End Point
Corporation
If our employee table had department and salary
information... Why and Why Not
Joins
# SELECT first, last, salary, department CROSS JOIN
INNER JOIN
FROM employee; OUTER JOIN
NATURAL JOIN
first | last | salary | department Self Joins
Other Useful
--------+----------+--------+---------------- Operations
fred | rogers | 97000 | sales Subqueries
Set Operations
carly | fiorina | 95000 | sales Common Operations
Advanced
johnny | carson | 89000 | sales Operations
speedy | gonzales | 96000 | development Common Table
Expressions
Window Functions
hans | reiser | 93000 | development
Real, Live Queries
martha | stewart | 90000 | development Something Simple
Something Fun
john | doe | 99000 | administration Key Points
(7 rows)

39. Fun With SQL
Window Functions Example
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
Rank employees in each department by salary CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
SELECT first, last, salary, department, Self Joins
RANK() OVER ( Other Useful
Operations
PARTITION BY department Subqueries
Set Operations
ORDER BY salary DESC Common Operations
) Advanced
Operations
FROM employee Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

40. Fun With SQL
Window Functions Example
Joshua Tolley
End Point
Corporation
Why and Why Not
... and get this: Joins
CROSS JOIN
INNER JOIN
first | last | salary | department | rank OUTER JOIN
--------+----------+--------+----------------+------ NATURAL JOIN
Self Joins
john | doe | 99000 | administration | 1
Other Useful
speedy | gonzales | 96000 | development | 1 Operations
hans | reiser | 93000 | development | 2 Subqueries
Set Operations
martha | stewart | 90000 | development | 3 Common Operations
fred | rogers | 97000 | sales | 1 Advanced
carly | fiorina | 95000 | sales | 2 Operations
Common Table
johnny | carson | 89000 | sales | 3 Expressions
Window Functions
(7 rows)
Real, Live Queries
Something Simple
Something Fun
Key Points

41. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
Real, live queries Other Useful
Operations
Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

42. Fun With SQL
Something Simple
Joshua Tolley
End Point
Corporation
Why and Why Not
The slow version:
Joins
CROSS JOIN
SELECT DISTINCT(sync) FROM bucardo.bucardo_rate INNER JOIN
OUTER JOIN
ORDER BY 1 NATURAL JOIN
Self Joins
Other Useful
Operations
The fast version: Subqueries
Set Operations
Common Operations
SELECT name FROM sync WHERE EXISTS ( Advanced
Operations
SELECT 1 FROM bucardo_rate Common Table
Expressions
WHERE sync = name LIMIT 1) Window Functions
Real, Live Queries
ORDER BY 1 Something Simple
Something Fun
Key Points

43. Fun With SQL
Something Simple
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
The bucardo rate table is huge, with few distinct values NATURAL JOIN
Self Joins
finding ”DISTINCT sync” requires a long table scan Other Useful
Operations
The sync table contains a list of all possible values in Subqueries
Set Operations
the bucardo rate.sync column Common Operations
Advanced
So instead of a big table scan, we scan the small table, Operations
and filter out values can’t find in bucardo rate Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

44. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
Something Fun Other Useful
Operations
Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

45. Fun With SQL
Joshua Tolley
SELECT End Point
id, idname, Corporation
COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) AS avgtime,
COALESCE(SUM(total), 0) AS count
FROM ( Why and Why Not
SELECT slavecommit,
EXTRACT(EPOCH FROM slavecommit - mastercommit) AS synctime, Joins
total CROSS JOIN
FROM bucardo.bucardo_rate INNER JOIN
WHERE sync = ’RO_everything’ AND OUTER JOIN
mastercommit > (NOW() - (15 + 1) * INTERVAL ’1 HOUR’) NATURAL JOIN
) i Self Joins
RIGHT JOIN (
Other Useful
SELECT id, idname,
Operations
TO_TIMESTAMP(start - start::INTEGER % 3600) AS start,
Subqueries
TO_TIMESTAMP(stop - stop::INTEGER % 3600) AS stop
Set Operations
FROM (
Common Operations
SELECT id,
TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Advanced
’Dy Mon DD HH:MI AM’) AS idname, Operations
EXTRACT(EPOCH FROM NOW() - id * INTERVAL ’1 HOUR’) AS start, Common Table
EXTRACT(EPOCH FROM NOW() - (id - 1) * INTERVAL ’1 HOUR’) AS stop Expressions
FROM ( Window Functions
SELECT GENERATE_SERIES(1, 15) AS id
Real, Live Queries
) f
) g Something Simple
Something Fun
) h ON (slavecommit BETWEEN start AND stop)
GROUP BY id, idname Key Points
ORDER BY id DESC;

46. Fun With SQL
Something Fun
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
The table contains replication data NATURAL JOIN
Self Joins
Time of commit on master Other Useful
Time of commit on slave Operations
Subqueries
Number of rows replicated Set Operations
Common Operations
The user wants a graph of replication speed over time, Advanced
given a user-determined range of time Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

47. Fun With SQL
Something Fun
Joshua Tolley
End Point
We want to average replication times over a series of Corporation
buckets. The first part of our query creates those buckets,
Why and Why Not
based on generate series(). Here we create buckets for
Joins
15 hours CROSS JOIN
INNER JOIN
OUTER JOIN
SELECT NATURAL JOIN
Self Joins
id, Other Useful
Operations
TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Subqueries
’Dy Mon DD HH:MI AM’) AS idname, Set Operations
Common Operations
EXTRACT(EPOCH FROM NOW() - id * Advanced
Operations
INTERVAL ’1 HOUR’) AS start, Common Table
Expressions
EXTRACT(EPOCH FROM NOW() - (id - 1) * Window Functions
INTERVAL ’1 HOUR’) AS stop Real, Live Queries
Something Simple
FROM ( Something Fun
SELECT GENERATE_SERIES(1, 15) AS id Key Points
) f

48. Fun With SQL
Something Fun
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
This gives us: INNER JOIN
OUTER JOIN
NATURAL JOIN
id | idname | start | stop Self Joins
----+---------------------+------------------+------------------ Other Useful
1 | Sat Mar 14 10:23 PM | 1237091036.95657 | 1237094636.95657 Operations
2 | Sat Mar 14 09:23 PM | 1237087436.95657 | 1237091036.95657 Subqueries
Set Operations
3 | Sat Mar 14 08:23 PM | 1237083836.95657 | 1237087436.95657 Common Operations
4 | Sat Mar 14 07:23 PM | 1237080236.95657 | 1237083836.95657
Advanced
... Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

49. Fun With SQL
Something Fun
Joshua Tolley
End Point
Corporation
Why and Why Not
Make the buckets end on nice time boundaries: Joins
CROSS JOIN
INNER JOIN
SELECT id, idname, OUTER JOIN
NATURAL JOIN
TO_TIMESTAMP(start - start::INTEGER % 3600) Self Joins
Other Useful
AS start, Operations
TO_TIMESTAMP(stop - stop::INTEGER % 3600) Subqueries
Set Operations
AS stop Common Operations
Advanced
FROM ( Operations
-- The bucket query, shown earlier, goes here Common Table
Expressions
Window Functions
) g
Real, Live Queries
Something Simple
Something Fun
Key Points

50. Fun With SQL
Something Fun
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
That gives us this: NATURAL JOIN
Self Joins
id | idname | start | stop
----+---------------------+-------------------------------+------------------------------- Other Useful
1 | Sat Mar 14 10:23 PM | 2009-03-14 21:59:59.956568-06 | 2009-03-14 22:59:59.956568-06 Operations
2 | Sat Mar 14 09:23 PM | 2009-03-14 20:59:59.956568-06 | 2009-03-14 21:59:59.956568-06 Subqueries
3 | Sat Mar 14 08:23 PM | 2009-03-14 19:59:59.956568-06 | 2009-03-14 20:59:59.956568-06 Set Operations
4 | Sat Mar 14 07:23 PM | 2009-03-14 18:59:59.956568-06 | 2009-03-14 19:59:59.956568-06 Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

51. Fun With SQL
Something Fun
Joshua Tolley
End Point
Corporation
In an different subquery, select everything from the table of
Why and Why Not
the right time period and right sync. Call this the ”stats”
Joins
query: CROSS JOIN
INNER JOIN
OUTER JOIN
SELECT NATURAL JOIN
Self Joins
slavecommit, Other Useful
EXTRACT(EPOCH FROM slavecommit - mastercommit) Operations
Subqueries
AS synctime, Set Operations
Common Operations
total Advanced
FROM bucardo.bucardo_rate Operations
Common Table
WHERE Expressions
Window Functions
sync = ’RO_everything’ AND Real, Live Queries
Something Simple
mastercommit > (NOW() - (15 + 1) * Something Fun
INTERVAL ’1 HOUR’) Key Points

52. Fun With SQL
Something Fun
Joshua Tolley
End Point
Corporation
Why and Why Not
...which gives us this: Joins
CROSS JOIN
INNER JOIN
slavecommit | synctime | total OUTER JOIN
-------------------------------+------------------+-------- NATURAL JOIN
Self Joins
2009-03-14 07:32:00.103759-06 | 5.65614098310471 | 1
Other Useful
2009-03-14 07:32:04.31508-06 | 5.25827997922897 | 3 Operations
2009-03-14 07:32:04.31508-06 | 5.25827997922897 | 5 Subqueries
Set Operations
2009-03-14 07:32:08.700184-06 | 7.71899098157883 | 1 Common Operations
2009-03-14 07:32:08.700184-06 | 8.22490698099136 | 1 Advanced
2009-03-14 07:32:12.675518-06 | 7.85176599025726 | 6 Operations
Common Table
2009-03-14 07:32:12.675518-06 | 7.15798497200012 | 6 Expressions
Window Functions
...
Real, Live Queries
Something Simple
Something Fun
Key Points

53. Fun With SQL
Something Fun
Joshua Tolley
End Point
Corporation
Now, join the two queries:
Why and Why Not
SELECT Joins
CROSS JOIN
id, idname, INNER JOIN
OUTER JOIN
COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) NATURAL JOIN
Self Joins
AS avgtime, Other Useful
COALESCE(SUM(total), 0) AS count Operations
Subqueries
FROM ( Set Operations
Common Operations
<STATS QUERY> Advanced
) RIGHT JOIN ( Operations
Common Table
<CALENDAR QUERY> Expressions
Window Functions
) ON (slavecommit BETWEEN start AND stop) Real, Live Queries
Something Simple
GROUP BY id, idname Something Fun
ORDER BY id DESC; Key Points

54. Fun With SQL
Something Fun
Joshua Tolley
End Point
...and get this: Corporation
Why and Why Not
id | idname | avgtime | count
Joins
----+---------------------+---------+------- CROSS JOIN
INNER JOIN
15 | Sat Mar 14 08:35 AM | 7.9 | 14219 OUTER JOIN
NATURAL JOIN
14 | Sat Mar 14 09:35 AM | 6.9 | 16444 Self Joins
13 | Sat Mar 14 10:35 AM | 6.5 | 62100 Other Useful
Operations
12 | Sat Mar 14 11:35 AM | 6.2 | 47349 Subqueries
Set Operations
11 | Sat Mar 14 12:35 PM | 0 | 0 Common Operations
Advanced
10 | Sat Mar 14 01:35 PM | 4.6 | 21348 Operations
Common Table
Expressions
Window Functions
This is the average replication time and total replicated rows Real, Live Queries
Something Simple
per hour. Note that this correctly returns zeroes when no Something Fun
rows are replicated, and still returns a value for that time Key Points
slot. This prevents some amount of application-side
processing.

55. Fun With SQL
Something Fun
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
Other Useful
Operations
That query again: Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

56. Fun With SQL
Joshua Tolley
SELECT End Point
id, idname, Corporation
COALESCE(ROUND(AVG(synctime)::NUMERIC, 1), 0) AS avgtime,
COALESCE(SUM(total), 0) AS count
FROM ( Why and Why Not
SELECT slavecommit,
EXTRACT(EPOCH FROM slavecommit - mastercommit) AS synctime, Joins
total CROSS JOIN
FROM bucardo.bucardo_rate INNER JOIN
WHERE sync = ’RO_everything’ AND OUTER JOIN
mastercommit > (NOW() - (15 + 1) * INTERVAL ’1 HOUR’) NATURAL JOIN
) i Self Joins
RIGHT JOIN (
Other Useful
SELECT id, idname,
Operations
TO_TIMESTAMP(start - start::INTEGER % 3600) AS start,
Subqueries
TO_TIMESTAMP(stop - stop::INTEGER % 3600) AS stop
Set Operations
FROM (
Common Operations
SELECT id,
TO_CHAR(NOW() - id * INTERVAL ’1 HOUR’, Advanced
’Dy Mon DD HH:MI AM’) AS idname, Operations
EXTRACT(EPOCH FROM NOW() - id * INTERVAL ’1 HOUR’) AS start, Common Table
EXTRACT(EPOCH FROM NOW() - (id - 1) * INTERVAL ’1 HOUR’) AS stop Expressions
FROM ( Window Functions
SELECT GENERATE_SERIES(1, 15) AS id
Real, Live Queries
) f
) g Something Simple
Something Fun
) h ON (slavecommit BETWEEN start AND stop)
GROUP BY id, idname Key Points
ORDER BY id DESC;

57. Fun With SQL
Key Points
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Understand join types, and use them Self Joins
Other Useful
Know what functions and set operations your database Operations
Subqueries
provides Set Operations
Common Operations
Build large queries piece by piece Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points

58. Fun With SQL
Joshua Tolley
End Point
Corporation
Why and Why Not
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
NATURAL JOIN
Self Joins
Questions? Other Useful
Operations
Subqueries
Set Operations
Common Operations
Advanced
Operations
Common Table
Expressions
Window Functions
Real, Live Queries
Something Simple
Something Fun
Key Points