This presentation reviews the top ten new features that will appear in the Postgres 9.5 release. Postgres 9.5 adds many features designed to enhance the productivity of developers: UPSERT, CUBE, ROLLUP, JSONB functions, and PostGIS improvements. For administrators, it has row-level security, a new index type, and performance enhancements for large servers.

1. Major Features: Postgres 9.5
BRUCE MOMJIAN
POSTGRESQL is an open-source, full-featured relational database.
This presentation gives an overview of the Postgres 9.5 release.
Creative Commons Attribution License http://momjian.us/presentations
Last updated: November, 2015
1 / 40

2. PostgreSQL the database…
◮ Open Source Object Relational DBMS since 1996
◮ Distributed under the PostgreSQL License
◮ Similar technical heritage as Oracle, SQL Server & DB2
◮ However, a strong adherence to standards (ANSI-SQL 2008)
◮ Highly extensible and adaptable design
◮ Languages, indexing, data types, etc.
◮ E.g. PostGIS, JSONB, SQL/MED
◮ Extensive use throughout the world for applications and
organizations of all types
◮ Bundled into Red Hat Enterprise Linux, Ubuntu, CentOS
and Amazon Linux
Major Features: Postgres 9.5 2 / 40

3. PostgreSQL the community…
◮ Independent community led by a Core Team of six
◮ Large, active and vibrant community
◮ www.postgresql.org
◮ Downloads, Mailing lists, Documentation
◮ Sponsors sampler:
◮ Google, Red Hat, VMWare, Skype, Salesforce, HP and
EnterpriseDB
◮ http://www.postgresql.org/community/
Major Features: Postgres 9.5 3 / 40

4. EnterpriseDB the company…
◮ Worldwide leader of Postgres based products and services
founded in 2004
◮ Customers include 63 of the Fortune 500 and 137 of the
Forbes Global 2000
◮ 2,700 customers range across many industries including
financial, healthcare, government, retail, telcommunications,
and insurance
◮ Enterprise offerings:
◮ PostgreSQL Support, Services and Training
◮ Postgres Plus Advanced Server with Oracle Compatibility
◮ Tools for Monitoring, Replication, HA, Backup & Recovery
Community
◮ Citizenship
◮ Contributor of key features: Materialized Views, JSON, &
more
◮ Nine community members on staff
Major Features: Postgres 9.5 4 / 40

5. EnterpriseDB the company…
Major Features: Postgres 9.5 5 / 40

6. EnterpriseDB Is a Leader
The Gartner report, Magic Quadrant for Operational Database Management Systems, by
Donald Feinberg, Merv Adrian, Nick Heudecker, Adam Ronthal, and Terilyn Palanca was
published October 12, 2015.
Major Features: Postgres 9.5 6 / 40

7. 9.5 Feature Outline
1. INSERT … ON CONFLICT, also known as “UPSERT”
2. Block-Range Indexes (BRIN) which enable compact indexing of
very large tables
3. Analytic operations GROUPING SETS, CUBE, and ROLLUP
4. Row-Level Security (RLS)
5. In-memory sorting and hashing performance improvements
6. Multi-core and large memory scalability improvements
7. Automated management of the number of WAL files
8. Additional JSONB data manipulation functions and operators
9. Enhancements to Foreign Data Wrappers
10. Allow Indexed PostGIS LIMIT distance calculations
without CTEs
To be released in 2015, full item list at
http://www.postgresql.org/docs/devel/static/release-9-5.html
Major Features: Postgres 9.5 7 / 40

8. 1. INSERT … ON CONFLICT
◮ Turns a conflicting INSERT into an UPDATE
◮ Works for VALUES and SELECT as a row source
◮ Handles concurrent operations without errors
◮ Is row-oriented, unlike MERGE, which is batch-oriented
◮ Does not have the problems associated with the
UPSERT/MERGE implementations of other vendors
(http://www.pgcon.org/2014/schedule/attachments/327_upsert_weird.pdf)
Major Features: Postgres 9.5 8 / 40

9. INSERT … ON CONFLICT Example
CREATE TABLE ins_update_test (x INTEGER PRIMARY KEY);
INSERT INTO ins_update_test VALUES (1);
INSERT INTO ins_update_test VALUES (1);
ERROR: duplicate key value violates unique constraint
"ins_update_test_pkey"
DETAIL: Key (x)=(1) already exists.
Major Features: Postgres 9.5 9 / 40

10. INSERT … ON CONFLICT Example
INSERT INTO ins_update_test VALUES (1)
ON CONFLICT DO NOTHING;
INSERT 0 0
INSERT INTO ins_update_test VALUES (1)
ON CONFLICT (x) DO UPDATE SET x = 2;
INSERT 0 1
SELECT * FROM ins_update_test;
x
---
2
Major Features: Postgres 9.5 10 / 40

11. INSERT … ON CONFLICT … EXCLUDED Example
CREATE TABLE customer (cust_id INTEGER PRIMARY KEY, name TEXT);
INSERT INTO customer VALUES (100, ’Big customer’);
INSERT INTO customer VALUES (100, ’Non-paying customer’);
ERROR: duplicate key value violates unique constraint
"customer_pkey"
DETAIL: Key (cust_id)=(100) already exists.
INSERT INTO customer VALUES (100, ’Non-paying customer’)
ON CONFLICT (cust_id) DO UPDATE SET name = EXCLUDED.name;
SELECT * FROM customer;
cust_id | name
---------+---------------------
100 | Non-paying customer
Major Features: Postgres 9.5 11 / 40

12. INSERT … ON CONFLICT with SELECT
CREATE TABLE merge (x INTEGER PRIMARY KEY);
INSERT INTO merge VALUES (1), (3), (5);
INSERT INTO merge SELECT * FROM generate_series(1, 5);
ERROR: duplicate key value violates unique constraint
"merge_pkey"
DETAIL: Key (x)=(1) already exists
Major Features: Postgres 9.5 12 / 40

13. INSERT … ON CONFLICT with SELECT
INSERT INTO merge SELECT * FROM generate_series(1, 5)
ON CONFLICT DO NOTHING;
SELECT * FROM merge;
x
---
1
3
5
2
4
Major Features: Postgres 9.5 13 / 40

14. INSERT … ON CONFLICT … UPDATE with SELECT
CREATE TABLE merge2 (x INTEGER PRIMARY KEY, status TEXT);
INSERT INTO merge2 VALUES (1, ’old’), (3, ’old’), (5, ’old’);
INSERT INTO merge2 SELECT *, ’new’ FROM generate_series(2, 5)
ON CONFLICT (x) DO
UPDATE SET status = ’conflict’;
SELECT * FROM merge2;
x | status
---+----------
1 | old
2 | new
3 | conflict
4 | new
5 | conflict
Major Features: Postgres 9.5 14 / 40

15. 2. Block-Range Indexes (BRIN)
◮ Tiny indexes designed for large tables
◮ Minimum/maximum values stored for a range of blocks
(default 1MB, 128 8k pages)
◮ Allows skipping large sections of the table that cannot
contain matching values
◮ Ideally for naturally-ordered tables, e.g. insert-only tables are
chronologically ordered
◮ Index is 0.003% the size of the heap
◮ Indexes are inexpensive to update
◮ Index every column at little cost
◮ Slower lookups than btree
Major Features: Postgres 9.5 15 / 40

16. Block-Range Indexes (BRIN) Example
CREATE TABLE brin_example AS
SELECT generate_series(1,100000000) AS id;
CREATE INDEX btree_index ON brin_example(id);
CREATE INDEX brin_index ON brin_example USING brin(id);
SELECT relname, pg_size_pretty(pg_relation_size(oid))
FROM pg_class
WHERE relname LIKE ’brin_%’ OR relname = ’btree_index’
ORDER BY relname;
relname | pg_size_pretty
--------------+----------------
brin_example | 3457 MB
btree_index | 2142 MB
brin_index | 104 kB
http://michael.otacoo.com/postgresql-2/postgres-9-5-feature-highl
Major Features: Postgres 9.5 16 / 40

17. 3. Analytic Operations GROUPING SETS,
CUBE, and ROLLUP
◮ Allows specification of multiple GROUP BY combinations in a
single query
◮ Avoids the need for UNION ALL and recomputation
◮ Empty fields are left NULL
Major Features: Postgres 9.5 17 / 40

18. Employee Table
SELECT * FROM employee ORDER BY name;
name | office | department
-------+--------+------------
Jill | PHL | Marketing
Lilly | SFO | Sales
Mark | PHL | Marketing
Nancy | PHL | Sales
Sam | SFO | Sales
Tim | PHL | Shipping
Major Features: Postgres 9.5 18 / 40

19. GROUP BY Example
SELECT office, COUNT(*)
FROM employee
GROUP BY office;
office | count
--------+-------
SFO | 2
PHL | 4
SELECT department, COUNT(*)
FROM employee
GROUP BY department;
department | count
------------+-------
Marketing | 2
Shipping | 1
Sales | 3
Major Features: Postgres 9.5 19 / 40

20. GROUP BY with UNION ALL
SELECT office, NULL as department, COUNT(*)
FROM employee
GROUP BY office
UNION ALL
SELECT NULL as office, department, COUNT(*)
FROM employee
GROUP BY department
ORDER BY 1;
office | department | count
--------+------------+-------
PHL | | 4
SFO | | 2
| Marketing | 2
| Shipping | 1
| Sales | 3
Major Features: Postgres 9.5 20 / 40

21. GROUPING SETS Example
SELECT office, department, COUNT(*)
FROM employee
GROUP BY GROUPING SETS (office, department)
ORDER BY office, department;
office | department | count
--------+------------+-------
PHL | | 4
SFO | | 2
| Marketing | 2
| Sales | 3
| Shipping | 1
Major Features: Postgres 9.5 21 / 40

22. ROLLUP Example
SELECT office, department, COUNT(*)
FROM employee
GROUP BY ROLLUP (office, department)
ORDER BY office, department;
office | department | count
--------+------------+-------
PHL | Marketing | 2
PHL | Sales | 1
PHL | Shipping | 1
PHL | | 4
SFO | Sales | 2
SFO | | 2
| | 6
Major Features: Postgres 9.5 22 / 40

23. CUBE Example
SELECT office, department, COUNT(*)
FROM employee
GROUP BY CUBE (office, department)
ORDER BY office, department;
office | department | count
--------+------------+-------
PHL | Marketing | 2
PHL | Sales | 1
PHL | Shipping | 1
PHL | | 4
SFO | Sales | 2
SFO | | 2
| Marketing | 2
| Sales | 3
| Shipping | 1
| | 6
Major Features: Postgres 9.5 23 / 40

24. GROUPING SETS Equivalent of CUBE
SELECT office, department, COUNT(*)
FROM employee
GROUP BY GROUPING SETS
((office, department), office, department, ())
ORDER BY office, department;
office | department | count
--------+------------+-------
PHL | Marketing | 2
PHL | Sales | 1
PHL | Shipping | 1
PHL | | 4
SFO | Sales | 2
SFO | | 2
| Marketing | 2
| Sales | 3
| Shipping | 1
| | 6
Major Features: Postgres 9.5 24 / 40

25. 4. Row-Level Security (RLS)
◮ Allows SELECT, INSERT, UPDATE, OR DELETE permission
control over existing rows with USING expression
◮ Also INSERT or UPDATE control over added and modified
rows with CHECK expression
◮ Expressions can contain checks for the current user,
subqueries, time comparisons, and function calls
◮ Enabled with GUC row_security, CREATE POLICY, and ALTER
TABLE … ENABLE ROW LEVEL SECURITY
Major Features: Postgres 9.5 25 / 40

26. Row-Level Security Example
Table Setup
SHOW row_security;
row_security
--------------
on
CREATE TABLE orders (id INTEGER, product TEXT,
entered_by TEXT);
ALTER TABLE orders ENABLE ROW LEVEL SECURITY;
CREATE POLICY orders_control ON orders FOR ALL TO PUBLIC
USING (entered_by = CURRENT_USER);
GRANT ALL ON TABLE orders TO PUBLIC;
Major Features: Postgres 9.5 26 / 40

27. Row-Level Security Example
User Setup
CREATE USER emp1;
CREATE USER emp2;
SET SESSION AUTHORIZATION emp1;
INSERT INTO orders VALUES (101, ’fuse’, CURRENT_USER);
SET SESSION AUTHORIZATION emp2;
INSERT INTO orders VALUES (102, ’bolt’, CURRENT_USER);
Major Features: Postgres 9.5 27 / 40

28. Row-Level Security Example
Testing
SET SESSION AUTHORIZATION postgres;
SELECT * FROM orders;
id | product | entered_by
-----+---------+------------
101 | fuse | emp1
102 | bolt | emp2
Major Features: Postgres 9.5 28 / 40

29. Row-Level Security Example
Testing
SET SESSION AUTHORIZATION emp1;
SELECT * FROM orders;
id | product | entered_by
-----+---------+------------
101 | fuse | emp1
SET SESSION AUTHORIZATION emp2;
SELECT * FROM orders;
id | product | entered_by
-----+---------+------------
102 | bolt | emp2
Major Features: Postgres 9.5 29 / 40

30. 5. In-Memory Sorting and
Hashing Performance Improvements
◮ Allow VARCHAR(), TEXT and NUMERIC() to use the
abbreviated sorting optimization
◮ Use memcmp()as quick string equality checks before collation
comparisons
◮ Decrease the average number of hash entries per bucket from
10 to 1
◮ Pre-allocate the maximum number of hash buckets in cases
where we are likely to use multiple work_mem-sized batches
◮ Allow CREATE INDEX, REINDEX, and CLUSTER to use inlined
sorting
◮ Allow use of 128-bit accumulators for aggregate
computations
Major Features: Postgres 9.5 30 / 40

31. 6. Multi-Core and Large Memory
Scalability Improvements
◮ Improve concurrency of shared buffer replacement
◮ Reduce the number of page locks and pins during index scans
◮ Make backend local tracking of buffer pins memory efficient
◮ Improve lock scalability on multi-socket systems
◮ Increase the number of shared buffer mapping hash table
entries from 16 to 128
◮ Allow searching for a free shared buffer to use minimal
locking
◮ Force buffer descriptors to be CPU-cache aligned (128 bytes)
◮ Reduce btree page pinning
Major Features: Postgres 9.5 31 / 40

32. 7. Automated Management of the Number of WAL
Files
◮ New GUC variables min_wal_size and max_wal_size control
the minimum and maximum size of the pg_xlog directory
◮ Previously checkpoint_segments controlled only the
maximum directory size (previously WAL files were not
removed)
◮ Size specified in bytes, not segment files
◮ Allows use of additional WAL files only when needed
Major Features: Postgres 9.5 32 / 40

33. Management of WAL Files
000000000000
111111111111
000000000
111111111
000000000000
111111111111
000000
000
111111
111
00000000
0000
11111111
1111
00000000
0000
11111111
1111
000000000
111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000
111111111
0000000000000000
1111111111111111
0000000000000000
1111111111111111
000000000000
111111111111
111 1 1Begin 1
Rotate 2 22
2 2 2End 1
2 2 21 1
PostgreSQL Shared Buffer Cache Write−Ahead Log
Major Features: Postgres 9.5 33 / 40

34. 8. Additional JSONB Data Manipulation Functions
and Operators
◮ Add jsonb_set(), which allows replacement of or addition
to JSONB documents
◮ Allow removal of JSONB documents using the subtraction
operator
◮ Allow merging of JSONB documents using the concatenation
(|| operator)
◮ Add function to remove null values from documents
Major Features: Postgres 9.5 34 / 40

35. 9. Enhancements to Foreign Data Wrappers
◮ Add IMPORT FOREIGN SCHEMA to create a local table
matching the schema of a foreign table
◮ Allow foreign tables to be part of inheritance trees
◮ Allow CHECK constraints on foreign tables
◮ Add infrastructure for foreign table join pushdown
Major Features: Postgres 9.5 35 / 40

36. 10. Allow Indexed PostGIS LIMIT
Distance Calculations without CTEs
◮ Nearest neighbor searches allow index lookups to return the
closest matches, e.g. return the 10 nearest points to a given
point
◮ Only the bounding boxes of two-dimensional objects are
indexed, e.g. polygon, circle, line
◮ Previously LIMIT could not combine bounding box index
lookups with accurate calculations
◮ Now LIMIT bounding box index filtering can recheck using
accurate distance calculations
◮ Workaround was to use a CTE with a 10x limit, then an outer
query to do accurate distance calculations
Major Features: Postgres 9.5 36 / 40

37. Pre-9.5 LIMIT Distance Example
WITH index_query AS (
SELECT st_distance(geom,
’SRID=3005;POINT(1011102 450541)’) AS distance,
parcel_id, address
FROM parcels
ORDER BY geom <-> ’SRID=3005;POINT(1011102 450541)’
LIMIT 100
)
SELECT *
FROM index_query
ORDER BY distance
LIMIT 10;
http://boundlessgeo.com/2011/09/indexed-nearest-neighbour-search-in-postgis/
http://shisaa.jp/postset/postgis-postgresqls-spatial-partner-part-3.html
Major Features: Postgres 9.5 37 / 40

38. 9.5 LIMIT Distance Example
SELECT st_distance(geom,
’SRID=3005;POINT(1011102 450541)’) AS distance,
parcel_id, address
FROM parcels
ORDER BY geom <-> ’SRID=3005;POINT(1011102 450541)’
LIMIT 10
http://www.postgresonline.com/journal/archives/350-PostGIS-2.2-leveraging-power-of-PostgreSQL-9.5.html
http://postgis.net/docs/manual-dev/geometry_distance_knn.html
Major Features: Postgres 9.5 38 / 40

39. Additional Resources…
◮ Postgres Downloads:
◮ www.enterprisedb.com/downloads
◮ Product and Services information:
◮ info@enterprisedb.com
Major Features: Postgres 9.5 39 / 40

40. Conclusion
http://momjian.us/presentations https://www.flickr.com/photos/thevlue/
Major Features: Postgres 9.5 40 / 40