1. Consistent State www.consistentstate.com kevink@consistentstate.com
PostgreSQL Backup and Recovery Methods
Kevin Kempter Chief Data Architect
Wednesday, September 18, 13

2. -or-
How NOT to be this guy
2
Wednesday, September 18, 13

3. Session
Topics
✓Overview
✓Backup Options
✓Restore Options
✓Point in Time Recovery
3
Wednesday, September 18, 13

4. Overview
• Multiple backup methods
• Multiple backup file formats
• Many recovery choices / options if a
pg_restore compatible format is used
• PITR
• PITR is the base construct for WAL
shipping (Warm Standby)
4
Wednesday, September 18, 13

5. PostgreSQL
Backup
Options
✓pg_dump
✓pg_dumpall
5
Wednesday, September 18, 13

6. pg_dump
✓ Utility to dump a snapshot of a single
database
✓ Multiple output options
✓ Non-blocking
✓ Creates a “Consistent” backup - even if the
database is in use
6
Wednesday, September 18, 13

7. pg_dump
• Syntax
• pg_dump [connection-options] [dump-options] [dbname]
• Connection Options
• -h, --host=HOSTNAME
• -p, --port=PORT
• -U, --username=NAME
• -w, --no-password
• -W, --password (should happen automatically)
• --role=ROLENAME (do SET ROLE before dump)
• Environment Variables
• PGDATABASE
• PGHOST
• PGOPTIONS
• PGPORT
• PGUSER
7
Wednesday, September 18, 13

8. pg_dump - Common Options
• -s [--schema-only]
• Dump schema (DDL) only, no data
• -a [--data-only]
• Dump data only - no DDL
• -c [--clean]
• Generate drop statements for all created objects
• -C [--create]
• Generate a “CREATE DATABASE” statement
• -n schema [--schema=schema]
• Only dump the specified schema, wildcard characters are allowed, also multiple -n’s are
allowed
8
Wednesday, September 18, 13

9. pg_dump - Common Options (continued)
• -N schema [--exclude-schema=schema]
• Exclude specified schema
• -F format [--format=format]
• Output format
• p (Plain) plain sql file #default
• c (Custom) custom binary format
• t (tar) tar format
• d (directory) Creates a directory with one file per table/blob, plus a TOC file in a binary
format that pg_restore can read
• -o [--oids]
• Dump table OID’s
• -O [--no-owner]
• Do not generate ownership commands
9
Wednesday, September 18, 13

10. pg_dump - Common Options (continued)
• -t table [--table=table]
• Only dump the specified table, wildcard characters are allowed, also multiple -t’s are
allowed, over-rides -n and -N options
• -x [--no-privileges] [--no-acl]
• Do not dump access privileges
• --inserts
• Generate INSERT statements
• --disable-triggers
• disable triggers during restore (for a data only restore) when doing a data-only dump
• --lock-wait-timeout=timeout
• fail if shared lock on an object cannot be acquired within timeout time
10
Wednesday, September 18, 13

11. pg_dump - Common Options (continued)
• -Z 0..9 [--compress=0..9]
• Specify compression level for custom format or plain format (not supported for tar
format)
• -v [--verbose]
• -V [--version]
11
Wednesday, September 18, 13

12. pg_dump - Examples
$ pg_dump -C --inserts prod1_db > prod1_db.sql
Creates a dump of insert statements including a create database statement
$ pg_dump --data-only --table=customer -Fc prod1_db > prod1_db.cust.fc.dmp
Dump the customer table data (data only) in a custom format from the prod1_db database
$ pg_dump -s prod1_db > prod1_db.ddl_only.sql
Creates a DDL only dump of the prod1_db database
$ pg_dump --schema=gold -Ft prod1_db > prod1_db.gold_schema.dmp
Creates a dump of the gold schema in the prod1_db database in a tar format
12
Wednesday, September 18, 13

13. pg_dump
Summary
13
Wednesday, September 18, 13

14. pg_dumpall
✓ Utility to dump a snapshot of a full
database cluster (or cluster-wide
constructs)
✓ Dumps only to plain sql format
✓ Non-blocking
✓ Creates a “Consistent” backup - even
if the database is in use
14
Wednesday, September 18, 13

15. pg_dumpall
• Syntax
• pg_dump [connection-options] [dump-options]
• Connection Options
• -h, --host=HOSTNAME
• -p, --port=PORT
• -U, --username=NAME
• -w, --no-password
• -W, --password (should happen automatically)
• --role=ROLENAME (do SET ROLE before dump)
• Environment Variables
• PGDATABASE
• PGHOST
• PGOPTIONS
• PGPORT
• PGUSER
15
Wednesday, September 18, 13

16. pg_dumpall - Common Options
• -s [--schema-only]
• Dump schema (DDL) only, no data
• -a [--data-only]
• Dump data only - no DDL
• -c [--clean]
• Generate drop statements for all created objects
• -o [--oids]
• Dump table OID’s
• -O [--no-owner]
• Do not generate ownership commands
16
Wednesday, September 18, 13

17. pg_dumpall - Common Options (continued)
• -r [--roles-only]
• Dump only CREATE ROLE data
• -t [--tablespaces-only]
• Dump only CREATE TABLESPACE data
• -g [--globals-only]
• Dump Global Structures (Roles and Tablespaces)
• --no-tablespaces
• Do NOT dump CREATE TABLESPACE Data
• --inserts
• Generate INSERT statements
17
Wednesday, September 18, 13

18. pg_dumpall - Common Options (continued)
• --disable-triggers
• disable triggers during restore (for a data only restore) when doing a data-only dump
• --lock-wait-timeout=timeout
• fail if shared lock on an object cannot be acquired within timeout time
• -v [--verbose]
• -V [--version]
18
Wednesday, September 18, 13

19. pg_dumpall - Examples
$ pg_dumpall -g > prod1_db_cluster.global_structures.sql
Creates a cluster dump containing only the cluster global structures
$ pg_dumpall --tablespaces-only > prod1_db_cluster.tablespaces.sql
Dump the cluster tablespaces
$ pg_dumpall --no-tablespaces > prod1_db_cluster.no_tablespaces.sql
Creates a dump of the cluster without any tablespace references
$ pg_dumpall -a > prod1_db_cluster.data_only.sql
Creates a dump of the cluster - data only
19
Wednesday, September 18, 13

20. pg_dumpall
summary
20
Wednesday, September 18, 13

21. PostgreSQL
Restore
Options
✓psql
✓pg_restore
21
Wednesday, September 18, 13

22. Restoring
with psql
$ psql -ef prod1_db.sql > load_db.log 2>&1
$ pg_dump prod1_db | psql -h qa_server
$ pg_dumpall -g | psql -h dev_server -p 5433 > load_dev.log 2>&1
$ pg_dump prod1_db | psql -e test_db > load_test_db.log 2>&1
22
Wednesday, September 18, 13

23. pg_restore
✓ Utility to restore a data file created by
pg_dump
• Works only with non plain text file
formats
23
Wednesday, September 18, 13

24. pg_restore
• Syntax
• pg_restore [connection-option...] [option...] [filename]
• Connection Options
• -h, --host=HOSTNAME
• -p, --port=PORT
• -U, --username=NAME
• -w, --no-password
• -W, --password (should happen automatically)
• --role=ROLENAME (do SET ROLE before dump)
• Environment Variables
• PGDATABASE
• PGHOST
• PGOPTIONS
• PGPORT
• PGUSER
24
Wednesday, September 18, 13

25. pg_restore - Common Options
• -d dbname [--dbname=dbname]
• -C [--create]
• Create the specified database before restore
• -c [--clean]
• Generate drop statements for all created objects
• -s [--schema-only]
• Dump schema (DDL) only, no data
• -a [--data-only]
• Dump data only - no DDL
25
Wednesday, September 18, 13

26. pg_restore - Common Options (continued)
• -n namespace [--schema=schema]
• Restore only objects in the specified schema
• -O [--no-owner]
• Do not restore ownership of objects
• -I index [--index=index]
• Restore specified index only
• -P function-name(argtype [, ...]) [ --function=function-name(argtype [, ...]) ]
• Restore specified function only
• -T trigger [--trigger=trigger]
• Restore specified trigger only
26
Wednesday, September 18, 13

27. pg_restore - Common Options (continued)
• -t table [--table=table]
• Restore specified table only
• --no-tablespaces
• Do not restore any TABLESPACES
• -F format [--format=format]
• Output format
• c (Custom) custom binary format
• t (tar) tar format
• d (directory)
27
Wednesday, September 18, 13

28. pg_restore - Common Options (continued)
• -t table [--table=table]
• Restore specified table only
• --no-tablespaces
• Do not restore any TABLESPACES
• -j number-of-jobs [--jobs=number-of-jobs]
• Use parallel jobs to perform the restore
• --disable-triggers
• Disable triggers during the restore (for a data only restore)
• -e [--exit-on-error]
• Exits upon any error
28
Wednesday, September 18, 13

29. pg_restore - Common Options (continued)
• -l [--list]
• Create a list (TOC) file
• -L list-file [--use-list=list-file]
• Restore based on the specified list file
• -V [--version]
• -v [--verbose]
29
Wednesday, September 18, 13

30. pg_restore - Examples
$ pg_restore -a -Fc -d prod2_db prod1_db.fc.dmp
Restores data only from a custom formatted file into database prod2_db
$ pg_restore -c --schema=gold_partners -v -Ft -d prod2_db prod.tar.dmp
Cleans (removes data & structures first) then restores the gold_partners
schema from a tar formatted file into the prod2_db database (with verbose
output)
$ pg_restore --schema-only -d qa1_db -Fc -j 10 prod1_db.fc.dmp
Restores the schema only (DDL) from a custom formatted file
into the qa1_db database using 10 parallel streams to do the restore
30
Wednesday, September 18, 13

31. Restoring
via a list file
• pg_restore can create a list file
from a pg_dump file
• List file will contain one line per
needed operation such as:
• CREATE TABLE
• COPY
• CREATE INDEX
• List file can be modified as desired
to create a custom restore
31
Wednesday, September 18, 13

32. Create a list file from the pg_dump file
$ pg_dump -Ft db1 > db1.fc.dmp
$ pg_restore -Ft -l db1.dmp > db1.lst
32
Wednesday, September 18, 13

33. Sample list file header
;
; Archive created at Tue Sep 10 09:42:24 2013
; dbname: testdb
; TOC Entries: 34
; Compression: -1
; Dump Version: 1.12-0
; Format: CUSTOM
; Integer: 4 bytes
; Offset: 8 bytes
; Dumped from database version: 9.2.4
; Dumped by pg_dump version: 9.2.4
;
;
33
Wednesday, September 18, 13

34. Sample list file contents
; Selected TOC Entries:
;
1981; 1262 16386 DATABASE - testdb_old postgres
6; 2615 2200 SCHEMA - public postgres
1982; 0 0 COMMENT - SCHEMA public postgres
1983; 0 0 ACL - public postgres
181; 3079 11730 EXTENSION - plpgsql
1984; 0 0 COMMENT - EXTENSION plpgsql
168; 1259 16411 TABLE public testdb_jasper_metrics_tables postgres
169; 1259 16414 TABLE public testdb_jasper_metrics_tables_tmp1 postgres
170; 1259 16417 TABLE public testdb_metrics_activity postgres
171; 1259 16423 TABLE public testdb_metrics_database postgres
172; 1259 16426 TABLE public testdb_postgres_metrics_bgwriter postgres
173; 1259 16429 TABLE public testdb_postgres_metricsio_user_tables postgres
174; 1259 16432 TABLE public testdb_testdb_gf_metrics_tables postgres
175; 1259 16435 TABLE public testdb_testdb_transition_metrics_tables postgres
176; 1259 16438 TABLE public testdb_testdb_transition_metrics_tables_tmp1 postgres
177; 1259 16441 TABLE public testdb_testdb_transition_metrics_tables_tmp2 postgres
34
Wednesday, September 18, 13

35. Sample list file contents (cont)
178; 1259 16444 TABLE public idle_conn_metrics postgres
179; 1259 16447 TABLE public total_conn_metrics postgres
180; 1259 16450 TABLE public waiting_conn_metrics postgres
1964; 0 16411 TABLE DATA public testdb_jasper_metrics_tables postgres
1965; 0 16414 TABLE DATA public testdb_jasper_metrics_tables_tmp1 postgres
1966; 0 16417 TABLE DATA public testdb_metrics_activity postgres
1967; 0 16423 TABLE DATA public testdb_metrics_database postgres
1968; 0 16426 TABLE DATA public testdb_postgres_metrics_bgwriter postgres
1969; 0 16429 TABLE DATA public testdb_postgres_metricsio_user_tables postgres
1970; 0 16432 TABLE DATA public testdb_testdb_gf_metrics_tables postgres
1971; 0 16435 TABLE DATA public testdb_testdb_transition_metrics_tables postgres
1972; 0 16438 TABLE DATA public testdb_testdb_transition_metrics_tables_tmp1 postgres
1973; 0 16441 TABLE DATA public testdb_testdb_transition_metrics_tables_tmp2 postgres
1974; 0 16444 TABLE DATA public idle_conn_metrics postgres
1975; 0 16447 TABLE DATA public total_conn_metrics postgres
1976; 0 16450 TABLE DATA public waiting_conn_metrics postgres
35
Wednesday, September 18, 13

36. Restore via list file - example
$ pg_dump -Ft prod_db > prod_db.fc.dmp
$ pg_restore -Ft -l prod_db.dmp > prod_db.lst
$ createdb qadb3
Edit prod_db.lst as needed / desired
$ pg_restore -L prod_db.lst -Ft -d qadb3 prod_db.dmp
36
Wednesday, September 18, 13

37. Restore
Options
Summary
37
Wednesday, September 18, 13

38. Point In Time
Recovery
(PITR)
38
Wednesday, September 18, 13

39. PITR Overview
• PITR Backups
• Archiving the WAL segments
• Making Base Backups
• PITR Recovery
• Restore the last Base Backup
• Prepare the recovered system data directory
• Create a recovery.conf file
• Start the postmaster
39
Wednesday, September 18, 13

40. PITR Setup
• Enable / set the following parameters in the
postgresql.conf file:
• wal_level = archive (or hot_standby)
• archive_mode = on
• archive_command = 'valid archive command'
Can be any valid shell command (including scripts)
• archive_timeout = [timeout]
• Special archive_command (and recovery.conf file) tags
• %p = full path (absolute path) and the file name of the WAL
segment to be archived
• %f = only the file name of the WAL segment
• %% = insert a % character in the command string.
40
Wednesday, September 18, 13

41. PITR Example
• Enable / set the following parameters in the postgresql.conf file:
• wal_level = archive
• archive_mode = on
• archive_command = 'cp %p /stage/wal/%f'
Can be any valid shell command (including scripts)
• archive_timeout = 0
• mkdir /stage/wal
• chown postgres:postgres /stage/wal
• Re-start the Server
41
Wednesday, September 18, 13

42. PITR Example - create transactions
• Execute SQL commands / transactions
• Enable access, turn on applications, etc
• This should force the creation of multiple archived WAL
files in the /stage/wal directory
• WAL segments are copied when:
• The WAL segment is full (see checkpoint_segments)
• Number of seconds specified in archive_timeout has passed
42
Wednesday, September 18, 13

43. PITR Example - create base backup
• Execute pg_start_backup
$ psql pitr_test
# select pg_start_backup ('tag') ;
• Archive the cluster data directory (and any related
tablespaces)
$ tar -czvf /backups/pitr/<date>.data.tar.gz ./data
rsync
other copy methods
• Execute pg_stop_backup
$ psql pitr_test
# select pg_stop_backup () ;
43
Wednesday, September 18, 13

44. PITR Example - create more transactions
• Execute SQL commands / transactions
• The application, user connections, etc will continue to
generate transactions (and archived WAL segments)
• Verify the creation of additional archived WAL files in
the /stage/wal directory
44
Wednesday, September 18, 13

45. PITR - recovery.conf file (common options)
Recovery settings are placed in the file 'recovery.conf'
• restore_command (string)
must return nonzero
• restore_command = 'cp /stage/wal/%f %p'
• restore_command = '/usr/local/bin/restore_shell.sh %p %f'
45
Wednesday, September 18, 13

46. PITR - recovery.conf file (common options)
recovery_target_time (timestamp)
• specifies the time stamp up to which recovery will proceed.
• recovery_target_time and recovery_target_xid are mutually
exclusive
• The default is to recover to the end of the WAL log.
46
Wednesday, September 18, 13

47. PITR Recovery
(1) If available copy the original cluster data directory to an
alternate location
if space is an issue at least copy the old pg_xlog dir it may contain
additional unarchived WAL segments
(2) Ensure the postmaster is not running
47
Wednesday, September 18, 13

48. PITR Recovery
If your backup was an rsync to a second server then skip steps 3 & 4
(3) Remove the cluster data directory and any tablespace
directories
(4) Restore your last system backup
• make sure permissions are retained
• If you're using tablespaces then verify that the symbolic links in
pg_tblspc/ were restored
48
Wednesday, September 18, 13

49. PITR Recovery
(5) Remove any wal segments from the pg_xlog dir that were
restored from the backup
If you didn't backup pg_xlog then create it, make sure you
re-establish it as a symbolic link if needed
If needed also re-create the pg_xlog/archive_status directory
(6) Copy the files from the original pg_xlog dir (if available) into
the new pg_xlog dir
do a copy as opposed to a move in case you need to start over
49
Wednesday, September 18, 13

50. PITR Recovery
(7) Create a recovery command (recovery.conf) in the cluster
data directory.
(8) [Optional] Temporarily modify pg_hba.conf to prevent
ordinary users from connecting until the recovery is complete
(9) Start the server.
The server will go into recovery mode via the recovery.conf file.
Once the recovery is complete then the server will become available and
rename the recovery.conf file to recovery.done
If an error interrupts the recovery (or stops the server) then simply
re-starting the server will restart the recovery
50
Wednesday, September 18, 13

51. PITR Recovery
(10) Verify the recovery.
If the database was not recovered properly (or to a state that you desire) then go
back to step 1
(11) restore the pg_hba.conf to its original state and run a
pg_ctl reload (if it was modified for the recovery)
51
Wednesday, September 18, 13

52. Questions?
Thank You!
52
Wednesday, September 18, 13