Slides of my talk given at PGDay.EU in Stuttgart Germany in December 2010.

1. PL/Perl New Features
in PostgreSQL 9.0
Tim Bunce - Dec 2010
Creative Commons BY-NC-SA 3.0

2. PL/Perl Changes
‣ USER
‣ INTERNAL
‣ DBA
‣ NYTPROF
‣ FUTURE
Some features are in recent 8.x.y due to back-porting of security changes

3. PL/Perl Changes
‣ USER
‣ INTERNAL
‣ DBA
‣ NYTPROF
‣ FUTURE

4. New Builtins

5. quote_...
quote_literal( "foo" ) ==> "'foo'"
quote_literal( "don't "carp"" ) ==> "'don''t "carp"'"
quote_literal( "" ) ==> "''"
quote_nullable( "foo" ) ==> "'foo'"
quote_nullable( "don't "carp"" ) ==> "'don''t "carp"'"
quote_nullable( "" ) ==> "''"
quote_ident( "foo" ) ==> "foo"
quote_ident( "don't "carp"" ) ==> ""don't ""carp""""
quote_ident( "" ) ==> """"

6. quote_...(undef)
quote_literal( undef ) ==> undef
quote_nullable( undef ) ==> "NULL"
quote_ident( undef ) ==> """" (warn)

7. {en,de}code_bytea
encode_bytea( "foo" ) ==> "x666f6f"
decode_bytea( "x666f6f" ) ==> "foo"
decode_bytea( "146157157") ==> "foo"
encode_bytea( "x{263a}" ) ==> "xe298ba" UTF8
decode_bytea( "xe298ba" ) ==> "342230272" Not UTF8
encode_bytea( "" ) ==> "x"
decode_bytea( "x" ) ==> ""
decode_bytea( "" ) ==> ""
encode_bytea( undef ) ==> "x" (warn)
decode_bytea( undef ) ==> "" (warn)

8. looks_like_number
looks_like_number( 1 ) ==> 1
looks_like_number( 0 ) ==> 1
looks_like_number( "+7.2e-9" ) ==> 1
looks_like_number( "foo" ) ==> 0
looks_like_number( "" ) ==> 0
looks_like_number( undef ) ==> undef
looks_like_number( " 4 " ) ==> 1
looks_like_number( "5plus" ) ==> 0 (but '5plus'+0=5)

9. encode_array_*
encode_array_literal( ["foo","bar"] )
==> "{"foo", "bar"}"
encode_array_constructor( ["foo","bar"] )
==> "ARRAY['foo', 'bar']"
encode_array_literal( [1,[2,[undef]]] )
==> "{"1", {"2", {NULL}}}"
encode_array_constructor( [1,[2,[undef]]] )
==> "ARRAY['1', ARRAY['2', ARRAY[NULL]]]"

10. encode_array_*
encode_array_literal( "foo" ) ==> "foo"
encode_array_constructor( "foo" ) ==> "'foo'"
encode_array_literal( undef ) ==> undef
encode_array_constructor( undef ) ==> "NULL"

11. Trusted require/use
• require/use work for already loaded modules
use strict; # old way: BEGIN { strict->import(); }
• extra pre-loaded modules
use warnings;
use Carp;
use feature qw(say); # for perl 5.10 or later
use utf8; # if server_encoding is utf8

12. CONTEXT: ...
• PL/Perl tracks the context of log messages
- before:
WARNING: ...some warning from perl code...
- now:
WARNING: ...some warning from perl code...
CONTEXT: PL/Perl function "..."
- Thanks to Alexey Klyukin.

13. DO '...' LANGUAGE ...;
• Arbitrary chunks of code can be executed
directly from psql, or client apps, via DO
- Thanks to Petr Jelinek, Joshua Tolley, Hannu Valtonen
• No need to create and run a stored procedure
each time:
DO $$
spi_exec("... $_ ...") for 'a'..'z';
$$ language plperl;

14. Other Changes
• Using $a and $b in sort blocks now works!
• eval { ... } and eval "..." now work!
• END blocks are now run at end of session
- they can't (currently) access the database
• Warnings from perl are now WARNINGs
- they used to be NOTICE

15. PL/Perl Changes
‣ USER
‣ INTERNAL
‣ DBA
‣ NYTPROF
‣ FUTURE

16. INTERNAL
• The Safe module is no longer used for plperl
- Now faster, simpler, and more secure
• Validates return values are in server encoding
- ERROR: invalid byte sequence for encoding
- Thanks to Andrew Dunstan
• Internal code refactoring and cleanup

17. PL/Perl Changes
‣ USER
‣ INTERNAL
‣ DBA
‣ NYTPROF
‣ FUTURE

18. New plperl.* Config
• Specify perl code to run during initialization:
plperl.on_init = '...perl code...'
plperl.on_plperlu_init = '...perl code...'
plperl.on_plperl_init = '...perl code...'
• Can only be set by superuser or postgres.conf
• Code can't access the database

19. Interpreter Lifecycle

20. Birth
1. Perl interpreter created
2. Options from PERL5OPT env var are processed
3. PL/Perl support bootstrap code is executed
4. plperl.on_init code runs (unrestricted)
Above steps may happen in postmaster process at startup,
if plperl is loaded via shared_preload_libraries.
Otherwise they happen at first use.
No access to database.

21. Adolescence
6. Interpreter is specialised for plperl (if that’s used first)
• Modules loaded: strict, warnings, features, Carp
• Unsafe perl ops are restricted (require, open etc.)
• DynaLoader package is deleted
• plperl.on_plperl_init code runs (restricted)
7. Database access is enabled
8. Perl interpreter is made available for use
9. Executes whatever action called it into existence

22. Siblings
• If plplerlu code is run later
- then a new interpreter is created
- similarly if plperlu is run first and plperl run later
• If plplerl with a different security context is run
- then a new interpreter is created for the ROLE
- That’s a recent security fix:
http://wiki.postgresql.org/wiki/20101005securityrelease
Note impact on shared_preload_libraries in these cases

23. Death
• Finally, when the session ends:
- Access to the database is disabled
- END blocks, if any, are run (if exiting cleanly)

24. plperl.on_init
• Handy to set global perl configuation
plperl.on_init='use lib qw(/myapp); use ...;'
plperl.on_init='require "plperloninit.pl";'
Effectively defines ‘approved’ modules for plperl
• SECURITY RISK!
Only load modules you're happy for plperl code to use.
Also check any other modules loaded as dependencies!
Use Devel::TraceLoad to see what's actually loaded:
PERL5OPT='-MDevel::Trace=summary' pg_ctl ...

25. PL/Perl Best Practice
• Include explicit use statements in functions
For plperlu that'll actually load if needed
For plperl it'll check that module is loaded
- so you'll get an immediate clear failure if not
- (e.g., on a replica with old postgres.conf file)

26. plperl.on_plperl_init
• Originally intended for things like
- PGOPTIONS="-c plperl.on_plperl_init='...'"
- to enable debug or profiling for a session
• But...
• Can only be set by superuser or postgres.conf
- due to SECURITY DEFINER risk at the time
- that’s now been patched (20101005, CVE-2010-3433)
- so this restriction may be removed in future

27. PL/Perl Changes
‣ USER
‣ INTERNAL
‣ DBA
‣ NYTPROF
‣ FUTURE

28. Devel::NYTProf
Perl Source Code Profiler
PostgreSQL::PLPerl::NYTProf

29. Enabling NYTProf
• Via postgres.conf:
plperl.on_init='use PostgreSQL::PLPerl::NYTProf'
• Via environment variable:
PERL5OPT='-MPostgreSQL::PLPerl::NYTProf' pg_ctl ...
• Is immediately active for all connections.
• To enable on demand for one connection:
NYTPROF=start=no PERL5OPT=... pg_ctl ...
DO 'DB::enable_profile' LANGUAGE plperl;

30. Reporting from NYTProf
• Writes per-backend data files:
$PGDATA/nytprof.out.$pid
• To generate a report:
nytprofhtml --file=$PGDATA/nytprof.out.4321 --open

31. ~ Demo ~
Of plperl.on_init in postgresql.conf
And use of PostgreSQL::PLPerl::NYTProf
Screencast: http://timbunce.blip.tv/file/3691795/
Video: http://www.fosslc.org/drupal/content/plperl-new-features-90

32. PL/Perl Changes
‣ USER
‣ INTERNAL
‣ DBA
‣ NYTPROF
‣ FUTURE

33. Future Possibilities
• Optimize trigger handling overheads
• Remove overheads of threaded perls
• $array_ref = decode_array_literal(‘{...}’)
• $hash_ref = decode_hstore_literal(‘x=>42’)
• Array params as array refs (with overloading)
• Rewrite encode_array_literal in C.

34. Questions?
Tim.Bunce@pobox.com
http://blog.timbunce.org
@timbunce on twitter

35. http://xkcd.com/519/