1. DBIx::Class Département
Office
vs.
DBIx::DataModel
FPW 2012, Strasbourg
laurent.dami@justice.ge.ch
Département
Office
29.06.12 - Page 1

2. Agenda
• Introduction
• Schema definition
• Data retrieval
• Joins
• Classes and methods
• Resultset/Statement objects
• Customization
• Conclusion

3. Introduction

4. ORM layer
Perl program
Object-Relational Mapper
DBI
DBD driver
Database

5. ORM useful for …
• dynamic SQL generation
– navigation between tables
– generate complex SQL queries from Perl datastructures
– better than phrasebook or string concatenation
• automatic data conversions (inflation / deflation)
• transaction encapsulation
• data validation
• computed fields
• caching
• expansion of tree data structures coded in the relational model
• …

6. CPAN ORM Landscape
• Discussed here
– DBIx::Class (a.k.a. DBIC)
– DBIx::DataModel (a.k.a. DBIDM)
DISCLAIMER
- I'm not an expert of DBIC
- I'll try to be neutral in comparisons, but …
- Won't cover all topics
• Many other
– Rose::DB, Jifty::DBI, Fey::ORM, ORM, DBIx::ORM::Declarative,
Tangram, Coat::Persistent, ORLite,
DBR, DBIx::Sunny, DBIx::Skinny, DBI::Easy, …

7. A bit of 2005 history
Class::DBI (1999)
SQL::Abstract (2001)
DBIx::DataModel private (feb.05)
Class::DBI::Sweet (29.05.05)
- SQL::Abstract
Class::DBI::Sweet 0.02 (29.06.05)
-prefetch joins
DBIx::Class 0.01 (08.08.05)
- SQL::Abstract
DBIx::Class 0.03 (19.09.05) DBIx::DataModel 0.10 (16.09.05)
- prefetch joins

8. Some figures
• DBIC • DBIDM
– thousands of users – a couple of users
– dozens of contributors – 1 contributor
– 162 files – 31 files
– 167 packages – 40 packages
– 1042 subs/methods – 175 subs/methods
– 16759 lines of Perl – 3098 lines of Perl
– 1817 comment lines – 613 comment lines
– 19589 lines of POD – 8146 lines of POD
– 70 transitive dependencies – 54 transitive dependencies

9. Our example : CPAN model
straight from CPAN::SQLite
Author
1
*
1 contains ► 1..*
Distribution Module
* *
depends_on ►

10. Département
Office
Schema definition
29.06.12 - Page 1

11. DBIC Schema class
Département
Office
use utf8;
package FPW12::DBIC;
use strict;
use warnings;
use base 'DBIx::Class::Schema';
__PACKAGE__->load_namespaces;
1;
29.06.12 - Page 1

12. DBIC 'Dist' table class
Département
Office
use utf8;
package FPW12::DBIC::Result::Dist;
use strict; use warnings;
use base 'DBIx::Class::Core';
__PACKAGE__->table("dists");
__PACKAGE__->add_columns(
dist_id => { data_type => "integer", is_nullable => 0,
is_auto_increment => 1 },
dist_vers => { data_type => "varchar", is_nullable => 1, size => 20 },
... # other columns
);
__PACKAGE__->set_primary_key("dist_id");
__PACKAGE__->belongs_to ('auth', 'FPW12::DBIC::Result::Auth', 'auth_id');
__PACKAGE__->has_many ('mods', 'FPW12::DBIC::Result::Mod', 'dist_id');
__PACKAGE__->has_many ('depends', 'FPW12::DBIC::Result::Depends', 'dist_id');
__PACKAGE__->many_to_many(prereq_mods => 'depends', 'mod');
# idem for classes Auth, Mod, etc.
29.06.12 - Page 1

13. DBIDM centralized declarations
use DBIx::DataModel;
DBIx::DataModel->Schema("FPW12::DBIDM")
# class table primary key
# ===== ===== ===========
->Table(qw/Auth auths auth_id /)
->Table(qw/Dist dists dist_id /)
->Table(qw/Mod mods mod_id /)
->Table(qw/Depends depends dist_id mod_id/)
# class role multipl. (optional join keys)
# ===== ==== ======= ====================
->Association([qw/Auth auth 1 auth_id /],
[qw/Dist dists * auth_id /])
->Composition([qw/Dist dist 1 /],
[qw/Mod mods * /])
->Association([qw/Dist dist 1 /],
[qw/Depends depends * /])
->Association([qw/Mod mod 1 /],
[qw/Depends depends * /])
->Association([qw/Dist used_in_dist * depends distrib /],
[qw/Mod prereq_mods * depends mod /])
;

14. Schema def. comparison
Département
Office
• DBIC • DBIDM
– one file for each class – centralized file
– regular Perl classes – dynamic class creation
– full column info – no column info (except pkey)
– 1-way "relationships" – 2-ways "associations" with
• belongs_to • multiplicities
• has_many • role names
• may_have • diff association/composition
• ….
– custom join conditions – custom column names
• any operator • only equalities
• 'foreign' and 'self'
29.06.12 - Page 1

15. Département
Office
Data retrieval
29.06.12 - Page 1

16. DBIC: Search
Département
Office
use FPW12::DBIC;
my $schema = FPW12::DBIC->connect($data_source);
my @dists = $schema->resultset('Dist')->search(
{dist_name => {-like => 'DBIx%'}},
{columns => [qw/dist_name dist_vers/]},
);
foreach my $dist (@dists) {
printf "%s (%s)n", $dist->dist_name, $dist->dist_vers;
}
29.06.12 - Page 1

17. DBIDM: Search
Département
Office
use FPW12::DBIDM;
use DBI;
my $datasource = "dbi:SQLite:dbname=../cpandb.sql";
my $dbh = DBI->connect($datasource, "", "",
{RaiseError => 1, AutoCommit => 1});
FPW12::DBIDM->dbh($dbh);
my $dists = FPW12::DBIDM::Dist->select(
-columns => [qw/dist_name dist_vers/],
-where => {dist_name => {-like => 'DBIx%'}},
);
foreach my $dist (@$dists) {
print "$dist->{dist_name} ($dist->{dist_vers})n";
}
29.06.12 - Page 1

18. Simple search comparison
Département
Office
• DBIC • DBIDM
– schema is an object – schema is a class (default) or an object
– result is a list or a resultset object – result is an arrayref (default)
– accessor methods for columns – hash entries for columns
– uses SQL::Abstract – uses SQL::Abstract::More
• mostly hidden • user can supply a custom $sqla obj
29.06.12 - Page 1

19. SQL::Abstract new()
Département
Office
• special operators
– ex: DBMS-independent fulltext search
# where {field => {-contains => [qw/foo bar/]}
my $sqla = SQL::Abstract->new(special_ops => [
{regex => qr/^contains(:?_all|_any)?$/i,
handler => sub {
my ($self, $field, $op, $arg) = @_;
my $connector = ($op =~ /any$/) ? ' | ' : ' & ';
my @vals = ref $arg ? @$arg : ($arg);
@vals = map { split /s+/ } grep {$_} @vals;
my $sql = sprintf "CONTAINS($field, '%s') > 0",
join $connector, @vals;
return ($sql); # no @bind
}
]);
29.06.12 - Page 1

20. DBIC: Find single record
Département
Office
my $auth1 = $schema->resultset('Auth')
->find(123);
my $auth2 = $schema->resultset('Auth')
->find({cpanid => 'DAMI'});
29.06.12 - Page 1

21. DBIDM: Find single record
Département
Office
my $auth1 = FPW12::DBIDM::Auth->fetch(123);
my $auth2 = FPW12::DBIDM::Auth->search(
-where => {cpanid => 'DAMI'},
-result_as => 'firstrow',
);
29.06.12 - Page 1

22. DBIC: Search args
Département
Office
• 1st arg : "where" criteria
• 2nd arg : attributes
– distinct
– order_by
– group_by, having
– columns # list of columns to retrieve from main table
– +columns # additional columns from joined tables or from functions
– join # see later
– prefetch # see later
– for
– page, rows, offset, etc.
– cache
29.06.12 - Page 1

23. DBIDM: Select args
my $result = $source->select(
-columns => @columns,,
-where => %where,
-group_by => @groupings,
-having => %criteria,
-order_by => @order,
-for => 'read only',
-post_SQL => sub { … },
-pre_exec => sub { … },
-post_exec => sub { … },
-post_bless => sub { … },
-page_size => …, -page_index => …,
-limit => …, -offset => …,
-column_types => %types,
-result_as => $result_type,
);

24. DBIDM: Polymorphic result
-result_as =>
– 'rows' (default) : arrayref of row objects
– 'firstrow' : a single row object (or undef)
– 'hashref' : hashref keyed by primary keys
– [hashref => @cols] : cascaded hashref
– 'flat_arrayref' : flattened values from each row
– 'statement' : a statement object (iterator)
– 'fast_statement' : statement reusing same memory
– 'sth' : DBI statement handle
– 'sql' : ($sql, @bind_values)
– 'subquery' : ["($sql)", @bind]
  don't need method variants : select_hashref(), select_arrayref(), etc.

25. DBIDM: Fast statement
• like a regular statement
– but reuses the same memory location for each row
– see DBI::bind_col()
my $statement = $source->select(
. . . ,
-result_as => 'fast_statement'
);
while (my $row = $statement->next) {
. . .
# DO THIS : print $row->{col1}, $row->{col2}
# BUT DON'T DO THIS : push @results, $row;
}

26. Advanced search comparison
• DBIC • DBIDM
– 'find' by any unique constraint – 'fetch' by primary key only
– "inner" vs "outer" columns – all columns equal citizens
– result is context-dependent – polymorphic result
– optional caching – no caching
– statement-specific inflators
– callbacks

27. Joins

28. Task : list distributions and their authors
Author
1
*
1 contains ► 1..*
Distribution Module
* *
depends_on ►

29. DBIC: Join
my @dists = $schema->resultset('Dist')->search(
{dist_name => {-like => 'DBIx%'}},
{columns => [qw/dist_name dist_vers/],
join => 'auth',
'+columns' => [{'fullname' => 'auth.fullname'}],
},
);
foreach my $dist (@dists) {
printf "%s (%s) by %sn",
$dist->dist_name, $dist->dist_vers,
$dist->get_column('fullname'); # no accessor meth
}

30. DBIC: Join with prefetch
my @dists = $schema->resultset('Dist')->search(
{dist_name => {-like => 'DBIx%'}},
{columns => [qw/dist_name dist_vers/],
prefetch => 'auth',
},
);
foreach my $dist (@dists) {
printf "%s (%s) by %sn",
$dist->dist_name, $dist->dist_vers,
$dist->auth->fullname; # already in memory
}

31. DBIDM: Join
my $rows = FPW12::DBIDM->join(qw/Dist auth/)->select(
-columns => [qw/dist_name dist_vers fullname/],
-where => {dist_name => {-like => 'DBIx%'}},
);
foreach my $row (@$rows) {
print "$row->{dist_name} ($row->{dist_vers}) "
. "by $row->{fullname}n";
}
Dist Auth
DBIDM::Source::Join
new class created on the fly ..::AutoJoin::…

32. Multiple joins
• DBIC
join => [ { abc => { def => 'ghi' } },
{ jkl => 'mno' },
'pqr' ]
• DBIDM
->join(qw/Root abc def ghi jkl mno pqr/)

33. Join from an existing record
• DBIC
my $rs = $auth->dists(undef, {join|prefetch => 'mods'});
• DBIDM
my $rows = $auth->join(qw/dists mods/)->select;

34. DBIC: left/inner joins
• attribute when declaring relationships
# in a Book class (where Author has_many Books)
__PACKAGE__->belongs_to( author => 'My::DBIC::Schema::Author',
'author',
{ join_type => 'left' } );
• cannot change later (when invoking the relationship)

35. DBIDM: Left / inner joins
->Association([qw/Author author 1 /],
[qw/Distribution distribs 0..* /])
# default : LEFT OUTER JOIN
->Composition([qw/Distribution distrib 1 /],
[qw/Module modules 1..* /]);
# default : INNER JOIN
# but defaults can be overridden
My::DB->join([qw/Author <=> distribs/)-> . . .
My::DB->join([qw/Distribution => modules /)-> . . .

36. Join comparison
• DBIC • DBIDM
– rows belong to 1 table class only – rows inherit from all joined tables
– joined columns are either – flattening of all columns
• "side-products",
• prefetched (all of them, no choice)
– fixed join type – default join type, can be overridden

37. Speed

38. Select speed
Département
Office
list mods join Auth dist mods
(109349 rows) (113895 rows)
DBI 0.43 secs 1.36 secs
DBIC regular 11.09 secs 15.50 secs
DBIC prefetch n.a. 146.29 secs
DBIC hashref inflator 10.06 secs 14.17 secs
DBIDM regular 4.00 secs 5.01 secs
DBIDM 2.25 secs 3.28 secs
fast_statement
29.06.12 - Page 1

39. Insert speed
Département
Office
insert (72993 rows)
DBI 5.8 secs
DBIC regular 40.35 secs
DBIC populate 5.6 secs
DBIDM regular 32.81 secs
DBIDM bulk 32.57 secs
29.06.12 - Page 1

40. Département
Office
Classes and methods
29.06.12 - Page 1

41. DBIC : methods of a row
DB<1> m $auth # 152 methods DB<4> x mro::get_linear_isa(ref $auth)
_auth_id_accessor 0 ARRAY(0x13826c4)
_cpanid_accessor 0 'FPW12::DBIC::Result::Auth'
_email_accessor 1 'DBIx::Class::Core'
_fullname_accessor 2 'DBIx::Class::Relationship'
_result_source_instance_accessor 3 'DBIx::Class::Relationship::Helpers'
add_to_dists 4 'DBIx::Class::Relationship::HasMany'
auth_id 5 'DBIx::Class::Relationship::HasOne'
cpanid 6 'DBIx::Class::Relationship::BelongsTo'
dists 7 'DBIx::Class::Relationship::ManyToMany'
dists_rs 8 'DBIx::Class'
email 9 'DBIx::Class::Componentised'
fullname 10 'Class::C3::Componentised'
result_source_instance 11 'DBIx::Class::AccessorGroup'
via DBIx::Class::Core -> DBIx::Class::Relationship 12 'Class::Accessor::Grouped'
-> DBIx::Class::Relationship::Helpers -> 13 'DBIx::Class::Relationship::Accessor'
DBIx::Class::Relationship::HasMany: has_many 14 'DBIx::Class::Relationship::CascadeActions'
via DBIx::Class::Core -> DBIx::Class::Relationship
-> DBIx::Class::Relationship::Helpers -> 15 'DBIx::Class::Relationship::ProxyMethods'
DBIx::Class::Relationship::HasOne: 16 'DBIx::Class::Relationship::Base'
_get_primary_key 17 'DBIx::Class::InflateColumn'
via DBIx::Class::Core -> DBIx::Class::Relationship 18 'DBIx::Class::Row'
-> DBIx::Class::Relationship::Helpers -> 19 'DBIx::Class::PK::Auto'
DBIx::Class::Relationship::HasOne: _has_one
20 'DBIx::Class::PK'
via DBIx::Class::Core -> DBIx::Class::Relationship
-> DBIx::Class::Relationship::Helpers -> 21 'DBIx::Class::ResultSourceProxy::Table'
DBIx::Class::Relationship::HasOne: 22 'DBIx::Class::ResultSourceProxy'
_validate_has_one_condition
via DBIx::Class::Core -> DBIx::Class::Relationship
-> DBIx::Class::Relationship::Helpers ->
DBIx::Class::Relationship::HasOne: has_one
...

42. DBIDM : methods of a row
DB<1> m $auth # 36 methods DB<4> x mro::get_linear_isa(ref $auth)
dists
insert_into_dists 0 ARRAY(0x145275c)
metadm 0 'FPW12::DBIDM::Auth'
via DBIx::DataModel::Source::Table:
_get_last_insert_id 1 'DBIx::DataModel::Source::Table'
via DBIx::DataModel::Source::Table: 2 'DBIx::DataModel::Source'
_insert_subtrees
via DBIx::DataModel::Source::Table: _rawInsert
via DBIx::DataModel::Source::Table: _singleInsert
via DBIx::DataModel::Source::Table:
_weed_out_subtrees
via DBIx::DataModel::Source::Table: delete
via DBIx::DataModel::Source::Table:
has_invalid_columns
via DBIx::DataModel::Source::Table: insert
via DBIx::DataModel::Source::Table: update
via DBIx::DataModel::Source::Table ->
DBIx::DataModel::Source: TO_JSON
via DBIx::DataModel::Source::Table ->
DBIx::DataModel::Source: apply_column_handler
via DBIx::DataModel::Source::Table ->
DBIx::DataModel::Source: auto_expand
via DBIx::DataModel::Source::Table ->
DBIx::DataModel::Source: bless_from_DB
via DBIx::DataModel::Source::Table ->
DBIx::DataModel::Source: expand
via DBIx::DataModel::Source::Table ->
DBIx::DataModel::Source: fetch

43. DBIC classes
Componentised AccessorGroup
BelongsTo, HasMany, etc.
BelongsTo, HasMany, etc.
Class
RLHelpers RLBase Row RSProxy ResultSource
RS::Table
Relationship InflateColumn PK::Auto PK RSProxy::Table
Schema Core ResultSet
application classes
My::DB My::DB::Result::Table_n My::DB::ResultSet::Table_n
objects
schema row resultset
instantiation inheritance delegation

44. DBIDM classes
Source
Schema Table Join Statement
application
classes
My::DB My::DB::Table_n
My::DB::AutoJoin::
objects
schema row row statement

45. DBIDM Meta-Architecture
Meta::Schema Meta::Association Meta::Path Meta::Source Meta::Type
meta::assoc meta::path
Meta::Table Meta::Join
meta::type
Schema Table Join
My::DB meta::schema
My::DB::Table_n meta::table
My::DB::Auto_join meta::join

46. DBIC introspection methods
• Schema
– sources(), source($source_name)
• ResultSource
– primary_key
– columns(), column_info($column_name)
– relationships(), relationship_info($relname)

47. Architecture comparison
• DBIC • DBIDM
– very complex class structure – classes quite close to DBI concepts
– no distinction front/meta layser – front classes and Meta classes
– methods for – methods for
• CRUD • CRUD
• navigation to related objs • navigation to related objs
• introspection • access to meta
• setting up columns
• setting up relationships
• …

48. Département
Office
Resultset/Statement objects
29.06.12 - Page 1

49. Example task
Département
Office
• list names of authors
– of distribs starting with 'DBIx'
– and version number > 2
29.06.12 - Page 1

50. DBIC ResultSet chaining
Département
Office
my $dists = $schema->resultset('Dist')->search(
{dist_name => {-like => 'DBIx%'}},
);
my $big_vers = $dists->search({dist_vers => { ">" => 2}});
my @auths = $big_vers->search_related('auth', undef,
{distinct => 1, order_by => 'fullname'});
say $_->fullname foreach @auths;
# Magical join ! Magical "group by" !
SELECT auth.auth_id, auth.email, auth.fullname, auth.cpanid
FROM dists me JOIN auths auth ON auth.auth_id = me.auth_id
WHERE ( ( dist_vers > ? AND dist_name LIKE ? ) )
GROUP BY auth.auth_id, auth.email, auth.fullname, auth.cpanid ORDER BY fullname
29.06.12 - Page 1

51. DBIDM Statement lifecycle
new()
schema + source
new
refine()
sqlize() bind()
-post_SQL sqlized
prepare() bind()
-pre_exec prepared
execute() bind()
-post_exec executed
execute() -post_bless
next() / all() bind()
column types applied blessed data row(s)

52. DBIDM Statement::refine()
my $stmt = FPW12::DBIDM->join(qw/Dist auth/)->select(
-where => {dist_name => {-like => 'DBIx%'}},
-result_as => 'statement',
);
$stmt->refine(
-columns => [-DISTINCT => 'fullname'],
-where => {dist_vers => { ">" => 0}},
-order_by => 'fullname',
);
say $_->{fullname} while $_ = $stmt->next;

53. DBIDM subquery
my $stmt = FPW12::DBIDM::Dist->select(
-where => {dist_name => {-like => 'DBIx%'}},
-result_as => 'statement',
);
my $subquery = $stmt->select(
-columns => 'auth_id',
-where => {dist_vers => { ">" => 2}},
-result_as => 'subquery',
);
my $auths = FPW12::DBIDM::Auth->select(
-columns => 'fullname',
-where => {auth_id => {-in => $subquery}},
);
say $_->{fullname} foreach @$rows;

54. Statement comparison
• DBIC • DBIDM
– powerful refinement constructs – limited refinement constructs
• more "where" criteria – explicit control of status
• navigation to related source
• column restriction
• aggregation operators (e.g.
"count")

55. Département
Office
Other features
29.06.12 - Page 1

56. Transactions
Département
Office
• DBIC • DBIDM
$schema->txn_do($sub); $sch->do_transaction($sub);
– can be nested – can be nested
– can have intermediate savepoints – no intermediate savepoints
29.06.12 - Page 1

57. DBIC inflation/deflation
__PACKAGE__->inflate_column('insert_time',
{inflate => sub { DateTime::Format::Pg->parse_datetime(shift); },
deflate => sub { DateTime::Format::Pg->format_datetime(shift) },
});

58. DBIDM Types (inflate/deflate)
# declare a Type
My::DB->Type(Multivalue =>
from_DB => sub {$_[0] = [split /;/, $_[0]] },
to_DB => sub {$_[0] = join ";", @$_[0] },
);
# apply it to some columns in a table
My::DB::Author->metadm->define_column_type(
Multivalue => qw/hobbies languages/,
);

59. Extending/customizing DBIC
• Subclassing
– result classes
– resultset classes
– storage
– sqlmaker

60. Extending / customizing DBIDM
• Schema hooks for
– SQL dialects (join syntax, alias syntax, limit / offset, etc.)
– last_insert_id
• Ad hoc subclasses for
– SQL::Abstract
– Table
– Join
– Statements
• Statement callbacks
• Extending table classes
– additional methods
– redefining _singleInsert method

61. Not covered here
• inserts, updates, deletes
• Schema generator
• Schema versioning
• inflation/deflation

62. Département
Office
THANK YOU FOR YOUR ATTENTION
29.06.12 - Page 1

63. Département
Office
Bonus slides
29.06.12 - Page 1

64. Why hashref instead of OO accessors ?
• Perl builtin rich API for hashes (keys, values, slices,
string interpolation)
• good for import / export in YAML/XML/JSON
• easier to follow steps in Perl debugger
• faster than OO accessor methods
• visually clear distinction between lvalue / rvalue
– my $val = $hashref->{column};
– $hashref->{column} = $val;
• visually clear distinction between
– $row->{column} / $row->remote_table()

65. SQL::Abstract::More : extensions
• -columns => [qw/col1|alias1 max(col2)|alias2/]
– SELECT col1 AS alias1, max(col2) AS alias2
• -columns => [-DISTINCT => qw/col1 col2 col3/]
– SELECT DISTINCT col1, col2, col3
• -order_by => [qw/col1 +col2 –col3/]
– SELECT … ORDER BY col1, col2 ASC, col3 DESC
• -for => "update" || "read only"
– SELECT … FOR UPDATE