Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
23.08.14 - Page 1
Département
Office
SQL::Abstract::FromQuery
Building DB requests from Web queries
YAPC::EU::2014, Sofia
...
Power search : the "widget" way
Source: https://rt.cpan.org/Search/Build.html
Power search : the "parsed" way
Name : Smi*
Gender : M
Salary : > 4000
Job : ! programmer, analyst
Birth : BETWEEN 01.01.1...
Agenda
• Background : SQL::Abstract & SQL::Abstract::More
• SQL::Abstract::FromQuery
– purpose
– API
• internals
– Regexp:...
Background : SQL::Abstract
SQL::Abstract::More
Translating Perl datastructures
into SQL
SQL::Abstract ('SQLA') : example
my $sqla = SQL::Abstract->new;
my ($sql, @bind) = $sqla->select(
$table_name,
[qw/col1 co...
Purpose
• generate SQL from Perl data structures
– direct from config or from user input
– avoid painful string manipulati...
"Special operators" in SQLA
• support for user-defined SQL extensions. For ex :
– fulltext index
– multivalued columns
– c...
SQL::Abstract::More ('SQLAM')
• fully compatible with SQLA
• named parameters
• more SQL constructs (group by, limit/offse...
23.08.14 - Page 1
Département
Office
SQLA, SQLAM are for programmers
 what about users ?
SQL
Perl
datastructure
SQLA, SQL...
HTTP query
Name : Smi*
Gender : M
Salary : > 4000
Job : ! programmer, analyst
Birth : BETWEEN 01.01.1970 AND 31.12.1990
Ad...
Expected SQLA structure
{
Name => {-like => 'Smi%'},
Gender => 'M',
Salary => {'>' => 4000},
Job => {-not_in => [qw/progra...
SYNOPSIS
my $parser = SQL::Abstract::FromQuery->new();
my $criteria = $parser->parse($http_query);
# $http_query is an obj...
Options
my $parser = SQL::Abstract::FromQuery->new(
# additional components (optional)
-components => [qw/FR Oracle/],
# g...
Components
• FR
– dates au format dd.mm.yyyy
– mots-clés en français (VRAI, FAUX, ENTRE, NUL, etc.)
– messages d'erreur en...
Field types : which grammar to apply
• Standard
– a plain value (number, string, date or time).
– a list of values, separa...
Internals
Regexp::Grammars
use Regexp::Grammars; qr{
<grammar: SQL::Abstract::FromQuery>
<rule: standard>
<MATCH=between> | <MATCH=o...
Autoactions hooked to the grammar
sub between {
my ($self, $h) = @_;
return {-between => [$h->{min}, $h->{max}]};
}
sub va...
Grammar Inheritance
<grammar: SQL::Abstract::FromQuery::FR>
<extends: SQL::Abstract::FromQuery>
<rule: null>
NULL?
<rule: ...
Rules inheritance
package SQL::Abstract::FromQuery::Oracle;
use parent 'SQL::Abstract::FromQuery';
use mro 'c3';
¨sub date...
23.08.14 - Page 1
Département
Office
Conclusion
Features
• flexible syntax for user input
– plain value
– list of values
– comparison operators
– patterns
– special const...
Usage
• See App::AutoCRUD
  come to tomorrow's talk 
Prochain SlideShare
Chargement dans…5
×

Sql abstract from_query

974 vues

Publié le

Parse search forms from applications and generate SQL

Publié dans : Technologie
  • Soyez le premier à commenter

Sql abstract from_query

  1. 1. 23.08.14 - Page 1 Département Office SQL::Abstract::FromQuery Building DB requests from Web queries YAPC::EU::2014, Sofia laurent.dami@justice.ge.ch Etat de Genève, Pouvoir Judiciaire Département Office
  2. 2. Power search : the "widget" way Source: https://rt.cpan.org/Search/Build.html
  3. 3. Power search : the "parsed" way Name : Smi* Gender : M Salary : > 4000 Job : ! programmer, analyst Birth : BETWEEN 01.01.1970 AND 31.12.1990 Address: ! NULL
  4. 4. Agenda • Background : SQL::Abstract & SQL::Abstract::More • SQL::Abstract::FromQuery – purpose – API • internals – Regexp::Grammars – multiple inheritance
  5. 5. Background : SQL::Abstract SQL::Abstract::More Translating Perl datastructures into SQL
  6. 6. SQL::Abstract ('SQLA') : example my $sqla = SQL::Abstract->new; my ($sql, @bind) = $sqla->select( $table_name, [qw/col1 col2 col3/], { col1 => 'val1', col2 => {-in => [qw/foo bar/]}, col3 => {-between => [qw/2012-01-01 2012-07-01/]}, col4 => {">" => 999}, col5 => {-like => 'foo%'}, }, [qw/col1 col2/], ); # SELECT col1, col2, col3 FROM $table_name # WHERE col1 = ? AND col2 IN (?, ?) AND col3 … etc # ORDER BY col1, col2
  7. 7. Purpose • generate SQL from Perl data structures – direct from config or from user input – avoid painful string manipulations with regex / join / etc. • placeholders and bind values – avoid datatype distinctions (strings / numbers /dates / etc.) – avoid SQL injection   used by DBIx::Class, DBIx::DataModel, …
  8. 8. "Special operators" in SQLA • support for user-defined SQL extensions. For ex : – fulltext index – multivalued columns – case-insensitive – LIKE / GLOB / etc. • useful for DBMS-independent queries in apps – the "special operator" translates to DBMS-specific syntax { col1 => {-contains => [qw/YAPC Sofia/]}, col2 => {-anyval => [12, 34, 56]}, col3 => {-glob => 'foo*'}, }
  9. 9. SQL::Abstract::More ('SQLAM') • fully compatible with SQLA • named parameters • more SQL constructs (group by, limit/offset, union/intersect, etc.) • more syntax (aliases, joins, ordering direction, etc.) my $sqla = SQL::Abstract::More->new; my ($sql, @bind) = $sqla->select( -columns => [qw/table1.col|ct1 table2.col|ct2 … /], -from => $table_name, -where => { col1 => 'val1', col2 => {-in => [qw/foo bar/]}, … }, -union => [-columns => …, -from => …, -where => …], -order_by => [qw/+col1 -col2/], );
  10. 10. 23.08.14 - Page 1 Département Office SQLA, SQLAM are for programmers  what about users ? SQL Perl datastructure SQLA, SQLAMHTTP query SQLA::FromQuery
  11. 11. HTTP query Name : Smi* Gender : M Salary : > 4000 Job : ! programmer, analyst Birth : BETWEEN 01.01.1970 AND 31.12.1990 Address: ! NULL
  12. 12. Expected SQLA structure { Name => {-like => 'Smi%'}, Gender => 'M', Salary => {'>' => 4000}, Job => {-not_in => [qw/programmer analyst/]}, Birth => {-between => [qw/1970-01-01 1990-12-31/]}, Address => {'<>' => undef}, } SELECT * FROM people WHERE Name LIKE 'Smi%' AND Gender = 'M' AND Salary > 4000 AND Job NOT IN ('programmer', 'analyst') AND Birth BETWEEN 1970-01-01 AND 1990-12-31 AND Address IS NOT NULL
  13. 13. SYNOPSIS my $parser = SQL::Abstract::FromQuery->new(); my $criteria = $parser->parse($http_query); # $http_query is an object with a ->param() method # .. or it can be just a hashref my ($sql, @bind) = $sqla->select($table, @cols, $criteria); # or my ($sql, @bind) = $sqlam->select( -columns => @cols, -from => $table, -where => $criteria, );
  14. 14. Options my $parser = SQL::Abstract::FromQuery->new( # additional components (optional) -components => [qw/FR Oracle/], # grammar rules for some specific fields (optional) -fields => { standard => [qw/field1 field2 .../], bool => [qw/bool_field1/], ... } );
  15. 15. Components • FR – dates au format dd.mm.yyyy – mots-clés en français (VRAI, FAUX, ENTRE, NUL, etc.) – messages d'erreur en français • Oracle – queries on date and time • ["to_date(?, '$date_format')", $date] • ["to_date(?, '$time_format')", $date]  WHERE D_BIRTH > TO_DATE('DD.MM.YYYY', ?) • Contains – generate fulltext queries • … (user-defined)
  16. 16. Field types : which grammar to apply • Standard – a plain value (number, string, date or time). – a list of values, separated by ','. – a negated value or list of values; negation is expressed by ! or != or - or <> – a comparison operator <=, <, >=, > followed by a plain value – the special word NULL – BETWEEN val1 AND val2 – boolean values YES, NO, TRUE or FALSE • Restricted – string / bool / date / … (user-defined)
  17. 17. Internals
  18. 18. Regexp::Grammars use Regexp::Grammars; qr{ <grammar: SQL::Abstract::FromQuery> <rule: standard> <MATCH=between> | <MATCH=op_and_value> | <MATCH=values> <rule: values> <[value]>+ % , <token: compare> <= | < | >= | > <rule: value> <MATCH=null> | <MATCH=date> | <MATCH=time> | <MATCH=string> <rule: date> <day=(dd?)>.<month=(dd?)>.<year=(ddd?d?)> | <year=(ddd?d?)>-<month=(dd?)>-<day=(dd?)>
  19. 19. Autoactions hooked to the grammar sub between { my ($self, $h) = @_; return {-between => [$h->{min}, $h->{max}]}; } sub values { my ($self, $h) = @_; my $n_values = @{$h->{value}}; return $n_values > 1 ? {-in => $h->{value}} : $h->{value}[0]; }
  20. 20. Grammar Inheritance <grammar: SQL::Abstract::FromQuery::FR> <extends: SQL::Abstract::FromQuery> <rule: null> NULL? <rule: between> <SQL::Abstract::FromQuery::between> | ENTRE (*COMMIT) (?: <min=value> ET <max=value> | <error:> ) <rule: bool> O(?:UI)? (?{ $MATCH = 1 }) | V(?:RAI)? (?{ $MATCH = 1 }) | NO?N? (?{ $MATCH = 0 }) | F(?:AUX|ALSE)? (?{ $MATCH = 0 }) | Y(?:ES)? (?{ $MATCH = 1 })
  21. 21. Rules inheritance package SQL::Abstract::FromQuery::Oracle; use parent 'SQL::Abstract::FromQuery'; use mro 'c3'; ¨sub date { my ($self, $h) = @_; my $date_format = $self->{date_format} || 'YYYY-MM-DD'; my $date = $self->next::method($h); return ["to_date(?, '$date_format')", $date]; }
  22. 22. 23.08.14 - Page 1 Département Office Conclusion
  23. 23. Features • flexible syntax for user input – plain value – list of values – comparison operators – patterns – special constructs (BETWEEN, MATCH, etc.) – Including SQLA "special ops" • internationalization • automate specific behaviours for some form fields – data conversion – inject special ops
  24. 24. Usage • See App::AutoCRUD   come to tomorrow's talk 

×