An admin application for editing database, but with configurable features (grouping and ordering of tables and columns, hyperlink navigation between related records, etc.)

1. 24.08.2014 - Page 1
Département
OfficeApp::AutoCRUD
An application for searching and
manipulating data in relational
databases
YAPC::EU::2014
laurent.dami@justice.ge.ch
Etat de Genève, Pouvoir Judiciaire
Département
Office

2. Agenda
• Introduction
• Overview (screenshots)
• Getting started
• Architecture
• Extensibility
• Conclusion

3. 24.08.2014 - Page 1
Département
Office
Introduction

4. Business need : tool for the support team
Regular
Users
database
Regular
App
Database
Admins
Admin
Tools
Support
Team
? ? ?

5. Data access for support team
Needs
• Browse DB schema
• CRUD on all data
• Navigate
– Follow relationships
– Optimize for most common
tables and colums
Solutions
• Program an admin GUI
–  too much work !
• Give them DBA tools
–  too low-level
–  no customization
 Need a CRUD app !

6. CRUD landscape
Admin tools
Oracle SQLDeveloper
SQLite Sqliteman
Postgres pgadmin
phpMyAdmin
...
Ready
to use
Needs
programming
Fixed
features
Customizable
Scaffolding tools
Ruby on Rails
Catalyst
CatalystX::CRUD
...
AutoCRUD apps
Catalyst::Plugin::AutoCRUD (2011)
App::AutoCRUD (Jan 2014)
RapidApp (Feb 2014)
[ WebAPI::DBIC ] (Jul 2014)
Interchange TableEditor (2013??)

7. App::AutoCRUD main features
• Powerful search syntax
• Distinctive URL for every resource
–  no session, all information in URL
–  easy admin links from the "real App"
• Joins as hyperlinks
• Bulk update / delete operations
• Customizable table order / column order
• Navigation through tree navigator

8. 24.08.2014 - Page 1
Département
Office
Quick overview
To run the demo :
cd examples
perl download_db.pl
plackup

9. Homepage
Specified in
config file
http://chinookdatabase.codeplex.com/
https://code.google.com/p/sakila-sample-database-ports/
Demo: free
sample
databases

10. Datasource entry point
Auto-generated
datamodel
Grouping, order & comments
from config file

11. Table description
Grouping, order & comments
from config file
Hyperlinks to foreign tables
from DB schema

12. Search form
Multiple values (OR)
LIKE
Multiple views
Choose
columns
BETWEEN

13. Results from search
Hyperlinks to foreign records or lists
Only show non-null columns
Also navigate through
LEFT/RIGHT arrow keys

14. Excel output

15. YAML output

16. Bulk update (selective)
2) Call the update form
1) Check records to be modified

17. Update form
Keys of selected records

18. Bulk update (from WHERE criteria)
Call the update form

19. Update form
Criteria for the update
Fields to update

20. ± RESTful URLs
• /Chinook/table/Track/search?GenreId=1,3&Name=Ab*
– GET  display pre-filled search form
– POST  redirect to list
• /Chinook/table/Track/list?GenreId=1,3&Name=Ab*
–  search results
• /Chinook/table/Track/id/399
–  single record
• /Chinook/table/Track/delete?where_pk.TrackId=399
– GET  display pre-filled delete form
– POST  database operation
Note : browser-oriented, not API-oriented
 No use of HTTP methods PUT and DELETE

21. 24.08.2014 - Page 1
Département
Office
Getting Started

22. YAML Config : connection parameters
app:
name: Test AutoCRUD
datasources :
Source1 :
dbh:
connect:
- dbi:SQLite:dbname=some_file
- "" # user
- "" # password
- RaiseError : 1
sqlite_unicode: 1

23. Config : specifying table grouping / ordering
tablegroups :
- name: Music
descr: Tables describing music content
node: open
tables :
- Artist
- Album
- Track
- name: Playlist
...

24. Config : specifying col grouping / ordering
Employee:
colgroups:
- name: keys
columns:
- name: EmployeeId
descr: primary key
- name: ReportsTo
descr: foreign key to the manager
- name: Employee details
columns:
- name: Title
- name: FirstName
- name: LastName

25. "crud.psgi" startup file
use App::AutoCRUD;
use YAML qw/LoadFile/;
my $config = LoadFile "/path/to/config.yaml";
my $crud = App::AutoCRUD->new(config => $config);
my $app = $crud->to_app;

26. Start the application
• From the command line
plackup crud.psgi
• Within Apache
<Location /crud>
SetHandler perl-script
PerlResponseHandler Plack::Handler::Apache2
PerlSetVar psgi_app /path/to/crud.psgi
</Location>
• Other possibilities : see L<Plack>

27. 24.08.2014 - Page 1
Département
Office
Architecture : external modules

28. External modules : Global view
Template
Toolkit
Alien:
:GvaScript
Prototype.js
Excel::Writer:
:XLSX
View
DBIx::DataModel
DBI
SQL::Abstract:
:More
Model
Plack
CGI::Expand
SQL::Abstract:
:FromQuery
Controller
Infrastructure
Moose Data::DomainYAML

29. Infrastructure modules
• Moose for OO
– Modern Perl OO framework
• YAML for config data
– Human-readable
– Support for directed acyclic graph (DAG)  reusable subtrees
– But can be replaced by JSON, XML, Config::General or Perl source code
• Data::Domain for data validation

30. Controller modules
• Plack for HTTP support
– Abstraction over various engines
– Many middleware options
• CGI::Expand for transforming form inputs
– <input name="foo.1.bar" value=1234>
–  { foo => [ {...}, { bar => 1234 } ] }
• SQL::Abstract::FromQuery for parsing user queries
– HTML form  SQL::Abstract::More  SQL
– Parses booleans, < > !=, LIKE, BETWEEN, etc.

31. Model modules
• DBI for database independance
– Perl standard for databases
• DBIx::DataModel for object-relational mapping
– Why not DBIC ? See
http://www.slideshare.net/ldami/dbixclass-vs-
dbixdatamodel
• SQL::Abstract::More for SQL generation
– Emit SQL from Perl datastructures

32. View modules
• Template Toolkit for HTML generation
• Alien::GvaScript for widgets
– Tree::Navigator
• Prototype.js for Javascript abstraction
– Navigator abstraction, OO programming, etc.
• Excel::Writer::XLSX for generating Excel worksheets
–

33. 24.08.2014 - Page 1
Département
Office
Architecture : internal structure

34. Internal structure : global view
AutoCRUD
AutoCRUD:
:View
AutoCRUD:
:Controller
AutoCRUD:
:Context
AutoCRUD:
:DataSource
Plack::Component Moose::Object
Controller:
:Table
Controller:
:Schema
View::XLSXView::TT ...

35. App::AutoCRUD
• Entry point
• Autodiscovers component subclasses
• isa Plack::Component
• Has
– Name
– Config
– Datasources

36. App::AutoCRUD::DataSource
• Encapsulates DBIx::DataModel schema
– Autogenerate Perl code if not already present
• Gathers and merges metadata
– From the DB schema
• Tables, columns, foreign keys
– From config
• Table groups, column groups, ordering, display parameters

37. App::AutoCRUD::Context
• Gathers contextual information for the current request
– Plack::Request object
– Datasource object
– View to apply
– Parameters for the view (for example TT template name)

38. Controller::Schema
• Lists table groups & tables
• Publishes the DBIx::DataModel schema

39. Controller::Table
• Implements CRUD operations
• Methods
– Descr
– List
– Id
– Search
– Update
– Delete
– Insert

40. View::*
• Render results
– TT
– XLSX
– YAML
– JSON
– ...

41. 24.08.2014 - Page 1
Département
Office
Extensibility

42. Regular subclassing
• Extend the app
package My::Private::AutoCRUD;
use Moose;
extends 'App::AutoCRUD';
• Subclass and redefine existing methods
– Automatic subclass discovery
– Good candidates :
• DataSource::prepare_for_request
• DataSource::_query_parser

43. Adding URLs
• New controllers
– Crud/<db_name>/<controller_name>/....
• New methods in existing controllers
– Crud/<db_name>/table/<table_name>/<method>/...

44. Adding Views
• Create View/XYZ.pm with "render" method
– Serves /path/to/resource.xyz?p1=v1&p2=v2&...

45. Relooking
• Override CSS
– My/Private/AutoCRUD/share/static/css/styles.css
• Override TT templates
– My/Private/AutoCRUD/share/templates/src/table/list.tt

46. 24.08.2014 - Page 1
Département
Office
Conclusion

47. Status
• Still young, lots of TODO
• Already in production at Geneva Justice
• Has some original features not seen other CRUD apps
• Could be useful for many people
– Also outside the Perl community 
•  Perl marketing needs general-purpose apps
– Needs stabilization & doc
– Needs marketing
– You can help !