SlideShare une entreprise Scribd logo
1  sur  86
Chicago, October 19 - 22, 2010


Using Spring with NoSQL
databases
Mark Pollack
Chris Richardson
Goal of this talk



• The benefits and drawbacks of
       NoSQL databases
  • How the Spring Data project
   simplifies the development of
       NoSQL applications
About Chris


              •Grew up in England and live in Oakland, CA
              •Over 25+ years of software development
              experience including 14 years of Java
              •Speaker at JavaOne, SpringOne, NFJS,
              JavaPolis, Spring Experience, etc.
              •Organize the Oakland JUG and the Groovy
              Grails meetup




               http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
About Mark

• Spring committer since 2003
• Founder of Spring.NET
• Work on some other new open source projects
   – Spring Gemfire
   – Spring AMQP
• Used to work on large scale data collection/analysis in
  High Energy Physics (~10yrs ago)
Agenda

•   Why NoSQL?
•   Some NoSQL Databases
•   Spring NoSQL projects
•   Demos/Code/Deep dive
Why NoSQL?
Relational databases are great

• Well understood by developers
• Well supported by frameworks and tools, e.g. Spring JDBC,
  Hibernate, JPA
• Well understood by operations
   –   Configuration
   –   Care and feeding
   –   Backups
   –   Tuning
   –   Failure and recovery
   –   Performance characteristics



• But….
The trouble with relational database

• Object/relational impedance mismatch
   – Complicated to map rich domain model to relational schema
   – Relational schema is rigid
• Extremely difficult/impossible to scale writes:
   – Vertical scaling is limited/expensive
   – Horizontal scaling is limited or requires $$
• Performance can be suboptimal
Why NoSQL?
 Internet scale is the primary driver in the use of non-relational DBs
    Large Data Size – TBs to PBs
    High Transaction rates
    Need for High Availability




                                                Pictures courtesy of Gilt Group
Relational DBs
       Harder to scale to larger size
       Transactions and joins reduce access rates
Different priorities

• Relational Databases with ACID semantics
   – Atomic, Consistent, Isolated, Durable
• Internet Systems with BASE semantics
   – Basically Available, Scalable, Eventually Consistent

• NoSQL/BASE values remove standard DB behaviors
   – No Joins
   – Very limited "transactions"
   – Different Durability/consistency guarantees

• NoSQL datastores emerge as “point solutions”
   – Amazon/Google papers
   – Facebook, LinkedIn …
The NoSQL Landscape
   There are a variety of non-relational datastores
        Many are Open Source
        Many with companies behind them
        Rapidly changing feature sets
                                                                             Scalaris
Neo4J        db4o          Objectivity    Amazon Dynamo

                                                Terrastore        Gemfire     Sones
                   HBase           CouchDB
InfoGrid                                                RavenDB
           Hadoop       Scalaris                                      Tokyo Cabinet
                                          MongoDB
      Hypertable           Cassandra                    Azure Table         Coherence
Mnesia              Voldemort            Gigaspaces
           Riak
                                                                 Amazon SimpleDB
                      Redis      BerkeleyDB     MemcacheDB
           Tamino
                                                 InfiniteGraph        Chordless
  Expect consolidation over time
  With so many choices, picking which one to use can be difficult
Stops on a tour through NoSQL-land
 What is the data model?
 How to I access my data?
 What are the primary use cases?
 What notable companies that are using it?
Data Model
 Key Value
                                K1   V1
   Key -> value                 K2   V2
   Redis, Riak, Voldemort…      K3   V2

   “Amazon Dynamo inspired”


 Column
   Key -> ‘column families’
   HBase, Cassandra
   “Google Bigtable inspired”
Data Model
Document
  Collections containing semi-       { id: ‘4b2b9f67a1f631733d917a7b"),’
                                       author: ‘joe’,
  structured data                      tags : [‘example’, ‘db’],
                                       comments : [ { author: 'jim', comment: 'OK' },
                                                    { author: ‘ida', comment: ‘Bad' }
        key-values , JSON, XML,                   ]

  CouchDB, MongoDB                   { id: ‘4b2b9f67a1f631733d917a7c"),
                                       author: ‘ida’, ...
                                     { id: ‘4b2b9f67a1f631733d917a7d"),
                                       author: ‘jim’, ...




Graph
  Graph: nodes and edges
        Edges can be directional
        Each can contain key-value
        attributes
  Neo4j, Sones, InfiniteGraph
How to I access my data?
How to I access my data?
 Query Mechanism
   Key lookup
   Map-Reduce
   Query by example
   Query language
 ‘out of box’ query power usually linked to data model
 Increasing power
   KV -> Column -> Document/Graph
 Counter examples
   Riak is key-value and supports Map-Reduce
   Spring Data Mapper…


    .
Reality Check - Relational DBs are not going away

    NoSQL usage small
    by comparison…
    But growing…
Future = multi-paradigm data storage for
 enterprise applications




IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg
Agenda

•   Why NoSQL?
•   Some NoSQL Databases
•   Spring NoSQL projects
•   Demos/Code/Deep dive
Redis
• Advanced key-value store
   – Think memcached on steroids (the good kind)
   – Values can be binary strings, Lists, Sets, Ordered    K1   V1
     Sets, Hash maps, ..
   – Operations for each data type, e.g. appending         K2   V2
     to a list, adding to a set, retrieving a slice of a
                                                           K3   V2
     list, …
• Very fast: ~100K operations/second on entry-
  level hardware
• Persistent
   – Periodic snapshots of memory OR append
     commands to log file
   – Limits are size of keys retained in memory.
• Has “transactions”
• Libraries – Many languages, all separate
  open source projects
Redis CLI
redis> sadd myset a
(integer) 1
redis> sadd myset b
(integer) 1
redis> smembers myset
1. "a"
2. "b"
redis> srem myset a
(integer) 1
redis> smembers myset
1. "b"
Scaling Redis

• Master/slave replication
   – Tree of Redis servers
   – Non-persistent master can replicate to a persistent slave
   – Use slaves for read-only queries
• Sharding
   – Client-side only – consistent hashing based on key
   – Server-side sharding – coming one day
• Run multiple servers per physical host
   – Leverage multiple CPUs
   – 32 bit more efficient than 64 bit
• Optional "virtual memory"
   – Ideally data should fit in RAM
   – Values (not keys) written to disc
Redis use cases

• Use in conjunction with another database as the SOR
• Drop-in replacement for Memcached
    – Session state
    – Cache of data retrieved from SOR
•   Denormalized datastore for high-performance queries
•   Hit counts using INCR command
•   Randomly selecting an item – SRANDMEMBER
•   Queuing – Lists with LPOP, RPUSH, ….
•   High score tables – Sorted sets

• Notable users: github, guardian.co.uk, ….
Cassandra

• Originally developed by Facebook for inbox search
   – Now an Apache open-source project
• Extremely scalable
• Data is replicated and sharded
• The data model will hurt your brain
   • Column-oriented database
   • 4 or 5-dimensional hash map
Cassandra data model – insert/update
  My Column family (within a key space)
    Keys    Columns

     a      colA: value1    colB: value2   colC: value3          This can
                                                                 be data!
     b       colA: value    colD: value     colE: value



                      Insert(key=a, columName=colZ, value=foo)

    Keys    Columns

     a      colA: value1    colB: value2   colC: value3     colZ: foo


     b       colA: value    colD: value     colE: value
Cassandra query – range slice
   Keys    Columns

    a       colA: value1   colB: value2   colC: value3   colZ: foo


    b       colA: value    colD: value    colE: value



          rangeSlice(startKey=a, endKey=b,
                     startColumn=colA, endColumnName=colB)

   Keys    Columns

    a       colA: value1   colB: value2

    b       colA: value
Cassandra CLI

$ bin/cassandra-cli
cassandra> connect localhost/9160
Connected to "Test Cluster" on localhost/9160
cassandra> get Keyspace1.Standard1['foo']
Returned 0 results
cassandra> set Keyspace1.Standard1['foo']['a'] = '2'
Value inserted
cassandra> get Keyspace1.Standard1['foo']
=> (Column=61, value=1, timestamp...)
cassandra> set Keyspace1.Standard1['foo']['bar'] = 'abc'
Value inserted
cassandra> get Keyspace1.Standard1['foo']
=> (Column=626172, value=abc, timestamp...)
=> (Column=61, value=1, timestamp...)
cassandra> count Keyspace1.Standard1['foo']
2 columns
cassandra>
Scaling Cassandra through sharding/replication
                Keys = [D, A]                       •Client connects to any node
                                                    •Dynamically add/remove
                                                    nodes
                                Node 1
                                                    •Configurable # replicas
                           Token = A                •Reads/Writes specify how
                                                    many nodes
                Replicates to     Replicates to        Keys = [A, B]

           Node 4                                 Node 2
         Token = D                            Token = B

Keys = [C, D]   Replicates to                Replicates to

                                Node 3
                           Token = C

                                       Keys = [B, C]
Cassandra use cases

• Use cases
   • Big data
   • Persistent cache
   • (Write intensive) Logging

• Who is using it
   – Digg, Facebook, Twitter, Reddit, Rackspace
   – Cloudkick, Cisco, SimpleGeo, Ooyala, OpenX
   – “The largest production cluster has over 100 TB of data in
     over 150 machines.“ – Casssandra web site
MongoDB

• JSON-style documents
    – Lists, Maps, Primitive values
• Documents organized into collections (~table)
• Full or partial document updates
    – Transactional update in place on one document
    – Atomic Modifiers
•   GridFS for efficiently storing large files
•   Index support – secondary and compound
•   Rich query language for dynamic queries
•   Map/Reduce
•   Replication and Auto Sharding
MongoDB Data Model
                               {
                                 "name" : "Ajanta",
• BSON data types                "type" : "Indian",
                                 "serviceArea" : [
• Documents types                   "94619",
  – Data                            "94618"
                                 ],
  – Query                        "openingHours" : [
  – Modifier for updates            {
• Query and Modify                     "dayOfWeek" : Monday,
                                       "open" : 1730,
  operators                            "close" : 2130
  – $tl, $gt, $ne, $in, $or,        }
    $elemMatch, $orderBy,        ],
    $group,                      "_id" :
                               ObjectId("4bddc2f49d1505567c
  – $where + javascript
                               6220a0")
  – $inc, $set, $push, $pull   }
MongoDB query by example

• Find a restaurant that serves the 94619 zip code and is
  open at 6pm on a Monday
   {
     serviceArea:"94619",
     openingHours: {
       $elemMatch : {
             "dayOfWeek" : Monday,
             "open": {$lte: 1800},
             "close": {$gte: 1800}
        }
     }
   }       DBCursor cursor = collection.find(qbeObject);
           while (cursor.hasNext()) {
              DBObject o = cursor.next();
              …
            }
MongoDB CLI
$ bin/mongo
> use mydb
> r1 = {name: 'Ajanta'}
{name: 'Ajanta'}
> r2 = {name: 'Montclair Egg Shop'}
{name: 'Montclair Egg Shop'}
> db.restaurants.save(r1)
> r1
{ _id: ObjectId("98…"), name: "Ajanta"}
> db.restaurants.save(r2)
> r2
{ _id: ObjectId("66…"), name: "Montclair Egg Shop"}
> db.restaurants.find()
{ _id: ObjectId("98…"), name: "Ajanta"}
{ _id: ObjectId("66…"), name: "Montclair Egg Shop"}
> db.restaurants.find(name: /^A/)
{ _id: ObjectId("98…"), name: "Ajanta"}
Scaling MongoDB
   Shard 1                         Shard 2
             mongod                          mongod


     mongod      mongod              mongod      mongod



                                                         A shard
                                                         consists of a
Config Server                                            replica set =
                                                         generalization
  mongod              mongos            mongos           of master slave

  mongod
                                                       Collections
  mongod                                               spread over
                          client                      multiple shards
MongoDB use cases

• Use cases
   –   Real-time analytics
   –   Content management systems
   –   Single document partial update
   –   Caching
   –   High volume writes
• Who is using it?
   –   Shutterfly, Foursquare
   –   Bit.ly Intuit
   –   SourceForge, NY Times
   –   GILT Groupe, Evite,
   –   SugarCRM
CouchDB

• JSON documents
• Database is flat collection of documents
• Queries are implemented as views using Map/Reduce in
  Javascript
    – Queries typically defined up front
    – Views are indexed and incrementally updated
• MVCC based
    – Need to deal with reconciliation
•   Multi-Master (bi) replication
•   Scale out with replication
•   Mobile version - Android
•   Use as a Server-Side JavaScript application server
    – “CouchApps”
• Cloudant provides Java VM based views
CouchDB with curl

$ curl -X PUT http://localhost:5984/contacts
{"ok":true}

$ curl -X PUT http://localhost:5984/contacts/markpollack -d
'{"firstName":"Mark",
  "lastName":"Pollack",
  "email":"mpollack@vmware.com"}‘
{"ok":true,"id":"markpollack","rev":"1-7bb43f219a8d4306614dbd526ed577fe"}

$curl -X GET http://localhost:5984/contacts/markpollack
{"_id":"markpollack",
 "_rev":"1-7bb43f219a8d4306614dbd526ed577fe",
"firstName":"Mark",
"lastName":"Pollack",
"email":"mpollack@vmware.com"}
Futon
Futon
CouchDB use cases

• Use cases
   – Synchronization
      • master-master replication
      • Offline replication for mobile devices: “Ground Computing”
   – Web apps that consume JSON
      • Server side JavaScript using CouchApps
   – Content management systems
   – Caching
• Who is using it?
   – Mozilla, BBC, Palm, Apple,
   – IBM, Ubuntu, meebo, Engine Yard
   – Cloudant, UNICEF, Credit Suisse
Neo4j

• Graph Database
• DB is a collection of graph nodes, relationships
   – Nodes and relationships have properties
• Querying is done via a traversal API
• Indexes on node/relationship properties
• Written in Java, can be embedded
   – Also a REST API
• Transactional (ACID)
• Master-Slave replication in next release
Neo4j Data Model
Neo4j Use Cases

• Use Cases
   – Social networking
   – Bioinformatics
   – Fraud detection
• Who is using it?
   – Fanbox, Jayway
   – Large companies that wish to remain anonymous
Agenda

•   Why NoSQL?
•   Some NoSQL Databases
•   Spring NoSQL projects
•   Demos/Code/Deep dive
Spring Data Project Goals

• Bring classic Spring value propositions to NoSQL
   – Productivity
   – Programming model consistency
• Support for a wide range of NoSQL databases
• Many entry points to use
   –   Low level data access APIs
   –   Opinionated APIs (Think JdbcTemplate)
   –   Object Mapping (Java and GORM)
   –   Cross Store Persistence Programming model
   –   Productivity support in Roo and Grails
   –   Vertical Applications
   –   Guidance
Spring Data

• Working together with various projects/companies
   –   Redis / VMware
   –   Riak / Basho
   –   CouchDB / CouchOne
   –   Mongo / 10gen
   –   Neo4j / Neo Technologies
Spring Data Projects
 Data Commons                 Planned
   Polyglot persistence         Guidance Docs
 Datastore Key-Value               The big picture
   Redis                        Datastore Column
                                   Cassandra, HBase
 Datastore Document
                                Hadoop
   MongoDB, CouchDB
                                Datastore Key-Value
 Datastore Graph
                                   Riak
   Neo4j
                                Datastore Mapping
 Datastore Mapping
                                   Additional DB Map
   Redis, Gemfire
                                Blob storage
                                   Amazon, Atmos, Azure
 Reaching M1 release in Q4.     SQL - Generic DAOs
                                Grails/Roo support
Spring Datastore Key-Value (Redis)
 Interface based low-level Redis Client API
    Method names are Redis command names
    Pluggable implementations (Jedis, JRedis)
    Translation to Spring DAO exception hierarchy
    Connection Pooling
 RedisTemplate
    Descriptive method names, grouped into natural categories
       ServerOps, SetOps
       Convenience methods – ‘redis recipies’
    getAndConvert, convertAndSet for payload/value serialization
       Java, JSON, XML
       Protocol Buffers, Avro, to come…
 Redis backed java.util.Set, List, Map, Capped Collections
 RedisPlatformTransactionManager
 Redis Messaging
    RedisMessageTemplate, RedisMessageListenerContainer
Spring Redis Examples
RedisClientFactory clientFactory = new JedisClientFactory();
RedisClient client = clientFactory.createClient();
client.sadd("s1", "1"); client.sadd("s1", "2");
client.sadd("s1", "3");
client.sadd("s2", "2"); client.sadd("s2", "3");
Set<String> intersection = client.sinter("s1", "s2");
System.out.println(intersection); // [3,2]

RedisTemplate redisTemplate = new RedisTemplate(client)
Person p = new Person("Joe", "Trader", 33);
redisTemplate.convertAndSet("trader:1", p);
p = redisTemplate.getAndConvert("trader:1", Person.class);
intersection =
  redisTemplate.getSetOps().getIntersectionOfSets(“s1”,”s2”)

RedisSet s1 = new RedisSet(template, "s1");
s1.add("1"); s1.add("2"); s1.add("3");
RedisSet s2 = new RedisSet(template, "s2");
s2.add("2"); s2.add("3");
Set s3 = s1.intersection("s3", s1, s2); // [3,2]
Spring MongoDB

• MongoTemplate
  – ‘RowMapper’ like functionality for mapping Mongo query
    results.
  – Basic POJO mapping support for simple CRUD
     • Leverage Spring 3.0 TypeConverters and SpEL
  – Connection affinity callback
• Support common map-reduce operations from Mongo
  Cookbook
• Higher level integration
  – File Upload
  – Spring MVC activity monitoring and analysis
Spring MongoDB Example
Mongo mongo = new Mongo();
DB db = mongo.getDB("mvc");
MongoTemplate mongoTemplate = new MongoTemplate(db,
                                                "people");
Person p = new Person("Joe", "Trader", 33);
mongoTemplate.save(p);
List<People> people =
       mongoTemplate.queryForCollection(Person.class);
Spring CouchDB

 CouchTemplate
   RestTemplate based API for CouchDB
   Basic JSON/POJO mapping support
      CouchDbMappingJacksonHttpMessageConverter
   Android implementation
   .NET implementation planned
 Repository base classes
   Out of the box CRUD
   Embedded View Definitions
   Automatic view generation
 Plans to work with ektrop OS project
   http://code.google.com/p/ektorp/
Spring CouchDB Example
RestTemplate restTemplate = new RestTemplate();
restTemplate.setMessageConverters(couchDbConverter);

List<Restaurants> rests = (List<Restaurant>)
       restTemplate.getForObject(getDbURL() +
                "_design/demo/_view/all", Restaurant.class);


UserAccount userAccount = new UserAccount("u1");
restTemplate.put(getDbURL() + userAccount.getUserName(),
                 userAccount);

//Note, not handling response to get new revision string…
Ektrop CouchDB Example
Spring Neo4j

• Easy POJO based programming model
• Support for graph Entities, Relationships, and Properties
    – Properties leverage Spring 3.0 converters
• DAO finder support
    – findAll, count, findById, findByPropertyValue,
      findAllByTraversal

See Graph Database Persistence Using Neo4j with Spring and Roo
Tomorrow @ 10:15am in Lilac C
Spring Cross Store

• Use Case
   – An existing application want to use NOSQL database as a
     part of its existing SQL-based application
      • Pick a problem space for which NOSQL is well suited
• How to link together the two worlds?
   – Developer application code task
   – No longer a single domain entity
• Spring Cross store allows for a single entity to be
  persisted across multiple datastores
   – AspectJ manages NOSQL entities/fields in change sets
   – Change sets are flushed when JPA commits
   – Interception to rehydrate when performing JPA queries
Spring Cross Store Example
Spring Cross Store Example
Spring Cross Store Example
Spring Mapping

• GORM for Redis, Gemfire….
   – Built on pure Java
• Mapping metadata is not tied to GORM
   – Can use annotations, XML etc.
• Will be working on making this mapping framework
  available to Java developers
Spring Mapping
Spring Mapping
Agenda

•   Why NoSQL?
•   Some NoSQL Databases
•   Spring NoSQL projects
•   Demos/Code/Deep dive
Redis meets POJOs in Action
• Finding restaurants that are open at the delivery time (e.g.
  94619) and serve the delivery zip code (e.g. 6pm)
select r.*
from FTGO_RESTAURANT_ZIPCODE zip,
  FTGO_RESTAURANT_TIME_RANGE tr, FTGO_RESTAURANT r
where r.RESTAURANT_ID = zip.RESTAURANT_ID
  and tr.RESTAURANT_ID = r.RESTAURANT_ID;
  and zip.ZIPCODE = '94619'
  and tr.day_Of_week = 'Monday'
                                                     Joins and
  and tr.open_time <= 1800
                                                      complex
  and 1800<= tr.close_time                         where clause!


   No way Redis can handle this! Right?
Redis – denormalizing the data is the key
 Tr_id      Day of      Open_time Close_tim       Zip code
            Week                  e
 Ajanta10   Monday      1130        1430          94707
 Ajanta10   Monday      1130        1430          94619
 Ajanta11   Monday      1730        2130          94707
 Ajanta11   Monday      1730        2130          94619
 Egg30      Monday      0700        1430          94707
 …                                         Two simpler queries
(SELECT tr_id
 FROM time_range_zip_code
 where day_of_week = ? and zip_code = ? open_time < ?)
INTERSECTION
(SELECT tr_id
 FROM time_range_zip_code
 where day_of_week = ? and zip_code = ? and ? < close_time)
Redis – storing the opening hours using sorted
 sets
Sorted Set
                  Keys     Members = TimeRangeId with opening time as score

   open:94707:Monday       Ajanta10:1130       Ajanta11:1730

   open:94619:Monday       Egg30:0700      Ajanta10:1130        Ajanta11:1730




                                ZRANGEBYSCORE open:94619:Monday 0 1800



       {Egg30, Ajanta10, Ajanta11, …}      Finds all time ranges that open
                                           before 6pm
Redis – storing closing times with sorted sets

Ordered Set
                 Keys     Members = TimeRangeId with closing time as score

  close:94707:Monday      Ajanta10:1430      Ajanta11:2130

  close:94619:Monday      Egg30:1430      Ajanta10:1430      Ajanta11:2130




                        ZRANGEBYSCORE close:94619:Monday 1800 2400

                          Finds all time ranges that close
       {Ajanta11, …}      after 6pm
Redis – computing the intersection

             {Egg30, Ajanta10, Ajanta11, …}

                          Intersection

                     {Ajanta11, …}
                               =

                     {Ajanta11, …}



• This is super-cool!
• Illustrates the feasibility of using Redis as a queryable
  datastore to improve scalability
Cassandra – Food to Go
                            AvailableRestaurants Column Family
        Keys   Columns

               1430
    94619_
                                                       Super
    Monday_       Egg30: {…}                           column
     0700


               1430
    94619_
    Monday_      Ajanta10: {…}
     1130


               2130
    94619_
    Monday_      Ajanta11: {…}
     1730
Cassandra – Food to Go
   rangeSlice(startKey=94619_Monday_0000,
             endKey=94619_Monday_1800,
             startColumn=1800,
             endColumnName=2359)




        Keys   Columns


                 2130
    94619_
    Monday_         Ajanta11: {…}
     1730
Food to Go - MongoDB
{                               {
  "name" : "Ajanta",                serviceArea:"94619",
  "type" : "Indian",                openingHours: {
  "serviceArea" : [                   $elemMatch : {
     "94619",                               "dayOfWeek" : Monday,
     "94618"                                "open": {$lte: 1800},
  ],                                        "close": {$gte: 1800}
  "openingHours" : [                   }
     {                              }
        "dayOfWeek" : Monday,   }
        "open" : 1730,
        "close" : 2130
     }
  ],                            Straightforward to model a
  "_id" :                       restaurant aggregate as a MongoDB
ObjectId("4bddc2f49d1505567c    document and execute queries
6220a0")
}
MongoDB – MVC Analytics

• Monitoring page view and web application activity is
  critical to analyzing effectiveness of
   – Marketing campaigns
   – Site design / new features
• Standard solutions are problematic aside from scaling
   – Google Analytics
      • not real time, hard to setup
   – MySQL
      • Downtime for schema changes
• Solutions needs to handle high write volume, handle large
  amounts of data, allow for flexible data exploration.
Web Analytics - SQL
                                     Still difficult to
                                     scale writes

            Application    MySQL
              Server       Master



            Reporting
                          MySQL
             Server        MySQL
                          Slave
                            MySQL
                           Slave
                             Slave
Web Analytics - MongoDB


           Application
             Server

                          MongoD
            Reporting
             Server
Web Analytics - MongoDB


           Application             MongoD
             Server

                          MongoS
            Reporting
             Server                MongoD
Spring MVC based solution

• Custom Handler Adapter with a ActionInterceptor
• ActionInterceptor has full context of web action
   –   Controller class, method name, and parameters
   –   WebRequest
   –   ModelAndView
   –   Exception
• Similar to Ruby on Rails/ASP.NET MVC Action Filters
• Collect data in MongoDB
   – MvcEvent
   – Increment counters per controller and action method
• AnalyticsController to display results
   – Action method parameter information is critical for reporting


• Also a good fit for an Insight plugin!
Collected Sample Data - Counters


{ "_id" : ObjectId("4cba8e0a5a49000000004961"),
 "name" : "SignUpController",
 "count" : 8,
 "methods" : { "create" : 2, "createForm" : 2, "show" : 4 }
}
{ "_id" : ObjectId("4cba8ea85a49000000004962"),
 "name" : "RestaurantController",
 "count" : 9,
 "methods" : { "addFavoriteRestaurant" : 2, "list" : 5, "show" : 2 }
}
Collected Sample Data
{
  "_id" : ObjectId("4cba929e4b445bc779b65c5f"),
  "action" : "addFavoriteRestaurant",
  "controller" : "RestaurantController",
  "date" : "Sun Oct 17 2010 02:07:26 GMT-0400 (Eastern Daylight Time)",
  "parameters" : {
    "p1" : "5",
    "p2" : "1",
    "p3" : "{currentUserAccountId= 1, userAccount_birthdate_date_format=M/d/yy,
useraccount=Id: 1, Version: 1, UserName: mpollack, FirstName: Mark, LastName:
Pollack, BirthDate: 2010-10-27 00:00:00.0, Favorites: 1, itemId=5}" },
  "remoteUser" : "mpollack",
  "requestAddress" : "127.0.0.1",
  "requestUri" : "/myrestaurants-analytics/restaurants/5/1"
}
Restaurants Web Application
Favorite Restaurants Analysis
Controller Invocation
CouchDB – Mobile offline access and sync

•   Smartphones have huge data capacity
•   Can carry large data sets with you.
•   Mobile devices are not always connected
•   Data sync is a difficult problem to solve
    – CouchOne Mobile handles the dirty bits of sync
    – Available for Android
• Same programming model as on server side
• Spring RestTemplate based CouchDB API
    – On Server and Phone
• Can also be used a full-stack environment on mobile
    – Database, JavaScript, Web Server
CouchDB – Offline data access and sync


               Continuous
               bi-directional
               replication      Application
                                  Server
Sync Favorite Restaurants
Summary

• Many new options for persistence
   – A particular use case in your application might be a good fit
   – Polyglot persistence will be increasingly common
• Trade-offs with
   – Normalization
   – Transactions
   – Familiarity
• The Spring Data project
   – Make it easier to build Spring-powered applications that use
     these new technologies
• More info at http://www.springframework.org/spring-data
   – Welcome contributors, comments…
Q&A



Chris Richardson         Mark Pollack
crichardson@vmware.com   mpollack@vmware.com
@crichardson             @markpollack

Contenu connexe

Tendances

Indic threads pune12-polyglot & functional programming on jvm
Indic threads pune12-polyglot & functional programming on jvmIndic threads pune12-polyglot & functional programming on jvm
Indic threads pune12-polyglot & functional programming on jvmIndicThreads
 
A Brief, but Dense, Intro to Scala
A Brief, but Dense, Intro to ScalaA Brief, but Dense, Intro to Scala
A Brief, but Dense, Intro to ScalaDerek Chen-Becker
 
OWL 2 Quick Reference Card Letter Size
OWL 2 Quick Reference Card Letter SizeOWL 2 Quick Reference Card Letter Size
OWL 2 Quick Reference Card Letter SizeJie Bao
 
Owl 2 quick reference card a4 size
Owl 2 quick reference card a4 sizeOwl 2 quick reference card a4 size
Owl 2 quick reference card a4 sizeJie Bao
 
Avro - More Than Just a Serialization Framework - CHUG - 20120416
Avro - More Than Just a Serialization Framework - CHUG - 20120416Avro - More Than Just a Serialization Framework - CHUG - 20120416
Avro - More Than Just a Serialization Framework - CHUG - 20120416Chicago Hadoop Users Group
 
Scalable and Flexible Machine Learning With Scala @ LinkedIn
Scalable and Flexible Machine Learning With Scala @ LinkedInScalable and Flexible Machine Learning With Scala @ LinkedIn
Scalable and Flexible Machine Learning With Scala @ LinkedInVitaly Gordon
 
Scala - The Simple Parts, SFScala presentation
Scala - The Simple Parts, SFScala presentationScala - The Simple Parts, SFScala presentation
Scala - The Simple Parts, SFScala presentationMartin Odersky
 
What's a macro?: Learning by Examples / Scalaのマクロに実用例から触れてみよう!
What's a macro?: Learning by Examples / Scalaのマクロに実用例から触れてみよう!What's a macro?: Learning by Examples / Scalaのマクロに実用例から触れてみよう!
What's a macro?: Learning by Examples / Scalaのマクロに実用例から触れてみよう!scalaconfjp
 
Ruby — An introduction
Ruby — An introductionRuby — An introduction
Ruby — An introductionGonçalo Silva
 
Clojure - An Introduction for Lisp Programmers
Clojure - An Introduction for Lisp ProgrammersClojure - An Introduction for Lisp Programmers
Clojure - An Introduction for Lisp Programmerselliando dias
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to ScalaRahul Jain
 
Clojure - An Introduction for Java Programmers
Clojure - An Introduction for Java ProgrammersClojure - An Introduction for Java Programmers
Clojure - An Introduction for Java Programmerselliando dias
 
あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法x1 ichi
 
Java 7 Whats New(), Whats Next() from Oredev
Java 7 Whats New(), Whats Next() from OredevJava 7 Whats New(), Whats Next() from Oredev
Java 7 Whats New(), Whats Next() from OredevMattias Karlsson
 
Introduction to Clojure
Introduction to ClojureIntroduction to Clojure
Introduction to ClojureRenzo Borgatti
 

Tendances (19)

Indic threads pune12-polyglot & functional programming on jvm
Indic threads pune12-polyglot & functional programming on jvmIndic threads pune12-polyglot & functional programming on jvm
Indic threads pune12-polyglot & functional programming on jvm
 
A Brief, but Dense, Intro to Scala
A Brief, but Dense, Intro to ScalaA Brief, but Dense, Intro to Scala
A Brief, but Dense, Intro to Scala
 
OWL 2 Quick Reference Card Letter Size
OWL 2 Quick Reference Card Letter SizeOWL 2 Quick Reference Card Letter Size
OWL 2 Quick Reference Card Letter Size
 
Pfm technical-inside
Pfm technical-insidePfm technical-inside
Pfm technical-inside
 
Owl 2 quick reference card a4 size
Owl 2 quick reference card a4 sizeOwl 2 quick reference card a4 size
Owl 2 quick reference card a4 size
 
Avro - More Than Just a Serialization Framework - CHUG - 20120416
Avro - More Than Just a Serialization Framework - CHUG - 20120416Avro - More Than Just a Serialization Framework - CHUG - 20120416
Avro - More Than Just a Serialization Framework - CHUG - 20120416
 
Scalable and Flexible Machine Learning With Scala @ LinkedIn
Scalable and Flexible Machine Learning With Scala @ LinkedInScalable and Flexible Machine Learning With Scala @ LinkedIn
Scalable and Flexible Machine Learning With Scala @ LinkedIn
 
Groovy Programming Language
Groovy Programming LanguageGroovy Programming Language
Groovy Programming Language
 
Scala - The Simple Parts, SFScala presentation
Scala - The Simple Parts, SFScala presentationScala - The Simple Parts, SFScala presentation
Scala - The Simple Parts, SFScala presentation
 
What's a macro?: Learning by Examples / Scalaのマクロに実用例から触れてみよう!
What's a macro?: Learning by Examples / Scalaのマクロに実用例から触れてみよう!What's a macro?: Learning by Examples / Scalaのマクロに実用例から触れてみよう!
What's a macro?: Learning by Examples / Scalaのマクロに実用例から触れてみよう!
 
Ruby — An introduction
Ruby — An introductionRuby — An introduction
Ruby — An introduction
 
J
JJ
J
 
Clojure - An Introduction for Lisp Programmers
Clojure - An Introduction for Lisp ProgrammersClojure - An Introduction for Lisp Programmers
Clojure - An Introduction for Lisp Programmers
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to Scala
 
Clojure - An Introduction for Java Programmers
Clojure - An Introduction for Java ProgrammersClojure - An Introduction for Java Programmers
Clojure - An Introduction for Java Programmers
 
Rjb
RjbRjb
Rjb
 
あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法
 
Java 7 Whats New(), Whats Next() from Oredev
Java 7 Whats New(), Whats Next() from OredevJava 7 Whats New(), Whats Next() from Oredev
Java 7 Whats New(), Whats Next() from Oredev
 
Introduction to Clojure
Introduction to ClojureIntroduction to Clojure
Introduction to Clojure
 

En vedette

Toshiba satellite l40 17 s batería at www-baterias-portatil-es
Toshiba satellite l40 17 s batería at www-baterias-portatil-esToshiba satellite l40 17 s batería at www-baterias-portatil-es
Toshiba satellite l40 17 s batería at www-baterias-portatil-esbatteryes
 
CURRICULUM VITA1.bosco
CURRICULUM VITA1.boscoCURRICULUM VITA1.bosco
CURRICULUM VITA1.boscobosco sachuma
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitterRoger Xia
 
Java programing considering performance
Java programing considering performanceJava programing considering performance
Java programing considering performanceRoger Xia
 
Converged open platform for enterprise
Converged open platform for enterpriseConverged open platform for enterprise
Converged open platform for enterpriseRoger Xia
 
Virtuoso - music retrieval and recommendation
Virtuoso -  music retrieval and recommendationVirtuoso -  music retrieval and recommendation
Virtuoso - music retrieval and recommendationeanes pereira
 
Student Film Permission
Student Film PermissionStudent Film Permission
Student Film PermissionTheMightyGabes
 
EXASolution für das Gesundheitswesen
EXASolution für das GesundheitswesenEXASolution für das Gesundheitswesen
EXASolution für das GesundheitswesenEXASOL AG
 

En vedette (16)

Sustancias psicoactivas
Sustancias psicoactivasSustancias psicoactivas
Sustancias psicoactivas
 
Copia de cuento juan luis
Copia de cuento juan luisCopia de cuento juan luis
Copia de cuento juan luis
 
Toshiba satellite l40 17 s batería at www-baterias-portatil-es
Toshiba satellite l40 17 s batería at www-baterias-portatil-esToshiba satellite l40 17 s batería at www-baterias-portatil-es
Toshiba satellite l40 17 s batería at www-baterias-portatil-es
 
CURRICULUM VITA1.bosco
CURRICULUM VITA1.boscoCURRICULUM VITA1.bosco
CURRICULUM VITA1.bosco
 
Lyric analysis
Lyric analysisLyric analysis
Lyric analysis
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitter
 
Java programing considering performance
Java programing considering performanceJava programing considering performance
Java programing considering performance
 
Converged open platform for enterprise
Converged open platform for enterpriseConverged open platform for enterprise
Converged open platform for enterprise
 
Navidad 2012
Navidad 2012Navidad 2012
Navidad 2012
 
Shoot schedule re made
Shoot schedule re madeShoot schedule re made
Shoot schedule re made
 
Acculturer
AcculturerAcculturer
Acculturer
 
Shooting schedule
Shooting scheduleShooting schedule
Shooting schedule
 
Music video
Music videoMusic video
Music video
 
Virtuoso - music retrieval and recommendation
Virtuoso -  music retrieval and recommendationVirtuoso -  music retrieval and recommendation
Virtuoso - music retrieval and recommendation
 
Student Film Permission
Student Film PermissionStudent Film Permission
Student Film Permission
 
EXASolution für das Gesundheitswesen
EXASolution für das GesundheitswesenEXASolution für das Gesundheitswesen
EXASolution für das Gesundheitswesen
 

Similaire à Spring one2gx2010 spring-nonrelational_data

Using Spring with NoSQL databases (SpringOne China 2012)
Using Spring with NoSQL databases (SpringOne China 2012)Using Spring with NoSQL databases (SpringOne China 2012)
Using Spring with NoSQL databases (SpringOne China 2012)Chris Richardson
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQLYan Cui
 
No sql solutions - 공개용
No sql solutions - 공개용No sql solutions - 공개용
No sql solutions - 공개용Byeongweon Moon
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesshnkr_rmchndrn
 
On Rails with Apache Cassandra
On Rails with Apache CassandraOn Rails with Apache Cassandra
On Rails with Apache CassandraStu Hood
 
High-Performance Storage Services with HailDB and Java
High-Performance Storage Services with HailDB and JavaHigh-Performance Storage Services with HailDB and Java
High-Performance Storage Services with HailDB and Javasunnygleason
 
Building DSLs with Scala
Building DSLs with ScalaBuilding DSLs with Scala
Building DSLs with ScalaMohit Jaggi
 
NoSQL Intro with cassandra
NoSQL Intro with cassandraNoSQL Intro with cassandra
NoSQL Intro with cassandraBrian Enochson
 
Nosql databases for the .net developer
Nosql databases for the .net developerNosql databases for the .net developer
Nosql databases for the .net developerJesus Rodriguez
 
MongoDB London PHP
MongoDB London PHPMongoDB London PHP
MongoDB London PHPMike Dirolf
 

Similaire à Spring one2gx2010 spring-nonrelational_data (20)

Using Spring with NoSQL databases (SpringOne China 2012)
Using Spring with NoSQL databases (SpringOne China 2012)Using Spring with NoSQL databases (SpringOne China 2012)
Using Spring with NoSQL databases (SpringOne China 2012)
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQL
 
No sql solutions - 공개용
No sql solutions - 공개용No sql solutions - 공개용
No sql solutions - 공개용
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skies
 
NoSQL
NoSQLNoSQL
NoSQL
 
Drop acid
Drop acidDrop acid
Drop acid
 
MongoDB is the MashupDB
MongoDB is the MashupDBMongoDB is the MashupDB
MongoDB is the MashupDB
 
On Rails with Apache Cassandra
On Rails with Apache CassandraOn Rails with Apache Cassandra
On Rails with Apache Cassandra
 
High-Performance Storage Services with HailDB and Java
High-Performance Storage Services with HailDB and JavaHigh-Performance Storage Services with HailDB and Java
High-Performance Storage Services with HailDB and Java
 
Building DSLs with Scala
Building DSLs with ScalaBuilding DSLs with Scala
Building DSLs with Scala
 
KeyValue Stores
KeyValue StoresKeyValue Stores
KeyValue Stores
 
NoSQL Intro with cassandra
NoSQL Intro with cassandraNoSQL Intro with cassandra
NoSQL Intro with cassandra
 
Wmware NoSQL
Wmware NoSQLWmware NoSQL
Wmware NoSQL
 
Nosql databases for the .net developer
Nosql databases for the .net developerNosql databases for the .net developer
Nosql databases for the .net developer
 
MongoDB London PHP
MongoDB London PHPMongoDB London PHP
MongoDB London PHP
 
No sql Database
No sql DatabaseNo sql Database
No sql Database
 
Mongodb my
Mongodb myMongodb my
Mongodb my
 
MongoDB
MongoDBMongoDB
MongoDB
 
MongoDB
MongoDBMongoDB
MongoDB
 
Einführung in MongoDB
Einführung in MongoDBEinführung in MongoDB
Einführung in MongoDB
 

Plus de Roger Xia

机器学习推动金融数据智能
机器学习推动金融数据智能机器学习推动金融数据智能
机器学习推动金融数据智能Roger Xia
 
Code reviews
Code reviewsCode reviews
Code reviewsRoger Xia
 
Python introduction
Python introductionPython introduction
Python introductionRoger Xia
 
Learning notes ruby
Learning notes rubyLearning notes ruby
Learning notes rubyRoger Xia
 
Code reviews
Code reviewsCode reviews
Code reviewsRoger Xia
 
E commerce search strategies
E commerce search strategiesE commerce search strategies
E commerce search strategiesRoger Xia
 
Indefero source code_managment
Indefero source code_managmentIndefero source code_managment
Indefero source code_managmentRoger Xia
 
Web Services Atomic Transactio
 Web Services Atomic Transactio Web Services Atomic Transactio
Web Services Atomic TransactioRoger Xia
 
Web service through cxf
Web service through cxfWeb service through cxf
Web service through cxfRoger Xia
 
Q con london2011-matthewwall-whyichosemongodbforguardiancouk
Q con london2011-matthewwall-whyichosemongodbforguardiancoukQ con london2011-matthewwall-whyichosemongodbforguardiancouk
Q con london2011-matthewwall-whyichosemongodbforguardiancoukRoger Xia
 
Consistency-New-Generation-Databases
Consistency-New-Generation-DatabasesConsistency-New-Generation-Databases
Consistency-New-Generation-DatabasesRoger Xia
 
Java explore
Java exploreJava explore
Java exploreRoger Xia
 
Mongo db实战
Mongo db实战Mongo db实战
Mongo db实战Roger Xia
 
Ca siteminder
Ca siteminderCa siteminder
Ca siteminderRoger Xia
 
Eclipse plug in mylyn & tasktop
Eclipse plug in mylyn & tasktopEclipse plug in mylyn & tasktop
Eclipse plug in mylyn & tasktopRoger Xia
 
新浪微博架构猜想
新浪微博架构猜想新浪微博架构猜想
新浪微博架构猜想Roger Xia
 
构建高效能的Web网站 精选版-by-infoq
构建高效能的Web网站 精选版-by-infoq构建高效能的Web网站 精选版-by-infoq
构建高效能的Web网站 精选版-by-infoqRoger Xia
 

Plus de Roger Xia (20)

机器学习推动金融数据智能
机器学习推动金融数据智能机器学习推动金融数据智能
机器学习推动金融数据智能
 
Code reviews
Code reviewsCode reviews
Code reviews
 
Python introduction
Python introductionPython introduction
Python introduction
 
Learning notes ruby
Learning notes rubyLearning notes ruby
Learning notes ruby
 
Code reviews
Code reviewsCode reviews
Code reviews
 
E commerce search strategies
E commerce search strategiesE commerce search strategies
E commerce search strategies
 
Saml
SamlSaml
Saml
 
JavaEE6
JavaEE6JavaEE6
JavaEE6
 
Indefero source code_managment
Indefero source code_managmentIndefero source code_managment
Indefero source code_managment
 
Web Services Atomic Transactio
 Web Services Atomic Transactio Web Services Atomic Transactio
Web Services Atomic Transactio
 
Web service through cxf
Web service through cxfWeb service through cxf
Web service through cxf
 
Q con london2011-matthewwall-whyichosemongodbforguardiancouk
Q con london2011-matthewwall-whyichosemongodbforguardiancoukQ con london2011-matthewwall-whyichosemongodbforguardiancouk
Q con london2011-matthewwall-whyichosemongodbforguardiancouk
 
Consistency-New-Generation-Databases
Consistency-New-Generation-DatabasesConsistency-New-Generation-Databases
Consistency-New-Generation-Databases
 
Java explore
Java exploreJava explore
Java explore
 
Mongo db实战
Mongo db实战Mongo db实战
Mongo db实战
 
Ca siteminder
Ca siteminderCa siteminder
Ca siteminder
 
Eclipse plug in mylyn & tasktop
Eclipse plug in mylyn & tasktopEclipse plug in mylyn & tasktop
Eclipse plug in mylyn & tasktop
 
新浪微博架构猜想
新浪微博架构猜想新浪微博架构猜想
新浪微博架构猜想
 
构建高效能的Web网站 精选版-by-infoq
构建高效能的Web网站 精选版-by-infoq构建高效能的Web网站 精选版-by-infoq
构建高效能的Web网站 精选版-by-infoq
 
Jenkins
JenkinsJenkins
Jenkins
 

Dernier

Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embeddingZilliz
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 

Dernier (20)

Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embedding
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 

Spring one2gx2010 spring-nonrelational_data

  • 1. Chicago, October 19 - 22, 2010 Using Spring with NoSQL databases Mark Pollack Chris Richardson
  • 2. Goal of this talk • The benefits and drawbacks of NoSQL databases • How the Spring Data project simplifies the development of NoSQL applications
  • 3. About Chris •Grew up in England and live in Oakland, CA •Over 25+ years of software development experience including 14 years of Java •Speaker at JavaOne, SpringOne, NFJS, JavaPolis, Spring Experience, etc. •Organize the Oakland JUG and the Groovy Grails meetup http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
  • 4. About Mark • Spring committer since 2003 • Founder of Spring.NET • Work on some other new open source projects – Spring Gemfire – Spring AMQP • Used to work on large scale data collection/analysis in High Energy Physics (~10yrs ago)
  • 5. Agenda • Why NoSQL? • Some NoSQL Databases • Spring NoSQL projects • Demos/Code/Deep dive
  • 7. Relational databases are great • Well understood by developers • Well supported by frameworks and tools, e.g. Spring JDBC, Hibernate, JPA • Well understood by operations – Configuration – Care and feeding – Backups – Tuning – Failure and recovery – Performance characteristics • But….
  • 8. The trouble with relational database • Object/relational impedance mismatch – Complicated to map rich domain model to relational schema – Relational schema is rigid • Extremely difficult/impossible to scale writes: – Vertical scaling is limited/expensive – Horizontal scaling is limited or requires $$ • Performance can be suboptimal
  • 9. Why NoSQL? Internet scale is the primary driver in the use of non-relational DBs Large Data Size – TBs to PBs High Transaction rates Need for High Availability Pictures courtesy of Gilt Group Relational DBs Harder to scale to larger size Transactions and joins reduce access rates
  • 10. Different priorities • Relational Databases with ACID semantics – Atomic, Consistent, Isolated, Durable • Internet Systems with BASE semantics – Basically Available, Scalable, Eventually Consistent • NoSQL/BASE values remove standard DB behaviors – No Joins – Very limited "transactions" – Different Durability/consistency guarantees • NoSQL datastores emerge as “point solutions” – Amazon/Google papers – Facebook, LinkedIn …
  • 11. The NoSQL Landscape There are a variety of non-relational datastores Many are Open Source Many with companies behind them Rapidly changing feature sets Scalaris Neo4J db4o Objectivity Amazon Dynamo Terrastore Gemfire Sones HBase CouchDB InfoGrid RavenDB Hadoop Scalaris Tokyo Cabinet MongoDB Hypertable Cassandra Azure Table Coherence Mnesia Voldemort Gigaspaces Riak Amazon SimpleDB Redis BerkeleyDB MemcacheDB Tamino InfiniteGraph Chordless Expect consolidation over time With so many choices, picking which one to use can be difficult
  • 12. Stops on a tour through NoSQL-land What is the data model? How to I access my data? What are the primary use cases? What notable companies that are using it?
  • 13. Data Model Key Value K1 V1 Key -> value K2 V2 Redis, Riak, Voldemort… K3 V2 “Amazon Dynamo inspired” Column Key -> ‘column families’ HBase, Cassandra “Google Bigtable inspired”
  • 14. Data Model Document Collections containing semi- { id: ‘4b2b9f67a1f631733d917a7b"),’ author: ‘joe’, structured data tags : [‘example’, ‘db’], comments : [ { author: 'jim', comment: 'OK' }, { author: ‘ida', comment: ‘Bad' } key-values , JSON, XML, ] CouchDB, MongoDB { id: ‘4b2b9f67a1f631733d917a7c"), author: ‘ida’, ... { id: ‘4b2b9f67a1f631733d917a7d"), author: ‘jim’, ... Graph Graph: nodes and edges Edges can be directional Each can contain key-value attributes Neo4j, Sones, InfiniteGraph
  • 15. How to I access my data?
  • 16. How to I access my data? Query Mechanism Key lookup Map-Reduce Query by example Query language ‘out of box’ query power usually linked to data model Increasing power KV -> Column -> Document/Graph Counter examples Riak is key-value and supports Map-Reduce Spring Data Mapper… .
  • 17. Reality Check - Relational DBs are not going away NoSQL usage small by comparison… But growing…
  • 18. Future = multi-paradigm data storage for enterprise applications IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg
  • 19. Agenda • Why NoSQL? • Some NoSQL Databases • Spring NoSQL projects • Demos/Code/Deep dive
  • 20. Redis • Advanced key-value store – Think memcached on steroids (the good kind) – Values can be binary strings, Lists, Sets, Ordered K1 V1 Sets, Hash maps, .. – Operations for each data type, e.g. appending K2 V2 to a list, adding to a set, retrieving a slice of a K3 V2 list, … • Very fast: ~100K operations/second on entry- level hardware • Persistent – Periodic snapshots of memory OR append commands to log file – Limits are size of keys retained in memory. • Has “transactions” • Libraries – Many languages, all separate open source projects
  • 21. Redis CLI redis> sadd myset a (integer) 1 redis> sadd myset b (integer) 1 redis> smembers myset 1. "a" 2. "b" redis> srem myset a (integer) 1 redis> smembers myset 1. "b"
  • 22. Scaling Redis • Master/slave replication – Tree of Redis servers – Non-persistent master can replicate to a persistent slave – Use slaves for read-only queries • Sharding – Client-side only – consistent hashing based on key – Server-side sharding – coming one day • Run multiple servers per physical host – Leverage multiple CPUs – 32 bit more efficient than 64 bit • Optional "virtual memory" – Ideally data should fit in RAM – Values (not keys) written to disc
  • 23. Redis use cases • Use in conjunction with another database as the SOR • Drop-in replacement for Memcached – Session state – Cache of data retrieved from SOR • Denormalized datastore for high-performance queries • Hit counts using INCR command • Randomly selecting an item – SRANDMEMBER • Queuing – Lists with LPOP, RPUSH, …. • High score tables – Sorted sets • Notable users: github, guardian.co.uk, ….
  • 24. Cassandra • Originally developed by Facebook for inbox search – Now an Apache open-source project • Extremely scalable • Data is replicated and sharded • The data model will hurt your brain • Column-oriented database • 4 or 5-dimensional hash map
  • 25. Cassandra data model – insert/update My Column family (within a key space) Keys Columns a colA: value1 colB: value2 colC: value3 This can be data! b colA: value colD: value colE: value Insert(key=a, columName=colZ, value=foo) Keys Columns a colA: value1 colB: value2 colC: value3 colZ: foo b colA: value colD: value colE: value
  • 26. Cassandra query – range slice Keys Columns a colA: value1 colB: value2 colC: value3 colZ: foo b colA: value colD: value colE: value rangeSlice(startKey=a, endKey=b, startColumn=colA, endColumnName=colB) Keys Columns a colA: value1 colB: value2 b colA: value
  • 27. Cassandra CLI $ bin/cassandra-cli cassandra> connect localhost/9160 Connected to "Test Cluster" on localhost/9160 cassandra> get Keyspace1.Standard1['foo'] Returned 0 results cassandra> set Keyspace1.Standard1['foo']['a'] = '2' Value inserted cassandra> get Keyspace1.Standard1['foo'] => (Column=61, value=1, timestamp...) cassandra> set Keyspace1.Standard1['foo']['bar'] = 'abc' Value inserted cassandra> get Keyspace1.Standard1['foo'] => (Column=626172, value=abc, timestamp...) => (Column=61, value=1, timestamp...) cassandra> count Keyspace1.Standard1['foo'] 2 columns cassandra>
  • 28. Scaling Cassandra through sharding/replication Keys = [D, A] •Client connects to any node •Dynamically add/remove nodes Node 1 •Configurable # replicas Token = A •Reads/Writes specify how many nodes Replicates to Replicates to Keys = [A, B] Node 4 Node 2 Token = D Token = B Keys = [C, D] Replicates to Replicates to Node 3 Token = C Keys = [B, C]
  • 29. Cassandra use cases • Use cases • Big data • Persistent cache • (Write intensive) Logging • Who is using it – Digg, Facebook, Twitter, Reddit, Rackspace – Cloudkick, Cisco, SimpleGeo, Ooyala, OpenX – “The largest production cluster has over 100 TB of data in over 150 machines.“ – Casssandra web site
  • 30. MongoDB • JSON-style documents – Lists, Maps, Primitive values • Documents organized into collections (~table) • Full or partial document updates – Transactional update in place on one document – Atomic Modifiers • GridFS for efficiently storing large files • Index support – secondary and compound • Rich query language for dynamic queries • Map/Reduce • Replication and Auto Sharding
  • 31. MongoDB Data Model { "name" : "Ajanta", • BSON data types "type" : "Indian", "serviceArea" : [ • Documents types "94619", – Data "94618" ], – Query "openingHours" : [ – Modifier for updates { • Query and Modify "dayOfWeek" : Monday, "open" : 1730, operators "close" : 2130 – $tl, $gt, $ne, $in, $or, } $elemMatch, $orderBy, ], $group, "_id" : ObjectId("4bddc2f49d1505567c – $where + javascript 6220a0") – $inc, $set, $push, $pull }
  • 32. MongoDB query by example • Find a restaurant that serves the 94619 zip code and is open at 6pm on a Monday { serviceArea:"94619", openingHours: { $elemMatch : { "dayOfWeek" : Monday, "open": {$lte: 1800}, "close": {$gte: 1800} } } } DBCursor cursor = collection.find(qbeObject); while (cursor.hasNext()) { DBObject o = cursor.next(); … }
  • 33. MongoDB CLI $ bin/mongo > use mydb > r1 = {name: 'Ajanta'} {name: 'Ajanta'} > r2 = {name: 'Montclair Egg Shop'} {name: 'Montclair Egg Shop'} > db.restaurants.save(r1) > r1 { _id: ObjectId("98…"), name: "Ajanta"} > db.restaurants.save(r2) > r2 { _id: ObjectId("66…"), name: "Montclair Egg Shop"} > db.restaurants.find() { _id: ObjectId("98…"), name: "Ajanta"} { _id: ObjectId("66…"), name: "Montclair Egg Shop"} > db.restaurants.find(name: /^A/) { _id: ObjectId("98…"), name: "Ajanta"}
  • 34. Scaling MongoDB Shard 1 Shard 2 mongod mongod mongod mongod mongod mongod A shard consists of a Config Server replica set = generalization mongod mongos mongos of master slave mongod Collections mongod spread over client multiple shards
  • 35. MongoDB use cases • Use cases – Real-time analytics – Content management systems – Single document partial update – Caching – High volume writes • Who is using it? – Shutterfly, Foursquare – Bit.ly Intuit – SourceForge, NY Times – GILT Groupe, Evite, – SugarCRM
  • 36. CouchDB • JSON documents • Database is flat collection of documents • Queries are implemented as views using Map/Reduce in Javascript – Queries typically defined up front – Views are indexed and incrementally updated • MVCC based – Need to deal with reconciliation • Multi-Master (bi) replication • Scale out with replication • Mobile version - Android • Use as a Server-Side JavaScript application server – “CouchApps” • Cloudant provides Java VM based views
  • 37. CouchDB with curl $ curl -X PUT http://localhost:5984/contacts {"ok":true} $ curl -X PUT http://localhost:5984/contacts/markpollack -d '{"firstName":"Mark", "lastName":"Pollack", "email":"mpollack@vmware.com"}‘ {"ok":true,"id":"markpollack","rev":"1-7bb43f219a8d4306614dbd526ed577fe"} $curl -X GET http://localhost:5984/contacts/markpollack {"_id":"markpollack", "_rev":"1-7bb43f219a8d4306614dbd526ed577fe", "firstName":"Mark", "lastName":"Pollack", "email":"mpollack@vmware.com"}
  • 38. Futon
  • 39. Futon
  • 40. CouchDB use cases • Use cases – Synchronization • master-master replication • Offline replication for mobile devices: “Ground Computing” – Web apps that consume JSON • Server side JavaScript using CouchApps – Content management systems – Caching • Who is using it? – Mozilla, BBC, Palm, Apple, – IBM, Ubuntu, meebo, Engine Yard – Cloudant, UNICEF, Credit Suisse
  • 41. Neo4j • Graph Database • DB is a collection of graph nodes, relationships – Nodes and relationships have properties • Querying is done via a traversal API • Indexes on node/relationship properties • Written in Java, can be embedded – Also a REST API • Transactional (ACID) • Master-Slave replication in next release
  • 43. Neo4j Use Cases • Use Cases – Social networking – Bioinformatics – Fraud detection • Who is using it? – Fanbox, Jayway – Large companies that wish to remain anonymous
  • 44. Agenda • Why NoSQL? • Some NoSQL Databases • Spring NoSQL projects • Demos/Code/Deep dive
  • 45. Spring Data Project Goals • Bring classic Spring value propositions to NoSQL – Productivity – Programming model consistency • Support for a wide range of NoSQL databases • Many entry points to use – Low level data access APIs – Opinionated APIs (Think JdbcTemplate) – Object Mapping (Java and GORM) – Cross Store Persistence Programming model – Productivity support in Roo and Grails – Vertical Applications – Guidance
  • 46. Spring Data • Working together with various projects/companies – Redis / VMware – Riak / Basho – CouchDB / CouchOne – Mongo / 10gen – Neo4j / Neo Technologies
  • 47. Spring Data Projects Data Commons Planned Polyglot persistence Guidance Docs Datastore Key-Value The big picture Redis Datastore Column Cassandra, HBase Datastore Document Hadoop MongoDB, CouchDB Datastore Key-Value Datastore Graph Riak Neo4j Datastore Mapping Datastore Mapping Additional DB Map Redis, Gemfire Blob storage Amazon, Atmos, Azure Reaching M1 release in Q4. SQL - Generic DAOs Grails/Roo support
  • 48. Spring Datastore Key-Value (Redis) Interface based low-level Redis Client API Method names are Redis command names Pluggable implementations (Jedis, JRedis) Translation to Spring DAO exception hierarchy Connection Pooling RedisTemplate Descriptive method names, grouped into natural categories ServerOps, SetOps Convenience methods – ‘redis recipies’ getAndConvert, convertAndSet for payload/value serialization Java, JSON, XML Protocol Buffers, Avro, to come… Redis backed java.util.Set, List, Map, Capped Collections RedisPlatformTransactionManager Redis Messaging RedisMessageTemplate, RedisMessageListenerContainer
  • 49. Spring Redis Examples RedisClientFactory clientFactory = new JedisClientFactory(); RedisClient client = clientFactory.createClient(); client.sadd("s1", "1"); client.sadd("s1", "2"); client.sadd("s1", "3"); client.sadd("s2", "2"); client.sadd("s2", "3"); Set<String> intersection = client.sinter("s1", "s2"); System.out.println(intersection); // [3,2] RedisTemplate redisTemplate = new RedisTemplate(client) Person p = new Person("Joe", "Trader", 33); redisTemplate.convertAndSet("trader:1", p); p = redisTemplate.getAndConvert("trader:1", Person.class); intersection = redisTemplate.getSetOps().getIntersectionOfSets(“s1”,”s2”) RedisSet s1 = new RedisSet(template, "s1"); s1.add("1"); s1.add("2"); s1.add("3"); RedisSet s2 = new RedisSet(template, "s2"); s2.add("2"); s2.add("3"); Set s3 = s1.intersection("s3", s1, s2); // [3,2]
  • 50. Spring MongoDB • MongoTemplate – ‘RowMapper’ like functionality for mapping Mongo query results. – Basic POJO mapping support for simple CRUD • Leverage Spring 3.0 TypeConverters and SpEL – Connection affinity callback • Support common map-reduce operations from Mongo Cookbook • Higher level integration – File Upload – Spring MVC activity monitoring and analysis
  • 51. Spring MongoDB Example Mongo mongo = new Mongo(); DB db = mongo.getDB("mvc"); MongoTemplate mongoTemplate = new MongoTemplate(db, "people"); Person p = new Person("Joe", "Trader", 33); mongoTemplate.save(p); List<People> people = mongoTemplate.queryForCollection(Person.class);
  • 52. Spring CouchDB CouchTemplate RestTemplate based API for CouchDB Basic JSON/POJO mapping support CouchDbMappingJacksonHttpMessageConverter Android implementation .NET implementation planned Repository base classes Out of the box CRUD Embedded View Definitions Automatic view generation Plans to work with ektrop OS project http://code.google.com/p/ektorp/
  • 53. Spring CouchDB Example RestTemplate restTemplate = new RestTemplate(); restTemplate.setMessageConverters(couchDbConverter); List<Restaurants> rests = (List<Restaurant>) restTemplate.getForObject(getDbURL() + "_design/demo/_view/all", Restaurant.class); UserAccount userAccount = new UserAccount("u1"); restTemplate.put(getDbURL() + userAccount.getUserName(), userAccount); //Note, not handling response to get new revision string…
  • 55. Spring Neo4j • Easy POJO based programming model • Support for graph Entities, Relationships, and Properties – Properties leverage Spring 3.0 converters • DAO finder support – findAll, count, findById, findByPropertyValue, findAllByTraversal See Graph Database Persistence Using Neo4j with Spring and Roo Tomorrow @ 10:15am in Lilac C
  • 56. Spring Cross Store • Use Case – An existing application want to use NOSQL database as a part of its existing SQL-based application • Pick a problem space for which NOSQL is well suited • How to link together the two worlds? – Developer application code task – No longer a single domain entity • Spring Cross store allows for a single entity to be persisted across multiple datastores – AspectJ manages NOSQL entities/fields in change sets – Change sets are flushed when JPA commits – Interception to rehydrate when performing JPA queries
  • 60. Spring Mapping • GORM for Redis, Gemfire…. – Built on pure Java • Mapping metadata is not tied to GORM – Can use annotations, XML etc. • Will be working on making this mapping framework available to Java developers
  • 63. Agenda • Why NoSQL? • Some NoSQL Databases • Spring NoSQL projects • Demos/Code/Deep dive
  • 64. Redis meets POJOs in Action • Finding restaurants that are open at the delivery time (e.g. 94619) and serve the delivery zip code (e.g. 6pm) select r.* from FTGO_RESTAURANT_ZIPCODE zip, FTGO_RESTAURANT_TIME_RANGE tr, FTGO_RESTAURANT r where r.RESTAURANT_ID = zip.RESTAURANT_ID and tr.RESTAURANT_ID = r.RESTAURANT_ID; and zip.ZIPCODE = '94619' and tr.day_Of_week = 'Monday' Joins and and tr.open_time <= 1800 complex and 1800<= tr.close_time where clause! No way Redis can handle this! Right?
  • 65. Redis – denormalizing the data is the key Tr_id Day of Open_time Close_tim Zip code Week e Ajanta10 Monday 1130 1430 94707 Ajanta10 Monday 1130 1430 94619 Ajanta11 Monday 1730 2130 94707 Ajanta11 Monday 1730 2130 94619 Egg30 Monday 0700 1430 94707 … Two simpler queries (SELECT tr_id FROM time_range_zip_code where day_of_week = ? and zip_code = ? open_time < ?) INTERSECTION (SELECT tr_id FROM time_range_zip_code where day_of_week = ? and zip_code = ? and ? < close_time)
  • 66. Redis – storing the opening hours using sorted sets Sorted Set Keys Members = TimeRangeId with opening time as score open:94707:Monday Ajanta10:1130 Ajanta11:1730 open:94619:Monday Egg30:0700 Ajanta10:1130 Ajanta11:1730 ZRANGEBYSCORE open:94619:Monday 0 1800 {Egg30, Ajanta10, Ajanta11, …} Finds all time ranges that open before 6pm
  • 67. Redis – storing closing times with sorted sets Ordered Set Keys Members = TimeRangeId with closing time as score close:94707:Monday Ajanta10:1430 Ajanta11:2130 close:94619:Monday Egg30:1430 Ajanta10:1430 Ajanta11:2130 ZRANGEBYSCORE close:94619:Monday 1800 2400 Finds all time ranges that close {Ajanta11, …} after 6pm
  • 68. Redis – computing the intersection {Egg30, Ajanta10, Ajanta11, …} Intersection {Ajanta11, …} = {Ajanta11, …} • This is super-cool! • Illustrates the feasibility of using Redis as a queryable datastore to improve scalability
  • 69. Cassandra – Food to Go AvailableRestaurants Column Family Keys Columns 1430 94619_ Super Monday_ Egg30: {…} column 0700 1430 94619_ Monday_ Ajanta10: {…} 1130 2130 94619_ Monday_ Ajanta11: {…} 1730
  • 70. Cassandra – Food to Go rangeSlice(startKey=94619_Monday_0000, endKey=94619_Monday_1800, startColumn=1800, endColumnName=2359) Keys Columns 2130 94619_ Monday_ Ajanta11: {…} 1730
  • 71. Food to Go - MongoDB { { "name" : "Ajanta", serviceArea:"94619", "type" : "Indian", openingHours: { "serviceArea" : [ $elemMatch : { "94619", "dayOfWeek" : Monday, "94618" "open": {$lte: 1800}, ], "close": {$gte: 1800} "openingHours" : [ } { } "dayOfWeek" : Monday, } "open" : 1730, "close" : 2130 } ], Straightforward to model a "_id" : restaurant aggregate as a MongoDB ObjectId("4bddc2f49d1505567c document and execute queries 6220a0") }
  • 72. MongoDB – MVC Analytics • Monitoring page view and web application activity is critical to analyzing effectiveness of – Marketing campaigns – Site design / new features • Standard solutions are problematic aside from scaling – Google Analytics • not real time, hard to setup – MySQL • Downtime for schema changes • Solutions needs to handle high write volume, handle large amounts of data, allow for flexible data exploration.
  • 73. Web Analytics - SQL Still difficult to scale writes Application MySQL Server Master Reporting MySQL Server MySQL Slave MySQL Slave Slave
  • 74. Web Analytics - MongoDB Application Server MongoD Reporting Server
  • 75. Web Analytics - MongoDB Application MongoD Server MongoS Reporting Server MongoD
  • 76. Spring MVC based solution • Custom Handler Adapter with a ActionInterceptor • ActionInterceptor has full context of web action – Controller class, method name, and parameters – WebRequest – ModelAndView – Exception • Similar to Ruby on Rails/ASP.NET MVC Action Filters • Collect data in MongoDB – MvcEvent – Increment counters per controller and action method • AnalyticsController to display results – Action method parameter information is critical for reporting • Also a good fit for an Insight plugin!
  • 77. Collected Sample Data - Counters { "_id" : ObjectId("4cba8e0a5a49000000004961"), "name" : "SignUpController", "count" : 8, "methods" : { "create" : 2, "createForm" : 2, "show" : 4 } } { "_id" : ObjectId("4cba8ea85a49000000004962"), "name" : "RestaurantController", "count" : 9, "methods" : { "addFavoriteRestaurant" : 2, "list" : 5, "show" : 2 } }
  • 78. Collected Sample Data { "_id" : ObjectId("4cba929e4b445bc779b65c5f"), "action" : "addFavoriteRestaurant", "controller" : "RestaurantController", "date" : "Sun Oct 17 2010 02:07:26 GMT-0400 (Eastern Daylight Time)", "parameters" : { "p1" : "5", "p2" : "1", "p3" : "{currentUserAccountId= 1, userAccount_birthdate_date_format=M/d/yy, useraccount=Id: 1, Version: 1, UserName: mpollack, FirstName: Mark, LastName: Pollack, BirthDate: 2010-10-27 00:00:00.0, Favorites: 1, itemId=5}" }, "remoteUser" : "mpollack", "requestAddress" : "127.0.0.1", "requestUri" : "/myrestaurants-analytics/restaurants/5/1" }
  • 82. CouchDB – Mobile offline access and sync • Smartphones have huge data capacity • Can carry large data sets with you. • Mobile devices are not always connected • Data sync is a difficult problem to solve – CouchOne Mobile handles the dirty bits of sync – Available for Android • Same programming model as on server side • Spring RestTemplate based CouchDB API – On Server and Phone • Can also be used a full-stack environment on mobile – Database, JavaScript, Web Server
  • 83. CouchDB – Offline data access and sync Continuous bi-directional replication Application Server
  • 85. Summary • Many new options for persistence – A particular use case in your application might be a good fit – Polyglot persistence will be increasingly common • Trade-offs with – Normalization – Transactions – Familiarity • The Spring Data project – Make it easier to build Spring-powered applications that use these new technologies • More info at http://www.springframework.org/spring-data – Welcome contributors, comments…
  • 86. Q&A Chris Richardson Mark Pollack crichardson@vmware.com mpollack@vmware.com @crichardson @markpollack