Groovy University
Groovy update, Groovy
ecosystem, and Gaelyk
Guillaume Laforge
Groovy Project Manager
SpringSource,
a divison of VMWare
lundi 15 novembre 2010
Guillaume Laforge
• Groovy Project Manager
• JSR-241 Spec Lead
• Head of Groovy Development
at SpringSource
• Initiator of the Grails framework
• Founder of the Gaelyk toolkit
• Co-author of Groovy in Action
• Speaker: JavaOne, QCon, JavaZone, Sun
TechDays, Devoxx, The Spring Experience,
SpringOne2GX, JAX, Dynamic Language World,
IJTC, and more... 2
lundi 15 novembre 2010
Bug killer... real insects with legs!
• Thanks to Groovy, with one hand left for coding,
I’m still more productive than with Java!
3
lundi 15 novembre 2010
Bug killer... real insects with legs!
• Thanks to Groovy, with one hand left for coding,
I’m still more productive than with Java!
3
lundi 15 novembre 2010
Agenda: 3 big parts!
• Groovy Update
– past / present / future
• Groovy’s Ecosystem
– a rich and lively ecosystem
• Gaelyk, Groovy into the Cloud
– a lightweight Groovy toolkit
for Google App Engine
4
lundi 15 novembre 2010
AST Transformation (1/2)
• Groovy 1.6 introduced AST Transformations
–AST: Abstract Syntax Tree
• Ability to change what’s being compiled by the
Groovy compiler... at compile time
–No runtime impact!
–Change the semantics of your programs! Even hijack
the Groovy syntax!
–Implementing recurring patterns in your code base
–Remove boiler-plate code
• Two kinds: global and local
–triggered by annotations
11
lundi 15 novembre 2010
AST Transformations (2/2)
• Transformations introduced in 1.6
–@Singleton
–@Immutable, @Lazy, @Delegate
–@Newify
–@Category, @Mixin
–@PackageScope
–Swing’s @Bindable and @Vetoable
–Grape’s own @Grab
12
lundi 15 novembre 2010
@Immutable
• To properly implement immutable classes
–No mutations — state musn’t change
–Private final fields
–Defensive copying of mutable components
–Proper equals() / hashCode() / toString()
for comparisons or as keys in maps
@Immutable class Coordinates {
Double lat, lng
}
def c1 = new Coordinates(lat: 48.8, lng: 2.5)
def c2 = new Coordinates(48.8, 2.5)
assert c1 == c2
13
lundi 15 novembre 2010
Grab a grape!
• Simple distribution and sharing of Groovy scripts
• Dependencies stored locally
–Can even use your own local repositories
@Grab(group = 'org.mortbay.jetty',
module = 'jetty‐embedded',
version = '6.1.0')
def startServer() {
def srv = new Server(8080)
def ctx = new Context(srv , "/", SESSIONS)
ctx.resourceBase = "."
ctx.addServlet(GroovyServlet, "*.groovy")
srv.start()
}
14
lundi 15 novembre 2010
Anotations anywhere
• You can now put annotations
–on imports
–on packages
–on variable declarations
• Examples with @Grab following...
23
lundi 15 novembre 2010
Grape improvements (1/4)
• @Grab on import
@Grab(group = 'net.sf.json‐lib',
module = 'json‐lib',
version = '2.3',
classifier = 'jdk15')
import net.sf.json.groovy.*
assert new JsonSlurper().parseText(
new JsonGroovyBuilder().json {
book(title: "Groovy in Action",
author: "Dierk König et al")
}.toString()).book.title == "Groovy in Action"
24
lundi 15 novembre 2010
Grape improvements (2/4)
• Shorter module / artifact / version parameter
–Example of an annotation on a variable declaration
@Grab('net.sf.json‐lib:json‐lib:2.3:jdk15')
def builder = new net.sf.json.groovy.JsonGroovyBuilder()
def books = builder.books {
book(title: "Groovy in Action", author: "Dierk Koenig")
}
assert books.toString() ==
'{"books":{"book":{"title":"Groovy in Action",' +
'"author":"Dierk Koenig"}}}'
25
lundi 15 novembre 2010
Grape improvements (3/4)
• Groovy 1.7 introduced Grab resolver
–For when you need to specify a specific repository
for a given dependency
@GrabResolver(
name = 'restlet.org',
root = 'http://maven.restlet.org')
@Grab('org.restlet:org.restlet:1.1.6')
import org.restlet.Restlet
26
lundi 15 novembre 2010
Grape improvements (4/4)
• Groovy 1.7.5 even further shrinks the grab
resolver definition:
27
@GrabResolver('http://maven.restlet.org')
@Grab('org.restlet:org.restlet:1.1.6')
import org.restlet.Restlet
lundi 15 novembre 2010
Power Asserts (1/2)
• Much better assert statement!
–Invented and developed in the Spock framework
• Given this script...
def energy = 7200 * 10**15 + 1
def mass = 80
def celerity = 300000000
assert energy == mass * celerity ** 2
28
lundi 15 novembre 2010
Easier AST Transformations
• AST Transformations are a very powerful feature
• But are still rather hard to develop
–Need to know the AST API closely
• To help with authoring your own transformations,
we’ve introduced
–the AST Viewer in the Groovy Console
–the AST Builder
30
lundi 15 novembre 2010
Customize the Groovy Truth!
• Ability to customize the truth by implementing a
boolean asBoolean() method
class Predicate {
boolean value
boolean asBoolean() { value }
}
def truePred = new Predicate(value: true)
def falsePred = new Predicate(value: false)
assert truePred && !falsePred
33
lundi 15 novembre 2010
Groovy 1.7.x changes
• Since Groovy 1.7.0...
Groovy 1.7.1, 1.7.2, 1.7.3, 1.7.4 and 1.7.5
have been released already!
• Here’s what’s new!
35
lundi 15 novembre 2010
XML back to String
• Ability to retrieve the XML string from a node
from an XmlSlurper GPathResult
def xml = """
<books>
<book isbn="12345">Groovy in Action</book>
</books>
"""
def root = new XmlSlurper().parseText(xml)
def someNode = root.book
def builder = new StreamingMarkupBuilder()
assert builder.bindNode(someNode).toString() ==
"<book isbn='12345'>Groovy in Action</book>"
37
lundi 15 novembre 2010
What we’re working on
• More runtime performance improvements
• Closures
–annotation parameters
composition, memoization, and trampoline
• Native JSON support
–builder and parser
• New AST transformations
• Gradle build
• Modularizing Groovy
• Align with JDK 7 / Java 7 / Project Coin
• Enhanced DSL support 43
lundi 15 novembre 2010
Closure annotation parameters
• Groovy 1.5 brought Java 5 annotations
• What if... we could go beyond what Java offered?
–In 1.7, we can put annotations on packages, imports
and variable declarations
–But annotations are still limited in terms of parameters
they allow
• Here comes closure annotation parameters!
–Groovy 1.8 will give us the ability to access annotation
with closure parameters at runtime
44
lundi 15 novembre 2010
GContracts
• Closures are already allowed in the Groovy 1.7
Antlr grammar
–André Steingreß created GContracts,
a «design by contract» module
// a class invariant
@Invariant({ name.size() > 0 && age > ageLimit() })
class Person { String name; int age }
// a method pre‐condition
@Requires({ message != null })
void greet(String message) { ... }
// a method post‐condition
@Ensures({ returnResult % 2 == 0 })
int evenResult() { ... }
45
lundi 15 novembre 2010
Closure memoization
• Memoization: remember the outcome of previous
(ideally side-effect free) invocations
47
def c = { a, b ‐> sleep 1000; a + b }.memoize()
assert c(1, 2) == 3 // after 1000ms
assert c(1, 2) == 3 // return immediately
// other forms:
// at least 10 invocations cached
def cAtLeast = { ... }.memoizeAtLeast(10)
// at most 10 invocations cached
def cAtMost = { ... }.memoizeAtMost(10)
// between 10 and 20 invocations cached
def cAtLeast = { ... }.memoizeBetween(10, 20)
lundi 15 novembre 2010
Closure trampoline
• No more infamous StackOvervlow errors for
deeply recursive algorithms
–if you use closure trampolines explicitely
48
def fact
fact = { int n, BigInteger accu ‐>
n > 1 ? fact.curry(n ‐ 1, n * accu) : accu
}
def factorial = { int n ‐> fact(n, 1) }
assert factorial.trampoline(1) == 1
assert factorial.trampoline(3) == 6
assert factorial.trampoline(1000) == 402387260077...
lundi 15 novembre 2010
New AST Transformations
• Many new transformations coming up for removing even
more boiler plate code
49
@Log inject a logger in your classes
@Field creates a field in your scripts
@PackageScope improvements (methods & fields)
@Synchronized providing safer synchronization semantics
@InheritConstructor ex. when extending Exception
@IndexedProperties JavaBeans indexed property support
@AutoClone automatic cloning of your beans
@AutoExternalizable automatic externalization of your beans
@Canonical adding equals, hashCode, toString
@EqualsAndHashCode only adding equals and hashCode
@ToString only adding toString
@TupleConstructor for adding a tuple constructor
lundi 15 novembre 2010
More modular build
• «Not everybody needs everything!» ™
• A lighter Groovy-core
–what’s in groovy-all?
• Modules
–test, jmx, swing, xml, sql, web, template
–integration (bsf, jsr-223)
–tools (groovydoc, groovyc, shell, console, java2groovy)
51
lundi 15 novembre 2010
Java 7 / 8 / Project Coin
• JSR-292 InvokeDynamic
• Simple Closures (8)
• Proposals from Project Coin
–Strings in switch (7)
–Automatic Resource Management (7)
–Improved generics type inference (diamond <>) (7)
–Simplified varargs method invocation (7)
–Better integral literals (7)
–Language support for collections (8?) 52
lundi 15 novembre 2010
Java 7 / 8 / Project Coin
• JSR-292 InvokeDynamic
• Simple Closures (8)
• Proposals from Project Coin
–Strings in switch (7)
–Automatic Resource Management (7)
–Improved generics type inference (diamond <>) (7)
–Simplified varargs method invocation (7)
–Better integral literals (7)
–Language support for collections (8?) 52
lundi 15 novembre 2010
Java 7 / 8 / Project Coin
• JSR-292 InvokeDynamic
• Simple Closures (8)
• Proposals from Project Coin
–Strings in switch (7)
–Automatic Resource Management (7)
–Improved generics type inference (diamond <>) (7)
–Simplified varargs method invocation (7)
–Better integral literals (7)
–Language support for collections (8?) 52
lundi 15 novembre 2010
Java 7 / 8 / Project Coin
• JSR-292 InvokeDynamic
• Simple Closures (8)
• Proposals from Project Coin
–Strings in switch (7)
–Automatic Resource Management (7)
–Improved generics type inference (diamond <>) (7)
–Simplified varargs method invocation (7)
–Better integral literals (7)
–Language support for collections (8?) 52
lundi 15 novembre 2010
Java 7 / 8 / Project Coin
• JSR-292 InvokeDynamic
• Simple Closures (8)
• Proposals from Project Coin
–Strings in switch (7)
–Automatic Resource Management (7)
–Improved generics type inference (diamond <>) (7)
–Simplified varargs method invocation (7)
–Better integral literals (7)
–Language support for collections (8?) 52
lundi 15 novembre 2010
Improved DSL support
• GEP-3: an extended command expression DSL
–Groovy Extension Proposal #3
• Command expressions
–basically top-level statements without parens
–combine named and non-named arguments in the mix
• for nicer Domain-Specific Languages
–(methodName arguments)*
53
lundi 15 novembre 2010
Before GEP-3
• The idea: extend command-expressions, beyond
top-level statements, for chained method calls
• Before
send("Hello").to("Graeme")
check(that: margherita).tastes(good)
sell(100.shares).of(MSFT)
take(2.pills).of(chloroquinine).after(6.hours)
wait(10.minutes).and(execute { })
blend(red, green).of(acrylic)
54
lundi 15 novembre 2010
With GEP-3
• The idea: extend command-expressions, beyond
top-level statements, for chained method calls
• After
send "Hello" to "Graeme"
check that: margherita tastes good
sell 100.shares of MSFT
take 2.pills of chloroquinine after 6.hours
wait 10.minutes and execute { }
blend red, green of acrylic
55
lundi 15 novembre 2010
With GEP-3
• The idea: extend command-expressions, beyond
top-level statements, for chained method calls
• After
send "Hello" to "Graeme"
check that: margherita tastes good
sell 100.shares of MSFT
take 2.pills of chloroquinine after 6.hours
wait 10.minutes and execute { }
blend red, green of acrylic
Less parens& commas
55
lundi 15 novembre 2010
Summary
• No need to wait for Java 7, 8, 9...
–closures, properties, interpolated strings, extended
annotations, metaprogramming, [YOU NAME IT]...
57
lundi 15 novembre 2010
Summary
• No need to wait for Java 7, 8, 9...
–closures, properties, interpolated strings, extended
annotations, metaprogramming, [YOU NAME IT]...
Groovy’s still
innovative
since 2003!
57
lundi 15 novembre 2010
Summary
• No need to wait for Java 7, 8, 9...
–closures, properties, interpolated strings, extended
annotations, metaprogramming, [YOU NAME IT]...
Groovy’s still
innovative
since 2003!
57
lundi 15 novembre 2010
Groovy Ecosystem
A very rich & lively ecosystem
For concurrency, for testing,
for building applications...
lundi 15 novembre 2010
The Groovy Ecosystem
• Many projects based on Groovy
• Serve various purposes:
–build applications (ie. frameworks)
• Grails (web apps), Griffon (Swing apps),
Gaelyk (Google App Engine)
–tame new concurrency paradigms
• GPars (actors, agents, fork/join, map/filter/reduce...)
–enhanced testing
• Easyb (BDD), Spock, Gmock (mocking), SoapUI (WS)
–help building projects
• Gant (ant sugar), Gradle (adhoc build system)
–web services interaction
60
lundi 15 novembre 2010
Grails
–Don’t miss Graeme’s session on Grails!
–And Christian on Eclipse tooling for Groovy and Grails
61
lundi 15 novembre 2010
Griffon (1/4)
• Griffon is a Grails-like application framework for
developing rich desktop applications
–MVC, DRY, CoC patterns
–Easy threading,
data binding
and observing
–Support in IDEs
62
lundi 15 novembre 2010
Griffon (2/4)
• Let’s create a simple demo console
for executing Groovy scripts
• A data model
63
import groovy.beans.Bindable
class DemoConsoleModel {
String scriptSource
@Bindable def scriptResult
@Bindable boolean enabled = true
}
lundi 15 novembre 2010
Griffon (3/4)
• A controller
64
class DemoConsoleController {
GroovyShell shell = new GroovyShell()
// automatically injected by Griffon
def model, view
def executeScript(ActionEvent evt = null) {
model.enabled = false
doOutside {
def result = shell.evaluate(model.scriptSource)
edt {
model.enabled = true
model.scriptResult = result
}
}
}
}
lundi 15 novembre 2010
GPars (1/6)
• Actors, dataflow,
concurrency, and more
–for harnessing our
multicore platforms
• GPars web site:
–http://gpars.codehaus.org
66
lundi 15 novembre 2010
GPars (2/6)
• Improving over JSR-166y’s ParallelArray
• Asynchronous closures returning Futures
67
GParsPool.withPool { // using fork/join
// a map-reduce functional style
def smallestSelfPortrait =
images.parallel.filter { it.contains me }
.map { it.resize() }
.min { it.sizeInMB }
}
GParsExecutorsPool.withPool { // using JDK executors
// adds callAsync() to closures
assert 6 == { it * 2 }.callAsync(3).get()
}
lundi 15 novembre 2010
GPars (3/6)
• Improving over JSR-166y’s Fork / Join
68
withPool(2) { pool ->
println "Number of files: " +
runForkJoin(new File("./src")) { File file ->
long count = 0
file.eachFile {
if (it.isDirectory()) {
println "Forking a child task for $it"
// fork a child task
forkOffChild(it)
} else {
count++
}
}
// use results of children tasks
// to calculate and store own result
return count + childrenResults.sum(0)
}
}
lundi 15 novembre 2010
GPars (4/6)
• Actors
–(see also dynamic dispatch actors, and more)
69
loop {
println 'Waiting for a gift'
react { gift ->
if (myWife.likes(gift)) {
reply 'Thank you!'
} else {
reply 'Try again, please'
react { anotherGift ->
if (myChildren.like(anotherGift))
reply 'Thank you!'
}
}
}
}
lundi 15 novembre 2010
GPars (5/6)
• DataFlow concurrency
70
new DataFlows().with {
task {
z = x + y
println "Result: ${z.val}"
}
task {
x = 10
}
task {
y = 5
}
}
lundi 15 novembre 2010
GPars (6/6)
• Agents
71
class Conference extends Agent<Integer> {
def Conference() { super(0) }
private register(long num) { data += num }
private unregister(long num) { data -= num }
}
final Agent conference = new Conference()
// send commands in parallel threads
final Thread t1 = Thread.start {
conference << { register 10 }
}
final Thread t2 = Thread.start {
conference << { register 5 }
}
final Thread t3 = Thread.start {
conference << { unregister 3 }
}
[t1, t2, t3]*.join()
assert 12 == conference.val
lundi 15 novembre 2010
Easyb (1/4)
• From the website:
–«Easyb is a behavior driven development (BDD)
framework for the Java platform. By using a
specification based Domain-Specific Language, Easyb
aims to enable executable, yet readable
documentation»
• Write specifications in Groovy
• Run them from CLI, Maven, or Ant
–provides various report formats
• BDD == given / when / then paradigm
72
lundi 15 novembre 2010
Easyb (2/4)
• First, let’s write a story
73
given "an invalid zip code"
and "given the zipcodevalidator is initialized"
when "validate is invoked with the invalid zip code"
then "the validator instance should return false"
lundi 15 novembre 2010
Easyb (3/4)
• Let’s write the test itself
74
given "an invalid zip code", {
invalidzipcode = "221o1"
}
and "given the zipcodevalidator is initialized", {
zipvalidate = new ZipCodeValidator()
}
when "validate is invoked with the invalid zip code", {
value = zipvalidate.validate(invalidzipcode)
}
then "the validator instance should return false", {
value.shouldBe false
}
lundi 15 novembre 2010
Easyb (4/4)
• There’s also the specification format
75
before "initialize zipcodevalidator instance", {
zipvalidate = new ZipCodeValidator()
}
it "should deny invalid zip codes", {
["221o1", "2210", "22010-121o"].each { zip ->
zipvalidate.validate(zip).is false
}
}
it "should accept valid zip codes", {
["22101", "22100", "22010-1210"].each { zip ->
zipvalidate.validate(zip).shouldBe true
}
}
lundi 15 novembre 2010
Spock (1/3)
• From the horse’s mouth:
–« Spock is a testing and specification framework for
Java and Groovy applications. What makes it stand out
from the crowd is its beautiful and highly expressive
specification language. »
76
class HelloSpock extends Specification {
def "can you figure out what I'm up to?"() {
expect:
name.size() == size
where:
name | size
"Kirk" | 4
"Spock" | 5
"Scotty" | 6
}
}
lundi 15 novembre 2010
Spock (2/3)
• Extensible (but transparent) use of AST
transformations to «pervert» the Groovy
language
–reuse of labels for the BDD actions
• Spock brought Groovy 1.7’s enhanced asserts
77
Condition not satisfied:
max(a, b) == c
| | | | |
3 1 3 | 2
false
lundi 15 novembre 2010
Spock (3/3)
• Another example of the expressive language
–with powerful mocking capabilities
78
def "subscribers receive published events at least once"() {
when: publisher.send(event)
then: (1.._) * subscriber.receive(event)
where: event << ["started", "paused", "stopped"]
}
lundi 15 novembre 2010
GMock (1/3)
• GMock is a mocking framework for Groovy
• Mocking capabilities
79
File mockFile = mock(File, constructor("/a/path/file.txt"))
mockFile.getName().returns("file.txt")
play {
def file = new File("/a/path/file.txt")
assertEquals "file.txt", file.getName()
}
–method calls
–property access
–static calls
–partial mocks
–constructor calls
–time matching
–order matching
–regex method matching
lundi 15 novembre 2010
Gant
• Gant is a tool for scripting Ant tasks using Groovy
instead of XML to specify the logic
82
includeTargets << gant.targets.Clean
cleanPattern << ['**/*~', '**/*.bak']
cleanDirectory << 'build'
target(stuff: 'A target to do some stuff.') {
println 'Stuff'
depends clean
echo message: 'A default message from Ant.'
otherStuff()
}
target(otherStuff: 'A target to do some other stuff') {
println 'OtherStuff'
echo message: 'Another message from Ant.'
clean()
}
setDefaultTarget stuff
lundi 15 novembre 2010
Gradle (1/3)
• Gradle is an enterprise-grade build system
–A very flexible general purpose build tool like Ant
–Switchable, build-by-convention frameworks à la
Maven, for Java, Groovy and Scala projects
–Groovy build scripts
–Powerful support for multi-project builds
–Dependency management based on Ivy
–Full support for existing Maven or Ivy repository infra
–Support for transitive dependency management
–Ant tasks and builds as first class citizens
83
lundi 15 novembre 2010
Gradle (2/3)
• For example, for a classical Java project
84
apply plugin: ‘java’
repositories {
mavenCentral()
}
dependencies {
compile group: ‘commons-collection’,
module: ‘commons-collection’, version: ‘3.2’
testCompile group: ‘junit’,
module: ‘junit’, version: ‘4.+’
}
lundi 15 novembre 2010
Gradle (3/3)
• Major projects migrate to Gradle
–Hibernate
–Various SpringSource projects
–Groovy -based projects: Grails, Groovy, GPars, Gaelyk
–Customers with 200+ multi-projects
85
lundi 15 novembre 2010
Other build and quality tools
• You can use Ant, Gant, Maven (GMaven plugin)
or Gradle to build your projects
–fits nicely in any existing build and continuous
integration infrastructure
• A few Groovy-friendly tools provide handy
metrics
–CodeNarc: provides various rules for static analysis of
your Groovy codebase (style, size, braces, naming...)
–Simian: detects duplication in your Groovy code base
–Cobertura, Clover: give code coverage metrics
86
lundi 15 novembre 2010
HTTPBuilder (3/3)
• Posting XML content to a URL
89
http.request(POST, XML) {
body = {
auth {
user 'Bob'
password 'pass'
}
}
}
lundi 15 novembre 2010
GroovyWS (1/2)
• Publishing and consuming WS-I
compliant SOAP web services
• Can also deal with complex objects
90
import groovyx.net.ws.WSClient
def proxy = new WSClient(
"http://www.w3schools.com/webservices/tempconvert.asmx?WSDL",
this.class.classLoader)
proxy.initialize()
def result = proxy.CelsiusToFahrenheit(0)
println "You are probably freezing at ${result} degrees Farhenheit"
lundi 15 novembre 2010
GroovyWS (2/2)
• Publishing a service
–considering a service
–publishing the service
91
class MathService {
double add(double arg0, double arg1) { arg0 + arg1 }
double square(double arg0) { arg0 * arg0 }
}
import groovyx.net.ws.WSServer
def server = new WSServer()
server.setNode("MathService", "http://localhost:6980/MathService")
server.start()
lundi 15 novembre 2010
Summary
• «Groovy» is way more than just a language!
• It’s also a very rich and active ecosystem!
–Grails, Griffon, Gradle, GPars, Spock, Gaelyk...
93
lundi 15 novembre 2010
IaaS, PaaS, SaaS
• Software as a Service
–Gmail, SalesForce.com
• Platform as a Service
–Google App Engine
• Infrastructure as a Service
–Amazon EC2
SaaS
PaaS
IaaS
97
lundi 15 novembre 2010
IaaS, PaaS, SaaS
• Software as a Service
–Gmail, SalesForce.com
• Platform as a Service
–Google App Engine
• Infrastructure as a Service
–Amazon EC2
SaaS
PaaS
IaaS
97
lundi 15 novembre 2010
IaaS, PaaS, SaaS
• Software as a Service
–Gmail, SalesForce.com
• Platform as a Service
–Google App Engine
• Infrastructure as a Service
–Amazon EC2
SaaS
PaaS
IaaS
97
lundi 15 novembre 2010
Google App Engine
• Google’s PaaS solution
• Run your app on Google’s infrastructure
• Initially just Python supported
• 1+ year ago, Java supported added too
–Sandboxed JVM
–Jetty servlet container
• Several JVM-compatible language supported
98
lundi 15 novembre 2010
Key aspects
• You can use most of your usual web frameworks for
developping apps on App Engine Java
–A WAR file, basically!
–Uploading to the cloud by sending deltas of changes
• No OS image, or software to install
–Unlike with Amazon EC2
• All the scaling aspects are handled for you
–Database / session replication, load balancing...
• There are quotas, but you need a high traffic
application to start being charged
–Free to get started 99
lundi 15 novembre 2010
Available services
• Memcache
–JCache implementation
–Save on CPU and DB
• URL Fetch
–Access remote resources
–HttpUrlConnection
• Mail
–Support both incoming
and outgoing emails
• Images
–Resize, crop, rotate...
• XMPP
–Send / receive Jabber
messages (GTalk)
• User
–Use Google’s user/
authentication system
–OAuth support
• Cron & Task queues
–Schedule tasks at regular
intervals
–Queue units of work
• Blobstore
–For storing large content
• And much more...100
lundi 15 novembre 2010
Limitations
• Not our usual relational database
–key / value datastore
• 30 seconds request duration limit
• Forbidden to
–write on the file system
–create threads
–use raw sockets
–issue system calls
–use IO / Swing / etc. directly
• There’s a whitelist of classes allowed
• Number of files and their size are limited 101
lundi 15 novembre 2010
The datastore...
• It’s not your father’s relational database! «NoSQL»
• Distributed key / value store
–Based on Google’s «BigTable»
–Schema-less approach
• Supporting
–Transactions and partitioning
–Hierarchies through entity groups
• Data access APIs
–JPA and JDO
• but adds a big request load time factor
–Direct low-level APIs
106
lundi 15 novembre 2010
...and its «limitations»
• You’re not using SQL
–No joins
–No database constraints
–No aggregation functions (count, avg...)
• In a query, you can only filter on one column for
inequality
• Transactions only available in entity groups
• You can only update an entity once in a transaction
107
lundi 15 novembre 2010
Gaelyk
• Gaelyk is a lightweight Groovy toolkit on top of
the Google App Engine Java SDK
• Gaelyk builds on Groovy’s servlet support
–Groovlets: Groovy scripts instead of raw servlets!
–Groovy templates: JSP-like template engine
–Both allow for a clean separation of views and logic
• Gaelyk provides several enhancements around
the GAE Java SDK to make life easier, thanks to
Groovy’s dynamic nature
110
lundi 15 novembre 2010
Why Groovy?
• Groovy is a dynamic language for the JVM
–very flexible, malleable, expressive and concise syntax
–easy to learn for Java developers
• deriving from the Java 5 grammar
–provides powerful APIs to simplify the life of developers
• possibility to dynamically enrich existing APIs
–support for Groovlets and its own template engine
• The truth: We worked with the Google App Engine
Java team before the official launch of the platform,
to ensure Groovy would run well on this new
environment, and Gaelyk emerged from our test-bed
project, and people asked me to Open-Source it!111
lundi 15 novembre 2010
First steps...
•Go to http://gaelyk.appspot.com
•Download the template project
•Put your first Groovlet in /WEB-INF/groovy
•And your templates in /WEB-INF/pages
•And you’re ready to go!
•Launch dev_appserver.sh
•Go to http://localhost:8080/
lundi 15 novembre 2010
MVC: Groovlets and templates
Groovlets
(controllers)
Templates
(views)
En55es
(domain)
lundi 15 novembre 2010
A groovlet
• Instead of writing full-blown servlets, just write
Groovy scripts (aka Groovlets)
def numbers = [1, 2, 3, 4]
def now = new Date()
html.html {
body {
numbers.each { number -> p number }
p now
}
}
116
lundi 15 novembre 2010
A groovlet
• Instead of writing full-blown servlets, just write
Groovy scripts (aka Groovlets)
def numbers = [1, 2, 3, 4]
def now = new Date()
html.html {
body {
numbers.each { number -> p number }
p now
}
}
116
Auto‐reloading
lundi 15 novembre 2010
Sending emails with Gaelyk
mail.send to: 'foobar@gmail.com',
from: 'other@gmail.com',
subject: 'Hello World',
htmlBody: '<bold>Hello</bold>'
121
lundi 15 novembre 2010
...compared to Java
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
String msgBody = "...";
try {
Message msg = new MimeMessage(session);
msg.setFrom(new InternetAddress("admin@example.com", "Admin"));
msg.addRecipient(Message.RecipientType.TO,
new InternetAddress("user@example.com", "Mr. User"));
msg.setSubject("Your Example.com account has been activated");
msg.setText(msgBody);
Transport.send(msg);
} catch (AddressException e) {}
} catch (MessagingException e) {}
122
lundi 15 novembre 2010
Accessing the datastore
• Direct interaction with the low-level datastore API
import com.google.appengine.api.datastore.Entity
Entity entity = new Entity("person")
// subscript notation, like when accessing a map
entity['name'] = "Guillaume Laforge"
// normal property access notation
entity.age = 32
entity.save()
entity.delete()
datastore.withTransaction {
// do stuff with your entities
// within the transaction
}
123
lundi 15 novembre 2010
Querying to be improved...
import com.google.appengine.api.datastore.*
import static com.google.appengine.api.datastore.FetchOptions.Builder.*
// query the scripts stored in the datastore
def query = new Query("savedscript")
// sort results by descending order of the creation date
query.addSort("dateCreated", Query.SortDirection.DESCENDING)
// filters the entities so as to return only scripts by a certain author
query.addFilter("author", Query.FilterOperator.EQUAL, params.author)
PreparedQuery preparedQuery = datastore.prepare(query)
// return only the first 10 results
def entities = preparedQuery.asList( withLimit(10) )
124
lundi 15 novembre 2010
...into something groovier?
def entities = datastore.query {
select all from savedscript
sort desc by dateCreated
where author == params.author
limit 10
} as List
125
lundi 15 novembre 2010
...into something groovier?
def entities = datastore.query {
select all from savedscript
sort desc by dateCreated
where author == params.author
limit 10
} as List
Not Yet Implemented!
125
lundi 15 novembre 2010
Task queue API
// access a configured queue using the subscript notation
queues['dailyEmailQueue']
// or using the property access notation
queues.dailyEmailQueue
// you can also access the default queue with:
queues.default
defaultQueue
// add a task to the queue
queue << [
countdownMillis: 1000, url: "/task/dailyEmail",
taskName: "sendDailyEmailNewsletter",
method: 'PUT', params: [date: '20090914'],
payload: content
]
126
lundi 15 novembre 2010
Jabber / XMPP support (1/3)
• Sending instant messages
String recipient = "someone@gmail.com"
// check if the user is online
if (xmpp.getPresence(recipient).isAvailable()) {
// send the message
def status = xmpp.send(to: recipient,
body: "Hello, how are you?")
// checks the message was successfully
// delivered to all the recipients
assert status.isSuccessful()
}
127
lundi 15 novembre 2010
Jabber / XMPP support (2/3)
• Sending instant messages with an XML payload
String recipient = "service@gmail.com"
// check if the service is online
if (xmpp.getPresence(recipient).isAvailable()) {
// send the message
def status = xmpp.send(to: recipient, xml: {
customers {
customer(id: 1) {
name 'Google'
}
}
})
// checks the message was successfully delivered to the service
assert status.isSuccessful()
}
128
lundi 15 novembre 2010
Jabber / XMPP support (2/3)
• Sending instant messages with an XML payload
String recipient = "service@gmail.com"
// check if the service is online
if (xmpp.getPresence(recipient).isAvailable()) {
// send the message
def status = xmpp.send(to: recipient, xml: {
customers {
customer(id: 1) {
name 'Google'
}
}
})
// checks the message was successfully delivered to the service
assert status.isSuccessful()
}
<customers>
<customer id=’1’>
<name>Google</name>
</customer>
</customers>
128
lundi 15 novembre 2010
Jabber / XMPP support (3/3)
• Receving incoming instant messages
–Configure the XmppServlet in web.xml
–Add the inbound message service in appengine-
web.xml
// get the body of the message
message.body
// get the sender Jabber ID
message.from
// get the list of recipients Jabber IDs
message.recipients
// if the message is an XML document instead of a raw string message
if (message.isXml()) {
// get the raw XML
message.stanza
// get a document parsed with XmlSlurper
message.xml
}
129
lundi 15 novembre 2010
Memcache service
• Map notation access to the cache
class Country implements Serialzable { String name }
def countryFr = new Country(name: 'France')
// use the subscript notation to put a country object in the cache
// (you can also use non-string keys)
memcache['FR'] = countryFr
// check that a key is present in the cache
if ('FR' in memcache) {
// use the subscript notation to get an entry from the cache using a key
def countryFromCache = memcache['FR']
}
130
lundi 15 novembre 2010
Closure memoization
• Cache the return values for each dinstinct
invocation (for a given arguments set)
131
def countEntities = memcache.memoize { String kind ->
datastore.prepare( new Query(kind) )
.countEntities()
}
// first call
def totalPics = countEntityes('photos')
// second call, hitting the cache
totalPics = countEntityes('photos')
lundi 15 novembre 2010
Blobstore enhancements
• The blobstore allows to store some large content
–images, videos, etc.
132
def blob = ...
print blob.filename // contentType, creation, size
// output the content of the blob to the response
blob.serve response
// read the content of the blob
blob.withReader { Reader r -> ... }
blob.withStream { InputStream is -> ... }
// delete the blob
blob.delete()
lundi 15 novembre 2010
Capabilities
• Google App Engine allows you to know the status
and availability of the various services
–DATASTORE, DATESTORE_WRITE, MEMCACHE...
–ENABLED, DISABLED, UNKNOWN,
SCHEDULED_MAINTENANCE
–is() and not() methods
134
if (capabilities[DATASTORE_WRITE].is(ENABLED)) {
// write some content in the datastore
} else {
// otherwise, redirect to some maintenance page
}
lundi 15 novembre 2010
URL Routing system (1/3)
• You can have friendly URL mappings with the
URL routing system
–You’ll have to configure the RouteFilter in web.xml
all "/blog/@year/@month/@day/@title",
forward: "/blog.groovy?year=@year&month=@month@day=@day@title=@title"
get "/blog/@year/@month/@day",
forward: "/blog.groovy?year=@year&month=@month@day=@day"
all "/aboutus",
redirect: "/blog/2008/10/20/about-us"
get "/book/isbn/@isbn",
forward: "/book.groovy?isbn=@isbn",
validate: { isbn ==~ /d{9}(d|X)/ }
135
lundi 15 novembre 2010
URL Routing system (2/3)
• You can also define caching times
–Nice for GAE’s infamous «loading requests»
get "/aboutus", cache: 24.hours,
forward: "/aboutus.gtpl"
get "/breaking-news", cache: 1.minute,
forward: "/news.groovy?last=10"
136
lundi 15 novembre 2010
URL Routing system (3/3)
• Namespace awareness: nice for multitenancy
• Capability awareness: for graceful degradation
137
// @cust customer variable could be « acme »
post "/@cust/update", forward: "/update.groovy",
namespace: { "ns-$cust" }
// different destinations depending on the GAE services status
get "/speakers", forward {
to "/speakers.groovy" // default destination
// when the datastore is not available
to "/unavailable.gtpl" on DATASTORE not ENABLED
// show some maintenance is upcoming
to "/speakers.groovy?maintenance=true" on DATASTORE
is SCHEDULED_MAINTENANCE
}
lundi 15 novembre 2010
Simple plugin system (1/3)
• Gaelyk features a simple plugin system for
extending your apps and share commonalities
• A plugin lets you
–provide additional groovlets and templates
–contribute new URL routes
–add new categories
–define variables in the binding
–provide any static content
–add new libraries
–do any initialization
138
lundi 15 novembre 2010
Simple plugin system (2/3)
• A plugin is actually just a zip file!
–Basically, just a Gaelyk application, minus...
• the Groovy / Gaelyk / GAE JARs
• the web.xml and appengine-web.xml descriptors
–But with a /WEB-INF/plugins/myplugin.groovy
descriptor
• A plugin must be referenced in /WEB-INF/
plugins.groovy with
–install myplugin
• shortcut to /WEB-INF/plugins/myplugin.groovy
139
lundi 15 novembre 2010
Simple plugin system (3/3)
• An example plugin descriptor
–/WEB-INF/plugins/jsonPlugin.groovy
import net.sf.json.*
import net.sf.json.groovy.*
// add new variables in the binding
binding {
jsonLibVersion = "2.3" // a simple string variable
json = new JsonGroovyBuilder() // an instance of a class of a 3rd‐party JAR
}
// add new routes with the usual routing system format
routes {
get "/json", forward: "/json.groovy"
}
// install a category you've developped
categories jsonlib.JsonlibCategory
// any other initialization code you'd need
140
lundi 15 novembre 2010
Summary
• Easy access to a cloud solution
–Deploying Java apps, as easily as you would with PHP
• Familiar to Java folks
–Your good old Servlet centric webapps style
• Pretty cheap
–You need a high-trafficed website to reach the quotas
• Gaelyk provides a simplified approach to
creating Servlet centric webapps in a productive
manner
–Leveraging Groovy’s servlet / template support and
dynamic capabilities
142
lundi 15 novembre 2010
What’s coming next?
• Gaelyk 0.5.6 released couple weeks ago with
additional URL routing support
–capabilities, namespace support
• Expect more sugar around the Datastore
–An SQL-like query DSL
–Easier relationship management (builder)
• Perhaps pre-compiled groovlets and templates
• Testing facilities specific to Gaelyk
• More generally...
–Anything that’ll come up in upcoming GAE SDK
versions 143
lundi 15 novembre 2010
Thanks for your attention!
Guillaume Laforge
Head of Groovy Development
Email: glaforge@gmail.com
TwiEer: @glaforge
• References:
• http://gaelyk.appspot.com/
• http://groovy.codehaus.org/
• http://code.google.com/appengine/
145
lundi 15 novembre 2010
Images used in this presentation
Three fingers http://markyoungtrainingsystems.com/wp-content/uploads/2009/10/three-fingers.jpg
Bandage http://image.made-in-china.com/2f0j00fIGaTPEYDQnB/Elastic-Crepe-Bandage-Gauze-.jpg
RIP http://gipsydan.files.wordpress.com/2009/11/rip.jpg
House / past: http://www.flickr.com/photos/jasonepowell/3680030831/sizes/o/
Present clock: http://www.flickr.com/photos/38629278@N04/3784344944/sizes/o/
Future: http://www.flickr.com/photos/befuddledsenses/2904000882/sizes/l/
Cooking: http://www.flickr.com/photos/eole/449958332/sizes/l/
Puzzle: http://www.everystockphoto.com/photo.php?imageId=263521
Light bulb: https://newsline.llnl.gov/retooling/mar/03.28.08_images/lightBulb.png
Speed limit : http://www.morguefile.com/archive/display/18492
Warehouse : http://www.morguefile.com/archive/display/85628
Check mark: http://www.lnl.infn.it/~epics/WikiDumps/localhost/600px-symbol_ok.svg.png
Puzzle: http://www.everystockphoto.com/photo.php?imageId=263521
Light bulb: https://newsline.llnl.gov/retooling/mar/03.28.08_images/lightBulb.png
Clouds http://www.morguefile.com/archive/display/627059
http://www.morguefile.com/archive/display/625552
http://www.morguefile.com/archive/display/629785
Duke ok GAE http://weblogs.java.net/blog/felipegaucho/archive/ae_gwt_java.png
Python logo : http://python.org/images/python-logo.gif
Gaelyc cross with clouds : http://www.morguefile.com/archive/display/37889
Speed limit : http://www.morguefile.com/archive/display/18492
Warehouse : http://www.morguefile.com/archive/display/85628
Snow foot steps : http://www.flickr.com/photos/robinvanmourik/2875929243/
Sugar : http://www.flickr.com/photos/ayelie/441101223/sizes/l/
Press release: http://www.napleswebdesign.net/wp-content/uploads/2009/06/press_release_11.jpg
Green leaf: http://www.flickr.com/photos/epsos/3384297473/
Jeepers creepers: http://www.flickr.com/photos/27663074@N07/3413698337/
Earth: http://www.flickr.com/photos/wwworks/2222523978/
Trafic light: http://rihancompany.com/var/243/35581-Traffic%20light%20drawing.jpg
Butt kick: http://www.brentcsutoras.com/wp-content/uploads/2009/12/kick-ass-1.jpg
Griffon: http://aldaria02.a.l.pic.centerblog.net/lz2levrz.jpg
Soap: http://www.flickr.com/photos/geishaboy500/104137788/
146
lundi 15 novembre 2010