Monitoring and Feature Toggle Pattern with JMX @ Paris Devops Meetup

1. Monitoring and
Feature Toggle Pattern
with JMX
Bruno Bonfils
Cyrille Le Clerc 15/06/2011
Thursday, June 16, 2011

2. Speaker
@cyrilleleclerc
blog.xebia.fr
Cyrille Le Clerc
Large Scale
In Memory Data Grid
Open Source
(Apache CXF, ...)
“you build it, you run it”
2
Thursday, June 16, 2011

3. The use case
3
Thursday, June 16, 2011

4. The Use Case
Corporate Data Center
travel-ecommerce
anti-fraud
Tomcat
Tomcat
travel-ecommerce anti-fraud
Tomcat Tomcat
Credit Card Service
Travel e-commerce application
4
Thursday, June 16, 2011

5. The Use Case
 xebia-spring-travel source code
▶ http://xebia-france.googlecode.com/svn/training/xebia-spring-travel/tags/xebia-spring-travel-1.0.0/
▶ Groovy JMX scripts
» http://xebia-france.googlecode.com/svn/training/xebia-spring-travel/tags/xebia-spring-travel-1.0.0/xebia-spring-travel-
ecommerce/src/main/scripts/
▶ JMXTrans Configuration
» http://xebia-france.googlecode.com/svn/training/xebia-spring-travel/tags/xebia-spring-travel-1.0.0/xebia-spring-travel-
ecommerce/src/main/jmxtrans/
▶ JMeter plan
» http://xebia-france.googlecode.com/svn/training/xebia-spring-travel/tags/xebia-spring-travel-1.0.0/xebia-spring-travel-
ecommerce/src/main/jmeter/
 xebia-management-extras JMX library
▶ http://code.google.com/p/xebia-france/wiki/XebiaManagementExtras
5
Thursday, June 16, 2011

6. Part 1
Monitoring with JMX
6
Thursday, June 16, 2011

7. Simplified Use Case
7
Thursday, June 16, 2011

8. Simplified Use Case
Corporate Data Center
travel-ecommerce
Tomcat
Credit Card Service
Mock
Monitoring Booking and Credit Card Service
8
Thursday, June 16, 2011

9. Simplified Use Case
Corporate Data Center
travel-ecommerce
Tomcat
Credit Card Service
Mock
Monitoring Booking and Credit Card Service
9
Thursday, June 16, 2011

10. Custom/business indicator monitoring
with JMX
10
Thursday, June 16, 2011

11. Custom Indicator Monitoring with JMX
CreditCardService
#purchase()
CreditCardService MonitoringImpl CreditCardServiceImpl
#purchase() #purchase()
monitoring logic isolated with a delegate pattern
11
Thursday, June 16, 2011

12. Custom Indicator Monitoring with JMX
@ManagedResource
public class CreditCardServiceMonitoringImpl implements CreditCardService {
// delegate
private CreditCardService creditCardService;
// counters
private final AtomicInteger purchaseInvocationCounter = new AtomicInteger();
private final AtomicLong purchaseInvocationDurationInNanosCounter = new AtomicLong();
private final AtomicInteger threeDSecureVerificationExceptionCounter = new AtomicInteger();
@Override
public PaymentTransaction purchase(MonetaryAmount total, Order order, String requestId) {
long nanosBefore = System.nanoTime();
try {
return creditCardService.purchase(total, order, requestId);
} catch (ThreeDSecureVerificationException e) {
threeDSecureVerificationExceptionCounter.incrementAndGet();
throw e;
} finally {
purchaseInvocationCounter.incrementAndGet();
purchaseInvocationDurationInNanosCounter.addAndGet(System.nanoTime() - nanosBefore);
}
}
http://code.google.com/p/xebia-france/source/browse/training/xebia-spring-travel/tags/xebia-spring-travel-1.0.0/xebia-spring-travel-ecommerce/src/main/java/fr/
xebia/monitoring/demo/payment/CreditCardServiceMonitoringImpl.java
12
Thursday, June 16, 2011

13. Accessing JMX is not so difficult
13
Thursday, June 16, 2011

14. JMX Monitoring with Visual VM
14
Thursday, June 16, 2011

15. JMX Monitoring with JSP Pages
Powered by JMX !
 Human friendly or script friendly pages
 Beware of security
▶ BasicAuth
▶ Obfuscated URL : /my-app/seye5E7E/jmx/cxf.jsp
15
Thursday, June 16, 2011

16. Graphite - JMXTrans Style Reporting Tools
{
"servers" : [ {
"port" : "6969",
"host" : "my-server-1",
"alias" : "travel-ecommerce-1",
"queries" : [
{
"outputWriters" : [ {
"@class" : "com.googlecode.jmxtrans.model.output.GraphiteWriter",
"settings" : { "port" : 2003, "host" : "graphite-server" }
}],
"obj": "travel-ecommerce:name=CreditCardServiceMonitoringImpl,...",
Graphite
"resultAlias" : "CreditCardService",
"attr":["PurchaseInvocationCount", "ThreeDSecureVerificationExceptionCount", ...]
}
],
"numQueryThreads" : 2
JMX Trans Configuration
} ]
}
http://code.google.com/p/xebia-france/source/browse/training/xebia-spring-travel/tags/xebia-spring-travel-1.0.0/xebia-spring-travel-
ecommerce/src/main/jmxtrans/xebia-spring-travel-ecommerce-jmxtrans.json 16
Thursday, June 16, 2011

17. Monotoring systems
AppDynamics
Hyperic
All others ...
17
Thursday, June 16, 2011

18. JVM Based Scripting Language
http://code.google.com/p/xebia-france/source/browse/training/xebia-spring-travel/tags/xebia-spring-
travel-1.0.0/xebia-spring-travel-ecommerce/src/main/scripts/getAntiFraudVerificationStatus.groovy
18
Thursday, June 16, 2011

19. Part 2
Feature Toggle Pattern with JMX
19
Thursday, June 16, 2011

20. Simplified Use Case
20
Thursday, June 16, 2011

21. Simplified Use Case
Corporate Data Center
travel-ecommerce
anti-fraud
Tomcat
Tomcat
Credit Card Service
Enable anti-fraud
21
Thursday, June 16, 2011

22. What ? Why ?
22
Thursday, June 16, 2011

23. What is it ?
 Technique to enable/disable a feature without
redeploying
 Can be simple : on/off
 Can be sophisticated : f(user), f(server), etc
23
Thursday, June 16, 2011

24. Benefits of the Feature Toggle Pattern
 Dissociate deployment & feature activation
 Progressive activation of a feature
Canary Testing
 Measure impacts of a new version
A/B Testing
 Differ feature activation on production
 Trunk based development
24
Thursday, June 16, 2011

25. Coding Patterns
25
Thursday, June 16, 2011

26. Coding Patterns
Dispatcher
AntiFraudService
<<Interface>>
AntiFraudServiceV1Impl
AntiFraudService
DispatchingImpl
AntiFraudServiceV2Impl
JMX
transient persistent
The dispatcher holds the feature toggle
http://code.google.com/p/xebia-france/source/browse/training/xebia-spring-travel/tags/xebia-spring-
travel-1.0.0/xebia-spring-travel-ecommerce/src/main/java/fr/xebia/ws/travel/antifraud/v1_0/
AntiFraudServiceDispatchingImpl.java 26
Thursday, June 16, 2011

27. Coding Patterns
To duplicate or not to duplicate ?
Dispatching
Smart reuse and mutualization
Dispatching
Brutal duplication
Version 1
Version 2
27
Thursday, June 16, 2011

28. Coding Patterns
To duplicate or not to duplicate ?
Dispatching
Smart reuse and mutualization
Dispatching
Brutal duplication
Version 1
Version 2
28
Thursday, June 16, 2011

29. Coding Patterns
To duplicate or not to duplicate ?
Dispatching
Smart reuse and mutualization
Dispatching
Brutal duplication
Version 1
Version 2
29
Thursday, June 16, 2011

30. Coding Patterns
To duplicate or not to duplicate ?
Old version removal requires cleanup
Smart reuse and mutualization
Old version removal is clean
Brutal duplication
Version 1
Version 2
30
Thursday, June 16, 2011

31. Coding Patterns
To duplicate or not to duplicate ?
Old code removal is simpler with brutal duplication
31
Thursday, June 16, 2011

32. Demo
32
Thursday, June 16, 2011

33. Questions / Answers
?
33
Thursday, June 16, 2011