OSGi Community Event 2014 Abstract: Salesforce.com is heavily invested in automating its worldwide data centers and their operations. The goal is to perform the workflows of the entire data center lifecycle from the bootstrapping of physical servers to the deployment and maintenance of applications. The expectation is that this will have two important effects. One, it will reduce risk by making workflows more deterministic and less prone to human error. And two, it will relieve the burden of human operators to do simple, everyday tasks and instead allow them to concentrate their time on more complicated tasks like troubleshooting and diagnostics. Because the data center is at the core of Salesforce.com’s business, there is extra emphasis placed on engineers to write software systems which are easy to reason about, test, configure, and be deployed with minimal down time. As such, several of the data center automation software teams have embraced the OSGi framework to help them meet these goals. These teams have found that the principle of separated software services as enforced by the OSGi framework have eased the design, implementation and deployment of such critical software systems. In addition, the framework is providing key functionality like Declarative Services for dependency injection and Config Admin for configuration injection which are instrumental in making these software projects successful.

1. Case Study: Using OSGi within the
Salesforce Data Center Automation
Initiative
Tim Kral, Principal Engineer
tkral@salesforce.com

2. Safe harbor statement under the Private Securities Litigation Reform Act of 1995:
This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties
materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results
expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be
deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other
financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any
statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.
The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new
functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our
operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any
litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our
relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of
our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to
larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is
included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent
fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor
Information section of our Web site.
Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently
available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based
upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-
looking statements.
Safe Harbor

3. Why Am I Giving This Talk?
Once upon a time, my co-workers were skeptical...
“OSGi seems great but I’m not sure who uses it in the real world.”
- Lee
“OSGi seems to have a high cost and I’m not sure what benefit we are
getting from it.”
- Austin
“Every time we encounter an OSGi problem, I’m going to make you put
a dollar in a jar so we have beer money.”
- Peter

4. What’s going on with the Salesforce Data Center Automation (DCA) initiative
Why Salesforce DCA use OSGi
OSGi Tricks, Tips and Lessons Learned
Demonstration of installing plugins with zero downtime
Answer your questions
Real people use OSGi in the real world
Why Am I Giving This Talk?

5. What’s going on with the Salesforce Data Center Automation (DCA) initiative
Why Salesforce DCA use OSGi
OSGi Tricks, Tips and Lessons Learned
Demonstration of installing plugins with zero downtime
Answer your questions
Real people use OSGi in the real world
Why Am I Giving This Talk?

6. 21B messages sent in a month
3.1B API calls in a month (nearly 50% of all transactions)
45 major releases
AppExchange: 2000+ app with 2M installs
Real people use OSGi in the real world
About Salesforce

7. The Cloud Computing Pyramid
Infrastructure as a Service
Virtualized Containers, Network Services
Storage, Image Management, Security
Platform as a Service
Database Services, User Management
Application Servers, Caching
Software as
a Service
Business software
consumed by end
users

8. Salesforce DCA Initiative
Infrastructure as a Service
Virtualized Containers, Network Services
Storage, Image Management, Security
Application Deployment
Services
Code / Configuration Deployment,
Maintain Cardinality
Software as
a Service
Physical Data Center Bootstrapping
OS Installation, Role Assignment
Static Network Configuration, Asset Registration
Health
Monitoring
Runtime Signal
Monitoring,
Health
Inferences,
Automated
Repair Loop
Internal Salesforce
engineers deploy services
for consumption

9. Workflow: Data Center Expansion
Application Deployment
Services
Infrastructure as a Service
Software as
a Service
Physical Data Center Bootstrapping
Health
Monitoring
1. Roll in Rack
2. Install OS
3. Register
Servers w/ Roles

10. Workflow: Software Deployment
Application Deployment
Services
Infrastructure as a Service
Software as
a Service
Physical Data Center Bootstrapping
Health
Monitoring
1. Service team
submits artifact +
configuration
2. New VM and LB
request
3. Provision VM and
LB based on
registered servers
4. Deploy app and
maintain cardinality

11. Workflow: Health Monitoring
Application Deployment
Services
Infrastructure as a Service
Physical Data Center Bootstrapping
Health
Monitoring
4. Perform repair
action
2. Aggregate
signal and
infer health
Software as
a Service
3. Request
repair if
service
unhealthy
1. Deployed app is
polled for health signal

12. Where is OSGi Used?
Application Deployment
Services
Infrastructure as a Service
Physical Data Center Bootstrapping
Health
Monitoring
Software as
a Service
OSGi !!!
Java:
Ruby:
Language agnostic:

13. What’s going on with the Salesforce Data Center Automation (DCA) initiative
Why Salesforce DCA use OSGi
OSGi Tricks, Tips and Lessons Learned
Demonstration of installing plugins with zero downtime
Answer your questions
Real people use OSGi in the real world
Why Am I Giving This Talk?

14. Salesforce Core App
• Legacy code
• 100s of software developers
• OSGi conversion
Data Center Automation
• Green field projects
• 10s of software developers
• OSGi native
A Tale of Two Use Cases

15. Why Use OSGi: Bundle Modularity
Public (API) Bundle Private (Implementation) Bundle
Simple Rules:
1. Content: Only public Java Interfaces and
shared beans
2. Export all public interfaces and classes
3. Consumer can have compile time
dependency
Simple Rules:
1. Contents: Implementations of some
interfaces from API bundle
2. Export nothing
3. Consumers may not have compile
time dependencies
Contents are available to the outside world Hidden implementations
OSGi class loading
rules prevent you
from cheating!

16. Example: Bundle Modularity
Signal Receiver
Public Bundle
App Deploy
Service
Health
Signal
Publisher
Signal
Receiver
Metrics Bus
Redis Bus
Redis Signal
Receiver Impl
Modified Kafka (Ajna)
Bus
Ajna Signal
Receiver Impl
Kafka Bus
Kafka Signal
Receiver Impl
AWS SQS Bus
AWS SQS Signal
Receiver Impl

17. Why Use OSGi: Service Modularity
Service Definition
(Public Java interface)
Service Definition
(Implementing Java class)
Service Registry
Request for Service

18. Example: Service Modularity
Health
Signal
Source
Signal
Receiver
Metrics Bus
Signal
Source Signal
Mapper
Service implementations
could be in the same
bundle or different ones
Signal Mapper
Signal Mapper
Registry
Server Network I/O
Signal Mapper
Oracle Stats
Signal Mapper
Container Memory
Use Signal Mapper
Daemon Running
Signal Mapper
Container Network
I/O Signal Mapper
Container Disk I/O
Signal Mapper
Libvirt Stats
Signal Mapper
Deployed App
Signal Mapper

19. Why Use OSGi: Engineer Efficiency
App Deploy
Service
Health
Signal
Publisher
Signal
Receiver
Modified Kakfa (Ajna)
Bus
Signal Receiver
Public Bundle
Ajna Plugin w/ Ajna
Signal Receiver Impl
Tim
Contract: Signal protocol +
SignalReceiver Java
interface
Lee
Peter

20. Efficiency: Test and Release
App Deploy
Service
Health
Signal
Publisher
Signal
Receiver
Modified Kakfa (Ajna)
Bus
Signal Receiver
Public Bundle
Ajna Plugin w/ Ajna
Signal Receiver Impl
Tim
1. Separate git projects each tested against contract
(Eclipse model)
2. Semantic versioning to ensure compatibility
3. Zero downtime releases
Lee
Peter

21. Why Use OSGi: Available Features
Things you (almost)
always need:
1. Dependency
injection
2. Configuration
management
3. HTTP server
4. Admin UI
Available OSGi
features:
1. Declarative
Services*
2. ConfigAdmin
3. HTTP Service (Jetty
implementation)
4. Felix WebConsole
*Includes service lifecycle management

22. What’s going on with the Salesforce Data Center Automation (DCA) initiative
Why Salesforce DCA use OSGi
OSGi Tricks, Tips and Lessons Learned
Demonstration of installing plugins with zero downtime
Answer your questions
Real people use OSGi in the real world
Why Am I Giving This Talk?

23. OSGi Tip #1: Use BND
Bundle = +
Metadata Files
BND auto-magically generates your bundle
metadata files
Java Files Pom Files
BND
+ =
Metadata Files
+ + +
Other available
integrations. See
http://www.aqute.
biz/Bnd/Download
Metadata Files
by hand = RAGE!

24. OSGi Tip #2: Platform vs. App Bundles
Platform Bundles: Outside runtime
dependencies for your Application
Order of Operations:
1. Install platform bundles
2. Install app bundles
3. Start platform bundles
4. Start app bundles
App Bundles: The contents of your
Application

25. OSGi Tip #3: Trust the Framework
Platform Bundles: Outside runtime
dependencies for your Application
“In general trying to control start order is a disaster in the
long run.”
- Peter Kriens*
“...the correct approach is to always install ALL of the
bundles before starting ANY of them. If you do this then the
order of installation does not matter.”
- Neil Bartlett**
App Bundles: The contents of your
Application
*Source: http://stackoverflow.com/questions/19088373/how-can-i-assign-start-level-of-osgi-
modules
**Source: http://stackoverflow.com/questions/17443500/fuse-esb-fabric-feature-bundle-load-order

26. OSGi Tip #4: Use Felix WebConsole
Installed
Bundles!!
Running
Services!!
More Information:
http://felix.apache.org/site/apache-felix-web-console.html

27. What’s going on with the Salesforce Data Center Automation (DCA) initiative
Why Salesforce DCA use OSGi
OSGi Tricks, Tips and Lessons Learned
Demonstration of installing plugins with zero downtime
Answer your questions
Real people use OSGi in the real world
Why Am I Giving This Talk?

28. Demo Bundle
Demo: Installing Signal Mappers
Health
Demo1 Signal
Source
Redis Signal
Receiver
Redis Metrics Bus
Demo2 Signal
Source Signal
Mapper
Signal Mapper
Signal Mapper
Registry
Demo2
Signal Mapper
Demo1
Signal Mapper
1. Without the proper
mappers these signal
sources mean nothing
2. Install the demo
bundle into the Health
server
3. Activate the demo
bundle which registers
the signal mappers
4. Signals can now be
interpreted!!!!!
No magic. Only native
OSGi components:
0. Equinox framework
1. Felix ConfigAdmin
2. Felix FileInstall
3. Felix Jetty HTTP
Service
4. Felix Metatype
5. Felix SCR
6. Felix WebConsole

29. What’s going on with the Salesforce Data Center Automation (DCA) initiative
Why Salesforce DCA use OSGi
OSGi Tricks, Tips and Lessons Learned
Demonstration of installing plugins with zero downtime
Answer your questions
Real people use OSGi in the real world
Why Am I Giving This Talk?

30. Thank you