3. IBM’s statements regarding its plans, directions, and intent are subject to change or
withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product
direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a
commitment, promise, or legal obligation to deliver any material, code or functionality.
Information about potential future products may not be incorporated into any contract. The
development, release, and timing of any future features or functionality described for our
products remains at our sole discretion.
3
5. Serdar Basegmez
IBM Collaboration Solutions Champion (2011-2013)
Owner of Developi Information Systems (Istanbul, Turkey)
Founder and Co-leader of LUGTR – Turkish Lotus User Group
Bilingual Blogger at LotusNotus.com (Turkish/English)
OpenNTF Guy in Turkey
– Contributor in XSnippets and CollaborationToday.info
– Member Director at OpenNTF Board
IBM Notes/Domino Design Partner
Writer at The View Magazine
Expertise on XPages, IBM Domino, IBM Connections, etc.
5
8. What’s all the fuss about DOTS about?
XPages is great!
– We all know it...
But...
– XPages are based on interaction.
– Users demand, they reply back.
What if…
– Processing takes too long?
– You need housekeeping?
– You have scheduled reports?
– You need to send reminders?
– Your application needs batch processing?
– You require unattended tasks?
You have Agents, right?
– We have a much better idea...
• Much better idea…
8
9. Some Basics
XPages ≈ Java
IBM Domino
– Java Application Server (almost)
– JSF Container
– OSGi platform
Positioning Java on XPages
– For classical Domino developers:
• Formula SSJS
• Lotusscript Java (Truth hurts!)
– Performance
– Reusability
– Modularity
– Separation of Powers (UI-Data-Logic)
OSGi is the key technology!
9
10. What is OSGi?
Open Services Gateway Initiative
– OSGi™ Alliance (IBM involved)
– Developed first in 1999
– Continues to be improved
Module System and Service Platform
– OSGi Bundles (Applications/components)
– Bundles can be remotely...
• Installed / Uninstalled
• Started / Stopped
• Updated
– No reboot required for operation!
10
11. OSGi within IBM Notes®/Domino®...
IBM Domino Server 8.5.2 9+
– OSGi introduced in HTTP task
– XPages runtime as OSGi plugin
– Extension Points
IBM Domino Server 8.5.3+
– OSGi support extended to;
• XPages Extension Library, 8.5.3
• REST APIs,
• Social Enabler
• DOTS (Domino OSGi Tasklet Service)
• Domino Servlet Container
• OpenSocial Container 8.5.2
• ...
11
13. Meet DOTS – your new best friend
Domino OSGi Tasklet Services: Next Generation Agents for Domino
DOTS provides ability to develop and run OSGi level server Tasklets for Domino
– Easy deployment.
– Tooling with Eclipse IDE and its Java and plugins editors and Java debugger.
– Leverage existing OSGi assets (within the company or open source).
– Tasklets can be run:
• Manually (local and remote calls)
• Scheduled or on server start
• Triggered (Hooks through data events) Not supported on Domino 9 SE Public Beta!
OpenNTF project by IBM since April, 2011 (replaced JAVADDIN project)
Included in IBM Domino 9 Social Edition Public Beta
13
14. What is a Tasklet?
Component to process a step in your business logic
– Lightweight addin task.
– DOTS Tasklets are simply Java methods within OSGi bundles.
– Declared as OSGi extension point
Tasklet is functionally less capable than a Server Task
Tasks DOTS Tasklets
Loaded once, run continuously Loaded once, run when needed
Manages own threads DOTS manages threading
Should be native (or encapsulated) Runs inside DOTS container
Unmanaged security Managed security
Hard to develop Easy to develop
14
15. DOTS Service Architecture
Source: Lotusphere 2012 – SHOW112 session by David Taieb, Paul Fiore, Elizabeth Sawyer
15
16. DOTS vs. Agents
DOTS Tasklets outperforms over Java Agents!
– Why?
16
17. DOTS vs. Agents
DOTS Tasklets outperforms over Java Agents!
AMGR Prepare
JVM Load Agent
Java Agent: launches a
Thread
Initializes
Java and JNI
interfaces
Bytecode
Run!
DOTS tasklet: Everything is ready to run Run!
> tell amgr run "testXPagescrash.nsf" 'LongJobAgent'
09.11.2012 19:38:39 JVM: Java Virtual Machine initialized.
09.11.2012 19:38:39 AMgr: Start executing agent 'LongJobAgent' in 'testXPagescrash.nsf'
09.11.2012 19:38:39 Agent Manager: Agent printing: 181349
09.11.2012 19:41:02 Agent Manager: Agent printing: 2227
09.11.2012 19:41:02 Agent Manager: Agent printing: Finished in 143 secs... -
09.11.2012 19:41:02 AMgr: Agent 'LongJobAgent' in 'testXPagescrash.nsf' completed execution
> load dots
> Listening for transport dt_socket at address: 8001
09.11.2012 19:42:40 Domino OSGi Tasklet Container started ( profile DOTS )
> 181349
> 2227
09.11.2012 19:43:22 [DOTS] (annotated) Finished in 41 secs...
17
18. DOTS vs. Agents
Amgr is old and complicated
– Security model problems
– Limited functionality (like issues on class loading, reflections)
Range
– Agents are database-dependent
– DOTS runs server-wide
More flexible scheduling
– Precise timings like 45 seconds
– Programmatic control over scheduling
DOTS provides wide option of database hooks Not supported on Domino 9 SE Public Beta!
18
19. Examples on DOTS
IBM SmartCloud™ for Social Business (a.k.a. LotusLive)
– Some part of back-end processing has been implemented using DOTS
CollaborationToday.info
– Feed Aggregation is implemented using DOTS
Other possible uses...
– Workflow Engine implementations
– Social Media Monitoring
– Database Monitoring
– Application Architecture Assessment
19
21. Preparing Runtime Environment
DOTS should be installed into IBM Domino server
– IBM Domino 9.0 Social Edition Public Beta does not need installation!
– For IBM Domino 8.5.2 or 8.5.3;
• Download DOTS package from OpenNTF
• You will also need some Eclipse plugins from Eclipse 3.6.2+
21
22. Preparing Runtime Environment
Prepare directory structure under Domino folder
– [Domino Program Files]osgi-dotsrcpeclipseplugins
– [Domino Program Files]osgi-dotssharedeclipseplugins
These plugins will be
copied from Eclipse
installation
[Eclipse]plugins
22
23. Preparing Runtime Environment
DOTS package contains executable files and Java libraries
– Copy JAR files into folders you have created
– Copy executable and libraries into Domino program folder
Copy to
[Domino]osgi-dotssharedeclipseplugins
Copy to
[Domino]osgi-dots
23
24. Preparing Development Environment
DOTS Tasklets consist of methods in Java classes as OSGi bundles.
Eclipse IDE is needed for development and debugging.
– In theory, Domino Designer can be used with lots of unsupported configuration changes.
Configuration is easy.
– You will see steps for setting up Eclipse to develop OSGi plugins for Domino
environment.
• Eclipse (Development Environment) OSGi on Domino (Runtime)
– Resulting Eclipse environment is useful to develop...
• DOTS
• XPages extensions
• Custom validators, data sources, converters, etc.
• Servlets
• J2EE applications for Domino
24
25. Preparing Development Environment
Download and extract Eclipse IDE...
– http://www.eclipse.org/downloads
– Eclipse 3.6.2 (Eclipse IDE for Java EE Developers) needed at least.
Download and extract necessary OpenNTF project releases
– Domino OSGi Tasklet Service : http://tiny.cc/DOTS
– Domino Debug Plug-in from XPages SDK for Eclipse RCP: http://tiny.cc/XPagesSDK
Prepare IBM Domino Server 8.5.2+
– DOTS should be installed properly.
– A local IBM Domino Server is strongly recommended.
– Use 32-bit for Eclipse and Domino Server for development/testing
25
26. Development Environment Basics
Target Platform:
– Simulate the runtime environment (e.g. JRE settings, classpaths, plug-ins...)
Notes Java API:
– DOTS Tasklets use Notes objects (e.g. Session).
– Needed for development only.
OSGi Framework
– Comes with the Debug Plug-in.
– Contains information about OSGi runtime (e.g. workspace)
33. Configuring Development Environment
Configure Notes Java API project
– Add all packages to the «Exported Packages»...
– Save the Manifest file (Ctrl+S or File Save)
33
34. Configuring Development Environment
Install Domino Debug Plugin from XPages SDK for Eclipse
– Help Install New Software...
– Add Local Point to the update site folder extracted from OpenNTF Package
– Accept warnings and restart Eclipse...
34
36. You are Ready!
You are ready to develop your first Tasklet...
Here are some resources for you!
– Lotusphere 2012 – Show 112 Session by David Taieb, Paul Fiore and Elizabeth Sawyer
• Available on Greenhouse – Lotusphere Community
– YouTube: OSGi Tasklet Service for IBM Lotus Domino by Niklas Heidloff
• http://youtu.be/CRuGeKkddVI
– NotesIn9 Episode #93 – «Introduction to DOTS» by Serdar Basegmez
• http://notesin9.com/index.php/2012/12/04/notesin9-093-introduction-to-dots/
– XPages Portable Command Guide by Martin Donnelly, Maire Kehoe, Tony
McGuckin, Dan O'Connor
• Chapter #3: Working with the Console
36
38. Tasklet Development Basics
A Tasklet project is a Plug-in Project contains a number of Java classes
– Same project may contain lots of Tasklets.
Extension Points
– Extension point to be defined for «com.ibm.dots.task»
Imported Packages
– Notes Java API to be linked for compile-time.
OSGi Runtime Configuration
– OSGi on Domino will start bundles from its own repository.
– A link between Eclipse workspace and OSGi Framework is needed.
• Eclipse will tell DOTS that «I have bundles to install!»
Debug Configuration
– Eclipse needs to know how it will connect to the target JVM for debug
38
39. Developing your first Tasklet
Create a new Plug-in Project
– File New Plug-in Project
39
40. Developing your first Tasklet
Import DOTS extensions
– Open Manifest file and click extensions
40
41. Developing your first Tasklet
Import DOTS extensions
– Deselect «Show only extension points from the required plug-ins»
– Find & pick «com.ibm.dots.task» extension from the list
– Correct the warning in the Manifest.mf tab, then Save & Close Manifest view.
41
42. Developing your first Tasklet
Import «lotus.domino» package into your project
42
S
46. Developing your first Tasklet
We will define how it will run!
– Open Plugin.xml (Plug-in Manifest) from the Package Explorer...
– These markups declare your Tasklet
Name of your class
An id for your tasklet
46
47. Running your first Tasklet
So far, we have created a Java class, written some code and Eclipse compiled.
DOTS task running on server will load and run our tasklet, but how?
– Normal way, we should deploy the plugin into server.
– For development/testing, we will tell DOTS to load my bundle directly from Eclipse PDE.
There are two ways:
– Run only,
– Run and Debug
47
49. Running your first Tasklet
Domino OSGi PDE Configuration should be made for the first time
Eclipse created a special file in DOTS workspace
– So DOTS task will load bundles from our run configuration...
49
50. Running your first Tasklet
Now, you can launch DOTS task in the server
– Also check for tasklist to see that our Tasklet has been loaded.
– ...and run!
50
51. Debugging your first Tasklet
For debugging, Eclipse has to connect to DOTS task.
– Eclipse will know what runs on DOTS
More settings needed now...
– Close DOTS task
• «tell dots quit»
– Set Notes.ini parameters for debugging DOTS:
• «set config DOTS_DEBUGADDRESS=8001» Security Alert!
These settings create a security breach
• «set config DOTS_DEBUGSUSPEND=y» and should be used with caution...
– Reload DOTS task
• «load dots»
• DOTS task will not start until Debugger connects.
51
52. Debugging your first Tasklet
Server is listening. Now, Eclipse has to know where to connect
– Create a new Debug Configuration
– Use the port we have set before!
52
53. Debugging your first Tasklet
You can now set a breakpoint and start debugging.
– Eclipse will switch to debug perspective as DOTS reach your breakpoint.
– Your changes will be applied immediately!
53
54. A couple of tips on Debugging & Testing...
Suspending DOTS...
– «set config DOTS_DEBUGSUSPEND=y»
– This setting will prevent DOTS task starting until a debugger connects.
– Suspending DOTS task would be useful to debug scheduled Tasklets or those which
starts on server start. But if your debugger cannot connect for some reason, server will
be unstable!
You can also enable debugging for DOTS temporarily...
– load dots -debugaddress=<port> -debugsuspend=<y/n>
Eclipse utilizes ‘hot-replace’ feature of JVM
– In Debug mode, we can change the source code and bundle will be refreshed upon
compiling!
54
55. A couple of tips on Debugging & Testing...
Refresh bundles...
– Without debug, there will be no «hot-replace».
• Meaning: Recompiling classes will have no effect until DOTS task restarted.
– OSGi allows you to refresh bundles without restart, here is how:
• Find out the «bundle-id» for the plug-in
• Refresh the bundle
– Doesn’t work for scheduled tasklets.
55
56. Scheduling Tasklets
DOTS looks for extension point configuration in «plugin.xml» to find out how to
run tasklets.
<plugin>
<extension
point="com.ibm.dots.task">
<task
class="com.ic13.samples.helloworld.HelloWorld" Tasklet class
id="helloworld"> Tasket id
<run every="5" unit="second" /> Add this!
</task>
</extension>
</plugin>
56
57. Magic Word: Annotation
So far, we have declared our tasklets with an XML file.
– A Java class «doRun» method A Tasklet
DOTS provides annotated declarations within Java classes
– Remember, you still need to create an extension point in «plugin.xml»
<plugin>
<extension point="com.ibm.dots.task">
<task class="com.ic13.samples.helloworld.AnnotatedTasklet" id="someid" />
</extension>
</plugin>
@RunOnStart
public void runOnStart( IProgressMonitor monitor ){
logMessage("Annotated onStart method");
}
@Run( id="manual")
public void runManual( String[] args, IProgressMonitor monitor ){
logMessage("Annotated run method with id=manual");
}
@RunOnStart
@RunEvery( every=60, unit=RunUnit.second )
@HungPossibleAfter( timeInMinutes=1 )
public void runEvery60seconds( IProgressMonitor monitor ){
logMessage( "Called from annotated method every 60 seconds");
}
57
59. Deploying Tasklets - Basics
What we develop in Eclipse IDE is a «Plug-in»
Plug-ins can be deployed into OSGi by...
– Referencing PDE tool to the Eclipse Run Configuration
• For testing & development... We have seen this...
– Putting JAR file into OSGi plug-ins folder,
• Needs file-level connection to the server and (sometimes) begging to your Admin!
– Referencing an update site
• DOTS provides profiles!
Before deploying Tasklets, make sure:
– Delete PDE runtime configuration (pde.launch.ini)
– Disable suspend feature in debugging (DOTS_DEBUGSUSPEND)
59
60. Deploying Tasklets as Plug-in
Export Plug-in to a JAR file
– Right Click Export Deployable plug-ins and fragments...
60
61. Deploying Tasklets as Plug-in
Select your plug-in project and a destination
Place exported JAR file into plugins folder, then restart DOTS task
– [Domino Data Folder]dominoworkspace-dotsapplicationseclipseplugins
61
62. DOTS Profiles
A DOTS profile is a specific DOTS container.
– Each profile defines «Tasklet Container» that has different settings for;
• Remote Controller
• Security Configuration
• Specific update sites/folders to load plug-ins
DOTS support multiple profiles for different needs.
Profiles can be stored in XML files or Notes Databases
– It can be any Notes Database like names.nsf.
Every profile has a unique name (Message Queue Name – mqName)
– Profiles accept separate tell commands, like «tell profileName run XYZ»
– DOTS loaded with a default profile named as «DOTS»
62
63. Creating Profile Repository as NSF
As a best practice, store DOTS Profiles in a Notes database
– Create an empty database and secure it with ACL.
On Server, configure DOTS to use Profiles database
– tell dots quit
– set config OSGI_CONFIGURATION_DB=dotsprofiles.nsf
– load dots
– tell dots profileCreate dots
DOTS will create necessary design
elements
By using DOTS profiles, we can import
Tasklets from NSF-based update sites...
63
64. Update Sites and Features
Update Site is used to organize and export features.
– So they can be installed into eclipse-based platforms.
Feature is a collection of plug-ins.
– Made of plug-ins that can be installed/updated together.
Update site, therefore is a directory structure.
Plug-ins can be deployed via:
– File system
– Archive file
– HTTP/HTTPs
– Virtual file system
• e.g. Domino OSGi creates VFS for NSF databases
64
68. Creating Update Site Database
Create an empty database from Eclipse Update Site template.
Import update site we just created.
Confirm our feature and plug-in has been imported into database.
68
69. Configure Profile to use our Update Site
Open OSGi profile we have created.
– Add update site database into Sites section.
– You may also select Features you want to
include.
Restart DOTS task and check for tasklist.
When you need to update your plug-
in, load a new version to the update site
database and refresh the bundle.
69
70. Security with Profiles
You might define additional Java security to the profile.
One reminder:
– File-based update sites or plug-ins have unlimited security by default.
– NSF-based update sites have a default set of permissions. Check DOTS documentation!
70
71. Multiple Profiles
You may create more than one profiles.
– For testing
– Tasklets that you use occasionally
– Tasklets with different security needs
– Heavily loaded Tasklets
You can create a second profile from the
server console.
Second profile should be loaded separately.
71
73. Tips and Tricks on Programmability
Logging
– Try not to use «System.out.println». (No output to log.nsf)
– Instead, use «logMessage(...)» and «logException(...)»
runWhen argument helps
– If you overload doRun method (run the same class for manual and scheduled operations
at the same time), runWhen will help you to identify which.
Parameterization
– Manual tasks can get parameters through Console or Plug-in manifest (plugin.xml)
• «tell dots run XYZ param1 param2».
• «args[n]», «getNthArgument(n)» or «getKeyedArgument(key)» can be used.
– For other tasks, you have multiple options.
• «notes.ini» settings
• Argument Resolver (refer to DOTS samples)
• Profile parameterization extension via DXL (refer to DOTS readme.pdf)
73
74. Tips and Tricks on Programmability
Profile Configuration Document:
74
75. Tips and Tricks on Programmability
Life-cycle for a scheduled Tasklet
– When a tasklet runs on schedule, DOTS class loader will create an instance from your
tasklet class. This instance will not be disposed until DOTS process stops.
– This is a great advantage!
75
76. Tips and Tricks on Programmability
Multiple run-mode for the same Tasklet and different Java Instances
– When building multi-purpose tasklets,
• Accomplish tasks with more than one functions via schedules & manual calls
– Each run will create a new Java object independent from each other
• Scheduled calls persistent objects
• Manual calls will be unloaded from memory afterwards
An example: Feed Reader for CollaborationToday.info
– A managed queue for feeds to be consumed
– Lots of tasklets
• Scheduled tasklet to read the next feed for new stories every 2 minutes
• Scheduled tasklet to check if there is a new feed definition every 60 minutes
• Manual tasklet to force refresh queue (when we have added a new feed)
• Manual tasklet to force one or more feeds to be refreshed immediately
– All can be defined in a single class but will run on different Java objects!!!
76
77. Tips and Tricks on Programmability
Instantiation Problem
– You can employ singleton pattern to create a global object within bundle.
• Activator class can be used to initialize your objects...
• Be careful about synchronization.
• Domino-based objects will be recycled after each run. Don’t persist those objects!
– In Feed Reader example;
• We can design a bundle-level Queue manager,
• Initialize it on bundle start (and even save it when bundle stops).
77
78. Tips and Tricks on Programmability
Feed Reader – Singleton Approach
– Still, careful about lifecycles… NotesSession should be used carefully!
Bundle.start Initialize the Queue Bundle
Bundle.stop Save the Queue
Tasklet Class
@RunEvery( every=2, unit=RunUnit.minute )
QueueManager Sched1() qm.readNextFeed()
(singleton object) @RunEvery( every=60, unit=RunUnit.minute )
Sched2() qm.refreshQueue()
readNextFeed()
refreshQueue()
readFeed(id) @Run( id="refreshQueue" )
Manual1(args) qm.refreshQueue()
@Run( id="refreshFeed" )
Manual21(args) qm.readFeed(args[0])
78
79. Tips and Tricks on Programmability
Remote Controller
– DOTS might accept remote commands from other applications (e.g. XPages apps)
– Remote commands are accepted from localhost only.
– Listener should be configured from Profile document.
Source: DOTS Readme File
79
80. Tips and Tricks on Programmability
Progress Monitoring
– «IProgressMonitor monitor» argument passed on tasklet methods.
– Very useful for tasklets running long time.
– Progress monitor can be used to...
• Inform DOTS task manager about how much tasklet has completed.
• Aware of situation that might break execution.
Vector<?> views = db.getViews();
monitor.beginTask( "List the views", views.size() ); Init with # of steps
for ( Object view : views ){
if ( monitor.isCanceled() ){
break; Break if canceled.
}
logMessage("t" + ((View)view).getName() );
try {
Thread.sleep( 1000 );
} catch (InterruptedException e) {
e.printStackTrace();
}
monitor.worked( 1 ); One step completed
}
80
81. Problems you may encounter using DOTS
OSGi Context
– DOTS runs in a different OSGi level than HTTP.
– So there is almost no connection between DOTS and XPages
• You can start DOTS tasks from XPages, using remote controller socket
• Refer to Readme documentation for configuration and a basic example
– Some possibilities
• Communicate over Notes Document
• Using REST API to send/receive information between DOTS and XPages
Extension Manager events
– Problems with queuing
– This feature has been removed in IBM Domino SE 9.0 Public Beta
Class Loading issues
– Some Open Source APIs might have issues with OSGi class loading (e.g. Rome project)
81