LimeDS is a toolkit for rapidly building cloud applications using Java. It allows developers to visually or programmatically combine modular components into data flows. Components have JSON inputs and outputs and handle tasks like processing, storage, or invoking external APIs. LimeDS provides capabilities like caching, validation, security, and failure handling to components. It is lightweight, dynamically wires and rewires components, and facilitates application robustness. LimeDS will be released as open source to build a community around sharing and reusing components.
1. Rapid Cloud Application Development
with LimeDS
Wannes Kerckhove, Thomas Dupont, Bruno Volckaert
Context
The Cloud revolutionized the IT industry and led to the emergence of a rich ecosystem that
offers countless services and new opportunities to a wide array of different actors. Startup
companies and small businesses can benefit greatly from the availability of countless reliable
and affordable Infrastructure as a Service (IaaS) providers, allowing instant scaling of resources
without the need for upfront investments in expensive hardware. Enabled by operational tools
such as Docker and Juju, which facilitate the configuration, provisioning and monitoring of cloud
deployments, Platform as a Service (PaaS) offerings provide dependable and easily
manageable runtime environments for all kinds of software platforms. By leveraging PaaS
solutions, operational overhead can be reduced and development cycles can be accelerated.
Finally, Software as a Service (SaaS) has become one of the prime ways in which people
interact with software (next to mobile apps) and often provide accessible Application
Programming Interfaces (APIs) that enable interesting integrationpossibilities with other cloud
applications and services.
The quality and scope of an ecosystem is an important factor, but at the core of its success are
the applications (cfr. the landslide in the mobile industry after the release of the first iPhone and
later Android). However, developing quality cloud applications & services is not a trivial task: the
inherent distributed nature of the cloud and the additional effort required for testing are some of
the obstacles faced in this regard. At the same time, engineering teams all over the world are
still holding on to development practices and technologies that were designed before the
emergence of cloud technologies: large enterprise applications stacks that are hard to configure
and take a long time to boot are not compatible with the Cloud philosophy of rapidly deployable
instances. Middleware platforms that are difficult to extend with new communication protocols or
are having trouble accepting dynamic data due to the use of some rigorously defined format.
Libraries on the other hand are not tuned to running in a fully distributed environment and can’t
take full advantage of the power of the infrastructure.
The rise of Open Data initiatives along with the APIs provided by Cloud services such as
Google Maps, Twitter, OpenWeatherMap make it now easier than ever to create meaningful
applications that can influence the way we live our lives. Unfortunately, these sources cannot
always be relied upon because of service interruptions (Open Data is mostly provided on a
voluntarily basis, no SLAs) or because of the limits imposed by the APIs (sometimes as a
means of revenue, but often as a way to bound system load).
2. LimeDS: Wiring Applications in The Cloud
LimeDS (LIghtweight Modular Environment for Dataoriented Services) is a toolkit for rapidly
building JVMbased Cloud applications with a strong focus on developer productivity and
interoperability with other services and devices. Development started at the Internet Based
Communication Networks and Services (IBCN) research group of Ghent University with the goal
of creating a platform that can provide a fast and qualitative solution to the recurring R&D
challenges IBCN faces in collaboration with the industry.
In the last two years, LimeDS has been deployed in a public transport research project (with a
focus on providing a robust service in a harsh railway environment) and is currently being used
in several Internet of Things and Mechatronics projects as an integration and control flow engine
between the edgedevices and the Cloud. This successful track record has allowed us to
expand upon the platform and we are now looking to take on new applications domains.
Conceptually, LimeDS relies on a simple abstraction: functional components with JSON
formatted input and output are used to model modular pieces of logic ranging from basic filter or
3. processing steps, to storage operations or network service invocations. These components form
the building blocks of a LimeDS application. Specific services and functionality can then be
created by combining these building blocks in what we call Data Flows. A visual editor (see the
screenshot above) enables users to design Data Flows in a very intuitive way by drawing
connections between components and by adding small snippets of Javascript code, which is
wellsuited to perform transformations on the JSON message format. Power users can utilize
the LimeDS Java API to create Data Flows programmatically or to extend the modular platform
with e.g. additional authentication algorithms or new communication protocols.
The power of this abstraction lies in the uniformity of the components. While the Data Flow
paradigm can be used to create complex applications with extensive logical flows, the individual
components always adhere to a simple interface with JSON flowing in and out. This allows the
framework to provide generic but configurable capabilities to each of the components without
needing to know the details of its operation. Examples of such capabilities are component
caching, I/O validation, security, failure detection, load balancing, etc. Additionally, each
component can be configured to be triggered by HTTP requests, incoming messages or
scheduling events.
LimeDS is implemented on top of OSGi, a mature Java middleware framework for creating
modular software. OSGi embraces the dynamic and uncertain nature of software (Is Service X
available? Has the system loaded a compatible version of library Y?) and provides developers
with the necessary hooks to react to atruntime changes in a production environment. Powered
by OSGi, LimeDS will dynamically wire and rewire links between the components while the
software is running. If a component is malfunctioning or a connection with a remote system can
no longer be established, LimeDS is able to discover new paths in the Data Flow, allowing the
application to recover from otherwise systembreaking failures.
The headless version of LimeDS (without the visual editor) has a footprint of under 10 MB, can
be deployed on any platform with a Java 8 compatible JVM, boots in a few seconds, can run
validated, highperformance RESTful Web services and integrate with external Web APIs
outofthebox.
4. In order to highlight some strengths of the LimeDS platform, we show a concise example of a
Data Flow:
The above Java code uses the LimeDS annotation API to construct a component that
periodically fetches car park information from another component (InfoGetter), filters the
information for available spots and then uses the LimeDS EventBus to publish this new
information to enable other components to react (e.g. by notifying a user). The component that
fetches the car park information is defined as follows:
5. Note that this component has a HTTP assertion configured that will enable the framework to
monitor the availability of a remote API. If the API goes down, LimeDS will react by deactivating
the component and all components that depend on it. For the example presented here, this
implies that when the remote API fails, the FreeSpotPublisher component will no longer be
scheduled to run until the API becomes available again. Image all the boilerplate code that
would be required to implement this kind of behaviour in a standard Java application.
Additionally, the component is configured to cache its responses, reducing the load on the
remote API.
This functionality can of course also be visually constructed using the visual flow builder, which
will result in the same behaviour running on top of the LimeDS instance.
The screenshot above shows a preview of how this flow would look like in the visual builder.
This time Javascript was chosen as the implementation language of choice for the
FreeSpotPublisher component.
To conclude we would like to state that LimeDS is not a silver bullet and there are certainly
aspects of the framework where other technologies have a definite edge. However, by
combining a (visual) building block approach to development with a dynamic runtime that
facilitates application robustness and with productivity features aimed at reducing boilerplate
code, all in a lightweight package, we are confident that LimeDS can be an interesting addition
to the toolkit of any developer working in the Cloud ecosystem.
6. What’s next?
We are planning to release LimeDS under an Open Source License as we want it to be used by
both corporations and private individuals. Our ambition is to build a community around the
platform to promote the sharing and reuse of components and to ensure the quality, robustness
and scalability of the platform by incorporating community feedback.
To this end, we are developing a LimeDS repository system that will facilitate the sharing of
components through means of a Web interface and will provide the necessary mechanisms to
enable the provisioning and update management of running LimeDS instances. This system will
also be made Open Source in order to provide teams with a way to setup their own private
repositories.
To show our commitment to the platform, we will use the repository system to share a number of
components that will allow LimeDS users to easily integrate with industrystandard technologies
and APIs.
In the long run, we would like to expose the LimeDS platform as a Cloud Service (PaaS) with a
wide range of management and debugging tools, to enable users to create productionready
and publicly accessible services in a matter of minutes.
Contact
For more information about LimeDS, visit our website at http://limeds.intec.ugent.be. Feel free to
use the contact form for further enquiries.