2. Usually elements are classiļ¬ed in a number of subclasses
which further speciļ¬es their properties. The most general
classiļ¬cation of elements include input elements, output, labels
GUI Renderer VUI Renderer
and groups.
ā¢ Input elements are elements which value can be changed
by the user. Input elements may also be seen as output-
input as they show the state before and after the user
interaction.
Meta Model
ā¢ Output elements, on the other hand, do not enable the
user to change their status, and they are used to display
static information to the user.
ā¢ Labels help modelling the tag that the rest of elements App1
App2 App3
should have, displaying the name of the user or an
icon. The use for these labels also includes modelling
of the options provided for elements that involve the user Figure 1: Layered UI architecture
making a selection out of a discrete set of options.
ā¢ Groups are used to bond a set of elements together to help
the UI renderer, and eventually the user to understand that so. The task for the UI renderer is to translate from the meta
those elements relate conceptually. model into the underlying technologyās components, and the
For each of these classes there are specializations, for in- user input back into the meta model.
stance, to display and retrieve boolean values, others specialize As most of the information is modelled as modality neutral,
in simple text, or numerical values within a speciļ¬c range. there can be many UI renderers, each one managing different
The choice of these subclasses is domain of the application. set of channels with different modalities.
Applications use the meta model by making instances of the
C. Interaction model among Applications, Meta Models and
desired container, and ļ¬lling it with instances of the elements
UI Renderers
they need to show to the user, or need to retrieve from
the user. This model is then presented to the user using a The global picture of how the whole system works is
UI renderer, when the user interacts with the renderer the presented in ļ¬gure 1. Application developers focus on the
latter will inject the information provided by the user into the whole stack using a canonical MVC approach, they do not
model. The application then receive this information through need a meta model nor an independent UI renderer. By
the mechanism the model provides, either a callback or an introducing the meta model application developers can focus
event. on the requirements of their application and ignoring details
This process forces the application developers to design UI about the user interface. The meta model also allow several
in neutral modality terms, preventing them from manipulating applications to share a common interface, and look.
the end result, and keeping them agnostic of the hardware or Applications create instances of the meta model, including
technology used in the actual presentation. the required containers with their properties, and pass them to
The use of the meta model forces also the use of MVC UI renderers. In some cases this process of transporting the
pattern, as the meta model itself represents the view part of meta model instance from the application to the UI renderer is
any application, and provides mechanisms to help the design done through another component. When this occurs normally
of the controller section. it is because the meta model is part of a platform, and
the mechanism of transportation is also a sub system of
B. UI Renderers the UI management. The case of platforms with this kind
The function of a UI renderer is to interpret the meta of UI approach usually enable different applications and UI
model into a useful form that the user understands and can renderers to run at the same time, just like shown in ļ¬gure 1.
interact with. Usually one UI renderer is associated to one
technology (and the compatible hardware) and one or more D. Nested and Hierarchical MVC
communication channels. This approach implies a MVC stacking. The applications
A UI renderer may be designed for a wide spectrum of use MVC where the view is a meta model instance and the
users, but can also be specialized to ļ¬t a set of users with UI renderer also uses MVC, but this very same instance is
special needs. For example a GUI renderer may be conļ¬gured the model. This concept is called nested MVC, as the MVC
to show bigger fonts if the user has mild visual impairment or for the UI renderer plays within the view of the MVC of the
may be purposely built for cognitive deprived users that are application (see ļ¬gure 2 a).
unable to remember short term events or can not read. This concept should not be confused with hierarchical
The containers deļ¬ned in the meta model are rendered ac- MVC (HMVC) (see ļ¬gure 2). HMVC refers to the MVC
cording to their deļ¬nition as close as the hardware, technology being applied several times at different levels. In the scope
and/or communication channel enables the UI renderer to do of this paper, HMVC is the application of MVC for meta
3. C. Maintainability
M C Maintainability is the measure of how the project can be re-
M C paired, evolve, or be adapted in the future. The most signiļ¬cant
contribution to maintainability is a clear and understandable
V V design, where the key features are immediately spotted and
developers have easy access to them.
M C D. Testability
V M C M C Testability in user interaction always implies tricky tech-
niques, as the UI by deļ¬nition needs a user to interact with.
V V Nevertheless testability means also the easiness to test indi-
vidual parts of the design independently; with the decoupling
(a) NMVC (b) HMVC of the model and view, both can be tested independently. The
proposed design also provides an easy way to test individ-
Figure 2: Diference between Nested and Hierarchical MVC
ual meta model container interpretations and even individual
elements.
model containers in top level, then grouping elements, and E. Scalability
simple elements on the bottom level. HMVC means that each Scalability means that the design is equally applicable for
component of the meta model is treated autonomously, and increased functionality. In this context increased functionality
independent from all other components of the meta model, is the extension of the meta model, with new containers
the only relationship being a hierarchical one. or elements. We propose the use of reļ¬ective properties,
applicable in the objective programming language like Java,
III. M ETHODOLOGY in order to guarantee dynamic mapping between model and
view. This way any change in the meta model is immediately
UI renderers should follow the MVC pattern, not just
applicable by either modifying an existing class or adding a
because the MVC is imposed by the usage of a meta model,
new class per new element.
but because it helps the design and clarity of the implementa-
tion. In this scope the model is represented by the incoming IV. R ESULTS
instances of the meta model, the view role is played by the
This section presents the results of the design and the
underlying rendering technology, and the controller is the UI
implementation of a UI renderer within the AAL project
rendererās logic.
universAAL. Section IV-A proposes an architectural design
The main idea behind this design is to make highly conļ¬g- which is independent from Meta model and the underlying
urable, extensible and maintainable UI renderer. technology used for generating the user interaction. Section
The advantages of the adoption of such methodology can be IV-B1 shows how we implemented this design in the univer-
classiļ¬ed in terms of extensibility, re-usability, maintainabil- sAAL platform using Java Swing as the graphical renderer.
ity, testability and scalability of the produced UI renderers.
Following, these ļ¬ve aspects are analysed separately. A. UI Renderer Design
The proposed architecture is composed of 4 main com-
A. Extensibility ponents, a Meta Model Component (MMC), a Container
Expected extensions of the renderer comprise meta model Management Unit (CMU), a View Mapper (VM) and a Main
extensions, with new containers and/or new elements, and the Hub (MH), along with view modules per container and per
conļ¬guration of the actual look and feel of the components of element, as shown in ļ¬gure 3.
the underlying technology (for example the usage of different The MMC is the module in charge of interfacing with the
colours, fonts, or layouts for GUI renderers). By separating meta model, listening to the meta model events and delegating
the model from the controller and the view, these extension the received instances to CMU.
are dealt separately in different modules of the solution. The CMU decides which container to show at each time,
according to the context or meta-data included in the meta
model. Meta-data included in the meta model can describe
B. Reusability
speciļ¬c needs, for example they can describe the priority a
Reusability is the capability of applying the same solution to container has, or the language used in the content, to the
other UI renderers, in fact not only the design can be applied emphasis of using one modality over another. The CMU
but also great part of the implementation has great potential to may implement the logic to queue a container that has less
be reused. The MVC model decouples model form view this priority than the container that is currently being displayed at
way the controller part keeps unaltered when the view part is that moment, or ļ¬lter those containers that do not match the
changed to develop a new UI renderer. userās language or the rendererās modality. The CMU is then
4. MH
L&F Extension
Meta Model
CMU VM MMC
Extension
UI Renderer
Container View Element View
Figure 3: Proposed UI renderer architecture
Meta Model
Figure 4: Extensions context
responsible for obtaining the implementation of the container
by querying the the VM to get the container view component.
The VM is a piece of software that locates the view
component for any container or element instance of the meta developers a simple, yet powerful, way to specialize the UI
model. For example when the VM is required to search for an renderer for certain impairments. For example the renderer
element or container of type T, then the viewing component will normally use fancy colours and fonts, while in presence
for that type is loaded, for instance on the basis of a naming of visual impaired users, the renderer would adopt another
convention that associates T with TView. Reļ¬ection can be template with bigger sizes, simpliļ¬ed fonts and high-contrast
used to easily implement this mechanism. colours.
The MH will act as a place holder for all common needed This design inherently allows meta model extensions and
variables, including the MMC and the CMU. This allows to L&F extensions. These extensions can be provided in separate
replace these components, when for example another meta packages and dynamically incorporated to the UI renderer (see
model is used, or when another display management is re- ļ¬gure 4). Being reļ¬ection used in VM, the new meta model
quired. components are directly mapped to the provided viewing
The container view component will make use of the VM components.
to ļ¬nd the element view component for each of itās children For instance it is possible to enrich the choice of L&F
(matching the expected behaviour of top level components of templates by adding them to the VM (e.g. simpliļ¬ed L&F,
the HMVC hierarchy). Grouping element views would do the customised colours etc.), or it is possible to enrich the meta
same technique but their hierarchical level is lower as they can model by extending it and by providing proper viewing
be āownedā by containers or other grouping elements. components for it (e.g. a viewer that plots arrays on a two
View components in general (element and container view axis graph).
components) are atomic translators for the meta model com- L&F packages are loaded by instructing the VM to load
ponents. For each meta model componentās MVC (as part of the L&F components instead of the proper view packages.
the HMVC) there would be a view component, which will As L&F components are extensions of the proper component,
allocate all the necessary instructions to do the translation into they incorporate all the needed functionality required from a
the underlying technologyās model. viewing component.
Furthermore, each view component can be decoupled into Meta model extensions can be very useful, for example
two major parts, the information translation and the look and an application may want to display graphs, which are not
feel (L&F) layers. The idea is to decouple what a component usually deļ¬ned in the original meta model; as it will be
shows from how it shows it. deļ¬ned in neutral modality and graphs only make sense in
The L&F is independent from proper viewing component graphical modalities. The way to provide this functionality
(information translation layer), and several L&F templates can is by providing meta model extensions, in some sense these
be developed for the same UI renderer. L&F templates will extensions are stackable, so they may be thought of as plugins.
take care of colours, fonts, layouts, shapes, sizes and every The proposed design automatically incorporates these plu-
displaying detail that the proper view component does not gins. When a reļ¬ection VM is used, the new meta model
take care of. The capability of providing different L&Fs is components are directly mapped to the provided viewing
something appreciated by any user, but in the case of AAL components. So the packaged meta model extension plugin
users is specially important. This mechanism offers AAL for a UI renderer just contains the viewing components for
5. the given extension.
B. UniversAAL Technological Solution
UniversAAL is an European research funded project [11],
which copes with the challenge to create a standard open
platform that makes technically feasible and economically
viable the development of personalized AAL solutions.
As part of its platform universAAL provides UI support
as a basic functionality. The main software artefacts of the
platform around which the UI management is built are the
input and output buses. In universAAl a bus is a virtual channel
of communication through which semantic rich information
travels, these buses connect pieces of software within the same
node and between nodes providing the power of distributed Figure 5: Simple test showing some of the key features of the
processing. The output bus is the mechanism through which default look and feel package.
output events are presented to the user, likewise the input bus
does the same job but inversely, taking user input events and
presenting them to the applications. to commit their mission, that is: to send all user input when
The meta model in universAAL, wrapped in these input and the submit is activated by the user.
output events, is a deļ¬nition of resources called IO RDF, even The handler had no need for a queued CMU as univer-
though itās implementation is not in actual RDF [9], but a Java sAALās DM already manages the priority.
equivalent representation. This meta model features 4 types of Look and feel is supported by means of the standard Java
forms, grouping elements, several input and output elements, Swing look and feel templates. A standard template for all
and a special kind of element called submit, which triggers the universAAL applications has been produced, although it leaves
userās input events and terminates dialogs. This meta model the possibility of extending it or implementing a new one.
is very basic, as in any AAL environments UI is not expected The visual results of the implementation produce an elegant
to be very complex due to usersā limitations. The IO RDF is and simple graphical output which can run seamlessly on dif-
based on XForms [10], and like it, it is susceptible of being ferent environments and operating systems and with different
extended. screen resolutions.
UniversAAL adds another component to the IO buses and V. C ONCLUSIONS AND FUTURE WORK
the IO RDF: the Dialog Manager (DM). The DM is a dialog This paper shows how to apply hierarchical MVC patterns
orchestrator which coordinates all output and input events to the design of AAL user interaction renderers. The proposed
coming from and to applications, playing the role of the architecture and the example of implementation provided for
CMU at system wise level. This coordination involves multi the universAAL platform prove that the use of this pattern
user coordination, multi modality synchronization, priority simpliļ¬es the effort in developing UI renderers.
management, impairment mapping, and many other important The Swing Handler has been compared with an already
administration of the whole UI. existing handler of the platform. Thanks to our design, locating
Lastly but not least important, the UI renderers, here called functionalities of each IO RDF resource representation was
UI handlers, cope with human-machine interaction and map- much easier on our implementation than the existing one.
ping to and from the IO RDF (meta model). Moreover, the Swing Handler allows the redeļ¬nition of the
1) The Swing Handler: Within this framework, we have IO RDF, extending it without hardly the need to change the
developed a UI renderer based on the well-established Java central logic of the handler. The adoption of Swing templates
Swing library [5] as graphical rendering engine, namely the allows also the capability of changing the look and feel, even
Swing Handler, to prove the applicability of the HMVC in at runtime.
UI renderers for an AAL environment. This Swing Handler Nevertheless the implementation of a single class for each
follows the design proposed in section IV-A, but specially element in the meta model was tedious work, and for large
adapted to univesAAL unique features. meta models it would require a bigger development team.
UniversAAL provides two elements to receive and publish The duality of view component and L&F template, makes that
output and input events respectively, these are the output the development of new templates is equally human resource
subscriber and the input publisher, both form the meta model demanding. But usually already created L&F packages will
component as their role is to interact directly with the IO serve as reference and source templates to develop new l&F
buses. Elements, or Form Controls as IO RDF deļ¬nes them, packages, which eases the procedure to create new L&Fs.
have their view following a HMVC paradigm. In the case of As future improvements we propose the development of
submit form controls have a main role in the control part of plugins for the Swing handler. Including L&F packages spe-
the handlerās MVC. They are connected to the input publisher cialized for speciļ¬c hardware like hand held devices or TV
6. displays. Other interesting L&F packages are AAL impairment Communication and Interaction, Human-Computer Interaction Series,
speciļ¬c like short sighted or cognitive deprived users. As far pages 114ā131. Springer London, 2008. 10.1007/978-1-84628-927-9_7.
[2] Laura Burzagli, Pier Luigi Emiliani, and Francesco Gabbanini. Design
as L&F template enables there should be a speciļ¬c template for all in action: An example of analysis and implementation. Expert
per hardware and impairment, so we propose the research on Systems with Applications, 36(2, Part 1):985 ā 994, 2009.
how to combine L&F packages. This will enable to easily [3] Annette De Vito Dabbs, Brad A. Myers, Kenneth R. Mc Curry, Jacque-
line Dunbar-Jacob, Robert P. Hawkins, Alex Begey, and Mary Amanda
develop a speciļ¬c handler, for example a new L&F template Dew. User-centered design and interactive health technologies for
could be created from combining existing templates for hand patients. CIN: Computers, Informatics, Nursing, 27, May/June 2009.
held device and mild sight impairment, in stead of creating [4] P. L. Emiliani and C. Stephanidis. Universal access to ambient
intelligence environments: opportunities and challenges for people with
the same handler from scratch. disabilities. IBM Syst. J., 44:605ā619, August 2005.
Finally, another future improvements involves the other type [5] Amy Fowler. A swing architecture overview. http://java.sun.com/
of plugin, the meta model extension. We propose the creation products/jfc/tsc/articles/architecture/.
[6] Sonja MĆ¼ller, Ingo Meyer, Ilse Bierhoff, Sarah Delaney, Andrew Six-
of GUI modal speciļ¬c features, like for example a graph and smith, and Sandra Sproll. Iterative User Involvement in Ambient Assisted
chart element which will draw a graphical representation of Living Research and Development Processes: Does It Really Make a
signal data or any type of numerical data series. Difference? 2011.
[7] AALuis project. Facilitate the connection of different services to
ACKNOWLEDGEMENTS different types of user interfaces and thus enable the future users of
ambient assisted living systems to use more services interacting in their
We would like to thank the whole universAAL Consortium preferred way. http://www.aaluis.eu/.
for their valuable contribution for the realization of this work. [8] GoldUI Project. European aal project for adaptive embedded human
interfaces designed for older people. http://www.goldui.eu/.
This work has been partially funded by European Union under [9] W3C RDF Working Group. Resource description framework a standard
the 7th Framework Program through the universAAL (FP7- model for data interchange on the web. http://www.w3.org/RDF/.
247950) research project. [10] W3C The Forms Working Group. Next generation of forms technology
for the world wide web. http://www.w3.org/MarkUp/Forms/.
R EFERENCES [11] universAAL project. Universal open platform and reference speciļ¬cation
for ambient assisted living. http://www.universaal.org/.
[1] Jan O. Borchers. A pattern approach to interaction design. In
Satinder Gill, John Karat, and Jean Vanderdonckt, editors, Cognition,