NEMOSHELL DEMO: Windowing System for Concurrent Applications on Multi-user Interactive Surfaces, ACM ITS 2014 (Interactive Tabletops and Surfaces), Nov.16-19, 2014, Germany Junghan Kim, Inhyeok Kim, Taehyoung Kim, and Young Ik Eom Sungkyunkwan University - Abstract Recently, the prevalence of large interactive surfaces renewed interests in windowing systems because of the advantages of enabling concurrent applications. We present the NEMOSHELL windowing system for multi-user interactive surfaces. We developed the system based on the Wayland system, a replacement for the Linux X. NEMOSHELL is designed to support multiple simultaneous applications, legacy input devices, legacy applications, and dynamic user interfaces. Finally, our demonstrations illustrate the potential of our system design. - Contact nemoux00@gmail.com www.nemoux.net

1. nemoux00@gmail.com
www.nemoux.net
ACM ITS (Interactive Tabletops and Surfaces) 2014

2. Surface computing becomes like Sci-Fi movies
Dynamic UI Legacy & WEB & Native
Performance! Reliability!
Multi-
Device
Multi-
User
Cross
Platform
X86/ARM
Security Multiple GPU/Display/Inputs
Fun User
Experience
Easy
Development
OPEN
PLATFORM
Focus on Engineering Development
Windowing system based on Linux’s Wayland
General-purpose surface computing
Scalability
Multiple GPUs / displays / input devices
Compatibility
Legacy app. / HTML5 (Chrome) / Android
Security
User execution protection
Performance
Multi-user * Multi-tasking
Ultra-high-definition / High frame rate
NEEDS of Open SW Platform

3. Multiple displays
No hardware dependency
Free display customization
Multi-display synchronization
Multiple GPUs
Hardware accelerated rendering
Buffer sharing between GPUs
Multiple input devices
Multiple touch panels
Multiple keyboards and mice
Applications
Window manager
GPU-0
GPU-1
SCREEN-0:0
App
app
SCREEN-0:1
App
SCREEN-1:0
App
SCREEN-1:1
App
App
…
Interconnection with multiple displays
and multiple GPUs on NEMOSHELL
Multiple Display Environments (MDEs)
Hardware Configuration

4. Multiple virtual keyboards
Physical keyboard emulation
Window group management
Group task management
Group user interface
Security
Secure execution for each users
based on virtualization
Dynamic user experience
Vector/animation engine (SVG)
: Isolation
: Isolation
Window manager
Container
aapppp
App
Container
aapppp
App
: Frame Buffer
app
App
App
App
App
App
A secure multi-user environment
based on the OS-level container
Multiple Virtual Keyboards

5. Legacy support
Wayland window protocol
■ Compatible with X/Wayland apps
Desktop and mobile/web
■ GTK/QT, Chrome, and Android
Language support
C/C++, JavaScript/Python/C#
Native SDK
Dynamic user interface
Touch based toolkit engine
Chromium, OpenGL, Wayland applications
QT / GTK applications
Native applications
(file browser, video player, system monitor … )

6. Open source oriented development
Based on Linux system
Using latest open-sources
Cross platform
Support X86/ARM
Distribution
LIVECD distribution on Yocto
Connectivity
Support IoT framework
NEMOSHELL: Practical software platform
NEMOSHELL on Samsung Chromebook 2 (ARM)
Photoshare with Android – Alljoyn framework

7. HTML5/ANDROID
Chrome
X applications
GTK+/QT/EFL/…
ozone-waylandxserver
NEMOTOOLWayland-backend
NEMOSHELL (window and group management, …)
NEMOCOMPZ (compositing, focus management, …)
Linux kernel, System libraries, …
Renderer
gstreamer/…
xwayland-backend
Interface
gl-renderer
(mesa/drm)
pixman-renderer
(pixman)
evdev
mtdev + tuio
GPU/displayKeyboard/mouse
Multi-touch
…
Security
Virtualization
(Docker)
OPENGL/EGL + SHM
KEYBOARD/MOUSE/TOUCH

8. 2014/Q4 ~ 2015/Q4
Crosswalk/Android
Gesture/Vision
(KINECT/MYO ...)
Ubuntu Package
NEMOSHELL Distribution
DEMO & Marketing
Commercial Support
2014 Q4
2015 Q1
2015 Q2
2015 Q32015 Q4
IPS
(Indoor Positioning System)
LIVE CD
Application Store