The document provides an agenda for an Intel Ultrabook AppLab event being held on September 3rd in Berlin. The day-long event includes technical presentations on Ultrabook development, Windows UI design, touch, sensors and tools. There will also be demonstrations of Ultrabook features and networking sessions for developers. The goal is to provide Ultrabook developers with resources and information to help develop applications.
31. Using touch in a desktop application
• Touch and gestures
• Do’s and Don’t of gesture interfaces
• Demo: Use touch and gestures to select, move
and manipulate three images
• The Windows Touch API
• Using touch with XMAL
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
31
32. Adding touch to a desktop application
• Touch is not a mouse
• Make touch targets large enough
• Support standard gestures and behaviors
• Portrait vs. Landscape
• Do not use touch just for touch’s sake
• Touch should be forgiving
• Microsoft User Experience Guidelines
– msdn.microsoft.com/en-
us/library/windows/desktop/cc872774.aspx
– blogs.msdn.com/b/ie/archive/2012/04/20/guidelines-for-
building-touch-friendly-sites.aspx
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
32
33. Designing for Touch User Experience
• Design UI to include traditional laptop style and tablet
mode usages
• Space and size controls to register fuzzy touch input
– Common Controls: 23X23 pixels
– Command Controls: 40X40 pixels
• Use multiple form tabs for touch efficiency
• Consider the target size while designing your
application UI
• Use natural and intuitive controls
• Ultrabook Optimal Resolution: 1366X768
Source: MSDN
http://msdn.microsoft.com/en-us/library/windows/desktop/cc872774.aspx#guidelines
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
33
34. Designing for Touch User Experience
• Use common controls as much as possible
• Choose custom controls that support touch
• Prefer constrained controls to unconstrained
controls
• Provide default values and auto text-
completion
• Use check boxes instead of multiple selection
lists
Source: MSDN
http://msdn.microsoft.com/en-us/library/windows/desktop/cc872774.aspx#guidelines
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
34
35. Designing for Touch User Experience
• Place controls in areas better utilized for touch
• Command controls should be more easily
accessible
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
35
37. Using Windows Touch API
• Touch and gesture events are delivered via Windows Touch
messages
– Windows must register for Windows Touch input:
RegisterTouchWindow
• WM_TOUCH reports action, position and identifier
• WM_GESTURE describes the gesture via GESTUREINFO
structure
• Special interfaces to help process gesture messages
– IManipulationProcessor
– IInertiaProcessor
• API reference at msdn.microsoft.com/en-
us/library/windows/desktop/dd371406%28v=vs.85%29.aspx
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
37
38. WinNative: Receive gesture message
LRESULT CALLBACK
WndProc(…)
{
…
switch (message)
{
• Check for case WM_GESTURE:
// Call code to
WM_GESTURE message // interpret the
in WndProc // gesture
return
DecodeGesture(…);
…
}
(Source: Microsoft)
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
38
39. WinNative: Process gesture
LRESULT DecodeGesture(…)
{
• Retrieve the additional …
BOOL bResult = GetGestureInfo(…);
gesture information from BOOL bHandled = FALSE;
the GESTUREINFO if (bResult)
structure {
// now interpret the gesture
switch (gi.dwID){
case GID_ZOOM :
// Code for zooming goes here
bHandled = TRUE;
• Handle each of the break;
case GID_PAN :
possible gestures …
case GID_ROTATE :
…
case GID_TWOFINGERTAP:
…
case GID_PRESSANDTAP:
…
default:
// A gesture was not recognized
break;
} … (Source: Microsoft)
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
39
42. Using touch and XAML/.NET
• XAML provides access to WPF touch-enabled
UI components
– WPF touchs event are available in both Windows* 7 and
Windows* 8.
• Typical gesture events:
– ManipulationStarting
– ManipulationDelta
– ManipulationInertiaStarting
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
42
43. XAML: TouchControl.xaml
<UserControl x:Class="Win8Demo.TouchControl"
• Three images placed in …>
a Grid inside of a <Grid x:Name="cont" ClipToBounds="True" >
<!-- Set the IsManipulationEnabled to
TouchControl true -->
<Image x:Name="photo_1"
Source="/Assets/ultrabook1.png“
IsManipulationEnabled="True"
Width="500" />
• Each image has <Image x:Name="photo_2“
Source="/Assets/ultrabook2.png“
IsManipulationEnabled IsManipulationEnabled="True“
Width="500" Margin="125,-79,25,79"/>
<Image x:Name="photo_3“
set to true Source="/Assets/ultrabook3.png“
IsManipulationEnabled="True“
Width="500" Margin="0,-59,150,59" />
</Grid>
</UserControl>
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
43
47. XAML: Manipulation starting event
handler
• ManipulationStarting event void
handler is called at start of m_rect_ManipulationStarting
touch event (…)
{
…
• By setting Manipulation- e.ManipulationContainer =
Container to the touched this;
image all subsequent
manipulation events will be }
relative to that element
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
47
49. XAML: Manipulation delta event
handler
• ManipulationDelta void m_rect_ManipulationDelta(…)
{
event happens as user …
//Get the manipulation’s delta
moves fingers var delta = e.DeltaManipulation;
//Compute the transformed center point
Point rectCenter = new Point(rect.ActualWidth *
0.5, rect.ActualHeight * 0.5);
• Compute new image rectCenter = matrix.Transform(rectCenter);
center point, scale, and //Adjust the element’s scale, rotation and
translation
rotation using event matrix.ScaleAt(delta.Scale.X, delta.Scale.Y,
rectCenter.X, rectCenter.Y);
data matrix.RotateAt(delta.Rotation, rectCenter.X,
rectCenter.Y);
matrix.Translate(delta.Translation.X,
delta.Translation.Y);
• Set Handled to true so //Update the element’s render transformation
other handlers don’t rect.RenderTransform =
new MatrixTransform(matrix);
process same event e.Handled = true;
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
49
51. XAML: Inertia starting event handler
• Inertia actions are handled by the void m_rect_ManipulationInertiaStarting(…)
{
ManipulationInertiaStarting event //Set the manipulations inertia values
handler e.TranslationBehavior = new
InertiaTranslationBehavior()
• Define the behavior of the inertia {
by specifying InitialVelocity =
e.InitialVelocities.LinearVelocity,
• Linear velocity DesiredDeceleration =…
• Deceleration };
• Expansion velocity (used in e.ExpansionBehavior = new
InertiaExpansionBehavior()
pinch or spread) {
• Angular velocity (used in InitialVelocity =
e.InitialVelocities.ExpansionVelocity,
rotation) DesiredDeceleration = …
};
e.RotationBehavior = new
InertiaRotationBehavior()
{
InitialVelocity =
• Again, set Handled to true so other e.InitialVelocities.AngularVelocity,
handlers don’t process same event };
DesiredDeceleration = …
e.Handled = true;…
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
51
53. XAML: Handle inertia manipulation
if (e.IsInertial)
• Check if inertia is moving the {
image across the grid //Get the containing element’s size
Rect containingRect = …
(e.ManipulationContainer).RenderSize);
//Get the transformed element’s new
//bounds
Rect shapeBounds =
rect.RenderTransform.TransformBounds
(…);
//If element falls out of bounds
• If so, make sure image does not if
(!containingRect.Contains(shapeBounds))
travel outside of the grid by {
calling the Complete method to //Report boundary feedback
e.ReportBoundaryFeedback(…);
end the inertia //Stop any after inertia
e.Complete();
}
}
…
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
53
60. Agenda
• Overview of available sensors
• Using sensors in a Windows 8* application
- In C++ desktop applications, using COM interface
- In C++ or C#/.NET Windows 8* store and desktop
applications, using WinRT APIs
• Considerations on Efficiency
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
60
61. Who am I ?
Xavier Hallade (@ph0b)
Technical Marketing Engineer at Intel
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
61
62. Sensors
OVERVIEW OF AVAILABLE SENSORS
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
62
63. Available Sensors
Compass Accelerometer Gyroscope GPS Ambient Light NFC
+ with Sensor Fusion: Inclinometer, Orientation, SimpleOrientation
• Sensor Fusion : virtual sensors created by combining data
from compass, accelerometer and gyroscope
• Ambient Light, GPS, 3D Motion and orientation sensors are
all mandatory on Windows 8* Tablets and Convertibles
• Applications can subscribe to sensor events, set callbacks or
directly poll their current state.
64. Orientation and Motion
• Gyrometer returns yaw, pitch and roll velocities
• Accelerometer returns acceleration along x, y and z,
and optionally a Shake event
• Simple Orientation : returns device orientation into
one of four quadrants, face-up, or face-down
• Orientation Sensor and inclinometer both returns
the complete device orientation
– Inclinometer returns yaw, pitch, and roll angles
– Orientation Sensor returns a quaternion as well as a 3x3
orientation matrix
65. Orientation and Motion detection
same convention as on Android
and in the HTML5 specification
0.0° ≤ Yaw < 360.0°
-180.0° ≤ Pitch < 180.0°
-90.0° ≤ Roll < 90.0°
66. Ambient Light
• Can be used to adapt application interface
depending on light conditions : contrast,
font size
• Screen brightness control should be handled
by the operating system.
Lighting condition Iuminance (lux)
Pitch black 1
Very dark 10
Dark indoors 50
Dim indoors 100
Normal indoors 300
Bright indoors 1,000
Dim outdoors 5,000
Cloudy outdoors 30,000
Direct sunlight 100,000
67. Location
Returns latitude, longitude:
- using Wi-Fi if available
- using IP Address otherwise
- using GPS if available and desired accuracy is set
on High
- then it can also return Altitude, Speed and Heading
Depending on user’s choice, its use can be denied.
A Windows 8* store app has to declare the
“location” capability in its manifest.
68. Near Field Communication
• Short range (<4cm)
• Contactless
• Used between active and/or passive devices
– Active: Smartphone, Tablet, Ultrabook™…
– Passive: Tags, Smart cards…
• Small amount of data, low data transfer rate
• Easy to use, feels instantaneous (<1/10s)
-> Great user experience !
69. Near Field Communication
• Pairing
– with Bluetooth* peripherals, at the OS level
– with another running application, by establishing
a BT or Wi-Fi direct socket, at the application level
• Short data read/write on NFC tags
• Short data exchange with active devices
– Sharing URIs, contacts, positions
– Secure transactions
70. NFC messages
Protocol Pub. Sub. Message contents
Windows X X binary data.
WindowsUri X X UTF-16LE encoded string that is a URI.
use PublishUriMessage
WindowsMime X X message data of a specific mime type.
For example, "WindowsMime.image/jpeg".
You can subscribe to "WindowsMime" message
without specifying the mime type. If so, the first 256
bytes received is the ASCII mime type string.
Windows:WriteTag X Windows message to be written to a static tag.
WindowsUri:WriteTag X WindowsUri message to be written to a static tag.
WindowsMime:Write X WindowsMime message to be written to a static tag.
Tag Example, “WindowsMime:WriteTag.image/jpeg”
LaunchApp:WriteTag X Write a tag that can be used to launch a specific app.
more information on msdn.
WriteableTag X if a writeable tag is brought in to proximity, a
proximity message is received that contains an
int32 indicating the maximum writeable size.
71. NFC messages
Windows messages are fully compatible with NDEF messages,
but you can also directly manipulate them :
Protocol Pub. Sub. Message contents
NDEF X X properly formatted NDEF records.
NDEF:ext X application defined NDEF records (TNF field value of 0x04).
NDEF:MIME X properly formatted NDEF mime message (TNF field value
of 0x02). For example, "NDEF:MIME.image/jpeg".
NDEF:URI X properly formatted NDEF message of a type defined by a
URI namespace (TNF field value of 0x03). For example,
"NDEF:URI.http://contoso.com/sometype".
NDEF:wkt X properly formatted NDEF message of a type defined by the
NFC forum (TNF field value of 0x01).An example of this
type is "NDEF:wkt.U" for the well known type of URI.
NDEF:WriteTag X message data that should be written to an NFC forum
standard static tag.
NDEF:Unknown X untyped NDEF message (TNF field value of 0x05).
72. Sensors
LET’S DIVE INTO SOME CODE !
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
72
73. Sensors and Location Platform
using COM interface
FROM C++
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
73
74. Sensors and Location Platform
• Same interface as on Windows 7
– Sensor fusion available through new GUIDs
• Sensor manager controls sensors
– Use sensor manager to get to sensors
– Notifies when a sensor connects
• Sensors report data, changes in state and disconnection
• Access both via COM interfaces
– Use API to communicate to sensors
– Sensor events handled using callbacks
– API reference at msdn.microsoft.com/en-
us/library/windows/desktop/dd318953%28v=vs.85%29.aspx
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
74
75. COM: Connect to sensor manager
// Create the sensor manager
• Access to sensors is through the
sensor manager ISensorManager pSensorManager;
hr = CoCreateInstance(CLSID_SensorManager,
• Group policy settings may deny NULL, CLSCTX_INPROC_SERVER,
IID_PPV_ARGS(&pSensorManager));
access to the system sensors
if(hr == HRESULT_FROM_WIN32
(ERROR_ACCESS_DISABLED_BY_POLICY))
{
// Unable to retrieve sensor manager due
// to group policy settings.
// Alert the user.
}
(Source: Microsoft)
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
75
76. COM: Retrieve a sensor object
// Get the sensor collection
• Request sensors by category, type hr = pSensorManager->GetSensorsByCategory(
SENSOR_CATEGORY_ALL,
or ID &pSensorColl);
if(SUCCEEDED(hr))
{
ULONG ulCount = 0;
// Verify that the collection contains at
// least one sensor
hr = pSensorColl->GetCount(&ulCount);
if(SUCCEEDED(hr))
{
if(ulCount < 1)
{
• Request permission to use the wprintf_s(L"nNo sensors of the
requested category.n");
hr = E_UNEXPECTED;
sensor(s) } else
{
• Windows 8 will open a dialog // Request permissions for all sensors
// in the collection
box to ask the user hr = pSensorManager->
RequestPermissions(0, pSensorColl,
• Granting permission triggers }
FALSE);
…
the OnStateChanged event
(Source: Microsoft)
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
76
77. COM: Sensor event callback
class CMyEvents : public ISensorEvents
• Receive event notifications by
{
implementing required COM public:
interfaces
STDMETHODIMP QueryInterface(…) {…}
STDMETHODIMP_(ULONG) AddRef() {…}
• Sensor events require STDMETHODIMP_(ULONG) Release() {…}
ISensorEvents
// ISensorEvents methods. //
STDMETHODIMP OnEvent(…) {…}
• Sensor Manager events require STDMETHODIMP OnDataUpdated(…) {…}
STDMETHODIMP OnLeave(…) {…}
ISensorManagerEvents STDMETHODIMP OnStateChanged(…) {…}
…
(Source: Microsoft)
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
77
78. COM: Start receiving events
// Create an instance of the event
• To start receiving events provide // class
the sensor with a pointer to your pEventClass = new CMyEvents();
callback if(SUCCEEDED(hr))
{
• To stop receiving events call // Retrieve the pointer to the
// callback interface
SetEventSink with a parameter of hr = pEventClass->QueryInterface(
NULL IID_PPV_ARGS(&pMyEvents));
}
if(SUCCEEDED(hr))
{
// Start receiving events
hr = pSensor->
SetEventSink(pMyEvents);
}
(Source: Microsoft)
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
78
79. Ambient Light Sensor
DEMO
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
79
80. Using WinRT Sensors and
Location API
FROM C# AND C++/CX
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
80
87. C#/WPF Accelerometer Demo
• Ellipse (ball) drawn on a canvas
• Accelerometer data is used to update X/Y/Z
display and location of ball
• Data is read using
accelerometer events in
conjunction with a
timer to move the ball
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
87
88. C#/WPF NFC DEMO
• Subscribe to incoming WindowsURI messages
• Publish a WindowsURI message to any active
device
• Write that
message to any
writable tag
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
88
89. USING SENSORS EFFICIENTLY
Copyright(C) 2012 Intel Corporation. All rights reserved.
*Other brands and names are properties of their respective owners.
89
90. General Considerations
• The effects of applications on power use
• Both idle workloads and active workloads
• The rules of power management
– Computational efficiency
– Maximize idle
– Data efficiency
– Power-aware behavior
• Align, coalesce or batch activity whenever
possible
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
90
91. Regarding Sensors
• Do not set a small ReportInterval if your application doesn’t
need it
• Avoid periodic registry or disk accesses in order to let the
hard disk spin down
• Avoid status polling, statistics & background bookkeeping
• Avoid using sensors when application is not active
• Consider using accelerometer or gyroscope alone if full
orientation is not needed.
Sensors objects in Sensors API: At least one client connected
Hardware: Accelerometer Gyroscope Inclinometer Compass Device Orientation
Accelerometer On Off On On On
Gyro Off On On On On
Magnetometer Off Off On On On
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
91
92. Advanced : Context/power-aware
behavior
• Handling sleep transitions seamlessly
– Saving an restoring state
– Stopping ongoing activity prior to sleeping
– Closing open resources & disconnecting communications
• Respond/adapt to system power events
– Switching from AC to battery; low battery status
• Scale behavior based on power state
• Context awareness toolkits
– software.intel.com/en-us/mobility
– softwarecommunity.intel.com/articles/eng/1070.htm
– softwarecommunity.intel.com/articles/eng/1026.htm
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
92
93. Intel Software Development Products
Intel Power Tools
• Intel® Power Checker
– Reveals how much power your application is using
– software.intel.com/partner/app/software-
assessment/?cid=sw:ubpower004
• Energy-Efficient Software Checklist
– software.intel.com/en-us/articles/energy-
efficient-software-checklist/
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
93
94. References
• Intel® Software Network
– software.intel.com
• COM Sensor API
– msdn.microsoft.com/en-us/library/windows/desktop/dd318953(v=vs.85).aspx
• COM Location API
– msdn.microsoft.com/en-us/library/dd464636(v=VS.85).aspx
• WinRT Sensors API
– msdn.microsoft.com/en-us/library/windows/apps/br206408.aspx
• WinRT Geolocation API
– msdn.microsoft.com/en-us/library/windows/apps/windows.devices.geolocation.aspx
• WinRT ProximityAPI
– msdn.microsoft.com/en-us/library/windows/apps/windows.networking.proximity.aspx
• Detecting Ultrabook Sensors
– software.intel.com/en-us/blogs/2012/07/26/detecting-ultrabook-sensors
• Ultrabook and Tablet Windows 8* Sensors Development Guide
– software.intel.com/en-us/articles/ultrabook-and-tablet-windows-8-sensors-development-guide/
• Intel® Energy Efficient Software Guidelines
– software.intel.com/partner/assets/pdf/misc/Energy_Efficient_Software_Guidelines_v3_4_10_11.pdf
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
94
95. Summary
– Sensors are a natural way to interact with end
users
– Desktop apps can utilize them through WinRT as
well as Windows 8* Store Apps
– Utilize efficient designs and coding practices to
save battery life
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
95
96. Q&A
Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.
96