A presentation based on a game controller design for the Microsoft Surface. Goes into details about the design of the overall system, as well as the potential for testing and future development. This was presented at the NEC-HFES conference in October 2011.
2. Tangible User Interfaces
for Multi-touch Surfaces
• Why?
• How are we implementing it?
• What can this lead?
3. Why TUIs?
• Tactile response
– Buttons, switches, typing, writing, driving, …..
– Part of everyday life
• Enhanced operation
– Touch screens are not suited
for certain applications
Why bother? – Touch screens are convienent because they don’t require any other input, so why try to add another input device?How is it implemented? – TUI can range from simple stylus’ to complex micro controllers that provide more informationWhat can we learn from this? – What conclusions can we draw from experimenting with TUIVideo of TUI
“We use Tangible User Interfaces everyday, almost all of the devices we use give us a tactile response ie. Buttons, switches, keyboards, driving. We use them in almost every mechanical device we use, the turning of a handle for instance, we know that our action was carried out effectively because we can feel it being confirmed as we do it (Family Guy Picture), we enjoy using objects with tactile feedback so much, its difficult to not do it. Touchscreens however lack a tactile response, which most users can find difficult to use depending on what they are trying to do. In some cases touchscreens are great and don’t need any external hardware. However you would not like to try and create a spreadsheet, or use photoshop, or use a 3DModeling software. This prevents these programs from being truly usable on a touchscreen, although if they did have some sort of controller that could preformspecifc actions for the software it might improve the likelyhood of these products use on touchscreens.”Tactile responses are present in nearly every electronic/mechanical product we use. (Buttons, Switches, typing, writing, driving). It is around in nearly every functional thing we use, the subtle confirmation that the physical action we just did actually was detected by the deviceEnhanced operation – TUI’s offer a way to extend the functionality of touch screens, which could add practicality to touch-screen for certain applications (photoshop, 3dmodeling, work, gaming)
Testing: How are we going to be able to test a notion like this? Give users multiple interfaces and gauge their responses to the application. (Taking surveys because there is no other way to really determine enjoyment, it will vary from user to user)Intuitive and Simple: Need to provide a user experience that is as simple and just as efficient as using their hands. We can’t have them preform a difficult task because it would bias results (assuming they would get better at the task as each time they preformed it) This requires us to give them a task that is simple from the start, but provides a unique experience everytimePresenting the difference: We want the user to “care” how they preform, which would lead them to use each interface to the fullest potential, How can we present this test in a consistent and engaging manner?
Airhockey: open-ended as far as what control method is prefered, to the point that each method is comparable in usability, so the only varying factor would be the users preference. By having the user play a timed session of the game there will be a desire to preform well, so they will master the interface as quickly as possible. Simple gameplay: the gameplay is super intuitive and methods of control can be varied simply without any change in objectiveDemonstrate: By using a game we allow the user to experience a practical application of the device rather than a mundane task that faills to demonstrate a plausible day to day situation, by providing the user with an objective with multiple correct solutions it gives the user more freedom in their personal of each method.Video of games
As given by the title, our presentation concerns TUI’s for touchscreens not just touchscreen applicationsWe are using the Arduino Uno, in combination with a bluetooth, batterypack, as well an IMU5DOF:The bluetooth allows us to communicate serially through a wireless network, and keeping with the need for the device to remain wireless we have attached a battery-pack for the device to be mobile. The Imu is used for the combination of its accelerometer and gyroscope. This allows us to use one sensor to manipulate two forms of positioning
Gyroscope: serves as the sensor that allows us to use the tilt positioning, serves the same funtionality as a joystick[tilt forward move forward, tilt backward move backward] the gyroscope allows us to detect changes in rotation about the x and y axis, also known as pitch and roll. A 6DOF sensor is available with the additional z axis which will provide ‘yaw’. We call this indirect positioning because subtle physical movements can continually move the mallet for as long as its remained tilted.The accelerometer: this sensor is used in both the tilt and swipe motions, the tilt motion: it provides a downward acceleration vector so you can continue to determine how long the device is tilted, the gyroscope only provides ‘change’ where the accelerometer can provide the downward acceleration vector which provides the data of which axis has the largest effect of gravity. The swipe: this method of movement is the classic airhockey motion, we try to determine acceleration through position which proves to be difficult because of the C^2 amount of offset error. This is where we use the acceleration in combonation with the surface. (The following can probably be omitted because its covered in the surface slide)~ Everytime the controller is at rest the surface can detect its location, we then use this location as a new zero coordinate. In theory we expect that the constantly changing zero might help offset the error of the acceleration vector/ help improve the accuracy by keeping variations small.
Concurrent – allows each aspect of the program to function independently controller breaks and db gets no new info game can dynamically switch multiple controllers no waiting for blocking IO operations Matlab - create comparisons between accelerometer data and surface input to determine drift?Replay – storing positions will allow for replays, not necessarily important but a nice sideeffect of storing this information
Xna allows for quick and easy graphic development without a background in computer graphicsDesigned to run on windows which ensures compatibility with surfaceJitter is an open source physics engine, limited capabilities but plenty enough to simulate air hockeyPotential Video Spot
Name each part of the picture but don’t explain it, just stick to the general interactions maybe add some animation so you can only see the pieces weve explained
Smartphones: Built-in sensor technology, already are storing user data which could personalize control and experience. Network capable, store user information for cross platform collaboration. Proprietary Controllers: For software that is generally to difficult to use with just a touch-screen it could provide hardware specific technology to make the software more usable on multiple interfaces.Digital Wallets: Increased efficiency in daily use, provides even more functionality to handheld devices. Pull data from phone so that less user input is required to be submitted manually, Login information/user preferences/ banking and checkout information
