March 29, 2012 lecture; CS 354 Computer Graphics; University of Texas at Austin

1. CS 354
Interaction &
Project 2
Mark Kilgard
University of Texas
March 29, 2012

2. CS 354 2
Today’s material
 In-class quiz
 On compression lecture
 Lecture topic
 Project 2
 Interaction

3. CS 354 3
My Office Hours
 Tuesday, before class
 Painter (PAI) 5.35
 8:45 a.m. to 9:15
 Thursday, after class
 ACE 6.302
 11:00 a.m. to 12
 Randy’s office hours
 Monday & Wednesday
 11 a.m. to 12:00
 Painter (PAI) 5.33

4. CS 354 4
Last time, this time
 Last lecture, we discussed
 Project 2 on Programmable Shaders
 Compression for Computer Graphics
 This lecture
 Project 2 discussion
 Get started now
 Interaction

5. CS 354 5
On a sheet of paper
Daily Quiz • Write your EID, name, and date
• Write #1, #2, #3, #4 followed by its answer
 True or False: A  True or False: Run-
hardware texture length encoding works
compression scheme well for digital
must be capable of photographs.
random access
decompression.  If the surface normals in
a mesh are each stored
 True or False: Floating as an index into a set of
point values are required 256 quantized normals
to have at least 23 bits rather than as three 32-
devoted to the mantissa bit floating point
and 8 bits for the numbers, what
exponent. compression ratio is
achieved over the
floating-point
representation?

6. CS 354 6
Project 2
 Project is NOW available
 PDF + code to use
 Requirements
 Mostly writing GLSL shaders
 Small amount of C/C++ code
 Convert height map to normal map
 Implement NormalMap::computeNormal method
 Intermediate milestone
 Monday, April 2nd
 Submit snapshots and GLSL code for first two shaders
 Complete project
 Due Friday, April 6th
 All 9 shaders
 Embellish for additional credit

7. CS 354 7
By April 2
 Have these first two shaders tasks
implemented
Task 1: apply decal
Task 0: roll torus

8. CS 354 8
Procedurally Generating a
Torus from a 2D Grid
2D grid over (s,t)∈[0,1]
Tessellated torus

9. CS 354 9
GLSL Standard Library Routines
You’ll Need for Project 2
 texture2D—accesses a 2D texture through a sampler2D and a 2-component
texture coordinate set (s,t)
 textureCube—access a cube map with a samplerCube and a 3-component
texture coordinate set (s,t,r)
 normalize—normalizes a vector
 cross—computes a cross product of 2 vectors
 dot—computes a dot (inner) product of 2 vectors
 max—compute the maximum of two values
 reflect—compute a reflection vector given an incident vector and a normal
vector
 vec3—constructor for 3-component vector from scalars
 mat3—constructor for 3x3 matrix from column vectors
 *—matrix-by-vector or vector-by-matrix multiplication
 sin—sine trigonometric function
 cos—cosine trigonometric function
 pow—raise a number to a power, exponentiation (hint: specular)

10. CS 354 10
Interaction with
Computer Graphics
 Graphics isn’t just about pretty pictures
 It’s also about interacting with pictures
SketchPad
1963
Ivan Sutherland

11. CS 354 11
Event Models
 Polling
 “while loop” reads events & processes them
 Good for events arriving at a regular rate
 Appropriate when events aren’t “digital”
 May require sampling analog signals
 Event-driven
 Structured to avoid “busy waiting”
 Callback-driven programming
 GLUT’s event loop is callback-driven

12. CS 354 12
Physical Events vs.
Logical Events
 Physical events  Logical events
 Real-world input  Examples
 Examples  Key press delivered to
a particular window the
 Key presses
mouse cusor is in
 Physical link layer for  Byte stream from a
networking
network socket
 Charged-coupled  Mouse clicked on an
device (CCD) captures
object in a 3D scene
image
 May require signal
 Higher-level
processing to make
into a digital event
 Prone to noise

13. CS 354 13
Routing Events
 What entity is interested in an event?
 How is the event routed to the interested
entity?
 Example: window system events
 Mouse events generated by mouse
 Button clicks and movement must go to the
correct window region
 Window regions arranged hierarchically
 Two or more events may be connected
 The press & release of a mouse must be paired

14. CS 354 14
Model-View-Controller
 Software architecture approach
 Separates concerns
Model
updates modifies
Data and logic
View Controller
modifies
Interface / display User input
user responds to view with input

15. CS 354 15
Interactivity
 Interaction Latency
 Time from visual display to user action to
displayed response to user interaction
 Low latency provides illusion of continuous
interaction
 Graphics device display frames
 Roughly 60 frames/second provides
continuous interaction
 Roughly 16.6 milliseconds
 Display devices tuned to this

16. CS 354 16
Frame Buffering
 Problem with just one framebuffer
 Updating the buffer you are displaying results in flicker and
tearing
 One solution
 Timing framebuffer updates to not be updating pixels about to be
scanned out
 Known as “racing the beam”
 Solution: Two buffers
 One to display NOW
 One to render NOW so to display NEXT
 Ping-pong between the two buffers
 Alternatively, copy “draw” buffer to “display” buffer every frame
 Either approach needs synchornization

17. CS 354 17
Synchronized Buffer Swaps
 Stall generate next frame until displaying
the last frame
16.6 milliseconds

18. CS 354 18
Unsynchronized Buffers Swaps
 More frame rendered and lower latency to display
 But tearing artifacts on displays
 And some frames rendered but never displayed
 Wasted work
16.6 milliseconds
tearing artifacts

19. CS 354 19
Triple Buffering
 Expensive Compromise
 Needs more memory, now 3 buffers
 Renders frames that may never be displayed
 But avoids tearing while minimizing latency
16.6 milliseconds

20. CS 354 20
Complex Interactions
 Dragging objects
 Cut-and-paste
 Pop-up and pull-down menus
 Selecting an object within a 3D scene

21. CS 354 21
State Machines to Managed
Complex Interactions
 Think of on-screen “button”
 You can click with the mouse
 How does it really logically work?

22. CS 354 22
Manipulators and Choosers
Open Inventor

23. CS 354 23
Intuitive Interactions
 Make computer behave in a way that’s natural to
humans
 NOT the other way around
 Good rules of thumb
 Hint what the result of an interaction will be
 Locate highlight buttons
 Change cursor shape based on its position
 Provide immediate and continuous display of state
 Show mercy, people make mistakes, be able to undo
 Make manipulation as direct as possible
 Examples
 Dragging and dropping objects
 Pointing to indicate interest

24. CS 354 24
Back Projection
 User clicks on screen…
 Question: What object got “clicked”?
 Known as “picking” in 3D applications
 Basic operation in 3D applications
 Process
 Window system maps click to (x,y) in a particular window
 Delivers the event to application selecting events on the window
 Applications gets (x,y) in window space
 Walk through scene graph
 Invert viewport, projection, and modelview transforms
 Back project point to object space ray
 Intersect the ray with each object in scene graph
 Does the ray intersect the object?
 What intersected object is the closest?
 Return the nearest object intersecting the ray as a “hit”

25. CS 354 25
Conceptual Vertex Transformation
glVertex*
object-space coordinates Modelview eye-space coordinates User-defined
API (xo,yo,zo,wo) matrix (xe,ye,ze,we) clip planes
commands
(xe,ye,ze,we)
clipped eye-space coordinates
(xe,ye,ze,we)
clip-space clipped clip-space
Projection coordinates View-frustum coordinates Perspective
matrix (xc,yc,zc,wc) clip planes (xc,yc,zc,wc) division
normalized device coordinates (NDC)
(xn,yn,zn,1/wc)
window-space
Viewport + Depth Range coordinates to primitive
transformation (xw,yw,zw,1/wc) rasterization

26. CS 354 26
Reversed Vertex Transformation
glVertex*
object-space coordinates Modelview eye-space coordinates User-defined
API (xo,yo,zo,wo) matrix (xe,ye,ze,we) clip planes
commands
(xe,ye,ze,we)
clipped eye-space coordinates
(xe,ye,ze,we)
clip-space clipped clip-space
Projection coordinates View-frustum coordinates Perspective
matrix (xc,yc,zc,wc) clip planes (xc,yc,zc,wc) division
normalized device coordinates (NDC)
(xn,yn,zn,1/wc)
(x,y) mouse event
window-space
Viewport + Depth Range coordinates to primitive
transformation (xw,yw,zw,1/wc) rasterization

27. CS 354 27
New Current
Interaction Paradigms
 Multi-touch screens
 Phones, tablets… work surfaces?
Perceptive Pixel

28. CS 354 28
Where next?
 Visual, tactile, gestures, speech, highly
responsive, emotionally aware
 Minority Report interface…

29. CS 354 29
Next Class
 Next lecture
 Procedural methods
 Making shape, appearance, and animation from
programs—instead of data
 Project 2
 Start learning GLSL and doing project
 Due Friday, April 6th
 Incremental deadline on April 2nd
 First two tasks