Recommended
More Related Content
What's hot
What's hot (20)
Similar to Realtime Per Face Texture Mapping (PTEX)
Similar to Realtime Per Face Texture Mapping (PTEX) (20)
Recently uploaded
Recently uploaded (20)
Realtime Per Face Texture Mapping (PTEX)
- 2. Per-Face Texture Mapping forPer-Face Texture Mapping for Realtime RenderingRealtime Rendering ““Realtime Ptex”Realtime Ptex”
- 3. Quick NoteQuick Note • For CUDA/Compute folks:For CUDA/Compute folks: – Ptex != PTXPtex != PTX
- 4. GoalsGoals Render native Ptex datasets in real-time on commodity hardware Remove texture seams from textured modelsRemove texture seams from textured models Remove expensive, manual model unwrap step from artRemove expensive, manual model unwrap step from art pipelinepipeline Support arbitrary resolutions on a per-face basisSupport arbitrary resolutions on a per-face basis
- 5. Video Time!Video Time! © Disney. Used with permission.
- 6. Video RecapVideo Recap Stored as 8Kx8K Rendered as 2Kx2K © Disney. Used with permission.
- 7. Model StatisticsModel Statistics • ~800 FPS on GTX 460 with no optimization~800 FPS on GTX 460 with no optimization • 278M of Color Texture Data278M of Color Texture Data • 5812 Patches5812 Patches © Disney. Used with permission.
- 8. General StepsGeneral Steps Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 9. Load ModelLoad Model Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 10. Load ModelLoad Model • Vertex DataVertex Data – Any geometry arranged as a quad-based meshAny geometry arranged as a quad-based mesh – Example: Wavefront OBJExample: Wavefront OBJ • Patch TexturePatch Texture – Power-of-two texture imagesPower-of-two texture images • Adjacency InformationAdjacency Information – 4 Neighbors of each quad patch4 Neighbors of each quad patch • Easily load with library available fromEasily load with library available from http://ptex.us/http://ptex.us/
- 11. Load Model (cont’d)Load Model (cont’d) • Texel DataTexel Data – Per face, load the largest available mipmap level from thePer face, load the largest available mipmap level from the source filessource files – In memory, place the loaded texel data into a memoryIn memory, place the loaded texel data into a memory buffer that has a fixed-size texel border regionbuffer that has a fixed-size texel border region – The borders must be big enough for the largest filter kernelThe borders must be big enough for the largest filter kernel you will support (Border size = ½ filter kernel size)you will support (Border size = ½ filter kernel size) • Bilinear (2x2 kernel): 1 pixel borderBilinear (2x2 kernel): 1 pixel border • 16x Aniso (16x16 kernel): 8 pixel border16x Aniso (16x16 kernel): 8 pixel border
- 12. Load ModelLoad Model VB: … IB:
- 13. Load ModelLoad Model VB: … IB:
- 14. Bucket and SortBucket and Sort Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 15. Bucket and SortBucket and Sort • Bucket ptex surfaces into groups by Aspect RatioBucket ptex surfaces into groups by Aspect Ratio – Each Aspect Ratio will be one bucketEach Aspect Ratio will be one bucket – 1:1 bucket, 2:1 bucket, 4:1 bucket, etc1:1 bucket, 2:1 bucket, 4:1 bucket, etc • Then, within each bucket, sort by decreasing surfaceThen, within each bucket, sort by decreasing surface size and assign IDs.size and assign IDs. – This allows us to densely pack texture arrays, avoiding “empty”This allows us to densely pack texture arrays, avoiding “empty” surfaces.surfaces.
- 16. Bucket and SortBucket and Sort Bucket 4:1 Bucket 1:1 0
- 17. Reorder Index BufferReorder Index Buffer Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 18. Reorder Index BufferReorder Index Buffer • OriginalOriginal • Post Sort, we have 2!Post Sort, we have 2! IB: IB(1:1): IB(4:1):
- 19. Generate MipmapsGenerate Mipmaps Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 20. Generate MipmapsGenerate Mipmaps • Walk through surfaces, and generate a mipmap chainWalk through surfaces, and generate a mipmap chain from native size to (MinFilterSize x MinFilterSize) forfrom native size to (MinFilterSize x MinFilterSize) for each surface.each surface. – Bilinear stops at 2x2, 8xAniso stops at 8x8Bilinear stops at 2x2, 8xAniso stops at 8x8 • Ptex data files do not guarantee complete mipmapPtex data files do not guarantee complete mipmap chains—although the library can generate all levels forchains—although the library can generate all levels for you—with pre-multiplied alpha.you—with pre-multiplied alpha. • Mipmap chains stop to allow for unique pinning valuesMipmap chains stop to allow for unique pinning values in the cornersin the corners
- 21. Generate MipmapsGenerate Mipmaps • Done for every surface, but only inside the surface—theDone for every surface, but only inside the surface—the border is not touched.border is not touched.
- 22. Fill BordersFill Borders Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 23. Fill BordersFill Borders • Copy neighbor texels into border area of this surface’sCopy neighbor texels into border area of this surface’s mip levelmip level – Match source and destination number of pixels whenMatch source and destination number of pixels when possiblepossible • Bordered textures are the heart of the logical realtimeBordered textures are the heart of the logical realtime ptex solutionptex solution • Allows 1-2 texture lookups per ptex sample requestAllows 1-2 texture lookups per ptex sample request – 1 if not performing tween-mip-level interpolation, 2 otherwise1 if not performing tween-mip-level interpolation, 2 otherwise
- 29. Pack Texture ArraysPack Texture Arrays Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 30. Texture ArraysTexture Arrays • Like 3D / Volume Textures, except:Like 3D / Volume Textures, except: – No filtering between 2D slicesNo filtering between 2D slices – Only X and Y decrease with mipmap level (Z doesn’t)Only X and Y decrease with mipmap level (Z doesn’t) – Z indexed by integer index, not [0,1]Z indexed by integer index, not [0,1] • E.g. (0.5, 0.5, 4) would be (0.5, 0.5) from the 5E.g. (0.5, 0.5, 4) would be (0.5, 0.5) from the 5thth sliceslice • API SupportAPI Support – Direct3D 10+: Texture2DArrayDirect3D 10+: Texture2DArray – OpenGL 3.0+: GL_TEXTURE_2D_ARRAYOpenGL 3.0+: GL_TEXTURE_2D_ARRAY
- 31. Pack Texture ArraysPack Texture Arrays • Copy all generated data into Texture2DArrayCopy all generated data into Texture2DArray • Each Texture2DArray represents a single mipmap levelEach Texture2DArray represents a single mipmap level – Texture2DArrays present a view of the data that is efficientTexture2DArrays present a view of the data that is efficient for GPU layoutfor GPU layout – Logical Textures cut across the same page index of everyLogical Textures cut across the same page index of every Texture2DArrayTexture2DArray
- 32. Pack Texture ArraysPack Texture Arrays 0 1 2 3 Logical Texture Layout
- 33. Pack Texture ArraysPack Texture Arrays 10x10x3 (1+8+1)x(1+8+1)x3 6x6x4 (1+4+1)x(1+4+1)x4 4x4x4 (1+2+1)x(1+2+1)x4 GPU Layout Texture2DArray gColor[0] gColor[1] gColor[2]
- 34. Pack Patch ConstantsPack Patch Constants Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 35. Pack Patch ConstantsPack Patch Constants • Each primitive has a “PatchInfo” struct:Each primitive has a “PatchInfo” struct: – TextureId – which array slice contains our dataTextureId – which array slice contains our data – TopMipLevel – the index of the top-most mipmap level for thisTopMipLevel – the index of the top-most mipmap level for this texturetexture – FlipUVs – whether or not to flip UVs, allows 1:2 and 2:1 to beFlipUVs – whether or not to flip UVs, allows 1:2 and 2:1 to be grouped into same bucketgrouped into same bucket – MaxMipLevels – Maximum mipmap level for each edgeMaxMipLevels – Maximum mipmap level for each edge
- 36. Fill BordersFill Borders Load Model Load Model RenderRender Preprocess Draw Time Bucket and Sort Bucket and Sort Generate Mipmaps Generate Mipmaps Fill Borders Fill Borders Pack Texture Arrays Pack Texture Arrays Reorder Index Buffer Reorder Index Buffer Pack Patch Constants Pack Patch Constants Red: Vertex and Index data Green: Patch Constant information Blue: Texel data Orange: Adjacency information
- 37. RenderRender • Brief Recap of D3D11 Pipe stagesBrief Recap of D3D11 Pipe stages New Direct3D 11 StagesNew Direct3D 11 Stages IAIA VS HS TSTS DS GS RASRAS PS OMOM Programmable (Shader) Fixed Function
- 38. RenderRender • In the Hull ShaderIn the Hull Shader – Store pre-expansion PrimitiveID to output control pointsStore pre-expansion PrimitiveID to output control points – This is used everywhere to determine which set of PatchThis is used everywhere to determine which set of Patch Constants are owned by the currently running thread (inConstants are owned by the currently running thread (in Domain, Geometry or Pixel Shaders)Domain, Geometry or Pixel Shaders) New Direct3D 11 StagesNew Direct3D 11 Stages IAIA VS HS TSTS DS GS RASRAS PS OMOM
- 39. RenderRender • In the Domain ShaderIn the Domain Shader – Vertices belonging to a quad meshes are evaluated with aVertices belonging to a quad meshes are evaluated with a domain location, which is (0,0)-(1,1) for each patchdomain location, which is (0,0)-(1,1) for each patch – Use this value to store our UV locationUse this value to store our UV location New Direct3D 11 StagesNew Direct3D 11 Stages IAIA VS HS TSTS DS GS RASRAS PS OMOM
- 40. RenderRender • Texture lookups in Domain or Pixel Shader are replacedTexture lookups in Domain or Pixel Shader are replaced with a “ptex” sample function.with a “ptex” sample function. – Determines which logical texture to work fromDetermines which logical texture to work from – Compute mipmap level(s) to accessCompute mipmap level(s) to access – Scale and bias computed (u,v) by mipmap sizeScale and bias computed (u,v) by mipmap size – Lookup texels, return weighted averageLookup texels, return weighted average
- 41. Texture Lookup Shader CodeTexture Lookup Shader Code • Traditional (D3D11)Traditional (D3D11) returnreturn gTxDiffuse.Sample(gTxDiffuse.Sample( gSampler,gSampler, I.fTextureUV );I.fTextureUV ); • PtexPtex returnreturn ptex( gTxDiffuse,ptex( gTxDiffuse, gSampler,gSampler, I.uPrimitiveId,I.uPrimitiveId, I.fTextureUV );I.fTextureUV ); Complex logic hidden in single function call
- 42. Questions?Questions? • jmcdonald at nvidia dot comjmcdonald at nvidia dot com • Brent dot Burley at disneyanimation dot comBrent dot Burley at disneyanimation dot com • http://ptex.us/http://ptex.us/ • http://groups.google.com/group/ptexhttp://groups.google.com/group/ptex • Or visit our studio sessionOr visit our studio session – Monday, 8 August @ 4:30 PM – 5:00 PMMonday, 8 August @ 4:30 PM – 5:00 PM – The Studio / West Building, Ballroom AThe Studio / West Building, Ballroom A
- 43. Additional ConsiderationsAdditional Considerations • Filtering across edges with differing numbers of pixelsFiltering across edges with differing numbers of pixels • Filtering across cornersFiltering across corners
- 44. Filtering across edgesFiltering across edges
- 45. Filtering across edgesFiltering across edges
- 46. Filtering across edgesFiltering across edges
- 47. Filtering across cornersFiltering across corners • Corners with valence > 4Corners with valence > 4 cannot be exactlycannot be exactly matched with amatched with a bilerp (4 samples)bilerp (4 samples) © Disney. Used with permission.
- 48. Filtering across cornersFiltering across corners • The solution is a bit more involved.The solution is a bit more involved. – First, walk the mesh and determine which corners are sharedFirst, walk the mesh and determine which corners are shared – For each shared group, determine the correct value for whenFor each shared group, determine the correct value for when we’re exactly at that corner. E.g. Simple Average.we’re exactly at that corner. E.g. Simple Average. – Then, modify every mipmap level of every surface of thatThen, modify every mipmap level of every surface of that group s.t. the shared corner has the same valuegroup s.t. the shared corner has the same value – When you’re in the corner, everyone will perform the sameWhen you’re in the corner, everyone will perform the same lookup—regardless of mipmap level—and continuity prevailslookup—regardless of mipmap level—and continuity prevails
- 49. Filtering across cornersFiltering across corners • For Realtime Ptex, we apply pinning to all corners,For Realtime Ptex, we apply pinning to all corners, regardless of valence.regardless of valence.
- 51. Questions redux?Questions redux? • jmcdonald at nvidia dot comjmcdonald at nvidia dot com • Brent dot Burley at disneyanimation dot comBrent dot Burley at disneyanimation dot com • http://ptex.us/http://ptex.us/ • http://groups.google.com/group/ptexhttp://groups.google.com/group/ptex • Or visit our studio sessionOr visit our studio session – Monday, 8 August @ 4:30 PM – 5:00 PMMonday, 8 August @ 4:30 PM – 5:00 PM – The Studio / West Building, Ballroom AThe Studio / West Building, Ballroom A
Editor's Notes
- Top Mipmap level size is color coded in Roy G. Biv ordering, followed by white->grayscale Red is 2048, Yellow is 512, Green is 256 and so on.
- Given this simple model, let’s walk through a graphical representation of the preparation. We’re going to focus on the middle surface, the one with the blue border.
- Assign the surfaces ids based on decreasing size. Because the bottom surface is a 4:1 aspect ratio, it’ll be in a different bucket than our 1:1 surfaces above the dividing line.
- For each surface, for each edge, for each mip level. Use adjacency information loaded from ptex file format
- Next, we need to fill in the borders. The surfaces with matching sizes are just copied into the border region, as shown here with the left edge
- And again with the right.
- Surfaces that are smaller than this surface have duplicated pixels pushed into the borders. However, these borders should never be used as we will later see.
- Along the bottom edge, the adjacent edge (from the neighbor) has more pixels than we do.
- Drop to the next mipmap level and copy the pixels straight across from that mipmap.
- Introduced in Direct3D 10
- For each render chunk
- Note that although not shown, all borders (including corners) would be filled out by this time.
- Red: Vertex and Index data Green: Patch Constant information Blue: Image data Orange: Adjacency information
- Determines which surface to work from Compute Mip Level(s) and weight factors Scale and bias computed uv for each mip level Lookup texels, return weighted average of two mip levels
- Corners: Filtering across corners requires a pre-process, where a pinned value for each shared corner is computed Both improvements require clamping the bottom mip level s.t. a filtered lookup can
- Given the above surfaces, which are neighbors. Both surfaces have been setup with the neighbor’s pixel data, so the data outside the red lines is within the surface, and the pixels within the red and brown lines are the border texels from the neighbors.
- When a bilerp is taken from a theoretically matching UV, the values being computed can be radically different!
- The solution is to detect that we are along an edge, and then clamp the available Mipmaps to those shared with the neighbor. This information is conveyed in the Patch Constant data.
- Pre-walk mesh, determine which corners are grouped together. For each group, determine a “correct” weighted value to represent that corner For each surface in that group, paint the corner (including a portion of the surface) the pinned value. This value will pollute a sample-sized footprint of the border and interior of the surface, in the corner. Only really problematic for displacement mapping—where the tessellator conveniently ensures exact, matching UVs for all surfaces