WIXLIB

To DoAdvanced Computer GraphicsCSE 190 [Spring 2015], Lecture 12§ Assignment 2 due May 15§ Should already be well on way.§ Contact us for difficulties etc.Ravi Ramamoorthihttp://www.cs.ucsd.edu/ ravirMotivationReal-Time Rendering§ Today, create photorealistic computer graphics§ Complex geometry, lighting, materials, shadows§ Computer-generated movies/special effects (difficultor impossible to tell real from rendered )§ Goal: interactive rendering. Critical in many apps§ Games, visualization, computer-aided design, § Until 10-15 years ago, focus on complex geometry§ CSE 168 images from rendering competition (2011)§ But algorithms are very slow (hours to days)Evolution of 3D graphics renderingInteractive 3D graphics pipeline as in OpenGL§ Earliest SGI machines (Clark 82) to today§ Most of focus on more geometry, texture mapping§ Some tweaks for realism (shadow mapping, accum. buffer)§ Chasm between interactivity, realismOffline 3D Graphics RenderingRay tracing, radiosity, photon mapping§ High realism (global illum, shadows, refraction, lighting,.)§ But historically very slow techniques“So, while you and your children’s children are waiting for ray tracing to takeover the world, what do you do in the meantime?” Real-Time RenderingSGI Reality Engine 93(Kurt Akeley)Pictures courtesy Henrik Wann Jensen1

New Trend: Acquired Data15 years ago§ Image-Based Rendering: Real/precomputed images as input§ High quality rendering: ray tracing, global illumination§ Little change in CSE 168 syllabus, from 2003 to today§ Also, acquire geometry, lighting, materials from real world§ Easy to obtain or precompute lots of high quality data. Buthow do we represent and reuse this for (real-time) rendering?§ Real-Time rendering: Interactive 3D geometry with simpletexture mapping, fake shadows (OpenGL, DirectX)§ Complex environment lighting, real materials (velvet, satin,paints), soft shadows, caustics often omitted in both§ Realism, interactivity at cross purposesToday: Real-Time Game RenderingsToday§ Vast increase in CPU power, modern instrs (SSE, Multi-Core)§ Real-time raytracing techniques are possible (even onhardware: NVIDIA Optix)Unreal Engine 4https://www.youtube.com/watch?v gtHamLNPXyk#t 33§ 4th generation of graphics hardware is programmable§ (First 3 gens were wireframe, shaded, textured)§ Modern NVIDIA, ATI cards allow vertex, fragment shaders§ Great deal of current work on acquiring and rendering withrealistic lighting, materials [Especially at UCSD]Digital Ira: NVIDIA, USC§ Focus on quality of rendering, not quantity of polygons, textureGoals§ Overview of basic techniques for high-qualityreal-time rendering§ Survey of important concepts and ideas, but donot go into details of writing codeOutline§ Motivation and Demos§ Programmable Graphics Pipeline§ Shadow Maps§ Environment Mapping§ Some pointers to resources, others on web§ One possibility for assignment 3, will need tothink about some ideas on your own2

High quality real-time renderingHigh quality real-time rendering§ Photorealism, not just more polygons§ Photorealism, not just more polygons§ Natural lighting, materials, shadows§ Natural lighting, materials, shadowsGlass VaseInteriors by architect Frank Gehry. Note rich lighting, rangingfrom localized sources to reflections off vast sheets of glass.High quality real-time renderingGlass Star (courtesy Intel)Peacock featherReal materials diverse and not easy to represent by simpleparameteric models. Want to support measured reflectance.Today: Full Global Illumination§ Photorealism, not just more polygons§ Natural lighting, materials, shadowssmall area light, sharp shadows soft and hard shadowsAgrawala et al. 00Ng et al. 03Natural lighting creates a mix of soft diffuse and hard shadows.ApplicationsProgrammable Graphics Hardware§ Entertainment: Lighting design§ Architectural visualization§ Material design: Automobile industry§ Realistic Video games§ Electronic commerce3

Programmable Graphics HardwarePrecomputation-Based Methods§ Static geometry§ Precomputation§ Real-Time Rendering (relight all-frequency effects)NVIDIA a new dawn plications/a-new-dawn/videosRelit Images§ Involves sophisticated representations, algorithmsVideo: Real-Time RelightingNg, Ramamoorthi, Hanrahan 04Spherical Harmonic LightingInteractive RayTracingAdvantages§ Very complex scenes relatively easy (hierarchical bbox)§ Complex materials and shading for free§ Easy to add global illumination, specularities etc.Disadvantages§ Hard to access data in memory-coherent way§ Many samples for complex lighting and materials§ Global illumination possible but expensiveModern developments: Leverage power of modern CPUs,develop cache-aware, parallel implementationsAvatar 2010, based on Ramamoorthi and Hanrahan 01, Sloan lications/design-garage/videos4

Sparse Sampling, ReconstructionSparse Sampling, Reconstruction§ Same algorithm as offline Monte Carlo rendering§ But with smart sampling and filtering (current work)OutlineBasic Hardware Pipeline§ Motivation and DemosApplication§ Programmable Graphics PipelineRasterizerGeometryGPUCPU§ Shadow Maps§ Environment MappingCreate geometry, lights,materials, textures,cubemaps, as inputsGeometry or Vertex PipelineModel, ViewTransformLightingProjectionClippingTransform and lighting calcs.Apply per-vertex operations Textures, CubemapsPer-pixel (per-fragment)operationsPixel or Fragment PipelineScreenThese fixed function stages can be replaced by a general per-vertexcalculation using vertex shaders in modern programmable hardwareRasterization(scan se fixed function stages can be replaced by a general per-fragmentcalculation using fragment shaders in modern programmable hardware5

OpenGL Rendering PipelineProgrammable inModern GPUs(Vertex rize)§ User specifies vertices (vertex buffer object)Programmable inModern y§ For each vertex in parallel§ OpenGL calls user-specified vertex shader:Transform vertex (ModelView, Projection), other plified OpenGL PipelineTraditional Approach: Fixed function pipeline (state machine)New Development (2003-): Programmable pipeline§ For each primitive, OpenGL rasterizes§ Generates a fragment for each pixel the fragment covers§ For each fragment in parallel§ OpenGL calls user-specified fragment shader:Shading and lighting calculations§ OpenGL handles z-buffer depth test unless overwritten§ Modern OpenGL is “lite” basically just a rasterizer§ “Real” action in user-defined vertex, fragment shadersShading Languages§ Vertex / Fragment shading described by small programShader SetupInitializing (shader itself discussed later)§ § Written in language similar to C but with restrictions1. Create shader (Vertex and Fragment)§ Long history. Cook’s paper on Shade Trees,Renderman for offline rendering2. Compile shader§ Stanford Real-Time Shading Language, work at SGI3. Attach shader to program4. Link program§ Cg from NVIDIA, HLSL5. Use program§ GLSL directly compatible with OpenGL 2.0 (So, you canjust read the OpenGL Red Book to get started)§ Shader source is just sequence of strings§ Similar steps to compile a normal programShader Initialization CodeGLuint initshaders (GLenum type, const char *filename) {// Using GLSL shaders, OpenGL book, page 679GLuint shader glCreateShader(type) ;Linking Shader ProgramGLuint initprogram (GLuint vertexshader, GLuint fragmentshader){GLint compiled ;GLint linked ;string str textFileRead (filename) ;glAttachShader(program, vertexshader) ;GLchar * cstr new GLchar[str.size() 1] ;glAttachShader(program, fragmentshader) ;const GLchar * cstr2 cstr ; // Weirdness to get a const charglLinkProgram(program) ;strcpy(cstr,str.c str()) ;glGetProgramiv(program, GL LINK STATUS, &linked) ;glShaderSource (shader, 1, &cstr2, NULL) ;if (linked) glUseProgram(program) ;glCompileShader (shader) ;glGetShaderiv (shader, GL COMPILE STATUS, &compiled) ;else {programerrors(program) ;if (!compiled) {shadererrors (shader) ;}}throw 3 ;return shader ;GLuint program glCreateProgram() ;}}throw 4 ;return program ;6

Cliff Lindsay web.cs.wpi.edu/ rich/courses/imgd4000-d09/lectures/gpu.pdfCliff Lindsay web.cs.wpi.edu/ ragment Shader Compute Lightingvec4 ComputeLight (const in vec3 direction, const in vec4lightcolor, const in vec3 normal, const in vec3 halfvec, constin vec4 mydiffuse, const in vec4 myspecular, const in floatmyshininess) {float nDotL dot(normal, direction);§ Motivation and Demos§ Programmable Graphics Pipelinevec4 lambert mydiffuse * lightcolor * max (nDotL, 0.0) ;§ Shadow Mapsfloat nDotH dot(normal, halfvec) ;§ Environment Mappingvec4 phong myspecular * lightcolor * pow (max(nDotH, 0.0),myshininess) ;vec4 retval lambert phong ;return retval ;}Shadow and Environment MapsCommon Real-time Shadow Techniques§ Basic methods to add realism to interactive renderingProjectedplanarshadows§ Shadow maps: image-based way hard shadows§ § § § Very old technique. Originally Williams 78Many recent (and older) extensionsWidely used even in software rendering (RenderMan)Simple alternative to raytracing for shadowsShadowvolumes§ Environment maps: image-based complex lighting§ Again, very old technique. Blinn and Newell 76§ Huge amount of recent work (some covered in course)Hybridapproaches§ Together, give most of realistic effects we want§ But cannot be easily combined!!§ See Annen 08 [real-time all-frequency shadows dynamicscenes] for one approach: convolution soft shadowsLight mapsThis slide, others courtesy Mark Kilgard7

ProblemsMostly tricks with lots of limitations§ Projected planar shadowsworks well only on flat surfaces§ Stenciled shadow volumesdetermining the shadow volume is hard work§ Light mapstotally unsuited for dynamic shadows§ In general, hard to get everything shadowing everythingPhase 1: Render from Light§ Depth image from light sourcePhase 2: Render from Eye§ Standard image (with depth) from eyeEyeShadow Mapping§ Lance Williams: Brute Force in image space(shadow maps in 1978, but other similar ideas likeZ buffer, bump mapping using textures and so on)§ Completely image-space algorithm§ no knowledge of scene’s geometry is required§ must deal with aliasing artifacts§ Well known software rendering technique§ Basic shadowing technique for Toy Story, etc.Phase 1: Render from Light§ Depth image from light sourcePhase 2 : Project to light for shadows§ Project visible points in eye view back to light sourceEye(Reprojected) depths match for light and eye. VISIBLE8

Phase 2 : Project to light for shadows§ Project visible points in eye view back to light sourceVisualizing Shadow Mapping§ A fairly complex scene with shadowsthe pointlight sourceEye(Reprojected) depths from light, eye not the same. BLOCKED!!Visualizing Shadow Mapping§ Compare with and without shadowsVisualizing Shadow Mapping§ The scene from the light’s point-of-viewFYI: from theeye’s point-of-viewagainwith shadowswithout shadowsVisualizing Shadow Mapping§ The depth buffer from the light’s point-of-viewFYI: from thelight’s point-of-viewagainVisualizing Shadow Mapping§ Projecting the depth map onto the eye’s viewFYI: depth map forlight’s point-of-viewagain9