WIXLIB

REAL-TIME RAY TRACING WITH MDLIgnacio Llamas & Maksim Eisenstein, 03.21.2019 - GPU Technology Conference

A brief Introduction to MDLUsing MDL in a Monte Carlo RendererAGENDAUsing MDL in a Real-Time Ray Tracing Renderer2

A BRIEFINTRODUCTION TOMDL3

The NVIDIA Material Definition Language (MDL)A language to declaratively and procedurallydefine physically-based materials for physicallybased rendering solutions.Describes what we need to know to model howlight interacts with a materialDoes not specify how to render.That's up to the renderer4

HOW IS MDL DIFFERENTHLSL, GLSLMATERIALX SHADERXLower level, generic proceduralshading languagesFormat for network-based CG looksNot specific to materials in any waySpecific to a production pipeline and toolsNo standard xDFsOSLMDLLanguage for programmable shading inadvanced renderersEnergy-conserving BxDF, EDF and VDFMaterial closures, support a few BxDFs,VDFs, EDFs. No combinersElemental xDFs Combiner xDFs xDF graphsDeclarative and Procedural language5

REAL-TIME RENDERING ENGINE MATERIALSComparing to MDL10 models, 17 params:Lit: Diffuse GGXClear Coat: Lit 1 GGXOthers: variesGeneralization able torepresent many materials:10 elemental BSDFs3 elemental EDFs1 elemental VDF2 mixing BSDFs, EDFs, VDFs4 layering BSDFs7 shading models:Lit: Disney Diffuse GGX10 shading models:5 bsdf modifiers1 edf modifierUE4MDLUnity6

MDL MATERIAL femissionintensitybackface vdfgeometryscatteringdisplacementscattering coefficientcutout opacitynormalabsorption coefficientiorthin walled7

DEFINING A MATERIAL USING MDLMDL is a ‘C’ like language. The material and its components viewed as a structstruct material {boolthin walledmaterial surface surfacematerial surface backfacecoloriormaterial volume volumematerial geometry geometry}; struct material surface {bsdfscatteringmaterial emission emission}; bsdf(); material emission();false;material surface();material surface();color(1.0);material volume();material geometry();8

MDLELEMENTAL DISTRIBUTION onsMicrofacet models: beckmann smith ggx smith beckmann vcavities ggx vcavities ward geisler moroderDiffuse ReflectionDiffuse Transmission Glossy / MicrofacetSpecular ReflectionSpec. Refl. Transm. Measured BSDFBackscatter9 Glossy

MDLELEMENTAL DISTRIBUTION ES n10

MDLTintDISTRIBUTION FUNCTION MODIFIERSThin FilmDirectional FactorMeasured Curve Factor11

MDLDISTRIBUTION FUNCTIONS COMBINERSNormalized MixClamped MixWeighted LayerFresnel LayerCustom Curve LayerMeasured Curve Layer12

EXAMPLE: DIFFUSE GGX MDLsurface.scatteringmaterialfresnel layerbaseiorlayerdiffuse reflection bsdftint simple glossy bsdftint 13

EXAMPLE: UE4 LITMetallicGGXtint: base colornormalized mixDielectricexport material Lit(color base color color(0.8, 0.8, 0.8),float metallic 0.0,float specular 0.5,float roughness 0.2,color emissive color color(0.0, 0.0, 0.0)float opacity mask 1.0,float3 normal state::normal(),) let {float alpha roughness * roughness;float grazing refl math::max((1.0 - roughness), 0.0);bsdf dielectric component df::custom curve layer(weight: specular,normal reflectivity: 0.08,grazing reflectivity: grazing refl,layer: df::microfacet ggx vcavities bsdf(roughness u: alpha),base: df::diffuse reflection bsdf(tint: base color));bsdf metallic component df::microfacet ggx vcavities bsdf(tint: base color, roughness u: alpha);GGXtint: whitecustom curve layerDiffusetint: base colorbsdf dielectric metal mix df::normalized mix(components: df::bsdf component[](df::bsdf component(component: metallic component, weight: metallic),df::bsdf component(component: dielectric component, weight: 1.0-metallic)));}in material(surface: material surface(scattering: dielectric metal mix,emission: material emission (emission: df::diffuse edf (),intensity: emissive color)),geometry: material geometry(cutout opacity: opacity mask,normal: normal));14

USING MDL IN AMONTE CARLORENDERER15

MDL sourceMDL SDK 2019What you getResolve, parse, storeMDLSDKDatabase of contentEditorCompile ed DAGview on materialDistillSamplesDocs16

WORKING WITH A COMPILED MATERIALInspect:Examine graph structure of compiled materialCompile: Use MDL backends to generate target code for texturing functions distribution functionsDistill: Use Distiller API to convert material to a fixed material model like UE4 bake texturing functions into textures17

FROM MATERIAL TO RENDERING CODEImplementing the declarative part of the materialActual shading code for material description can be highly renderer specific Renderer may analyze declarative part of compiled material instance(in particular all BSDFs) Renderer can implement its own building blocks for all of MDL’s df module Renderer needs to wire up BSDF hierarchy and parameters within its own datastructures Renderer can “interpret” that at runtimeOr we just let the MDL SDK create code for the BSDFs18

MDL-GENERATED CODE FOR SURFACE BSDFSEssential blocks for a physically based Monte Carlo rendererbsdf init:Shared initialization for the current shading pointbsdf evaluate:Evaluation of the BSDF for a given outgoing and incoming directionbsdf sample:Importance sampling of an incoming for a given outgoing directionbsdf pdf:Probability density computation of that importance samplingedf eval:Evaluation of the EDF for a given outgoing direction19

CALLING MDL-GENERATED CODEContract 1: Renderer to MDL Shader Code Interfacevoid bsdf init(Shading state material state, inout packed tex results p);Stores 'texturing function' results in 'p'. Reuse in sample/eval/pdfvoid bsdf sample(Shading state material state, inout Packed tex results res,inout uint seed, in float3 V, inout float3 L,inout float3 bsdfOverPdf, inout float pdf);float3 bsdf eval(Shading state material state, inout Packed tex results res,in float3 V, in float3 L);float bsdf pdf(Shading state material state, inout Packed tex results res,in float3 V, in float3 L /* direction to light */ );20

EXECUTING CODE GENERATED BY MDL SDKContract 1: Renderer to MDL - Renderer-Provided Shading Statestruct Shading state material {float3normal;float3geom normal;float3position;floatanimation time;float3text coords[N];float3tangent u[N];float3tangent v[N];//////////////state::normal()state::geom normal()state::position()state::animation time()state::texture coordinate() tablestate::texture tangent u() tablestate::texture tangent v() tablefloat4x4float4x4uintworld to object;object to world;object id;// world-to-object transform matrix// object-to-world transform matrix// state::object id()uintuintarg block offset;ro data segment offset;// offset to arguments in user buffer// offset to read-only data in user buffer};21

EXECUTING CODE GENERATED BY MDL SDKContract 2: MDL to Renderer Interface.Texture and Parameter Accessfloat mdl read argblock as float(uint offs)float mdl read rodata as float(uint offs) { {return asfloat(gSceneParams.blockBuffer.Load(offs 2)); return asfloat(gSceneParams.blockBuffer.Load(offs 2));} }int mdl read argblock as int(uint offs)int mdl read rodata as int(uint offs) { {return asint(gSceneParams.blockBuffer.Load(offs 2)); return asint(gSceneParams.blockBuffer.Load(offs 2));} }uint mdl read argblock as uint(uint offs)uint mdl read rodata as uint(uint offs) { {return gSceneParams.blockBuffer.Load(offs 2); return asuint(gSceneParams.blockBuffer.Load(offs 2));} }bool mdl read argblock as bool(uint offs)bool mdl read rodata as bool(uint offs) { {uint val gSceneParams.blockBuffer.Load(offs 2);return (val & (0xff (8 * (offs & 3)))) ! 0;uint val gSceneParams.blockBuffer.Load(offs 2); return (val & (0xff (8 * (offs & 3)))) ! 0; } }22

EXECUTING CODE GENERATED BY MDL SDKContract 2: MDL to Renderer Interface.Texture and Parameter Accessstatic uint mdlArgBlockByteOffset;uint convertMdlTexIndexToInternalIndex(uint tex){uint textureDescriptorsRangeStart gBlockBuffer.Load(mdlArgBlockByteOffset 2);return{const uint texIdx convertMdlTexIndexToInternalIndex(tex);return getTextureWidth(texIdx);}uint tex height 2d(uint tex, int2 uvTile){const uint texIdx convertMdlTexIndexToInternalIndex(tex);return art tex - 1;}}bool tex isvalid(uint tex){return tex ! 0;}uint tex width 2d(uint tex, int2 uvTile)float4 tex lookup float4 2d(uint tex, float2 coord,int wrapU, int wrapV,float2 cropU, float2cropV){const uint texIdx convertMdlTexIndexToInternalIndex(tex);const int samplerIndex getSamplerIndex(wrapU, wrapV);return dex], coord, 0);}23

A SIMPLE PATH TRACER WITH MDLWith just BSDF scattering and Emission. No light samplingSimplest unidirectional path tracer just needs bsdf init, bsdf sample and edf evalMDL SDK generates all the shader code necessary for these functions.for every pixel: float3 color 0;for every sample:float3 throughput 1.0f; float3 radiance 0;float3 L generatePrimaryRay(pixel);for every bounce:Shading state material state; Packed tex results texRes;TraceRay(L, , /*out*/state);bsdf init(state, /*out*/ texRes);bsdf sample(state, texRes, seed, V, /*out*/L, /*out*/bsdfOverPdf, /*out*/pdf);float3 emission edf eval(state);radiance throughput * emission;throughput * bsdfOverPdf;if (pdf 0) break;color radiance;color / sampleCount;24

25

USING MDL IN AREAL-TIMERAY TRACINGRENDERER26

MDL FOR REAL-TIME RAY TRACINGVS. MDL FOR MONTE CARLO RENDERINGReal-time Ray Tracing has different needs:Cannot generate 100s or 1000s of samples per pixel per frameInstead: Split light paths into segments and contribution types, Generate buffers with samples for these Denoise them and combine them into a final image27

REAL-TIME RAY TRACINGSplitting Light PathsDirect Lighting from Analytical LightsIndirect Diffuse /Ambient OcclusionIndirect Specular: Reflections28

REAL-TIME RAY TRACINGDirect Lighting from Analytical Lights For single sample:න 𝑑𝜔𝑖 𝐿 𝜔𝑖 cos 𝜃𝑖 𝑓𝐵𝑅𝐷𝐹 𝐷𝑒𝑛𝑜𝑖𝑠𝑒 𝐿 𝜔𝑗ΩLinear Transform of Cosines approximation (LTC)(Heitz et al., 2016), Real Shading in UE4 (Karis 2013) Generalization for multiple samples:𝑉 𝜔𝑗 𝐿 𝜔𝑗 cos 𝜃𝑗 𝑓𝐵𝑅𝐷𝐹𝑓Ω𝑖𝐿 𝜔𝑗 cos 𝜃𝑗 𝑓𝐵𝑅𝐷𝐹𝐷𝑒𝑛𝑜𝑖𝑠𝑒 σ𝑗𝑓Ω𝑖𝐷𝑒𝑛𝑜𝑖𝑠𝑒 σ𝑗 Ω 𝑑𝜔𝑖 𝐿 𝜔𝑖 cos 𝜃𝑖 𝑓𝐵𝑅𝐷𝐹 Combining Analytic Direct Illumination and StochasticShadows (Heitz et al., 05 Combining-Analytic-Direct)29

REAL-TIME RAY TRACINGIndirect Diffuse and Ambient Occlusion Ambient Occlusion'Diffuse reflectance' * 'Denoised AO ray visibility '(short rays sampling hemisphere about normal) Indirect Diffuse GI'Diffuse reflectance' * 'Denoised irradiance'30

REAL-TIME RAY TRACINGIndirect Specular: Reflection / Refraction Indirect Specular: Reflections'Pre-integrated BSDF' * 'Denoised incoming radiance'Getting More Physical in Call of Duty: Black Ops II (Lazarov 2013)This approximation is for isotropic GGX only.Generalizing to arbitrary BSDFs harder. Open issue. Indirect Specular: Smooth Translucency31

REAL-TIME RAY TRACINGCombined Denoised Light Path Segments32

REAL-TIME RAY TRACING WITH MDLHow do we do it?Recall a few slides earlier:"Actual shading code [.] can be highly renderer specific""A renderer may analyze the declarative part of the compiled material instance""Renderer can implement its own building blocks for all of MDL’s df module"So. that's what we do.33

WORKING WITH A COMPILED MATERIALGraph of compiled materialMaterial model ibution functionsdiffuseTexturing functionstintroughnessspeculartintweight34

REAL-TIME RAY TRACING WITH MDLCustom Code GenerationPartial port of MDL SDK libbsdf.cpp to HLSLMap all glossy/microfacet BSDFs to GGX initiallyGenerate per-light-type BSDF analytic integral evaluation.Using LTC approximation (Heitz et al. 2016)Sphere, Rectangle, Disk, Line, Distant Light with Cone Angle (using virtual sphere)Generate custom HLSL functions, similar to MDL SDK Distiller: Weighted Diffuse Tint (reflectance) for all diffuse layers Weighted Specular Reflectance for all specular/glossy layers. Used for reflections Roughness for Top N Layers. Used for reflections35