WIXLIB

Vulkan LoaderDeep DiveMark Young and Lenny Komow

Welcome!

Who We Are. LunarG:–– Mark Young:––– Software engineering firm focusing on Graphics and Shader technologiesValve provides financing to support Vulkan Ecosystem: Vulkan desktop loader Validation layers Tools (Trace/Replay, VIA)Current Vulkan desktop loader ownerOpenGL User-Mode drivers 9.5 years (ATI/AMD)DirectX Kernel-Mode drivers 5 years (Qualcomm)Lenny Komow––Desktop Loader support (working towards co-owner)2D/3D ECAD rendering 1.5 years (Cadence Design)

Ask Yourself, “What Do I Want to Learn?”“You’ve got to be very careful if you don’tknow where you are going, because youmight not get there.”-Yogi Berra

What This Is Not. An introduction to Vulkan A Vulkan tutorial on writing:– Applications– Layers– Drivers Don’t abandon ship yet!An overview of using Validation LayersLook for that and more on the Khronos Vulkan Resources page

We Will Be Covering.-A detailed look at the Vulkan Loader and its interfaces to:- Applications- Drivers- Layers-Debug settings-Possible optimizations

Vulkan App Developer View

Loader Position (High-Level)

Really Loader(s) Plural Intent is only one loader to rule them all, but reality is– Two different loaders: Desktop Loader– Same source used for Linux/Windows Android Loader– Nougat devicesStill, one loader interface design (in GitHub and LunarG Vulkan SDK)

Loader Interface Doc Available in Vulkan-LoaderAndValidationLayers Github./loader/LoaderAndLayerInterface.md Valid for all Vulkan loaders

Vulkan Desktop Loader Versus OpenGL LoaderControlled ByUpdatedAvailable SourceWho Can Contribute* CLA required for contributionsOpenGL LoaderVulkan LoaderOperating/Windowing SystemKhronosInfrequently4-6 WeeksIn some casesOpen Source (Github)Limited set of developersAnyone*

Github

How Does the Loader Get on a User’s System? Android:– Installed as part of Android OS with Nougat (or newer) Desktop:– Driver Packages– Application Installs– LunarG SDKs– Manually built/installed (GitHub)You may have multiple copies,but that’s ok. Really!

Example of Multiple Installs

Which File to Use? Possibly multiple files on your machine Original file contains full version (major, minor, lkan-1-1-0-39-1.dllvulkan-1-1-0-42-0.dll Apps should target the base name shared object/DLLlibvulkan.so.1vulkan-1.dll symbolic link to latestcopy of latestThe 1 is the ABI version–Indicates backwards compatibility

Your Ticket to the Fast Lane!The loader is designed to be: Optimized Efficient IgnorantIt sets you up and steps out of your way as much as possibleWhere there is great powerthere is great responsibilityWinston Churchill

With Some Caution- Loader will crash if you use it improperly- Examples:- Attempting to use extensions that aren’t enabled- Use Device procs queried from a different device- vkGetDeviceProcAddr for vkQueueSubmit onPhysical Device A (Nvidia)- Call it with Physical Device B (Intel)- Passing in a wrong handle to a call- Using a VkInstance in place of a VkDevice

Expensive Loader Calls Most expensive loader calls rateXXX Loader has to generate state on these (but calls are supposed to bestateless) Some overhead can include loader re-gen/re-alloc data every call Why?– External GPUs––Layer info changingSystem state change Switching to battery

Loader DebuggingDesktop Loader Debug [Environment Variable]:VK LOADER DEBUGEnable Validation Layerswarn, error, info,perf, debug, or all

Heading Deeper Dispatchable Objects Instance versus Device Objects/Commands Trampolines and Terminators Manifest Files (Desktop) What’s Exposed by Default? Extension Handling Loader/ICD Interaction Loader/Layer Interface Loader Security

Dispatchable Objects Most commands take an opaque dispatchable object as first parameterVkResult vkGetEventStatus( VkDevice device , VkEvent event); First field in each dispatchable object is a dispatch table pointer––Used by loader trampoline codeICDs must reserve first element of created objects for a pointer the loader will fill inVkDevice*Dispatch TablevkCreateDevicevkDestroyDevice.

Instance Versus Device Objects/CommandsInstance:High-level construct (similar, but not the same, as a GL Context)Works with all ICDs and Physical DevicesIncludes: VkInstance and VkPhysicalDeviceDevice:Logical accessor to a particular Physical Device (through a particular ICD)Includes: VkDevice, VkQueue, VkCmdBuffer and other objects derived from these

Trampolines and Terminators

Trampolines and TerminatorsTrampolineLoader entrypoint for a commandTriggers the proper call-chain startStart could be a layer, the terminator, or an ICDTerminatorLoader code that distributes calls to multiple ICDsAll Instance and certain Device commands

Call ChainsSequence of function calls from the application to its final destinationMay include stepping into:- Loader (one or more times)- Any enabled layers- One or more ICDsDifferent for Instance versus Device commandsInstance chain generated during vkCreateInstanceDevice chain generated during vkCreateDevice

Example Call ChainsInstance Call ChainDevice Call Chain using loader exports *Device Call Chain using vkGetDeviceProcAddr ** Some special cases still require a specific device call chain to include atrampoline/terminator

Manifest FilesBased on Manifest lists used by shipsListed all Crew, Passengers, Cargo

Manifest File Usage Only used by Desktop Loader– Primary purpose to reduce security riskJSON file format, read on layer/ICD info merateInstanceExtensionProperties Lists all important properties of interest to the app/loader Loader interaction with actual library delated––ICD interaction not initiated until Manifest file appears validLayer interaction not initiated until enabled during vkCreateInstance

What’s Exposed By Default?Loader exports following symbols by default: Core Vulkan (currently 1.0)WSI for specific platform– All Platforms: –Windows Additional: –VK KHR win32 surfaceLinux Additional: –VK KHR surfaceVK KHR swapchainVK KHR displayVK KHR display swapchainVK KHR xlib surfaceVK KHR xcb surfaceVK KHR wayland surfaceVK KHR mir surfaceAndroid Additional: VK KHR android surface

Extension HandlingBehavior Dependent on Loader KnowledgeKnown extension, “No”-n problemsOk, sorry about the bad pun!Unknown Device Extensions: Loader handles automaticallyUnknown Instance Extensions: Loader must know in order to handleproperly

Why Unknown Instance Extensions cause problemsDevice Call Chain using vkGetDeviceProcAddrInstance Call ChainCan handle unknownparameter list easily withtail-call optimizationCan’t handle unknownparameter list because mustprocess for individual ICDs

Instance Extension Name FilteringLoader filters out Unknown Instance Extension names Won’t appear in vkEnumerateInstanceExtensionProperties Throw an error in vkCreateInstanceDisable Environment Variable:VK LOADER DISABLE INST EXT FILTERBeware:Can cause crashes if youattempt to use a command froma ‘forced-on’ instanceextension!

Loader Generated CodeNew Vulkan header/xml integrated into tree about once a weekAll extension code automatically generated Python scripts (under scripts folder) Run every buildLoader script of interest scripts/loader extension generator.py

Still Awake?

Installable Client Drivers (ICDs) Provided by hardware company (or Open Source community) Vulkan Drivers have associated JSON Manifest file– Install should set this up properly Versioned interface with loader– Detailed in LoaderAndLayerInterface.md in Github– Also detailed in SDK as LoaderAndLayerInterface.html

Desktop ICD Manifest Search LocationsWindows Registry: [Simply points to location of JSON file]HKEY LOCAL MACHINE\SOFTWARE\Khronos\Vulkan\DriversOn 64-bit Windows, 32-bit ICD JSON found in:HKEY LOCAL sLinux: [Actual icd.d HOME/.local/share/vulkan/icd.d

Example ICD Manifest FileFor Intel’s Open Source Mesa Vulkan DriverManifest JSON file format{"file format version": "1.0.0","ICD": {"library path": "/usr/lib64/libvulkan intel.so","api version": "1.0.3"}All ICDData}Location of library (dll/so)Version of Vulkan API written against

Disabling Windows ICDs in RegistrySimply write 1 to the ICD 32-bit Key ValueLoader will now ignore it

Desktop ICD Debug Environment VariableForce a particular ICD or set of ICDs:VK ICD FILENAMESDelimited list of specific driver JSON Manifest files,should contain full path to JSON fileexport VK ICD FILENAMES /home/./intel icd/i915.json:/home/./nvidia.json

Loader/ICD InterfaceICDs should expose at least three commands:vk icdNegotiateLoaderICDInterfacevk icdGetInstanceProcAddrvk icdGetPhysicalDeviceProcAddrNames can actually be different (overridden in Manifest file)Loader looks for Manifest files during certain teInstanceLoader then “negotiates” the interface version with the ICD

Loader/ICD Interface Version NegotiationLoader calls ICD usingVkResult vk icdNegotiateLoaderICDInterfaceVersion(uint32 t *pVersion);*pVersion initially contains loader’s desired interface versionICD then determines if it can run that version: If it can, it returns VK SUCCESSIf it can’t, it determines if it can run at a lower version––If it can, it updates *pVersion with the new interface version and return VK SUCCESSIf not, it returns VK ERROR INCOMPATIBLE DRIVERNow the loader queries all the appropriate commands using vk icdGetInstanceProcAddr.

More Details on Loader/ICD InterfaceDocumented in LoaderAndLayerInterface.md fileUnder “Loader and ICD Interface Negotiation” sectionInterface defined in vk icd.hIn Github and in the installed SDK as include/vulkan/vk icd.h

LayersLayers generated by many different sourcesDetails we’ll cover: Instance and Device Layers Implicit versus Explicit Layers Layers manifest files Loader/Layer Interface

Instance and Device LayersBoth vkCreateInstance and vkCreateDevice have places to enable layersvkCreateInstance in VkInstanceCreateInfo structuretypedef struct VkInstanceCreateInfo {VkStructureTypesType;const void*pNext;.uint32 tenabledLayerCount;const char* const*ppEnabledLayerNames;.} VkInstanceCreateInfo;vkCreateDevice in VkDeviceCreateInfo structuretypedef struct VkDeviceCreateInfo {VkStructureTypesType;const void*pNext;.uint32 tenabledLayerCount;const char* const*ppEnabledLayerNames;.} VkDeviceCreateInfo;

Device Layers Deprecated!Not used anymoretypedef struct VkDeviceCreateInfo {VkStructureTypeconst void*VkDeviceCreateFlagsuint32 tconst VkDeviceQueueCreateInfo*uint32 tDeprecated!const char* const*uint32 tconst char* const*const VkPhysicalDeviceFeatures*} sionNames;pEnabledFeatures;All Layers enabled in InstanceDevice Layers are ignoredMakes it simpler for you

Desktop Layer LoadingImplicitEnabled by defaultDisable with Environment Variable (Defined in JSON)Example:Nvidia’s Optimus LayerValve’s Vulkan Steam Overlay LayerExplicitMust be enabled by app or environmentExample:Validation LayersOnly“Explicit Layers” onAndroid

Example Explicit Layer Manifest FileManifest JSON file formatVkTrace Layer Manifest File:{Layer Name"file format version" : "1.0.0",Layer Type (only GLOBAL or"layer" : {INSTANCE)"name": "VK LAYER LUNARG vktrace","type": "GLOBAL","library path": ".\\VkLayer vktrace layer.dll",Location of library (dll/so)"api version": "1.0.42","implementation version": "1",Version of Vulkan API written against"description": "Vktrace tracing library","functions" : {Layer implementation version"vkGetInstanceProcAddr" : " HOOKED vkGetInstanceProcAddr","vkGetDeviceProcAddr" : " HOOKED vkGetDeviceProcAddr"}}All LayerData}Overridingfunctionnames