Transcription
Introduction InfrastructureProgrammingPresented by // DAN DALYOCTOBER, 20211
IPDK OverviewIPDK is acommunity-driventarget agnosticframework forinfrastructureprogrammingthat runs on aCPU, IPU, DPU,or switch.2IaaSPaaSInline TOML/AI5G/EDGEIPDK Infrastructure Application InterfaceCompilerDrivenOpenCommunityIPDK Target Abstraction InterfaceIPDKTargetsCPU TargetIPU TargetSwitchTargetIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
Agenda Charter Approach Use cases GE3FIREWALL,IDSCRYPTOML/AI5G/EDGEIPDK Infrastructure Application InterfaceCompilerDrivenOpenCommunity– Example: Infrastructureas-a-Service Roadmap Call to actionInline AccelerationIPDK Target Abstraction InterfaceIPDKTargetsCPU TargetIPU TargetSwitchTargetIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
IPDK Charter4Open Community:Target Agnostic:Use-Case Driven:Open-source project withstandard interfacesAny CPU, IPU,DPU, or switchUse programmability foradaptabilityIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
ApproachEXISTING ENVIRONMENTDELINEATE BETWEENBUSINESS LOGIC, INFRASTRUCTURE,AND MANAGEMENTSIMPLIFY MANAGEMENTWITH STANDARD INTERFACESBETWEEN LOGICAL BOUNDRIESOPTIMIZE BY COMPILING TARGET AGNOSTICPROGRAMS INTO SOFTWARE AND NT1. Delineate Business Logic vs. Infrastructure2. Simplify Infrastructure Management3. Optimize using a Compiler-Driven Target Abstraction5IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
Use-Case Driven1.2.3.6Infrastructure-as-a-ServiceVirtual networking, storageand Crypto across VMs,containers and bare metalIaaSInline TOML/AI5G/EDGEIPDK Infrastructure Application InterfacePlatform-as-a-ServiceContainer Networking(Kubernetes) Sidecars(Envoy, MongoDB)Inline AccelerationFirewall, IDS, NetworkTelemetry 5G/WirelessInfrastructure, AI/MLPaaSCompilerDrivenOpenCommunityIPDK Target Abstraction InterfaceIPDKTargetsCPU TargetIPU TargetSwitchTargetIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
Open-Source Development RecipesDelineate, simplify, and optimizefor each major use case andfunctional area Open-Source Development &GovernanceNew patches, agents, andinterface supportOpen-Source definitions of interfacesDockerfiles and pre-built containers Development has started, come join!Collaborate on Slack , Github & IPDK.io7IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
IPDK Standard Interfaces InfrastructureApplication InterfaceDevices & Services forWorkload AppsPlatform CapabilitiesOffered to Mgmt Target AbstractionInterface (TAI)Target CapabilitiesOffered to IPDK8IaaSPaaSInline TOML/AI5G/EDGEIPDK Infrastructure Application InterfaceCompilerDrivenOpenCommunityIPDK Target Abstraction InterfaceIPDKTargetsCPU TargetIPU TargetSwitchTargetIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
Take-Home Demo IPDK Container Traffic Gen Container Storage Target ContainerCreate Virtual DevicesHOSTVM Create virtual port Create virtual diskGenerate network trafficand storage read/writes9IPDK ContainerKVMDownload ContainersOpenConfig RPC or CLI create port host1 virtio-net 10Gbps netdev1 create disk host1 virtio-blk 100KIOPS bdev1KVM TargetidevgemuTGContainerveth orEthernetIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & SlackStorageTarget
Where Can IPDK Run?KVM HOSTVIRTUAL INSTANCEPHYSICAL DEVICEInstanceVMIPDK AppsInstanceHostIPDK AppsKVMIPDK AppsDeviceidevidevidevgemuKVM/Cloud10 Insert devices into VMs Sockets in/out of VM apps/containers Direct attached devices (drives,NICs, Accel) Insert devices into instances(VM, bare metal) Sockets in/out of instance apps/containers Virtual devices (drives, NICs, Accel) Insert devices into hosts (VM, bare metal) Sockets in/out of host or VMapps/containers Direct and virtual devices (drives, NICs,Accel)Software HypervisorSoftware Virtual SwitchCloud HypervisorCloud Virtual SwitchHardware HypervisorHardware Virtual SwitchKVM can also run in each instanceKVM can also run in each instanceIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
Call to ActionCome Join the Community Develop IPDK-compatibleApplications Contribute an IPDK TargetIaaS11Inline TOML/AI5G/EDGEIPDK Infrastructure Application Interface– Software, Switch, IPU,DPU, etc. Develop New Recipes Contribute to ExistingRecipes Contribute to the InterfacesPaaSCompilerDrivenOpenCommunityIPDK Target Abstraction InterfaceIPDKTargetsCPU TargetIPU TargetSwitchTargetIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
IPDK ContainerManagedOpenConfigManagedService Mesh StorageP4Runtime KubernetesProtocol(xDS)Host ServerVMBare Metal /ContainersINTIPUContainerteamdBGPEnvoyqemu vDPAvDisk(s)vPort(s)vDisk(s)IPU (vhost)SPDKInitiatorLibraryP4 OVSP4RuntimeOpenConfigP4DPDKTargetvPort(s)Inverse Devices(vhost)QuickAssistLookasideCPU Telemetry& Power MgmtK8S CNIP4MEVTargetNIC DevicesOperating System (Customer’s Choice)Secure Boot / Attestation12IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
IaaS – Network VirtualizationOpenConfigLoad DriverTraffic Send/Receive3Host Server or VM1Create PortP4RuntimeINTIPDKContainerteamdBGPP4 OVSP4RuntimeOpenConfigvDisk(s)2vPort(s)Hotplug Portassoc. netdevSWTargetInverse DevicesIPU/vhostNIC DevicesVPC ExceptionsInfrastructure Operating System413VPC DataplaneHWTargetNetwork ControlPlane PacketsIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
IaaS – Storage Virtualization3StorageProtocolLoad DriverStorage RequestsHost Server or VMOpenConfigIPUContainer2vPort(s)Storage Request 4IPU (vhost)Create DisksP4RuntimeSPDKInitiatorLibraryvDisk(s)1P4 OVSP4RuntimeOpenConfigHotplug Diskassoc. bdevInverse Devices(vhost)NIC DevicesInfrastructure Operating System14IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack5StorageCompletion
IaaS – Storage Virtualization - Interfaces1. OpenConfig Operation’add virtio-blk disk1 bdev5 10TB’2. TDI (Table Driven Interface)‘addEntry(vdev, “disk1”, config1)3. Backend Device Operation– KVM: qemu hotplug– Canyon: pci hotplug– MEV: simulated hotplug4. SPDK Operationassociate device w/ ‘bdev5’15VirtualStorageDeviceProtocol OpenConfigSPDKInitiatorLibraryP4 OVSP4RuntimeOpenConfigAdd/Del/Modify Virtual Device TableAssociate Device bdevvDiskIPDK Target BackendHotplug Virtual DeviceTarget(qemu or Hardware)IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
IaaS – Host Virtualization – NetworkCreate Portw/ VF #P4RuntimeOpenConfig3VM Loads DriverTraffic Send/ReceiveHost ServerVMqemu vDPAvDisk(s)IaaS –Host Virt. eamdBGPIaaS –IPDK Plat. ManagerP4 OVSP4RuntimeOpenConfiggRPC2vPort(s)Assign VFassoc. netdevSWTargetInverse DevicesIPUInfrastructure Operating System16VPC DataplaneHWTargetNIC DevicesVPC Exceptions41Network ControlPlane PacketsIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
IaaS – Host Virtualization – Disk3StorageProtocolVM Loads DriverTraffic Send/ReceiveHost ServerVMqemu vDPAvDisk(s)IaaS –Host Virt. AgentLiveMigrationvPort(s)Storage Request 4vDisk(s)Create Diskw/ VF #P4RuntimeOpenConfigINTIPUContainerteamdBGPIaaS –IPDK Plat. ManagerSPDKInitiatorLibrarygRPC2vPort(s)P4 OVSP4RuntimeOpenConfigAssign VFassoc. bdevSWTargetInverse DevicesIPU1HWTargetNIC DevicesVPC ExceptionsInfrastructure Operating System417VPC DataplaneIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack5StorageCompletion
IaaS – Host Virtualization – Net Disk3Flexibility of PrimaryControl Point- Host or IPUVM Loads DriverTraffic & StorageHost ServerVMqemu vDPAvDisk(s)IaaS –Host Virt. INTIPUContainerteamdBGPIaaS –IPDK Plat. ManagerSPDKInitiatorLibrarygRPC2vPort(s)Inverse DevicesIPUCreate Port/Diskw/ VF #P4RuntimeOpenConfigP4 OVSP4RuntimeOpenConfigAssign VFassoc. netdevQuickAssistLookasideVPC ExceptionsP4DPDKTargetCPU Telemetry& Power MgmtOperating System (Customer’s Choice)418VPC DataplaneP4MEVTargetNIC DevicesNetwork ControlPlane PacketsSecure Boot / AttestationIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
PaaS – CSI & CNIOptional Optimizations: Container Dedicated Queues (CDQ) Localhost SocketsHost Server or VMContainers sPaaS – CSIvDisk(s)PaaS – CNIManagedOpenConfigManagedService Mesh StorageP4Runtime ySPDKInitiatorLibraryP4DPDKTargetvPort(s)IPU (vhost)P4 OVSP4RuntimeOpenConfigInverse Devices(vhost)QuickAssistLookasideCPU Telemetry& Power MgmtK8S CNIP4MEVTargetNIC DevicesOperating System (Customer’s Choice)Secure Boot / Attestation19IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
PaaS – K8s Delineations0. Bare Metal, no IPU2. IPU anchored control (Managed Services)YAMLspeckubeletHost NodeK8s control planeYAMLspeckubeletHost NodeIPDK CNIkublet agentIPUIPDK CNIkubeproxyK8s control planeIPDK kubelet agentIPDK kubeproxy agent1. Trusted host anchored control (Self Managed)3. Split Tenant & Managed InfrastructureYAMLspecHost NodeTenantK8s control planeHost NodeK8s control planeCalico AgentkubeletIPDK CNIkubeletkubeproxyIPDK CNIIPUIPUYAMLspecInfraK8s control planekubeletkubeproxyCalico Agent20IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & SlackCalico Agent
PaaS – Calico Running on IPUOptional Optimizations: Container Dedicated Queues (CDQ) Localhost SocketsHost Server or VMContainers sPaaS – CSIvDisk(s)PaaS – CNIManagedOpenConfigManagedService Mesh StorageP4Runtime ySPDKInitiatorLibraryP4DPDKTargetvPort(s)IPU (vhost)P4 OVSP4RuntimeOpenConfigInverse Devices(vhost)QuickAssistLookasideCPU Telemetry& Power MgmtPaaS –Calico P4 AgentK8S CNI(Calico)P4MEVTargetgRPCNIC DevicesOperating System (Customer’s Choice)Secure Boot / Attestation21IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
PaaS – Envoy DelineationsCurrent Service Mesh on HostNew PluginDevelopmentService Mesh on IPU worker nodeHost NodeHost Node4Service ’1.2.3.4.5.22vethService A3trafficredirectIPU Worker Node2veth1compute5Expose IPU compute resources as a k8s worker nodeModify k8s orchestration to inject Envoy to IPU Worker NodeConfigure IPU dataplane offload for ‘iptables’ traffic redirectOrchestrate tenant serviceIstio configuration of EnvoyIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
PaaS – Envoy on IPUOptional Optimizations: Container Dedicated Queues (CDQ) Localhost SocketsHost Server or VMContainers sPaaS – CSIvDisk(s)PaaS – CNIManagedOpenConfigManagedService Mesh StorageP4Runtime KubernetesProtocol(xDS)INTIPUContainerPaaS –Envoy P4 rgetvPort(s)IPU (vhost)P4 OVSP4RuntimeOpenConfigInverse Devices(vhost)QuickAssistLookasideCPU Telemetry& Power MgmtPaaS –Calico P4 AgentK8S CNI(Calico)gRPCP4MEVTargetNIC DevicesOperating System (Customer’s Choice)Secure Boot / Attestation23IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
PaaS – Multi-Tenancy (Service:Sidecar) 1:1 Service Mesh (Envoy) containers Resource allocation & life cycle CDQ enable scheduling & BW allocations Separation of mTLS Certificates No restriction on Istio deploymentYAMLInfra managed per Tenant Control& Mgmt. Planes.Istio Config Statistics Logs CertificatesHost NodeIPU Support various deployments such as multipleEnvoy instances, soft multi-tenancy, & RBACnamespaces.ServiceAService BCDQ IPU dataplane used for traffic redirection toEnvoy proxy and offload of CNICDQService AYAMLIPU NodeIPU DataplaneCDQ Additional services (OpenTelemetry) can beorchestratedCDQOpenTelemetryCollector Optional to use Istio soft multi-tenancyMetricsGrafana UI24PrometheusIPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack
Thank you.IPDK.io: Infrastructure Programmer Development KitCollaborate with the community on Github & Slack25
NETWORK STORAGE CRYPTO FIREWALL,IDS ML/AI 5G/EDGE IPDK Infrastructure Application Interface CPU Target IPU Target Switch Target. 3 . Approach Use cases Open-source development -Example: Infrastructure-as-a-Service Roadmap Call to action IPDK.io: Infrastructure Programmer Development Kit Collaborate with the community on .
