WIXLIB

Case StudySmartOur Switchtec Gen 4 PCIeswitches are GPU-optimized withlow pin-to-pin latency and lowlatency variation for optimalmachine learning workloadperformance.ConnectedSwitchtec features include up to100 line-rate capable PCIe lanes aswell as an integrated highperformance cut-through DirectMemory Access (DMA) engine.SecureSwitchtec’s high-reliability featuresinclude hot- and surprise-plugsupport, end-to-end data integrityand debugging with the ChipLinkdiagnostics tool.GigaIO Introduces PCI Express (PCIe ) Network Fabric Solutionfor the Age of AIThe emergence of Artificial Intelligence (AI) and the associated Machine Learning (ML)and Deep Learning (DL) applications are fueling demand for fundamental changes inthe creation of compute and storage clusters. Faster and larger storage arrays anda rapid proliferation of specialized compute accelerators, like Graphics ProcessingUnits (GPUs), Field-Programmable Gate Arrays (FPGAs) and Application-SpecificIntegrated Circuits (ASICs), are creating bottlenecks and configuration problems forthe interconnect systems. Traditional networks were never designed to handle theperformance requirements of these workloads and devices.microchip.com

Today’s data streams are far wider and deeper than ever,with workflows so varied that they may combine differentinfrastructure setups within a single workflow. Today, asingle workload may require different types of accelerators:NVIDIA V100 Tensor Core V100 GPUs for compute projects,NVIDIA GeForce RTX graphics cards for visualization orFPGAs for encryption, all within the same workflow.But today’s advanced AI-driven data center modelscome with a tax on efficiency. High-dollar investmentsin compute resources often result in ultra-powerfulaccelerators remaining idle during down periods whenthey aren’t attacking today’s AI workloads.In a traditional data center, managers must guess anddecide at server refresh time what hardware their users willneed for the next three to five years and provision for peakdemand, which means resources may sit idle—the GPUindustry estimates that the average utilization rate is around15 percent—despite being some of the most expensiveresources in the rack.Half a century ago, large corporations and universitieswere making multi-million-dollar bets on mainframecomputers. The “big iron” managers who had to justifythe unprecedented spending to the executive suitecontinually needed to prove that their goliath machines were producing economies of scale. Resourcesthat sat idle were their enemies as well.Having made sizable hardware and software investments topursue their High-Performance Computing (HPC) and AI datacenter initiatives, managers often look to allocate expensivenew resources such as GPUs and FPGAs found inside serversto other workloads during idle periods.“Buy the largest computer you could afford and findways to maximize the amount of work done on it” wasa common refrain. 1The philosophy of resource sharing is nothing new. Today’snetwork architects and administrators have practicallygrown up with the concept of virtualization, where datacenter resources have transitioned beyond physical serversin a fixed location to cloud-based virtual environments.Highly inefficient designs of the past where server hardwareresources were allocated only to specific operating systemsand enterprise applications seem severely outdated. Modernapproaches not only allow servers to operate near capacitybut they also reduce power requirements, operating costsand downtime. Beyond these efficiencies and economies ofscale, virtual environments even make enterprise softwareapplications perform better.Echoes of the past are being heard again.

The Rise of Composable Disaggregated Infrastructure (CDI)CDI, under the umbrella of SDI (Software-Defined Infrastructure), has garnered growing interest in the data center. Composable infrastructure disaggregates compute, storage andnetworking resources into shared resource pools that can beavailable for on-demand allocation (i.e., “composable”).CDI is a data center paradigm with a software layer that abstracts a hardware layer consisting of IT resources across avirtual fabric and organizes them into logical resource pools, notdissimilar to a Hyper-Converged Infrastructure (HCI) cluster. Butrather than having a linked-up series of self-contained nodes ofcompute, storage, network and memory as in HCI, composableinfrastructure is a rack-scale solution like Converged Infrastructure (CI). Unlike CI, however, composable infrastructure is, at itscore, an infrastructure-as-code delivery model with an open andextensible unifying API layer that communicates and controlshardware resources.The remarkable increase in the amount of data being collected,analyzed and stored is driving the rapid adoption of advanceddata analytics and AI that is challenging the fundamental architectures of today’s data centers. This challenge explains muchof the move to the public cloud, which promises the ultimate inresource flexibility but comes with a high price tag, especially forHPC and AI workloads.The rapid pace of change in acceleration technology and AI software fuels the necessity for highly flexible and easily upgradeable architectures capable of incorporating new hardware technology without forklift upgrades to expensive equipment. Thismeans breaking the server box and disaggregating elementsof the traditional server into separate pieces that can be easilyshared. But to effectively disaggregate storage and accelerators,the interconnects must support both exceptional low latencyand high bandwidth.Data center managers also want to drive high utilization of expensive new storage and acceleration components to keep thecosts of Capital Expenditure (CapEx) and Operating Expenses(OpEx) down.Today’s advanced scale computing, enterprise, cloud and edgedata centers need both scale-up and scale-out resources acrossthe cluster and require a network technology that will scaleaccordingly.

The PCIe Standard:Evolution at WorkPCIe is a widely deployed bus interconnect interfacecommonly used in server platforms. It is also beingincreasingly used as a storage and application acceleratorinterconnect solution.With the cooperation and support from more than800-member companies that represent the modernconnectivity ecosystem, the PCI Special Interest Group(PCI-SIG) manages the development of PCI specifications asopen industry standards. The organization has seen the PCIestandard grow to become a widespread solution over thecourse of two decades. PCI-SIG advocates like to point outthat Input/Output (I/O) bandwidth doubles every three years,and over the course of five generations of the standard, PCIeis now being deployed to serve devices and applications thatcould not have been imagined during the early days of itsdevelopment.The popularity of display adapters for 3D graphics andvideo and the evolution of new storage paradigms in theearly 2000s led PC-based developers to seek alternativesfor predecessor connectivity standards that couldn’t deliverthe computer-to-peripheral communications speed andperformance that users required. The very first PCIe standardoffered a common protocol for transferring data at 250 Mb/s.As industry demands for higher-performing I/O are being metby each succeeding generation of the standard, the PCI-SIGmandates backwards compatibility, cost-efficiency, highperformance, a processor-agnostic stance and a high degreeof scalability.“PCIe is seen as a dominant datatransfer protocol that has expandeddevice support to include Add-In Cards(AICs), network adapters, NetworkInterface Cards (NICs), accelerationcards, FPGAs and NVMe Flashdrives.”The breadth of its influence can be seen well beyondtraditional compute environments as its footprint nowextends from PC, server, storage and cloud sectors intomobile, IoT and automotive connectivity.

Enter GigaIOWith any fundamental shift in computing, networkingor storage technology, new entrants typicallyemerge to take advantage of market opportunitiesbefore they reach mass adoption. Such is the casewith GigaIO Networks, Inc. of Carlsbad, CA. Thecompany was established in 2017 by networkingand high-performance computing veterans whoassembled a highly skilled team with deep networkarchitecture, hardware, software and silicondevelopment capabilities. The team invented thefirst truly composable cloud-class, software-defineduniversal infrastructure fabric, which empowersusers to accelerate workloads on demand, usingindustry-standard PCIe technology. GigaIO’s patentedtechnology optimizes cluster and rack systemperformance in a way that also reduces the Total Costof Ownership (TCO) for customers. They have quiteliterally taken PCIe outside the box with a softwarecentric approach to enable on-demand resources forAI workloads.

The ChallengeWith deep ties to the high-performance computingand storage communities and a vision for expandingthe role of PCIe connectivity, the team at GigaIOrecognized a gap in the market. The demand forGPU-based solutions and accelerator hardwareto address AI-driven workloads was increasing.Applications for intense data processing werecrossing over from research institutions and intothe enterprise. Geophysical modeling, weatherforecasting, bioinformatics, physical simulations,edge computing, fintech trading platforms, advancedvisualizations, analytics and cloud computing all useadvanced heterogeneous computation architecturesto enable faster execution of compute-intensivetasks. The rise of real-world AI and ML has boostedGPU and related hardware sales. But what if thesenew assets could be leveraged on-demand toaddress unprecedented workloads? While they offerpeak performance when in use, they often sit idle.By asking the right questions, the GigaIO teamaddressed the challenge to architect a rack-scalecomposable infrastructure solution that deliversthe unlimited flexibility and agility of the cloud butat a fraction of the cost. They identified a series offoundational requirements: The solution would need to orchestrate anycompute, acceleration (GPUs, FPGAs, ASICs),storage, memory or networking resource for anyworkload using an enterprise-class, easy-to-useand open standard, high-performance network. It would need the ability to deploy, expand, reduceor replace all rack resources dynamically in realtime. It would need to fully automate bare-metal, cloudclass infrastructure, delivering higher throughputat a fraction of the cost.