Transcription
Dell EMC PowerEdge MX SmartFabric Configurationand Troubleshooting GuideAbstractThis document provides the steps for configuring and troubleshooting theDell EMC PowerEdge MX networking switches in SmartFabric mode.Configuration examples include Dell EMC Networking, Cisco Nexus, andCisco ACI environments.This document replaces the Dell EMC PowerEdge MX SmartFabricMode Deployment Guide, which is now deprecated.May 2019Dell EMC Configuration and Troubleshooting Guide
RevisionsDateDescriptionMay 2019Initial ReleaseThe information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in thispublication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.Use, copying, and distribution of any software described in this publication requires an applicable software license. 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Othertrademarks may be trademarks of their respective owners.Dell believes the information in this document is accurate as of its publication date. The information is subject to change without notice.2Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide
Table of contentsRevisions. 212Introduction . 81.1Typographical conventions .101.2Attachments .10SmartFabric Services for PowerEdge MX overview .112.1Dell EMC OS10 Enterprise Edition .112.2Operating modes .112.2.1 Full Switch mode .122.2.2 SmartFabric mode .122.3Changing operating modes .142.4MX9116n Fabric Switching Engine (FSE): virtual ports .142.5Virtual Link Trunking (VLT) .162.6Server templates, identities, networks, and deployment .162.6.1 Templates .162.6.2 Identities .172.6.3 Networks and automated QoS .172.6.4 Deployment.183SmartFabric mode requirements, guidelines, and restrictions .193.1Create multi-chassis management group .193.2Upstream network requirements .193.3Spanning Tree Protocol .203.4VLAN scaling guidelines .203.5Configuring port speed and breakout .213.6Storage Uplinks .213.7Switch slot placement for SmartFabric mode .223.7.1 Two MX9116n Fabric Switching Engines in different chassis .223.7.2 Two MX5108n Ethernet switches in the same chassis .223.7.3 Two MX9116n Fabric Switching Engines in the same chassis .233.8Switch-to-Switch cabling .233.9NIC teaming guidelines .243.10 Identity pools .243.11 Other restrictions and guidelines .2543Creating a SmartFabric .264.1Physically cable MX chassis and upstream switches .264.2Define VLANs .26Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide
54.3Create the SmartFabric .274.4Configure uplink port speed or breakout, if needed .284.5Create the Ethernet uplinks .294.6Configure the upstream switch and connect uplink cables .31Deploying a server .325.1Server preparation .325.1.1 Reset server CNAs to factory defaults .325.1.2 Configure NIC partitioning on CNAs .32675.2Create a server template .335.3Create identity pools .355.4Associate server template with networks .365.5Deploy a server template .36SmartFabric operations .376.1Viewing the fabric .376.2Editing the fabric .396.3Editing uplinks.406.4Editing VLANs on a deployed server .42Switch operations .447.1Switch management page overview .447.1.1 Switch overview .447.1.2 Hardware tab .467.1.3 Firmware tab .477.1.4 Alerts tab .477.1.5 Settings tab .4887.2Configure Ethernet switch ports from OME-M .507.3Upgrading OS10EE .52Validating the SmartFabric deployment .558.1View the MCM group topology .558.2View the SmartFabric status .568.3View port status .588.4CLI commands.598.4.1 show switch-operating-mode .598.4.2 show discovered-expanders .598.4.3 show unit-provision .598.4.4 show lldp neighbors .598.4.5 show qos system .614Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide
8.4.6 show policy-map .618.4.7 show class-map .618.4.8 show vlt domain-id .618.4.9 show vlt domain-id vlt-port-detail .628.4.109show interface port channel summary .62Scenario 1 - SmartFabric deployment with Dell EMC PowerSwitch Z9100-ON upstream switches .639.1Dell EMC PowerSwitch Z9100-ON switch configuration .649.2Dell EMC PowerSwitch Z9100-ON validation .669.2.1 show vlt .669.2.2 show lldp neighbors .669.2.3 show spanning-tree brief .6610 Scenario 2 - SmartFabric connected to Cisco Nexus 3232C switches .6810.1 Cisco Nexus 3232C switch configuration .6910.2 Configuration validation .7110.2.1show vpc .7110.2.2show vpc consistency-parameters .7210.2.3show lldp neighbors.7310.2.4show spanning-tree summary .7311 Scenario 3 - SmartFabric connected to Cisco ACI leaf switches .7411.1 Validated environment .7511.2 Cisco APIC configuration .7711.3 Deploy a SmartFabric .7811.3.1Define VLANs .7811.3.2Create the SmartFabric .7911.3.3Define uplinks .8111.4 Deploy servers .8311.4.1Create server templates .8311.4.2Add VLANs to the server templates .8311.4.3Deploy the server templates .8411.5 vCenter configuration overview .8511.6 Verify configuration .8711.6.1Cisco ACI validation .8711.6.2Verify connectivity between VMs .9212 SmartFabric troubleshooting .9312.1 Troubleshooting errors encountered for port group breakout .9312.2 Troubleshooting Spanning Tree Protocol (STP) .955Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide
12.2.1Verify if STP is enabled on upstream switches .9512.2.2Verify if type of STP is the same on MX and upstream switches .9512.3 Verify VLT/vPC configuration on upstream switches .9612.4 Discovery of FEM and compute sleds .9612.5 Troubleshooting uplink errors .9712.5.1Toggle auto negotiation .9712.5.2Set uplink ports to administratively up.9812.5.3Verify MTU size .9812.5.4Verify auto negotiation settings on upstream switches .9812.5.5Verify LACP .9912.6 Troubleshooting FC/FCoE .101AHardware overview .102A.1Dell EMC PowerEdge MX7000 chassis .102A.2Dell EMC PowerEdge MX740c compute sled .104A.3Dell EMC PowerEdge MX840c compute sled .105A.4Dell EMC PowerEdge MX9002m module .106A.5Dell EMC Networking MX9116n Fabric Switching Engine .107A.6Dell EMC Networking MX7116n Fabric Expander Module.108A.7Dell EMC Networking MX5108n Ethernet switch .108A.8PowerEdge MX7000 Fabrics I/O slots.109A.9Scalable fabric architecture overview .110A.10 QSFP28 double density connectors .113A.11 OOB management network .113BCD6OpenManage Enterprise Modular console .115B.1PowerEdge MX9002m module cabling .115B.2PowerEdge MX7000 initial deployment .115B.3PowerEdge MX Ethernet I/O Module initial deployment .117Rack-mounted switches .119C.1Dell EMC PowerSwitch S3048-ON.119C.2Dell EMC PowerSwitch Z9100-ON .119C.3Cisco Nexus 3232C .119C.4Cisco Nexus C93180YC-EX .119C.5Cisco Nexus C9336-PQ .119Additional information .120D.1Delete a SmartFabric .120D.2Delete an MCM group .120Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide
ED.3Reset chassis using RACADM .120D.4Reset an OS10EE switch to factory defaults .121D.5Reset Cisco Nexus 3232C to factory defaults .121Validated components .122E.1Scenarios 1 and 2 .122E.1.1 Dell EMC Networking switches .122E.1.2 Dell EMC PowerEdge MX7000 chassis and components .122E.1.3 Cisco Nexus switches .123E.2Scenario 3 .123E.2.1 Dell EMC Networking switches .123E.2.2 Dell EMC PowerEdge MX7000 chassis and components .123E.2.3 Cisco ACI components .1247FTechnical resources .125GSupport and feedback .126Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide
1IntroductionThe Dell EMC PowerEdge MX is a unified, high-performance data center infrastructure. The PowerEdge MXprovides the agility, resiliency, and efficiency to optimize a wide variety of traditional and new, emerging datacenter workloads and applications. With its kinetic architecture and agile management, PowerEdge MXdynamically configures compute, storage, and fabric, increases team effectiveness, and acceleratesoperations. The responsive design delivers the innovation and longevity that customers need for their IT anddigital business transformations.As part of the PowerEdge MX platform, Dell EMC Networking OS10 Enterprise Edition includes SmartFabricServices. SmartFabric Services is a network automation and orchestration solution that is fully integrated withthe MX Platform.Dell EMC PowerEdge MX7000 chassisThis document provides information about OS10 Enterprise Edition SmartFabric Services running on thePowerEdge MX platform. This document also provides examples for the deployment of two PowerEdgeMX7000 chassis and the setup and configuration of the SmartFabric. In SmartFabric mode, switches operateas Layer 2 I/O aggregation fabric and are managed through the Open Manage Enterprise Modular (OME-M)console.This guide also demonstrates connectivity with different upstream switch options, including: 8Dell EMC PowerSwitch Z9100-ONCisco Nexus 3232CCisco Nexus C93180YC-EX in Application Centric Infrastructure (ACI) modeDell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide
NOTE: For a detailed overview of the PowerEdge MX hardware, see Appendix A. For more information about thePowerEdge MX network architecture, see the Dell EMC PowerEdge MX Networking Architecture Guide.NOTE: The examples in document assume that the MX7000 chassis are configured in a Multi-ChassisManagement group and that no errors have been found. Additionally, this guide assumes the reader has a basicunderstanding of the PowerEdge MX platform.Four important terminologies and their definitions are as follows:Scalable Fabric – Exclusive to the MX7000 platform. This is an architecture comprised of the Dell EMCNetworking MX9116n Fabric Switching Engine and Dell EMC Networking MX7116n Fabric Expander Moduleallowing a fabric to span up to ten MX7000 chassis. This creates a single network fabric enabling efficienteast/west traffic flows between participating chassis. Supported in both SmartFabric and Full Switch modes.SmartFabric Mode - SmartFabric Mode leverages Smart Fabric Services (see below) to create a Layer 2network leveraging one to ten MX7000 chassis. Switches operating in SmartFabric Mode are administeredthrough the OpenManage Enterprise - Modular (OME-M) GUI interfaces that provide complete lifecyclemanagement of the network fabric.Full Switch Mode – When operating in Full Switch Mode, the switch can perform any functionality supportedby the version of OS10 running on the switch. Most of the configuration is performed using the CLI, not theOME-M GUI.Smart Fabric Services (SFS) – In PowerEdge MX, SFS technology provides the underlying networkautomation and orchestration to support all automated network operations. SFS is the underlying technologyfor all Dell EMC Networking OS10 automation efforts including PowerEdge MX, Isilon back-end storagenetworking, VxRail network automation, and so on.Table 1 outlines what this document is and is not. Also, this guide assumes a basic understanding of thePowerEdge MX platform.Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide - is/is notThis guide isThis guide is not/does notA reference for the most used features o
8 Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide 1 Introduction The Dell EMC PowerEdge MX is a unified, high-performance data center infrastructure. The PowerEdge MX provides the agility, resiliency, and efficiency to optimize a wide variety of traditional and new, emerging data center workloads and applications.
