“You Can’t Control What You Can’t Measure” Tom DeMarco PDF Free Download

2y ago

87 Views

1 Downloads

3.82 MB

56 Pages

Report/dmca

Download PDF

Transcription

Data network visibility and control“You can’t control what you can’t measure”Tom DeMarco1Monday, July 16, 12

Reason 1: Widely supported industry standard2Monday, July 16, 12

Reason 2: ComprehensiveIPFIX, NetFlow RoutingSwitchingTraffic SentinelServersVirtual Switching All Devices All Servers All Applications All the timeVirtual Servershttp://www.sflow.orgMay 09, 2011Monday, July 16, 12ApplicationsCopyright InMon Corp. 2010 All Rights Reserved3

sFlow Overview: replaces counter polling“De-synchronized, Parallel Push” sFlow agent automatically pushes full set of SNMP ifTablecounters1 Compared to SNMP polling, counter push results in 10-20xfewer packets on network, reduces CPU load on switch andon network management software (XDR is easier toencode/decode than SNMP) Single sFlow collector can easily monitor 200,000 switchports with 1 minute granularity. SNMP polling with 5 minutegranularity requires 5-10 collectors.1.ifIndex, ifType, ifType, ifSpeed, ifDirection, ifAdminStatus, ifOperStatus, ifInOctets, ifInUcastPkts,ifInMulticastPkts, ifInBroadcastPkts, ifInDiscards, ifInErrors, ifInUnknownProtos, ifOutOctets,ifOutUcastPkts, ifOutMulticastPkts, ifOutBroadcastPkts, ifOutDiscards, ifOutErrors,ifPromiscuousModeSeptember 08, 2011Monday, July 16, 12Copyright InMon Corp. 2010 All Rights Reserved4

sFlow Overview: monitors all protocols Simple agents: packet headers sent to sFlow collector for decoding.Easier to add decodes to central collector than to every device in amulti-vendor network (e.g. IPv6, FCoE etc.)Captures complex layering (e.g. MAC/VLAN/MPLS/IPv4/IPv6): criticalfor tracing packet paths through network.September 08, 2011Monday, July 16, 12Copyright InMon Corp. 2010 All Rights Reserved6

Traffic Sentinel: Traffic Breakdown MAC, VLAN, IP, IPv6,TCP, UDP, MPLS, TRILL,RTP etc. (over 100 fields) 1-minute granularity Thresholds/alerts Automatic de-duplication Subnet rollups03/08/07Monday, July 16, 12Copyright InMon Corp. 2006 All Rights Reserved

TRILL Fabrics - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Outer Destination MAC Address - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Outer Destination MAC Address Outer Source MAC Address - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Outer Source MAC Address - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Ethertype IEEE 802.1Q UP C Outer VID - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Ethertype TRILL V Hop Limit M Reserved - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Egress RBridge Address Ingress RBridge Address - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Inner Destination MAC Address - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Inner Destination MAC Address InnerSource MAC Address - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Inner Source MAC Address - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Ethertype IEEE 802.1Q UP C Inner VID - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Original Ethernet Payload - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - New FCS - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Monday, July 16, 12 sFlow on existing Ethernetswitches captures the followingTRILL fields––––TRILL RBridge AddressesForwarding pathHop countBroadcast bit As well as inner/outer MACaddresses and encapsulated TCP/IP etc. data sFlow monitoring providesinformation about path utilization,applications using a path etc.Critical for load balancing andtroubleshooting TRILLdeployments

TRILL FabricsMonday, July 16, 12

sFlow Overview: captures packet pathInternetSeptember 08, 2011Monday, July 16, 12 Each packet sample capturesthe forwarding path for thepacket Threading together the pathsprovides a constantlyupdating picture of networktopology and host locations The combination offorwarding table data andpacket headers provides anintegrated view of traffic. E.g.you can filter on forwardingattributes (VLAN, MPLS,route) and see traffic, or filteron traffic and identifyforwarding paths.Copyright InMon Corp. 2010 All Rights Reserved9

Traffic Sentinel: Multivendor topologydiscoveryUses: sFlow CDP FDP LLDP Spanning-tree Bridge-tables and more. Auto-layout Mouse-wheel zoom Show Status,Traffic(refreshed every minute)03/08/07Monday, July 16, 12Copyright InMon Corp. 2006 All Rights Reserved

ArchitectureMonday, July 16, 12

ArchitectureSimple agentsMonday, July 16, 12

ArchitectureSimple agentsMonday, July 16, 12Smart collector

ArchitectureSimple agentsEasy to implementMonday, July 16, 12Smart collector

ArchitectureSimple agentsEasy to implementEmbedded, wire-speedMonday, July 16, 12Smart collector

ArchitectureSimple agentsEasy to implementEmbedded, wire-speedLow costMonday, July 16, 12Smart collector

ArchitectureSimple agentsEasy to implementEmbedded, wire-speedLow costCounter-pushMonday, July 16, 12Smart collector

ArchitectureSimple agentsEasy to implementEmbedded, wire-speedLow costCounter-push packet/transaction samplingMonday, July 16, 12Smart collector

ArchitectureSimple agentsEasy to implementEmbedded, wire-speedLow costCounter-push packet/transaction samplingMonday, July 16, 12Smart collectorNetwork-wide,integrated,visibility and control

ArchitectureSimple agentsSmart collectorInternetAnalyzerMonday, July 16, 12

ArchitectureSimple agentsNetworkSmart collectorInternetAnalyzerMonday, July 16, 12

ArchitectureSimple agentsNetworkServersMonday, July 16, 12Smart collectorInternetAnalyzer

ArchitectureSimple agentsNetworkServersVirtual switchesMonday, July 16, 12Smart collectorInternetAnalyzer

ArchitectureSimple agentsNetworkServersVirtual switchesVirtual serversMonday, July 16, 12Smart collectorInternetAnalyzer

ArchitectureSimple agentsNetworkServersVirtual switchesVirtual serversApplicationsMonday, July 16, 12Smart collectorInternetAnalyzer

Cross-layer correlation: Application, Host andNetworkStandard measurements fromdifferent sources designed tobe joinable at the collectore.g. application response time increase correlated directly to congestion on network path13Monday, July 16, 12

host-sflow.sourceforge.net14Monday, July 16, 12

sFlow-APPLICATIONExamples:NFS/CIFS transactions (file path, bytes, response-time, socket)HTTP requests (URL, user-agent, mime-type, bytes, response-time, socket)Memcached lookups (key, value-bytes, hit/miss, socket)Database queries (query#, response-time, socket)Application layer measurements much more valuable when correlated with performance ofevery component in underlying infrastructure!15Monday, July 16, 12

Host sFlow distributed APIsflowovsdovs-vsctlOpen vSwitchhsflowdDNS-SD/etc/hsflowd.confMonday, July 16, 12sFlowcollectorDNSserver

Host sFlow - JSON API{"flow sample:{"app name":"myapp","sampling rate": 100"app operation": {"operation":"task.start","attributes":"id 123&user root","status descr":"OK","status":0,"req bytes":43,"resp bytes":234,"uS":2000},"app initiator": {"actor":"123"},"app target": {"actor":"231"},"extended socket ipv4": {"protocol":6,"local ip":"10.0.0.1","remote ip":"10.0.0.23","local port":123,"remote port":43032}}}Monday, July 16, 12 UDP msg to hsflowd onlocalhost:36343 most fields optional sampling in app. or hsflowd counters in app. or hsflowd hsflowd sends binary sFlowAPPLICATION feed toconfigured collectors.

XenMotion bandwidthMonday, July 16, 12

Application throughput and response timeMonday, July 16, 12

Application throughput and response timeDrop in throughputduring XenMotion ofMemcache clientMonday, July 16, 12

sFlow-APPLICATION example: transactions20Monday, July 16, 12

sFlow-APPLICATION example: transactiondetail21Monday, July 16, 12

sFlow-APPLICATION example: latency22Monday, July 16, 12

Why Monitor Everything?1. Troubleshooting - always have contextwherewhowhatwhen23Monday, July 16, 12

Why Monitor Everything?1. Troubleshooting - always have contexttrace pathlocate hosts24Monday, July 16, 12

Why Monitor Everything?2. Put Network and Server teams on same pageInternetServerStorageControlNetwork3. Full “Observability” required for automated controlMay 09, 2011Monday, July 16, 12Copyright InMon Corp. 2010 All Rights Reserved25

OpenFlow sFlow Adaptive ControlMonday, July 16, 12

OpenFlow sFlow Adaptive ControlMonitorMonday, July 16, 12

OpenFlow sFlow Adaptive ControlMonitor Detailed, low-latency measurements from sFlow allows OpenFlowcontroller to adapt network to changing traffic patterns (load balancing,DDoS mitigation etc.). OpenFlow can be optimized for efficiency (e.g. by using wildcards), sFlowprovides visibility to detect and manage large flows.Monday, July 16, 12

ApplicationApplicationApplicationControl PlaneNetwork OSOpen APIsData PlaneHostsOpen APIsConfigurationNETCONF/OF-ConfigMonday, July 16, 12ForwardingVisibility

sFlow vs NetFlow/IPFIXNetFlow/IPFIXsampledecodehashflow cacheflushsendpacketssamplesendi/f countersMonday, July 16, 12poll

sFlow vs NetFlow/IPFIX : system-wide NetFlow/IPFIXPacketsNetFlow/IPFIXCollector Limited fields Large, unpredictablelatencyApplications: TCP/IP Accounting Security (associations)Monday, July 16, 12

sFlow vs NetFlow/IPFIX : system-wide sFlowPacketsCollector All fields All protocols All encapsulations Sub-second latency De-synchronized I/F counters tooApplications: Almost anything(software defined)Monday, July 16, 12

More ay, July 16, 12blog.sflow.com

Jul 16, 2012 · sFlow vs NetFlow/IPFIX : system-wide sFlow Collector All ﬁelds All protocols All encapsulations Sub-second latency De-synchronized I/F counters too Packets Applications