ISP Edge DesignISP Edge Design - Cisco PDF Free Download

2y ago

105 Views

1 Downloads

3.74 MB

61 Pages

Report/dmca

Download PDF

Transcription

Categorising ISPsGlobal ISP Global ISPGlobal ISPGlobal ISPRegional ISPRegional ISPRegional ISPRegional ISPIXPAccess ISPIXPAccess ISPAccess ISP 2006 Cisco Systems, Inc. All rights reserved.Access ISPAccess ISPAccess ISPCisco Internal3

Peering and Transit TransitCarrying traffic across a networky for a feeUsuallyExample: Access provider connects to a regional provider PeeringExchanging routing information and trafficUsually for no feeSometimes called settlement free peeringExample: Regional provider connects to another regionalprovider 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal4

Private Interconnect Two ISPs connect their networks over a private linkCan be peering arrangementg for trafficNo chargeShare cost of the linkCan be transit arrangementO ISP chargesOnehtheth otherth forf traffict ffiOne ISP (the customer) pays for the linkISP 1 2006 Cisco Systems, Inc. All rights reserved.ISP 2Cisco Internal5

Public Interconnect Several ISPs meeting in a common neutral location andinterconnect their networksUsually is a peering arrangement between their networksISP 1ISP 6ISP 3IXPISP 5 2006 Cisco Systems, Inc. All rights reserved.ISP 2Cisco InternalISPS 46

IXP (Internet eXchange Point)A physical network infrastructure operated by a singleentity with the purpose to facilitate the exchange ofInternet traffic between Autonomous Systems. Theyconnected should atnumber of Autonomous Systemsleast be three and there must be a clear and openpolicy for others to join. High-speed/Low-cost Internet Traffic Exchange A.k.a.A k PublicP bli PPeeringi or SSettlement-FreettltFPeeringP i Non-Profit Associations or Commercial Datacenters Around 300 big IXPs in the world 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal8

Euro IXEuro-IXEuro-IX (European Internet Exchange Association)was formed in May 2001 with the intention to furtherdevelop, strengthen and improve the Internet ExchangeyPoint ((IXP)) community 105 IXPs in 102 citiesin 31 countries 9 non-europeannon european members www.euro-ix.net 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal12

Example: GoogleNet A PortalNetPortalNet Dedicated CDNCDN Parallel Internet BackBoneGoogleNet (Faster, Cheaper, More Reliable)DataCenters can be colocated at Peering Points10GN*10G40G100GN*100GIX IX IX IX IX Tier2 ISP’sS ’ invest in massive LocalLoop upgrades to support IPTV.Upgrades/ UsersCisco Internal Google peers locally, often on aSettlement Free Basis, withE b ll CEyeballCarriers.i Google distributes it’s DataCentersto be virtually ONnet to Eyeballnetworks. Google is now only afew Hops away from Any User onthe Internet.IPTV Local Loop 2006 Cisco Systems, Inc. All rights reserved. Google builds it’s own worldwideIP Backbone. Google can send any amount oftraffic into the Internet withoutpaying anyone, they are Nobody’sCustomer.Some 300 Exchanges WorldwideGoogle-WIFI Google has been buying Fiber ona Worldwide basisMobile Google drives Net Neutrality sothat whatever Traffic they send,can’t be impaired. Google can now addressesService Substitution (Google TV,Voice )16

ISP designg –ppeeringg layeryINTERNETUpstream ISP’sIXPeBGPNationalIGWIPv4 RouteReflectorsInternationalIGWiBGPPMPLS CoreISP TransitRouterseBGPISP Customers 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal22

Cisco Internet Gateway RoutersThroughputScalabilityFIB entriesNetflow entriesASR 1000CRS-1/4CRS-1/8CRS-1/16CRS-1 MC20 Gbps40 Gbps2 Million2 Million320 Gbps960 Gbps2 Million4 Million640 Gbps1.92 Tbps2 Million8 Million1.28 Tbps3.84 Tbps2 Million16 Million10 Tbps100 Tbps2 Million100 MillionExistingEi ti deploymentsd lt ( 60%( 60% marketshare)k t h) The most used ISP GW is Cisco 12000 (GSR) Many deployments are based on Cisco 7600 Many small IGW’s are still Cisco 7200 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal26

IGW – Essential Feature setBroad LAN and WAN interfaces support international links – POS STM-1/4/16/64 national links – GE, 10GE, future full-rate 100GEIPv4 and IPv6 Routing and Forwarding 2M hardware entries (IPv4 IPv6) – no compression tricks! BGP, OSPF/ISIS, BFD – fast, prefixprefix-independentindependent convergenceIPv4 and IPv6 filters (access-lists) thousands of L3/L4 entries (IPv4 IPv6) – no impact on forwarding rate! loose uRPF (Unicast Reverse Path Forwarding)IPv4 and IPv6 netflow monotoring at least 1:1000 sampling rate, V9 exportDDoS attack protection and Control Plane protection in-hardware protection of router’s brainanti-hackinghacking tools – management plane protection anti 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal27

IGW – some optional featuresMPLS support rarely used on IGW, but sometimes yes MPLS Netflow is requiredqtooTraffic Shaping with RED – per-interface or per-VLAN if the circuit runs over MAN or ISP subrate service shaping prevents unnecessary drops and improves TCP goodputAccounting BGP Policy Accounting – per-AS accounting for large networks BGP Policy Propagation – packet marking based on BGP Communities MAC accounting – for peering/transit via IXPSecure Virtualization of the router Logical Routers with secure resources allocationCarrier Grade NAT IPv4 exhaustion is close! large scale IPv4 NAT and IPv6 AFT with V6 Tunneling is desirableLI (Lawful Intercept) if used as a ISP Transit, LI may be mandatory 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal28

Anti-spoofingRFC2827/BCP38 Ingress Packet FilteringAnti-spoofing filter (ingress filter on source IP)allow only source addresses from the customer’s 96.0.X.X/24RFC2827 and RFC3704 (BCP 38 and 84)Bogon filter (ingress filter on destination IP)Drops packetsDk t withith “insane”“i”ddestinationti ti IP addressddRFC1918, own block, internal IP core, ISP’s CustomerAllocationoca o Block:oc96.0.0.0/1996.0.18.0/24Anti-spoofing Filter Appliedingress on DownstreamAggregation or NAS Routers 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal30

uRPF (Unicast Reverse Path Forwarding)“StrictStrict Mode”Mode (v1) and “LooseLoose Mode”Mode (v2)router(config-if)# ip verify unicast source reachable-via rxi/f 2i/f 1i/f 2i/f 31S D i/fdatai/f 11S D i/fdataFIB:.S - i/f 1D - i/f 3.i/f 3“Strict Mode”(aka “v1”)FIB:.S - i/f 2D - i/f 3.Same i/f:FORWARDOther i/f:DROProuter(config-if)# ip verify unicast source reachable-via anyi/f 2i/f 1i/f 2i/f 31S D i/fdatai/f 11S D i/fdataFIB:.S - i/f xD - i/f 3.Any i/f:FORWARD 2006 Cisco Systems, Inc. All rights reserved.i/f 3“Loose Mode”(aka “v2”)v2 )FIB:.D - i/f 3.?Src not in FIBor route null0:DROPCisco Internal31

Bogons A Bogon prefix is a route that should never appear in the Internetrouting table Different from DSUA.Bogons are defined as Martians (private and reserved addressesdefined by RFC 1918 and RFC 3330) and netblocks that have notbeen allocated to a (RIR) by IANA CYMRU maintains list of Bogons, works with IANA and RIR etc. http://www.cymru.com/Bogons/index.html BOGON List Keeps on Changing as IANA allocates routes.BE AWARE!The bogon prefixes are announced unaggregated by the bogon routeservers is 65333:888; as of 14 JUL 2008 this includes 45 prefixes BOGON Router Server.Peer with CYMRU Route Server keep BOGON list upto date. 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal32

Hardware protection against DOS attacksCRS-1CRS1 Control Plane ProtectionCPUInput processesRPCoPPCSAR queueIngress LCCPUTo RPqueueraw queues4: Multiple queues toLC and RP CPU3 LPTS in3:i iFIB policeli traffict ffiASIC2b: Skip LC CPU!2a: LPTS iFIB lookup (Match, BTSH/GTSM)1: Ingress iACL, uRPF 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal33

IOS XR – Dynamic Control Plane ProtectionRouter bgpneighbor 202.4.48.99 ttl security!mpls ldp !LC 1 PreIFIB TCAM HW TSLocalbgpldpLC 2 PreIFIB TCAM HW Entries 2006 Cisco Systems, Inc. All rights reserved.Cisco InternalTCP Handshake34

Netflow is a Security tool #1 today!7 Keys define a flowSource AddressAddress, Destination AddressAddress, SourcePort, Destination Port, Layer 3 Protocol Type,TOS byte (DSCP), Input Logical Interface((ifIndex))A flow is unidirectionalTurning it on (generic):interface GigabitEthernet 1/1/1ip route-cache flow [sampled]Export (optional):ip flow-export destination 172.17.246.225 9995Sampled Netflow (mostly used for Security):ip flow-sampling-mode packet-interval x 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal36

Flow Is Defined By Seven Unique KeysTraffic Source IP address Destination IP addressEnable NetFlowNewSNMP MIBInterface Source port Destination port Layer 3 protocol type TOS byte (DSCP) Input logical interface (ifIndex)NetFlowNtFlExport PacketsTraditional Export &CollectorSNMP PollerGUI 2004CiscoSystems,Inc. All rights Ciscoreserved. 2006 Cisco Systems,Inc.All rightsreserved.Internal3737

NetFlow Cache Example1. Create and update flows in NetFlow 510.0.23.2104024.514 2. ExpirationInactive timer expired (15 sec is default)Active timer expired (30 min (1800 sec) is default)NetFlow cache is full (oldest flows are expired)RST or FIN TCP 1100000A2/24500A2/241510.0.23.21528180043. Aggregatione.g. Protocol-Port AggregationScheme Becomes4. Export version5. Transport protocol 2006 Cisco Systems, Inc. All rights reserved.ExportPacketCisco InternalHeaderNon-Aggregatedgg gFlows—Exportp Version 5 or 111100000A200A21528Aggregated Flows—Export Version 8 or 938

Netlow Export – V5 fixed formatUsage PacketP k t CountCt Byte CountTimeof Day Start sysUpTime End sysUpTimePortUtilizationQoSSSourceIPIPAddressAdd SourceSAddressAddDestination IPIPAddress DestinationAddress Source TCP/UDP Port Destination TCP/UDP Port Input ifIndex Output ifIndex NextN t HopH AddAddress Source AS Number Dest. AS Number Source Prefix Mask Dest. Prefix Maskyp of Service Type TCP Flags ProtocolFrom/ToApplicationRoutingandPeeringVersion 5 used extensively today 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal39

NetFlow Export – V9 flexible formatExample of Export Packet right after router boot or NetFlow configurationHeader(version,# packets,seq ence #sequence#,Source ID)Template FlowSetTemplateRecordTemplate IDTemplateRecordTemplate IDTemplateRecordTemplate IDTemplateRecordTemplate ID(specific Fieldtypes andlengths)(specific Fieldtypes andlengths)(specific Fieldtypes andlengths)(specific Fieldtypes andlengths)OptionTemplateFlowSetTemplate ID((specificifiField typesandlengths)Option DataFlowSetFlowSet ieldvalues)Example of Export Packets containing mostly flow informationHeader(version,# packets,sequence #,Source ID)Dataa a FlowSeto SeDataa a FlowSeto SeFlowSet IDFlowSet ues)((Fieldvalues) 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal40

BGP Next Hop TOS AggregationTypical ExampleAS1CustomersAS2AS3AS4AS5PEPEMPLS CoreorIP CoreCwithith BGP RRoutest OOnlylPEPEPEPEPoPPoPServer Farm 1CustomersServer Farm 2Internal Traffic: “PoP to PoP”External Traffic Matrix PoP to BGP AS 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal45

Customer is DOSedBefore – CoCo-LateralLateral DamagePeer AIXP-WIXPWAPeer BIXP-EUpstream AUpstreamABDCUpstreamBEUpstreamBTargetCustomersF POP 2006 Cisco Systems, Inc. All rights reserved.Attack causesCo-LateralDamageCisco InternalGNOC48

Customer is DOSedAfter – Packet Drops Pushed to the EdgePeer AIXP-WIXPWAPeer BIXP-EUpstream AUpstreamABDCUpstreamBETargetGF POP 2006 Cisco Systems, Inc. All rights reserved.Cisco InternalNOCUpstreamBiBGPAdvertisesList of BlackHoledPrefixes49

BGP Blackholing: Reacting to an AttackBGP Sent – 171.68.1.0/24 Next-Hop 192.0.2.1Static Route in Edge Router – 192.0.2.1 Null0171.68.1.0/24 192.0.2.1 Null0Next hop of 171.68.1.0/24 is nowto Null0equal Remote Triggered Black Hole filtering is the foundation for a whole seriesof techniques to traceback and react to DDOS attacks on an ISP’s network. Easy preparation, does not effect ISP operations or performance. It does adds the option to an ISP’s security toolkit. 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal50

BGP Blackholing: IOS configuration place a host-routehost route to Null on every BGP routerip route 192.0.2.1 255.255.255.255 Null0 prepare a injection into BGP with the blackhole next-hoprouter bgp 10redistribute static route-map set-blackholeroute-map set-blackhole permit 10match tag 666set ipi next-hoph192 0 2 1192.0.2.1set community 10:666 no-exportset local-preference 50 simply filter it out everywhere by one command:BH(config)# ip route 1.2.2.2 255.255.255.255 Null0 tag 666 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal51

BGP Blackholing: Filtering on source IPaddress loose uRPF (unicast reverse path forwarding)ip route 192.0.2.2 255.255.255.255 Null0int PoS 1/0/0ip verify unicast source reachable-via any!!! packet with source IP prefix pointing to Null0 will be dropped !!! prepare a injection into BGP with the blackhole nextnext-hophoproute-map set-blackhole permit 20match tag 667set ip next-hopnext hop 192.0.2.2set community 10:667 no-exportset local-preference 50 simply filter it out everywhere by one command:BH(config)# ip route 1.2.2.3 255.255.255.255 Null0 tag 667 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal52

BGP Triggered Rate LimitingQPPB (QoS Policy Propagation via BGP)router bgp 10table-map DOS-Activateneighbor 200200.200.14.4200 14 4 remote-asremote as 10neighbor 200.200.14.4 update-source Loopback 0neighbor 200.200.14.4 send-community!ip bgp-community new-format!ip community-list 1 permit 10:666!route-map DOS-Activate permit 10match community 1set ip qosqos-groupgroup 66!route-map DOS-Activate permit 20!interface PoS 0/0/0bbgp-policylisource iip-qos-maprate-limit input qos-group 66 256000 8000 8000conform-action transmitexceed-action drop QPPB marking is done before rate-limit or policing hardware support in Cisco 10000, 12000, CRS-1 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal53

Default Route & the InternetBHole(config-router)# default-information originate always Advertising Default from the SinkHole will pull down all sort of junktraffic.RouterAdvertisesDefaultCustomer Traffic when circuits flap.Network ScansFailed AttacksCode Red/NIMDABackscatter Can place tracking tools (Netflowcache) and IDS in the Sink Holenetwork to monitor the noise. BCP: Default should be always ablackhole (Null0 or Static ARP) !! 2006 Cisco Systems, Inc. All rights reserved.Cisco InternalSink HoleNetworkCustomers172.168.20.0/24 – target’s network172.168.20.1 is attacked55

Target Routers are Expendable# ip route 0.0.0.0 0.0.0.0 192.0.2.253# arp 192.0.2.253 0007.ecbd.e000 arpaTo ISP BackboneSink Hole GatewayTarget RouterTo ISPBackboneSniffers andAnalyzersTo ISP Backbone Sink Hole Gateway Generates the more specific iBGPAnnouncement. Pull the DOS/DDOS attack to the sink hole and forwards theattack to the target router. StStaticti ARP tot theth targettt routert keepsktheth SinkSi k HoleH l OperationalOtil–Target Router can crash from the attack and the static ARP willkeep the gateway forwarding traffic to the ethernet switch. 2006 Cisco Systems, Inc. All rights reserved.Cisco Internal56

What to Monitor in a Sinkhole? Scans on dark IP (allocated and announced butunassigned address space)Who is scoping out the network—pre-attack planning, worms Scans on bogons (unallocated)Worms, infected machines, and Bot creation Backscatter from spoofe

A physical network infrastructure operated by a single entity with the purpose to facilitate the exchange of Internet traffic between Autonomous Systems. The . ISP design – ppgyeering layer Upstream ISP’s INTERNET IXP eBGP International IGW National IGW IPv4 Route P