Transcription
Designing ISP ArchitecturesSun Microsystems, Inc.901 San Antonio RoadPalo Alto, CA 94303-4900U.S.A. 650-960-1300Part No. 816-0917-10March 2002, Revision 01Send comments about this document to: docfeedback@sun.com
Copyright 2002 Sun Microsystems, Inc., 901 San Antonio Road Palo Alto, CA 94303-4900 USA. All rights reserved.This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation.No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors,if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark inthe U.S. and other countries, exclusively licensed through X/Open Company, Ltd. For Netscape Communicator , the following notice applies:Copyright 1995 Netscape Communications Corporation. All rights reserved.Sun, Sun Microsystems, the Sun logo, AnswerBook2, docs.sun.com, SunDocs, Solaris, Sun BluePrints, UNIX, Solaris Resource Manager, SolarisBandwidth Manager, iPlanet Directory Server, Solstice Backup, Netra t1, Sun Fire 280R, Sun StorEdge, Sun Management Center, SolsticeDiskSuite, iPlanet Web Server, iPlanet Application Server, Java, and JavaScript are trademarks, registered trademarks, or service marks of SunMicrosystems, Inc. in the U.S. and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarksof SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC trademarks are based upon an architecture developed bySun Microsystems, Inc.The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledgesthe pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sunholds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun’s licensees who implement OPENLOOK GUIs and otherwise comply with Sun’s written license agreements.RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) andFAR 52.227-19(6/87), or DFAR 252.227-7015(b)(6/95) and DFAR 227.7202-3(a).DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.Copyright 2002 Sun Microsystems, Inc., 901 San Antonio Road Palo Alto, CA 94303-4900 Etats-Unis. Tous droits réservés.Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et ladécompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sansl’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en a. Le logiciel détenu par des tiers, et qui comprend la technologierelative aux polices de caractères, est protégé par un copyright et licencié par des fournisseurs de Sun.Des parties de ce produit pourront être dérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marquedéposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd. La notice suivante est applicable àNetscape Communicator : Copyright 1995 Netscape Communications Corporation. Tous droits réservés.Sun, Sun Microsystems, the Sun logo, AnswerBook2, docs.sun.com, SunDocs, et Solaris docs.sun.com, Solaris, Sun BluePrints,UNIX, SolarisResource Manager, Solaris Bandwidth Manager, iPlanet Directory Server, Solstice Backup, Netra t1, Sun Fire 280R, Sun StorEdge, SunManagement Center, Solstice DiskSuite, iPlanet Web Server, iPlanet Application Server, Java, and JavaScript sont des marques de fabrique oudes marques déposées, ou marques de service, de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays. Toutes les marques SPARC sontutilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARC International, Inc. aux Etats-Unis et dans d’autrespays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc.L’interface d’utilisation graphique OPEN LOOK et Sun a été développée par Sun Microsystems, Inc. pour ses utilisateurs et licenciés. Sunreconnaît les efforts de pionniers de Xerox pour la recherche et le développement du concept des interfaces d’utilisation visuelle ou graphiquepour l’industrie de l’informatique. Sun détient une licence non exclusive de Xerox sur l’interface d’utilisation graphique Xerox, cette licencecouvrant également les licenciés de Sun qui mettent en place l’interface d’utilisation graphique OPEN LOOK et qui en outre se conforment auxlicences écrites de Sun.CETTE PUBLICATION EST FOURNIE "EN L’ETAT" ET AUCUNE GARANTIE, EXPRESSE OU IMPLICITE, N’EST ACCORDEE, Y COMPRISDES GARANTIES CONCERNANT LA VALEUR MARCHANDE, L’APTITUDE DE LA PUBLICATION A REPONDRE A UNE UTILISATIONPARTICULIERE, OU LE FAIT QU’ELLE NE SOIT PAS CONTREFAISANTE DE PRODUIT DE TIERS. CE DENI DE GARANTIE NES’APPLIQUERAIT PAS, DANS LA MESURE OU IL SERAIT TENU JURIDIQUEMENT NUL ET NON AVENU.PleaseRecycle
AcknowledgementsAs with any major publication, the result is an aggregation of effort andcollaboration from many contributors, both directly and indirectly. In particular, thisbook is the result of a major effort from the Advanced Internet Practice (SunProfessional Services) and the Sun BluePrints (Enterprise Engineering) groupswithin Sun Microsystems. The outcome is a collective knowledge of designprinciples and preferred practices based upon professional experiences, gained fromdesigning very large-scale solutions for service providers.Special thanks to my manager Dan Berg (Director, Advanced Internet Practice) andBill Sprouse (Manager, Enterprise Engineering) for the opportunity with thisinitiative. Additionally, special thanks to Barbara Jugo (Publications Manager, SunBluePrints), Jeff Wheelock (Manager, Sun BluePrints), and Charles Alexander(Director, Enterprise Engineering) for their support in keeping things in perspective.Thanks to Meredith Rose (Systems Engineer, Foundry) for technical assistance withnetwork design using Foundry products, Nikki Kester (Regional Sales Manager,Foundry) for support and resources, and Robert Cosme (Systems Engineer, CiscoSystems) for technical assistance with network design using Cisco products.Thanks to all reviewers for their insights and comments: Tom Bialaski (EnterpriseEngineering), Kirk Brown (Advanced Internet Practice), Jason Carolan (AdvancedInternet Practice), Ron Cotten (Advanced Internet Practice), David Deeths(Enterprise Engineering), Alex Noordergraaf (Enterprise Engineering), and KemerThomson (Enterprise Engineering).Finally, last but not least, a very special thanks to Rex Casey (Senior Technical Writer,Sun BluePrints) for spending countless hours in translating my complex thoughtsand technical jargon to something that is readable and understandable. Specialthanks to Terry Williams (Technical Writer, Sun BluePrints) and Dany Galgani(Graphic Designer, IPG Publications) for transforming napkin sketches into beautifulartwork; Tim Marsh (Enterprise Engineering) for his hard work in setting up theprototype in the lab; and Minerva Ontiveros (Enterprise Engineering) for handlingall the administrative logistics. Without all of you and your support, I could nothave done it with such an aggressive schedule.iii
ivDesigning ISP Architectures March 2002
ContentsAcknowledgementsPrefaceiiixxiAbout This BookxxiSun BluePrints ProgramxxiiWho Should Read This BookBefore You Read This BookIntroducing FijiNetxxiixxiiixxiiiHow This Book Is OrganizedxxiiiOrdering Sun DocumentationxxvAccessing Sun Documentation OnlineRecommended PublicationsUsing UNIX CommandsTypographic ConventionsxxvxxvixxvixxviiShell Prompts in Command ExamplesSun Welcomes Your Commentsxxviixxviiiv
1.Introduction1Defining an ISP and Its ArchitectureIdentifying Market TrendsValue Proposition23Market Positioning3Present and Future TrendsObtaining Market DataKey Challenges7Considerations869Alternative Approach2.56Challenges in Becoming an ISPRequirements210Formulating Design Requirements11Formulating Requirements for a DesignObtain Requirements1212Evaluate Requirements16Establish Assumptions17Determine Design Trade-Offs19Address Architectural Limitations25Formulating Requirements for FijiNet28Obtain Requirements for FijiNet28Evaluate Requirements for FijiNet31Establish Assumptions for FijiNet38Determine Design Trade-Offs for FijiNet40Address Architectural Limitations for FijiNetviDesigning ISP Architectures March 200241
3.Establishing an Architectural ModelUnderstanding the Model44Identifying Key ComponentsISP Services4545Operating EnvironmentOperating Platform5050Applying Architectural ance59Open System59Applying the Model to FijiNet60Identify Key Components for FijiNet60Apply Architectural Principles to FijiNet4.Creating a Logical Design65Creating a High-Level FrameworkIdentify High-Level Topology6666Identify Services Within the TopologyDefine Service Flows637086Define Networking Components96Creating a High-Level Framework for FijiNetIdentify High-Level Topology for FijiNet107107Identify Services Within the FijiNet Topology110Contentsvii
Define Service Flows for FijiNet113Define Networking Components for FijiNet5.Creating a Physical Design123Creating a High-Level Network Design124Build a Network Design Diagram125Create IP Address SchemaPlanning Capacity127132Estimate Software CapacityEstimate Server Capacity133156Estimate Network Capacity157Creating a Network Design for FijiNet164Build a Network Design Diagram for FijiNetCreate IP Address Schema for FijiNetPlanning Capacity for FijiNet167Estimate Server Capacity for FijiNetSelecting Software167186Estimate Network Capacity for FijiNetSelecting Components164165Estimate Software Capacity for FijiNet6.121187193194Choose Software for Basic Services195Choose Software for Value-Added Services198Choose Software for Infrastructure Services200Choose Software for Operation and Management ServicesChoose an Operating EnvironmentviiiDesigning ISP Architectures March 2002205204
Selecting Servers206Determine Server Types206Choose Enterprise Servers207Choose Storage Equipment207Selecting Network Components210Choose Routers and SwitchesChoose Load BalancersChoose Firewalls210211211Choose Intrusion Detection System (IDS)Choose Console Servers213Choose Network Access Servers213Selecting Hardware Rack Equipment214Selecting Software for FijiNet212215Choose Software for FijiNet’s Basic Services215Choose Software for FijiNet’s Infrastructure Services217Choose Software for FijiNet’s Operation and Management ServicesChoose Operating Environment for FijiNetSelecting Servers for FijiNet221Choose Server Equipment for FijiNet222Choose Storage Equipment for FijiNet223Selecting Network Components for FijiNet225Choose Routers and SwitchesChoose Firewalls220221Determine Server Types for FijiNetChoose Load Balancers218226228228Choose Intrusion Detection SystemsChoose Console Servers229229Contentsix
Choose Network Access Servers230Selecting Hardware Rack Equipment for FijiNet7.Implementing a Solution233Implementing a Prototype234Implementing a Design235Apply an Incremental Approach235Test and Optimize Your ImplementationImplementing an Operating PlatformImplementing an Operating SystemImplementing Infrastructure ServicesImplementing Basic Services237237238240240Usage Pattern ChangesTechnology Changes241241Business Strategy ChangesA.241Questions for Obtaining Design RequirementsGeneral QuestionsSupport Questions244244Systems and Network Management QuestionsEnd-User Questions245Registration Questions245Customer Care QuestionsBilling System Questions246246Service Availability QuestionsSecurity Questions243244Business-Related Questionsx236239Implementing Value-Added ServicesAdapting to Changes231247Designing ISP Architectures March 2002246245
Demographic Questions248Networking QuestionsDial-up Questions249Directory QuestionsEmail Questions249250250Web Hosting Questions251Search Engine Questions252Caching Proxy Questions253Internet Relay Chat QuestionsFTP Questions253254Internet News Questions254Development and Staging QuestionsB.Sample Network ConfigurationsCisco 2651 Router257258Cisco 3512-XL SwitchCisco PIX 525 Firewall260262Cisco AS 5400 Access ServerC.264Cisco AS 2511 Console Server271Sample DNS Configurations273External DNS Configurations274Primary External Server274Secondary External Servers279Internal DNS Configurations282Primary Internal Server282Secondary Internal ServersD.255DHCP Server Configuration288291Contentsxi
E.NTP Server Configuration295F.DNS Benchmark Data for Sun Enterprise ServersBenchmark Data for BIND v8.1.2299Benchmark Data for BIND v8.2.2-P7Benchmark Data for BIND v9.1.0G.Network Capacity297300301303North American Digital Hierarchy304Committee of European Postal and Telephone HierarchySynchronous Digital HierarchyH.I.HTTP ThroughputPort and Protocol ListBibliographyIndexxii307335341Designing ISP Architectures March 2002309305304
FiguresFIGURE 3-1ISP Architectural Model 44FIGURE 3-2Basic Services 46FIGURE 3-3Value-Added Services47FIGURE 3-4Infrastructure Services48FIGURE 3-5Operation and Management ServicesFIGURE 3-6ISP Architectural Model 51FIGURE 4-1Generic ISP High-Level Topology 66FIGURE 4-2POP TopologyFIGURE 4-3Logical Network TopologyFIGURE 4-4DMZ NetworkFIGURE 4-5Services Network 75FIGURE 4-6Content Network 80FIGURE 4-7Staging Network 83FIGURE 4-8Management NetworkFIGURE 4-9DNS Service FlowFIGURE 4-10LDAP Service Flow88FIGURE 4-11DHCP Service Flow89FIGURE 4-12RADIUS Service Flow 91FIGURE 4-13NTP Service Flow49676971848792xiii
xivFIGURE 4-14Email Service FlowFIGURE 4-15Web Hosting Service Flow 94FIGURE 4-16News Service FlowFIGURE 4-17Hierarchical Network Components ModelFIGURE 4-18FijiNet TopologyFIGURE 4-19FijiNet POP TopologyFIGURE 4-20FijiNet Logical Network TopologyFIGURE 4-21FijiNet External NetworkFIGURE 4-22FijiNet Internal NetworkFIGURE 4-23FijiNet Management NetworkFIGURE 4-24FijiNet DNSFIGURE 4-25FijiNet LDAPFIGURE 4-26FijiNet DHCP Server 115FIGURE 4-27FijiNet RADIUS 116FIGURE 4-28FijiNet NTPFIGURE 4-29FijiNet Email ServiceFIGURE 4-30FijiNet Web Service 119FIGURE 4-31FijiNet News ServiceFIGURE 5-1Sample Network Design 126FIGURE 5-2Sample IP Address Schema for an ISP Infrastructure 131FIGURE 5-3Daily Number of News ArticlesFIGURE 5-4Storage Requirements for Full Feed 140FIGURE 5-5Future Storage Estimate 140FIGURE 5-6FijiNet Network Design 164FIGURE 5-7IP Address Schema for FijiNetFIGURE 6-1Sample Application Characteristics g ISP Architectures March 2002139166
TablesTABLE 1-1U.S. Residential High-Speed Access Growth and Trends (by Millions) 5TABLE 2-1FijiNet Business RequirementsTABLE 2-2FijiNet Functional Requirements 30TABLE 2-3FijiNet Business Requirements Evaluation 32TABLE 2-4FijiNet Functional Requirements EvaluationTABLE 2-5Evaluation of AssumptionsTABLE 2-6Determining Design Trade-Offs for FijiNetTABLE 3-1Scaling Model for Servers 52TABLE 3-2Availability LevelsTABLE 3-3Operating Environment for FijiNet 61TABLE 3-4Operating Platform for FijiNet 62TABLE 5-1Sample Address Masking at Core LayerTABLE 5-2Sample Address Masking at Distribution Layer 129TABLE 5-3Sample Address Masking at Access Layer 130TABLE 5-4Estimating Storage for Email Service134TABLE 5-5Estimating Memory for Email Service135TABLE 5-6Estimating Storage for Web Service 136TABLE 5-7Estimating Memory for Web Service 137TABLE 5-8Estimating Storage for News Service2935384054128139xv
xviTABLE 5-9Estimating Memory for News ServiceTABLE 5-10Estimating Storage for FTP Service142TABLE 5-11Estimating Memory for FTP Service142TABLE 5-12Estimating Storage for DNS Service143TABLE 5-13Estimating Memory for DNS Service144TABLE 5-14Estimating Storage for RADIUS Service144TABLE 5-15Estimating Memory for RADIUS Service145TABLE 5-16Estimating Storage for Directory Service146TABLE 5-17Estimating Memory for Directory Service146TABLE 5-18Estimating Storage for DHCP Service 147TABLE 5-19Estimating Memory for DHCP Service 148TABLE 5-20Estimating Storage for NTP Service148TABLE 5-21Estimating Memory for NTP Service149TABLE 5-22Estimating Storage for Backup Service 150TABLE 5-23Estimating Memory for Backup Service 150TABLE 5-24Estimating Storage for Host-Based Firewall Service 151TABLE 5-25Estimating Memory for Firewall Service 151TABLE 5-26Estimating Storage for Log Service152TABLE 5-27Estimating Memory for Log Service152TABLE 5-28Estimating Storage for System Disk 154TABLE 5-29Filesystem Layout for System Disk 155TABLE 5-30Filesystem Layout for Data155TABLE 5-31Sizing an Enterprise Server156TABLE 5-32Estimating Network Bandwidth for UsersTABLE 5-33Estimating Modems Needed for Dial-up AccessTABLE 5-34Estimating Links Needed for Internet Connectivity 159TABLE 5-35Estimating Links Needed for Dial-up AccessTABLE 5-36Estimating Ports for RoutersDesigning ISP Architectures March 2002161141157158160
TABLE 5-37Estimating Network Ports for Switches 162TABLE 5-38Console Port Estimation for Console Servers 163TABLE 5-39IP Address Schema for FijiNetTABLE 5-40FijiNet: Estimating Storage for Email Service168TABLE 5-41FijiNet: Estimating Memory for Email Service169TABLE 5-42FijiNet: Estimating Storage for Web Service 171TABLE 5-43FijiNet: Estimating Memory for Web Service 172TABLE 5-44FijiNet: Estimating Storage for News Service173TABLE 5-45FijiNet: Estimating Memory for News Service173TABLE 5-46FijiNet: Estimating Storage for FTP Service174TABLE 5-47FijiNet: Estimating Memory for FTP Service174TABLE 5-48FijiNet: Estimating Storage for DNS Service175TABLE 5-49FijiNet: Estimating Memory for DNS Service175TABLE 5-50FijiNet: Estimating Storage for RADIUS ServiceTABLE 5-51FijiNet: Estimating Memory for RADIUS Service 177TABLE 5-52FijiNet: Estimating Storage for Directory Service177TABLE 5-53FijiNet: Estimating Memory for Directory Service178TABLE 5-54FijiNet: Estimating Storage for DHCP Service178TABLE 5-55FijiNet: Estimating Memory for DHCP Service179TABLE 5-56FijiNet: Estimating Storage for NTP Service179TABLE 5-57FijiNet: Estimating Memory for NTP Service180TABLE 5-58FijiNet: Estimating Storage for Backup Service 181TABLE 5-59FijiNet: Estimating Memory for Backup Service 181TABLE 5-60FijiNet: Estimating Storage for Firewall Service 182TABLE 5-61FijiNet: Estimating Memory for Firewall Service 182TABLE 5-62FijiNet: Estimating Storage for Log Service 183TABLE 5-63FijiNet: Estimating Memory for Log Service 183TABLE 5-64FijiNet: Estimating Storage for System Disk 184165176Tablesxvii
xviiiTABLE 5-65Filesystem Layout for System Disk 185TABLE 5-66Filesystem Layout for DataTABLE 5-67FijiNet: Estimating Server SizingTABLE 5-68FijiNet: Estimating Network Bandwidth for UsersTABLE 5-69FijiNet: Estimating Modems for Dial-Up Access 188TABLE 5-70FijiNet: Estimating Links for Internet Connectivity 189TABLE 5-71FijiNet: Estimating Links for Dial-Up Access 189TABLE 5-72FijiNet: Estimating Ports for RoutersTABLE 5-73FijiNet: Estimating Network Ports for Switches 191TABLE 5-74FijiNet: Estimating Ports for Console ServerTABLE 6-1Mail Servers 195TABLE 6-2POP/IMAP Servers 196TABLE 6-3Web Servers 196TABLE 6-4News ServersTABLE 6-5UseNet Providers 197TABLE 6-6FTP ServersTABLE 6-7Application ServersTABLE 6-8Database ServersTABLE 6-9DNS Server
viii Designing ISP Architectures March 2002 Define Service Flows for FijiNet 113 Define Networking Components for FijiNet 121 5. Creating a Physical Design 123 Creating a High-Level Network Design 124 Build a Network Design Diagram 125 Create IP Address Schema 127 Planning Capacity 132 Estimate Software Capacity 133 Estimate Server Capacity 156
