WIXLIB

USE IP TABLES TODETECT ANDPRE VENTSystem administrators need to stay ahead of newsecurity vulnerabilities that leave their networks exposedevery day. A firewall and an intrusion detection system(IDS) are two important weapons in that fight, enablingyou to proactively deny access and monitor networktraffic for signs of an attack. Tools for visualizing iptables logsLinux Firewalls discusses the technical details of theiptables firewall and the Netfilter framework that arebuilt into the Linux kernel, and it explains how theyprovide strong filtering, Network Address Translation(NAT), state tracking, and application layer inspectioncapabilities that rival many commercial tools. You’lllearn how to deploy iptables as an IDS with psad andfwsnort and how to build a strong, passive authentication layer around iptables with fwknop.If you’re responsible for keeping a network secure,you’ll find Linux Firewalls invaluable in your attempt tounderstand attacks and use iptables—along with psadand fwsnort—to detect and even prevent compromises.Concrete examples illustrate concepts such as firewalllog analysis and policies, passive network authentication and authorization, exploit packet traces, Snortruleset emulation, and more with coverage of: Application layer attack detection with the iptablesstring match extension and fwsnort Building an iptables ruleset that emulates a Snort ruleset Passive OS fingerprinting with iptablesPerl and C code snippets offer practical examplesthat will help you to maximize your deployment ofLinux firewalls.ABOUT THE AUTHORMichael Rash is a security architect with EnterasysNetworks, Inc., where he develops the Dragonintrusion detection and prevention system. He is afrequent contributor to open source projects and thecreator of psad, fwknop, and fwsnort. Rash is anexpert on firewalls, intrusion detection systems, passiveOS fingerprinting, and the Snort rules language. He isco-author of Snort 2.1 Intrusion Detection (Syngress,2004) and author of Intrusion Prevention and ActiveResponse (Syngress, 2005), and he has writtensecurity articles for Linux Journal, Sys Admin magazine, and ;login:. Port knocking vs. Single Packet Authorization (SPA)“ I L AY F L AT .”This book uses RepKover — a durable binding that won’t snap shut.ATTACK DETECTION A N D RESPONSE W I T HI P T A B L E S ,P S A D ,A N DF W S N O R TMICHAEL R ASHLinux Firewalls is a great book.— From the foreword by Richard Bejtlichof TaoSecurity.com 49.95 ( 59.95 CDN)SHELVE IN:COMPUTER SECURITY/NETWORKINGw w w.nostarch.com LINUXFIREWALLSR ASHT H E F I N E ST I N G E E K E N T E RTA I N M E N T LINU X FIRE WA LL SN E T W O R K- B A S E DAT TACKS Printed on recycled paperwww.it-ebooks.info

www.it-ebooks.info

fire PRAISE.fm Page i Wednesday, April 9, 2008 5:18 PMPRAISE FOR LINUX FIREWALLS“Right from the start, the book presented valuable information and pulled me in.Each of the central topics were thoroughly explained in an informative, yetengaging manner. Essentially, I did not want to stop reading.”–SLASHDOT“What really makes this book different from the others I’ve seen over the yearsis that the author approaches the subject in a layered method while exposingpotential vulnerabilities at each step. So for those that are new to the securitygame, the book also takes a stab at teaching the basics of network security whileteaching you the tools to build a modern firewall.”–INFOWORLD“This admirable, eminently usable text goes much further than advertised.”–LINUX USER AND DEVELOPER“This well-researched book heightens an average system administrator’sawareness to the vulnerabilities in his or her infrastructure, and the potentialto find hardening solutions.”–FREE SOFTWARE MAGAZINE“If you or anyone you know is responsible for keeping a secure network, LinuxFirewalls is an invaluable resource to have by your side.”–LINUXSECURITY.COM“If you’re building a Linux firewall and want to know what all the bells andwhistles are, when you might want to set them off, and how to hook themtogether, here you go.”–;LOGIN“If you run one or more Linux based firewalls, this book will not only help you toconfigure them securely, it will help you understand how they can be monitoredto discover evidence of probes, abuse and denial of service attacks.”–RON GULA, CTO & CO-FOUNDER OF TENABLE NETWORK SECURITYwww.it-ebooks.info

www.it-ebooks.info

LINUX FIRE WALLSAttack Detection andResponse with iptables,psad, and fwsnortby Mi cha el R as h San Franciscowww.it-ebooks.info

fire TITLE COPY.fm Page iv Monday, April 14, 2008 10:48 AMLINUX FIREWALLS. Copyright 2007 by Michael Rash.All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic ormechanical, including photocopying, recording, or by any information storage or retrieval system, without the priorwritten permission of the copyright owner and the publisher.Printed on recycled paper in the United States of America11 10 09 0823456789ISBN-10: 1-59327-141-7ISBN-13: 978-1-59327-141-1Publisher: William PollockProduction Editor: Christina SamuellCover and Interior Design: Octopod StudiosDevelopmental Editor: William PollockTechnical Reviewer: Pablo Neira AyusoCopyeditors: Megan Dunchak and Bonnie GranatCompositors: Christina Samuell and Riley HoffmanProofreaders: Karol Jurado and Riley HoffmanIndexer: Nancy GuentherFor information on book distributors or translations, please contact No Starch Press, Inc. directly:No Starch Press, Inc.555 De Haro Street, Suite 250, San Francisco, CA 94107phone: 415.863.9900; fax: 415.863.9950; info@nostarch.com; www.nostarch.comLibrar y of Congress Cataloging-in-Publication DataRash, Michael.Linux firewalls : attack detection and response with iptables, psad, and fwsnort / Michael Rash.p. cm.Includes index.ISBN-13: 978-1-59327-141-1ISBN-10: 1-59327-141-71. Computers--Access control. 2. Firewalls (Computer security) 3. Linux. I. Title.QA76.9.A25R36 2007005.8--dc222006026679No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product andcompany names mentioned herein may be the trademarks of their respective owners. Rather than use a trademarksymbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to thebenefit of the trademark owner, with no intention of infringement of the trademark.The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has beentaken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to anyperson or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by theinformation contained in it.www.it-ebooks.info

To Katie and little Bellawww.it-ebooks.info

www.it-ebooks.info

BRIEF CONTENTSAcknowledgments .xvForeword by Richard Bejtlich . xviiIntroduction .1Chapter 1: Care and Feeding of iptables .9Chapter 2: Network Layer Attacks and Defense .35Chapter 3: Transport Layer Attacks and Defense .49Chapter 4: Application Layer Attacks and Defense .69Chapter 5: Introducing psad: The Port Scan Attack Detector .81Chapter 6: psad Operations: Detecting Suspicious Traffic .99Chapter 7: Advanced psad Topics: From Signature Matching to OS Fingerprinting .113Chapter 8: Active Response with psad.131Chapter 9: Translating Snort Rules into iptables Rules .149Chapter 10: Deploying fwsnort .173Chapter 11: Combining psad and fwsnort .193Chapter 12: Port Knocking vs. Single Packet Authorization .213Chapter 13: Introducing fwknop .231Chapter 14: Visualizing iptables Logs.257Appendix A: Attack Spoofing .279Appendix B: A Complete fwsnort Script .285Index .291www.it-ebooks.info

www.it-ebooks.info

CONTENTS IN DETAILA CK N O W LE D G M E N T SxvF O R E W O R D b y R i c h a r d B e jt li c hI NT R O D U C T I O Nxvii1Why Detect Attacks with iptables? . 2What About Dedicated Network Intrusion Detection Systems? . 3Defense in Depth . 4Prerequisites . 4Technical References . 5About the Website . 5Chapter Summaries . 61C AR E A N D FE E D I N G O F IP T A B L E S9iptables . 9Packet Filtering with iptables . 10Tables . 11Chains . 11Matches . 12Targets . 12Installing iptables . 12Kernel Configuration . 14Essential Netfilter Compilation Options . 15Finishing the Kernel Configuration . 16Loadable Kernel Modules vs. Built-in Compilation and Security . 16Security and Minimal Compilation . 17Kernel Compilation and Installation . 18Installing the iptables Userland Binaries . 19Default iptables Policy . 20Policy Requirements . 20iptables.sh Script Preamble . 22The INPUT Chain . 22The OUTPUT Chain . 24The FORWARD Chain . 25Network Address Translation . 26Activating the Policy . 27iptables-save and iptables-restore . 27Testing the Policy: TCP . 29Testing the Policy: UDP . 31Testing the Policy: ICMP . 32Concluding Thoughts . 33www.it-ebooks.info

2N E TW O R K LA Y E R A TTA C KS AN D D E F E NS E35Logging Network Layer Headers with iptables . 35Logging the IP Header . 36Network Layer Attack Definitions . 38Abusing the Network Layer . 39Nmap ICMP Ping . 39IP Spoofing . 40IP Fragmentation . 41Low TTL Values . 42The Smurf Attack . 43DDoS Attacks . 44Linux Kernel IGMP Attack . 44Network Layer Responses . 45Network Layer Filtering Response . 45Network Layer Thresholding Response . 45Combining Responses Across Layers . 463TR AN S P O R T L A Y E R AT TA CK S A N D DE F E N S E49Logging Transport Layer Headers with iptables . 50Logging the TCP Header . 50Logging the UDP Header . 52Transport Layer Attack Definitions . 52Abusing the Transport Layer . 53Port Scans . 53Port Sweeps . 61TCP Sequence Prediction Attacks . 61SYN Floods . 62Transport Layer Responses . 62TCP Responses . 62UDP Responses . 66Firewall Rules and Router ACLs . 674A PP L IC A T I O N L A Y E R AT T A CK S A N D D E F E N S E69Application Layer String Matching with iptables . 70Observing the String Match Extension in Action . 70Matching Non-Printable Application Layer Data . 71Application Layer Attack Definitions . 72Abusing the Application Layer . 73Snort Signatures . 74Buffer Overflow Exploits . 74SQL Injection Attacks . 76Gray Matter Hacking . 77Encryption and Application Encodings . 79Application Layer Responses . 80xC on t en ts in D et ai lwww.it-ebooks.info

5I NT R O D U C I N G P S AD :THE PORT SCAN ATTACK DETECTOR81History . 81Why Analyze Firewall Logs? . 82psad Features . 83psad Installation . 83psad Administration . 85Starting and Stopping psad . 85Daemon Process Uniqueness . 86iptables Policy Configuration . 86syslog Configuration . 88whois Client . 89psad Configuration . 90/etc/psad/psad.conf . 90/etc/psad/auto dl . 96/etc/psad/signatures . 96/etc/psad/snort rule dl . 97/etc/psad/ip options . 97/etc/psad/pf.os . 97Concluding Thoughts . 986P SA D O P E R A T I O N S : D E T E CT IN G S US P I CI O U S T R AF F IC99Port Scan Detection with psad . 100TCP connect() Scan . 101TCP SYN or Half-Open Scan . 103TCP FIN, XMAS, and NULL Scans . 105UDP Scan . 106Alerts and Reporting with psad . 108psad Email Alerts . 108psad syslog Reporting . 110Concluding Thoughts . 1127A D V A NC E D P S AD T O PI C S : F R O M S I G N A T U R EM A T C HI N G T O O S F IN G E R P R I N T I NG113Attack Detection with Snort Rules . 113Detecting the ipEye Port Scanner . 115Detecting the LAND Attack . 116Detecting TCP Port 0 Traffic . 116Detecting Zero TTL Traffic . 117Detecting the Naptha Denial of Service Attack . 117Detecting Source Routing Attempts . 118Detecting Windows Messenger Pop-up Spam . 118psad Signature Updates . 119OS Fingerprinting . 120Active OS Fingerprinting with Nmap . 120Passive OS Fingerprinting with p0f . 121C on t en ts in D et ailwww.it-ebooks.infoxi