Transcription
NEXT GENERATION FIREWALL TEST REPORTForcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5Author – Jeff Bowermon
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5OverviewNSS Labs performed an independent test of the Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5. Theproduct was subjected to thorough testing at the NSS facility in Austin, Texas, based on the Next GenerationFirewall (NGFW) Test Methodology v6.0 available at www.nsslabs.com. This test was conducted free of charge andNSS did not receive any compensation in return for Forcepoint’s participation. For additional information on NGFWtechnology, refer to the NSS Analyst Brief, What Do CIOs Need to Know About Next Generation Firewalls? 1While the companion Comparative Reports on security, performance, and total cost of ownership (TCO) willprovide information about all tested products, this Test Report provides detailed information not availableelsewhere.NSS research indicates that NGFW devices are typically deployed to protect users rather than data center assets,and that the majority of enterprises will not separately tune intrusion prevention system (IPS) modules within theirNGFWs. Therefore, during NSS testing, NGFW products are configured with the vendor’s pre-defined orrecommended (i.e., “out-of-the-box”) settings in order to provide readers with relevant security effectiveness andperformance dimensions based on their expected usage.ProductForcepoint Stonesoft NGFW 1402v5.8.5NSS ExploitLibrary BlockRate2NSS-TestedThroughput3-Year TCO(List Price)3-Year TCO(Street Price)98.6%2,642 Mbps 64,560 48,570Firewall ty andReliabilityPASSPASSPASSPASSFigure 1 – Overall Test ResultsUsing the recommended policy, the Stonesoft NGFW 1402 blocked 98.8% of attacks against server applications,98.5% of attacks against client applications, and 98.6% of attacks overall. The device proved effective against allevasion techniques tested. The device also passed all stability and reliability tests.The Stonesoft NGFW 1402 is rated by NSS at 2,642 Mbps, which is lower than the vendor-claimed performance;Forcepoint rates this device at 4 Gbps. NSS-Tested Throughput is calculated as an average of all of the “real-world”protocol mixes and the 21 KB HTTP response-based capacity test.1What Do CIOs Need to Know About Next Generation Firewalls? NSS Labs2 Defined asthe rate at which the device under test blocked exploits from the NSS Exploit Library. This value is a component of the overall blockrate, which is reported in the NSS Labs Security Value Map 2
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5Table of ContentsOverview. 2Security Effectiveness . 5Firewall Policy Enforcement .5Application Control .6NSS Exploit Library .6False Positive Testing .7Coverage by Attack Vector .7Coverage by Impact Type.7Coverage by Date.8Coverage by Target Vendor .8Resistance to Evasion Techniques .9Performance . 10Raw Packet Processing Performance (UDP Throughput) .10Raw Packet Processing Performance (UDP Latency) .11Maximum Capacity .11HTTP Capacity with No Transaction Delays .13Application Average Response Time – HTTP .13HTTP Capacity with Transaction Delays .14Real-World Traffic Mixes .14Stability and Reliability . 16Management and Configuration . 17Total Cost of Ownership (TCO) . 18Installation Hours .18List Price and Total Cost of Ownership .19Street Price and Total Cost of Ownership .19Detailed Product Scorecard . 20Test Methodology . 25Contact Information . 253
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5Table of FiguresFigure 1 – Overall Test Results.2Figure 2 – Firewall Policy Enforcement .5Figure 3 – Application Control .6Figure 4 – Number of Exploits Blocked (%) .6Figure 5 – Coverage by Attack Vector .7Figure 6 – Product Coverage by Date .8Figure 7 – Product Coverage by Target Vendor .8Figure 8 – Resistance to Evasion Results .9Figure 9 – Raw Packet Processing Performance (UDP Traffic) .10Figure 10 – UDP Latency in Microseconds .11Figure 11 – Concurrency and Connection Rates .12Figure 12 – HTTP Capacity with No Transaction Delays .13Figure 13 – Average Application Response Time (Milliseconds) .13Figure 14 – HTTP Capacity with Transaction Delays .14Figure 15 – Real-World Traffic Mixes.15Figure 16 – Stability and Reliability Results .16Figure 17 – Sensor Installation Time (Hours) .18Figure 18 – List Price 3-Year TCO .19Figure 19 – Street Price 3-Year TCO .19Figure 20 – Detailed Scorecard .244
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5Security EffectivenessThis section verifies that the device under test (DUT) is capable of enforcing the security policy effectively.Firewall Policy EnforcementPolicies are rules configured on a firewall to permit or deny access from one network resource to another based onidentifying criteria such as source, destination, and service. A term typically used to define the demarcation pointof a network where policy is applied is demilitarized zone (DMZ). Policies are typically written to permit or denynetwork traffic from one or more of the following zones: Untrusted – This is typically an external network and is consideredunknown and nonsecure. An example of an untrusted network wouldbe the Internet.DMZ – This is a network that is being isolated by the firewall restrictingnetwork traffic to and from hosts contained within the isolatednetwork.Trusted – This is typically an internal network; a network that isconsidered secure and protected.The NSS firewall tests verify performance and the ability to enforce policybetween the following: Trusted to UntrustedUntrusted to DMZTrusted to DMZNote: Firewalls must provide at a minimum one DMZ interface in order toprovide a DMZ or “transition point” between untrusted and trustednetworks.Test ProcedureResultBaseline PolicySimple PolicyComplex PolicyStatic NATDynamic/Hide NATSYN Flood ProtectionIP Address Spoofing ProtectionTCP Split Handshake SpoofPASSPASSPASSPASSPASSPASSPASSPASSFigure 2 – Firewall Policy Enforcement5
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5Application ControlAn NGFW must provide granular control based on applications, not just ports. This capability is needed toreestablish a secure perimeter where unwanted applications are unable to tunnel over HTTP/S. As such, granularapplication control is a requirement of an NGFW since it enables the administrator to define security policies basedon applications rather than on ports alone.Test ProcedureResultBlock Unwanted ApplicationsBlock Specific ActionsPASSPASSFigure 3 – Application ControlOur testing found that the Stonesoft NGFW 1402 correctly enforced complex outbound and inbound policiesconsisting of multiple rules, objects, and applications. NSS engineers verified that the device successfullydetermined the correct application and took the appropriate action based on the policy.NSS Exploit LibraryNSS’ security effectiveness testing leverages the deep expertise of our engineers who utilize multiple commercial,open-source, and proprietary tools, including NSS’ network live stack test environment 3 as appropriate. With 1999exploits, this is the industry’s most comprehensive test to date. Most notably, all of the exploits and payloads inthis test have been validated such that: A reverse shell is returnedA bind shell is opened on the target, allowing the attacker to execute arbitrary commandsArbitrary code executionA malicious payload is installedA system is rendered unresponsiveEtc.ProductForcepoint Stonesoft NGFW 1402v5.8.5Total Number ofExploits RunTotal NumberBlockedBlockPercentage1,9991,97298.6%Figure 4 – Number of Exploits Blocked (%)3See the NSS Cyber Advanced Warning System for more details.6
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5False Positive TestingThe Stonesoft NGFW 1402 correctly identified traffic and did not fire alerts for non-malicious content.Coverage by Attack VectorBecause a failure to block attacks could result in significant compromise and could severely impact critical businesssystems, NGFWs should be evaluated against a broad set of exploits. Exploits can be categorized as either attackerinitiated or target initiated. Attacker-initiated exploits are threats executed remotely against a vulnerableapplication and/or operating system by an individual, while target-initiated exploits are initiated by the vulnerabletarget. Target-initated exploits are the most common type of attack experienced by the end user, and the attackerhas little or no control as to when the threat is executed.Figure 5 – Coverage by Attack VectorCoverage by Impact TypeThe most serious exploits are those that result in a remote system compromise, providing the attacker with theability to execute arbitrary system-level commands. Most exploits in this class are “weaponized” and offer theattacker a fully interactive remote shell on the target client or server. Slightly less serious are attacks that result inan individual service compromise, but not arbitrary system-level command execution. Finally, there are attacksthat result in a system- or service-level fault that crashes the targeted service or application and requiresadministrative action to restart the service or reboot the system. Clients can contact NSS for more informationabout these tests.7
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5Coverage by DateFigure 6 provides insight into whether or not a vendor is aging out protection signatures aggressively enough topreserve performance levels. It also reveals whether a product lags behind in protection for the most currentvulnerabilities. NSS reports exploits by individual years for the past ten years. Exploits older than ten years aregrouped together.Figure 6 – Product Coverage by DateCoverage by Target VendorExploits within the NSS Exploit Library target a wide range of protocols and applications. Figure 7 depicts thecoverage offered by the Stonesoft NGFW 1402 for five of the top vendors targeted in this test. More than 70vendors are represented in the test. Clients can contact NSS for more information about this test.Figure 7 – Product Coverage by Target Vendor8
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5Resistance to Evasion TechniquesEvasion techniques are a means of disguising and modifying attacks at the point of delivery to avoid detection andblocking by security products. Failure of a security device to correctly identify a specific type of evasion potentiallyallows an attacker to use an entire class of exploits for which the device is assumed to have protection. Thisrenders the device virtually useless. Many of the techniques used in this test have been widely known for yearsand should be considered minimum requirements for the NGFW product category.Providing exploit protection results without fully factoring in evasion can be misleading. The more classes ofevasion that are missed (such as IP packet fragmentation, stream segmentation, RPC fragmentation, URLobfuscation, HTML obfuscation, payload encoding, and FTP evasion), the less effective the device. For example, itis better to miss all techniques in one evasion category, such as FTP evasion, than one technique in each category,which would result in a broader attack surface.Furthermore, evasions operating at the lower layers of the network stack (IP packet fragmentation or streamsegmentation) have a greater impact on security effectiveness than those operating at the upper layers (HTTP orFTP obfuscation.) Lower-level evasions will potentially impact a wider number of exploits; missing TCPsegmentation, for example, is a much more serious issue than missing FTP obfuscation.Figure 8 provides the results of the evasion tests for the Stonesoft NGFW 1402.Test ProcedureResultIP Packet FragmentationPASSStream SegmentationPASSRPC FragmentationPASSURL ObfuscationPASSHTML ObfuscationPASSPayload EncodingPASSFTP EvasionPASSIP Packet Fragmentation TCP SegmentationPASSFigure 8 – Resistance to Evasion Results9
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5PerformanceThere is frequently a trade-off between security effectiveness and performance. Because of this trade-off, it isimportant to judge a product’s security effectiveness within the context of its performance and vice versa. Thisensures that new security protections do not adversely impact performance and that security shortcuts are nottaken to maintain or improve performance.Raw Packet Processing Performance (UDP Throughput)This test uses UDP packets of varying sizes generated by test equipment. A constant stream of the appropriatepacket size, with variable source and destination IP addresses transmitting from a fixed source port to a fixeddestination port, is transmitted bidirectionally through each port pair of the DUT.Each packet contains dummy data and is targeted at a valid port on a valid IP address on the target subnet. Thepercentage load and frames per second (fps) figures across each inline port pair are verified by network monitoringtools before each test begins. Multiple tests are run and averages are taken where necessary.This traffic does not attempt to simulate any form of “real-world” network condition. No TCP sessions are createdduring this test, and there is very little for the state engine to do. The aim of this test is to determine the rawpacket processing capability of each inline port pair of the DUT, and to determine the DUT’s effectiveness atforwarding packets quickly, in order to provide the highest level of network performance and with the leastamount of latency.Figure 9 – Raw Packet Processing Performance (UDP Traffic)10
NSS LabsNext Generation Firewall Test Report – Forcepoint Stonesoft Next-Generation Firewall 1402 v5.8.5Raw Packet Processing Performance (UDP Latency)NGFWs that introduce high levels of latency le
While the companion Comparative Reports on security, performance, and total cost of ownership (TCO) will provide information about all tested products, this Test Report provides detailed information not available elsewhere. NSS research indicates that NGFW devices are typically deployed to protect users rather than data center assets,File Size: 1003KB
