Transcription
Fast and Effective EndpointSecurity for BusinessComparative AnalysisJune t and Effective Endpoint Security for Business – Comparative AnalysisK. Lai, D. WrenPassMark Software15 June 2010Edition 1Fast and Effective Security for Business - Ed1 June 2010.docx
Fast and Effective Endpoint Security for BusinessPassMark SoftwareTable of ContentsTABLE OF CONTENTS. 2REVISION HISTORY. 3REFERENCES. 3INTRODUCTION . 4RATINGS AND SUMMARY . 5RATING CATEGORIES . 6STAR RATING DESCRIPTION . 6TASK DESCRIPTION . 7HARDWARE ENVIRONMENTS. 7PRODUCTS AND VERSIONS TESTED . 7PERFORMANCE BENCHMARK RESULTS . 8ESET SMART SECURITY BUSINESS EDITION . 11KASPERSKY BUSINESS SPACE SECURITY . 14MCAFEE TOTAL PROTECTION FOR ENDPOINT . 17MICROSOFT FOREFRONT CLIENT SECURITY . 20SOPHOS ENDPOINT SECURITY AND DATA PROTECTION . 23SYMANTEC ENDPOINT PROTECTION . 26TREND MICRO WORRY FREE BUSINESS SECURITY . 29DISCLAIMER AND DISCLOSURE . 32DISCLAIMER OF LIABILITY. 32DISCLOSURE . 32TRADEMARKS. 32CONTACT DETAILS . 32APPENDIX A – PERFORMANCE METHODOLOGY . 33Comparative AnalysisEdition 1Page 2 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessPassMark SoftwareRevision HistoryRevEdition 1Revision HistoryDateFirst edition of the document.Performance charts and comparative reviews added.15 Jun 2010ReferencesRef #DocumentAuthorDateAV Comparatives24 Dec 2009AV Comparatives17 Mar 2010AV Comparatives5 Jun 2010Virus BulletinJun 2010Nico Cuppen Software-AV Comparatives - Summary Report 2009Current Edition: December 20091The information in the most recent version of this report byAV Comparatives was used to determine the effectiveness ofreviewed business security solutions.AV Comparatives - On-Demand ComparativesCurrent Edition: Report Number 25, February 20102The information from the most recent version of this reportby AV Comparatives was used to determine the effectivenessof reviewed business security solutions.AV Comparatives – Retrospective/ProActive TestCurrent Edition: Report Number 26, May 20103Information from Retrospective/ProActive Test reports by AVComparatives was used in determining the effectiveness ofreviewed business security solutions.VB100 Test ResultsCurrent results: Windows Server 2008 R2, June 20104Overall test results obtained by VB100 were used indetermining the effectiveness of reviewed business securitysolutions.Network Traffic Monitor v2.015A tool used to monitor the amount of inbound and outboundnetwork traffic for the Update Size metric.Comparative AnalysisEdition 1Page 3 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessPassMark SoftwareIntroductionThe importance of effective security for businesses cannot be understated, with malware damage and costs to1businesses escalating every year . The nature of the threat landscape has become more sophisticated, withmalware events not only causing lost productivity and reputation for affected businesses but a staggering2potential for losses from theft of data and other security breaches from more targeted attacks.In response, businesses have been placing their confidence in a wide range of security solutions to meet theirneeds. With the large amount of software available from many vendors, the challenge for businesses nowbecomes determining which security solution is the most effective at mitigating the threat of malware, whileminimizing implementation cost and impact to existing business functions and workflow.This report presents a comparative analysis on the performance, effectiveness and usability of seven securitysolutions from some of the world’s largest security vendors. In this report, PassMark Software has evaluated thefollowing business security products: 12ESET Smart Security 4 Business EditionKaspersky Business Space SecurityMcAfee Total Protection for EndpointMicrosoft Forefront Client SecuritySymantec Endpoint ProtectionSophos Endpoint Security and Data ProtectionTrend Micro Worry-Free Business Security: Standard EditionA 2007 survey conducted by ‘Computer Economics Inc’ estimates that worldwide, malware costs businesses over US 13B each year.The Fifth Annual “Cost of a Data Breach” report by Ponemon Institute, Jan 2010, puts the average price of data breach at US 204 per compromised record.Comparative AnalysisEdition 1Page 4 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessPassMark SoftwareRatings and SummaryPassmark Software has given each security product a rating which reflects its overall performance, ease of use, design, features and level of excellence in that category.Categories represent major functions or feature sets common to the sphere of business security. These ratings represent PassMark Software’s subjective views andexperiences in installing, configuring and use of business security products to manage endpoints.The following table summarizes ratings in all categories for all products ecTrend MicroOverall RatingInstallation & ConfigurationMigrationDefault PoliciesClient InstallationInterface DesignClient & Policy ManagementRemote Management--UpdatesCommon Use CasesEffectivenessPerformanceComparative AnalysisEdition 1Page 5 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessPassMark SoftwareRating CategoriesThe table below describes the criteria and factors which were considered for each business security solution ineach category to determine a rating. Evaluation categories were determined prior to testing and were chosen asa set of expected features or functions which define business security products.CategoryOverall RatingInstallation & ConfigurationCategory DescriptionThe rating for this category is calculated as an average of all other ratings, with all categories carryingequal weight.This category evaluates the speed and relative ease of the installation and configuration process of servercomponents, including the quality and accuracy of documentation, the ability to install pre-requisites andthe level of installer integration.MigrationThis category rates the relative ease and simplicity of product migration from previous vendor solutionsor third party software solutions. Extra consideration is given to vendors who have documented themigration process well.Default PoliciesThis category considers whether the policies created by default during product installation ‘make sense’from a management perspective, taking into account whether different default choices are available.Client InstallationThis category evaluates the simplicity and ease of client installation, taking into account the speed ofdeployment and the level of impact of installation on endpoint users.Interface DesignThis category rates the design of the server console’s user interface for responsiveness, intuitiveness,consistency and functionality.Client & Policy ManagementThis category assesses the flexibility and granularity of policy management from the server console,taking into account the level of automation in setting up groups and the depth of configuration optionsavailable to administrators.Remote ManagementThis category examines how much support is provided for administrators to access the managementconsole from a remote terminal.UpdatesThis category rates the level of configuration required to enable the management server to update acentral repository, as well as the ease of deployment to and the timeliness of retrieval by endpointmachines.Common Use CasesThis category examines the relative ease of use and handling for three common management use casesby each security product. These scenarios are: conducting an ‘on-demand’ scan of selected endpoints;creating and viewing a malware report ‘on-demand’; and creating, assigning and deploying a new policy.EffectivenessThis category rates the anti-malware effectiveness of a security product based on information fromrecently published material at reputable, third party testing sites. The sources we have used for thiscategory are VB100 and AV Comparatives.PerformanceThis category assigns an overall rating based on a security product’s performance over ten performancebenchmark tests conducted by PassMark Software.Star Rating DescriptionThe table below explains the general significance of ratings relative to product performance, usability andfunctionality.Star Rating--Rating DescriptionUnsupported – This category was not supported by the business security solution. Support was not documented inproduct guides, the online knowledgebase or help files.Very Poor – The security solution offered very limited performance in this category. Products with this rating hadsparse or inaccurate documentation, extremely poor usability, or technical issues which severely hamperedproduct stability, usability and functionality.Poor – The security solution had inadequate or basic performance in this category, as a result of poor usability orfunctionality. Some products with this rating had bugs which hampered product performance in this category.Average – The security solution had adequate performance in this category with some room for improvement.Good– The security solution provides good performance in this category area with useful features and gooddocumentation.Exceptional – The security solution provides outstanding performance in a category area, with unique, thoughtfulor well-designed features that streamline usability or functionality and excellent documentation.Comparative AnalysisEdition 1Page 6 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessPassMark SoftwareTask DescriptionPassMark Software has conducted performance benchmark testing and subjective comparative analysis on theoverall ease of use, speed and effectiveness on seven (7) business security software products.Hardware EnvironmentsThe following hardware platforms were used in conducting our comparative analysis and performance tests, andare intended to represent a typical server and business deployment:Server Machine SpecificationThe following machine ran a virtual machine on which the server components of the security softwarewere installed:Operating System:CPU:Motherboard:RAM:HDD:Windows Server 2003 32-bitIntel Xeon CPU @ 3.4GHzHewlett-Packard 08B4h Motherboard4GB ECC RAMWestern Digital Raptor 74GB 10,000RPMVirtual Machine SpecificationOperating System:RAM:Windows Server 2003 32-bit1-2 GB (depending on the product’s requirements)Client Machine SpecificationOperating System:CPU:Video Card:RAM:HDD:Windows 7 Ultimate x64Intel Core i7 720 @ 2.67GHznVidia GeForce 8800 GT6 GB500 GBProducts and Versions TestedProduct NameServer Component and VersionClient Component and VersionESET Smart Security 4Business EditionRemote Administrator Console:v4.0.122.0Smart Security 4 Business Edition:v4.2.40Kaspersky BusinessSpace SecurityAdministration Kit:v8.0.2090Kaspersky Anti-virus 6.0:v6.0.4.1424McAfee Total Protectionfor EndpointePolicy Orchestrator:v4.5McAfee Agent:v4.5.0.1270Microsoft ForefrontClient SecurityForefront Client Security Console:v1.1.1710.90Forefront Client Security:v1.5.1981.0Symantec EndpointProtectionEndpoint Protection Manager:v11.0.600.550Symantec Endpoint Protection:v11.0.6000.550Sophos Endpoint Securityand Data ProtectionEnterprise Console:v4.0.0.2362Endpoint Security and Control:v9.05Trend Micro Worry-Free BusinessSecurity: Standard EditionWorry-Free Business Security:v6.0 SP2 build 3025Client/Server Security Agent:v16.0.3052Comparative AnalysisEdition 1Page 7 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessPassMark SoftwarePerformance Benchmark ResultsThe following performance categories have been selected as ‘real-life’ metrics which may impact heavily onendpoint system performance and responsiveness. These benchmarks allow the comparison of the level ofimpact that business security software products may have on endpoint machines. Products with goodperformance will have less impact on business activities, workflow and productivity.More detailed description of the methodology used can be found in Appendix A – Performance Methodology.Install SizeProtect endpoints without filling up disk spaceNewer versions of products often have increased diskspace requirements, ensuring disk space remains criticalfor endpoint systems. Endpoint clients with a largerinstallation footprint may be consuming more disk spacethan necessary.This metric measures the total additional disk spaceconsumed by the endpoint client after installation and amanual update. Our final result is measured in megabytes(MB).SymantecMcAfeeKasperskyTrend MicroESETSophosMicrosoft0 MB200 MB400 MB600 MB800 MBMemory usage commit chargeHave more system resources to perform tasksExtensive memory (or physical RAM) usage by securityproducts have significant impact on endpoint systemperformance and cause more reliance on hard disk drives,which have slower read and write speeds than RAM.Business security products which use more memory willvisibly slow performance on affected endpoints.This metric measures the total additional memory useconsumed by the endpoint machine during a period ofsystem idle where an endpoint security product has beeninstalled. Our final result is measured in megabytes (MB),and calculated from an average of 40 samples.MicrosoftMcAfeeSophosTrend MicroSymantecKasperskyESET0 MB60 MB120 MB180 MB240 MB300 MBWord document launch timeSpend less time waiting for documents to launchSecurity products may hinder launch times of applicationsand documents as a result of poorly performing antimalware functionality, such as real-time file scanning orbehavioural heuristics. Slow endpoint system responsetimes can bring about avoidable issues for productivity.The metric measures the total time taken to launch a largeMicrosoft Word 2007 document with a system restart priorto application launch. Our final result is measured inmilliseconds (ms), and calculated from an average of five(5) samples.Comparative AnalysisEdition 1McAfeeTrend MicroSophosMicrosoftSymantecKasperskyESET0 ms4,000 ms8,000 ms12,000 msPage 8 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessWord document restart timeRestart your applications without the waitRestarting applications is a common task for endpointusers, who may quickly close or switch betweendocuments and applications as they conduct work tasks.The amount of time taken by applications to restart is avisible measure of system responsiveness for endpointusers.This metric measures the total time taken to re-open alarge, mixed media Microsoft Word 2007 document, wherean endpoint security product has been installed. Our finalresult is measured in milliseconds (ms), and calculatedfrom an average of ten (10) samples.File copy time for small filesCopy your documents more quicklyTransferring files between devices and drives is a commonactivity undertaken by endpoint users. File copy times maybe negatively affected by poor performance of businesssecurity products functionality, such as file scanning orheuristics.This metric measures the total time taken to copy a set ofsmall files between directories, where an endpoint securityproduct has been installed. Our final result is measured inseconds (s), and calculated from an average of five (5)samples.PassMark SoftwareMcAfeeKasperskySymantecESETTrend MicroSophosMicrosoft1,800 msMcAfeeSymantecKasperskyESET0sKaspersky10 s20 s30 s40 s50 s60 sTrend MicroMcAfeeSophosSymantecESET80 sClient Signature Database SizeSymantecReduce the footprint of endpoint solutionsTrend MicroThis metric measures the total size of anti-malwaresignature files on the endpoint machine after a manualupdate. Our final result is measured in megabytes.2,600 msSophosMicrosoftThe size of the anti-malware signature database onendpoint machines gives an indication of a product’sfootprint on the local disk drive.2,400 msMicrosoftCopy your media files more quicklyThis metric measures the total time taken to copy a set oflarge files between directories, where an endpoint securityproduct has been installed. Our final result is measured inseconds (s), and calculated from an average of five (5)samples.2,200 msTrend MicroFile copy time for large filesCopying large files between directories may similarly beaffected by poor performance of anti-malwarefunctionality in business security products.2,000 ms82 s84 s86 s88 s90 sMcAfeeKasperskySophosMicrosoftESET0 MBComparative AnalysisEdition 140 MB80 MB120 MB160 MBPage 9 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessBoot timeSpend less time waiting for your computer to startMany business software suites create start up tasks andprocesses, causing machine boot times to take significantlylonger. End users can do little but wait for their machine tobecome responsive. Better performing products will haveless of an impact on boot time.This metric measures the time taken to boot the machinewhere an endpoint security product has been installed. Ourfinal result is measured in seconds (s) and calculated froman average of five (5) samples.Machine restart timeReduce the time needed to restart your computerExtra system resources consumed by processes andservices created by security software may delay shut downtime and the restart cycle of endpoint machines.This metric measures the time taken to restart the machinewhere an endpoint security product has been installed. Ourfinal result is measured in seconds (s) and calculated froman average of five (5) samples.Daily network trafficPassMark SoftwareMicrosoftTrend MicroKasperskySymantecMcAfeeSophosESET0s60 sSophosSymantecTrend MicroESETKasperskyMcAfee70 s75 s80 s85 s90 s0 MB20 MB40 MB60 MB80 MBMcAfeeKasperskyAll security solutions require the latest signatures andengine updates in order to provide managed networks thebest possible protection against malware. However, not allsecurity software is equal, with some products performingincremental updates and others downloading much more.MicrosoftComparative AnalysisEdition 140 sMicrosoftMinimize impact on the coporate networkThis metric measures the total daily inbound and outboundtraffic as a result of security software engine and signatureupdates to the repository on the server machine. Our finalresult is measured in megabytes and is calculated as anaverage of sixteen (16) days of data.20 sSymantecTrend MicroESETSophosPage 10 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessPassMark SoftwareESET Smart Security Business EditionInstallation and Configuration4/5Installing ESET Remote Administrator Server and Console,the management components of ESET Smart Security 4Business Edition, was a relatively fast and streamlinedprocess. It took approximately an hour to complete theinstallation, updates and basic configuration of managementcomponents.Review SummaryOverall RatingInstallation & ConfigurationMigrationDefault PoliciesClient InstallationInterface DesignClient & Policy ManagementRemote ManagementUpdatesCommon Use CasesEffectivenessPerformance Very little system impact on endpointmachines. Fastest, best overall performance. Low traffic overhead for updates (3-4MB perday on average) Flexibly manage older versions and variantsof ESET client solutions from a single,compatible console. Endpoints can be automatically sorted intoparametric groups as they join the network. Signature updates are transportable. Mirror server is not automatically createdduring a typical installation. Remote Administrator Console interface isfunctional, rather than aestheticallyappealing.Comparative AnalysisEdition 1The typical Administrator Server installation required verylittle input from the user, seamlessly installing all neededcomponents including the embedded Access JET database.ESET Remote Administrator Server also supports otherdatabases such as SQL Server, Oracle and MySQL, and canautomate the creation of an empty database for use wherean “Advanced” installation is selected. During installation,administrators can set security passwords but this step isentirely optional and passwords may be configured at a laterstage. After installing the Remote Administrator Server,installing the Remote Administrator Console took under tenminutes to install.The ESET Smart Security Business Edition Basic Setup guidewas extremely user-focused. The guide gave step-by-stepinstructions for users from purchase and download toinstallation and configuration, and even included estimatedtimes for completion of each phase. The guide also provideduseful tips for less technical users. ESET also provides acomprehensive User Guide which documents custominstallations and more advanced product functionality.Migration from Previous Solutions4.5/5The removal of third party software prior to ESET clientdeployment can be performed through the ESET RemoteAdministrator Console via a push installation. Administratorswill first need to download the relevant third party removaltool from a list of links provided by ESET, before creating acustom package for push installation.Some versions of ESET clients are automatically upgraded bythe Remote Administrator. In all cases, the RemoteAdministrator Console can manage most different types andversions of ESET security software, making it unnecessary toremove or upgrade existing ESET client software.Upgrading to a new version of ESET Remote Administrator iseasy. Administrators can simply install the latest version ofRemote Administrator and the existing database andsettings will be automatically migrated. For businesses withspecific upgrade or migration needs, ESET also provides a“Rip and Replace” service for North American customers.Page 11 of 3615 June 2010
Fast and Effective Endpoint Security for BusinessDefault Policies4.5/5Default policies are created during the RemoteAdministrator installation and generally provide agood balance for most deployments.The Basic Setup guide gives some starting points toassist in moving away from default settings, such asthe disabling of nonessential notifications forendpoint machines.Client Install4/5PassMark SoftwareInterface Design4/5The Remote Administrator console is designed toappear similar in format to many familiar Windowsapplications. Unfortunately, this means the consoleinterface is a bit bland and grey, with an emphasison functionality rather than aesthetics.Major functions can be accessed from categories oftabs or from the large menu buttons across the topof the interface. Navigating from the interface wasextremely responsive, taking a fraction of a secondto navigate between windows and menus.An endpoint status dashboard can be viewed fromthe Clients tab, with conditionally formatted panelsproviding a quick indication about warnings, errorsor alerts for each endpoint machine.Client and Policy ManagementScreenshot 1: ESET Smart Security 4 Business Edition (Client)The Basic Setup guide provides a checklist of itemsto confirm prior to installation. In our case, the onlyconfiguration change needed to start remoteinstallation was the enabling of the Remote Registryservice on the endpoint machine.5/5Endpoint machines can be arranged into groupsmanually or dynamically added to user-definedparametric groups when certain conditions are met.Parametric groups are extremely powerful,automatically assigning policies to clients as theyconnect to servers, or for filtering and reportingunder certain conditions. Businesses with existinggroup infrastructure, such as Active Directory, canalso make use of the Policy Rules Wizard to createuser-defined rules and automate the mapping ofpolicies to existing groups and objects.After pre-requisites are met, administrators canbrowse to the Remote Install tab and create an ESETSecurity Products installation package. Packagecreation is reasonably fast, taking around fiveminutes to configure and create by the managementserver. Once correctly set up, a single package canbe used to remotely install to both 32-bit and 64-bitendpoint systems.Administrators can also use Remote Administratorto flexibly deploy any custom install package, evennon-ESET applications, as long as the installer is a.msi file.Target systems can be discovered via NetBIOS, IPsearch or through existing Active Directoryinfrastructure. Once initiated, the client module tookroughly ten minutes to install and update on theendpoint machine. No machine restart was requiredfor functional protection.For larger deployments, client installation can beperformed through Windows Group PolicyManagement. This procedure is well-documented inthe User Guide.Comparative AnalysisEdition 1Screenshot 2: ESET Remote Administrator Console (Server)The propagation of policy changes is not immediate,but clients will update and enforce policies on thenext connection to the server. Should anadministrator need to revert to default policies, theycan simply click on the “Default” button from thepolicy manager.The Remote Administrator console is highlycompatible with most variants and legacy versions ofESET security products, all of which can be managedPage 12 of 3615 June 2010
Fast and Effective Endpoint Security for Businesssimultaneously from a single platform. The flexibilityof the Remote Administrator Console caters well toscenarios where client upgrades may not bepossible.All policies can be imported or exported in an XMLformat for backup or duplication purposes.Remote Management4.5/5Where installed on a terminal, the RemoteAdministrator Console (RAC) allows for fullyfunctional remote management of server modules.No additional configuration from the managementserver is required for remote access. If the defaultsettings are selected for the management serverinstallation, accessing the management server willnot require a password. ESET strongly recommendsthat users set a secure password in their installationguides.Updates4/5Basic server configuration included the setup of themirror server from which endpoint machines willretrieve signature updates. While the process wassimple, well-documented and requiring only a fewminutes, other products we tested automated thecreation of an update repository during theinstallation process.Administrators are initially required to enter theirusername and password details in order to accessthe ESET Update Servers. These details areremembered by the management server for futurerepository updates.PassMark SoftwareAll signature files downloaded by ESET SmartSecurity are transportable and able to be manuallymoved between machines where required. Thisfeature is advantageous for administrators managingmachines on isolated networks.Effectiveness5/5AV Comparatives awarded ESET NOD32 Antivirus(v4.0.474.0) the ADVANCED rank in their latest OnDemand Comparative, citing very few false positivesand a total detection rate of 97.7%.The most recent Summary Report (2009) from AVComparatives noted that ESET has high detectionrates, a very good heuristics engine and low systemimpact. Overall, this report awarded the ESET enginethird pl
McAfee Total Protection for Endpoint Microsoft Forefront Client Security Symantec Endpoint Protection Sophos Endpoint Security and Data Protection Trend Micro Worry-Free Business Security: Standard Edition
