Total Protection Suites - PassMark Software

Transcription

2012 Consumer Security Products PerformanceBenchmarks (Edition 3)Antivirus, Internet Security & Total SecurityWindows 7February 2 Consumer Security Products Performance Benchmarks (Edition 3)M. Baquiran, D. WrenPassMark Software16 February 20123totalprotectionsuites-feb2012.docx

Consumer Security ProductsPassMark SoftwareTable of ContentsTABLE OF CONTENTS. 2REVISION HISTORY. 4REFERENCES. 4EXECUTIVE SUMMARY . 5OVERALL SCORE . 6PRODUCTS AND VERSIONS . 8INTERNET SECURITY SOFTWARE . 8ANTIVIRUS SOFTWARE . 9TOTAL SECURITY SOFTWARE . 9PERFORMANCE METRICS SUMMARY . 10INTERNET SECURITY SOFTWARE – TEST RESULTS . 13BENCHMARK 1 – BOOT TIME . 13BENCHMARK 2 – SCAN TIME . 14BENCHMARK 3 – USER INTERFACE LAUNCH TIME . 15BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE . 16BENCHMARK 5 – BROWSE TIME . 17BENCHMARK 6 – INTERNET EXPLORER LAUNCH TIME . 18BENCHMARK 7 – INSTALLATION TIME . 19BENCHMARK 8 – INSTALLATION SIZE . 20BENCHMARK 9 – REGISTRY KEYS ADDED . 21BENCHMARK 10 – FILE COPY, MOVE AND DELETE . 22BENCHMARK 11 – INSTALLATION OF THIRD PARTY APPLICATIONS . 23BENCHMARK 12 – NETWORK THROUGHPUT . 24BENCHMARK 13 – FILE FORMAT CONVERSION . 25BENCHMARK 14 – FILE COMPRESSION AND DECOMPRESSION . 26BENCHMARK 15 – FILE WRITE, OPEN AND CLOSE . 27BENCHMARK 16 – PE SCAN TIME . 28BENCHMARK 17 – FILE COPY DISK TO DISK . 29ANTIVIRUS SOFTWARE – TEST RESULTS . 30BENCHMARK 1 – BOOT TIME . 30BENCHMARK 2 – SCAN TIME . 31BENCHMARK 3 – USER INTERFACE LAUNCH TIME . 32BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE . 33BENCHMARK 5 – BROWSE TIME . 34BENCHMARK 6 – INTERNET EXPLORER LAUNCH TIME . 35BENCHMARK 7 – INSTALLATION TIME . 36BENCHMARK 8 – INSTALLATION SIZE . 37BENCHMARK 9 – REGISTRY KEYS ADDED . 38BENCHMARK 10 – FILE COPY, MOVE AND DELETE . 39BENCHMARK 11 – INSTALLATION OF THIRD PARTY APPLICATIONS . 40Performance BenchmarkEdition 3Page 2 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareBENCHMARK 12 – NETWORK THROUGHPUT . 41BENCHMARK 13 – FILE FORMAT CONVERSION . 42BENCHMARK 14 – FILE COMPRESSION AND DECOMPRESSION . 43BENCHMARK 15 – FILE WRITE, OPEN AND CLOSE . 44BENCHMARK 16 – PE SCAN TIME . 45BENCHMARK 17 – FILE COPY DISK TO DISK . 46TOTAL SECURITY SOFTWARE – TEST RESULTS . 47BENCHMARK 1 – BOOT TIME . 47BENCHMARK 2 – SCAN TIME . 47BENCHMARK 3 – USER INTERFACE LAUNCH TIME . 48BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE . 48BENCHMARK 5 – BROWSE TIME . 49BENCHMARK 6 – INTERNET EXPLORER LAUNCH TIME . 49BENCHMARK 7 – INSTALLATION TIME . 50BENCHMARK 8 – INSTALLATION SIZE . 50BENCHMARK 9 – REGISTRY KEYS ADDED . 51BENCHMARK 10 – FILE COPY, MOVE AND DELETE . 51BENCHMARK 11 – INSTALLATION OF THIRD PARTY APPLICATIONS . 52BENCHMARK 12 – NETWORK THROUGHPUT . 52BENCHMARK 13 – FILE FORMAT CONVERSION . 53BENCHMARK 14 – FILE COMPRESSION AND DECOMPRESSION . 53BENCHMARK 15 – FILE WRITE, OPEN AND CLOSE . 54BENCHMARK 16 – PE SCAN TIME . 54BENCHMARK 17 – FILE COPY DISK TO DISK . 55DISCLAIMER AND DISCLOSURE . 56CONTACT DETAILS . 56APPENDIX 1 – TEST ENVIRONMENT . 57APPENDIX 2 – METHODOLOGY DESCRIPTION . 58Performance BenchmarkEdition 3Page 3 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareRevision HistoryRevRevision HistoryDateEdition 1Initial version of this report, includes new results for Norton Internet Security 2012, GData Internet Security 2012, Kaspersky Internet Security 2011, Panda Internet Security2012, Avast! Internet Security 6, Norton AntiVirus 2012, G Data AntiVirus 2012,Kaspersky Anti-Virus 2012, Panda Antivirus 2012 and Avast! Pro Antivirus 6.Edition 2Added results for 7 additional Internet Security products and 7 additional Antivirusproducts. For more details on these products, see Products and Versions.Edition 3Added results for 4 additional Free Antivirus products to the antivirus category. Addedresults for 5 products in the Total Security category. Replaced all Registry Key Countresults using RegistryCount.exe to results taken using OSForensics. For more details seeBenchmark 9 – Registry Key Count.11 August 201114 November 201124 January 2012ReferencesRef #1DocumentWhat Really Slows Windows Down (URL)Performance BenchmarkEdition 3AuthorO. Warner,The PC SpyDate2001-2009Page 4 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareExecutive SummaryPassMark Software conducted objective performance testing on thirteen (13) Internet Security softwareproducts, seventeen (17) Antivirus software products, and five (5) Total Security software products on Windows7 Ultimate Edition (64-bit) SP1 between April 2011 and January 2012. This report presents our results andfindings as a result of performance benchmark testing conducted for these consumer security products.Subsequent editions of this report will include new products released for 2012 as they are made available. Formore details on which versions were tested, please see the section “Products and Versions”.Testing was performed on all products using seventeen (17) performance metrics. These performance metricsare as follows: Boot Time; Scan Time; User Interface Launch Time; Memory Usage during System Idle; Browse Time; Internet Explorer Launch Time; Installation Time; Installation Size; Registry Keys Added; File Copy, Move and Delete; Installation of Third Party Applications; Network Throughput (previously named “Binary Download Test”) File Format Conversion; File Compression and Decompression; File Write, Open and Close; PE Scan Time and; File Copy Disk to Disk.Performance BenchmarkEdition 3Page 5 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareOverall ScorePassMark Software assigned every product a score depending on its ranking in each metric compared to otherproducts in the same category.Internet Security SoftwareIn the following table the highest possible score attainable is 221; in a hypothetical situation where a product hasattained first place all 17 metrics. Internet Security products have been ranked by their overall scores:Product NamePerformance BenchmarkEdition 3Overall ScoreWebroot SecureAnywhere Essentials195Norton Internet Security 2012158Trend Micro Virus Buster 2012134Avira Internet Security 2012133Avast! Internet Security 6128Kaspersky Internet Security 2012119AVG Internet Security 2012116Trend Micro Titanium Internet Security 2012114Panda Internet Security 2012101G Data Internet Security 2012103SourceNEXTstyle ZERO93McAfee Internet Security 201286BitDefender Internet Security 201268Page 6 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareAntivirus SoftwareIn the following table the highest possible score attainable is 289; in a hypothetical situation where a product hasattained first place in all 17 metrics. Antivirus products have been ranked by their overall scores:Product NameOverall ScoreWebroot SecureAnywhere AntiVirus 2012257Norton Antivirus 2012203Avast! Pro Antivirus 6183Kaspersky Anti-Virus 2012165Trend Micro Titanium 2012167PC Tools Spyware Doctor with AntiVirus 9164Avira Free Antivirus 12161Panda Cloud Antivirus Free152Avast! Free Antivirus 6148Microsoft Security Essentials148AVG Anti-Virus Free 2012141Quick Heal AntiVirus 2012138Avira Antivirus Premium 2012135AVG Anti-Virus 2012123G Data Antivirus 2012116Panda Antivirus Pro 2012108McAfee Antivirus 201290Total Security SoftwareIn the following table the highest possible score attainable is 85; in a hypothetical situation where a product hasattained first place in all 17 metrics. Total Security products have been ranked by their overall scores:Product NameOverall ScoreNorton 360 v669Trend Micro Titanium Maximum SecurityPerformance BenchmarkEdition 3201255McAfee Total Protection 201249Kaspersky PURE 942BitDefender Total Security 201240Page 7 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareProducts and VersionsIn all cases, we have tested the full, retail release of the newest generation (2012 versions) of security products.The names and versions of products which were tested can be found below from each category table:Internet Security SoftwareManufacturerProduct NameRelease YearProduct VersionDate TestedSymantec CorporationNorton Internet Security 2012201119.1.0.21Aug 2011G Data Software AGG Data Internet Security 2012201122.0.2.25Jun 2011Kaspersky Internet Security 2012201112.0.0.374Jun 2011Panda Internet Security 2012201117.00.00Jul 2011Avast! Internet Security 620116.0.1203Aug 2011AVG Internet Security 201220112012.0.1809Sep 2011Trend Micro Titanium InternetSecurity 201220115.0.1280Oct 2011SourceNEXTstyle ZERO201111.0.0047Oct 2011Trend Micro Inc.Trend Micro Virus Buster 201220115.0.1280Oct 2011Avira OperationsGmbH & Co. KGAvira Internet Security 2012201112.0.0.871Oct 2011BitDefenderBitDefender Internet Security 2012201115.0.31.1282Oct 2011McAfee, Inc.McAfee Internet Security 2012201111.0.393Nov 2011Webroot SecureAnywhere Essentials20118.0.0.66Feb-2012Kaspersky LabPanda Security SLAVAST Software a.s.AVG TechnologiesTrend Micro Inc.SourcenextCorporationWebrootPerformance BenchmarkEdition 3Page 8 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareAntivirus SoftwareManufacturerProduct NameRelease YearProduct VersionDate TestedSymantec CorporationNorton AntiVirus 2012201119.1.0.21Aug 2011G Data Software AGG Data Antivirus 2012201122.0.2.25Jun 2011Kaspersky LabKaspersky Antivirus 2012201112.0.0.374Jun 2011Panda SecurityPanda Antivirus Pro 2012201111.00.00Jul 2011Avast! Pro Antivirus 620116.0.1203Aug 2011AVG Anti-Virus 201220112012.0.1809Sep 2011Trend Micro Inc.Trend Micro Titanium 201220115.0.1280Oct 2011Avira OperationsGmbH & Co. KGAvira Antivirus Premium 2012201112.0.0.871Oct 2011Quick Heal AntiVirus 2012201113.00(6.0.0.1)Oct 2011McAfee AntiVirus 2012201111.0.623Nov 2011PC Tools Spyware Doctor withAntiVirus 920119.0.0.888Nov 2011Microsoft Security Essentials20112.1.1116.0Nov 2011Avast! Free Antivirus20116.0.1289Dec 2011AVG Anti-Virus Free Edition 201220112012.0.1873Dec 2011Panda Cloud Antivirus Free20111.5.1Dec 2011Webroot SecureAnywhere Essentials20118.0.0.60Dec 2011Avira Free Antivirus 12201112.0.0.870Dec 2011AVAST Software a.s.AVG TechnologiesQuick HealTechnologies (P) Ltd.McAfee, Inc.PC ToolsMicrosoft CorporationAVAST Software a.s.AVG TechnologiesPanda SecurityWebrootAvira OperationsGmbH & Co. KGTotal Security SoftwareManufacturerProduct NameRelease YearProduct VersionDate TestedSymantec CorporationNorton 360 v620116.0.0.141Dec 2011Trend Micro Titanium MaximumSecurity 201220115.0.1280Jan 2012Trend Micro Inc.McAfee, Inc.Kaspersky LabBitDefenderPerformance BenchmarkEdition 3SecurityCenter11.0.649McAfee Total Protection 20122011Kaspersky PURE 920119.1.0.124Jan 2012BitDefender Total Security 2012201115.0.35.1478Jan 2012Virus Scan15.0294Jan 2012Page 9 of 6716 February 2012

Consumer Security ProductsPassMark SoftwarePerformance Metrics SummaryWe have selected a set of objective metrics which provide a comprehensive and realistic indication of the areasin which an antivirus may impact system performance for end users. Our metrics test the impact of the antivirussoftware on common tasks that end-users would perform on a daily basis.All of PassMark Software’s test methods can be replicated by third parties using the same environment to obtainsimilar benchmark results. Detailed descriptions of the methodologies used in our tests are available as“Appendix 2 – Methodology Description” of this report.Benchmark 1 – Boot TimeThis metric measures the amount of time taken for the machine to boot into the operating system. Securitysoftware is generally launched at Windows startup, adding an additional amount of time and delaying thestartup of the operating system. Shorter boot times indicate that the application has had less impact on thenormal operation of the machine.Benchmark 2 – Scan TimeAll antivirus solutions have functionality designed to detect viruses and various other forms of malware byscanning files on the system. This metric measured the amount of time required to scan a set of clean files. Oursample file set comprised a total file size of 982 MB and was made up of files that would typically be found onend-user machines, such as media files, system files and Microsoft Office documents.Benchmark 3 – User Interface Launch TimeThis metric provides an objective indication as to how responsive a security product appears to the user, bymeasuring the amount of time it takes for the user interface of the antivirus software to launch from Windows.To allow for caching effects by the operating system, both the initial launch time and the subsequent launchtimes were measured. Our final result is an average of these two measurements.Benchmark 4 – Memory Usage during System IdleThis metric measures the amount of memory (RAM) used by the product while the machine and antivirussoftware are in an idle state. The total memory usage was calculated by identifying all antivirus softwareprocesses and the amount of memory used by each process.The amount of memory used while the machine is idle provides a good indication of the amount of systemresources being consumed by the antivirus software on a permanent basis. Better performing products occupyless memory while the machine is idle.Benchmark 5 – Browse TimeIt is common behavior for security products to scan data for malware as it is downloaded from the internet orintranet. This behavior may negatively impact browsing speed as products scan web content for malware. Thismetric measures the time taken to browse a set of popular internet sites to consecutively load from a localserver in a user’s browser window.Performance BenchmarkEdition 3Page 10 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareBenchmark 6 – Internet Explorer Launch TimeThis metric is one of many methods to objectively measure how much a security product impacts on theresponsiveness of the system. This metric measures the amount of time it takes to launch the user interface ofInternet Explorer 8. To allow for caching effects by the operating system, both the initial launch time and thesubsequent launch times were measured. Our final result is an average of these two measurements.Benchmark 7 – Installation TimeThe speed and ease of the installation process will strongly influence the user’s first impression of the antivirussoftware. This test measures the minimum installation time required by the antivirus software to be fullyfunctional and ready for use by the end user. Lower installation times represent antivirus products which arequicker for a user to install.Benchmark 8 – Installation SizeIn offering new features and functionality to users, antivirus software products tend to increase in size with eachnew release. Although new technologies push the size limits of hard drives each year, the growing disk spacerequirements of common applications and the increasing popularity of large media files (such as movies, photosand music) ensure that a product's installation size will remain of interest to home users.This metric aims to measure a product’s total installation size. This metric is defined as the total disk spaceconsumed by all new files added during a product's installation.Benchmark 9 – Registry Keys AddedA large registry increases a machine’s use of resources. This may negatively impact system performance,especially on much older machines. This test measures the amount of keys and values added to registry, afterrebooting the test machines, following a successful product installation. Lower numbers mean that a product hasadded fewer keys during installation and had less impact on the registry.Benchmark 10 – File Copy, Move and DeleteThis metric measures the amount of time taken to move, copy and delete a sample set of files. The sample fileset contains several types of file formats that a Windows user would encounter in daily use. These formatsinclude documents (e.g. Microsoft Office documents, Adobe PDF, Zip files, etc), media formats (e.g. images,movies and music) and system files (e.g. executables, libraries, etc).Benchmark 11 – Installing Third Party ApplicationsThis metric measures the amount of time taken to install and uninstall third party programs. The installationspeed of third party applications may be impacted by antivirus behavior such as heuristics or real time malwarescanning.Benchmark 12 – Network ThroughputThe metric measures the amount of time taken to download a variety of files from a local server using theHyperText Transfer Protocol (HTTP), which is the main protocol used on the web for browsing, linking and datatransfer. Files used in this test include file formats that users would typically download from the web, such asimages, archives, music files and movie files.Performance BenchmarkEdition 3Page 11 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareBenchmark 13 – File Format ConversionThis test measures the amount of time taken to convert an MP3 file to a WAV and subsequently, convert thesame MP3 file to a WMA format.Benchmark 14 – File Compression and DecompressionThis metric measures the amount of time taken to compress and decompress different types of files. Filesformats used in this test included documents, movies and images.Benchmark 15 – File Write, Open and CloseThis benchmark was derived from Oli Warner’s File I/O test at http://www.thepcspy.com (please see Reference#1: What Really Slows Windows Down). This metric measures the amount of time taken to write a file, thenopen and close that file.Benchmark 16 – PE Scan TimeAll antivirus solutions have functionality designed to detect viruses and various other forms of malware byscanning files on the system. This metric measured the amount of time required to scan a set of PE (PortableExecutable) files. Our sample file set comprised a total file size of 2.03GB and consisted of .exe (329MB), .dll(920MB) and .sys files (827MB).Benchmark 17 – File Copy Disk To DiskThis test measures the amount of time taken to copy files between two local drives. The data set comprised atotal file size of 5.44GB, and the formats used included documents, movies, images and executables.Performance BenchmarkEdition 3Page 12 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareInternet Security Software – Test ResultsIn the following charts, we have highlighted the results we obtained for Norton Internet Security 2012 in yellow.The average has also been highlighted in blue for ease of comparison.Benchmark 1 – Boot TimeThe following chart compares the average time taken for the system to boot (from a sample of five boots) foreach Internet Security product tested. Products with lower boot times are considered better performingproducts in this category.Webroot SecureAnywhere Essentials 201226.37Trend Micro Virus Buster 201227.32Avast! Internet Security 628.95AVG Internet Security 201230.78Trend Micro Titanium Internet Security 201230.80Norton Internet Security 201231.73Avira Internet Security 201232.61SourceNEXTstyle ZERO32.91Kaspersky Internet Security 201234.10Average36.26Panda Internet Security 201236.99G Data Internet Security 201238.21BitDefender Internet Security 201238.51McAfee Internet Security 201282.120sPerformance BenchmarkEdition 310 s20 s30 s40 s50 s60 s70 s80 s90 sPage 13 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareBenchmark 2 – Scan TimeThe following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for eachInternet Security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 25) scan times. Products with lower scan times are considered better performing products in this category.G Data Internet Security 201222.50Webroot SecureAnywhere Essentials 201222.63Norton Internet Security 201223.25AVG Internet Security 201225.38McAfee Internet Security 201227.50Trend Micro Titanium Internet Security 201236.21Avast! Internet Security 636.75Average41.08Avira Internet Security 201242.75BitDefender Internet Security 201243.50Trend Micro Virus Buster 201244.25Kaspersky Internet Security 201245.58Panda Internet Security 201275.79SourceNEXTstyle ZERO88.000sPerformance BenchmarkEdition 310 s20 s30 s40 s50 s60 s70 s80 s90 s100 sPage 14 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareBenchmark 3 – User Interface Launch TimeThe following chart compares the average time taken to launch a product’s user interface. Products with lowerlaunch times are considered better performing products in this category.Trend Micro Virus Buster 201231.04Trend Micro Titanium Internet Security 201232.63SourceNEXTstyle ZERO90.35McAfee Internet Security 2012127.77G Data Internet Security 2012Webroot SecureAnywhere Essentials 2012Avast! Internet Security 6Kaspersky Internet Security 2012Average241.30289.99305.06395.00523.33Avira Internet Security 2012789.20BitDefender Internet Security 2012790.14Norton Internet Security 2012AVG Internet Security 2012Panda Internet Security 20120 msPerformance BenchmarkEdition 3991.781,082.471,636.60200 ms 400 ms 600 ms 800 ms 1,000 ms 1,200 ms 1,400 ms 1,600 ms 1,800 msPage 15 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareBenchmark 4 – Memory Usage during System IdleThe following chart compares the average amount of RAM in use by an Internet Security product during a periodof system idle. This average is taken from a sample of ten memory snapshots taken at roughly 60 seconds apartafter reboot. Products with lower idle RAM usage are considered better performing products in this category.Webroot SecureAnywhere Essentials 20122.70Avast! Internet Security 6Norton Internet Security 2012Panda Internet Security 2012BitDefender Internet Security 201216.9324.0440.3044.37Average52.16Avira Internet Security 201252.54Kaspersky Internet Security 2012AVG Internet Security 2012Trend Micro Virus Buster 2012McAfee Internet Security 2012Trend Micro Titanium Internet Security 2012SourceNEXTstyle ZEROG Data Internet Security 20120 MBPerformance BenchmarkEdition 355.0356.5062.4971.1780.2685.1086.6010 MB 20 MB 30 MB 40 MB 50 MB 60 MB 70 MB 80 MB 90 MB 100 MBPage 16 of 6716 February 2012

Consumer Security ProductsPassMark SoftwareBenchmark 5 – Browse TimeThe following chart compares the average time taken for Internet Explorer to successively load a set of

Consumer Security Products PassMark Software Performance Benchmark Page 9 of 67 Edition 3 16 February 2012 Antivirus Software Manufacturer Product Name Release Year Product Version Date Tested Symantec Corporation Norton AntiVirus 2012 2011 19.1.0.21 Aug 2011 G Data Software AG G Data Antivirus 2012 2011 22.0.2.25 Jun 2011