Transcription
EPRI Software Development2016 Guide for Testing Your SoftwareSoftware Quality Assurance (SQA)
Usability Testing Sections Installation and Un-Installation Software Documentation Test Cases or Tutorial Graphical User Interface Stress Testing Security Vulnerability Testing 2016 Electric Power Research Institute, Inc. All rights reserved.2
InstallationEPRI Requirements:http://swdev.epri.com/req-install.asp Run a Virus Scan Verify Documentation– Network installation instructions ifnecessary.– Documentation required for Applicationlike Web Applications & Spreadsheets. 2016 Electric Power Research Institute, Inc. All rights reserved.3
Installation Installation Settings– Typical v. Custom Install Directories Shortcuts– Confirm successful installation &un-installation of Applications. Software Encryption– Input serial numbers or securitykeys if necessary Test invalid inputs for validation 2016 Electric Power Research Institute, Inc. All rights reserved.4
Software DocumentationEPRI Requirements:http://swdev.epri.com/req-doc.asp Check if the EPRI Software ManualTemplate was used.–Check headers and footer–Check for system requirements: Hardware and Software specifications Permissions such as Administrator rights–Check application feature descriptions–Check spelling and grammar 2016 Electric Power Research Institute, Inc. All rights reserved.5
Test CasesEPRI pReminder: One tutorial is required or at least threesolved example problems. Execute & confirm all tutorials for correct inputs andoutputs. Verify that the calculations, graphs, and screenshotsmatch the documentation.Note: If any inputs or results do not match, the softwarecan not be approved to send to customers. 2016 Electric Power Research Institute, Inc. All rights reserved.6
Graphical User InterfaceEPRI Requirements:http://swdev.epri.com/req-gui.asp Check for the Preproduction Splash Screen (if preproductionstage) Windows fit in the main application screen and nothing is cut-off ifwindows are resized Make sure all information is accessible Internationalization– Check compatibility– SI Units Change appearance settings Tab order and hot-keys (alt-keys) Check embedded Help feature, including buttons to open the Helpfeature 2016 Electric Power Research Institute, Inc. All rights reserved.7
Stress Testing Range checking– Boundaries of numeric inputs Input type– Numerical– Alphabetical– Special Characters Follow the solved example problems, but then skip astep or do them in a different sequence 2016 Electric Power Research Institute, Inc. All rights reserved.8
Stress Testing Check print featureTry different login combinationsCheck error messages for clarity.– Error messages should appear when theerror occurs. Check for spelling within the application 2016 Electric Power Research Institute, Inc. All rights reserved.9
Stress Testing For databases:– Ensure all connections through the application are validwhen accessing data– Ensure single quotes and double quotes are tested toverify they do not corrupt the database– Add duplicate records– Delete all records to make sure it does not crash theapplication Modify data files to make sure the application gives acorrect error message 2016 Electric Power Research Institute, Inc. All rights reserved.10
Stress TestingWith administrative feature Verify Admin privilege and howit differs from a regular userCheck for compatibility withMicrosoft Office applications ifapplicable (such as copy andpaste features)Test functionalities of buttonsCheck save feature 2016 Electric Power Research Institute, Inc. All rights reserved.11Without administrative feature
Stress Testing Check open file feature correct fileextensions, choosing incorrect filetype brings up error message, etc.)The International Standarddate notationDD-MM-YYYYUnited States Standard If there are graphs, check graphfeatures and settingsCheck options/settings not coveredin the sample problems.Check to make sure internationalunits are converted correctly 2016 Electric Power Research Institute, Inc. All rights reserved.12date notationMM-DD-YYYY
Stress Testing Maximize, minimize, and resizewindows to make sure theapplication responds correctly.Check keyboard shortcutsCheck all menu items, includingthe pop-up menus that come upwhen the user right-mouseclicks an itemIf there are hardware/softwarekeys, check to see if theapplication responds whenexecuted with the key(s), thenwithout the key(s) 2016 Electric Power Research Institute, Inc. All rights reserved.13XCV
Security Vulnerability Testing OWASP Top Ten Web Application Vulnerabilities– http://www.owasp.org/index.php/OWASP Top Ten Project1: Injection2: Cross-Site Scripting (XSS)3: Broken Authentication and Session Management4: Insecure Direct Object References5: Cross-Site Request Forgery (CSRF)6: Security Misconfiguration7: Insecure Cryptographic Storage8: Failure to Restrict URL Access9: Insufficient Transport Layer Protection10: Unvalidated Redirects and Forwards 2016 Electric Power Research Institute, Inc. All rights reserved.14
Security Vulnerability Testing Two vulnerabilities SQA will test for:– Structured Query Language (SQL) Injection– Cross-Site Scripting The developer is expected to address securityvulnerabilities when developing an application 2016 Electric Power Research Institute, Inc. All rights reserved.15
Security Vulnerability Testing SQL Injection – Injection of a SQL Query throughinput data, such as a querystring or form Examples:– In the querystring, enter a SQL Statement, suchas " ‘; Delete from users --’ ", into a querystringvariable– Enter in " ' OR 1 1 " into a form field orquerystring variable See the following for more information and testingexamples:http://www.owasp.org/index.php/SQL Injection 2016 Electric Power Research Institute, Inc. All rights reserved.16
Security Vulnerability Testing Cross-Site Scripting - Harmful scripts are entered intoweb sites via querystring or form field Example:– Enter in " script type "text/javascript" alert(‘hello’); /script " into a form field to check whether the formfield is validated Allows the user to execute scripts that are harmful See the following for more ite-scripting 2016 Electric Power Research Institute, Inc. All rights reserved.17
Security Vulnerability Testing Testing tools:– OWASP’s Web Scarab (Manual)– OWASP’s Zed Attack Proxy (Automated)– Nexpose (Automated)– Rapid 7 (Automated) Reference:– Open Web Application Security Project (OWASP)http://www.owasp.org/index.php/Main Page 2016 Electric Power Research Institute, Inc. All rights reserved.18
What SQA Does Not DoSQA software usability testing does not do:–––V&V (Verification and Validation) testingTest or validate real world data (this should bedone by beta testers)Exhaustive testing or “white box” (source code)testingSQA usability testing will not find all errors and isnot intended toAll errors are expected to be found by developers 2016 Electric Power Research Institute, Inc. All rights reserved.19
Together Shaping the Future of Electricity
- V&V (Verification and Validation) testing - Test or validate real world data (this should be done by beta testers) - Exhaustive testing or "white box" (source code) testing SQA usability testing will not find all errors and is not intended to
