Transcription
LTAT.05.006: Software TestingLecture 08: Black-BoxTesting (advanced) – GUITesting, Visual Testing,Security, Usability, andA/B TestingSpring 2022LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Dietmar Pfahlemail: dietmar.pfahl@ut.ee
Lectures Lecture 1 (10.02) – Introduction to Software Testing Lecture 2 (17.02) – Basic Black-Box Testing Techniques: Boundary ValueAnalysis & Equivalence Class Partitioning Lecture 3 (03.03) – BBT advanced: Combinatorial Testing Lecture 4 (10.03) – Basic White-Box Testing Techniques: Control-Flow Coverage Lecture 5 (17.03) – BBT adv.: State-Transition, Metamorphic, Random Testing Lecture 6 (24.03) – Test Levels, Test Tools, Test Automation Lecture 7 (31.03) – BBT adv.: Exploratory Testing, Behaviour Testing Lecture 8 (07.04) – BBT adv.: GUI / Visual Testing, Usability Testing, A/B Testing Lecture 9 (14.04) – Security Testing of Mobile Applications Lecture 10 (21.04) – WBT adv.: Data-Flow Testing / Mutation Testing Lecture 11 (28.04) – WBT adv.: Symbolic Execution, Static Code Analysis, Review Lecture 12 (05.05) – Defect Estimation / Test Documentation, Organisation andProcess Improvement (Test Maturity Model) Lecture 13 (12.05) – Exam Preparation Lecture 14 (19.05) – Advanced Topics (optional)LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Structure of Lecture 8 GUI Testing GUITAR TESTAR Visual Testing Sikuli IDE Lab 7 Security Testing Usability Testing A/B TestingLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Structure of Lecture 8 GUI Testing GUITAR TESTAR Visual Testing Sikuli IDE Lab 7 Security Testing Usability Testing A/B TestingLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022GUITAR
GUITARSource: 6 / Lecture 08 / Dietmar Pfahl 2022
GUITAR – GUI Ripping During ripping, the GUI application is executed automatically; theapplication’s windows are opened in a depth first manner. The Ripper extracts all the widgets and their properties from the GUI. Properties of widgets include basic attributes such as position, color, size, andenabled status. Properties also include information about widgets' events, such as: whether awidget opens a modal or modeless window or a menu, whether a widget closes awindow, and whether the widget is a button or an editable text-field. The Ripper extracts properties for widgets as well as their containing GUIwindows and stores the information in the GUI Tree. For each GUI window, the Ripper first extracts structural information of thatwindow and then executes widgets that invoke other GUI windows. The depth-first traversal terminates when all GUI windows are covered.LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
GUITAR – GUI RippingLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022GUI of an app with two windowsGUI tree with 2 nodes and subset
GUITAR –ModelConversion EFG Event FlowGraphLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
GUITAR – Test Case GenerationApplies traversal algorithms for the EFG:Defined length Lof eventsequencesExample (L 5):e1 - e2 - e3 - e4 - e5LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
GUITAR –Test Case Execution & EvaluationLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
GUITAR –Test Case Execution & EvaluationExpand the ‘Option’ menu - Open‘Journal abbreviation’ window - Click ‘Download’ button - enteran invalid URL - Click ‘OK’LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
GUITAR –Test Case Execution & EvaluationMalformedURLExceptionwith an invalid Journalabbreviation download URL:“CRASH ME”LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
GUITAR –Test Case Execution & EvaluationThe text “CRASH ME” was provided bythe testers – but the app should not havebeen crashed; instead it should havehandled the incorrect input properlyLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Structure of Lecture 8 GUI Testing GUITAR TESTAR Visual Testing SikuliX Lab 7 Security Testing Usability Testing A/B TestingLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022TESTAR
‘Scriptless’ testing tool Testar Scripts are generated automatically (on-the-fly) No maintenance of scripts needed Main purpose: Robustness testing– Robustness is the ability of a computer system to copewith errors during execution and cope with erroneousinputtestar.orgLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Testar tool (1) GUI contains graphical objects called widgets Widgets form a hierarchy called widget tree Widgets have properties (title, etc.) Widget tree and properies of each widget form a GUI state GUI state changes, if user executes an action (click, write, etc.)LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Testar tool (2) Example widget of Calculator programLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Testar tool (3) Widget tree ofCalculator programLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Testar tool (4) – What it can do .Testar always detects (noOracle needs to be defined) when SUT crashes when SUT hangscrashhangingLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Testar tool (4) – What it can do .Testar always detects (noOracle needs to be defined) when SUT crashes when SUT hangsTestar can also detect: incorrect behaviour if properoracles have been defined(checking widget titles)Defined list of suspicious titlesLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Testar tool (4) – What it can do .Testar always detects (noOracle needs to be defined) when SUT crashes when SUT hangsTestar can also detect: incorrect behaviour if properoracles have been defined(checking widget titles)LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Example:Checks a specific requirement,i.e., requesting that all imagesmust have a textual description(i.e., oracle checks that widget titletext is not empty)
Structure of Lecture 8 GUI Testing GUITAR TESTAR Visual Testing Sikuli IDE Lab 7 Security Testing Usability Testing A/B TestingLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Sikuli IDECapture imagesof the GUIThen write testsusing thoseimages e.g., to testemail forwardingwith Gmail LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Sikuli XxxxHow Sikuli X mimics the user’s behavior: LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Find all the check boxes for emailsShift coordinateClick on the subject lineWait for everything to loadSelect the options ‘Reply’ and‘Forward’ in the list of options
Sikuli IDEMany hints will be given in theLab 7 session and can be found in thedocumentation.LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Lab 7 – Visual GUI Testing (Sikuli IDE)Lab 7 (week 32: Apr 06 & 07) –Visual GUI Testing (9 points)InstructionsLab 8 Instructions & ToolsSpecSubmission Deadlines: Tuesday Labs: Wednesday Labs:Monday, 12 Apr, 23:59Tuesday, 13 Apr, 23:59 Penalties apply for late delivery: 50% penalty, ifsubmitted up to 24 hours late; 100 penalty, if submittedmore than 24 hours lateLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022ApplicationSikuli IDE /SikuliXTest Output
Structure of Lecture 8 GUI Testing GUITAR TESTAR Visual Testing Sikuli IDE Lab 7 Security Testing Usability Testing A/B TestingLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Software Product Quality Model– ISO 25010 StandardSecurityLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Security Testing – What? Security Testing Type of testing that intends to uncover vulnerabilities of the system and determine that its data and resources are protected from possibleintruders Focus Areas: Network security: This involves looking for vulnerabilities in thenetwork infrastructure (resources and policies). System software security: This involves assessing weaknesses inthe various software (operating system, database system, and othersoftware) the application depends on. Client-side application security: This deals with ensuring that theclient (browser or any such tool) cannot be manipulated. Server-side application security: This involves making sure thatthe server code and its technologies are robust enough to fend offany intrusion.LTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – Why? Primary purpose of security testing is to identify thevulnerabilities and subsequently repairing them Security Testing helps in improving the current system ensuring that the system will work for longer time finding out loopholes that can cause loss of importantinformationLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing –Basic Concepts 5.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing –Basic Concepts nAccountabilityNon-repudiation[Availability]is about ensuring that information is accessibleonly for those with authorized accessand preventing information theftLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing –Basic Concepts nAccountabilityNon-repudiation[Availability]is about allowing the receiver of data todetermine that the information which thedata is providing is correctLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing –Basic Concepts nAccountabilityNon-repudiation[Availability]is about establishing the identity of the userLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing –Basic Concepts nAccountabilityNon-repudiation[Availability]is about determining that a data requester isallowed to receive a service or performan operationLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing –Basic Concepts nAccountabilityNon-repudiation[Availability]is about taking responsibility (andtransparency about the responsibilitiestaken) for establishing securityLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing –Basic Concepts nAccountabilityNon-repudiation[Availability]is about preventing the later denial that anaction happened or a communicationtook place, etc.LTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing –Basic Concepts nAccountabilityNon-repudiation[Availability]is about assuring that information andcommunications services will be readyfor use when expected/neededLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include: Vulnerability Scanning Security Scanning Penetration Testing Ethical Hacking Risk Assessment Security Auditing Posture Assessment Password crackingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include:Involves scanning of the application for all Vulnerability Scanningknown vulnerabilities Security ScanningTypically done with the help of vulnerability Penetration Testingscanning software, e.g., Nessus, Nikto,Gendarme, Flawfinder, etc. Ethical Hacking Risk Assessment Security Auditing Posture Assessment Password crackingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include: Vulnerability ScanningInvolves scanning the usage of a systemin operation to monitor whether attacks on Security Scanningthe system are being tried or perhaps Penetration Testingprepared Ethical Hacking Risk Assessment Security Auditing Posture Assessment Password crackingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include: Vulnerability ScanningTesters try to (unauthorized) enter into the Security Scanningapplication with the help of another Penetration Testingsoftware system and via (combinations of)loopholes that the application has kept Ethical Hackingopen unknowingly Risk Assessment Security Auditing Posture Assessment Password crackingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include: Vulnerability Scanning Security ScanningInvolves the performance of penetration Penetration Testingtests over the wide network on the system Ethical Hackingunder testConducted by ethical hackers who try to Risk Assessmentfind possible problems in the system Security Auditing Posture Assessment Password crackingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include: Vulnerability Scanning Security Scanning Penetration TestingAnalyzing and deriving the risk as afunction of potential damage (or loss) and Ethical Hackingthe possibility of damage occurrence Risk AssessmentTypically carried out in the form of scenarioanalysis, interviews, and expert Security Auditingdiscussions Posture Assessment Password crackingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include: Vulnerability Scanning Security Scanning Penetration Testinginvolves hands on internal inspection of Ethical HackingOperating Systems and Applications, often Risk Assessmentvia line-by-line inspection of the codeA security audit is a systematic evaluation Security Auditingof the security of a software system used Posture Assessmentby a company Password crackingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include: Vulnerability Scanning Security Scanning Penetration Testing Ethical Hacking Risk Assessmentcombines Security Scanning, EthicalHacking and Risk Assessments to show Security Auditingan overall Security Posture ( attitudetowards security) of the organization using Posture Assessmenta software system Password CrackingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Main security testing practices include: Vulnerability Scanning Security Scanning Penetration Testing Ethical Hacking Risk AssessmentPassword cracking programs can be used Security Auditingto identify weak passwords Posture AssessmentPassword cracking verifies that users are Password Crackingemploying sufficiently strong passwordsLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Security Testing – What?Example of a basic security test: Log into the web application.Log out of the web application.Click the BACK button of the browser (Check if you are asked to log inagain or if you are provided the logged-in application.) Most types of security testing involve complex steps andout-of-the-box thinking but, sometimes, it is simple testslike the one above that help expose the most severesecurity ing.htmlLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Open Web Application Security Project The Open Web Application Security Project (OWASP) is agreat resource for software security professionals. Check out the Testing SP Testing ProjectOWASP Top 10 security threats for 2013 were: Injection Broken Authentication and Session Management Cross-Site Scripting (XSS) Insecure Direct Object References Security Misconfiguration Sensitive Data Exposure Missing Function Level Access Control Cross-Site Request Forgery (CSRF) Using Known Vulnerable Components Unvalidated Redirects and ForwardsLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
SELECT * FROM Users WHERE Username ' username' AND Password ' password' username 1' or '1' '1 password 1' or '1' '1SELECT * FROM Users WHERE Username '1' OR '1' '1' ANDPassword '1' OR '1' '1' The Open Web Application Security Project (OWASP) is agreat resource for software security professionals. Be sureto check out the Testing SP Testing ProjectOWASP Top 10 security threats for 2013 were: Injection Broken Authentication and Session Management Cross-Site Scripting (XSS) Insecure Direct Object References Security Misconfiguration Sensitive Data Exposure Missing Function Level Access Control Cross-Site Request Forgery (CSRF) Using Known Vulnerable Components Unvalidated Redirects and ForwardsLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
How to avoid SQL injectionvulnerability?Instead of:String query "SELECT * FROM Users WHERE Username ” request.getParameter(”username") “AND Password “ request.getParameter(”password");try {Statement statement connection.createStatement();ResultSet results statement.executeQuery(query);}Which might result in a SQL query string like this:SELECT * FROM Users WHERE Username '1' OR '1' '1' ANDPassword '1' OR '1' '1’LTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
How to avoid SQL injectionvulnerability?Use java ‘prepared statement’:String username request.getParameter(”username");String password request.getParameter(”password");// perform input validation to detect attacksString query "SELECT * FROM Users WHERE Username ? AND Password ?";PreparedStatement pstmt ( 1, username);pstmt.setString( 2, password);ResultSet results pstmt.executeQuery( );Example with Hibernate Query Language (HQL) can be found s/SQL Injection Prevention Cheat Sheet.htmlLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Structure of Lecture 8 GUI Testing GUITAR TESTAR Visual Testing Sikuli IDE Lab 7 Security Testing Usability Testing A/B TestingLTAT.05.003 / Lecture 8 / Dietmar Pfahl 2022
Software Product Quality Model– ISO 25010 StandardUsabilityLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Testing Usability RequirementsLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Testing Usability RequirementsHow to test:- Define several (typical) usagescenarios involving tasks Qand R- Select test users and classifyas ’novice’ and ’experienced’- Let 5 (or better 10, 15) novicesperform the secenarios- Observe what problems theyencounter- Classify and count observedproblemsLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Usability Test Types EnvironmentThink aloudRubin’s Types of Usability Tests(Rubin, 1994, p. 31-46)Exploratory test – early productdevelopmentAssessment test – most typical,either early or midway in the productdevelopmentValidation test – confirmation ofproduct’s usabilityComparison test – compare two ormore designs; can be used withother three types of testsLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Audio/videorecordingObserve
Usability Testing – What? How? Test Focus Understandability Easy to understand? Ease of learning Easy to learn? Operability Matches purpose & environmentof operation?Ergonomics: color, font, sound,. Communicativeness In accordance with psychologicalcharacteristics of user?LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022 Test Environments Free form tasksProcedure scriptsPaper screensMock-upsField trial
Evaluating UI DesignsInspection MethodsUsability itiveWalkthroughLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022GuidelinesReviewField Study
Evaluating UI DesignsInspection MethodsUsability itiveWalkthroughCheapLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022GuidelinesReviewField StudyExpensive
Evaluating UI DesignsInspection MethodsUsability Testing evaluates design on how well itLaboratorytaskHeuristicsupports user in learningExperimentEvaluation usually performed by expert incognitive psychologyCognitiveWalkthrough expertGuidelines walks though' design toidentify potential problems usingReviewpsychological principles Scenarios may be used to guideanalysisLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Field Study
Evaluating UI DesignsInspection MethodsHeuristicEvaluationCognitiveGuidelines usability criteria (heuristics)ReviewareWalkthroughidentified design examined by experts to see ifthese are violatedLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Usability TestingLaboratoryExperimentField Study
Heuristic Evaluation by InspectionList of 10 Heuristics according to (Nielsen, 2005):List violations of heuristics:Rank by severity: 0.40: positive (or neutral) aspect of system.4: major, catastrophic aspect of systemLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionVisibility of system status:At all times, the system shouldthe end userwhat isList of 10 Heuristics accordinginformto (Nielsen,2005):currently ongoing, e.g., viatimely and adequate feedback.List violations of heuristics:Rank by severity: 0.40: positive (or neutral) aspect of system.4: major, catastrophic aspect of systemLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionMatch between the system andthe real world:system shouldspeak theList of 10 Heuristics accordingTheto (Nielsen,2005):language of the end user (i.e., itshould use adequateterminology from the domain ofthe end user).List violationsof heuristics:The systemshouldfollow theconventionsof the0.4real worldRank by severity:0: positive (orneutral) aspect inof systemand presentinformationa.logic 4:andwell-structuredmajor,catastrophic aspect ofform.systemLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionUser control and freedom:End users often select a certainby accident;in thatList of 10 Heuristics accordingfunctionto (Nielsen,2005):case, they should be able toeasily find a way out of thisunwanted system state, e.g., bysimply choosing ‘undo’ or ‘redo’List violations of heuristics:functionality.Rank by severity: 0.40: positive (or neutral) aspect of system.4: major, catastrophic aspect of systemLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionConsistency and standards:End users should not have tothemselves2005):whetherList of 10 Heuristics accordingaskto (Nielsen,different words (or symbols) andactions mean the same or dothe same.Conventions that are typical forList violationsheuristics:the usedplatformofshouldbehonored.Rank by severity: 0.40: positive (or neutral) aspect of system.4: major, catastrophic aspect of systemLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionError prevention:Even better than erroris it todesgn the UIList of 10 Heuristics accordingmessagesto (Nielsen,2005):such that no errors occur, i.e.,situations in which the end usermight make a mistake shouldbe avoided.List violations of heuristics:Beforea transactionRankby severity: 0.4with0: positive(or neutral) aspectof systempotentiallyincorrectend user.input4:ismajor,executed,a aspectdialogcatastrophicof systemseeking confirmation is useful.LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionRecognition rather than recall:The cognitive load of the endcan be reducedby makingList of 10 Heuristics accordinguserto (Nielsen,2005):objects actions and optionsvisible and recognizable.End users should not be forcedList violationsof heuristics:to rememberinformationwhenmovingone dialogRankfromby severity:0.4 step to0: positive (or neutral) aspect of systemthe next.4: major, catastrophic aspect of systemHelp text should always beeasily visible or, at least, easyto find.LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionFlexibility and efficiency of use:Shortcuts and special keys –not known2005):by noviceList of 10 Heuristics accordingoftento (Nielsen,users – can be helpful to expertusers and make their interactionwith the system more efficient.List violationsof heuristics:The systemshouldbe usableaccordingthe needsRank bytoseverity:0.4 of0: positive(orgroupsneutral) aspect of systemdifferentuser.( flexibility).4: major, catastrophic aspect of systemLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionAesthetic and minimalistdesign:dialogsshouldn’tList of 10 Heuristics according(Standard)to (Nielsen,2005):contain any irrelevant or rarelyneeded information.Every unnecessary informationcompetes for the attention ofList userviolationsheuristics:the endwithoftherelevantinformationand thusRank by severity:0.4diminishes0: positive (or neutral) aspect of systemits visibility.4: major, catastrophic aspect of systemAesthetics: color, alignment,contrast, proximityLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionHelp users recognize,diagnose, and recover fromerrors:List of 10 Heuristics according to (Nielsen, 2005):Error messages should usesimple language, not containcryptic codes, clearly identifythe problem, and make aList violationsof heuristics:constructivesolutionto solvethe problem.Rank by severity: 0.40: positive (or neutral) aspect of system.4: major, catastrophic aspect of systemLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Heuristic Evaluation by InspectionHelp and documentation:Ideally, a system should bewithout documentationList of 10 Heuristics accordingusableto (Nielsen,2005):(à self-explanatory), sometimesit is useful to offer help.Help texts and documentationListbeviolationsheuristics:shouldeasy tooffindandsearch.Rank by severity: 0.40: positive (or neutral) aspect of system.Help 4:textsdocumentationmajor, andcatastrophicaspect of systemshould focus on the end users’(typical) needs and offerconcrete steps on how to usethe system for a specific task.LTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Evaluating UI DesignsInspection MethodsUsability itiveWalkthroughLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022GuidelinesReviewField Study
Evaluating UI DesignsWritten guidelines recommended for larger projects: Screen layout Appearance of objectsInspection MethodsUsability Testing Terminology Wording of prompts and error messages Menu’sLaboratoryHeuristic Direct manipulation actionsand feedbackExperimentEvaluation On-line help and other documentationCognitiveWalkthroughLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022GuidelinesReviewField Study
Guidelines for Screen Layout trastLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Guidelines for Screen Layout trastLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Objects that are close to another (related)Appear to form groups.
Guidelines for Screen Layout trastLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Alignment of objects generatesclarity / avoids confusion.
Guidelines for Screen Layout trastLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Visual aspects such as color, shapeand size link (similar) items together.
Guidelines for Screen Layout trastLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Contrast makes differences easierrecognizable.
Guidelines for Screen Layout trastLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Contrast makes differences easierrecognizable. Use: size, color, value, shape,position, direction,
Evaluating UI DesignsInspection MethodsUsability itiveWalkthroughLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022GuidelinesReviewField Study
Evaluating UI DesignsInspection MethodsHeuristicEvaluationUsability TestingLaboratoryExperimentUsability testing in a controlled environmentCognitiveGuidelines There is a test set of usersWalkthroughReview They perform pre-specified tasks Data is collected (quantitative and qualitative) Take mean and/or median value of measured attributes Compare to goal or another systemLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Field Study
Evaluating UI Designs1. Direct observation in actual usediscover new usesInspection Methodstake notes, don’t help, chat later2. Logging actual useobjective, not intrusiveHeuristicgreat for identifyingerrorsEvaluationwhich features are/are not usedprivacy concernsCognitiveWalkthroughLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022Usability TestingLaboratoryExperimentGuidelinesReviewField Study
Evaluating UI Designs3. Questionnaires and interviews with real usersask users to recall critical incidentsInspection MethodsUsability Testingquestionnaires must be short and easy to return4. Focus groups6-9 usersLaboratoryHeuristicskilled moderatorEvaluationwith pre-planned scriptExperimentcomputer conferencing?5 On-line direct feedback mechanismsCognitiveGuidelinesinitiatedby usersField StudyWalkthroughReviewmay signal change in user needstrust but verify6. Bulletin boards and user groupsLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
UI Design & UsabilityRecommendation for those who want to know moreabout UI Design (and Usability): Course: LTAT.05.007 - Human Computer Interaction Teacher: Alexander NolteLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
Structure of Lecture 8 GUI Testing GUITAR TESTAR Visual Testing Sikuli IDE Lab 7 Security Testing Usability Testing A/B TestingLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
A/B TestingTwo GUIVersionsA&BLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
A/B Testing (cont’d)Tool supportTwo GUIVersionsA&BRandomly selectedLTAT.05.006 / Lecture 08 / Dietmar Pfahl 2022
A/B Testing 25/best-testing-software/?LTAT.05.006
Lecture 1 (10.02) -Introduction to Software Testing . Penetration Testing Ethical Hacking Risk Assessment Security Auditing Posture Assessment Password cracking involves hands on internal inspection of Operating Systems and Applications, often via line-by-line inspection of the code
