Transcription
Chapter 13Design of Experiments (DoE)Prof. Dr. Mesut Güneş Ch. 13 Design of Experiments13.1
Contents Introduction to DoETypes of experimental designs2k Factorial design2kr Factorial design with replications2k-p Fractional factorial designProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.2
Introduction to DoEProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.3
Design of Experiments Example: Study the performance of a system in respect toparticular parameters System: routing algorithm for a MANET Parameters: Number of nodes: N {10, 20, 50, 100, 1000, 10000}Mobility: M {1 m/s, 3 m/s, 5 m/s, 10 m/s}Packet size: P {64 byte, 256 byte, 512 byte, 1024 byte}Number of parallel flows: F {1, 3, 5, 7, 10} Parameter space: N x M x P x F 6 x 4 x 4 x 5 480 Question: how to perform the experiments to understandthe effects of the parameters?Prof. Dr. Mesut Güneş Ch. 13 Design of Experiments13.4
Design of Experiments Answer: Design of Experiments (DoE) The goal is to obtainmaximum informationwith theminimum number of experimentsProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.5
TerminologyResponse variable:The outcome of an experimentFactor:Each variable that affects the response variableand has several alternativesLevel:The values that a factor can assumePrimary Factor:The factors whose effects need to be quantifiedSecondary Factor:Factors that impact the performance but whoseimpact we are not interested in quantifyingReplication:Repetition of all or some experimentsExperimental Unit:Any entity that is used for the experimentInteraction:Two factors A and B interact if the effect of onedepends upon the level of the otherProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.6
Interaction of factorsNo InteractionA1InteractionA1Prof. Dr. Mesut Güneş Ch. 13 Design of ExperimentsB1A1B2A2A2B1B2B1A1B2A2A2B1B213.7
Design Design: An experimental design consists of specifying thenumber of experiments, the factor level combinations foreach experiment, and the number of replications. In planning an experiment, you have to decide1. what measurement to make (the response)2. what conditions to study3. what experimental material to use (the units) Example1. Measure goodput and overhead of a routing protocol2. Network with n nodes in chain3. Routing protocol, type of nodes, type of links, trafficProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.8
Types of experimental designsProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.9
Types of experimental designs: Simple design Simple design Start with a configuration and vary one factor at a time Given k factors and the i-th factor having ni levels The required number of experimentskn 1 (ni 1)i 1 Example: k 3, {n1 3, n2 4, n3 2} n 1 (2 3 1) 7Prof. Dr. Mesut Güneş Ch. 13 Design of Experiments13.10
Types of experimental designs: Full factorial design Full factorial design Use all possible combinations at all levels of all factors Given k factors and the i-th factor having ni levels The required number of experimentskn nii 1 Example: k 3, {n1 3, n2 4, n3 2} n 3 4 2 24Prof. Dr. Mesut Güneş Ch. 13 Design of Experiments13.11
Types of experimental designsFractional factorial design Fractional factorial design When full factorial design results in a huge number ofexperiments, it may be not possible to run all Use subsets of levels of factors and the possible combinationsof these Given k factors and the i-th factor having ni levels, andselected subsets of levels mi ni . The required number of experimentskn mii 1 Example: k 3, {n1 3, n2 4, n3 2}, but use {m1 2, m2 2, m3 1} n 2 2 1 4Prof. Dr. Mesut Güneş Ch. 13 Design of Experiments13.12
Types of experimental designs Comparison of the design typesDesign TypeFactorsSimple designk 3, {n1 3, n2 4, n3 2}Full factorial designFractional factorial designProf. Dr. Mesut Güneş Ch. 13 Design of ExperimentsNumber ofexperiments724Use subset{m1 2, m2 2, m3 1}413.13
2k Factorial DesignsProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.14
2k Factorial Designs A 2k factorial design is used to determine the effect of kfactors Each factor has two levels Advantages It is easy to analyze Helps to identify important factorsÆ reduce the number of factors Often effect of a factor is unidirectional, i.e., performanceincrease or decrease Begin by experimenting at the minimum and maximum levelof a factor Æ two levelsProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.15
2k Factorial DesignsExample for k 2 Study impact of memoryand cache on performanceof a workstation Memory size, two levels Cache size, two levelsFactor 1Memory SizeFactor 2CacheSize4 MB16 MB115-1,-1451,-1225-1,11,175 Performance ofworkstation as regressionmodely q0 q A x A qB xB q AB x A xBProf. Dr. Mesut Güneş Ch. 13 Design of Experiments 1 if 4MB memoryx A 1 if 16MB memory 1 if 1kb cachexB 1 if 2kb cache13.16
2k Factorial DesignsExample for k 2 Regression modelExperimenty q0 q A x A qB xB q AB x A xB Substitute the results intoAByAB1-1-1y1121-1y2-1y1 q0 q A qB q AB3-11y3-1y2 q0 q A qB q AB411y41the modely3 q0 q A qB q ABy4 q0 q A qB q AB Solve equantions for qiq0 14 ( y1 y2 y3 y4 )q A 14 ( y1 y2 y3 y4 )qB 14 ( y1 y2 y3 y4 )y 40 20xA 10xB 5xA xBq AB 14 ( y1 y2 y3 y4 )Prof. Dr. Mesut Güneş Ch. 13 Design of Experiments13.17
2k Factorial DesignsExample for k 2: Sign table method Sign table contains the effect of 204010205TotalTotal/4Prof. Dr. Mesut Güneş Ch. 13 Design of ExperimentsResult13.18
2k Factorial DesignsExample for k 2: Allocation of variation Determine the importance of a factor Calculate the variance22s y2 (yi y)2i 122 1 Sum of squares total (SST): Total variation of y22y SST ( yi y ) 2i 1 For 22 design, the variation is given by2SST 2 2 q A2 2 2 qB2 22 q AB SSASSBSSAB SSA: part explained by factor A Fraction of variation explained by A: SSA/SSTProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.19
2k Factorial DesignsThe General Case In the general case there are k factors, each factor hastwo levels A total of 2k experiments are required Analysis produces 2k effects (results) k main effects k2 two-factor interactions k3 three-factor interactions Sign table method is used!Prof. Dr. Mesut Güneş Ch. 13 Design of Experiments13.20
2k Factorial DesignsThe General Case Sign table, example for k 3IA1A2A3 --- -y1 ---- y2 - -- - y3 - --y4 -- - y5 - ----y6 - -- -y7 y8Prof. Dr. Mesut Güneş Ch. 13 Design of ExperimentsA1A2 A1A3 A2A3A1A2A3y13.21
2k Factorial DesignsThe General Case Sign tableIA1A2A3 A1A2 A1A3 A1A2A3 y1-1y111y21-1y3 SumISumI/2kProf. Dr. Mesut Güneş Ch. 13 Design of Experiments TotalTotal/2k13.22
2kr Factorial Design with ReplicationsProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.23
2kr Factorial Design with Replications Problem with 2k factorial design is that it does not providethe estimation of experimental errors, since no repetitions Solution: Repeat an experiment r times Æ replication If each of the 2k experiments is repeated r timesÆ 2kr factorial design with replications Extended modely q0 q A x A qB xB q AB x A xB eExperimental errorProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.24
2kr Factorial Design with Replications For analysis, the same method is used, except for y, themean of the replications is 5TotalTotal/4 Experimental error is given: eij yij y Sum of squared errors (SSE) and the standard deviationof errors:22rSSE eij2i 1 j 1Prof. Dr. Mesut Güneş Ch. 13 Design of Experimentsse SSE2 2 ( r 1)13.25
2k-p Fractional Factorial DesignProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.26
2k-p Fractional Factorial Design When the number of factors is large, a full factorial designrequires a large number of experiments In that case fractional factorial design can be used Requires fewer experiments, e.g., 2k-1 requires half of theexperiments as a full factorial designProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.27
2k-p Fractional Factorial Design Preparing the sign table Choose k-p factors and prepare a complete sign table.Æ Sign table with 2k-p rows and 2k-p columns The first column will be marked I and consists of all 1s The next k-p columns will be marked with the k-p factors thatwere chosen The remaining columns are simply products of these factorsProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.28
2k-p Fractional Factorial Design Sign table, example for k 7, p 4 Æ 27-4 23k-p chosen factorsproducts of chosen factorsIF1F2F3F1F2F1F3F2F3F1F2F3 --- - ---- - -- - - -- -- - - ---- - -- - 2k-p rows2k-p columnsProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.29
2k-p Fractional Factorial Design Confounding with fractional factorial design some of the effects can not bedetermined only combined effects of several factors can be computed A fractional factorial design is not unique Design resolution The resolution of a design is measured by the order ofeffects that are confounded The order of effect is the number of factors included in itI ABC order of 3 Æ Resolution RIIII ABCD order of 4 Æ Resolution RIV A design of higher resolution is considered a better design.Prof. Dr. Mesut Güneş Ch. 13 Design of Experiments13.30
Summary Design of experiments provides a method for plannedexperiments Goal: Obtain maximum information with minimumexperiments Basic techniques Factorial design Factorial design with replications Fractional factorial designProf. Dr. Mesut Güneş Ch. 13 Design of Experiments13.31
13.8 Design Design: An experimental design consists of specifying the number of experiments, the factor level combinations for each experiment, and the number of replications. In planning an experiment, you have to decide 1. what measurement to make (the response)File Size: 654KB
