WIXLIB

416Decision Sciencesthemselves; on other occasions, it may be necessary only to group the data and present results in theform of tables or in a graphical form. The summarization and exposition of the different importantaspects of the data is commonly called descriptive statistics. This idea includes the condensation ofthe data in the form of tables, their graphical presentation, and computation of numerical indicatorsof the central tendency and variability.There are mainly two main aspects of describing a data set:1. Summarization and description of the overall pattern of the data byDownloaded by [Debasis Kundu] at 16:48 25 January 2017a. Presentation of tables and graphsb. Examination of the overall shape of the graphical data for important features, includingsymmetry or departure from itc. Scanning graphical data for any unusual observations, which seems to stick out fromthe major mass of the data2. Computation of the numerical measures fora. A typical or representative value that indicates the center of the datab. The amount of spread or variation present in the dataSummarization and description of the data can be done in different ways. For a univariate data,the most popular methods are histogram, bar chart, frequency tables, box plot, or the stem and leafplots. For bivariate or multivariate data, the useful methods are scatter plots or Chernoff faces. Awonderful exposition of the different exploratory data analysis techniques can be found in Tukey(1977), and for some recent development, see Theus and Urbanek (2008).A typical or representative value that indicates the center of the data is the average value or themean of the data. But since the mean is not a very robust estimate and is very much susceptible tothe outliers, often, median can be used to represent the center of the data. In case of a symmetricdistribution, both mean and median are the same, but in general, they are different. Other than meanor median, trimmed mean or the Windsorized mean can also be used to represent the central valueof a data set. The amount of spread or the variation present in a data set can be measured using thestandard deviation or the interquartile range.8.3 ProbabilityThe main aim of this section is to introduce the basic concepts of probability theory that are usedquite extensively in developing different statistical inference procedures. We will try to providethe basic assumptions needed for the axiomatic development of the probability theory and willpresent some of the important results that are essential tools for statistical inference. For furtherstudy, the readers may refer to some of the classical books in probability theory such as Doob(1953) or Billingsley (1995), and for some recent development and treatment, readers are referredto Athreya and Lahiri (2006).8.3.1 Sample Space and EventsThe concept of probability is relevant to experiments that have somewhat uncertain outcomes. Theseare the situations in which, despite every effort to maintain fixed conditions, some variation of theresult in repeated trials of the experiment is unavoidable. In probability, the term “experiment” is

Statistical Methods417not restricted to laboratory experiments but includes any activity that results in the collection ofdata pertaining to the phenomena that exhibit variation. The domain of probability encompasses allphenomena for which outcomes cannot be exactly predicted in advance. Therefore, an experimentis the process of collecting data relevant to phenomena that exhibits variation in its outcomes. Letus consider the following examples:Experiment (a). Let each of 10 persons taste a cup of instant coffee and a cup of percolatedcoffee. Report how many people prefer the instant coffee.Experiment (b). Give 10 children a specific dose of multivitamin in addition to their normaldiet. Observe the children’s height and weight after 12 weeks.Downloaded by [Debasis Kundu] at 16:48 25 January 2017Experiment (c). Note the sex of the first 2 new born babies in a particular hospital on a givenday.In all these examples, the experiment is described in terms of what is to be done and what aspect ofthe result is to be recorded. Although each experimental outcome is unpredictable, we can describethe collection of all possible outcomes.DefinitionThe collection of all possible distinct outcomes of an experiment is called the sample space of theexperiment, and each distinct outcome is called a simple event or an element of the sample space.The sample space is denoted by .In a given situation, the sample space is presented either by listing all possible results of theexperiments, using convenient symbols to identify the results or by making a descriptive statementcharacterizing the set of possible results. The sample space of the above three experiments can bedescribed as follows:Experiment (a). {0, 1, . . . , 10}.Experiment (b). Here, the experimental result consists of the measurements of two characteristics, height and weight. Both of these are measured on a continuous scale. Denoting themeasurements of gain in height and weight by x and y, respectively, the sample space canbe described as {(x, y); x nonnegative, y positive, negative or zero.}Experiment (c). {BB, BG, GB, GG}, where, for example, BG denotes the birth of a boyfirst and then followed by a girl. Similarly, the other symbols are also defined.In our study of probability, we are interested not only in the individual outcomes ofany collection of outcomes of .but also inDefinitionAn event is any collection of outcomes contained in the sample space . An event is said to besimple, if it consists of exactly one outcome, and compound, if it consists of more than one outcome.DefinitionA sample space consisting of either a finite or a countably infinite number of elements is called adiscrete sample space. When the sample space includes all the numbers in some interval (finite orinfinite) of the real line, it is called continuous sample space.

418Decision Sciences8.3.2 Axioms, Interpretations, and Properties of ProbabilityGiven an experiment and a sample space , the objective of probability is to assign to each event A, anumber P(A), called probability of the event A, which will give a precise measure of the chance thatA will occur. To ensure that the probability assignment will be consistent with our intuitive notionof probability, all assignments should satisfy the following axioms (basic properties) of probability: Axiom 1: For any event A, 0 P(A) 1.Axiom 2: P( ) 1. Axiom 3: If {A1 , A2 , . . .} is an infinite collection of mutually exclusive events, then P(A1 A2 A3 . . .) P(Ai ).Downloaded by [Debasis Kundu] at 16:48 25 January 2017i 1Axiom 1 reflects the intuitive notion that the chance of A occurring should be at least zero, sothat negative probabilities are not allowed. The sample space is by definition an event that mustoccur when the experiment performed ( ) contains all possible outcomes. So, Axiom 2 says thatthe maximum probability of occurrence is assigned to . The third axiom formalizes the idea that ifwe wish the probability that at least one of a number of events will occur, and no two of the eventscan occur simultaneously, then the chance of at least one occurring is the sum of the chances ofindividual events.Consider an experiment in which a single coin is tossed once. The sample space is {H , T }.The axioms specify P( ) 1, so to complete the probability assignment, it remains only to determine P(H ) and P(T ). Since H and T are disjoint events, and H T , Axiom 3 impliesthat 1 P( ) P(H ) P(T ). So, P(T ) 1 P(H ). Thus, the only freedom allowed by theaxioms in this experiment is the probability assigned to H . One possible assignment of probabilities is P(H ) 0.5, P(T ) 0.5, while another possible assignment is P(H ) 0.75, P(T ) 0.25.In fact, letting p represent any fixed number between 0 and 1, P(H ) p, P(T ) 1 p is anassignment consistent with the axioms.8.3.3 Borel σ-Field, Random Variables, and ConvergenceThe basic idea of probability is to define a set function whose domain is a class of subsets of thesample space , whose range is [0, 1], and it satisfies the three axioms mentioned in the previoussubsection. If is the collection of finite number or countable number of points, then it is quite easyto define the probability function always, for the class of all subsets of , so that it satisfies Axioms1–3. If is not countable, it is not always possible to define for the class of all subsets of . Forexample, if R, the whole real line, then the probability function (from now onward, we call itas a probability measure) is not possible to define for the class of all subsets of . Therefore, wedefine a particular class of subsets of R, called Borel σ-field (it will be denoted by B); see Billingsley(1995) for details, on which probability measure can be defined. The triplet ( , B, P) is called theprobability space, while or ( , B) is called the sample space.Random variable: A real-valued point function X (·) defined on the space ( , B, P) is calleda random variable of the set {ω : X (ω) x} B, for all x R.Distribution function: The point functionF(x) P{ω : X (ω) x} P(X 1 ( , x]),defined on R, is called the distribution function of X .

Statistical Methods419Now, we will define three important concepts of convergence of a sequence of randomvariables. Suppose {X n } is a sequence of random variables, and X is also a random variable,and all are defined of the same probability space ( , B, P).Convergence in probability or weakly: The sequence of random variables {X n } is said topconverge to X in probability (denoted by X n X ) if for all 0,lim P( X n X ) 0.n Almost sure convergence or strongly: The sequence of random variables {X n } is said toa.e.converge to X strongly (denoted by X n X ), if P lim X n X 1.Downloaded by [Debasis Kundu] at 16:48 25 January 2017n Convergence in distribution: The sequence of random variables {X n } is said to converge todX in distribution (denoted by X n X ), iflim Fn (x) F(x),n for all x, such that F is continuous at x. Here, Fn and F denote the distribution functionsof X n and X , respectively.8.3.4 Some Important ResultsIn this subsection, we present some of the most important results of probability theory that havedirect relevance in statistical sciences. The books by Chung (1974) or Serfling (1980) are referredfor details.The characteristic function of a random variable X with the distribution function F(x) is definedas follows: it X φ X (t) E eeit x d F(x), for t R, where i 1. The characteristic function uniquely defines a distribution function. For example,if φ1 (t) and φ2 (t) are the characteristic functions associated with the distribution functions F1 (x)and F2 (x), respectively, and φ1 (t) φ2 (t), for all t R, then F1 (x) F2 (x), for all x R.Chebyshev’s theorem: If {X n } is a sequence of random variables, such that E(X i ) μi ,V (X i ) σi2 , and they are uncorrelated, then1 2σ 0 limn n 2 i 1nni 11Xi n nμip 0.i 1Khinchine’s theorem: If {X n } is a sequence of independent and identically distributed randomvariables, such that E(X 1 ) μ , then1n nnpX i μ.limi 1

420Decision SciencesKolmogorov theorem 1: If {X n } is a sequence of independent random variables, such thatE(X i ) μi , V (X i ) σi2 , then i 1 σi2 i2n1ni 11Xi n nμia.s. 0.i 1Kolmogorov theorem 2: If {X n } is a sequence of independent and identically distributedrandom variables, then a necessary and sufficient condition thatnDownloaded by [Debasis Kundu] at 16:48 25 January 20171na.s.Xi μi 1is that E(X 1 ) , and it is equal to μ.Central limit theorem: If {X n } is a sequence of independent and identically distributedrandom variables, such that E(X 1 ) μ, and V (X 1 ) σ2 , then1 σ nnd(X i μ) Z .i 1Here, Z is a standard normal random variable with mean zero and variance 1.Example 8.1Suppose X 1 , X 2 , . . . is a sequence of i.i.d. exponential random variable with the followingprobability density function for x 0:e x0f (x) if x 0,if x 0.In this case, E(X 1 ) V (X 1 ) 1. Therefore, by the weak law of large numbers (WLLN) ofKhinchine, it immediately follows that1nnpX i 1,i 1and by Kolmogorov’s strong law of large numbers (SLLN),1nna.e.X i 1.i 1Further, by the central limit theorem (CLT), we have1 nnd(X i 1) Z N (0, 1).i 1

Statistical Methods4218.4 EstimationDownloaded by [Debasis Kundu] at 16:48 25 January 20178.4.1 IntroductionThe topic of parameter estimation deals with the estimation of some parameters from the data thatcharacterizes the underlying process or phenomenon. For example, one is posed with the data takenrepeatedly of the same temperature. These data are not equal, although the underlying true temperature was the same. In such a situation, one would like to obtain an estimate of the true temperaturefrom the given data.We may also be interested in finding the coefficient of resolution of a steel ball from the dataon successive heights to which the ball rose. One may be interested in obtaining the face flowspeed of vehicles from data on speed and density. All these estimation problems come under thepurview of parameter estimation. The question of estimation arises because one always tries toobtain knowledge on the parameters of the population from the information available through thesample. The estimate obtained depends on the sample collected. Further, one could generally obtainmore than one sample from a given population, and therefore the estimates of the same parametercould be different from one another. Most of the desirable properties of an estimate are definedkeeping in mind the variability of the estimates.In this discussion on the said topic, we will look into desirable properties of an estimator, andsome methods for obtaining estimates. We will also see some examples that will help to clarifysome of the salient features of parameter estimation. Finally, we will introduce the ideas of intervalestimation, and illustrate its relevance to real-world problems.8.4.2 Desirable PropertiesThe desirable properties of an estimator are defined keeping in mind that the estimates obtained arerandom. In the following discussion, T will represent an estimate while θ will represent the trueparameter value of a parameter. The properties that will be discussed are the following:Unbiasedness: The unbiasedness property states that E(T ) θ. The desirability of this property is self-evident. It basically implies that on an average the estimator should be equal tothe parameter value.Minimum variance: It is also desirable that any realization of T (i.e., any estimate) may not befar off from the true value. Alternatively stated, it means that the probability of θ being nearto θ should be high, or as high as possible. This is equivalent to saying that the varianceof T should be minimal. An estimator that has the minimum variance in the class of allunbiased estimators is called an efficient estimator.Sufficiency: An estimator is sufficient if it uses all the information about the population parameter, θ, that is available from the sample. For example, the sample median is not a sufficientestimator of the population mean, because median only utilizes the ranking of the samplevalues and not their relative distance. Sufficiency is important because it is a necessarycondition for the minimum variance property (i.e., efficiency).Consistency: The property of consistency demands that an estimate be very close to the truevalue of the parameter when the estimate is obtained from a large sample. More specifically,if limn P( T θ ) 1, for any 0, however small it might be, the estimator Tis said to be a consistent estimator of the parameter θ. It may be noted that if T has a zerobias, and the variance of T tends to zero, then T is a consistent estimator of θ.

422Decision SciencesAsymptotic properties: The asymptotic properties of estimators relate to the behavior of theestimators based on a large sample. Consistency is thus an asymptotic property of anestimator. Other asymptotic properties include asymptotic unbiasedness and asymptoticefficiency.As the nomenclature suggests, asymptotic unbiasedness refers to the unbiasedness of an estimatorbased on a large sample. Alternatively, it can be stated as follows:lim E(T ) θ.Downloaded by [Debasis Kundu] at 16:48 25 January 2017n For example, an estimator whose E(T ) θ (1/n) is an asymptotically unbiased estimator of θ.For small samples, however, this estimator has a finite negative bias.Similarly, asymptotic efficiency suggests that an asymptotically efficient estimator is the minimumvariance unbiased estimator of θ for large samples. Asymptotic efficiency may be thought of as thelarge sample equivalent of best unbiasedness, while asymptotic unbiasedness may be thought of asthe large sample equivalent of unbiasedness property.Minimum mean square error: The minimum mean square error (MSE) property states thatthe estimator T should be such that the quantity MSE defined below is minimum:MSE E(T θ)2 .Alternatively written,MSE V ar (T ) (E(T ) θ)2 .Intuitively, it is appealing because it looks for an estimator that has small bias (may be zero)and small variance. This property is appealing because it does not constrain an estimatorto be unbiased before looking at the variance of the estimator. Thus, the minimum MSEproperty does not give higher importance to unbiasedness than to variance. Both the factorsare considered simultaneously.Robustness: Another desirable property of an estimator is that the estimator should not bevery sensitive to the presence of outliers or obviously erroneous points in the data set.Such an estimator is called a robust estimator. The robust property is important because,loosely speaking, it captures the reliability of an estimator. There are different ways inwhich robustness is quantified. Influence function and breakdown point are two such methods. Influence functions describe the effect of one outlier on the estimator. Breakdown pointof an estimator is the proportion of incorrect observations (for example, arbitrarily largeobservations) an estimator can handle before g

Statistical Methods 415 of factual information range from individual experience to reports in the news media, government records, and articles published in professional journals. Weather forecasts, market reports, costs of living indexes, and the results of public opinion are some