5
USES OF T-TEST A one-sample location test of whether the mean of a normally distributed population has a value specified in a null hypothesis. A two sample location test of the null hypothesis that the means of two normally distributed populations are equal 4

6
USES OF T-TEST A test of the null hypothesis that the difference between two responses measured on the same statistical unit has a mean value of zero A test of whether the slope of a regression line differs significantly from 0 5

7
BACKGROUND If comparing means among > 2 groups, 3 or more t-tests are needed -Time-consuming (Number of t-tests increases) -Inherently flawed (Probability of making a Type I error increases) 6

9
HISTORY Fisher proposed a formal analysis of variance in his paper The Correlation Between Relatives on the Supposition of Mendelian Inheritance in His first application of the analysis of variance was published in Become widely known after being included in Fisher's 1925 book Statistical Methods for Research Workers in

10
DEFINITION An abbreviation for: ANalysis Of VAriance The procedure to consider means from k independent groups, where k is 2 or greater. 9

11
ANOVA and T-TEST ANOVA and T-Test are similar -Compare means between groups 2 groups, both work 2 or more groups, ANOVA is better 10

15
Definition ANOVA can determine whether there is a significant relationship between variables. It is also used to determine whether a measurable difference exists between two or more sample means. Objective: To identify important independent variables (predictor variables – y i ’s) and determine how they affect the response variables. One-way, two-way, or multi-way ANOVA depend on the number of independent variables there are in the experiment that affect the outcome of the hypothesis test. 14

31
Residual plot Violations of the basic assumptions and model adequacy can be easily investigated by the examination of residuals. We define the residual for observation j in treatment i as If the model is adequate, the residuals should be structureless; that is, they should contain no obvious patterns. 30

32
Normality Why normal? – ANOVA is an Analysis of Variance – Analysis of two variances, more specifically, the ratio of two variances – Statistical inference is based on the F distribution which is given by the ratio of two chi-squared distributions – No surprise that each variance in the ANOVA ratio come from a parent normal distribution Normality is only needed for statistical inference. 31

37
Independence Independent observations – No correlation between error terms – No correlation between independent variables and error Positively correlated data inflates standard error – The estimation of the treatment means are more accurate than the standard error shows. 36

38
SAS code for independence test The plot of the residual against the factor is used to check the independence. proc plot; plot resid* indu; run; 37

39
Independence Tests 38

40
Homogeneity of Variances Eisenhart (1947) describes the problem of unequal variances as follows – the ANOVA model is based on the proportion of the mean squares of the factors and the residual mean squares – The residual mean square is the unbiased estimator of  2, the variance of a single observation – The between treatment mean squares takes into account not only the differences between observations,  2, just like the residual mean squares, but also the variance between treatments – If there was non-constant variance among treatments, we can replace the residual mean square with some overall variance,  a 2, and a treatment variance,  t 2, which is some weighted version of  a 2 – The “neatness” of ANOVA is lost 39

41
Sas code for Homogeneity of Variances test The plot of residuals against the fitted value is used to check constant variance assumption. proc plot; plot resid* yhat; run; 40

48
Derivation – 1-Way ANOVA – Cont’ We can show that Using the above equation, we define 47

49
Derivation – 1-Way ANOVA – Cont’ Given the distributions of the MSS values, we can reject the null hypothesis if the between group variance is significantly higher than the within group variance. That is, We reject the null hypothesis if F > f n-1,N-n,α 48

59
Univariate Procedure Code proc univariate data=stockfit plot normal; var resid; We use the proc univariate to produce the stem-and-leaf and normal probability plots and we use the stem- leaf plot to visualize the overall distribution of a variable. 58

71
Conclusion After the analysis of one way anova test,we can get the result of F=1.00 and p= Since the p-value is bigger, we accept the null hypothesis which indicates that there is no difference between the mean of daily average percentage change of stocks of different industries. Thus, there is no different if we buy the stocks in different industries in the long term. 70

80
Model An extension of one way ANOVA. It provides more insight about how the two IVs interact and individually affect the DV. Thus, the main effects and interaction effects of two IVs have on the DV need to be tested. Model: Null hypothesis: 79

81
Sum of Squares Every term compared with the error term leads to F distribution. In this way, we can conclude whether there is main effect or interaction effect. SS TOTAL = SS A + SS B + SS INTERACTION + SS ERROR 80

82
Example Using the same data from the One-Way analysis, we will now separate the data further by introducing a second factor, Average Daily Volume. 81

88
To test the main effect of one IV, we should combine all the data of the other IV. And this is done in the one way ANOVA. From the ANOVA we know there is no significant main effects or interaction effect of the two IVs. To indicate if there is an interaction effect, we can plot of means of each cell formed by combination of all levels of IVs. 87 Using SAS

89
PLOT OF CELL MEANS Industry by Average Daily Volume 88

90
Interpreting the Output Given that the F tests were not significant we would normally stop our analysis here. If the F test is significant, we would want to know exactly which means are different from each other. Use Tukey’s Test. MEANS INDUSTRY | VOLUME / TUKEY CLDIFF; 89

92
Conclusion We cannot conclude that there is a significant difference between any of the group means. The two IVs have no effects on the DV. 91

93
92

94
M-way ANOVA (Derivation) Let us have n factors, A 1,A 2,…,A n, each with 2 or more levels, a 1,a 2,…,a n, respectively. Then there are N = a 1 a 2 …a n types of treatment to conduct, with each treatment having sample size n i. Let x i 1 i 2 …i n k be the k th observation from treatment i 1 i 2 …i n. By the assumption for ANOVA, x i 1 i 2 …i n k is a random variable that follows the normal distribution. Using the model x i 1 i 2 …i n k = µ i 1 i 2 …i n k + ε i 1 i 2 …i n k where each (residual) ε i 1 i 2 …i n k are i.i.d. and follows N(0,σ 2 ). 93

95
M-way ANOVA (Derivation) 94

96
M-way ANOVA (Derivation) 95

97
M-way ANOVA (Derivation) These are all distributed as independent χ 2 random variables (when multiplied by the correct constants and when some hypotheses hold) with d.f. satisfying the equation: 96

98
M-way ANOVA (Derivation) There are a total of 2 m hypotheses in an m- way ANOVA. – The null hypothesis, which states that there is no difference or interaction between factors – For k from 1 to m, there are mCk alternative hypotheses about the interaction between every collection of k factors. – Then we have 1 + mC1 + mC2 + … + mCm = 2 m by a well known combinatorial identity. 97

99
M-way ANOVA (Derivation) These hypotheses are: 98

100
M-way ANOVA (Derivation) We want to see if the variability between groups is larger that the variability within the groups. To do this, we use the F distribution as our pivotal quantity, and then we can derive the proper tests, very similar to the 1-way and 2- way tests. 99

103
What we know: – regression is the statistical model that you use to predict a continuous outcome on the basis of one or more continuous predictor variables. – ANOVA compares several groups (usually categorical predictor variables) in terms of a certain dependent variable(continuous outcome ) ( if there are mixture of categorical and continuous data, ANCOVA is an alternative method.) Take a second look: They are the just different sides of the same coin! 102

104
Review of ANOVA Compare the means of different groups n groups, n i elements for ith group, N element in total. SST= + SS between SS within How about only two group,X and Y, Each have n data? 103

105
Review of Simple Linear Regression We try to find a line y = β 0 + β 1 x that best fits our data so that we can calculate the best estimate of y from x It will find such β 0 and β 1 that minimize the distance Q between the actual and estimated score Let predicted value be of one group, while the other group consist all of original value.. It is a special (and also simple) case of ANOVA! Minimize me 104

111
Notes: Both of them are applicable only when outcome variables are continuous. They share basically the same procedure of checking the underlying assumption. 110

112
Robust ANOVA -Taguchi Method 111

113
What is Robustness? The term “robustness” is often used to refer to methods designed to be insensitive to distributional assumptions (such as normality) in general, and unusual observations (“outliers”) in particular. Why Robust ANOVA? There is always the possibility that some observations may contain excessive noise. excessive noise during experiments might lead to incorrect inferences. Widely used in Quality control 112

115
Robust ANOVA Our aim is to minimize by choosing β: In standard ANOVA, we let we can also try some other ρ(x). 114

116
Least absolute deviation It is well-known that the median is much more robust to outliers than the mean. least absolute deviation (LAD) estimate, which takes How is LAD related to median? the LAD estimator determines the “center” of the data set by minimizing the sum of the absolute deviations from the estimate of the center, which turns out to be the median. It has been shown to be quite effective in the presence of fat tailed data 115

117
M-estimation M-estimation is based on replacing ρ(.) with a function that is less sensitive to unusual observations than is the quadratic. The M means we should keep ρ follows MLE. LSD with, is an example of a robust M-estimator. Another popular choice of ρ : Tukey bisquare: and (;)1rcρ= otherwise, where r is the residual and c is a constant. 116

118
Suggestion these robust analyses may not take the place of standard ANOVA analyses in this context; Rather, we believe that the robust analyses should be undertaken as an adjunct to the standard analyses 117