Normal equations
+
+
Least-squares calculations
+
+
Solution for normal equations
+
+
Standard error of estimate
+
+
Test Statistic for inferences concerning β
+
Or it can be written as
+
+
Confidence interval for regression coefficient β
+
+
Confidence interval for the mean of y when x = x0
+
+
Coefficient of correlation
+
+
+
Estimated Equation
+
+
Where
+SST is the total sum of squares,
+SSR is the sum of squares due to regression,
+SSE is the sum of squares due to error.
++
Multiple Coefficient of Determination
+
Which is the % of variation of y can be explained by the sample regression line.
++
Adjusted Multiple Coefficient of Determination
+
Ra2 will always be smaller than R2.
++
The relationship between the explanatory X and the response variable Y should be linear.
+Methods for fitting a model to non-linear relationships exist but are beyond the scope of this course.
+Check using a scatterplot of the data, or a residuals plot.
++
The residuals should be nearly normal.
+This condition may not be satisfied when there are unusual observations that do not follow the trend of the rest of the data.
+Check using histogram or normal probability plot of residuals.
++
The variability of points around the least squares line should be roughly constant.
+This implies that the variability of residuals around the 0 line should be roughly constant as well.
+It is also called homoscedasticity.
+Check using histogram or normal probability plot of residuals.
++
To determine whether a significant relationship exists between the dependent variable y and the set of all the independent variables x.
++
Setting H0 and H1:
+
+
+
Setting H0 and H1:
+
+
The correlation among the independent variables.
+When the independent variables are highly correlated, say |r| > 0.7, it is not possible to determine the separate effect on the dependent variable.
+Every attempt should be made to avoid including independent variables that are highly correlated.
+Two predictor variables are said to be collinear when they are correlated, and this collinearity complicates model estimation.
+Predictors that are associated with each other are not preferred to be added into the model, as often the addition of such variables brings nothing to the table. Instead, the simplest model is preferred or say the parsimonious model.
+While it is not possible to avoid collinearity from arising in observational data, experiments are usually designed to prevent correlations among predictors.
++
Such as genders (male, female), method of payment (cash, check, credit card).
++
For example, X1 might represent gender where X1 = 0 indicates male and X1 = 1 indicates female.
+In this case, X1 is called a dummy or indicator variable.
++
If a qualitative variable has k levels, k - 1 dummy variables are required, with each dummy variable being coded as 0 or 1.
+For example, a variable with levels A, B, and C could be represented by X1 and X2 values of (0, 0) for A, (1, 0) for B, and (0, 1) for C.
+Care must be taken in defining and interpreting the dummy variables.
++
In multiple regression analysis it is preferable to use the residual plot against ŷ to determine if the model assumptions are satisfied.
++
Standardized residuals are frequently used in residual plots.
+Identifying outliers (typically, standardized residuals < -2 or > 2)
+Providing insight into the assumption that the error term e has a normal distribution.
++
The computation of the standardized residuals in multiple regression analysis is too complex to be done by hand, Excel regression tool can be used.