Lecture 9

Lecture 9

Outline

Goodness-of-Fit Tests

Given the following

  1. Counts of items in each of several categories

  2. A model that predicts the distribution of the relative frequencies

How good is the fit?

Assumptions and Condition

Example 1

| | | |-|-| |fit1|fit2|

Chi-Square Model

To decide if the null model is plausible, look at the differences between the observed values and the values expected if the model were true.

\[\chi^2 = \sum_{\text{all cells}} \frac{(Obs - Exp)^2}{Exp}\]

So, $\chi^2$ gets “big” when

  1. the data set is large and/or

  2. the model is a poor fit.

The Chi-Square Calculation

  1. Find the expected values. These come from the null hypothesis model. Every null model gives a hypothesized proportion for each cell. The expected value is the product of the total number of observations times this proportion.

Example 1 (cont.)

The Chi-Square Calculation (cont.)

  1. Compute the residuals. Once you have expected values for each cell, find the residuals, $Obs-Exp$.

  2. Square the residuals, $(Obs-Exp)^2$.

  3. Compute the components. Find for each cell, $(Obs-Exp)^2/Exp$

  4. Find the sum of the components. That’s the chi-square statistic,

\[\chi^2 = \sum_{\text{all cells}} \frac{(Obs - Exp)^2}{Exp}\]

The Chi-Square Calculation (cont.)

  1. Find the degrees of freedom. It’s equal to the number of cells minus one.

  2. Test the hypothesis. Large chi-square values mean lots of deviation from the hypothesized model, so they give small P-values. Look up the critical value from a table of chi-square values, or find the P-value directly.

Example 2

At a major credit card bank, the percentages of people who historically apply for the Silver, Gold, and Platinum cards are 60%, 30%, and 10% respectively. In a recent sample of customers, 110 applied for Silver, 55 for Gold, and 35 for Platinum. Is there evidence to suggest the percentages have changed?

What type of test do you conduct? This is a goodness-of-fit test comparing a single sample to previous information (the null model).

Example 2 (cont.)

What are the expected values?

ValuesSilverGoldPlatinum
Observed1105535
Expected1206020
\[\chi^2 = \frac{(110-120)^2}{120} + \frac{(55-60)^2}{60} + \frac{(35-20)^2}{20} = 12.499\]

Find the test statistic and P-value.

Reject the null hypothesis. There is sufficient evidence customers are not applying for cards in the traditional proportions.

Interpreting Chi-Square Values

The $\chi^2$ distribution is right-skewed and becomes broader with increasing degrees of freedom:

chi

The $\chi^2$ test is a one-sided test.

Examining the Residuals

\[\frac{Obs - Exp}{\sqrt{Exp}}\]

The Chi-Square Test for Homogeneity

How important is it to seek your utmost attractive appearance?

chi2h1

The Chi-Square Test for Homogeneity (cont.)

Convert the results to "column percentages":

chi2h2

The Chi-Square Test for Homogeneity (cont.)

It seems that India stands out from the others.

chi2h3

Our null hypothesis is that the relative frequency distributions are homogeneous for each country.

Test the hypothesis with a chi-square test for homogeneity.

The Chi-Square Test for Homogeneity (cont.)

The Chi-Square Test for Homogeneity (cont.)

chi2h4

The Chi-Square Test for Homogeneity (cont.)

Following the pattern of the goodness-of-fit test, we compute the component for each cell of the table:

\[Component = \frac{(Obs - Exp)^2}{Exp}\]

Summing these components across all cells gives $\chi^2$:

\[\chi^2 = \sum_{\text{all cells}} \frac{(Obs - Exp)^2}{Exp}\]

For a test of homogeneity, there are degrees of freedom, $(R-1)\times(C-1)$ where $R$ is the number of rows and $C$ is the number of columns.

Comparing Two Proportions

Are women more likely to graduate high school than men, or are the differences due to random variation?

twopr1

Use this proportion to compute the expected values.

Comparing Two Proportions (cont.)

Observed CountsExpected Values
twopr1twopr2

The chi-square statistic with (2-1)*(2-1) = 1 df is

\[\chi^2 = 54.941\]

A chi-square test with 1 df, is equivalent to testing whether two proportions are equal.

Chi-Square Test of Independence

Are Age and Appearance independent, or is there a relationship?

chi2i1

Chi-Square Test of Independence (cont.)

chi2i1

Test for independence using a chi-square test of independence.

Chi-Square Test of Independence (cont.)

The test is mechanically equivalent to the test for homogeneity, but with some differences in how we think about the data and the results:

Chi-Square Test of Independence (cont.)

Chi-Square Test of Independence (cont.)

For the Appearance and Age example, we reject the null hypothesis that the variables are independent.

\[\chi^2 = 170.7762\]
\[\text{P-value} < 0.001\]

So, it may be of interest to know how differently two age groups select the "very important" category (Appearance response 6 or 7).

Chi-Square Test of Independence (cont.)

Construct a confidence interval for the true difference in proportions

The 95% confidence interval is found below:

\[(\hat{p}_1 - \hat{p}_2) \pm SE(\hat{p}_1 -\hat{p}_2) =\]
\[(0.4517-0.3991) \pm 1.96 \times \sqrt{ \frac{(0.4517)(0.5483}{1596} + \frac{(0.3991)(0.6009}{1521}} =\]

= from 1.8% to 8.7%

This is a statistically significant difference, but now we can see that the difference may be as small as 1.8%.

Example 3

Consumer Reports uses surveys to measure reliability in automobiles. Annually they release survey results about problems that consumers have had with vehicles in the past 12 months and the origin of manufacturer. Is consumer satisfaction related to country of origin?

Example 3 (cont.)

State the hypotheses.

Find the test statistic.

\[\chi^2 = 2.928\]

Given p-value = 0.231, state your conclusion.