How do you determine whether two sample proportions are statistically different?

 
Relevant MM Lessons:

Overview: Comparing Two Proportions

This question has its own dedicated lesson ». Make sure to check it out if you need a more thorough read, or want to become an expert on how proportions differences can show up on tests.Note that this how-to focuses on the hypothesis test approach. There is also a confidence interval approach which is less involved but also less frequently tested. Read about the confidence interval approach in its own lesson ».Like any question of statistical significance, we need a test statistic and a theoretical limit based on a chosen level of significance. When you encounter the comparison of two proportions in AP or college intro Stats courses, we will always use the same assumptions, formulas, and process.When we test the hypothesis that two proportions are not statistically different from one another, our null hypothesis is always$$H_0: \,\,\,\, p_1 - p_2 = 0$$Depending on our belief, our alternative hypothesis may be one tailed$$H_A: \,\,\,\, p_1 - p_2 > 0$$or$$H_A: \,\,\,\, p_1 - p_2 < 0$$or, our hypothesis may be two-tailed$$H_A: \,\,\,\, p_1 - p_2 \neq 0$$Some problems will tell you directly. Otherwise you should infer the correct alternative hypothesis based on the question being asked in the problem.To practice setting up these hypothesis, and for a more rigorous look at the following how-to steps, including checking that the appropriate conditions are met, check out the full lesson » on comparing proportions.Without further ado, follow these steps to get the hypothesis test result.

Method 1: TI-Calculator Only

If you are allowed to work problems without touching a pencil, you are among the lucky ones! AP students should be experts at the TI-84 statistics functions, but also knowledgeable about the following by-hand approach to explain written questions step-by-step and conceptually rigorously.Begin the hypothesis test on your TI-84 by pressing STAT, scrolling right to TESTS, and scrolling down to option 6, "2-PropZTest".Enter in the summary statistics from your two samples. Specifically, it wants you to enter the number of successes in sample 1, the total sample size of sample 1, the same two items for sample 2, an finally, what type of one-tailed or two-tailed hypothesis test you want to evaluate.Note that the TI-84 always pools the samples together for the Standard Error calculation, and you should too. More about this below in Step 2.If the $p$-value in your results is lower than your chosen significance level, which is typically $5\%$, then we reject the null hypothesis and instead accept the alternative hypothesis that there is a meaningful difference between the two proportions.

Method 2: Step by Step

Most of us, and especially AP test takers, need to be able to produce the scratch work which demonstrates the underlying statistics that determine whether we should reject or fail to reject the null hypothesis.Step 1 - The DifferenceCalculate the average difference, $\hat{p}_1 - \hat{p}_2$, where $\hat{p}_1$ is the proportion of successes from sample one, and $\hat{p}_2$ is the proportion of successes from sample two. If you want to symbolize these using the variables that the TI-84 uses, we would say$$\hat{p}_1 = \frac{x_1}{n_1}$$$$\hat{p}_2 = \frac{x_2}{n_2}$$where $x_1$ and $x_2$ are the number of successes in sample one and sample two, respectively, and $n_1$ and $n_2$ are the total sample sizes for sample one and sample two, respectively.Step 2 - The Standard ErrorCalculate the Standard Error of the difference using the given formula.$$SE = \sqrt{\hat{p}(1-\hat{p})\left[ \frac{1}{n_1} + \frac{1}{n_2} \right]}$$where $\hat{p}$ is the pooled sample proportion, combining both samples into one big sample. In other words,$$\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}$$This Standard Error calculation, using the pooled together $\hat{p}$, is always used for hypothesis testing because it is a better estimate of Standard Error under the assumption of the null hypothesis that the two sample proportions are equal. On very rare occasion, problems or profs will ask you to use the Standard Error calculation that we use for two proportion confidence intervals to conduct a hypothesis test, but this is super rare (see the full lesson » on the confidence interval approach).Step 3 - LimitsThe $z$ critical value is the highest test statistic that your sample can produce without suggesting that we reject the null hypothesis. This value depends on two things: first whether you are conducting a one-tailed or two-tailed test, and second, your chosen level of significance. Problems will usually tell you what significance level to use, but if not, assume $5\%$.Once you know those two metrics, we are ready to get the $z$ critical cut-off value. Depending on what resources you are expected to use, you will either look up the $z$ critical value in a table or use your TI-84 to obtain the $z$ value.If you are looking up the $z$ critical value with your TI-84, go to 2nd $\longrightarrow$ VARS and scroll down to option 3, "InvNorm". Obtain the $z$ value by entering the desired cumulative probability with a standard normal distribution ($\mu = 0$, $\sigma = 1$).For a significance level of $\alpha$, your cumulative percent is $1 - \alpha$ for a one-tailed test, and $1 - \alpha/2$ for a two-tailed test. This is explained in more detail in the full lesson about understanding hypothesis tests ». For example, a two-tailed test at $5\%$ significance needs you to enter $0.975$ here, but a one-tailed test with $10\%$ significance needs you to enter $0.9$ here.If you are looking up the $z$ critical value with a table, you still need to figure out the cumulative probability which is equal to either $1 - \alpha$ for a one-tailed test or $1 - \alpha/2$ for a two-tailed test, and find the closest $z$ value that has this cumulative probability.Either way, if you are doing a one-tailed test, make sure the $z$ number you got becomes negative if needed. Again, this is determined by the setup of the hypothesis and is discussed in more detail here ». If you are doing a two-tailed test, you should include both the positive and the negative of the $z$ critical number you found. Some teachers also want you to sketch a picture of a normal curve to show the rejection area.With your critical $z$ in hand, we're ready to compute, compare, and choose whether to reject the null.Step 4 - Judgement DayCalculate your $z$ test statistic as the quotient of the average difference from step 1 and the Standard Error from step 2.$$Z_{\mathrm{test}} = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})\left[ \frac{1}{n_1} + \frac{1}{n_2} \right]}}$$If your $z$ test statistic is larger than your $z$ critical value in magnitude, you should reject the null of a two-tailed hypothesis test (plus or minus doesn't matter). If your $z$ test statistic is larger than your $z$ critical value and they are the same sign (both plus or both minus), you should reject the null hypothesis of a one-tailed test.

Tips

  • Use technology wisely, but be prepared to write and understand the step-by-step for explanation purposes
  • Make sure you state your conclusions (reject or fail to reject) in complete sentences and always state the level of significance used to make your decision
  • Be ready to draw a normal curve with your $z$ critical value(s) to show the region where we would reject computed $z$ test statistics