Hypothesis testing: A-level Maths OCR revision notes

2.05

The language and logic of a test

Hypotheses, tails and the significance level 2.05

Definitions

Null hypothesis $H_{0}$ : a specific claimed value for a population parameter, assumed true for the purpose of the calculation.
Alternative hypothesis $H_{1}$ : the statement of how the true value might differ from that claim.
One-tailed test: $H_{1}$ specifies a direction, such as $p > p_{0}$ or $μ < μ_{0}$ .
Two-tailed test: $H_{1}$ specifies only that the parameter differs, such as $μ \neq = μ_{0}$ .
Significance level: the threshold probability, fixed in advance, for how unlikely the observed data must be under $H_{0}$ before $H_{0}$ is rejected.

Key results

The significance level, commonly $5%$ , is also the probability of rejecting $H_{0}$ when it is in fact true, so lowering it makes false alarms rarer but genuine effects harder to detect.
In a two-tailed test at the $5%$ level the significance is split between the tails, giving $2.5%$ in each.

Notes

The significance level must be chosen before looking at the data; selecting it afterwards to make a result 'significant' invalidates the test.
The direction of a one-tailed test must also come from the context stated before the data is seen, not from the direction the data happens to point.
A two-tailed test is the more cautious default when there is no prior reason to expect the effect in a particular direction — reserve a one-tailed test for when the context genuinely rules out the other direction in advance.

The logic of a test and how to state a conclusion 2.05

Definitions

Test statistic: the quantity computed from the sample whose distribution is known under $H_{0}$ .
Critical region: the set of values of the test statistic for which $H_{0}$ is rejected.

Key results

The two possible outcomes are: reject $H_{0}$ in favour of $H_{1}$ (the data would have been sufficiently unlikely under $H_{0}$ ), or conclude there is insufficient evidence to reject $H_{0}$ .
The two comparison routes are equivalent: comparing the probability of the observed result with the significance level, or comparing the test statistic with the critical value.

Method

Define the parameter being tested in words, in context.
State $H_{0}$ and $H_{1}$ , and the significance level.
State the distribution of the test statistic assuming $H_{0}$ is true.
Compute either the probability of an outcome at least as extreme as the one observed, or the value of the test statistic.
Compare with the significance level or the critical value.
Write the conclusion in context, referring back to the original claim.

In practiceA coin is suspected of bias towards heads. In 20 throws it shows 15 heads. Test at the 5% level.

$p = P (head); H_{0} : p = 0.5, H_{1} : p > 0.5$ one-tailed, 5%
$X \sim B (20, 0.5) under H_{0}$
$P (X \geq 15) = 0.0207 < 0.05$ at least as extreme as observed
Reject $H_{0}$ : there is sufficient evidence at the 5% level that the coin is biased towards heads.

Notes

Never write that the test 'accepts' or 'proves' $H_{0}$ , since failing to find evidence against a claim is not the same as proving the claim true.
A conclusion stated only as 'reject $H_{0}$ ' is incomplete: it must be translated back into the language of the original context.
Statistical significance is not practical importance — a very large sample can make a tiny, meaningless difference significant.
A test that fails to reject $H_{0}$ has not shown $H_{0}$ is true — it has only shown the sample doesn't give enough evidence against it; a larger sample might yet reject it.

2.05

Testing a binomial proportion

Hypothesis testing for a binomial proportion 2.05

Key results

Under $H_{0}$ the number of successes follows $X \sim B (n, p_{0})$ , and the test finds the probability of an outcome at least as extreme as the one observed.
For an upper-tail test ( $H_{1} : p > p_{0}$ ) compute $P (X \geq x) = 1 - P (X \leq x - 1)$ ; for a lower-tail test ( $H_{1} : p < p_{0}$ ) compute $P (X \leq x)$ .
If that probability is less than the significance level, $H_{0}$ is rejected.

Method

Define $p$ as the population proportion in context and state $H_{0} : p = p_{0}$ against the appropriate $H_{1}$ .
State that $X \sim B (n, p_{0})$ under $H_{0}$ .
Compute the tail probability for the observed value, using cumulative binomial tables or a calculator.
Compare with the significance level (or with half of it, in each tail of a two-tailed test).
Conclude in context.

In practice30% of voters are claimed to support a party. In a random sample of 25, 3 do. Test at the 5% level whether the claim is wrong.

$H_{0} : p = 0.3, H_{1} : p \neq = 0.3$ two-tailed
$X \sim B (25, 0.3) under H_{0}$
$P (X \leq 3) = 0.0332 > 0.025$ compare with half the level, in the lower tail
Do not reject $H_{0}$ : insufficient evidence at the 5% level that the proportion differs from 30%.

Notes

For small samples, exact binomial probabilities are used rather than a normal approximation, since the conditions for that approximation ( $n p_{0} > 5$ and $n (1 - p_{0}) > 5$ ) may not hold — always check these conditions before deciding which method to use.
The tail probability must include the observed value itself, not just the outcomes beyond it: 'at least as extreme as' means $\geq x$ , not $> x$ .
State $p$ as a population proportion in the actual context ('the proportion of all such patients who improve'), not as an abstract symbol — a mark is routinely reserved for this contextual definition.

Critical regions and actual significance level 2.05

Definitions

Actual significance level: the true probability that the test statistic falls in the critical region when $H_{0}$ is true.

Key results

Because a binomial variable is discrete, the critical region cannot usually be made to carry exactly $5%$ of the probability, so its actual significance level is somewhat less than the nominal level.
The critical region is the largest tail whose total probability under $H_{0}$ still does not exceed the significance level.

Method

Tabulate cumulative probabilities under $H_{0}$ near the relevant tail.
For an upper tail, find the smallest $c$ with $P (X \geq c) <$ the significance level; the critical region is $X \geq c$ .
Quote the actual significance level as that probability $P (X \geq c)$ , expressed as a percentage.
Check the neighbouring value to confirm that including it would push the probability above the significance level.

In practiceUnder $H_{0}$ , $X \sim B (20, 0.3)$ , with $H_{1} : p > 0.3$ at the 5% level. Find the critical region.

$P (X \geq 9) = 0.1133, P (X \geq 10) = 0.0480, P (X \geq 11) = 0.0171$ upper-tail probabilities
$critical region: X \geq 10$ the smallest $c$ with $P (X \geq c) < 0.05$
$actual significance level = 4.80%$ including 9 would push it to 11.33%

Notes

Once the critical region is known, testing any observed value is immediate — this is why critical regions are computed in advance for repeated quality-control checks.
Always check the value just outside the critical region as well as the boundary itself — this confirms the region really is the largest one still under the significance level, not merely a plausible-looking guess.

Worked examples

Worked example

A coin is tossed $20$ times and lands heads $15$ times.

Test, at the $5%$ significance level, whether the coin is biased towards heads.

Show worked solution

Let $p$ be the probability that the coin lands heads.

$H_{0} : p = 0.5$ , $H_{1} : p > 0.5$ (one-tailed, since bias towards heads is specified).

Under $H_{0}$ , $X \sim B (20, 0.5)$ , where $X$ is the number of heads.

The probability of a result at least as extreme as the one observed is:

P (X \geq 15) = 1 - P (X \leq 14) = 1 - 0.9793 = 0.0207

Since:

0.0207 < 0.05

the result lies in the critical region, so $H_{0}$ is rejected.

There is significant evidence at the $5%$ level that the coin is biased towards heads.

Worked example

A standard treatment is known to succeed for $40%$ of patients.

In a trial of $20$ patients given a new treatment, only $3$ improve.

Test, at the $5%$ significance level, whether the new treatment has a lower success rate.

Show worked solution

Let $p$ be the probability that a patient improves under the new treatment.

$H_{0} : p = 0.4$ , $H_{1} : p < 0.4$ (one-tailed, lower).

Under $H_{0}$ , $X \sim B (20, 0.4)$ .

The lower-tail probability is:

P (X \leq 3) = 0.0160

(from cumulative binomial tables).

Since:

0.0160 < 0.05

$H_{0}$ is rejected.

There is significant evidence at the $5%$ level that the new treatment has a lower success rate than the standard one.

Note that a normal approximation would be inappropriate here: although:

n p_{0} = 8 > 5

the sample is small and the exact binomial is available.

Worked example

It is claimed that $30%$ of customers at a shop pay in cash.

A random sample of $20$ customers is taken to test $H_{0} : p = 0.3$ against $H_{1} : p > 0.3$ at the $5%$ significance level.

Find the critical region and the actual significance level of the test.

Show worked solution

Under $H_{0}$ , $X \sim B (20, 0.3)$ .

Using cumulative binomial tables,

P (X \leq 8) = 0.8867

and:

P (X \leq 9) = 0.9520

For an upper-tail critical region $X \geq c$ : taking $c = 9$ gives:

P (X \geq 9) = 1 - P (X \leq 8) = 1 - 0.8867 = 0.1133

which exceeds $0.05$ , so $9$ is not in the critical region.

Taking $c = 10$ gives:

P (X \geq 10) = 1 - P (X \leq 9) = 1 - 0.9520 = 0.0480

which is less than $0.05$ .

So the critical region is $X \geq 10$ , and the actual significance level is $0.0480$ , or $4.80%$ .

It falls short of the nominal $5%$ because $X$ is discrete: no critical region of this form has probability exactly $0.05$ .

2.05

Testing for correlation

Hypothesis testing for correlation 2.05

Key results

The test asks whether an observed sample correlation coefficient $r$ provides evidence of a genuine linear correlation in the underlying population, rather than arising purely by chance in this particular sample.
The hypotheses are stated in terms of the population correlation coefficient $ρ$ : $H_{0} : ρ = 0$ against $H_{1} : ρ > 0$ , $H_{1} : ρ < 0$ or $H_{1} : ρ \neq = 0$ .
The sample value $r$ is compared against a critical value read from tables, which depends on the sample size and the chosen significance level.
Reject $H_{0}$ when $∣ r ∣$ exceeds the critical value; larger samples have smaller critical values, so a modest $r$ can be significant if $n$ is large.

Notes

For a two-tailed test, use the critical value listed for half the significance level in each tail.
A significant result says only that a linear association exists in the population; it says nothing about how strong or how useful that association is.
Larger samples make smaller values of $r$ significant, so a 'significant' correlation from a large data set can still correspond to a fairly weak linear relationship in practical terms.

Correlation and causation 2.05

Key results

A statistically significant correlation is evidence of a linear association, not evidence of causation.
Two variables can be strongly correlated because one causes the other, because both are driven by a third factor, or by coincidence, and a hypothesis test for correlation cannot itself distinguish between these explanations.

Notes

Distinguishing causation requires a controlled experiment with randomised allocation, not a larger observational sample.
When asked to comment on a significant correlation, name a plausible third factor for the context given — that is what the mark is for.
'Reverse causation' is worth considering too — for two correlated variables $X$ and $Y$ , it's possible $Y$ influences $X$ rather than the assumed direction $X$ influences $Y$ .

Worked example

For a random sample of $12$ pairs of observations, the product moment correlation coefficient is $r = 0.582$ .

Test at the $5%$ significance level whether there is positive correlation in the population, and state whether the conclusion would change at the $1%$ level.

Show worked solution

Let $ρ$ be the population product moment correlation coefficient.

$H_{0} : ρ = 0$ , $H_{1} : ρ > 0$ (one-tailed).

For $n = 12$ at the $5%$ one-tailed level, the critical value from tables is $0.4973$ .

Since:

0.582 > 0.4973

$H_{0}$ is rejected: there is significant evidence at the $5%$ level of positive linear correlation in the population.

At the $1%$ one-tailed level the critical value for $n = 12$ is $0.6581$ , and:

0.582 < 0.6581

so at that level there would be insufficient evidence to reject $H_{0}$ .

In either case the test shows association, not causation.

2.05

Testing a normal mean

The distribution of the sample mean 2.05

Key results

If $X \sim N (μ, σ^{2})$ and a random sample of size $n$ is taken, then the sample mean satisfies $\overset{ˉ}{X} \sim N (μ, \frac{σ ^{2}}{n})$ .
The standard deviation of the sample mean is therefore $\frac{σ}{n}$ , which shrinks as $n$ grows — quadrupling the sample size halves the spread of the sample mean.
This is why larger samples give more precise estimates, and why the same observed difference from $μ_{0}$ becomes more significant as $n$ increases.

Notes

Note the divisor: it is the variance that is divided by $n$ , so the standard deviation is divided by $n$ , not by $n$ .
Don't confuse $σ$ (the spread of individual measurements) with $\frac{σ}{n}$ (the spread of the sample mean) — they answer different questions and are never interchangeable in a formula.

Hypothesis testing for a normal mean 2.05

Key results

Testing a claimed population mean $μ_{0}$ with known population standard deviation $σ$ uses $\overset{ˉ}{X} \sim N (μ_{0}, \frac{σ ^{2}}{n})$ under $H_{0}$ .
The test statistic is $z = \frac{x ˉ - μ _{0}}{σ / n}$ , compared against a critical value from the standard normal distribution.
A two-tailed test at the $5%$ significance level uses critical values $\pm 1.96$ ; if the calculated $z$ falls beyond either of these, $H_{0}$ is rejected in favour of $H_{1} : μ \neq = μ_{0}$ .
Other standard critical values: $\pm 1.645$ for a one-tailed $5%$ test, $\pm 2.326$ for a one-tailed $1%$ test, and $\pm 2.576$ for a two-tailed $1%$ test.

Method

State $H_{0} : μ = μ_{0}$ and the appropriate $H_{1}$ , with the significance level.
State $\overset{ˉ}{X} \sim N (μ_{0}, \frac{σ ^{2}}{n})$ under $H_{0}$ .
Compute $z = \frac{x ˉ - μ _{0}}{σ / n}$ .
Compare $z$ with the critical value, or equivalently compare the tail probability with the significance level.
Conclude in context.

In practiceMasses are $N (μ, 2. 5^{2})$ . Test $H_{0} : μ = 40$ against $H_{1} : μ \neq = 40$ at the 5% level, given $n = 16$ and $\overset{x}{ˉ} = 41.4$ .

$\overset{ˉ}{X} \sim N (40, \frac{2. 5 ^{2}}{16}) under H_{0}$
$z = \frac{41.4 - 40}{2.5/4} = 2.24$
$2.24 > 1.96$ two-tailed: 2.5% in each tail
Reject $H_{0}$ : there is evidence at the 5% level that the mean is not 40.

Notes

The test requires the population standard deviation to be known and the underlying variable to be normally distributed (or the sample to be large); if $σ$ is estimated from the sample, a different test is needed.
The two routes agree exactly, because the critical value is simply the $z$ whose tail probability equals the significance level — quoting both is a useful self-check.
For a two-tailed test, always compare $∣ z ∣$ (or the observed mean's absolute distance from $μ_{0}$ ) with the positive critical value — comparing a negative $z$ directly against a positive critical value gives a false 'not significant' result.

Worked examples

Worked example

A machine is set to fill bottles with a mean volume of $500$ ml; the population standard deviation is known to be $5$ ml.

A random sample of $25$ bottles has a mean volume of $497.5$ ml.

Test, at the $5%$ significance level, whether the mean volume has changed.

Show worked solution

$H_{0} : μ = 500$ , $H_{1} : μ \neq = 500$ (two-tailed).

Test statistic:

z = \frac{497.5 - 500}{5/ 25} = \frac{- 2.5}{1} = - 2.5

The two-tailed $5%$ critical values are $\pm 1.96$ .

Since:

∣ - 2.5∣ = 2.5 > 1.96

the test statistic lies in the critical region, so $H_{0}$ is rejected: there is significant evidence at the $5%$ level that the mean volume has changed.

Worked example

Packets are filled to a claimed mean of $250$ g with known population standard deviation $4$ g.

A sample of $16$ packets has mean $248.2$ g.

Test at the $5%$ level whether the mean has decreased, and comment on what would happen at the $1%$ level.

Show worked solution

$H_{0} : μ = 250$ , $H_{1} : μ < 250$ (one-tailed).

Under $H_{0}$ ,

\overset{ˉ}{X} \sim N (250, \frac{4 ^{2}}{16}) = N (250, 1)

so the standard deviation of the sample mean is:

\frac{4}{16} = 1

Test statistic:

z = \frac{248.2 - 250}{1} = - 1.8

The one-tailed $5%$ critical value is $- 1.645$ , and:

- 1.8 < - 1.645

so $H_{0}$ is rejected: there is significant evidence at the $5%$ level that the mean has decreased.

Equivalently, the tail probability is:

P (Z < - 1.8) = 1 - Φ (1.8) = 1 - 0.9641 = 0.0359 < 0.05

giving the same conclusion.

At the $1%$ level the critical value is $- 2.326$ , and $- 1.8$ does not reach it (equivalently:

0.0359 > 0.01

), so there would be insufficient evidence to reject $H_{0}$ — the same data can be significant at one level and not at another.