Further Statistics — Continuous distributions: A-level Further Maths OCR revision notes

5.03

Probability density functions

Probability density functions 5.03

Definitions

Probability density function (PDF): a function $f$ with $f (x) \geq 0$ and $\int_{- \infty}^{\infty} f (x) d x = 1$ , such that $P (a \leq X \leq b) = \int_{a}^{b} f (x) d x$ .
Mode: the value of $x$ at which $f (x)$ is greatest.

Key results

$E (X) = \int x f (x) d x$ and $E (g (X)) = \int g (x) f (x) d x$ .
$Var (X) = \int x^{2} f (x) d x - [E (X)]^{2}$ .
$P (X = a) = 0$ for any single value, so $<$ and $\leq$ give the same probabilities.

Method

Find an unknown constant from total area 1, over the whole range where $f$ is non-zero.
Sketch $f$ first. Symmetry often gives $E (X)$ at once, and the sketch locates the mode, which may be at an end point.

In practice $f (x) = k x (4 - x)$ for $0 \leq x \leq 4$ . Find $k$ , $E (X)$ and $Var (X)$ .

$k \int_{0}^{4} x (4 - x) d x = k [2 x^{2} - \frac{1}{3} x^{3}]_{0}^{4} = \frac{32}{3} k = 1 ⟹ k = \frac{3}{32}$ total area 1
$E (X) = 2$ the PDF is symmetric about $x = 2$
$E (X^{2}) = \frac{3}{32} \int_{0}^{4} (4 x^{3} - x^{4}) d x = \frac{3}{32} [x^{4} - \frac{1}{5} x^{5}]_{0}^{4} = \frac{3}{32} \cdot \frac{256}{5} = 4.8$
$Var (X) = 4.8 - 2^{2} = 0.8$

Notes

$f (x)$ is a density, not a probability: it can exceed 1, as on the sheet where $f (2) = 1.5$ .
For a piecewise PDF, integrate each piece over its own interval and add.

Worked example

f (x) = k (4 - x^{2})

for $0 \leq x \leq 2$ and 0 otherwise.

Find $k$ , $E (X)$ , $Var (X)$ and $P (X < 1)$ .

Show worked solution

k \int_{0}^{2} (4 - x^{2}) d x = k (8 - \frac{8}{3}) = \frac{16 k}{3} = 1

so:

k = \frac{3}{16}

E (X) = \frac{3}{16} \int_{0}^{2} (4 x - x^{3}) d x = \frac{3}{16} (8 - 4) = \frac{3}{4}

E (X^{2}) = \frac{3}{16} (\frac{32}{3} - \frac{32}{5}) = \frac{4}{5}

so:

Var (X) = \frac{4}{5} - \frac{9}{16} = \frac{19}{80}

P (X < 1) = \frac{3}{16} (4 - \frac{1}{3}) = \frac{11}{16}

5.03

Cumulative distribution functions

Cumulative distribution functions, medians and quartiles 5.03

Definitions

Cumulative distribution function (CDF): $F (x) = P (X \leq x) = \int_{- \infty}^{x} f (t) d t$ .
Median $m$ : $F (m) = \frac{1}{2}$ . Lower and upper quartiles: $F (q) = \frac{1}{4}$ and $F (q) = \frac{3}{4}$ .

Key results

$f(x)=F'(x)$ wherever $F$ is differentiable.
$F$ is continuous and non-decreasing, from 0 to 1. $P (a < X \leq b) = F (b) - F (a)$ .

Method

To find $F$ from a piecewise $f$ , integrate piece by piece, adding the total probability of the earlier pieces as the constant, so that $F$ is continuous at each join.
Define $F$ for all real $x$ , including $F (x) = 0$ below the range and $F (x) = 1$ above it.

In practice $f (x) = \frac{x}{8}$ for $0 \leq x \leq 4$ . Find $F (x)$ , the median and the mean.

$F (x) = \int_{0}^{x} \frac{t}{8} d t = \frac{x ^{2}}{16} (0 \leq x \leq 4)$
$F (x) = 0 (x < 0), F (x) = 1 (x > 4)$ define $F$ for every real $x$
$\frac{m ^{2}}{16} = \frac{1}{2} ⟹ m = 22 \approx 2.83$ reject $- 22$ : outside the range of $X$
$E (X) = \int_{0}^{4} \frac{x ^{2}}{8} d x = \frac{8}{3} \approx 2.67$ mean below the median: negative skew

Notes

When solving $F (m) = \frac{1}{2}$ produces more than one root, keep only the one inside the range of $X$ .
Comparing mean and median indicates skew. On the sheet, median $1.59 >$ mean $1.5$ : negative skew.

Worked example

F (x) = \frac{1}{8} (x^{2} + 2 x)

for $0 \leq x \leq 2$ , with $F = 0$ below and $F = 1$ above.

Find $f (x)$ and the median.

Show worked solution

f(x)=F&#039;(x)=\tfrac14(x+1)

for $0 \leq x \leq 2$ .

Median:

\frac{1}{8} (m^{2} + 2 m) = \frac{1}{2}

so $m^{2} + 2 m - 4 = 0$ and $m = - 1 \pm 5$ .

Only:

m = 5 - 1 = 1.24

lies in $[0, 2]$ .

5.03

The exponential and continuous uniform distributions

The exponential distribution 5.03

Definitions

Exponential distribution $X \sim Exp (λ)$ : $f (x) = λ e^{- λ x}$ for $x \geq 0$ .

Key results

$F (x) = 1 - e^{- λ x}$ , so $P (X > x) = e^{- λ x}$ .
$E (X) = \frac{1}{λ}$ and $Var (X) = \frac{1}{λ ^{2}}$ ; the median is $\frac{ln 2}{λ}$ .
Memoryless property: $P (X > s + t ∣ X > s) = P (X > t)$ .
If events occur as a Poisson process at rate $λ$ , the time between successive events is $Exp (λ)$ .

Notes

The memoryless property makes it a model for components that do not wear out: a bulb that has lasted 1000 hours is as good as new. Where wear matters, it is a poor model.
Watch the parameter: a mean lifetime of 200 hours means $λ = \frac{1}{200}$ , not 200.
Why it is memoryless: $P (X > s + t ∣ X > s) = \frac{P ( X > s + t )}{P ( X > s )} = \frac{e ^{- λ (s + t)}}{e ^{- λ s}} = e^{- λ t} = P (X > t)$ .

The continuous uniform distribution 5.03

Definitions

Continuous uniform (rectangular) distribution $X \sim U (a, b)$ : $f (x) = \frac{1}{b - a}$ for $a \leq x \leq b$ .

Key results

$E (X) = \frac{a + b}{2}$ and $Var (X) = \frac{( b - a ) ^{2}}{12}$ .
$P (c \leq X \leq d) = \frac{d - c}{b - a}$ for $a \leq c \leq d \leq b$ .

Notes

Rounding errors are uniform: a length rounded to the nearest cm has error $U (- 0.5, 0.5)$ .
Probabilities are lengths of intervals as fractions of $b - a$ ; no integration is needed.
Where the variance comes from: $E (X^{2}) = \frac{1}{b - a} \int_{a}^{b} x^{2} d x = \frac{a ^{2} + ab + b ^{2}}{3}$ , so $Var (X) = \frac{a ^{2} + ab + b ^{2}}{3} - \frac{( a + b ) ^{2}}{4} = \frac{( b - a ) ^{2}}{12}$ .

Worked examples

Worked example

Bulbs have exponentially distributed lifetimes with mean 200 hours.

Find the probability that a bulb lasts more than 300 hours, and the probability that a bulb that has lasted 200 hours lasts at least another 300.

Show worked solution

λ = \frac{1}{200}

so:

P (X > 300) = e^{- 300/200} = e^{- 1.5} = 0.223

By the memoryless property the second probability is the same, 0.223: the bulb's age is irrelevant under this model.

Worked example

Lengths are rounded to the nearest centimetre.

Find the variance of the rounding error and the probability that the error is less than 0.2 cm in size.

Show worked solution

The error is $U (- 0.5, 0.5)$ , with variance:

\frac{1 ^{2}}{12} = \frac{1}{12}

P (∣ E ∣ < 0.2) = \frac{0.4}{1} = 0.4

5.07

The Wilcoxon signed-rank test

The Wilcoxon signed-rank test 5.07

Definitions

Non-parametric test: a test that makes no assumption that the population follows a particular distribution, such as the normal; hypotheses are about the median.
Wilcoxon signed-rank test: a test for the median of a single population, or for paired data, using the ranks of the sizes of the differences from the hypothesised median.

Key results

$P$ = sum of the ranks of positive differences, $Q$ = sum of the ranks of negative differences, with $P + Q = \frac{1}{2} n (n + 1)$ .
Test statistic $T$ : for a two-tailed test, the smaller of $P$ and $Q$ ; for a one-tailed test, the sum that $H_{1}$ predicts will be small. Reject $H_{0}$ if $T$ is less than or equal to the critical value.
For large $n$ : $T \approx N (\frac{1}{4} n (n + 1), \frac{1}{24} n (n + 1) (2 n + 1))$ , used with a continuity correction.
Paired samples: apply the test to the differences within each pair, with $H_{0}$ : median difference $= 0$ .

Method

Subtract the hypothesised median from every value; discard zeros and reduce $n$ accordingly.
Rank the absolute differences from smallest to largest, giving tied values the mean of the ranks they occupy, then attach the signs.
Find $T$ and compare with the tabulated critical value for $n$ and the significance level.

In practiceData: 23, 18, 26, 21, 29, 25, 17, 24. Test $H_{0}$ : median $= 20$ against $H_{1}$ : median $> 20$ at the 5% level.

$d = x - 20 : 3, - 2, 6, 1, 9, 5, - 3, 4$ no zeros, so $n = 8$
$∣ d ∣ ranked: 1 \to 1, 2 \to 2, 3, 3 \to 3.5, 4 \to 5, 5 \to 6, 6 \to 7, 9 \to 8$ tied sizes share the mean of their ranks
$Q = 2 + 3.5 = 5.5, P = 36 - 5.5 = 30.5$ check: $P + Q = \frac{1}{2} (8) (9) = 36$
$T = Q = 5.5 > 5$ $H_{1}$ predicts small negative ranks; the critical value for $n = 8$ is 5
Do not reject $H_{0}$ at the 5% level, although the result is close.

Notes

The test assumes the population is symmetric about its median; it uses both the signs and the sizes of the differences.
Unlike most tests met so far, a small statistic is the significant one.

Worked example

Ten values are 53, 47, 58, 61, 49, 56, 52, 63, 45, 57.

Use a Wilcoxon signed-rank test at the 5% level to test whether the population median exceeds 50.

Show worked solution

Differences from 50: 3, −3, 8, 11, −1, 6, 2, 13, −5, 7.

Ranks of $∣ d ∣$ : 1→1, 2→2, the two 3s share 3.5, 5→5, 6→6, 7→7, 8→8, 11→9, 13→10.

Negative ranks sum to:

1 + 3.5 + 5 = 9.5

positive to:

55 - 9.5 = 45.5

so $T = 9.5$ .

The one-tailed 5% critical value for $n = 10$ is 10.

$9.5 \leq 10$ : reject $H_{0}$ .

There is evidence that the median exceeds 50.

5.07

The Wilcoxon rank-sum test

The Wilcoxon rank-sum test 5.07

Definitions

Wilcoxon rank-sum test: a test of whether two independent samples come from populations with the same median (identical populations, against one shifted relative to the other).

Key results

Rank all $m + n$ values together, where $m \leq n$ is the size of the smaller sample. $R_{m}$ = sum of the ranks of the smaller sample.
Test statistic $W$ = the smaller of $R_{m}$ and $m (m + n + 1) - R_{m}$ . Reject $H_{0}$ if $W$ is less than or equal to the critical value for $m$ and $n$ .
For large samples: $R_{m} \approx N (\frac{1}{2} m (m + n + 1), \frac{1}{12} mn (m + n + 1))$ .

Method

State hypotheses about the population medians, choosing one- or two-tailed from the context.
Rank the combined data, find $R_{m}$ , then $W$ , and compare with tables.
For a one-tailed test, check that the smaller sample's ranks are small (or large) in the direction $H_{1}$ predicts.

In practiceSamples A: 12, 15, 18, 20 and B: 14, 22, 25, 27, 30. Test at the 5% level (two-tailed) whether the population medians differ.

$1 2_{A}, 1 4_{B}, 1 5_{A}, 1 8_{A}, 2 0_{A}, 2 2_{B}, 2 5_{B}, 2 7_{B}, 3 0_{B}$ rank all nine together
$R_{m} = 1 + 3 + 4 + 5 = 13$ A is the smaller sample, $m = 4$ , $n = 5$
$m (m + n + 1) - R_{m} = 4 \times 10 - 13 = 27, W = min (13, 27) = 13$
$W = 13 > 11$ critical value for $m = 4$ , $n = 5$ , two-tailed 5%
Do not reject $H_{0}$ : no significant evidence that the medians differ.

Notes

Use the rank-sum test for two independent samples and the signed-rank test for paired data. Applying the wrong one is a common error.
The non-parametric tests are less powerful than a $t$ or $z$ test when the population really is normal, but valid when it is not.

Worked example

Times, in seconds, to complete a task: method A 12, 15, 19, 22; method B 17, 25, 28, 31, 34.

Test at the 5% level whether method A tends to give shorter times. (Critical value for $m = 4$ , $n = 5$ , one-tailed 5%: 12.)

Show worked solution

$H_{0}$ : the two populations have the same median; $H_{1}$ : A's median is lower.

Ranking all nine: A has ranks 1, 2, 4 and 5, so $R_{m} = 12$ ;

m (m + n + 1) - R_{m} = 40 - 12 = 28

so $W = 12$ .

$12 \leq 12$ : reject $H_{0}$ .

There is evidence at the 5% level that method A gives shorter times.

5.08

Correlation

Testing correlation: Pearson and Spearman 5.08

Definitions

Product moment correlation coefficient $r = \frac{S _{x y}}{S _{xx} S _{yy}}$ : the strength of linear association.
Spearman's rank correlation coefficient $r_{s}$ : the PMCC of the ranks, $r_{s} = 1 - \frac{6 \sum d ^{2}}{n ( n ^{2} - 1 )}$ when there are no ties.

Key results

Both lie between $- 1$ and $1$ . $r_{s} = \pm 1$ for any strictly monotonic relationship; $r = \pm 1$ only for an exact straight line.
Test $H_{0} : ρ = 0$ with $r$ when the data come from a bivariate normal distribution (an elliptical scatter).
Use $r_{s}$ when that is doubtful, when the data are ranks, or when the relationship is monotonic but not linear.

Method

Rank each variable separately. Find $d$ for each pair, then $\sum d^{2}$ and $r_{s}$ .
Compare $∣ r ∣$ or $∣ r_{s} ∣$ with the critical value for $n$ and the significance level, choosing one or two tails from the wording of $H_{1}$ .

In practiceTwo judges rank six entries: A gives 1, 2, 3, 4, 5, 6 and B gives 1, 3, 2, 4, 6, 5. Test for positive association at the 5% level.

$d : 0, - 1, 1, 0, - 1, 1 \sum d^{2} = 4$
$r_{s} = 1 - \frac{6 \times 4}{6 ( 36 - 1 )} = 1 - \frac{24}{210} = 0.886$ no ties, so the formula is exact
$0.886 > 0.829$ critical value for $n = 6$ , one-tailed 5%
Reject $H_{0} : ρ = 0$ : there is evidence of positive association between the judges' rankings.

Notes

Tied values receive the mean of the ranks they would occupy; the $\sum d^{2}$ formula is then only approximate, so compute the PMCC of the ranks.
Significant correlation does not show causation, and $r$ near 0 does not mean there is no relationship, only no linear one.

Worked example

Two judges rank 8 entries and $\sum d^{2} = 14$ .

Test at the 5% level whether there is positive agreement.

Show worked solution

r_{s} = 1 - \frac{6 ( 14 )}{8 ( 63 )} = 1 - \frac{84}{504} = 0.833

$H_{0} : ρ_{s} = 0$ , $H_{1} : ρ_{s} > 0$ .

The one-tailed 5% critical value for $n = 8$ is 0.6429.

Since:

0.833 > 0.6429

reject $H_{0}$ : there is evidence of positive agreement between the judges.

5.08

Linear regression

Least-squares regression 5.08

Definitions

Least-squares regression line of $y$ on $x$ : the line $y = a + b x$ that minimises the sum of the squares of the residuals.
Residual: the observed value of $y$ minus the value predicted by the line, $y_{i} - (a + b x_{i})$ .

Key results

$b = \frac{S _{x y}}{S _{xx}}$ and $a = \overset{y}{ˉ} - b \overset{x}{ˉ}$ , with $S_{xx} = \sum x^{2} - \frac{( \sum x ) ^{2}}{n}$ and $S_{x y} = \sum x y - \frac{\sum x \sum y}{n}$ .
The line always passes through $(\overset{x}{ˉ}, \overset{y}{ˉ})$ .
Use the line of $y$ on $x$ to estimate $y$ from a given $x$ , when $x$ is the independent (controlled) variable.

Method

Calculate $S_{xx}$ and $S_{x y}$ from the summary statistics, then $b$ and $a$ ; give the equation in context with units.
If the data have been coded, substitute the coding into the equation to recover the line in the original variables.

In practice $n = 8$ , $\sum x = 40$ , $\sum y = 96$ , $\sum x^{2} = 240$ , $\sum x y = 520$ . Find the regression line of $y$ on $x$ .

$S_{xx} = 240 - \frac{4 0 ^{2}}{8} = 40, S_{x y} = 520 - \frac{40 \times 96}{8} = 40$
$b = \frac{S _{x y}}{S _{xx}} = 1, a = \overset{y}{ˉ} - b \overset{x}{ˉ} = 12 - 5 = 7$
$y = 7 + x$ it passes through $(\overset{x}{ˉ}, \overset{y}{ˉ}) = (5, 12)$ , as it must

Notes

Predictions within the range of the data (interpolation) are reliable when the correlation is strong; extrapolation beyond it may not be.
Large residuals point to outliers or to a relationship that is not linear.

Worked example

For six data points, $\sum x = 36$ , $\sum y = 78.2$ , $\sum x^{2} = 258$ and $\sum x y = 549.7$ .

Find the regression line of $y$ on $x$ and estimate $y$ when $x = 9$ .

Show worked solution

S_{xx} = 258 - \frac{3 6 ^{2}}{6} = 42

and:

S_{x y} = 549.7 - \frac{36 \times 78.2}{6} = 80.5

b = \frac{80.5}{42} = 1.917

and:

a = 13.03 - 1.917 \times 6 = 1.53

So:

y = 1.53 + 1.92 x

and at $x = 9$ , $y \approx 18.8$ (interpolation, so reliable if the correlation is strong).