Type I & Type II Errors
What are Type I & Type II errors?
- There are four possible outcomes of a hypothesis test:
- H0 is false and H0 is rejected
- H0 is true and H0 is not rejected
- The test is accurate for these two outcomes
- H0 is true and H0 is rejected
- H0 is false and H0 is not rejected
- The test has led to an error for these two outcomes
- A Type I error occurs when a hypothesis test gives sufficient evidence to reject H0 despite it being true
- This is sometimes called a “false positive”
- In a court case this would be when the defendant is found guilty despite being innocent
- A Type II error is when a hypothesis test gives insufficient evidence to reject H0 despite it being false
- This is sometimes called a “false negative”
- In a court case this would be when the defendant is found innocent despite being guilty
How do I find the probabilities of a Type I or Type II error occurring?
- You should calculate the probability of errors occurring before a sample is taken
- The probabilities are determined by the critical region
- Equally it is determined by the significance level α%
- Critical regions are determined such that:
- They keep the probability of a Type I error less than or equal to the significance level
- They maximise the probability of a Type I error
- The probability of a Type I error occurring is equal to the probability of being in the critical region if H0 were true
- P(Type I error) = P(being in the critical region | H0 is true)
- For a continuous distribution (normal, t, χ²)
- P(Type I error) = α%
- For a discrete distribution (binomial, Poisson)
- P(Type I error) ≤ α%
- The probability of a Type II error occurring is equal to the probability of not being in the critical region given the actual value of the population parameter
- P(Type II error) = P(not being in critical region | actual population parameter)
- You need to know the actual population parameter in order to find the probability of a Type II error
- Once a sample has been taken you can determine which error could have occurred
- If you rejected H0 then you could have made a Type I error
- If you accepted H0 then you could have made a Type II error
Can I reduce the probabilities of making a Type I or Type II error?
- You can reduce the probability of a Type I error by reducing the significance level
- However this will increase the probability of a Type II error
- You can reduce the probability of a Type II error by increasing the significance level
- However this will increase the probability of a Type I error
- The only way to reduce both probabilities is by increasing the size of the sample
Exam Tip
- If an exam question asks you to find the probability of a Type I or II error then double check that the test has not been carried out yet
- The examiner could test your understanding of errors by asking you to state which error could not have occurred once the test has been carried out
Worked example
Lucy can hit the target 70% of the time when she throws an axe with her right hand. She claims that the proportion, p, of her throws that hit the target is higher than 70% when she uses her left hand. Lucy uses the hypotheses and to test her claim. Lucy makes 100 throws and will reject the null hypothesis if the axe hits the target more than 77 times.