What is Hypothesis Testing?
Hypothesis Testing is a method of statistical inference. It is used to test if a statement regarding a population parameter is statistically significant. Hypothesis testing is a powerful tool for testing the power of predictions. A Financial Analyst, for example, might want to make a prediction of the mean value a customer would pay for her firm’s product. She can then formulate a hypothesis, for example, “The average value that customers will pay for my product is larger than $5”. To statistically test this question, the firm owner could use hypothesis testing. This example is further explored down below.
Hypothesis Testing is a critical part of the scientific method, which is a systematic approach of assessing theories through observation. A good theory is one that can make accurate predictions. For an Analyst who makes predictions, hypothesis testing is a rigorous way of backing up his prediction with statistical analysis.
Hypothesis Testing Steps
Here are the steps for hypothesis testing:
- State the null hypothesis (H0) and alternative hypothesis (Ha).
- Consider the Statistical assumptions being made. Evaluate if these assumptions are coherent with the underlying population being evaluated. For example, is assuming the underlying distribution as a normal distribution sensible?
- Determine the appropriate probability distribution and select the appropriate test statistic.
- Select the Significance Level commonly denoted by the greek letter alpha (α), this is the probability threshold for which the null hypothesis will be rejected.
- Based on the significance level and on the the appropriate test, state the decision rule.
- Collect the observed sample data, and use it to calculate the test statistic
- Based on your results you should either reject the null hypothesis, or fail to reject the null hypothesis. This is known as the statistical decision.
- Consider any other economic issues that are applied to the problem. These are non-statistical considerations that need to be considered for a decision. For example, sometimes societal cultural shifts lead to changes in consumer behaviour, this must be taken in consideration in addition to the statistical decision for a final decision.
Stating the Null Hypothesis and Alternative Hypothesis
The Null Hypothesis is usually set as what we don’t want to be true. It is the hypothesis to be tested. Therefore, the Null Hypothesis is considered to be true, until we have sufficient evidence to reject it. If we reject the null hypothesis, we are led to the alternative hypothesis.
Going back to our initial example of the business owner who is looking for some customer insight. His null hypothesis would be:
H0 : The average value customers are willing to pay for my product is smaller than or equal to $5
H0 : µ ≤ 5
(µ = the population mean)
The alternative hypothesis would then be what we are evaluating, so in this case it would be:
Ha : The average value customers are willing to pay for the product is greater than $5
Ha : µ > 5
It is important to emphasize that the alternative hypothesis will only be considered, if the sample data that we gather provides evidence for it.
What are Type I and Type II Errors?
The binary nature of our decision, to reject or fail to reject the null hypothesis, gives rise to two possible errors. The table below illustrates all of the possible outcomes. A Type I Error arises when a true Null Hypothesis is rejected. The probability of making a Type I Error is also known as the level of significance of the test, which is commonly referred to as alpha (α). So, for example, if a test that has its alpha set as 0.01, there is a 1% probability of rejecting a true null hypothesis or a 1% probability of making a Type I Error.
A Type II Error arises when you fail to reject a False Null Hypothesis. The probability of making a Type II Error is commonly denoted by the greek letter beta (β). β is used to define the Power of a Test, which is the Probability of correctly rejecting a false null hypothesis. The Power of a Test is defined as 1-β. A test with more Power is more desirable, as there is a lower probability of making a Type II Error. However there is a tradeoff between the probability of making a Type I Error and the probability of making a Type II Error.
Hypothesis Testing Example
Lets go back to the business owner example. Let us remember the question that we are trying to answer:
Q: “Will customers pay, on average, more than $5 for our product?”
1. We have set above both the null and alternative hypothesis
H0 : µ ≤ 5
Ha : µ > 5
2. For this example let us assume that the firm sells organic apple juice boxes. This is consumed by a wide range of consumers of all ages, income levels, and cultural backgrounds. So given our product is widely used by a diverse group of consumers assuming a normal distribution is fair.
3. Let us assume that getting samples from our consumers and we will manage to get over 100 observations. So given we are confident with our assumption of a normal distribution for the underlying population, and have a large number of observations we will use a z-test.
4. We want to be confident of our result so let us pick our significance level as α = 5%, this will provide strong evidence of our result.
5. We are using a z-test with a significance level, and the null hypothesis is µ ≤ 5 so our rejection point will be z0.05 =1.645. Which means that if the z score calculated from our sample is is greater than 1.645 we reject the null hypothesis.
6. Now assume that we have collected our date and have out that from our sample of 100 observations, the mean price that customers were willing to pay for our juices is $5.02, and that the sample standard deviation was of $0.10. We can now calculate the z-score for our sample where we get a value of 2 given by [(5.02 – 5) / ( 0.1/ √100)].
7. Given our calculated z is greater than z0.05 =1.645, we have strong evidence to reject the null hypothesis at a 5% significance level. We are then in favour of the alternative hypothesis, that the average value customers are willing to pay for the product is greater than $5.
8. We now need to take in consideration any economic or qualitative issues that are not addressed through the statistical process. These are usually non quantifiable variables that have to be addressed when making a decision based on the findings. For example if the biggest competitor was going to lower the price of the competing product significantly, that may lower the average value they are willing to pay for your product.
If you want to learn more about topics related to Hypothesis Testing, check out resources on the Royal Statistics Society website.
CFI offers the Financial Modeling & Valuation Analyst (FMVA)™ certification program for those looking to take their careers to the next level. To keep learning and advancing your career, the following resources will also be helpful: