Hypothesis Testing — One-Sample vs Two-sample T-tests
When it comes to hypothesis testing, there are a variety of tests that we can choose from. Let’s travel to the Middle Earth, go through some missions and see how can we help the men, elves, or dwarfs, tackle the problem. Before we move on, Gandalf wants to remind us that we need to make sure that we are studying continuous variables for t-test. For discrete or ordinal variables, we will walk through some testing methods in the next blog.
One-sample T-test
One-tail
It is FO 50, Aragorn is celebrating his 50th years of ruling and hosting a banquet. Legolas is leading his elven bakers to make Lembas. To meet the deadline, bakers are expected to make 10 or more Lembas in an hour. To help Legolas determine if his bakers can meet the standard, we can use a one-sample T-test (one-tail). Legolas can now choose his statistical significance value (alpha). 0.05 is a common used value.
Null hypothesis: bakers makes less than 10 Lembas per hour.
Alternative hypothesis: bakers can make 10 or more Lembas per hour.
Using the above formula, Legolas will be able to get the t value. In this formula:
m — mean of the sample (baker’s arithmetic average productivity)
u — theoretical/population mean (10 as it is the threshold)
s — standard deviation of the sample (the standard deviation of baker’s productivity)
n — sample size (number of bakers on the team)
Then with the degree of freedom (n-1), he can check the T-distribution table to get the P-value. As he has chosen 0.05 as the statistical significance, if the P-value is less than 0.05, he can reject the null hypothesis and be confident about meeting the banquet deadline.
Two-tail
If making 10 Lembas per hour is not an ideal number that elves like, Legolas can use a one-sample T-test (two-tail) to determine is the productivity of bakers is not 10.
Null hypothesis: bakers makes 10 Lembas per hour.
Alternative hypothesis bakers can make more than 10 or less than 10 Lembas per hour.
The same formula and criteria can be used. To further determine if the bakers productivity is higher or lower than 10, we can look at the value of t; a negative t would indicate that the bakers average productivity is less than 10.
Two-sample T-test
One-tail
Legolas got 2 teams of bakers proposing the banquet work to him. He wants to know if team Mirkwood is faster than team Rivendell. He can use the two-sample t-test (one-tail).
Null hypothesis: team Mirkwood has the same or lower productivity than team Rivendell.
Alternative hypothesis: team Mirkwood has higher productivity than team Rivendell.
Using the above formula, Legolas will be able to get the t value. In this formula:
X1, X2 — mean of the samples (arithmetic average productivity of Mirkwood and Rivendell bakers)
S1, S2 — standard deviation of the sample (the standard deviation of Mirkwood and Rivendell bakers)
n — sample size (number of Mirkwood and Rivendell bakers)
Then with the degree of freedom (n-1), he can check the T-distribution table to get the P-value. As he has chosen 0.05 as the statistical significance, if the P-value is less than 0.05, he can reject the null hypothesis and be confident about his hometown team Mirkwood is more productive.
Two-tail
Now Legolas is looking at new oven product. He wants to know if this would affect the baking work. He is curious if the team (Mirkwood) with the new oven would have different productivity. He can use a two-sample t-test (two-tail).
Null hypothesis: team Mirkwood has the same productivity as team Rivendell.
Alternative hypothesis: team Mirkwood has different productivity comparing to team Rivendell.
The same formula and criteria can be used. To further determine if Mirkwood bakers productivity is higher or lower than Rivendell, Legolas can look at the value of t; a negative t would indicate that the Mirkwood bakers average productivity is less than team Rivendell (perhaps the oven is not really helping them).
Effect Size
During the testing, Legolas asked about a commonly encountered term — Effect Size. Effect size is simple how different are the means of the samples. In the previous two-sample case, Legolas might go further and measure how much more productive can the Mirkwood team be than the Rivendell team.