Chi Square – Explanation & Examples

The definition of the chi-square test is:

“The chi-square test compares two variables in a contingency table to see if they are related.”

In this topic, we will discuss the chi-square test from the following aspects:

1. What is the chi-square test?
2. Hypothesis testing using the chi-square test.
3. Steps of hypothesis testing performed by the chi-square test.
4. How to calculate the chi-square test?
5. Practice questions.
6. Answer key.

1. What is the chi-square test?

The chi-square test of independence, also called χ^2 test, is used to analyze the contingency table formed by two categorical variables.

The chi-square test evaluates whether there is a significant association between the categories of the two variables.

An R × C contingency table is a table with R rows and C columns. It displays the relationship between two variables, where the variable in the rows has R categories and the variable in the columns has C categories.

– Example of 2 X 2 contingency table

A study in 1994 examined 491 dogs that had developed cancer and 945 dogs as a control group (without cancer) to determine whether there is an increased risk of cancer in dogs that are exposed to the herbicide 2,4-Dichlorophenoxyacetic acid (2,4-D).

Reference:

Hayes HM, Tarone RE, Cantor KP, Jessen CR, McCurnin DM, and Richardson RC. 1991. Case-Control Study of Canine Malignant Lymphoma: Positive Association With Dog Owner’s Use of 2, 4- Dichlorophenoxyacetic Acid Herbicides. Journal of the National Cancer Institute 83(17):1226-1231.

The results of this study are shown in the following table.

We see that:

• The 2,4-D exposure categories are indicated along the rows and the cancer status categories are indicated along with the columns.
• The data are arranged in the form of a 2 × 2 contingency table because the 2,4-D exposure has 2 categories and the cancer status has 2 categories also.
• The “cancer” and “no cancer” columns for dogs that developed and did not develop cancer respectively.
• The “2,4-D” and “no 2,4-D” rows for dogs that were exposed and were not exposed to 2,4-D respectively.
• 191 dogs were exposed to 2,4-D and developed cancer.
• 304 dogs were exposed to 2,4-D and did not develop cancer.
• 300 dogs were not exposed to 2,4-D and developed cancer.
• 641 dogs were not exposed to 2,4-D and did not develop cancer.

We want to test for the relationship between exposure to 2,4-D and developing cancer. We use the χ^2 test to test if this relationship truly exists.

– Example of 2 X 5 contingency table

Survey responses for 20,000 responses to the Behavioral Risk Factor Surveillance System.

Source Office of Surveillance, Epidemiology, and Laboratory Services Behavioral Risk Factor Surveillance System, BRFSS 2010 Survey Data.

The results of this study are shown in the following table.

We see that:

• The health coverage categories are indicated along the rows and the health status categories are indicated along with the columns.
• The data are arranged in the form of a 2 × 5 contingency table because the health coverage has 2 categories and the health status has 5 categories.
• The “Excellent”, “Fair”, “Good”, “Poor”, and “Very good” columns are for the person’s health status.
• The “No” and “Yes” rows are for whether the person had health coverage or not.
• 459 persons had excellent health status and were not having health coverage.
• 4198 persons had excellent health status and were having health coverage.
• 385 persons had fair health status and were not having health coverage.
• 1634 persons had fair health status and were having health coverage.
• 854 persons had good health status and were not having health coverage.
• 4821 persons had good health status and were having health coverage.
• 99 persons had poor health status and were not having health coverage.
• 578 persons had poor health status and were having health coverage.
• 727 persons had very good health status and were not having health coverage.
• 6245 persons had very good health status and were having health coverage.

We want to test for the relationship between health status and health coverage. We use the χ^2 test to test if this relationship truly exists.

– Example of 4 X 3 contingency table

A 2010 Pew Research poll asked 1,306 Americans, “From what you’ve read and heard, is there solid evidence that the average temperature on earth has been getting warmer over the past few decades, or not?”

Source: Pew Research Center, Majority of Republicans No Longer See Evidence of Global Warming, data collected on October 27, 2010.

The results of this study are shown in the following table.

We see that:

• The party categories are indicated along the rows and the response categories are indicated along with the columns.
• The data are arranged in the form of a 4 × 3 contingency table because the party has 4 categories and the response has 3 categories.
• The “Don’t know / refuse to answer”, “Earth is warming”, and “Not warming” columns are the response categories.
• The “Conservative Republican”, “Liberal Democrat”, “Mod/Cons Democrat”, and “Mod/Lib Republican” rows are the party categories.
• 45 persons responded “Don’t know / refuse to answer” and were having “Conservative Republican” party, compared to 23 persons having “Liberal Democrat” party, 45 persons having “Mod/Cons Democrat” party, and 23 persons having “Mod/Lib Republican” party.
• 248 persons responded “Earth is warming” and were having “Conservative Republican” party, compared to 405 persons having “Liberal Democrat” party,
• 563 persons having “Mod/Cons Democrat” party, and 135 persons having “Mod/Lib Republican” party.
• 450 persons responded “Not warming” and were having “Conservative Republican” party, compared to 23 persons having “Liberal Democrat” party,
• 158 persons having “Mod/Cons Democrat” party, and 135 persons having “Mod/Lib Republican” party.

We want to test for the relation between the party categories and the response categories. We use the χ^2 test to test if this relationship truly exists.

2. Hypothesis testing using the chi-square test

Where you start with two exclusive possibilities for the unknown truth. Then, use the sample to choose between these two possibilities for the truth. The two possibilities are the Null hypothesis, Ho, and the Alternative hypothesis, Ha.

• The null hypothesis, Ho: There is no difference between the two populations or the two categorical variables, and the difference = zero.
• The alternative hypothesis, Ha: There is a difference between the two populations so the difference ≠ zero.

Hypothesis testing is denoted as:

• Ho: p_1=p_2 or p_1-p_2=0. The proportions of one variable are the same for different values of the other variable.

In testing the relation between exposure to 2,4-D and developing cancer, this means that the proportion of developing cancer is similar for dogs exposed and not exposed to 2,4-D.

• Ha: p_1≠p_2 or p_1-p_2≠0. In testing the relation between exposure to 2,4-D and developing cancer, this means that the proportion of developing cancer is different for dogs exposed and not exposed to 2,4-D.

Note: Although the hypothesis testing for the chi-square test compares proportions, the chi-square test uses the actual count to test that.

– Steps of hypothesis testing performed by the chi-square test

• The chi-square test uses the contingency table of data to calculate an expected table.

The expected table contains the theoretical data counts that would be expected when there is no relation between the rows and the columns i.e. the null hypothesis is true, Ho: p_1=p_2.

• The test calculates the discrepancies between each observed and expected value and aggregates them.
• If the null hypothesis is true, the aggregated value, called the χ^2 statistic, has a chi-square distribution. Define the probability of the aggregated value under this chi-square distribution. This is the p-value.

The p-value is the probability of our sample results if the null hypothesis is true.

The null hypothesis means that the proportion of developing cancer is similar for dogs exposed and not exposed to 2,4-D.

Generally in research, the cut-off used is 0.05. This 0.05 is called the rejection level, α level, or significance level.

• Make a decision, accept Ha, or fail to reject Ho.

If p-value < 0.05, so it is a statistically significant result at 0.05 level. Reject the null hypothesis and conclude that our sample data are unlikely under the Ho, null hypothesis, they have a probability of less than 0.05.

If p-value >= 0.05, so it is a statistically non-significant result at 0.05 level and we fail to reject the Null hypothesis.

We say fail to reject the Null hypothesis because if we have a p-value of 0.25. This means that our sample data have a probability of 25% under the null hypothesis which is considered a large percentage. In your opinion, you may consider it small and accept Ha.

Note: No expected value in the expected table is less than 5 (sometimes known as “the rule of five”). If any expected value is less than 5, the chi-square test is not applicable and other tests are applied (Fisher Exact test).

3. How to calculate the chi-square test?

– Example of 2 X 2 contingency table

A study in 1994 examined 491 dogs that had developed cancer and 945 dogs as a control group to determine whether there is an increased risk of cancer in dogs that are exposed to the herbicide 2,4-Dichlorophenoxyacetic acid (2,4-D).

The following 2 X 2 contingency table is obtained:

To see the different proportions of cancer development per 2,4-D exposure, we can use the following table:

The sum of every row is 1.00 or 100%.

We see that 0.39 or 39% of dogs exposed to 2,4-D developed cancer compared to 0.32 or 32% of dogs not exposed to 2,4-D.

We can plot these proportions in the following bar plot.

To test for the relationship between exposure to 2,4-D and developing cancer, we follow these steps:

• Use the 2 X 2 table to calculate the expected count of each cell.

The expected count in the (i, j) cell =
(the total count in the ith row X the total count in the jth column)/ the total count in the table.

The expected counts indicate no association between the rows and columns. In other words, there is no association between exposure to 2,4-D and cancer development.

The sum of the expected values across any row or column must equal the corresponding row or column total.

The expected count for dogs exposed to 2,4-D and developed cancer = (total in 1st row X total in 1st column)/table total = (495X491)/1436 = 169.2514.

The expected count for dogs not exposed to 2,4-D and developed cancer = (total in 2nd row X total in 1st column)/table total = (941X491)/1436 = 321.7486.

The expected count for dogs exposed to 2,4-D and did not develop cancer = (total in 1st row X total in 2nd column)/table total = (495X945)/1436 = 325.7486.

The expected count for dogs not exposed to 2,4-D and did not develop cancer = (total in 2nd row X total in 2nd column)/table total = (941X945)/1436 = 619.2514.

The following table will be produced:

We see that all expected values are larger than 5 so the chi-square test can be used.

To see the different proportions of cancer development per 2,4-D exposure in the expected table:

The sum of every row is 1.00 or 100%.

We see that 0.34 or 34% of dogs exposed to 2,4-D developed cancer and also 0.34 or 34% of dogs not exposed to 2,4-D.

• Make a table with 2 columns for different cells and their observed counts.

We have 4 cells in this 2 X 2 table:

• A cell for dogs exposed to 2,4-D and had cancer.
• A cell for dogs exposed to 2,4-D and had not cancer.
• A cell for dogs not exposed to 2,4-D and had cancer.
• A cell for dogs not exposed to 2,4-D and had not cancer.

• Add a column for the expected count of each cell.

• Subtract the expected value from the Observed value and place the result in the “obs-exp” column.

• Square the differences from Step 4 and place the result in the “(obs-exp)^2” column.

• Divide the squared differences by their respective expected value and place the result in the “(obs-exp)^2/exp” column.

• Sum all the values in the last column to get the chi-square statistic:

The χ^2 statistic = 2.80+1.47+1.45+0.76 = 6.48.

The last column is summed to get an overall measure of agreement between the observed and expected tables.

• If the null hypothesis is true, the χ^2 statistic has a chi-square distribution with (R − 1) × (C − 1) degrees of freedom or df.

Define the probability (or the p-value) of the χ^2 statistic under this chi-square distribution.

The p-value is given by the area to the right of the χ^2 statistic under this chi-square distribution.

In our 2X2 contingency table, the df = (2-1)X(2-1) = 1.

The following is the chi-square distribution with 1 df.

The total area under the curve is 1.00 or 100%.

In the first plot, we see that when the χ^2 value = 3.84, the area to the right or the p-value = 0.05.

In our contingency table, the χ^2 value = 6.84 (plotted as a vertical line in the second plot), so the p-value is smaller than 0.05.

• Make a decision, accept Ha, or fail to reject Ho.

The p-value < 0.05, so it is a statistically significant result. We reject the null hypothesis and conclude that our sample data are unlikely under the Ho, null hypothesis, they have a probability of less than 0.05.

We conclude that there is a significant relationship between exposure to 2,4-D and cancer development in dogs.

– Example of 2 X 5 contingency table

The Survey responses for 20,000 responses to the Behavioral Risk Factor Surveillance System.

The results of this study are shown in the following table.

To see the different proportions of health status per health coverage, we can use the following table:

The sum of every row is 1.00 or 100%.

We see that 0.18 or 18% of persons who do not have health coverage had excellent health status compared to 0.24 or 24% of persons who do have health coverage.

We see that 0.15 or 15% of persons who do not have health coverage had fair health status compared to 0.09 or 9% of persons who do have health coverage, and so on.

We can plot these proportions in the following bar plot.

To test for the relationship between health coverage and health status, we follow these steps:

• Use the 2 X 5 table to calculate the expected count of each cell.

The expected counts indicate no association between the rows and columns. In other words, there is no association between health coverage and health status.

The expected count in the (i, j) cell =
(the total count in the ith row X the total count in the jth column)/ the total count in the table.

For example, the expected count for persons who do not have a health coverage and with excellent health status = (row total X column total) / table total = (2524 X 4657)/20000 = 587.7134.

The following table will be produced:

We see that all expected values are larger than 5 so the chi-square test can be used.

To see the different proportions of health statuses per health coverage in the expected table:

We see that 0.23 or 23% of persons who do or do not have health coverage had excellent health status.

All other proportions of different health statuses across the health coverage are equal.

• Make a table with 2 columns for different cells and their observed counts.

We have 10 cells in this 2 X 5 table which are shown in the following table:

For example, the “No, Excellent” category means persons without health coverage and excellent health status.

• Add a column for the expected count of each cell.

• Subtract the expected value from the Observed value and place the result in the “obs-exp” column.

• Square the differences from Step 4 and place the result in the “(obs-exp)^2” column.

• Divide the squared differences by their respective expected value and place the result in the “(obs-exp)^2/exp” column.

• Sum all the values in the last column to get the chi-square statistic:

The χ^2 statistic = 28.19+ 4.07+ 66.53+ 9.61+ 26.52+ 3.83+ 2.15+ 0.31+ 26.56+ 3.84 = 171.61.

• If the null hypothesis is true, the χ^2 statistic has a chi-square distribution with (R − 1) × (C − 1) degrees of freedom.

In our 2X5 contingency table, the df = (2-1)X(5-1) = 4.

The following is the chi-square distribution with 4 df.

In the first plot, we see that when the χ^2 value = 9.49, the area to the right or the p-value = 0.05.

In our contingency table, the χ^2 value = 171.61 (plotted as a vertical line in the second plot), so the p-value is very much smaller than 0.05.

• Make a decision, accept Ha, or fail to reject Ho.

The p-value < 0.05, so it is a statistically significant result. We reject the null hypothesis and conclude that our sample data are unlikely under the Ho or the null hypothesis.

We conclude that there is a significant relationship between health statuses and health coverage in the persons surveyed.

– Example of 4 X 3 contingency table

A 2010 Pew Research poll asked 1,306 Americans, “From what you’ve read and heard, is there solid evidence that the average temperature on earth has been getting warmer over the past few decades, or not?”.

The results of this study are shown in the following table.

To see the different proportions of responses per different parties, we can use the following table:

The sum of every row is 1.00 or 100%.

We see that:

• 0.33 or 33% of “Conservative Republican” persons responded that Earth is warming, compared to 0.90 or 90% of “Liberal Democrat” persons, 0.73 or 73% of “Mod/Cons Democrat” persons, and 0.46 or 46% of “Mod/Lib Republican” persons.
• 0.61 or 61% of “Conservative Republican” persons responded that Earth is not warming, compared to only 0.05 or 5% of “Liberal Democrat” persons, 0.21 or 21% of “Mod/Cons Democrat” persons, and 0.46 or 46% of “Mod/Lib Republican” persons.

We can plot these proportions in the following bar plot.

To test for the relationship between parties and responses, we follow these steps:

• Use the 4 X 3 table to calculate the expected count of each cell.

The expected counts indicate no association between the rows and columns. In other words, there is no association between the parties and the responses.

The expected count in the (i, j) cell =
(the total count in the ith row X the total count in the jth column)/ the total count in the table.

For example, the expected count for “Conservative Republican” persons who responded that Earth is warming = (row total X column total) / table total = (743 X 1351)/2253 = 445.5362.

The following table will be produced:

We see that all expected values are larger than 5 so the chi-square test can be used.

To see the different proportions of responses per parties in the expected table:

All proportions of different responses across the different parties are the same.

• Make a table with 2 columns for different cells and their observed counts.

We have 12 cells in this 4 X 3 table which are shown in the following table:

For example, the “Conservative Republican,Earth is warming” category means “Conservative Republican” persons who responded that Earth is warming.

• Add a column for the expected count of each cell.

• Subtract the expected value from the Observed value and place the result in the “obs-exp” column.

• Square the differences from Step 4 and place the result in the “(obs-exp)^2” column.

• Divide the squared differences by their respective expected value and place the result in the “(obs-exp)^2/exp” column.

• Sum all the values in the last column to get the chi-square statistic:

The χ^2 statistic = 507.52.

• If the null hypothesis is true, the χ^2 statistic has a chi-square distribution with (R − 1) × (C − 1) degrees of freedom.

In our 4X3 contingency table, the df = (4-1)X(3-1) = 6.

The following is the chi-square distribution with 6 df.

In the first plot, we see that when the χ^2 value = 12.59, the area to the right or the p-value = 0.05.

In our contingency table, the χ^2 value = 507.52 (plotted as a vertical line in the second plot), so the p-value is very much smaller than 0.05.

• Make a decision, accept Ha, or fail to reject Ho.

The p-value < 0.05, so it is a statistically significant result. We reject the null hypothesis and conclude that our sample data are unlikely under the null hypothesis.

We conclude that there is a significant relationship between the different parties and the response type in the persons surveyed.

5. Practice questions

1. The Data from the 2010 General Social Survey shows the following table.

The rows are for the educational degree and the columns for answering the question “Do you think the use of marijuana should be made legal, or not?”.

Is there a relationship between educational degree and the answer type?

2. A sample of categorical variables from the General Social survey showed the following table.

The rows are for the reported income and the columns for race categories.

Is there a relationship between race and the reported income?

3. A study from the 1970s about whether gender influences hiring recommendations showed the following table.

The rows are for the gender and the columns for the promotions.

All expected values are larger than 5 so the chi-square test can be used.

The χ^2 statistic = 5.1692.

The following is the chi-square distribution with 1 df.

Is that a significant result?

4. The demographic information on every member of a 1000 random sample of the US armed forces showed the following table.

The rows are for the branch of the armed forces: air force, army, marine corps, or navy, and the columns for the gender.

All expected values are larger than 5 so the chi-square test can be used.

The χ^2 statistic = 17.534.

The following is the chi-square distribution with 3 df.

Is that a significant result?

5. The demographic information on every member of a random 2000 sample of the US armed forces showed the following table.

The rows are for the branch of the armed forces: air force, army, marine corps, or navy, and the columns for the race.

All expected values are larger than 5 so the chi-square test can be used.

The χ^2 statistic = 30.051.

The following is the chi-square distribution with 6 df.

Is that a significant result?

6. Answer key

1. We follow the same steps above to calculate the expected table.

We see that all expected values are larger than 5 so the chi-square test can be used.

Then, we follow the above steps to get the chi-square statistic.

The χ^2 statistic = 20.48.

In our 5X2 contingency table, the df = (5-1)X(2-1) = 4.

The following is the chi-square distribution with 4 df.

In the first plot, we see that when the χ^2 value = 9.49, the area to the right or the p-value = 0.05.

In our contingency table, the χ^2 value = 20.48 (plotted as a vertical line in the second plot), so the p-value is very much smaller than 0.05.

The p-value < 0.05, so it is a statistically significant result.

We reject the null hypothesis and conclude that there is a significant relationship between the different educational degrees and the response type in the persons surveyed.

This means that the proportions of response type are different across different educational degrees.

2. We follow the same steps above to calculate the expected table.

We see that all expected values are larger than 5 so the chi-square test can be used.

Then, we follow the above steps to get the chi-square statistic.

The χ^2 statistic = 148.13.

In our 12X3 contingency table, the df = (12-1)X(3-1) = 22.

The following is the chi-square distribution with 22 df.

In the first plot, we see that when the χ^2 value = 33.92, the area to the right or the p-value = 0.05.

In our contingency table, the χ^2 value = 148.13 (plotted as a vertical line in the second plot), so the p-value is very much smaller than 0.05.

The p-value < 0.05, so it is a statistically significant result.

We reject the null hypothesis and conclude that there is a significant relationship between race and the reported income in the persons surveyed.

This means that the proportions of incomes are different across different races.

3. We see, from the plot, that when the χ^2 value = 3.84, the area to the right or the p-value = 0.05.

In our contingency table, the χ^2 value = 5.1692, so the p-vale is smaller than 0.05.
The p-value < 0.05, so it is a statistically significant result.

We reject the null hypothesis and conclude that there is a significant relationship between gender and being promoted.

This means that the proportions of promotions are different across the 2 sexes.

4. We see, from the plot, that when the χ^2 value = 7.81, the area to the right or the p-value = 0.05.

In our contingency table, the χ^2 value = 17.534, so the p-vale is smaller than 0.05.
The p-value < 0.05, so it is a statistically significant result.

We reject the null hypothesis and conclude that there is a significant relationship between gender and the branch of the armed forces.

This means that the proportions of branches from the US armed forces are different across the 2 sexes.

5. We see, from the plot, that when the χ^2 value = 12.59, the area to the right or the p-value = 0.05.

In our contingency table, the χ^2 value = 30.051, so the p-value is smaller than 0.05.
The p-value < 0.05, so it is a statistically significant result.

We reject the null hypothesis and conclude that there is a significant relationship between race and the branch of the armed forces.

This means that the proportions of branches are different across the different races.

Previous Lesson | Main Page | Next Lesson