The Chi-square test is intended to test how likely it is that an observed distribution is due to chance. It is also called a

A Chi-square test is designed to analyze

Pass | Fail | |

Attended | 25 | 6 |

Skipped | 8 | 15 |

Another way to describe the Chi-square test is that it tests the null hypothesis that the variables are independent. The test compares the observed data to a model that distributes the data according to the expectation that the variables are independent. Wherever the observed data doesn't fit the model, the likelihood that the variables are dependent becomes stronger, thus proving the null hypothesis incorrect!

The following table would represent a possible input to the Chi-square test, using 2 variables to divide the data: gender and party affiliation. 2x2 grids like this one are often the basic example for the Chi-square test, but in actuality any size grid would work as well: 3x3, 4x2, etc.

Democrat | Republican | |

Male | 20 | 30 |

Female | 30 | 20 |

This shows the basic 2x2 grid. However, this is actually incomplete, in a sense; generally, the data table should include "marginal" information giving the total counts for each column and row, as well as for the whole data set:

Democrat | Republican | Total | |

Male | 20 | 30 | 50 |

Female | 30 | 20 | 50 |

Total | 50 | 50 | 100 |

We now have a complete data set on the distribution of 100 individuals into categories of gender (Male/Female) and party affiliation (Democrat/Republican). A Chi-square test would allow you to test

So, as implied, the null hypothesis in this case would be that gender and party affiliation are independent of one another. To test this hypothesis, we need to construct a

Democrat | Republican | Total | |

Male | 25 | 25 | 50 |

Female | 25 | 25 | 50 |

Total | 50 | 50 | 100 |

This is the information we would need to calculate the likelihood that gender and party affiliation are independent. I will discuss the next steps in calculating a Chi square value later, but for now I'll focus on the background information.

This is also an important question to tackle, of course. Using a statistical test without having a good idea of what it can and cannot do means that you may misuse the test, but also that you won't have a clear grasp of what your results really mean. Even if you don't understand the detailed mathematics underlying the test, it is not difficult to have a good comprehension of where it is or isn't appropriate to use. I mentioned some of this above, when contrasting types of data and so on. This section will consider other things that the Chi-square test is not meant to do.

First of all, the Chi-square test is

Some further considerations are necessary when selecting or organizing your data to run a Chi-square test. The variables you consider must be

You should also

It is also important that you have enough data to perform a viable Chi-square test.

A broader description of this topic can be found here.

The

The degrees of freedom for a Chi-square grid are equal to the number of rows minus one times the number of columns minus one: that is, (R-1)*(C-1). In our simple 2x2 grid, the degrees of independence are therefore (2-1)*(2-1), or

Degrees of freedom are important in a Chi-square test because they factor into your calculations of the probability of independence. Once you calculate a Chi-square value, you use this number and the degrees of freedom to decide the probability, or

Earlier, I showed a simple example of observed vs. expected data, using an artificial data set on the party affiliations of males and females. I show them again below.

Democrat | Republican | Total | |

Male | 20 | 30 | 50 |

Female | 30 | 20 | 50 |

Total | 50 | 50 | 100 |

Democrat | Republican | Total | |

Male | 25 | 25 | 50 |

Female | 25 | 25 | 50 |

Total | 50 | 50 | 100 |

We will focus on models based on the null hypothesis that the distribution of data is due to chance -- that is, our models will reflect the expected distribution of data when that hypothesis is assumed to be true. But as I mentioned before, the ease of dividing up this data is due to the simplicity of the distribution I chose. How do we calculate the expected distribution of a more complicated data set?

Pass | Fail | Total | |

Attended | 25 | 6 | 31 |

Skipped | 8 | 15 | 23 |

Total | 33 | 21 | 54 |

Here is the grid for an earlier example I discussed, showing how students who attended or skipped class performed on an exam. The numbers for this example are not so clean! Fortunately, we have a formula to guide us.

The estimated value for each cell is the total for its row multiplied by the total for its column, then divided by the total for the table: that is,

I'll show a different method for notating observed versus expected counts below: the expected frequency appears in parentheses below the observed frequency. This allows you to show all your data in one clean table.

Pass | Fail | Total | |

Attended | 25 (18.94) | 6 (12.05) | 31 |

Skipped | 8 (14.05) | 15 (8.94) | 23 |

Total | 33 | 21 | 54 |

We have now calculated the distribution of our totals based on the assumption that attending class will have absolutely no effect on your test performance. Let's all hope we can prove this null hypothesis wrong.

It's finally time to put our data to the test. You can find many programs that will calculate a Chi-square value for you, and later I will show you how to do it in Excel. For now, however, let's start by trying to understand the formula itself.

Using this formula, we find that the Chi-square value for our gender/party example is ((20-25)^2/25) + ((30-25)^2/25) + ((30-25)^2/25) + ((20-25)^2/25), or (25/25) + (25/25) + (25/25) + (25/25), or 1 + 1 + 1 + 1, which comes out to

(1) The Chi-square value.

(2) The degrees of freedom.

Once you have this information, there are a couple of methods you can use to get your p-value. For example, charts like this one or even Javascript programs like the one on this site will take the Chi-square value and degrees of freedom as input, and simply return a p-value. In the chart, you choose your degrees of freedom (df) value on the left, follow along its row to the closest number to your Chi-square value, and then check the corresponding number in the top row to see the approximate probability ("Significance Level") for that value. The Javascript program is more direct, as you simply input your numbers and click "calculate." Later, I will also show you how to make Excel do the work for you.

So, for our example, we take a Chi-square value of 4 and a df of 1, which gives us a p-value of

By convention, the "cutoff" point for a p-value is 0.05; anything below that can be considered a very low probability, while anything above it is considered a reasonable probability. However, that does not mean that we should take our 0.046 value and say, "Eureka! They're dependent!" Actually, 0.046 is so close to 0.05 that there's really not much we

So, let's try our other data set, and see if attending class really does affect your exam performance.

Pass | Fail | Total | |

Attended | 25 (18.94) | 6 (12.05) | 31 |

Skipped | 8 (14.05) | 15 (8.94) | 23 |

Total | 33 | 21 | 54 |

I'm going to skip the specific formula this time, and use the javascript program on this site to do the calculation for me. It returns a value of

This is all you need to know to calculate and understand Pearson's Chi-square test for independence. It's a widely popular test because once you know the formula, it can all be done on a pocket calculator, and then compared to simple charts to give you a probability value. You can also use this spreadsheet to play around with all the steps of the test (spreadsheet created by Bill Labov, with some small additions by Joel Wallenberg). The Chi-square test will prove to be a handy tool for analyzing all kinds of relationships; once you know the basics for a 2x2 grid, expanding to a larger set of values is easy. Good luck!