Degrees of Freedom

What are Degrees of Freedom?

The term “degrees of freedom” (often abbreviated as “d.f.” or “df”) describes the freedom for values, or variables, to vary. Put differently, a lower degrees of freedom means that there are more constraints to the variables.

History of Degrees of Freedom

The conceptual application of the degrees of freedom was recognized by mathematician Carl Friedrich Gauss as early as 1821. At the time, the concept was not defined as we know it today.

The first definition of degrees of freedom was provided by statistician William Sealy Gosset, known more commonly by his pseudonym, Student. Specifically, the usage of the concept was clearly laid out with his explanation of the Student’s t-distribution. On the other hand, the term “degrees of freedom” was popularized by statistician and biologist Ronald Fisher.

Intuitive Understanding of Degrees of Freedom

What does “freedom to vary” mean? In essence, freedom to vary is used to demonstrate a lack of constraint in a particular dataset or mathematical system.

Example

Say that you own seven shirts that you can wear in a week, and you decide to wear each shirt only once during the week.

On Sunday, you open the dresser and think to choose one of the seven shirts. You can choose to wear any of the seven shirts. On the second day, the shirt worn on the first day cannot be chosen, and you must choose from the remaining shirts. The pattern continues as follows:

• Sunday: 7 shirts to choose from
• Monday: 6 shirts to choose from
• Tuesday: 5 shirts to choose from
• Wednesday: 4 shirts to choose from
• Thursday: 3 shirts to choose from
• Friday: 2 shirts to choose from
• Saturday: 1 shirt to choose from

On the last day, Saturday, there is only one shirt to choose from, which, practically speaking, means that there is no choice. Put in different words, you are constrained on Saturday with your choice of which shirt you can wear.

In this one week that you are to choose one shirt per day, you have six days on which you are free to choose a shirt. It is identical to saying that your choice to choose a shirt is constrained on one day. Thus, this week, there are six degrees of freedom.

Mathematical Understanding of Degrees of Freedom

Taking a step closer to the application of degrees of freedom in statistics, we can utilize simple mathematical systems to showcase the applicability of degrees of freedom.

Simple Math System with No Constraints

Let’s think of a two-dimensional graph. Say, a graph with the traditional x- and y-axes.

If we are to choose a point with no constraints, or full freedom to vary, we can choose for x and any value for y. It means we can choose any point on the above graph without any restriction. It means, for such a system with two values, we have two degrees of freedom.

Simple Mathematical System with One Constraint

Elaborating on the previous example, let’s bring in a constraint:

• x + y = 7

Note that this is equivalent to y = 7 – x. Below is the line demonstrating the combinations of x and y that fit the criterion.

Now, thinking about the degrees of freedom again, notice that if we decide to choose x, then y is fixed. If we choose a value for y, then x must be fixed.

For example, if we choose x to be 4, then y must be 3. If we choose y to be 2, then x must be 5.

Since we can only choose one of the two variables without violating the constraint, we have one degree of freedom.

Simple Mathematical System with Two Constraints

Now, let’s bring in a second constraint to the system.

• x + y = 7 (in red)
• x – y = 1 (in blue)

To find out which values will not violate either criterion, we can use algebra. Alternatively, we can identify the intersection of the two curves, which shows x = 4 and y = 3.

The example shows that only when x = 4 and y = 3 are the constraints satisfied. Thus, we cannot choose any one of the two values, and we have zero degrees of freedom.

Intuition For Degrees of Freedom Using Mathematical Systems

From above, we see that as more constraints were added, the freedom to vary, and thus the degrees of freedom, decreased.

From a different perspective, we can think of the constraints as relationships between the two variables. With each additional interdependent relationship introduced between x and y, the degree of freedom for us to choose lowers by one degree.

Applicability of Degrees of Freedom in the Real World

Although the degree of freedom is an abstract idea and most frequently mentioned in statistics, it is very applicable in the real world.

For example, business owners looking to hire labor to produce output are faced with two variables – labor and output. In addition, the relationship between employees and output (i.e., the amount of output an employee can produce) is the constraint.

In such a case, the business owners can either decide the amount of output to be produced, which fixes the number of employees to be hired or decide on the number of employees, which fixes the amount of output produced. Thus, concerning output and employees, the owners have one degree of freedom.

More Resources

Thank you for reading CFI’s guide to Degrees of Freedom. To keep advancing your career, the additional CFI resources below will be useful:

• Decision Tree
• Game Theory
• Zero Sum Game
• Basic Statistics Concepts for Finance
• See all data science resources