Basis, Dimension, and Rank

TL;DR

A basis is a minimal set of linearly independent vectors that can create any other vector in a vector space. The dimension is simply the number of vectors in this basis, telling you "how big" the space is. Rank tells you the dimension of the output space (column space) of a matrix, which is crucial for understanding linear transformations.

1. The Mental Model

Think of a vector space like a room. A basis is like a minimal set of unique directions (e.g., along the floor, up the wall) that let you reach any point in that room. The dimension is the number of these unique directions. The rank of a matrix describes how many "effective" directions that matrix can transform vectors into.

2. The Core Material

What is a Basis?

Imagine you have a set of vectors. A basis for a vector space (or subspace) is a special set of vectors that meets two conditions:
1. Linear Independence: No vector in the set can be written as a combination of the others. They all point in "truly different" directions.
2. Spanning: Every other vector in the space can be written as a linear combination of the vectors in this set. They "reach" everywhere in the space.

A basis is like the fundamental building blocks for a vector space. For example, in 3D space, the vectors \[[1,0,0]\], \[[0,1,0]\], and \[[0,0,1]\] form a basis. You can create any 3D vector by combining these three.

Dimension of a Vector Space

The dimension of a vector space is simply the number of vectors in any basis for that space. It's a fundamental property that tells you how many "degrees of freedom" or independent directions are needed to describe the space. If a basis has 3 vectors, the dimension is 3.

Row Space, Column Space, and Null Space

When you have a matrix $A$, it defines several important vector spaces:
* Column Space (Col $A$): This is the span of the columns of $A$. It represents all possible output vectors when you multiply $A$ by any vector $x$ (i.e., $Ax$).
* Row Space (Row $A$): This is the span of the rows of $A$. It's the same as the column space of $A^T$.
* Null Space (Null $A$): This consists of all vectors $x$ such that $Ax = 0$. These are the vectors that the matrix transforms into the zero vector.

Rank of a Matrix

The rank of a matrix $A$ is the dimension of its column space (or, equivalently, its row space). It tells you how many linearly independent columns (or rows) the matrix has. It's a key indicator of how much "information" or "dimension" the matrix preserves when it acts as a linear transformation.

Here's how these concepts relate:

graph LR
    A["Matrix A"] --> B["Reduced Row Echelon Form (RREF)"]
    B --> C["Pivot Columns (in original A)"]
    B --> D["Non-Pivot Columns"]
    C --> E["Basis for Column Space"]
    C --> F["Rank (number of pivots)"]
    B --> G["Non-zero Rows (in RREF)"]
    G --> H["Basis for Row Space"]
    D --> I["Basis for Null Space"]
    F --> J["Dimension of Column Space"]
    F --> K["Dimension of Row Space"]
    I --> L["Dimension of Null Space (Nullity)"]
    J -- "Equals" --> K
    F -- "Rank-Nullity Theorem" --> L

Rank-Nullity Theorem

This is a powerful theorem that connects the rank of a matrix to the dimension of its null space (often called the nullity). For an $m \times n$ matrix $A$:

Rank($A$) + Nullity($A$) = $n$ (number of columns)

This means the number of linearly independent columns plus the dimension of the space of vectors that get mapped to zero equals the total number of columns.

3. Worked Example

Let's find the basis for the column space, the row space, and the null space, and then determine the rank and nullity of the following matrix:

$A = \begin{pmatrix} 1 & 2 & 1 & 0 \\ 0 & 0 & 1 & 2 \\ 0 & 0 & 0 & 0 \end{pmatrix}$

First, put $A$ into its Reduced Row Echelon Form (RREF). This matrix is already in RREF.

1. Basis for Column Space: The pivot columns are the 1st and 3rd columns. Take these columns from the original matrix $A$:
   Basis(Col $A$) = $\{ \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}, \begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix} \}$

2. Rank: The number of pivot columns is 2. So, Rank($A$) = 2.

3. Basis for Row Space: The non-zero rows in the RREF are the basis for the row space:
   Basis(Row $A$) = $\{ [1, 2, 1, 0], [0, 0, 1, 2] \}$
   (Notice that the dimension of the row space is also 2, which equals the rank!)

4. Basis for Null Space: To find the null space, we solve $Ax = 0$.
   From RREF, we have:
   $x_1 + 2x_2 + x_3 = 0$
   $x_3 + 2x_4 = 0$

   Let $x_2$ and $x_4$ be free variables (corresponding to non-pivot columns).
   From the second equation: $x_3 = -2x_4$
   Substitute into the first: $x_1 + 2x_2 + (-2x_4) = 0 \implies x_1 = -2x_2 + 2x_4$

   So, any vector $x$ in the null space looks like:
   $x = \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{pmatrix} = \begin{pmatrix} -2x_2 + 2x_4 \\ x_2 \\ -2x_4 \\ x_4 \end{pmatrix} = x_2 \begin{pmatrix} -2 \\ 1 \\ 0 \\ 0 \end{pmatrix} + x_4 \begin{pmatrix} 2 \\ 0 \\ -2 \\ 1 \end{pmatrix}$

   Basis(Null $A$) = $\{ \begin{pmatrix} -2 \\ 1 \\ 0 \\ 0 \end{pmatrix}, \begin{pmatrix} 2 \\ 0 \\ -2 \\ 1 \end{pmatrix} \}$

5. Nullity: The number of vectors in the null space basis is 2. So, Nullity($A$) = 2.

Let's check the Rank-Nullity Theorem: Rank($A$) + Nullity($A$) = 2 + 2 = 4. The number of columns $n$ is 4. It holds!

4. Key Takeaways

• A basis is a minimal, linearly independent set of vectors that spans the entire vector space.
• The dimension of a vector space is the number of vectors in any of its bases.
• The rank of a matrix is the dimension of its column space (or row space). It's the number of pivot columns in its RREF.
• The nullity of a matrix is the dimension of its null space, which is the number of free variables in the RREF.
• The Rank-Nullity Theorem (Rank + Nullity = number of columns) is a fundamental relationship connecting these concepts.
• To find a basis for the column space, use the original columns corresponding to pivot columns in the RREF.
• To find a basis for the row space, use the non-zero rows of the RREF itself.

Common Mistakes to Avoid:
- Don't use the pivot columns from the RREF to form the basis for the column space; use the corresponding columns from the original matrix.
- Confusing rank with the total number of rows or columns; rank is specifically about linearly independent columns/rows.
- Forgetting that the dimension of the row space is always equal to the dimension of the column space (which is the rank).
- Incorrectly identifying free variables when finding the null space, leading to an incorrect nullity.

5. Now Try It

Consider the matrix $B = \begin{pmatrix} 1 & 3 & 0 \\ 2 & 6 & 0 \\ -1 & -3 & 1 \end{pmatrix}$.
1. Find the RREF of $B$.
2. Identify a basis for the column space of $B$ and state its rank.
3. Identify a basis for the row space of $B$.
4. Identify a basis for the null space of $B$ and state its nullity.
5. Verify the Rank-Nullity Theorem for matrix $B$.

Success looks like: You should find the rank and nullity, and show they correctly sum to the total number of columns. Your bases should contain linearly independent vectors that span their respective spaces.