Eigenvalues and Eigenvectors (Introduction)

TL;DR

Eigenvalues and eigenvectors help us understand how a linear transformation stretches or shrinks vectors, identifying special directions that remain unchanged. An eigenvector is a vector whose direction isn't changed by a transformation, only scaled by a factor called the eigenvalue. These concepts are fundamental for simplifying complex systems and analyzing stability.

1. The Mental Model

Imagine you have a stretching machine that transforms shapes. Most vectors will change both their length and direction, but certain "special" vectors only get stretched or shrunk, staying on the same line. These special vectors are eigenvectors, and the factor by which they're scaled is the eigenvalue.

2. The Core Material

Eigenvalues and eigenvectors are fundamental to understanding linear transformations. They reveal the intrinsic properties of a matrix, showing how it "acts" on certain vectors.

What's a Linear Transformation?

A linear transformation is basically a function that takes a vector and outputs another vector, preserving certain properties like scaling and addition. Think of rotations, reflections, scaling, or shearing—these are all linear transformations, and they can be represented by matrices.

The Special Relationship

For a given square matrix $A$, an eigenvector $v$ is a non-zero vector that, when transformed by $A$, only changes in magnitude, not direction. This scaling factor is the eigenvalue $\lambda$. Mathematically, this relationship is expressed as:

$Av = \lambda v$

Where:
* $A$ is the square matrix representing the linear transformation.
* $v$ is the eigenvector (a non-zero vector).
* $\lambda$ is the eigenvalue (a scalar).

This equation means that applying the transformation $A$ to $v$ is the same as just scaling $v$ by $\lambda$.

Finding Eigenvalues

To find the eigenvalues, we rearrange the equation:
$Av - \lambda v = 0$
We can rewrite $\lambda v$ as $\lambda Iv$, where $I$ is the identity matrix of the same size as $A$. This lets us factor out $v$:
$(A - \lambda I)v = 0$

For $v$ to be a non-zero vector, the matrix $(A - \lambda I)$ must be singular (non-invertible). This happens when its determinant is zero:
$\det(A - \lambda I) = 0$

This equation is called the characteristic equation. Solving it for $\lambda$ will give you the eigenvalues. It usually results in a polynomial.

Finding Eigenvectors

Once you have the eigenvalues $\lambda$, you can find the corresponding eigenvectors. For each $\lambda$:
1. Substitute $\lambda$ back into the equation $(A - \lambda I)v = 0$.
2. Solve this system of linear equations for $v$. You'll find that there are infinitely many solutions for $v$ (any scalar multiple of an eigenvector is also an eigenvector for the same eigenvalue). We typically look for a basis for the null space of $(A - \lambda I)$, which is called the eigenspace for that eigenvalue.

Here's a flow of how you'd typically find them:

graph TD
    A["Start with matrix A"] --> B["Form (A - λI)"]
    B --> C["Calculate det(A - λI) = 0"]
    C --> D["Solve characteristic equation for λ (Eigenvalues)"]
    D --> E{For each λ:};
    E --> F["Substitute λ into (A - λI)v = 0"]
    F --> G["Solve the system for v (Eigenvectors)"]
    G --> H["End"]

3. Worked Example

Let's find the eigenvalues and eigenvectors for the matrix $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$.

Step 1: Find the eigenvalues.
First, set up the characteristic equation $\det(A - \lambda I) = 0$:
$A - \lambda I = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{pmatrix}$

Now, calculate the determinant:
$\det \begin{pmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{pmatrix} = (2-\lambda)(2-\lambda) - (1)(1) = (2-\lambda)^2 - 1$
Set the determinant to zero:
$(2-\lambda)^2 - 1 = 0$
$(2-\lambda)^2 = 1$
$2-\lambda = \pm 1$

This gives us two possible values for $\lambda$:
$2-\lambda = 1 \implies \lambda_1 = 1$
$2-\lambda = -1 \implies \lambda_2 = 3$

So, our eigenvalues are $\lambda_1 = 1$ and $\lambda_2 = 3$.

Step 2: Find the eigenvectors.

For $\lambda_1 = 1$:
Substitute $\lambda_1 = 1$ into $(A - \lambda I)v = 0$:
$\begin{pmatrix} 2-1 & 1 \\ 1 & 2-1 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$
$\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$

This gives us the system of equations:
$v_1 + v_2 = 0$
$v_1 + v_2 = 0$
Both equations are the same. From this, we can say $v_1 = -v_2$.
Let $v_2 = t$ (where $t$ is any non-zero scalar). Then $v_1 = -t$.
So, the eigenvector $v$ can be written as $t \begin{pmatrix} -1 \\ 1 \end{pmatrix}$.
A common choice (setting $t=1$) is $v_1 = \begin{pmatrix} -1 \\ 1 \end{pmatrix}$.

For $\lambda_2 = 3$:
Substitute $\lambda_2 = 3$ into $(A - \lambda I)v = 0$:
$\begin{pmatrix} 2-3 & 1 \\ 1 & 2-3 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$
$\begin{pmatrix} -1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$

This gives us the system of equations:
$-v_1 + v_2 = 0$
$v_1 - v_2 = 0$
Again, both equations are equivalent, giving $v_1 = v_2$.
Let $v_2 = s$. Then $v_1 = s$.
So, the eigenvector $v$ can be written as $s \begin{pmatrix} 1 \\ 1 \end{pmatrix}$.
A common choice (setting $s=1$) is $v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$.

4. Key Takeaways

• Eigenvectors are special vectors that only get scaled, not rotated or flipped, by a linear transformation.
• Eigenvalues are the scalar factors by which their corresponding eigenvectors are scaled.
• The characteristic equation, $\det(A - \lambda I) = 0$, is used to find eigenvalues.
• Once eigenvalues are known, substitute them back into $(A - \lambda I)v = 0$ to find the eigenvectors.
• An eigenvector is never the zero vector.
• Any scalar multiple of an eigenvector is also an eigenvector for the same eigenvalue.

Common Mistakes to Avoid:
- Forgetting that eigenvectors must be non-zero vectors.
- Making arithmetic errors when calculating the determinant or solving the system of equations.
- Not multiplying $\lambda$ by the identity matrix $I$ when forming $(A - \lambda I)$.
- Confusing $v$ with $\lambda v$ when writing the eigenvalue equation.

5. Now Try It

Find the eigenvalues and eigenvectors for the matrix $B = \begin{pmatrix} 3 & 0 \\ 0 & -2 \end{pmatrix}$.
What to do: Follow the steps: 1) Calculate $\det(B - \lambda I) = 0$ to find $\lambda$. 2) For each $\lambda$, solve $(B - \lambda I)v = 0$ for $v$.
What success looks like: You should find two distinct eigenvalues and two corresponding eigenvectors (or directions for them). One eigenvector will be along the x-axis, and the other along the y-axis, due to the diagonal nature of the matrix.