Linear Transformations and Matrices

TL;DR

Linear transformations are functions that map vectors from one space to another while preserving vector addition and scalar multiplication. Matrices provide a concrete way to represent these transformations, allowing us to perform complex geometric operations algebraically. Every linear transformation can be uniquely represented by a matrix, making them powerful tools for understanding and manipulating vector spaces.

1. The Mental Model

Think of a linear transformation as a machine that takes a vector as input and spits out a new vector. This machine doesn't just randomly change things; it moves, rotates, stretches, or shrinks vectors in a very predictable and consistent way. A matrix is simply the "instruction manual" for this machine.

2. The Core Material

A linear transformation is a special kind of function, let's call it $T$, that maps vectors from one vector space ($V$) to another vector space ($W$). It satisfies two key properties:

1. Additivity: $T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})$ for any vectors $\mathbf{u}, \mathbf{v}$ in $V$.
2. Homogeneity (Scalar Multiplication): $T(c\mathbf{u}) = cT(\mathbf{u})$ for any scalar $c$ and vector $\mathbf{u}$ in $V$.

These properties mean that linear transformations preserve the "structure" of vector spaces. They don't bend or break the grid lines, they just move them around.

Representing Transformations with Matrices

The remarkable thing is that every linear transformation can be represented by a matrix. If you have a linear transformation $T: \mathbb{R}^n \to \mathbb{R}^m$, you can find an $m \times n$ matrix $A$ such that $T(\mathbf{x}) = A\mathbf{x}$ for any vector $\mathbf{x}$ in $\mathbb{R}^n$.

How do you find this matrix $A$? You figure out what the transformation does to the standard basis vectors.
For $\mathbb{R}^n$, the standard basis vectors are:
$\mathbf{e}_1 = \begin{pmatrix} 1 \\ 0 \\ \vdots \\ 0 \end{pmatrix}$, $\mathbf{e}_2 = \begin{pmatrix} 0 \\ 1 \\ \vdots \\ 0 \end{pmatrix}$, ..., $\mathbf{e}_n = \begin{pmatrix} 0 \\ 0 \\ \vdots \\ 1 \end{pmatrix}$.

The columns of matrix $A$ are simply the transformed standard basis vectors:
$A = \begin{pmatrix} | & | & & | \\ T(\mathbf{e}_1) & T(\mathbf{e}_2) & \dots & T(\mathbf{e}_n) \\ | & | & & | \end{pmatrix}$

Let's look at some common 2D transformations:

Rotation

Rotating a vector in $\mathbb{R}^2$ counter-clockwise by an angle $\theta$.
$T\left(\begin{pmatrix} x \\ y \end{pmatrix}\right) = \begin{pmatrix} x \cos\theta - y \sin\theta \\ x \sin\theta + y \cos\theta \end{pmatrix}$
To find the matrix, apply $T$ to $\mathbf{e}_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $\mathbf{e}_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}$:
$T\left(\begin{pmatrix} 1 \\ 0 \end{pmatrix}\right) = \begin{pmatrix} \cos\theta \\ \sin\theta \end{pmatrix}$
$T\left(\begin{pmatrix} 0 \\ 1 \end{pmatrix}\right) = \begin{pmatrix} -\sin\theta \\ \cos\theta \end{pmatrix}$
So, the rotation matrix is $R_\theta = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}$.

Scaling

Scaling a vector by factors $s_x$ in the x-direction and $s_y$ in the y-direction.
$T\left(\begin{pmatrix} x \\ y \end{pmatrix}\right) = \begin{pmatrix} s_x x \\ s_y y \end{pmatrix}$
$T\left(\begin{pmatrix} 1 \\ 0 \end{pmatrix}\right) = \begin{pmatrix} s_x \\ 0 \end{pmatrix}$
$T\left(\begin{pmatrix} 0 \\ 1 \end{pmatrix}\right) = \begin{pmatrix} 0 \\ s_y \end{pmatrix}$
The scaling matrix is $S = \begin{pmatrix} s_x & 0 \\ 0 & s_y \end{pmatrix}$.

Composition of Transformations

If you apply one linear transformation $T_1$ and then another $T_2$, the result is also a linear transformation. If $T_1$ is represented by matrix $A_1$ and $T_2$ by $A_2$, then the combined transformation $T_2(T_1(\mathbf{x}))$ is represented by the matrix product $A_2 A_1$. Order matters! Applying $T_1$ then $T_2$ means multiplying by $A_2 A_1$.

graph TD
    A["Original Vector X"] --> B["Apply Transformation T1 (Matrix A1)"]
    B --> C["Intermediate Vector T1(X)"]
    C --> D["Apply Transformation T2 (Matrix A2)"]
    D --> E["Transformed Vector T2(T1(X))"]

    subgraph Mathematical Representation
        F["X"] --> G["A1 * X"]
        G --> H["(A2 * A1) * X"]
        H --> I["A_composite * X"]
    end

3. Worked Example

Let's find the matrix for a linear transformation $T: \mathbb{R}^2 \to \mathbb{R}^2$ that first rotates a vector by 90 degrees counter-clockwise, and then scales the x-component by 2 and the y-component by 3.

Step 1: Find the rotation matrix for 90 degrees.
For $\theta = 90^\circ = \pi/2$ radians:
$\cos(90^\circ) = 0$
$\sin(90^\circ) = 1$
The rotation matrix $R_{90} = \begin{pmatrix} \cos 90^\circ & -\sin 90^\circ \\ \sin 90^\circ & \cos 90^\circ \end{pmatrix} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$.

Step 2: Find the scaling matrix.
We want to scale the x-component by $s_x = 2$ and the y-component by $s_y = 3$.
The scaling matrix $S = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$.

Step 3: Combine the transformations.
Since we apply rotation FIRST, then scaling, the combined matrix $A$ will be $S \cdot R_{90}$.
$A = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix} \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} = \begin{pmatrix} (2)(0) + (0)(1) & (2)(-1) + (0)(0) \\ (0)(0) + (3)(1) & (0)(-1) + (3)(0) \end{pmatrix} = \begin{pmatrix} 0 & -2 \\ 3 & 0 \end{pmatrix}$.

So, the matrix $A = \begin{pmatrix} 0 & -2 \\ 3 & 0 \end{pmatrix}$ represents the linear transformation that first rotates by 90 degrees counter-clockwise, then scales by (2, 3).

Let's test it with a vector, say $\mathbf{v} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$.
Applying $R_{90}$ first: $R_{90}\mathbf{v} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} -1 \\ 1 \end{pmatrix}$. (Rotated 90 deg)
Then applying $S$: $S(R_{90}\mathbf{v}) = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix} \begin{pmatrix} -1 \\ 1 \end{pmatrix} = \begin{pmatrix} -2 \\ 3 \end{pmatrix}$. (Scaled)

Now, directly using the combined matrix $A$:
$A\mathbf{v} = \begin{pmatrix} 0 & -2 \\ 3 & 0 \end{pmatrix} \begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} (0)(1) + (-2)(1) \\ (3)(1) + (0)(1) \end{pmatrix} = \begin{pmatrix} -2 \\ 3 \end{pmatrix}$.
The results match!

4. Key Takeaways

• A linear transformation is a function that preserves vector addition and scalar multiplication.
• Every linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$ can be represented by a unique $m \times n$ matrix.
• The columns of a transformation matrix are the transformed standard basis vectors.
• Common geometric operations like rotation, scaling, reflection, and shear are linear transformations.
• Composing multiple linear transformations means multiplying their corresponding matrices (in reverse order of application).
• Matrix-vector multiplication $A\mathbf{x}$ is the mechanism by which the matrix $A$ transforms the vector $\mathbf{x}$.

Common Mistakes to Avoid:
- Mixing up order in matrix multiplication: For $T_1$ then $T_2$, the composite matrix is $A_2 A_1$, not $A_1 A_2$.
- Confusing the dimensions: An $m \times n$ matrix transforms $\mathbb{R}^n$ vectors into $\mathbb{R}^m$ vectors.
- Assuming all transformations are linear: Translations (shifting a vector) are NOT linear because they don't map the zero vector to the zero vector.
- Forgetting that the transformation matrix depends on the basis: We've been using the standard basis, but different bases lead to different matrix representations.

5. Now Try It

Consider a linear transformation $T: \mathbb{R}^2 \to \mathbb{R}^2$ that first reflects a vector across the y-axis, and then rotates it 45 degrees clockwise. Find the single matrix that represents this composite transformation.
What success looks like: You should derive a $2 \times 2$ matrix with specific numerical entries (involving $\sqrt{2}$).