Introduction to Vectors and Vector Spaces

TL;DR

Vectors are mathematical objects that have both magnitude and direction, commonly represented as arrows or lists of numbers. A vector space is a collection of vectors that satisfies specific rules, allowing them to be added together and scaled. Understanding vectors and vector spaces is fundamental for grasping concepts like transformations, dimensions, and data representation in linear algebra.

1. The Mental Model

Think of vectors as instructions for movement: "go 3 steps east, then 2 steps north." A vector space is like a city grid where all possible travel instructions (vectors) make sense and can be combined or extended.

2. The Core Material

What is a Vector?

A vector is a quantity that has both magnitude (how much) and direction (which way). Contrast this with a scalar, which only has magnitude (like temperature or speed). You can visualize a vector as an arrow starting from an origin point, pointing towards its destination.

In linear algebra, we usually represent vectors as ordered lists of numbers, often written as columns:

$$ \mathbf{v} = \begin{pmatrix} x \\ y \end{pmatrix} \quad \text{or} \quad \mathbf{w} = \begin{pmatrix} x \\ y \\ z \end{pmatrix} $$

The numbers inside the vector are called its components or entries. The number of components tells you the vector's dimension. So, $(3, 2)$ is a 2-dimensional vector.

Vector Operations

You can do two main things with vectors:

1. Vector Addition

To add two vectors, you add their corresponding components. This is like combining two sets of movement instructions. Geometrically, if you place the tail of the second vector at the head of the first, the sum is the vector from the first tail to the second head.

$$ \mathbf{u} = \begin{pmatrix} u_1 \\ u_2 \end{pmatrix}, \quad \mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} \implies \mathbf{u} + \mathbf{v} = \begin{pmatrix} u_1 + v_1 \\ u_2 + v_2 \end{pmatrix} $$

2. Scalar Multiplication

To multiply a vector by a scalar, you multiply each component of the vector by that scalar. This scales the vector's magnitude (makes it longer or shorter) and can reverse its direction if the scalar is negative.

$$ c \cdot \mathbf{v} = c \cdot \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \begin{pmatrix} c \cdot v_1 \\ c \cdot v_2 \end{pmatrix} $$

What is a Vector Space?

A vector space $V$ is a collection of vectors that "behaves nicely" under vector addition and scalar multiplication. This means that if you take any two vectors from the space and add them, the result is still in that space. If you take any vector from the space and multiply it by a scalar, the result is still in that space.

Formally, a vector space over a field (usually real numbers, $\mathbb{R}$) must satisfy ten axioms. Don't worry about memorizing all ten now, but the key idea is closure:

graph TD
    A["Collection of Objects (Candidates for Vectors)"] --> B{Can you add any two of them?};
    B -- Yes --> C{Is the result always in the collection?};
    C -- Yes --> D{Can you multiply any one by a scalar?};
    D -- Yes --> E{Is the result always in the collection?};
    E -- Yes --> F["It's a Vector Space!"];
    B -- No --> G["Not a Vector Space"];
    C -- No --> G;
    D -- No --> G;
    E -- No --> G;

Common examples of vector spaces you'll encounter are $\mathbb{R}^n$, which is the set of all $n$-dimensional real vectors. For example, $\mathbb{R}^2$ includes all vectors $(x, y)$ where $x, y$ are real numbers.

3. Worked Example

Let's take two vectors in $\mathbb{R}^2$:
$\mathbf{u} = \begin{pmatrix} 2 \\ 1 \end{pmatrix}$ and $\mathbf{v} = \begin{pmatrix} -1 \\ 3 \end{pmatrix}$.
Let's also use a scalar $c = 3$.

1. Vector Addition:
   $\mathbf{u} + \mathbf{v} = \begin{pmatrix} 2 \\ 1 \end{pmatrix} + \begin{pmatrix} -1 \\ 3 \end{pmatrix} = \begin{pmatrix} 2 + (-1) \\ 1 + 3 \end{pmatrix} = \begin{pmatrix} 1 \\ 4 \end{pmatrix}$

   The resulting vector $\begin{pmatrix} 1 \\ 4 \end{pmatrix}$ is also in $\mathbb{R}^2$.

2. Scalar Multiplication:
   $c \cdot \mathbf{u} = 3 \cdot \begin{pmatrix} 2 \\ 1 \end{pmatrix} = \begin{pmatrix} 3 \cdot 2 \\ 3 \cdot 1 \end{pmatrix} = \begin{pmatrix} 6 \\ 3 \end{pmatrix}$

   The resulting vector $\begin{pmatrix} 6 \\ 3 \end{pmatrix}$ is also in $\mathbb{R}^2$.

This example demonstrates that $\mathbb{R}^2$ is indeed closed under these operations, making it a vector space.

4. Key Takeaways

• Vectors have both magnitude and direction, often visualized as arrows or lists of numbers.
• Scalars are just magnitudes (single numbers).
• Vector addition means adding corresponding components.
• Scalar multiplication means multiplying each component by the scalar.
• A vector space is a set of vectors where adding any two vectors or multiplying any vector by a scalar always results in a vector still within that set (closure).
• $\mathbb{R}^n$ (all $n$-dimensional real vectors) is the most common example of a vector space.

Common mistakes to avoid:
- Confusing a scalar with a 1D vector; a scalar is just a number.
- Trying to add vectors of different dimensions (e.g., a 2D vector and a 3D vector).
- Forgetting that the result of vector operations (addition, scalar multiplication) must stay within the vector space.

5. Now Try It

Consider the set of all 2D vectors where the x-component is always zero: $V = \left\{ \begin{pmatrix} 0 \\ y \end{pmatrix} \mid y \in \mathbb{R} \right\}$.
1. Pick any two vectors from this set, say $\mathbf{u}$ and $\mathbf{v}$. Add them. Is the result still in $V$?
2. Pick any vector $\mathbf{w}$ from this set and any real scalar $c$. Multiply $\mathbf{w}$ by $c$. Is the result still in $V$?
If both answers are "yes," then $V$ is a vector space. What would success look like? Both your resulting vectors would have a zero in their first component, just like the original vectors from the set.