Introduction to Eigenvalues and Eigenvectors for Beginners

Eigenvalues and eigenvectors are core concepts in linear algebra. While the names sound intimidating, the underlying idea is surprisingly simple and visually intuitive. They are also incredibly powerful, with applications ranging from search engines to quantum mechanics.

The Core Idea: What Are They?

In linear algebra, a matrix can be thought of as a linear transformation. It takes a vector as input and produces a new vector as output. This transformation can stretch, shrink, rotate, or shear the original vector.

An eigenvector of a matrix is a special non-zero vector that, when transformed by the matrix, does not change its direction. The only effect the matrix has on an eigenvector is to scale it (make it longer or shorter).

The eigenvalue is the scalar factor by which the eigenvector is scaled.

The Defining Equation

This relationship is captured in one elegant equation:

$$Av = \lambda v$$

Where:

• A is the matrix (the transformation).
• v is the eigenvector (the vector that doesn't change direction).
• λ (lambda) is the eigenvalue (the scaling factor).

So, applying matrix A to its eigenvector v has the same effect as just multiplying v by a number λ.

In this example, the blue vector is an eigenvector. The matrix transformation (A) scales it, but its direction remains on the same line.

How to Find Eigenvalues and Eigenvectors

To find the eigenvalues and eigenvectors for a square matrix A, we start with the defining equation:

$$Av = \lambda v$$

We can rewrite this as:

$$Av - \lambda v = 0$$

To factor out the vector v, we need to introduce the identity matrix I (a matrix with 1s on the diagonal and 0s elsewhere):

$$Av - \lambda I v = 0$$

Now, we can factor out v:

$$(A - \lambda I)v = 0$$

Since we are looking for non-zero eigenvectors v, this means the matrix (A - \lambda I) must be "singular," which is another way of saying its determinant must be zero.

$$\det(A - \lambda I) = 0$$

This is called the characteristic equation. Solving it gives us the eigenvalues λ. Once we have the eigenvalues, we can plug them back into the equation (A - \lambda I)v = 0 to find the corresponding eigenvectors v.

A Simple 2x2 Example

Let's find the eigenvalues and eigenvectors for the matrix $A = \begin{pmatrix} 2 & 1 \ 1 & 2 \end{pmatrix}$.

Step 1: Set up the characteristic equation $\det(A - \lambda I) = 0$.

$$A - \lambda I = \begin{pmatrix} 2 & 1 \ 1 & 2 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \ 0 & 1 \end{pmatrix} = \begin{pmatrix} 2-\lambda & 1 \ 1 & 2-\lambda \end{pmatrix}$$

Now find the determinant:

$$\det \begin{pmatrix} 2-\lambda & 1 \ 1 & 2-\lambda \end{pmatrix} = (2-\lambda)(2-\lambda) - (1)(1) = 0$$

Step 2: Solve for the eigenvalues ($\lambda$).

$$(2-\lambda)^2 - 1 = 0$$ $$4 - 4\lambda + \lambda^2 - 1 = 0$$ $$\lambda^2 - 4\lambda + 3 = 0$$

Factoring this quadratic equation gives:

$$(\lambda - 3)(\lambda - 1) = 0$$

So, the eigenvalues are $\lambda_1 = 3$ and $\lambda_2 = 1$.

Step 3: Find the eigenvector for each eigenvalue.

For $\lambda_1 = 3$: We solve $(A - 3I)v = 0$:

$$\begin{pmatrix} 2-3 & 1 \ 1 & 2-3 \end{pmatrix} \begin{pmatrix} x \ y \end{pmatrix} = \begin{pmatrix} 0 \ 0 \end{pmatrix}$$ $$\begin{pmatrix} -1 & 1 \ 1 & -1 \end{pmatrix} \begin{pmatrix} x \ y \end{pmatrix} = \begin{pmatrix} 0 \ 0 \end{pmatrix}$$

This gives us the equation $-x + y = 0$, or $x = y$. Any vector where the components are equal (and not zero) is an eigenvector. A simple choice is $v_1 = \begin{pmatrix} 1 \ 1 \end{pmatrix}$.

For $\lambda_2 = 1$: We solve $(A - 1I)v = 0$:

$$\begin{pmatrix} 2-1 & 1 \ 1 & 2-1 \end{pmatrix} \begin{pmatrix} x \ y \end{pmatrix} = \begin{pmatrix} 0 \ 0 \end{pmatrix}$$ $$\begin{pmatrix} 1 & 1 \ 1 & 1 \end{pmatrix} \begin{pmatrix} x \ y \end{pmatrix} = \begin{pmatrix} 0 \ 0 \end{pmatrix}$$

This gives us the equation $x + y = 0$, or $y = -x$. A simple choice is $v_2 = \begin{pmatrix} 1 \ -1 \end{pmatrix}$.

Answer:

• Eigenvalue $\lambda_1 = 3$ has eigenvector $v_1 = \begin{pmatrix} 1 \ 1 \end{pmatrix}$.
• Eigenvalue $\lambda_2 = 1$ has eigenvector $v_2 = \begin{pmatrix} 1 \ -1 \end{pmatrix}$.

Why Are Eigenvalues and Eigenvectors Important?

Eigenvalues and eigenvectors reveal the fundamental properties of a matrix. They show which directions remain unchanged under the transformation and by how much the vectors in those directions are scaled. This has countless applications:

• Physics: Analyzing vibrations in mechanical structures. The eigenvectors are the modes of vibration, and the eigenvalues are their frequencies.
• Machine Learning: Principal Component Analysis (PCA) uses eigenvectors to reduce the dimensionality of data, finding the "principal" directions of variance.
• Search Engines: Google's original PageRank algorithm used the eigenvector of a massive matrix to rank the importance of web pages.
• Quantum Mechanics: The state of a system is described by vectors, and observable quantities (like energy) are represented by eigenvalues of operators (matrices).

Conclusion

Eigenvalues and eigenvectors are not just abstract mathematical concepts. They represent the "axes" of a linear transformation, revealing its most fundamental properties. By finding the vectors that are only scaled, not rotated, we gain deep insight into the behavior of the system the matrix represents.