What the Heck are Eigenvalues and Eigenvectors?

There are certain words that will put a speedy end to any dinner conversation.  Eigenvalue, eigenvector, and eigenfunction are probably in the top 100 such words. They are used in linear algebra and unfortunately, when they are used, they are rarely explained.  If you do happen to find them, then most likely, you have opened a technical book by mistake.

Consider the following conversation as an example:

Guest:  Do you feel that Google will continue its dominance of search?

Mathematician:  I feel confident that Google's use of eigenvectors places it in a unique position of importance among...

Guest:  (yawn) 

Last year, a book was written which attempted to explain the mathematics behind Google's page rank algorithm.  One of the chapters of the book is called:  The $25 billion dollar Eigenvector.

My goal in this hub is to explain the intuitions behind these terms in an effort to explain what they are, how they are used, and some basic ideas about them.

The prefix "eigen" is itself a German word which means "proper" or "characteristic (see here). Unfortunately, this doesn't help us very much in understanding what they are other than to suggest that they were invented by a German mathematician. There is some truth to this since possibly the first person to give them their current name was the German mathematician David Hilbert (see, here). Although, it may have also been the German physicist Hermann Ludwig Ferdinand von Helmholtz who was first (see, here).

Initially, eigenvalues were called "Proper Values" in the United States but that term is no longer used. Today, they are universally called eigenvalues and eigenvectors (for a complete history of the term, see here).

An eigenvalue is a number that is derived from a square matrix.  A square matrix is itself just a collection of n rows of n numbers.  An eigenvalue is usually represented by the Greek letter lamdba (λ).

Let A be a square matrix (a collections of n rows of n numbers which means that there are n x n numbers in total).

Let x be a nonzero vector.  A vector is just a column of numbers.  A nonzero vector is any vector where not all the numbers are 0.   By convention, a vector that consists entirely of 0's is called the 0 vector.

We say that a number is the eigenvalue for this square matrix if and only if there exists a nonzero vector x such that Ax = λx where:

A is the square matrix

x is the nonzero vector

λ is a nonzero value.

In this circumstance, λ is the eigenvalue and x is the eigenvector.

So far, we've shown that certain square matrices satisfy an equation such that:

Ax = x

So what?  Why should I care about matrices, nonzero vectors, eigenvalues, and eigenvectors?

The major reason for studying eigenvalues and eigenvectors is that they are used in many important mathematical results.  Perhaps, it makes sense to show one example of a real world application before answering the other questions.

On July 1, 1940, the Tacoma Narrows Bridge opened in Washington state.  It connected the city of Tacoma with the Kitsap Penninsula and ran over the Tacoma Narrows which is a strait across the Puget Sound.  Four months after it was built, it collapsed.  This was captured in film and was later nicknamed "Gallopin' Gertie".  The full story can be found here.

Believe it or not but this collapse can be explained in engineering terms using the idea eigenvalues.

Why did the bridge collapse?

One explanation centers around natural frequencies.  The natural frequency is "the frequency at which a system naturally vibrates once it has been set into motion" (from this article).  In other words, the natural frequency is the characteristic motion of structure.  It's the motion that a structure takes on in response to wind or being walked on.  It is especially important in the design of musical instruments and in the tuning of radios.

Mathematically, the natural frequency can be characterized by the eigenvalue of the smallest magnitude.

The model suggests that the "oscillations of the bridge were caused by the frequency of the wind being too close to the natural frequency of the bridge." (from this article)  When frequencies match, they compound which proved too strong a force for the bridge.

The same type of collapse can happen with soldiers marching.  If soldiers march in lock step too close to the natural frequency of a bridge, then it is possible, under some circumstances, for the bridge to collapse.