Chapter 18
By the end of this section, you should be able to answer the following questions:
Given an $n\times n$ matrix $A,$ we call $A$ orthogonally diagonalisable if there exists an orthogonal matrix $P$ such that $P^{-1}AP = P^{T}AP$ is diagonal. To understand this, we first need to know what is meant by an orthogonal matrix.
Recall from the previous chapter that an orthogonal matrix is a real square matrix $Q$ such that the columns of $Q$ are mutually orthogonal unit vectors with respect to the Euclidean inner product (i.e., ${\v}_i\pd {\v}_j=0$ if $i\ne j,$ and $||{\v}_i||=1$).
An orthogonal matrix is then a real square matrix $Q$ such that $Q^{-1}=Q^T.$ An immediate consequence of this is that $\det(Q) = \pm 1.$
An orthogonal matrix is then a real square matrix $Q$ such that $Q^{-1}=Q^T.$ An immediate consequence of this is that $\det(Q) = \pm 1.$
So we have that \[ Q^TQ= QQ^T = I. \] Thus
$\text{det}\left(Q\right)^2=$
$\text{det}\left(Q^T\right)\text{det}\left(Q\right)= \quad \;\qquad \qquad \qquad \qquad \qquad $
$\qquad \qquad \qquad \quad \text{det}\left(Q^TQ\right)$
$=\text{det}\left(QQ^T\right)$
$=\text{det}\left(I\right)$
$=1.$
So $\;\text{det}\left(Q\right)^2=1$ $\;\Ra\;$ $\text{det}\left(Q\right)=\pm 1.$
A matrix $A$ is symmetric if and only if $A=A^T$. Symmetric matrices are easy to identify due to their "mirror symmetry" about the main diagonal. For example, we can tell by inspection that $$ A=\left(\begin{array}{rrr}-3&1&0\\1&-2&1\\0&1&-3\end{array}\right) $$ is symmetric.
Proof: Given $A\v_1 = \lambda_1 \v_1$ and $A\v_2 = \lambda_2 \v_2$ with $\lambda_1 \neq \lambda_2$, we want to prove that \[ \v_1 \pd \v_2 = 0, \] or equivalently \[ \v_1^T \v_2 = 0. \] in terms of matrix multiplication.
Proof: Given $A\v_1 = \lambda_1 \v_1$ and $A\v_2 = \lambda_2 \v_2$ with $\lambda_1 \neq \lambda_2$. Consider
| $\lambda_1 \v_1^T \v_2$ | $=$ | $\left(\lambda_1\v_1 \right)^T \v_2$ $=\left(A \v_1\right)^T \v_2$ |
| $=$ | $\v_1^T A^T \v_2$ 👉 $\left(\left(A B\right)^T = B^T A^T\right)$ | |
| $=$ | $\v_1^T A \v_2$ 👉 $\left(A^T =A\right)$ | |
| $=$ | $\v_1^T \lambda_2 \v_2$ $= \lambda_2\v_1^T \v_2$ |
Then $\left(\lambda_1 - \lambda_2\right)\v_1^T \v_2 = 0.$ Since $\lambda_1\neq \lambda_2,$ $\;\Ra\; \v_1^T \v_2 = 0.$ So $\;\v_1 \pd \v_2 = 0.$ Therefore $\v_1$ and $\v_2$ are orthogonal. $\quad \blacksquare$
It is straightforward to show that if a matrix is orthogonally diagonalisable, then it is symmetric:
Proof: If $A = P D P^T$ is given, then
$A^T = \left(PDP^T\right)^T$ $= \left(P^T \right)^T D^T P^T$ $= P D P^T$ $= A.$
Note: $A$ being symetric is a necessary condition for $A$ to be othogonally diagonalisable.
In fact, the converse is also true (although difficult to prove), giving us the amazing result 😲:
An $n\times n$ real matrix is orthogonally diagonalisable if and only if it is symmetric.
The significance of this is that a symmetric matrix is always diagonalisable by an orthogonal matrix.
Let $A=\left(\begin{array}{rrr}-3&1&0\\1&-2&1\\0&1&-3\end{array}\right)$ (see examples in Chapter 16).
We already know the eigenvalues are $-3, -1, -4$ with corresponding eigenvectors $$ {\v}_1=\mat{r}{1\\ 0\\ -1},\ \ {\v}_2=\mat{c}{1\\ 2\\ 1},\ \ {\v}_3=\mat{r}{1\\ -1\\ 1}. $$ Note that $A$ is real symmetric, so ${\v}_1$, ${\v}_2$ and ${\v}_3$ should be pairwise orthogonal.
Let $A=\left(\begin{array}{rrr}-3&1&0\\1&-2&1\\0&1&-3\end{array}\right). $ We already know the eigenvalues are $-3, -1, -4$ with corresponding eigenvectors $ \;{\v}_1=\mat{r}{1\\ 0\\ -1},\ \ {\v}_2=\mat{c}{1\\ 2\\ 1},\ \ {\v}_3=\mat{r}{1\\ -1\\ 1}. $
Let's check the orthogonality: $\v_i \pd \v_j = 0$ for $i\neq j$.
We know from the previous result 18.2.1 that as the eigenvalues are distinct, the eigenvectors $\v_1, \v_2,$ and $\v_3$ are mutually orthogonal.
Are these vectors unitary? 🤔 We must normalize them! So we need:
$\norm{\v_1} $ $=\sqrt{1^2+0^2+(-1)^2}$ $=\sqrt{2},$ $\quad$ $\norm{\v_2}$ $=\sqrt{1^2+2^2+1^2}$ $=\sqrt{6},$
$\norm{\v_3} $ $=\sqrt{1^2+(-1)^2+1^2}$ $=\sqrt{3}.$
Now we normalize the eigenvectors to unit vectors: $\hat{\v}_i =\dfrac{\v_i}{\norm{\v_i}}$
Let $A=\left(\begin{array}{rrr}-3&1&0\\1&-2&1\\0&1&-3\end{array}\right). $ We already know the eigenvalues are $-3, -1, -4$ with corresponding eigenvectors $\; {\v}_1=\mat{r}{1\\ 0\\ -1},\ \ {\v}_2=\mat{c}{1\\ 2\\ 1},\ \ {\v}_3=\mat{r}{1\\ -1\\ 1}. $
$\norm{\v_1} $ $=\sqrt{1^2+0^2+1^2}$ $=\sqrt{2},$ $\quad$ $\norm{\v_2}$ $=\sqrt{1^2+2^2+1^2}$ $=\sqrt{6},$
$\norm{\v_3} $ $=\sqrt{1^2+(-1)^2+1^2}$ $=\sqrt{3}.$
Now we normalize the eigenvectors to unit vectors: $\hat{\v}_i =\dfrac{\v_i}{\norm{\v_i}}$
Then the unitary vectors are \[ \hat{\v}_1=\frac{1}{\sqrt{2}} \mat{r}{1\\ 0\\ -1},\ \ \hat{\v}_2=\frac{1}{\sqrt{6}} \mat{c}{1\\ 2\\ 1},\ \ \hat{\v}_3=\frac{1}{\sqrt{3}} \mat{r}{1\\ -1\\ 1}. \]
Thus the set $\left\{\hat{\v}_1, \hat{\v}_2, \hat{\v}_3\right\}$ is orthonormal.
Let $A=\left(\begin{array}{rrr}-3&1&0\\1&-2&1\\0&1&-3\end{array}\right). $ We already know the eigenvalues are $-3, -1, -4$ with corresponding eigenvectors $\; {\v}_1=\mat{r}{1\\ 0\\ -1},\ \ {\v}_2=\mat{c}{1\\ 2\\ 1},\ \ {\v}_3=\mat{r}{1\\ -1\\ 1}. $
Now we can form the orthogonal matrix $P$:
$\ds P = \big( \hat{\v}_1 ~| ~\hat{\v}_2 ~| ~\hat{\v}_3\big)$ $= \ds \left( \begin{array}{rrr} \dfrac{1}{\sqrt{2}} & \dfrac{1}{\sqrt{6}} & \dfrac{1}{\sqrt{3}}\\ 0 & \dfrac{2}{\sqrt{6}} & -\dfrac{1}{\sqrt{3}}\\ -\dfrac{1}{\sqrt{2}} & \dfrac{1}{\sqrt{6}} & \dfrac{1}{\sqrt{3}} \end{array} \right). $
It orthogonally diagonalises $A$, that is \[ P^TAP = \left(\begin{array}{rrr}-3&0&0\\0&-1&0\\0&0&-4\end{array}\right). \]