Linear Algebra · Book or PDF

A First Course in Linear Algebra

A.G. Hamilton · Study material generated by AI from this edition. Not affiliated with the publisher.

Chapter 4 · pp. 42–53

Special Matrices

Some matrices have shapes or properties that make them easy to work with and very useful later on. This chapter introduces zero, identity, diagonal, triangular, symmetric, skew-symmetric, orthogonal and elementary matrices. It also gives the rules for the transpose, including the one that reverses the order of a product.

Zero and identity matrices

A zero matrix has every entry equal to 0. It behaves like the number 0: adding it changes nothing, and multiplying by it gives a zero matrix. There are many zero matrices, one for each size, so always check that the sizes fit the calculation (p. 42).

An identity matrix is a square matrix with 1s on the main diagonal (top left to bottom right) and 0s everywhere else. It plays the role of the number 1. There is one $p \times p$ identity matrix for each $p$ , written $I_p$ , or just $I$ when the size is clear. For example,

$I_2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}.$

(p. 42)

The key property is

$AI = IA = A$

when $A$ is square and $I$ has the same size. Each row of $I$ has a single 1, so when it multiplies a column of $A$ it picks out exactly one entry and leaves $A$ unchanged. (p. 42)

Diagonal and triangular matrices

A diagonal matrix is a square matrix whose entries off the main diagonal are all 0. The diagonal entries themselves may be anything, including 0. So identity matrices, and even square zero matrices, count as diagonal. Adding or multiplying two $p \times p$ diagonal matrices gives another $p \times p$ diagonal matrix. For instance, diag(2, 3) times diag(4, 5) is diag(8, 15). (p. 42)

The main diagonal splits a square matrix into two triangles.

An upper triangular matrix has 0s at every position below the main diagonal.
A lower triangular matrix has 0s at every position above it.

(pp. 42–44)

Sums and products of upper triangular matrices are upper triangular. The same holds for lower triangular matrices. For example,

$\begin{pmatrix} 1 & 2 \\ 0 & 3 \end{pmatrix}\begin{pmatrix} 2 & 1 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 2 & 3 \\ 0 & 3 \end{pmatrix},$

and the answer still has a 0 below the diagonal. (pp. 43–44)

These matrices matter in practice. Applying Gaussian elimination to any square matrix always ends with an upper triangular matrix. (p. 44)

Transpose and symmetric matrices

The transpose $A^T$ is the matrix reflected in its main diagonal. Its $(i,j)$ -entry is $a_{ji}$ , so the rows of $A^T$ are the columns of $A$ , and the columns of $A^T$ are the rows of $A$ . (p. 44)

A square matrix is symmetric if $A^T = A$ , meaning the reflection changes nothing. For example,

$\begin{pmatrix} 1 & 2 \\ 2 & 3 \end{pmatrix}$

is symmetric because the two off-diagonal entries match. (pp. 43–44)

Sums stay symmetric. If $A$ and $B$ are symmetric and the same size, then $(A+B)^T = A^T + B^T = A + B$ . (p. 43)

Common mistake: products are not always symmetric. It is tempting to think that if sums of symmetric matrices are symmetric, products must be too. This is wrong. Using the rule for transposing a product (see below), $(AB)^T = B^T A^T = BA$ . This usually differs from $AB$ . Here is a check with two symmetric matrices:

$A = \begin{pmatrix} 1 & 2 \\ 2 & 3 \end{pmatrix},\quad B = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix},\quad AB = \begin{pmatrix} 2 & 1 \\ 3 & 2 \end{pmatrix}.$

The product has 1 in one off-diagonal position and 3 in the other, so it is not symmetric. (pp. 43–44)

The three transpose rules

For matrices of suitable sizes:

$(A^T)^T = A$
$(A+B)^T = A^T + B^T$
$(AB)^T = B^T A^T$

Reflecting twice returns the original matrix, which explains rule 1. Rule 2 holds because adding entries and then swapping positions gives the same result as swapping first. (p. 46)

Rule 3 is the important one, because the order reverses. Forgetting this is a very common error. The proof compares entries. The $(i,j)$ -entry of $(AB)^T$ is the $(j,i)$ -entry of $AB$ , which is $\sum_k a_{jk} b_{ki}$ . The $(i,j)$ -entry of $B^T A^T$ is $\sum_k (B^T)_{ik}(A^T)_{kj} = \sum_k b_{ki} a_{jk}$ . These sums are equal. (pp. 45–46)

Skew-symmetric and orthogonal matrices

A matrix is skew-symmetric if $A^T = -A$ . Entries opposite each other across the diagonal must have opposite signs. Each diagonal entry must satisfy $a_{ii} = -a_{ii}$ , so it must be 0. For example,

$\begin{pmatrix} 0 & 2 \\ -2 & 0 \end{pmatrix}$

is skew-symmetric. (pp. 45–46)

An orthogonal matrix is a square matrix with $A^T A = I$ and $A A^T = I$ . A simple example is

$A = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 & 1 \\ -1 & 1 \end{pmatrix}.$

Here $A^T A = \tfrac12\begin{pmatrix} 2 & 0 \\ 0 & 2 \end{pmatrix} = I$ . An orthogonal matrix and its transpose are inverses of each other. (pp. 45–46, 48)

Elementary matrices

An elementary matrix is an identity matrix changed by one elementary row operation. The three kinds of operation give three kinds of matrix:

swapping two rows of $I$ ;
multiplying one row of $I$ by a nonzero number (for example, row 3 by 5);
adding a multiple of one row of $I$ to another (for example, 3 times row 2 added to row 1).

(p. 47)

The main fact is that multiplying $A$ on the left by an elementary matrix $E$ does the same thing as applying that row operation directly to $A$ . Take $E$ as the identity with its first two rows swapped. Then $EA$ is $A$ with its first two rows swapped. If $A$ has rows (1, 2, 3), (4, 5, 6), (7, 8, 9), then $EA$ has rows (4, 5, 6), (1, 2, 3), (7, 8, 9). (pp. 47–48)

This means Gaussian elimination can be written as matrix multiplication. For any square matrix $A$ there are elementary matrices $E_1, E_2, \dots, E_r$ , one for each row operation used, such that

$E_r \cdots E_2 E_1 A$

is upper triangular. $E_1$ is applied first, so it sits next to $A$ . (pp. 48–49)

Every elementary matrix $E$ has a partner $F$ with $FE = EF = I$ , because a second row operation can undo the first. The two matrices are inverses of each other. For example:

A row swap is its own inverse.
Multiplying row 3 by 5 is undone by multiplying row 3 by $\tfrac15$ .
Adding 3 times row 2 to row 1 is undone by adding $-3$ times row 2 to row 1.

(pp. 48–49)

Check yourself

Why does $AI = A$ hold for a square matrix $A$ and an identity matrix $I$ of the same size?
Give two symmetric $2 \times 2$ matrices whose product is not symmetric, and explain why $(AB)^T = BA$ for symmetric $A$ and $B$ .
Why must every diagonal entry of a skew-symmetric matrix be 0?
Which elementary matrix undoes "add 4 times row 1 to row 3"? What is the product of the two matrices?
If $A$ is orthogonal, what is its inverse?

Make a test on this chapter Learn this shelf