Linear Algebra · Book or PDF

A First Course in Linear Algebra

A.G. Hamilton · Study material generated by AI from this edition. Not affiliated with the publisher.

Chapter 3 · pp. 32–41

Matrices and Algebraic Vectors

A matrix is a rectangular array of numbers. Arrays like this were first used as shorthand in Gaussian elimination, and this chapter turns them into objects with their own arithmetic. You will learn how to add, scale and multiply matrices, how a whole system of equations becomes one short equation, and why matrix multiplication breaks one familiar rule of number arithmetic.

Matrices and vectors: the basic vocabulary

A p×q matrix has p rows and q columns, and its numbers are called entries. The entry in row i and column j is the (i,j)-entry, written $a_{ij}$ . The first subscript is the row and the second is the column, so $a_{23}$ sits in row 2, column 3. The notation $[a_{ij}]_{p\times q}$ stands for the p×q matrix whose (i,j)-entry is $a_{ij}$ for every i and j. (p. 32)

Two special shapes come up constantly.

A row vector has only one row, such as $(4\ \ 7\ \ 9)$ , which is 1×3.
A column vector has only one column. A column vector with p entries is called a p-vector, which is a p×1 matrix. Entries 4, 7, 9 stacked in a column form a 3-vector.

Column vectors are the standard way to write the unknowns of simultaneous equations. (p. 32)

Adding, subtracting and scaling

Matrices are added or subtracted entry by entry, so they must be the same size. If A and B are both p×q, then A+B has (i,j)-entry $a_{ij}+b_{ij}$ , and A−B has (i,j)-entry $a_{ij}-b_{ij}$ . A 2×3 matrix and a 3×2 matrix cannot be added, because some entries would have no partner. The same rule holds for row and column vectors. (p. 32)

To multiply a matrix by a number k, multiply every entry by k. The (i,j)-entry of kA is $ka_{ij}$ . This idea comes from repeated addition: A+A doubles every entry, so A+A = 2A. The number k can be any real number, not just a whole number. For example, $\tfrac12$ times a matrix with entries 4 and 6 gives a matrix with entries 2 and 3. (p. 34)

Multiplying a matrix by a column vector

To multiply a p×q matrix A by a column vector with q entries, build each entry of the result from one row of A. Multiply that row's entries by the matching entries of the vector, then add. For example:

$\begin{pmatrix}1&2\\3&4\end{pmatrix}\begin{pmatrix}5\\6\end{pmatrix}=\begin{pmatrix}1\cdot5+2\cdot6\\3\cdot5+4\cdot6\end{pmatrix}=\begin{pmatrix}17\\39\end{pmatrix}$

The matrix must have as many columns as the vector has entries. The answer is a column vector with p entries, one for each row of the matrix. (pp. 33–34)

Writing equations as $A\mathbf{x}=\mathbf{h}$

The row-by-row rule is designed so that the product gives the left-hand sides of equations. That means any set of p linear equations in q unknowns can be written as a single matrix equation $A\mathbf{x}=\mathbf{h}$ .

A is the p×q matrix of coefficients.
$\mathbf{x}$ is the q-vector of unknowns.
$\mathbf{h}$ is the p-vector of right-hand sides.

For example, the equations $3x_1-2x_2=1$ and $4x_1+x_2=-2$ become

$\begin{pmatrix}3&-2\\4&1\end{pmatrix}\begin{pmatrix}x_1\\x_2\end{pmatrix}=\begin{pmatrix}1\\-2\end{pmatrix}.$

Multiplying out the left side gives $3x_1-2x_2$ in the first row and $4x_1+x_2$ in the second. This works no matter how many equations or unknowns there are. (pp. 33–34, 36)

Multiplying two matrices

If A is p×q and B is q×r, the product AB is a p×r matrix. Its (i,j)-entry is

$\sum_{k=1}^{q} a_{ik}b_{kj},$

the sum of products of row i of A with the matching entries of column j of B. Another way to see it: each column of AB is A times the matching column of B. This is just the matrix-times-vector rule applied once per column. (pp. 35–36)

Example: let $A=\begin{pmatrix}1&2&0\\3&1&4\end{pmatrix}$ (2×3) and $B=\begin{pmatrix}2&1\\0&1\\5&2\end{pmatrix}$ (3×2). Then AB is 2×2:

$AB=\begin{pmatrix}1\cdot2+2\cdot0+0\cdot5 & 1\cdot1+2\cdot1+0\cdot2\\ 3\cdot2+1\cdot0+4\cdot5 & 3\cdot1+1\cdot1+4\cdot2\end{pmatrix}=\begin{pmatrix}2&3\\26&12\end{pmatrix}.$

There are 4 entries, each a sum of 3 products.

When a product exists

AB exists only when the number of columns of A equals the number of rows of B. A helpful picture is dominoes. A p,q domino can sit next to a q,r domino because the middle numbers match, and the outer numbers p and r are what remain. (p. 36)

Because of this, AB may exist while BA does not. A 2×3 matrix times a 3×4 matrix works and gives a 2×4 matrix. In the reverse order, the 4 columns of the first matrix do not match the 2 rows of the second, so the product fails. (p. 36)

Rules that do hold

Whenever the sizes allow the sums and products, these laws are true:

$(AB)C=A(BC)$ (associativity)
$A(B+C)=AB+AC$ (distributivity; the same works on the other side)
$(kA)B=k(AB)$

They let you regroup and expand products much as you do with numbers. The rules about scaling are easy to check entry by entry. Associativity and distributivity need longer proofs. For distributivity, the key step is

$\sum_k a_{ik}(b_{kj}+c_{kj})=\sum_k a_{ik}b_{kj}+\sum_k a_{ik}c_{kj},$

which shows that the (i,j)-entry of $A(B+C)$ equals the sum of the (i,j)-entries of AB and AC. (pp. 36–38)

A rule that fails: AB is not always BA

Matrix multiplication is not commutative. Even when both AB and BA exist, they usually differ. Take $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$ and $B=\begin{pmatrix}1&0\\1&1\end{pmatrix}$ . Then

$AB=\begin{pmatrix}2&1\\1&1\end{pmatrix},\qquad BA=\begin{pmatrix}1&1\\1&2\end{pmatrix}.$

This changes how you expand brackets. Distributivity gives

$(A+B)(A+B)=A^2+AB+BA+B^2.$

You cannot merge the middle terms into $2AB$ unless AB = BA. So the school formula $(A+B)^2=A^2+2AB+B^2$ is not valid for matrices in general. (pp. 37–38)

Zero and identity matrices, diagonal and triangular shapes

A zero matrix has every entry 0. Multiplying it by any matrix of suitable size, on either side, gives a zero matrix. It acts like the number 0.

The identity matrix I has 1s on the main diagonal (top left to bottom right) and 0s everywhere else. Multiplying a matrix by I of suitable size, on either side, leaves it unchanged. It acts like the number 1. For instance,

$\begin{pmatrix}1&0\\0&1\end{pmatrix}\begin{pmatrix}a&b\\c&d\end{pmatrix}=\begin{pmatrix}a&b\\c&d\end{pmatrix}.$

Two more shapes are worth knowing. A diagonal matrix has nonzero entries only on the main diagonal. A triangular matrix has zeros on one side of the main diagonal: all zeros below it for upper triangular, all zeros above it for lower triangular. (p. 41)

Check yourself

In the matrix $[a_{ij}]$ , which entry is $a_{32}$ , and what do the two subscripts mean?
Can a 3×2 matrix be added to a 2×3 matrix? Can the two be multiplied in either order? Explain.
Write the equations $x_1+2x_2=5$ and $3x_1-x_2=4$ in the form $A\mathbf{x}=\mathbf{h}$ .
Find $\begin{pmatrix}2&0\\1&3\end{pmatrix}\begin{pmatrix}4\\-1\end{pmatrix}$ .
Why does $(A+B)^2$ not simplify to $A^2+2AB+B^2$ for matrices in general?

Make a test on this chapter Learn this shelf

A First Course in Linear Algebra

Matrices and Algebraic Vectors

Matrices and vectors: the basic vocabulary

Adding, subtracting and scaling

Multiplying a matrix by a column vector

Writing equations as Ax=hA\mathbf{x}=\mathbf{h}Ax=h

Multiplying two matrices

When a product exists

Rules that do hold

A rule that fails: AB is not always BA

Zero and identity matrices, diagonal and triangular shapes

Check yourself

Matrices and Algebraic Vectors

Matrices and vectors: the basic vocabulary

Adding, subtracting and scaling

Multiplying a matrix by a column vector

Writing equations as Ax=hA\mathbf{x}=\mathbf{h}Ax=h

Multiplying two matrices

When a product exists

Rules that do hold

A rule that fails: AB is not always BA

Zero and identity matrices, diagonal and triangular shapes

Check yourself

Writing equations as $A\mathbf{x}=\mathbf{h}$

Writing equations as $A\mathbf{x}=\mathbf{h}$