Working definition (not final): a two dimensional array of scalars with one or more columns and one or more rows.
ex. or or are 3x3, 1x3 and 3x1 matrices respectively. This is of the form rows x columns.
Generally, matrices are denoted as:
where and . Indexing can also start with zero. In this form, the matrix is:
Diagonal matrix and Triangular matrix
For a diagonal . In a diagonal matrix the non diagonal elements are zero.
ex.
the zero matrix (a matrix with only zeroes) is also a diagonal matrix.
A triangular matrix is a square matrix (same number of rows and columns) where elements above or below the main diagonal are all zeroes.
- if the elements above the main diagonal are zeroes, it’s a lower triangular matrix. ex.
- if the elements below the main diagonal are zeroes, it’s an upper triangular matrix. ex.
Identity matrix
A diagonal matrix whose all diagonal entries are 1.
ex.
Matrix algebra
Two matrices are said to be equal if their dimensions (rows x columns) are equal and all corresponding elements are equal.
ex.
Thus the two are equal.
Addition and subtraction
Addition or subtraction is only possible when two matrices have the same size or dimensions. Both operations yield a matrix of the same size as the operand matrices. Matrices of different sizes cannot be added or subtracted.
and
Scalar multiplication
Matrices can be multiplied by a scalar (a constant). The constant just gets multiplied to each element of the matrix individually, so:
Matrix multiplication
Product of two matrices is another matrix. The necessary condition for matrix multiplication is that the number of columns in the first matrix must be equal to the number of rows of the second matrix. That is, if two matrices of dimensions m x n and p x q are being multiplied, it must be true that n = p, and the resultant matrix would have dimensions m x q.
ex.
generally, if we start the indexing from 1 and is the number of columns in the first matrix, or the number of rows in the second matrix:
if you’re doing this by hand (why?) I have found this to be a nice aid:
which is just a dot product of vectors.
Some rules
some common pitfalls:
- is not generally equal to as may not be conformable (what is conformability)?
- If neither A nor B may necessarily be equal to 0 (zero matrix).
- If , is not necessarily equal to .
Transpose of a matrix
If a matrix is given as:
Some common rules:
If a matrix and its transpose are equal, i.e. , we say that the matrix is symmetric. It can be observed that it’s always of the form , or we can say that for every ,
A symmetric matrix:
Determinant of a matrix; determinant as a volume
Every square matrix has a determinant which is a scalar number. For a simple 2x2 matrix,
If the determinant is non-zero, we say that the matrix is regular and the inverse exists. i.e. the matrix is invertible.
We can consider a determinant as a mapping from a set of n vectors spanning an object in . The determinant is the signed volume of an n-dimensional parallelpiped formed by columns of the matrix A.
For n=2 the columns of the matrix form a parallelogram. If two vectors b and g form the columns of the matrix , the absolute value of the determinant of A is the area of the parallelogram with the vertices 0, b, g, b+g as shown in the figure below. In particular, if b and g are linearly dependent we can’t even have a parallelogram since .
Sign: The sign of the determinant signals the orientation of the spanning vectors with respect to the standard basis . Flipping the order to g,b instead of b,g swaps the columns of A and reverse the orientation of the shaded area.
The intuition here is that in 2 dimensions, the determinant represents the area spanned by two vectors. This intuition expands to higher dimensions too.
In we can consider three vectors spanning the edge of a parallelpiped, i.e. a solid with faces that are parallel parallelograms. The absolute value of the determinant of a 3x3 matrix is the volume of the solid.
Thus, the determinant acts as a function that measures the signed volume formed by column vectors composed in a matrix.
I asked claude about the sign change since I was confused on what it means, as “direction of the basis vectors” wasn’t that intuitive.
Claude on sign change:
The sign tells you whether the linear transformation preserves or flips orientation — think of it as: does it keep your basis vectors in the same rotational arrangement, or does it mirror them?
Concretely in 2D: take the standard basis e₁=(1,0), e₂=(0,1). Going from e₁ to e₂ is a counterclockwise turn. Now apply a transformation A to both vectors, getting Ae₁ and Ae₂ — these become the new columns of A. Ask: going from Ae₁ to Ae₂, is it still counterclockwise, or has it flipped to clockwise?
- det(A) > 0: orientation preserved — Ae₁ to Ae₂ is still counterclockwise, same rotational sense as the original basis. Rotations, scaling (by positive factors), shears — all orientation-preserving.
- det(A) < 0: orientation flipped — Ae₁ to Ae₂ is now clockwise. This happens with reflections (e.g., flipping across an axis) — the “handedness” of the space gets mirrored.
Trace of a matrix
The trace of a square matrix is defined as:
i.e. it’s the sum of the diagonal elements.
Rank of a matrix
The number of linearly independent columns of a matrix equals the number of linearly independent rows and is called the rank of A and is denoted by .
- The row rank equals the column rank, so
- The columns of span the subspace with . Later this subspace will be called the image or range. A basis of can be found by applying Gaussian elimination to to identify the pivot columns.
- The rows of span a subspace with . A basis can again be found via Gaussian elimination.
- For all it holds that is regular (invertible) if and only if .
- For all and all it holds that the linear equation system can be solved if and only if , where denotes the augmented system. More information in System of linear equations.
- For the subspace of solutions for possesses dimension . Later, we will call this subspace the kernel or the null space.
- A matrix has full rank if its rank equals the largest possible rank for a matrix of the same dimensions. This means that the rank of a full-rank matrix is the lesser of the number of rows and columns; i.e. . A matrix is said to be rank deficient if it does not have a full rank.
Now think about (4) and (7) for a second. If a square matrix () is rank deficient, it means that it’s not invertible. A matrix is only invertible if the determinant is non-zero. This means that square matrices with rank deficiency have zero determinant. We will use this insight in Eigenvalues and Eigenvectors.
Inverse of a matrix
The inverse of a scalar is the reciprocal or division by 1 by the scalar, ex. inverse of k = 7 is 1/k = 1/7.
In matrices division is not defined because there may be even when B and C are not equal.
Instead, the inverse of a square matrix is defined as the unique matrix where:
The inverse only exists when the matrix is non singular, or the determinant is non-zero.
Some properties:
Calculating the inverse
To compute the inverse of we need to find a matrix X that satisfies . Then we can say that .
Practically, with a pen and paper we can first create the augmented matrix and perform operations on it to bring it to reduced row-echelon form. Then we can get the inverse on the right hand side of the equation system. Recall that this is called Gaussian elimination.
ex.
Implementation
see https://numpy.org/doc/stable/reference/routines.linalg.html
using numpy:
import numpy as np
# arrays inside an array
matA = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
matB = np.array([[2, 3, 4], [5, 6, 7], [8, 9, 10]])
print(matA.shape)
print(matB.shape)
print(np.add(matA, matB))
print(np.subtract(matA, matB))
print(np.dot(matA, matB))
# a determinant of 0 means that the matrix is singular.
print(np.linalg.det(matA))
# this causes an error, as for a singular matrix the inverse cannot be found
# print(np.linalg.inv(matA))
matC = np.array([[1, -1], [2, 3]])
# non zero determinant ==> non singular.
print(np.linalg.det(matC))
print(np.linalg.inv(matC))my own naive implementations that I did a while ago for understanding:
class Matrix:
""" This class models a two dimensional matrix """
def __init__(self, rows: int, columns: int):
""" Initialise a R x C matrix with zeroes """
self.matrix = []
self.rows = rows
self.cols = columns
self.dimensions = rows, columns
for i in range(rows):
row = []
for j in range(columns):
row.append(0)
self.matrix.append(row)
def __repr__(self):
""" Prints the matrix """
string = ""
for row in self.matrix:
string += f"{row}\n"
return string.strip()
def user_set(self):
""" Takes user input for each element of the matrix """
self.matrix = []
for i in range(self.rows):
row = []
for j in range(self.cols):
user_input = int(input(f"Element [{i}][{j}]: "))
row.append(user_input)
self.matrix.append(row)
def __add__(self, another: Matrix):
""" Adds two matrices together """
if self.dimensions != another.dimensions:
raise ArithmeticError("The two matrices don't have the same dimensions!")
final_matrix = []
for i in range(self.rows):
row = []
for j in range(self.cols):
row.append(self.matrix[i][j] + another.matrix[i][j])
final_matrix.append(row)
new_matrix: Matrix = Matrix(self.rows, self.cols)
new_matrix.from_list(final_matrix)
return new_matrix
def __sub__(self, another: Matrix):
""" Subtracts the second matrix from the first """
if self.dimensions != another.dimensions:
raise ArithmeticError("The two matrices don't have the same dimensions!")
final_matrix = []
for i in range(self.rows):
row = []
for j in range(self.cols):
row.append(self.matrix[i][j] - another.matrix[i][j])
final_matrix.append(row)
new_matrix: Matrix = Matrix(self.rows, self.cols)
new_matrix.from_list(final_matrix)
return new_matrix
def __mul__(self, another: Matrix | float):
""" Multiplies two matrices together, or a matrix and a constant together """
if type(another) is Matrix:
if self.cols != another.rows:
raise ArithmeticError("For matrix multiplication, the number of columns in the first matrix must be equal to the number of rows in the second matrix.")
final_matrix = []
for i in range(self.rows):
new_row = []
for j in range(another.cols):
# final_matrix[i][j] = 0
element = 0
for k in range(self.cols):
element += self.matrix[i][k] * another.matrix[k][j]
new_row.append(element)
final_matrix.append(new_row)
new_matrix: Matrix = Matrix(self.rows, another.cols)
else:
final_matrix = []
for row in self.matrix:
new_row = []
for element in row:
new_element = another * element
new_row.append(new_element)
final_matrix.append(new_row)
new_matrix: Matrix = Matrix(self.rows, self.cols)
new_matrix.from_list(final_matrix)
return new_matrix
def transpose(self):
""" Returns the transpose of a matrix """
transpose = []
for i in range(self.cols):
new_row = []
for j in range(self.rows):
new_row.append(self.matrix[j][i])
transpose.append(new_row)
new_matrix: Matrix = Matrix(self.cols, self.rows)
new_matrix.from_list(transpose)
return new_matrix
def from_list(self, my_list: list):
""" Returns a Matrix from a list """
self.matrix = []
for row in my_list:
new_row = []
for element in row:
new_row.append(element)
self.matrix.append(row)
return self.matrix
if __name__ == "__main__":
new = Matrix(3, 3)
new.from_list([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
another = Matrix(2, 3)
another.from_list([[3, 2, 1], [2, 3, 6]])
# traversing column wise?
print(another)