Linear Algebra For AIML - Chapter 2 Matrices

Chapter 2 — Matrices

Matrices are at the heart of linear algebra and form the backbone of almost every AI and Machine Learning algorithm. This chapter explains matrices from first principles, builds intuition with examples, and shows practical use cases with Python (NumPy).

2.1 What is a Matrix?

A matrix is a 2D array of numbers arranged in rows and columns. It is denoted by a capital letter (e.g., A, B).

Example:

A = [ 1   2   3
      4   5   6 ]

This is a 2 × 3 matrix (2 rows, 3 columns).

2.2 Matrix Notation, Shape, Rows vs. Columns

- The shape of a matrix is written as m × n, meaning m rows and n columns.
- A row is a horizontal line of elements.
- A column is a vertical line of elements.

B = [ 7   8
      9  10
     11  12 ]   → Shape = 3 × 2

2.3 Matrix Addition & Subtraction

Matrices of the same shape can be added or subtracted element-wise.

A = [ 1  2
      3  4 ]

B = [ 4  5
      6  7 ]

A + B = [ 5   7
          9  11 ]

2.4 Scalar Multiplication

Multiply every element of the matrix by a scalar.

2 × [ 1  2
      3  4 ] = [ 2  4
                 6  8 ]

2.5 Matrix Multiplication (VERY Important)

This is the most important operation in AI/ML. Rule: If A is m × n and B is n × p, then A × B is m × p.

If A is m × n and B is n × p, then A × B is an m × p matrix.

A = [ 1  2
      3  4
      5  6 ]  (3 × 2)

B = [ 7   8   9
     10  11  12 ]  (2 × 3)

Result: A × B = (3 × 3 matrix)

Each entry is the dot product of a row of A with a column of B.

In practice, matrix multiplication allows us to apply linear transformations and propagate data through neural networks.

2.6 Transpose of a Matrix

The transpose of a matrix (denoted Aᵀ) flips its rows into columns.

A = [ 1  2  3
      4  5  6 ]

Aᵀ = [ 1  4
       2  5
       3  6 ]

2.7 Quick NumPy Examples (Practical)

Let’s try the above operations in Python using NumPy:

import numpy as np

# create matrices
A = np.array([[1, 2], [3, 4]])
B = np.array([[4, 5], [6, 7]])

# addition
print("A + B =", A + B)

# scalar multiplication
print("2 * A =", 2 * A)

# matrix multiplication
print("A @ B =", A @ B)   # "@" is matrix multiply

# transpose
print("A.T =", A.T)

2.8 Visual Intuition

- Think of a matrix as a table of numbers. - Multiplying a matrix by a vector applies a linear transformation (rotation, scaling, etc.). - Matrix multiplication is like "feeding data through layers" in a neural network.

2.9 AI/ML Use Cases (Why Matrices Matter)

Neural Networks: Inputs (matrices) × Weights (matrices) = Outputs (activations).
Images: Stored as matrices of pixel values. Color images are 3D tensors (height × width × channels).
Word Embeddings: Stored as embedding matrices, where each row is a word vector.
Transformers: Attention scores are computed using matrix multiplications of query, key, and value matrices.

2.10 Common Pitfalls & Tips

Matrix multiplication is not commutative (A×B ≠ B×A in general).
Always check matrix shapes before multiplication. Example: (m × n) × (n × p) → (m × p).
Transpose is often used to align shapes when multiplying.

2.11 Exercises

Compute C = A × B where A = [[1,2],[3,4]], B = [[2,0],[1,2]].
Find the transpose of [[1,3,5],[2,4,6]].
Explain why matrix multiplication is essential in neural networks.
Use NumPy to create a 3×3 random matrix and compute its transpose.

Answers / Hints

A × B = [[1*2+2*1, 1*0+2*2], [3*2+4*1, 3*0+4*2]] = [[4,4],[10,8]].
Transpose = [[1,2],[3,4],[5,6]].
Neural networks use matrices to store weights; multiplying inputs × weights computes outputs for each layer.

Example in NumPy:


M = np.random.randint(0,10,(3,3))
print(M)
print(M.T)

2.12 Practice Projects / Mini Tasks

Load a grayscale image using Python (PIL or OpenCV) and represent it as a NumPy matrix.
Implement a small neural network forward pass using only NumPy matrix multiplication.
Visualize how a matrix can rotate 2D points (use a rotation matrix).

2.13 Further Reading & Videos

3Blue1Brown — Essence of Linear Algebra (YouTube series) — excellent visuals.
NumPy documentation — for matrix and linear algebra functions.
Any standard Linear Algebra text (e.g., Gilbert Strang) — chapters on matrices and transformations.

Next chapter: Systems of Linear Equations — we'll learn how matrices help solve multiple equations at once, a key concept for optimization in ML.

IndianTechnoEra