Linear Algebra For AIML - Chapter 4 Determinants

Chapter 4 — Determinants

Determinants provide a scalar summary of a square matrix. They are crucial in linear algebra for checking invertibility, understanding transformations, and computing eigenvalues — all important in AI & ML applications like PCA and linear regression.

4.1 What is a Determinant?

The determinant of a square matrix is a scalar that encodes important properties of the matrix. It can be thought of as a scaling factor for the linear transformation represented by the matrix.

Example (2x2):

A = [[a, b],
     [c, d]]
det(A) = ad - bc

For 3x3 matrices:

B = [[a,b,c],
     [d,e,f],
     [g,h,i]]
det(B) = a(ei − fh) − b(di − fg) + c(dh − eg)

4.2 Properties of Determinants

det(I) = 1 for the identity matrix I.
det(Aᵀ) = det(A).
det(A × B) = det(A) × det(B).
Swapping two rows changes the sign of the determinant.
If a row (or column) is multiplied by a scalar, the determinant is multiplied by that scalar.
det(A) = 0 → matrix is singular (non-invertible).

Importance in AI/ML: Determinants help check if matrices can be inverted (needed in linear regression and least squares). They also appear in eigenvalue computations for PCA and other dimensionality reduction techniques.

4.3 Geometric Interpretation

- In 2D: |det(A)| gives the area scaling factor of the transformation. If det(A) = 2, areas double under the transformation.
- In 3D: |det(A)| gives the volume scaling factor.

Example: Transform a unit square with matrix [[2,0],[0,3]], the area becomes 2*3 = 6 → det(A) = 6.

4.4 Quick NumPy Examples (Practical)

import numpy as np

# 2x2 determinant
A = np.array([[2, 3], [1, 4]])
det_A = np.linalg.det(A)
print("det(A) =", det_A)

# 3x3 determinant
B = np.array([[1,2,3],[0,4,5],[1,0,6]])
det_B = np.linalg.det(B)
print("det(B) =", det_B)

# Check invertibility
if det_B != 0:
    print("Matrix B is invertible")
else:
    print("Matrix B is singular")

4.5 AI/ML Use Cases (Why Determinants Matter)

Checking if weight matrices are invertible in linear regression or least squares solutions.
Understanding volume scaling in transformations (useful in feature engineering).
Eigenvalues & eigenvectors: det(A - λI) = 0 is the characteristic equation used in PCA.

4.6 Exercises

Compute the determinant of [[3,2],[1,4]] by hand and verify with NumPy.
Check if [[1,2,3],[4,5,6],[7,8,9]] is invertible using determinant.
Transform a 2x2 square using [[2,0],[0,3]] and compute area scaling using determinant.
Verify det(A × B) = det(A) × det(B) for A=[[1,2],[3,4]] and B=[[2,0],[1,3]].

Answers / Hints

det([[3,2],[1,4]]) = 3*4 - 2*1 = 10
det([[1,2,3],[4,5,6],[7,8,9]]) = 0 → singular matrix
Area scaling = 2*3 = 6
Compute det(A) = -2, det(B) = 6 → det(A)×det(B) = -12, det(A×B) = -12 (verify with NumPy)

4.7 Practice Projects / Mini Tasks

Given a dataset matrix, compute the determinant of its covariance matrix to understand singularity issues.
Experiment with small 2D transformations and visualize area scaling using matplotlib.
Implement a function that checks if a matrix is invertible before using it in regression computations.

4.8 Further Reading & Videos

3Blue1Brown — Essence of Linear Algebra (YouTube series).
NumPy documentation: np.linalg.det
Gilbert Strang — Linear Algebra chapters on determinants.

Next chapter: Rank of a Matrix — learn how to determine rank, understand linear independence, and why rank matters in ML models.

IndianTechnoEra