Calculus For AI/ML - Chapter 5 Integration

Chapter 5 — Integration

Integration is the reverse process of differentiation. In ML, it is used in probability, expected values, and in solving continuous optimization problems.

5.1 What is Integration?

Integration is the process of finding the area under a curve of a function. If f(x) represents a rate of change, integration gives the accumulated value.

Importance in ML: Integration is used to compute probabilities, expected values, and in probabilistic models where continuous distributions are involved.

5.2 Indefinite vs Definite Integrals

Indefinite Integral: Gives the general antiderivative of a function:
```
∫ f(x) dx = F(x) + C
```
Example: ∫ 2x dx = x² + C
Definite Integral: Computes the area under the curve between two limits a and b:
```
∫[a to b] f(x) dx = F(b) - F(a)
```
Example: ∫[0 to 2] 2x dx = (2²) - (0²) = 4

5.3 Techniques of Integration

Substitution: Useful when an integral contains a composite function. Let u = g(x), then integrate w.r.t u.
```
∫ 2x * cos(x²) dx, let u = x² → du = 2x dx
∫ cos(u) du = sin(u) + C = sin(x²) + C
```

Integration by Parts: Based on product rule for derivatives. Formula:

∫ u dv = u*v - ∫ v du

Example:

∫ x e^x dx, u = x → du = dx, dv = e^x dx → v = e^x
∫ x e^x dx = x e^x - ∫ e^x dx = x e^x - e^x + C

5.4 Applications in Machine Learning

Probability distributions: Compute total probability, cumulative distribution functions.
Expected values: E[X] = ∫ x * p(x) dx for continuous random variables.
Loss functions: Integration may be used in continuous loss expectations.

5.5 Quick Python Examples

Using SymPy to compute integrals:

from sympy import symbols, integrate, exp, sin

x = symbols('x')

# Indefinite integral
f = 2*x
indef = integrate(f, x)
print("Indefinite Integral:", indef)  # x^2

# Definite integral
defint = integrate(f, (x, 0, 2))
print("Definite Integral:", defint)  # 4

# Substitution example: ∫ 2x*cos(x^2) dx
expr = 2*x * cos(x**2)
res = integrate(expr, x)
print("Substitution Integral:", res)  # sin(x^2)

# Integration by parts example: ∫ x * exp(x) dx
expr2 = x * exp(x)
res2 = integrate(expr2, x)
print("Integration by Parts:", res2)  # x*exp(x) - exp(x)

5.6 Exercises

Compute ∫ (3x² + 2x + 1) dx (indefinite).
Compute ∫[0 to 1] e^x dx (definite).
Use substitution to solve ∫ x * sin(x²) dx.
Use integration by parts to compute ∫ ln(x) dx.

Answers / Hints

∫ (3x² + 2x + 1) dx = x³ + x² + x + C
∫[0 to 1] e^x dx = e - 1
Let u = x² → du = 2x dx → adjust constants, then integrate sin(u)
∫ ln(x) dx = x ln(x) - x + C (by parts, u = ln(x), dv = dx)

5.7 Practice Projects / Mini Tasks

Compute expected value for a continuous probability distribution (e.g., uniform, normal).
Visualize ∫ f(x) dx for simple functions using Python and matplotlib.
Compare numerical integration (Simpson/Trapezoid) vs symbolic integration using SymPy.

5.8 Further Reading & Videos

Khan Academy — Integrals and Techniques of Integration
MIT OCW — Single and Multivariable Calculus
SymPy Documentation — integrate() function
Probability & Expected Values in ML — Deep Learning Book by Goodfellow et al.

Next chapter: Multivariable Integration & Multiple Integrals — we'll explore integrals over 2D and 3D domains, essential for probabilistic modeling in machine learning.

IndianTechnoEra