IndianTechnoEra: Discrete Mathematics For AIML - Chapter 8 Probability in Discrete Structures

Chapter 8 — Probability in Discrete Structures

This chapter covers probability concepts applied to discrete structures. Understanding discrete probability is essential for modeling uncertainty in AI/ML, especially in probabilistic graphical models and Bayesian networks.

8.1 Discrete Probability Spaces

A discrete probability space consists of a finite or countable sample space S, where each outcome w ∈ S has a probability P(w). AI/ML Context: Representing uncertainty in feature occurrence, classification outcomes, or event probabilities.

# Python example using NumPy
import numpy as np

# Coin toss: H=0, T=1
outcomes = [0, 1]
probabilities = [0.5, 0.5]

# Simulate 10 tosses
samples = np.random.choice(outcomes, size=10, p=probabilities)
print(samples)

8.2 Conditional Probability & Bayes’ Theorem

Conditional probability measures the likelihood of an event given another event has occurred: P(A|B) = P(A∩B)/P(B). Law of Total Probability: P(A) = Σ P(A|Bi)P(Bi) for partition {Bi}. Bayes’ Theorem: P(A|B) = P(B|A)P(A)/P(B)

AI/ML Context: Fundamental for Naive Bayes classifiers, spam detection, and probabilistic reasoning.

# Naive Bayes example
# P(Spam|Contains "Offer") = P("Offer"|Spam)*P(Spam)/P("Offer")
P_offer_given_spam = 0.8
P_spam = 0.3
P_offer = 0.5

P_spam_given_offer = (P_offer_given_spam * P_spam) / P_offer
print(P_spam_given_offer)

8.3 Random Variables (Discrete)

A discrete random variable maps outcomes to numerical values. PMF (Probability Mass Function): P(X=x) gives the probability of each outcome.

# Discrete random variable example
X = [0, 1, 2]           # possible values
pmf = [0.2, 0.5, 0.3]   # probabilities

# Expected value
E_X = sum(x*p for x,p in zip(X, pmf))
print("Expected value:", E_X)

8.4 Expectation & Variance

Expectation: E[X] = Σ x * P(X=x)
Variance: Var(X) = E[(X-E[X])²]
Standard Deviation: sqrt(Var(X))

AI/ML Context: Feature normalization, probabilistic predictions, risk estimation.

# Variance calculation
mean_X = E_X
Var_X = sum((x - mean_X)**2 * p for x,p in zip(X, pmf))
print("Variance:", Var_X)

8.5 ML Use Cases

• Probabilistic graphical models: encode dependencies between discrete variables.
• Naive Bayes classifiers for spam detection and text classification.
• Bayesian networks for decision-making and predictive modeling.
• Modeling discrete events in reinforcement learning or simulations.

8.6 Practical Python Implementation

import numpy as np

# Define discrete events and probabilities
events = ["Sunny", "Rainy", "Cloudy"]
prob = [0.6, 0.2, 0.2]

# Simulate weather for 7 days
sim_weather = np.random.choice(events, size=7, p=prob)
print(sim_weather)

# Expected value example: number of sunny days
X = np.array([0,1,2,3,4,5,6,7])
pmf_sunny = [0.0,0.1,0.2,0.3,0.2,0.1,0.05,0.05]
E_sunny = sum(x*p for x,p in zip(X, pmf_sunny))
print("Expected sunny days:", E_sunny)

8.7 Exercises

1. Simulate rolling a fair six-sided die 20 times and compute the PMF.
2. Compute the expected value and variance of the number of heads in 10 coin tosses.
3. Implement a small Naive Bayes classifier for a toy dataset.

Hints / Solutions

1. Use np.random.choice() with probabilities.
2. Use formula E[X] = Σ x*P(X=x) and Var(X) = Σ (x-E[X])²*P(X=x).
3. Count feature occurrences per class and apply Bayes’ theorem.

8.8 Further Reading & Videos

• Discrete Probability Theory (MIT OpenCourseWare)
• Python: NumPy and SciPy statistics modules
• Probabilistic Graphical Models by Daphne Koller
• YouTube: StatQuest — Bayes’ theorem and discrete probability

Next Chapter: Logic & Propositional Calculus — study propositions, truth tables, logical connectives, and rule-based AI applications.