Probability Distributions — Discrete and Continuous

This lesson on Probability Distributions — Discrete and Continuous is hands-on and example-driven. You will be able to define, visualize, and interpret statistical distributions using histograms and continuous curves. This allows you to assess the probability of various outcomes and model the likelihood structure of any dataset.

What You'll Be Able To Do

• Define a statistical distribution and its purpose in modeling likelihood.
• Construct a histogram by grouping continuous measurements into discrete bins.
• Transform discrete frequency counts into a continuous probability density model.
• Interpret the meaning of peaks and tails in a distribution curve.
• Calculate relative likelihoods based on the visual structure of a distribution.

Detailed Concept Walkthrough

1. Organizing Data for Likelihood

A statistical distribution organizes raw data measurements into groups to show the frequency or likelihood of different outcomes occurring. It transforms raw observations into a model of probability.

• Mechanism: Data collection involves measuring observations (e.g., height). To form a distribution, these continuous measurements must be grouped into discrete intervals called "bins" to make them countable.
• Under the Hood: The distribution answers the question: "If I select a random observation, what is the chance it falls within this specific range (bin)?" The total area under the distribution must always equal 1, representing 100% of the probability.
• Best Practice: Always ensure bins are mutually exclusive and collectively exhaustive; every observation must fall into exactly one bin to prevent double counting or missing data points in the final model.

Key Takeaway: Distributions model the likelihood of outcomes by grouping measurements into defined, countable ranges.

2. Constructing and Interpreting Histograms

The histogram is the initial visual representation of a statistical distribution, using vertical bars (bins) whose height represents the frequency or relative probability of data falling into that range.

• Mechanism: Data is sorted into predefined bins, and the count (frequency) for each bin determines the height of the corresponding bar. Taller bars indicate a higher likelihood of an observation falling into that range.
• Under the Hood: Histograms are descriptive statistics; they show the actual frequency counts from the sample data collected. The relative probability is calculated by dividing the bin frequency by the total number of observations.
• Best Practice: The choice of bin size significantly impacts the histogram's appearance; too few bins hide important detail, while too many bins result in a sparse, noisy visualization with many empty bars.

import pandas as pd
import matplotlib.pyplot as plt

# Assume 'heights' is a list of continuous measurements
data = {'heights': [5.5, 6.1, 5.8, 5.2, 6.5, 5.9, 5.7, 5.6, 6.0, 5.4]}
df = pd.DataFrame(data)

# Constructing a histogram with 5 bins
plt.hist(df['heights'], bins=5, edgecolor='black')
plt.title('Height Distribution Histogram')
plt.xlabel('Height (feet)')
plt.ylabel('Frequency Count')
plt.show()

Key Takeaway: Histograms map frequency counts to visual bars, providing an initial estimate of the underlying probability structure.

3. Refining Distributions and the PDF

By increasing the sample size and decreasing the bin width, the histogram approaches a smooth, continuous curve known as the Probability Density Function (PDF), which models the theoretical distribution.

• Mechanism: As bins become infinitesimally small, the histogram bars merge into a smooth curve representing probability density, not frequency count, allowing for precise probability calculations for any range.
• Under the Hood: The PDF is an inferential model, often approximated using parameters like the mean and standard deviation. Calculating the probability for a range (P(a < X < b)) requires finding the area under the curve between points $a$ and $b$ (integration).
• Best Practice: The continuous curve is essential for modeling continuous variables (like time or height) because the probability of observing exactly one specific value is zero; probability must always be calculated over a range.

from scipy.stats import norm
import numpy as np

# Generate points for a theoretical Normal Distribution (PDF)
x = np.linspace(4, 7, 100)
# Use mean=5.8, std dev=0.3 to model the curve
plt.plot(x, norm.pdf(x, loc=5.8, scale=0.3), label='Continuous PDF')
plt.title('Probability Density Function Model')
plt.show()

Key Takeaway: The continuous PDF curve models precise probability density, overcoming the limitations of discrete binning, especially for continuous data.

4. Interpreting Distribution Shapes

The shape of the distribution (histogram or curve) reveals how probability is spread across the possible outcomes, indicating which outcomes are most and least likely.

• Mechanism: Peaks (modes) in the distribution indicate the most frequently observed or most likely outcomes. The height of the curve or bar directly correlates with the likelihood of an observation occurring at that point or range.
• Under the Hood: The tails of the distribution represent extreme values or outliers. Observations falling far out in the tails have a very low probability density, meaning they are rare events.
• Best Practice: When interpreting a distribution, focus on symmetry, skewness (the direction the tail extends), and modality (number of peaks) to understand the central tendency and variability of the data.

Key Takeaway: Peaks show high likelihood (common outcomes), while tails show low likelihood (rare outcomes).

Topics Covered in Probability Distributions — Discrete and Continuous

1. Defining Distribution (00:18 - 01:07) — A statistical distribution organizes data to understand the likelihood of various outcomes.
2. Constructing Histograms (01:08 - 01:38) — Histograms visualize binned data where bar height shows the relative probability of a range.
3. Refining the Estimate (01:38 - 02:22) — Decreasing bin size and increasing sample size refines the histogram into a smooth curve.
4. Advantages of the Curve (02:22 - 03:14) — The continuous curve (PDF) allows precise probability calculation for any range using the area under the curve.
5. Interpreting Shapes (03:14 - 03:39) — Peaks indicate the most likely outcomes, while tails represent the least likely, extreme values.

Statistics for Data Science Cheat Sheet

• Statistical Distribution — Model showing how probability is spread across outcomes

• Histogram — Visual representation of binned frequency counts

  plt.hist(data, bins=10)
  

• Bin — Discrete interval used to group continuous data

  bins = [5.0, 5.5, 6.0, 6.5]
  

• Probability Density Function (PDF) — Continuous curve modeling precise probability density

  from scipy.stats import norm
  

• Area Under the Curve — Represents the probability for a specific range of outcomes

  P(a < X < b)
  

• Sample Size — Increasing this refines the distribution estimate

  len(observations) > 1000
  

Comparison Table

Common Pitfalls

• Mistake: Assuming histogram bar height is the exact probability. Avoid: Bar height is frequency; probability is relative frequency (count/total).
• Mistake: Using too few or too many bins in a histogram. Avoid: Choose bin size that reveals structure without excessive noise or loss of detail.
• Mistake: Thinking the PDF gives the probability of a single point. Avoid: Probability for a single continuous point is zero; calculate probability over a range.
• Mistake: Confusing the peak of the curve with the mean. Avoid: The peak is the mode (most frequent value); it may differ from the mean in skewed distributions.

FAQs

• Why do we need to bin continuous data? Binning transforms continuous measurements into discrete, countable groups, which is necessary to visualize frequency and define initial likelihoods.
• What is the main advantage of the continuous curve over the histogram? The curve allows for the calculation of precise probabilities for any range, overcoming the limitations and inaccuracies caused by fixed bin boundaries.
• How does increasing the sample size help the distribution estimate? A larger sample size provides a more representative view of the true population, making the observed frequency distribution closer to the theoretical probability distribution.
• What does 'area under the curve' mean practically? The area under the curve between two points represents the cumulative probability that a random observation will fall within that specific range of values.