Reproducibility + Docker

This lesson on Reproducibility + Docker is hands-on and example-driven. You will learn how to containerize your ML model environment using Docker best practices to guarantee identical execution across development, testing, and production. You will be able to write a robust Dockerfile that passes the critical cold-clone reproducibility check required for MLOps deployment.

What You'll Be Able To Do

• Pin all Python dependencies explicitly using a requirements.txt file generated from a locked environment.
• Structure a Dockerfile to leverage build cache efficiency by placing stable layers first.
• Utilize .dockerignore effectively to minimize build context size and prevent credential leakage.
• Execute a successful cold-clone build check against a new repository clone to verify environment reproducibility.
• Write multi-stage Dockerfiles that separate build-time dependencies from runtime images for security and size optimization.

Detailed Concept Walkthrough

1. Pinned Dependencies for Reproducibility

Reproducibility requires that every dependency, including transitive ones, resolves to the exact same version every time the environment is built. Pinning dependencies prevents unexpected failures caused by upstream library updates or removals.

• Mechanism: Tools like pip freeze capture the specific version number (e.g., numpy==1.24.3) for every package installed in the current environment. This locked list is the only source of truth for dependency installation inside the container.
• Best Practice: Always use pip install -r requirements.txt inside the Dockerfile, ensuring that requirements.txt contains explicitly pinned versions, not just major versions or ranges, to eliminate all non-determinism.
• Execution Flow: When the Docker build runs, the RUN pip install command reads the pinned file, guaranteeing that the Python environment inside the container is bit-for-bit identical to the one used during development and testing.
• Under the Hood: Without pinning, the package manager resolves to the latest compatible version, which changes over time, leading to 'works on my machine' failures when deployed to a new environment.

# requirements.txt must contain pinned versions (e.g., scikit-learn==1.3.0)
COPY requirements.txt /app/requirements.txt

# Install dependencies before copying application code
RUN pip install --no-cache-dir -r /app/requirements.txt

# Copy application code later
COPY . /app

Key Takeaway: Pinned dependencies are the non-negotiable foundation for achieving reliable, reproducible ML deployments.

2. Optimizing Docker Layering and Caching

Docker builds images in layers, caching the result of each instruction; efficient layering minimizes build times and resource consumption by maximizing cache hits. The key is placing the least frequently changing instructions earliest in the Dockerfile.

• Mechanism: Docker compares the instruction and the context files used in a layer against its cache. If both match, the cached layer is reused, skipping execution of that instruction and all subsequent instructions.
• Best Practice: The standard MLOps pattern is: FROM (Base Image) -> COPY requirements.txt -> RUN pip install (Stable Deps) -> COPY application code (Frequent Changes). This isolates the expensive dependency installation step.
• Under the Hood: Changing a file copied in a layer invalidates the cache for that layer and all subsequent layers. By copying only the dependency file first, we ensure that code changes do not force a full dependency reinstallation.
• Syntax Rule: Combine multiple related RUN commands using && and line continuation (\) into a single layer to reduce the total number of layers and improve image efficiency.

FROM python:3.10-slim AS base

# 1. Stable layer: Copy only the dependency list
COPY requirements.txt /app/requirements.txt

# 2. Stable layer: Install dependencies (only runs if requirements.txt changes)
RUN pip install --no-cache-dir -r /app/requirements.txt

# 3. Volatile layer: Copy application code (runs on every code change)
COPY src/ /app/src

Key Takeaway: Place stable, expensive operations (like dependency installation) early in the Dockerfile to maximize build cache utilization.

3. Controlling Context with .dockerignore

The build context is the set of files sent to the Docker daemon during the build process; using a .dockerignore file prevents unnecessary files from being transferred, speeding up builds and preventing sensitive data exposure.

• Mechanism: Before the build starts, the Docker client reads the .dockerignore file and filters the local directory contents, sending only the necessary files (the build context) to the Docker daemon.
• Best Practice: Always exclude development artifacts (.git, __pycache__), local environment files (.venv, venv/), and sensitive data (.env, credentials.json, data/raw/) to minimize transfer size and security risks.
• Syntax Rule: The .dockerignore syntax follows glob patterns similar to .gitignore. Patterns are relative to the root of the build context, and lines starting with ! can re-include files that were previously excluded by a pattern.
• Execution Flow: A large build context slows down the initial transfer phase, especially when building remotely (e.g., on AWS CodeBuild or ECR), even if those files are never explicitly copied into the image.

# .dockerignore file
# Exclude version control and local environment files
.git
.gitignore
.venv
venv/

# Exclude large or sensitive files
*.pyc
__pycache__
data/
credentials.json

Key Takeaway: A comprehensive .dockerignore file is mandatory for fast, secure, and efficient Docker builds in MLOps pipelines.

Topics Covered in Reproducibility + Docker

1. Reproducibility Defined (0:00 - 0:45) — Reproducibility ensures that the model environment is identical across all stages of the MLOps pipeline.
2. The Cold-Clone Check (0:45 - 1:45) — The cold-clone check verifies that a fresh clone of the repository can build and run the container successfully.
3. Pinned Dependencies (1:45 - 3:30) — Explicitly pinning all package versions prevents environment drift and non-deterministic builds.
4. Docker Layering Strategy (3:30 - 5:30) — Structuring the Dockerfile to place stable instructions first maximizes the use of the build cache.
5. Example Dockerfile Pattern (5:30 - 7:00) — The optimal pattern involves copying requirements before application code to isolate dependency installation.
6. Using .dockerignore (7:00 - 8:30) — The .dockerignore file minimizes the build context size and prevents sensitive files from being transferred to the daemon.
7. Review Core Concepts (8:30 - 10:00) — Reviewing the three core components—pinning, layering, and ignoring—required for production-ready containers.

MLOps + Cloud Deploy (AWS-First) Cheat Sheet

• pip freeze > requirements.txt — Locks all installed package versions explicitly

  pip freeze > requirements.txt
  

• COPY requirements.txt /app/ — Transfers dependency list before application code

  COPY requirements.txt /app/
  

• RUN pip install -r — Installs pinned dependencies inside the container

  RUN pip install --no-cache-dir -r /app/reqs.txt
  

• .dockerignore — Excludes files from the Docker build context

  data/
  .git
  *.log
  

• FROM python:3.10-slim — Defines the minimal base image for the runtime

  FROM python:3.10-slim
  

• Cold-Clone Check — Verifies environment reproducibility on a fresh clone

Comparison Table

Common Pitfalls

• Mistake: Copying the entire repository before installing dependencies. Avoid: Copy only requirements.txt first to maximize layer caching.
• Mistake: Forgetting to include a .dockerignore file in the root. Avoid: Create .dockerignore excluding .git, data/, and local environment files.
• Mistake: Using pip install package without explicit version pinning. Avoid: Always generate and use a fully pinned requirements.txt file.
• Mistake: Using latest tag for the base image in production. Avoid: Pin the base image version, such as python:3.10-slim.

FAQs

• Why is the cold-clone check the ultimate test? It simulates a fresh deployment environment, proving that the Dockerfile and source files alone contain everything needed to build and run successfully without relying on local cache or environment state.
• Should I use latest for my base image? No. Always pin the base image version (e.g., python:3.10-slim) to prevent unexpected breakage when the base image maintainers push updates.
• What is the primary benefit of using slim base images? Slim images are significantly smaller and contain only the necessary runtime components, reducing image size, attack surface, and deployment time.
• If I change one line of code, will Docker reinstall all dependencies? No, if you followed the layering best practice, the dependency installation layer is cached, and only the subsequent layers containing the application code will be rebuilt.