Project Structure & Packaging

Modern Python projects: src layout, pyproject.toml, uv. The 2026 way to organize code that ships. FIND_VIDEO: search 'python project structure pyproject toml uv' — recommended channel: ArjanCodes / mCoding. Aim for 10 min or under.

20 minutesVideo LessonPDF notes

Watch Project Structure & Packaging on YouTube

🎯 Free Guest Mode: You are learning for free. Sign in to save your completion progress and quiz answers.

Ready to continue?

Mark this lesson as complete when you're ready to proceed.

Key moments

Project Setup and Environment Isolation — Demonstrates why system Python should be isolated and configures a clean virtual environment using VS Code.
Creating and Importing Modules — Explains that any Python source file is a module by importing a function from simple.py into main.py.
Defining Package Directories — Introduces package directories as containers for modules and imports functions using dot-notation module paths.
Exposing Functions with __init__.py — Shows how adding a dunder init file allows functions to be imported directly from the package namespace.
Configuring pyproject.toml and Build Tools — Sets up PEP 518 package metadata and installs build dependencies to generate distribution artifacts.
Building and Installing Local Wheels — Executes python -m build to create wheel archives and installs the resulting artifact into a second independent project.

PDF notes

Bottom line: You will isolate project dependencies using dedicated virtual environments, organize reusable Python code into modular packages with package initialization files, and build distributable wheel artifacts using pyproject.toml specifications. You can then install and import your custom packages locally across independent projects or prepare them for repository publication.

This lesson on Project Structure & Packaging is hands-on and example-driven. You will isolate project dependencies using dedicated virtual environments, organize reusable Python code into modular packages with package initialization files, and build distributable wheel artifacts using pyproject.toml specifications. You can then install and import your custom packages locally across independent projects or prepare them for repository publication.

What You'll Be Able To Do

Isolate project dependencies by creating and activating clean virtual environments using venv.
Structure reusable code by creating modular Python files and package directory hierarchies.
Expose package-level functions and attributes directly through init.py files.
Configure project metadata, dependencies, and package discovery inside pyproject.toml.
Build distributable wheel and source distribution archives using python -m build.
Install local wheel builds across separate project environments using pip.

Detailed Concept Walkthrough

1. Virtual Environments and Dependency Isolation

A virtual environment is an isolated directory tree containing a dedicated Python interpreter and package set. It prevents dependency pollution across multiple projects by decoupling project-specific requirements from the global system Python installation.

Mechanism: Creating a virtual environment clones the global Python interpreter binary into a standalone workspace without copying globally installed third-party libraries. This ensures that new installations remain strictly scoped to the active project.
Under the Hood: When an environment is active, the workspace runtime directs pip to install packages into a local site-packages folder rather than the OS-wide directory, keeping global site-packages untouched.
Best Practice: Always inspect the active environment using pip list before installing libraries to verify that you are not polluting the global operating system runtime.

# List packages in the current environment to verify clean state
pip list

# Create a standard virtual environment named .venv using the built-in venv module
python -m venv .venv

# Inspect the environment to confirm only basic tooling (e.g., pip) is installed
pip list

Key Takeaway: Always isolate project dependencies in dedicated virtual environments to guarantee predictable, reproducible execution across environments.

2. Modules versus Packages

A module is an individual Python file (.py) containing executable code, functions, or classes, whereas a package is a directory containing one or more modules. Packages organize modules into a structured namespace hierarchy.

Mechanism: Any valid Python file in the working directory can be imported as a module using its filename prefix (e.g., importing simple from simple.py). A subdirectory acts as a package namespace, allowing nested modules to be accessed via dot notation (e.g., utils.add).
Under the Hood: When resolving imports, Python searches the current working directory and paths listed in sys.path. Nested modules are dynamically loaded into sys.modules under their fully qualified hierarchical names.
Syntax Rule: To import specific functions from nested modules within a package directory, use the syntax 'from package.module import function_name'.

# File: utils/add.py
def add_numbers(a, b):
    """Module-level function inside the utils package."""
    return a + b

# File: main.py
# Import the function from the 'add' module inside the 'utils' package directory
from utils.add import add_numbers

result = add_numbers(3, 2)
print(result)  # Outputs: 5

Key Takeaway: Modules are single Python source files, while packages are directory containers that group related modules into coherent namespaces.

3. Package Initialization with Dunder Init

The init.py (dunder init) file marks a directory as an importable package and allows the directory itself to behave like a top-level module. It provides an entry point for defining package-level symbols and orchestrating exposed APIs.

Mechanism: Placing an init.py file inside a folder enables direct imports from the package root without requiring consumers to reference internal submodule filenames.
Execution Flow: When a package is imported, Python automatically executes init.py first, initializing the package namespace and loading any functions, variables, or submodules defined within it.
Best Practice: Use init.py to export a clean public API for your package so consumers do not need to know the internal directory layout or individual module names.

# File: utils/__init__.py
def sub_numbers(a, b):
    """Function exposed at the root package level."""
    return a - b

# File: main.py
# Directly import from the package namespace without referencing __init__ or submodules
from utils import sub_numbers

result = sub_numbers(9, 1)
print(result)  # Outputs: 8

Key Takeaway: The init.py file enables a directory to be imported directly as a module and exposes its public interface.

4. Packaging and Distribution with pyproject.toml

PEP 518 establishes pyproject.toml as the single source of truth for build configuration, project metadata, and dependencies. It supersedes legacy setup.py and requirements.txt workflows by providing a standardized build interface.

Mechanism: The pyproject.toml configuration specifies the build backend (such as setuptools and wheel) along with project metadata, dependencies, and target package directories to include in distribution artifacts.
Under the Hood: Executing the build front-end (python -m build) parses pyproject.toml, gathers source packages containing init.py, and generates both a wheel (.whl) binary distribution and a source archive (.tar.gz) in a dist/ directory.
Best Practice: Separate the distributable package name defined in pyproject.toml from internal module directory names, and install the compiled .whl file in downstream projects using pip.

# File: pyproject.toml
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "my-package"
version = "0.1.0"
dependencies = []

[tool.setuptools.packages.find]
where = ["."]
include = ["utils*"]

Key Takeaway: Modern Python packaging standardizes build metadata in pyproject.toml to create portable wheel archives via python -m build.

Topics Covered in Project Structure & Packaging

Project Setup and Environment Isolation (0:00 - 1:05) — Demonstrates why system Python should be isolated and configures a clean virtual environment using VS Code.
Creating and Importing Modules (1:06 - 1:48) — Explains that any Python source file is a module by importing a function from simple.py into main.py.
Defining Package Directories (1:49 - 2:40) — Introduces package directories as containers for modules and imports functions using dot-notation module paths.
Exposing Functions with init.py (2:41 - 3:28) — Shows how adding a dunder init file allows functions to be imported directly from the package namespace.
Configuring pyproject.toml and Build Tools (3:29 - 4:20) — Sets up PEP 518 package metadata and installs build dependencies to generate distribution artifacts.
Building and Installing Local Wheels (4:21 - 5:30) — Executes python -m build to create wheel archives and installs the resulting artifact into a second independent project.

Python Cheat Sheet

pip list — Lists all installed packages in the active environment
```
pip list
```
from <module> import <func> — Imports a specific function from a Python file
```
from simple import say_hello
```
from <pkg>.<mod> import <func> — Imports a function from a module inside a package
```
from utils.add import add_numbers
```
__init__.py — Designates directory as package and defines root namespace
```
# utils/__init__.py
def sub_numbers(a, b): return a - b
```
python -m build — Builds wheel and source distributions into dist/ folder
```
python -m build
```
pip install <path_to_wheel> — Installs a pre-built wheel package from a local directory
```
pip install ../P1/dist/my_package-0.1.0-py3-none-any.whl
```

Comparison Table

Artifact / Concept	File Structure	Primary Purpose
Module	Single .py file	Stores reusable functions and classes
Package	Directory with modules & init.py	Groups related modules under namespace
Distribution Wheel (.whl)	Zipped archive from pyproject.toml	Distributes installable code across projects

Common Pitfalls

Mistake: Developing projects directly inside the global system Python environment. Avoid: Create and activate an isolated virtual environment before installing project-specific packages.
Mistake: Forgetting to create init.py in package subdirectories before running build tools. Avoid: Ensure every package directory contains an init.py file so build tools discover and package it.
Mistake: Relying on legacy setup.py scripts for modern project packaging configurations. Avoid: Define project metadata, build backend tools, and dependencies inside a standardized pyproject.toml file.

FAQs

Why should I use a virtual environment instead of system Python? A virtual environment isolates dependencies per project, preventing version conflicts and eliminating unnecessary bloat in the system Python installation.
Is init.py mandatory for importing submodules in Python 3? While Python 3 supports implicit namespace packages without it, init.py is required to expose root package functions and enable standard package discovery tools.
What is the difference between a wheel (.whl) and a source distribution (.tar.gz)? A wheel is a pre-built, ready-to-install package format, whereas a source distribution is an unbuilt archive containing raw source files.
Does pyproject.toml completely replace requirements.txt? Yes, pyproject.toml acts as the single source of truth for package dependencies and build metadata in modern packaging workflows under PEP 518.

Common pitfalls

Mistake: Developing projects directly inside the global system Python environment. Avoid: Create and activate an isolated virtual environment before installing project-specific packages.
Mistake: Forgetting to create __init__.py in package subdirectories before running build tools. Avoid: Ensure every package directory contains an __init__.py file so build tools discover and package it.
Mistake: Relying on legacy setup.py scripts for modern project packaging configurations. Avoid: Define project metadata, build backend tools, and dependencies inside a standardized pyproject.toml file.

Hello, and welcome to this presentation from The Urban Penguin. My name is Andrew Mallett, and today we're looking at that problem that you do want to create reusable code, but you need to know how to make it reusable across all of your projects within Python. So, we're going to be taking a look at modules and taking a look at packages, and looking at the difference between the two of them and seeing how we can set it up in our development environment. Now, come on then. Take a look at my working environment. Yes, I know it is Windows, but many people will be using Windows as their software development platform. We'll be taking a look at using Python and Visual Studio Code, all of which are available on Mac OS as well as on Linux. So, everybody should be able to follow through and get the most out of this exercise. I've just created a project's directory on my desktop with two empty subdirectories: P1 for project one, and P2 for project two. We're going to be looking at working with modules and packages inside of our P1 directory to begin with. Moving through to Visual Studio Code, we can navigate through to my desktop, projects, and I'm going to open up the empty project one directory. So, the project directory is shown then at the top of my Explorer. Let me just clear that warning about trusting the author, and this will then contain all of my configuration, my packages, and my modules. And we're going to be stepping through the process. The first thing that we're going to do is create ourselves a Python file so we can identify what type of project that we're working with. So, I'm just calling this main.py. Now, taking a look at the bottom right-hand corner of Visual Studio Code, we can see then the version of Python that we're using for this. And this is our global or system Python. Ideally, we don't develop our projects with our system Python. If I go through and open up a terminal, so if I just go View and Terminal, we can wait for it to open up as it builds the first time. If I do a pip list, I can list all of the modules that we have. These are representing not my project modules, but the modules and packages that have been installed into the system. As I work on more and more projects, it's going to be very difficult to differentiate the packages required for my project from the packages that I had anyway. So, working in a clean environment is going to lessen the overhead on your system Python as well as making it easier to determine what is required. So, I'm just going to close this down. I can just kill the terminal with the dustbin. Go back onto the versioning in the bottom right-hand corner and create a virtual environment based on venv. And I can choose then to clone my global Python interpreter, but we don't bring across any of the additional packages. We only have the packages that are built into Python. Once that has set up, I have got a clean and fresh environment. And if I go back through to my terminal now, it tells me that the virtual environment has been created and activated, but I don't see a venv in front of the prompt name. But I can verify by running pip list. And we can see now that it's a very different view. We've only got our Python installer, pip, as an available package. So, this is a great starting point, a clean and fresh Python environment for a new project. The next stage is to understand the term module. In reality, any Python file is a module. But let's just go through and check that my main.py file, my main Python file is working. And we can see there that we're printing hello. I probably clicked it twice, but I've got hello coming out twice. So, my virtual environment is up and running. I can see then the Python interpreter coming from my virtual environment, and everything is working well. But to investigate the concept of modules a little bit more, I'm going to go through to my P1 directory and create a new file. I'm just going to call this, I don't know, it doesn't really matter what, let me just call it simple.py. We'll have a function inside of this, so def, define, and then we can have something like, say, hello. I'm going to keep this really simple for the demonstration, so I'm not putting in any type hints. And then we can have good day to you. And of course, all of this needs to be in quotes. So, let me select the text and add in my double quotes. So, I would normally use double quotes for quoting text, although it makes no difference to Python. But you should be consistent. Keeping the consistency is important. I'm actually going to save this. I know you don't normally need to save files because if you run it, it's going to be saved, but I'm not really going to run this file. I'm going to reference it from my main.py. So, don't forget to save your simple.py file, the module. We can now see my import working, so: from simple import say_hello So, simple now, we can see, is a module, but there's nothing special about that. I can execute my main.py without saving it specifically, because it saves when I execute. Of course, the import worked, but I didn't actually call say_hello, so let's just add that in. And then, of course, we get hello and good day to you. So, in simple terms, a module is just a Python file, any Python file that we can access. So, this is within my home directory or the same working directory rather. This is in my working directory, so I can access it. If it wasn't in my working directory, I couldn't navigate back up my structure to be able to read it. We're going to create now a new directory, and I'm going to call this utils. Utils is going to act as a package. A package is a container for modules. So, a module is a Python file. A package is a container or directory that contains Python modules. I'm defining then add_numbers, so again, a very simple function. We'll just have a added to b. Again, not worrying with type hints, and returning a plus b. Would seem reasonable if we've called it add_numbers, don't you think? Again, not forgetting to save then my add.py file. I can move back into my main file now. And I go from, and then my package name, utils, navigating down to the module .add, so utils.add, import add_numbers. Now, we can quickly test that this is going to work. So, I can use the print statement, and then add_numbers, and we can then go through and add 3 and 2 together. And hopefully, if my maths are right, we're going to get 5. So, I get hello, good day to you, and we have 5. Utils now is acting as a package, a container for modules. But very often, you know when you import a module, you're accessing the resource directly. And that would be done by making the utils directory appear as a Python file, a module itself. If I go through to utils, and I'm going to create myself a new Python file, but this file has to have a very special name. So, __init__.py, that's double underscore init, double underscore .py. The dunder init file. It's the dunder init file that makes the directory appear as a module itself. So, again, I can put as many resources inside of my dunder init file as I want. So, here I'm just going to go through and put in one function. We might as well have sub for subtract numbers, again, keeping it really simple, and then just returning a subtract b. And with that done, don't forget to save the dunder init .py file. So, let's go up and save it. And now returning through to my main.py, from here, I can go through and we can check the import is working okay. So, from utils directly, I can then import sub_numbers. I don't need to navigate down to utils.__init__. Dunder init is making the utils directory appear as a module itself. But either way, utils is a directory, so is always a package, a container of modules. It can have separate modules, or it can be made a module itself by creating the __init__.py file. Remember, dunder init is double underscore init double underscore .py. The next stage is to go through and take a look at building a package that we can distribute to other projects, be it locally or through the PyPI repository. So, I'm creating a new file here called pyproject.toml, t-o-m-l, so pyproject.toml. I'm going to copy across an existing template that I use. So, I'm just going to drop out, copy this across, and then paste it in. The pyproject.toml file has been in place since PEP 518, 518. As we go through and take a look at it, we can see, well, when we build packages, what do we require? We can give our project a name, and we can then go through and specify my-package. I'm setting up a name deliberately different to our directory, our utils directory. As we scroll down, we can see then the dependencies. So, this is replacing the requirements.txt. This really becomes a single source of truth file that we can work with. It's replacing the requirements.txt, but it's also replacing the setup.py file that was previously used before PEP 518. Now we specify when we run setup, what packages we want to search, and I'm going to look for the utils directory. But this would be a list of subdirectories that had the __init__.py file in it. So, we've only got the one directory, and I'm going to go through and save the toml file. I'm going to go through now, and in my project, I'm going to install build, setuptools, and wheel. We said that build requires this, but I have to have the build module in any case to be able to do anything with it. They're going to be installed in this project, but they don't become requirements of the target project. With these installs, I can actually now just go through and execute Python and build myself a package that I can use in my second project, P2. As I run through then, python -m and build, we read the pyproject.toml file, and we're creating in the distribution directory the output, and it will give me the wheel output, which is my standard pip install, as well as a zipped-up version, the tar.gz file. So, this now has created a package that I can share with other projects, or even publish to the PyPI site. Now, of course, I know you want to see this working. So, let me go through and just close this folder, and open up the project two folder. So, I can navigate through to my desktop, projects, and project two. Again, it's an empty folder. I can trust the authors again. And the first thing I'm going to do is create myself the main.py file so I can identify this as a Python project. Having now created my Python file, I can create a virtual environment, basing it on then my global environment, as we did before. We can allow this to run through. So, this again is going to give me that clean Python environment with only pip installed. That's now done, and is up and running. And if I go through and open up my terminal, we should be able to see through here with pip list, only one package. So, we see there that we have pip. So, this is pretty much where we had things before. But what I'm going to do now is a pip install, moving to the parent directory, so .., and then navigating down to P1, and dist, and then just tab-completing then on my package, making sure that I've got the wheel file. And we can see then that we've installed my-package. And I can do a pip list, and we can see then that my-package, remember that's the name that I gave to the package, it's not the name of my module. Remember, I might have many subdirectories within that. So, we would then go through and use from the utils module, we can navigate down and import my sub_numbers. So, this is coming from the dunder init. And then we can prove that it's working through to the add.py, utils.add, and we can then go through and see import and our add_numbers. We want to check that both of these are working, so both can be imported, and can execute correctly. So, we will add in the simple print statements. But we've imported all of our packages, but not the modules at the top level. So, we didn't have our simple module or our main module. So now, as I go through and print sub_numbers, and we can have 9 - 1, and then maybe if we do almost the reverse of this, at least we know the maths are working nicely, let's have our add_numbers, and then 8 + 1, and this should be good to go. Let's execute it. And as we look at the glorious output, we can see then that we've got 8 and 9, so that is looking good to me. So, when you think of it, we created our package from our first project, and made it available to our second project by importing or installing the package. We've shown what are packages, a collection of modules in a directory, and the use of the dunder init Python module. So, working through a quick summary: You will have your project directory. Inside of your project directory, we can create the pyproject.toml file that then incorporates the requirements.txt as well as the setup.py file, all of it in one file. We then create packages within our project directory. Packages are directories themselves, but these directories contain other modules. One such module could well be the dunder init module that represents the directory or package itself. As well as the dunder init module, you can also create other named modules within side of your package directory. All that is left to say is, thank you for watching. If you've enjoyed this, then you're probably sadder than I first thought. For more of this drivel, please subscribe, like, and comment to @theurbanpenguin.

Frequently asked questions

Why should I use a virtual environment instead of system Python?

A virtual environment isolates dependencies per project, preventing version conflicts and eliminating unnecessary bloat in the system Python installation.

Is __init__.py mandatory for importing submodules in Python 3?

While Python 3 supports implicit namespace packages without it, __init__.py is required to expose root package functions and enable standard package discovery tools.

What is the difference between a wheel (.whl) and a source distribution (.tar.gz)?

A wheel is a pre-built, ready-to-install package format, whereas a source distribution is an unbuilt archive containing raw source files.

Does pyproject.toml completely replace requirements.txt?

Yes, pyproject.toml acts as the single source of truth for package dependencies and build metadata in modern packaging workflows under PEP 518.

How was this lesson?

Your feedback helps us refine explanations and catch bugs.