Error Handling Patterns

Exception hierarchies, retries with backoff, Result types, and the patterns that production code uses. FIND_VIDEO: search 'python exception handling patterns best practices' — recommended channel: ArjanCodes / mCoding. Aim for 10 min or under.

11 minutesVideo LessonPDF notes

🎯 Free Guest Mode: You are learning for free. Sign in to save your completion progress and quiz answers.

Ready to continue?

Mark this lesson as complete when you're ready to proceed.

Key moments

Try and Except Motivation — The instructor demonstrates how unhandled tracebacks break user experience and introduces the try/except construct.
Specific vs General Exceptions — The lesson demonstrates how broad exception handlers accidentally mask developer bugs like NameError.
Exception Aliasing — The instructor aliases exceptions using the as keyword to extract internal system error messages dynamically.
The Else Clause — The instructor explains why success-dependent execution logic belongs inside the else block rather than the try block.
The Finally Clause — The lesson covers how finally executes unconditionally to release resources like database handles.
Manual Error Raising — The instructor demonstrates manually raising custom exceptions using the raise keyword during validation checks.

PDF notes

Bottom line: You will intercept runtime exceptions, prevent application crashes, and present actionable feedback using Python's try, except, else, and finally blocks. You will master specific error catching in hierarchical order, isolate non-risky execution logic, guarantee resource cleanup, and trigger custom errors with raise.

This lesson on Error Handling Patterns is hands-on and example-driven. You will intercept runtime exceptions, prevent application crashes, and present actionable feedback using Python's try, except, else, and finally blocks. You will master specific error catching in hierarchical order, isolate non-risky execution logic, guarantee resource cleanup, and trigger custom errors with raise.

What You'll Be Able To Do

Intercept runtime errors using targeted try and except blocks instead of unhandled tracebacks.
Order multiple except clauses hierarchically from specific subclasses to general exceptions.
Inspect runtime error messages dynamically by binding exceptions with the as keyword.
Isolate post-success data processing from error-prone operations using the else block.
Guarantee cleanup tasks like closing file handles and database connections using the finally block.
Trigger custom error handling workflows programmatically using the raise keyword.

Detailed Concept Walkthrough

1. Targeted Exception Handling and Ordering

The try/except construct isolates risky operations and defines graceful fallback behavior when specific runtime errors occur. Matching must proceed from narrow subclasses to broader parent exceptions.

Mechanism: Code placed inside the try block executes normally until an unhandled error occurs, which immediately halts try execution and transfers control to the first matching except clause.
Hierarchy Evaluation: Python inspects except clauses from top to bottom; placing a broad Exception handler before a specific handler like FileNotFoundError will swallow the error prematurely.
Bug Isolation: Catching specific exceptions prevents masking developer bugs, such as NameError or typo assignments, which should raise a standard traceback during development rather than triggering a false error handler.

try:
    # Risky operation: opening an external file that might not exist
    f = open('testfile.txt')
    var = bad_var  # NameError: unexpected developer bug
except FileNotFoundError:
    print('Sorry, this file does not exist.')
except Exception as e:
    # Fallback for unexpected runtime issues
    print(f'General error intercepted: {e}')

Key Takeaway: Always list specific exception types before generic ones to avoid masking unintended bugs.

2. Exception Aliasing with the As Keyword

Binding an intercepted exception instance to a variable exposes Python's internal diagnostic message for logging and debugging without crashing the script.

Mechanism: Appending as <variable_name> (conventionally as e) to an except clause assigns the caught exception instance to a local variable.
Diagnostic Extraction: Printing or logging e stringifies the exception object, providing standard POSIX error codes, missing variable descriptions, or file path details.
Best Practice: Use the exception instance to generate structured telemetry or log messages while hiding raw, confusing tracebacks from end users.

try:
    f = open('missing_dataset.csv')
except FileNotFoundError as e:
    # Logs OS error number and missing path: [Errno 2] No such file or directory
    print(f'Handled FileNotFoundError instance: {e}')
except Exception as e:
    print(f'Handled general failure: {e}')

Key Takeaway: Alias exceptions using as e to capture internal error details without exposing tracebacks to end users.

3. The Else and Finally Lifecycle

The else block executes only when the try block succeeds without error, while the finally block executes unconditionally regardless of success, caught errors, or unhandled crashes.

Scope Separation: The else block isolates non-risky logic (e.g., reading payload contents) away from the try block so that errors in processing are not mistaken for acquisition errors.
Guaranteed Cleanup: The finally block always runs at the end of the lifecycle, even if an uncaught exception triggers or an except block handles an error.
Data Engineering Relevance: Use finally to release external resources such as database cursors, network connections, or open OS file descriptors.

try:
    f = open('test_file.txt')
except FileNotFoundError as e:
    print(f'File missing: {e}')
else:
    # Executes ONLY if try succeeded without throwing an exception
    print(f.read())
    f.close()
finally:
    # Executes no matter what happened above
    print('Cleaning up and closing pipeline handles...')

Key Takeaway: Use else for code that requires prior success, and finally for resource cleanup that must run unconditionally.

4. Manual Exception Dispatch with Raise

The raise keyword forces an immediate exception when runtime validation or domain constraints fail, even if the native Python statement is syntactically and structurally valid.

Mechanism: Executing raise <ExceptionClass> immediately interrupts the current execution block and jumps to the corresponding except clause.
Domain Validation: Pipelines use raise to halt data ingestion when file names, schema definitions, or header rows indicate corrupt or invalid data payloads.
Control Flow Integration: Raised exceptions seamlessly trigger downstream except and finally blocks within the active scope or bubble up the call stack if unhandled.

try:
    f = open('corrupt_file.txt')
    # Enforce domain validation on the opened resource
    if f.name == 'corrupt_file.txt':
        raise Exception('File marked as corrupted by ingestion validation.')
except Exception as e:
    print(f'Pipeline alerted: {e}')
finally:
    print('Validation cycle completed.')

Key Takeaway: Use raise to manually halt execution and trigger error handling when business validation rules fail.

Topics Covered in Error Handling Patterns

Try and Except Motivation (0:00 - 1:15) — The instructor demonstrates how unhandled tracebacks break user experience and introduces the try/except construct.
Specific vs General Exceptions (1:16 - 2:45) — The lesson demonstrates how broad exception handlers accidentally mask developer bugs like NameError.
Exception Aliasing (2:46 - 3:35) — The instructor aliases exceptions using the as keyword to extract internal system error messages dynamically.
The Else Clause (3:36 - 5:00) — The instructor explains why success-dependent execution logic belongs inside the else block rather than the try block.
The Finally Clause (5:01 - 6:15) — The lesson covers how finally executes unconditionally to release resources like database handles.
Manual Error Raising (6:16 - 7:30) — The instructor demonstrates manually raising custom exceptions using the raise keyword during validation checks.

Python Cheat Sheet

try / except — Catches runtime errors and executes fallback logic

try:
    f = open('test.txt')
except FileNotFoundError:
    print('File missing')

except <Error> as e — Binds caught error instance to a variable

try:
    f = open('test.txt')
except FileNotFoundError as e:
    print(e)

except Exception — Catches any standard built-in exception broadly

try:
    x = bad_var
except Exception as e:
    print(e)

else — Executes code only when try encounters zero errors

try:
    f = open('test.txt')
except FileNotFoundError:
    pass
else:
    print(f.read())

finally — Executes unconditionally for guaranteed resource cleanup

try:
    f = open('test.txt')
finally:
    print('Cleanup complete')

raise <Error> — Programmatically triggers an exception immediately

if f.name == 'corrupt.txt':
    raise Exception('Corrupted file detected')

Comparison Table

Clause	Trigger Condition	Data Engineering Use Case
try	Code executed on initial entry	Connecting to DB or opening files
except	Runs only when matching error occurs	Catching missing tables or paths
else	Runs only if try raised nothing	Processing payload data post-fetch
finally	Runs unconditionally after all blocks	Closing open DB connections safely

Common Pitfalls

Mistake: Placing general Exception above specific exceptions in except clauses. Avoid: Position specific exceptions like FileNotFoundError before broad Exception clauses.
Mistake: Putting safe data processing logic inside the try block. Avoid: Move non-failing operational code to the else block to isolate errors.
Mistake: Using a generic except block that masks developer code bugs. Avoid: Catch explicit exception classes so bugs like NameError raise tracebacks.
Mistake: Neglecting to close external resources when unexpected exceptions occur. Avoid: Place resource cleanup code like file or database disconnects in finally.

FAQs

Why should I use the else block instead of putting all code inside try? Putting all code inside try risks catching accidental exceptions in your processing logic under the wrong handler. The else block ensures you only handle exceptions originating from the specific setup operation.
What happens if an exception is raised that does not match any except clause? The script halts execution and prints the default Python traceback, unless an except Exception clause is defined at the end to catch it.
When should I manually trigger an error using the raise keyword? Use raise when application-level validation rules fail—such as encountering a corrupt data file—even if the underlying Python operations executed without error.

Common pitfalls

Mistake: Placing general Exception above specific exceptions in except clauses. Avoid: Position specific exceptions like FileNotFoundError before broad Exception clauses.
Mistake: Putting safe data processing logic inside the try block. Avoid: Move non-failing operational code to the else block to isolate errors.
Mistake: Using a generic except block that masks developer code bugs. Avoid: Catch explicit exception classes so bugs like NameError raise tracebacks.
Mistake: Neglecting to close external resources when unexpected exceptions occur. Avoid: Place resource cleanup code like file or database disconnects in finally.

Hey, how's it going, everybody? In this video, we'll be going over how to handle errors and exceptions in Python by learning how these try/except blocks work. Now, these try/except blocks can be confusing to people who aren't familiar with them because they can actually have a lot of different sections. Like we have this try, except, else, and finally. So, let's take a look at exactly what's going on here and how we can use these. Now, a lot of the time we're just going to use these first two, try and except. So, for now, I'm going to go ahead and comment out this else and finally. So, first of all, why would we even need a try and except block in the first place? Um, well, let's run some code that throws an error, and it might give us a uh better idea of why we need this. So, I'm going to try to open up a file here. If we look in the current folder that I'm in, we have this test_file.txt. Let's try to open that up, and let's accidentally leave off the underscore. So, if I do testfile.txt without the underscore, now if I run this, you can see that we get this long traceback error here. Now, this is useful for us as developers because it gives us a lot of useful information. We have this line number here, and FileNotFoundError, and exactly what happened. But, uh if we get an error like this, we wouldn't want something like this to be displayed to the people who are using our software. So, if we can anticipate uh sections of our code that uh might throw an error or an exception, then we can use these try and except blocks to handle them in the way that we want. So, what I'm going to do here is I'm going to move the code that threw the error into this uh try section here, and within this except block, I'm just going to print an Error and an exclamation point. Actually, let me make this a little bit more detailed. I'm going to say, uh, "Sorry, this file does not exist." So now if I run this code, uh you can see that we receive our custom error instead of the uh that a longer, more verbose error that Python gave us before. So what exactly is going on here? So what's going on is that within our try block, we're going to try to run some code and this throws an exception right here. So then, if it throws an exception, then we go to our except and then it's going to print out this line here. Now, this exception here is kind of vague. Um now if you remember when we ran this code outside of the try/except block, we got a FileNotFoundError, and we actually want to be as specific as possible when we catch catch our exceptions because the goal of the try/excepts isn't to just get around all of the exceptions and errors that we could could run into, uh it's meant to catch the errors that we expect and handle them properly. So this general exception here, it's not only going to uh catch FileNotFoundErrors, it's also going to catch a lot of other different problems that we could run into. Um so for example here, if I was to make this the correct file name, and then uh a line underneath here, let's just make a bad assignment. So I'll say var = bad_var, and if I run this, you can see that we get the exact same thing. It hits this exception and then it says, "Sorry, this file does not exist." Uh let me fix this little typo here. So what happened here is within our try block, uh this code actually ran successfully, and this is what threw the exception, but that's not clear right now because this is too general. So what I'm going to do is I'm going to change this to a uh catch a file FileNotFoundError. So now if I save and run this, now you can see that we get the regular Python traceback, which is a good thing. So what happened here is it came in and it opened up this file, and that was the error that we were trying to catch, and then this was an unexpected exception here. And uh you can see here that it is a NameError, and we're only catching FileNotFoundErrors. So it didn't get caught, and it didn't execute this um code here, and instead it just printed out the default Python traceback. Now if we do want to catch more general exceptions, then we can act- actually add another except um below this. And so if I say um except Exception, and then within here I'm just going to print uh let's see, I'll just print "Sorry, something went wrong." Now if you do handle multiple exceptions like this, uh be sure that you are putting the more specific exceptions at the top and the more general ones uh the further that you go down. Because if I was to put this general Exception here at the top, then it's always going to hit that one first, and it'll never get to the FileNotFoundError. Now currently we're printing out some custom messages here, but if I wanted uh to just print out the exception that it threw, then I can just do as, and you can name this whatever, I'm just going to say as e, and then I can print out e, and I can also do this for the FileNotFoundError too, so I'll do as e, and instead of this custom message I will print out e. And if I run that, you can see that it comes down to this Exception here, and instead of printing out a custom message, it prints out the exception that we hit, which is the bad_var is not defined. So, if I just get rid of this line of code, and then run that, then it runs successfully. If I give this a bad file name, and run that, you can see that it says error number two, no such file or directory uh and then the file name. Okay, so that's a quick look at the try/except blocks. Now, what's the deal with the else and finally blocks down here? Um so first, let's start out with the else, and we'll uncomment this. Now, what the else does is it runs code uh that needs to be executed if the try clause doesn't raise an exception. So uh so for example here, let's say that this didn't run an exception here. So I'm going to make this the correct file name, and now I'm just going to print out the contents here, and then I'm also going to close the file, so I'll do an f.close(). So if I run this, our try isn't going to throw an exception because we have the correct file name here, and then since it doesn't throw an exception, it's going to execute the code that's within the else clause. So it prints out the contents of the file, and then it closes the file. And if I open up this file here, you can see that it just says test file contents, and that's exactly what got printed out down here. Now, you might wonder why we just didn't put this in our try block. So we could move this up here and just execute this underneath. And if it throws an exception, then uh hopefully it's one that we'll catch anyway. Now, you could do that, but like I was saying earlier that we want to be specific about what it is that we're actually trying to catch here. Uh so if I move this code up here, then um then it may throw an exception, and we may catch that exception, but it may be something that we weren't trying to catch, and it may just be an accident. So it's better to just go ahead and put that in the else clause, or to uh just put it after the uh try block altogether. Okay, so now we come to our finally clause. Um so like I was saying, the else clause only runs if we don't throw an exception, but the finally runs no matter what happens, whether the code is successful or whether we throw an exception. So this is useful for making sure that you release certain resources regardless of what happened in the try/except. Uh so for example, say you were working with a database or something like that, uh then this would be an area where you could close down the database at this step. So I'm just going to put in a print statement here, and I'm just going to say that I am executing the finally. So you can see if I run this, then it runs our else clause, and then it also executes our finally. And now if uh within our try, if I make something that's going to throw an exception, and then I run this, you can see that it throws our exception, but it also still executes our finally. So again, that's a way that you can um make sure that you properly close down certain things. So like, you know, uh closing down a database or something like that. Something that needs to be done regardless if the code is successful or not. Okay, so we're just about done, but the last thing that I wanted uh you to know about exceptions is that you can actually raise exceptions on your own if you need to do that. So it doesn't have to be something that Python would have caught on its own. Um so for example here, uh I have this corrupt_file.txt, so if I open up this corrupt_file.txt, now there's nothing actually wrong with this file, but I'm just kind of using it for this example. Um so I want to raise my own exception here. So I'm going to say if f.name is equal to, and I'm going to say corrupt_file.txt, so I'll copy that and paste it in there. Now uh now I'm going to raise an exception manually. So to do that, it's as easy as saying raise, and then the exception that you want to raise. So I'll just raise this general Exception here. So if I save that and if I run that, so it tried to print this Exception, but it just doesn't have any details here, so I'll just put in an Error with an exclamation point. So if I run that again, you can see that this line did raise this exception, so then it went down to where uh this except line where we were handling that exception, and it printed out the error. So this allows us to manually raise exceptions. So if I was to comment out that line and run the code, now you can see that it did uh the correct thing that it would have done normally. So it opened up the file, and it came down here and printed out the contents and closed the file. The contents of the file, it just says corrupt file, and then it ran our finally clause. So that about does it for this video. Uh hopefully it gave you a large overview of exception handling and gave you some ideas of how you can use this in your projects. If you do have any questions, just ask in the comment section below. Uh be sure to subscribe for future videos, and thank you all for watching.

Frequently asked questions

Why should I use the else block instead of putting all code inside try?

Putting all code inside try risks catching accidental exceptions in your processing logic under the wrong handler. The else block ensures you only handle exceptions originating from the specific setup operation.

What happens if an exception is raised that does not match any except clause?

The script halts execution and prints the default Python traceback, unless an except Exception clause is defined at the end to catch it.

When should I manually trigger an error using the raise keyword?

Use raise when application-level validation rules fail—such as encountering a corrupt data file—even if the underlying Python operations executed without error.

How was this lesson?

Your feedback helps us refine explanations and catch bugs.