How Not to Handle Exceptions in Python

I see lots of people handling Python exceptions the wrong way. Perhaps this applies to you too. Does the following situation sound familiar?

You’re writing some code, but you know that the library you’re using might raise an exception. You don’t remember which one, exactly. At this point, it’s tempting to use so-called catch-all blocks and get on with the fun stuff.

The worst way to do it

The worst you can do is create a try-except block that catches anything. By catch-all, I mean something like:

try:
    ...
except:
    pass

Catch-all blocks like these are bad because:

1.  You have no idea what other exceptions might be raised (more on this later). 
2. We’re hiding the exception by silently using pass instead of logging the error.

Furthermore, an empty except will catch everything, including KeyboardInterrupt (control + c), SystemExit, and even NameErrors! This means that the following code can not be stopped cleanly:

from time import sleep

while True:
    try:
        print("Try and stop me")
        sleep(1)
    except:
        print("Don't.. stop.. me now!")

Feel free to try it. You need to close your terminal window or kill the Python process to stop this program. 

A somewhat better way to catch all exceptions

In contrast, when using except Exception, although still a quick and dirty way to catch too many exceptions, at least you’ll be able to stop the running process properly:

from time import sleep
while True:
    try:
        print("Try and stop me")
        sleep(1)
    except Exception:
        print("Ok I'll stop!")

When catching Exception you will not catch SystemExit, KeyboardInterrupt and other such exceptions. Why’s that, you ask?

All exceptions inherit from a class called BaseException. According to the official documentation: “In a try statement with an except clause that mentions a particular class, that clause also handles any exception classes derived from that class.” An empty except is equivalent to except BaseException, hence it will catch all possible exceptions.

In contrast, the class Exception is defined as: “All built-in, non-system-exiting exceptions are derived from this class. All user-defined exceptions should also be derived from this class.”

It gets even worse

In the following example, we import the module called os to get the current working directory. However, my fatty little fingers made a typo:

import os

try:
    working_dir = os.getcdw()
    print(working_dir)
except:
    print('error')

Because os.getcdw is not a function in the os module, a NameError is thrown. Instead of failing, the except clause will catch the error, print ‘error’ and the program will continue despite our blatant typo. Unfortunately, this one is not solvable by catching Exception either!

Apparently, our little trick from step one is not a fix for all our problems. So what should we do?

Catch what you can handle

A phrase that is often heard about exceptions, is: catch what you can handle. Many developers are tempted to directly deal with exceptions, while it’s often better to let the exception propagate to a part of your program that can actually handle it.

For example, consider the part of a text editor that opens and loads files, let’s call it the OpenFile class. If the user requested to open a file that does not exist, you can either directly handle that error, or let it propagate. 

In this case, it’s better to propagate the exception to the caller, because OpenFile has no idea how bad this exception is for the caller. The caller can handle the situation in multiple ways:

• It could create a new file with that name instead and go on
• Maybe the caller needs the file to be there, in which case it can show an error dialog to inform the user that this file does not exist. 

Either way, it’s not up the OpenFile class to decide what to do in case of a FileNotFoundError. 

So should an exception always be propagated? No. A possible exception that can be handled in the FileOpen class, is the TimeoutError. You might want to retry a few times, for example, without bothering the caller with the error. This is an exception that OpenFile can handle, so it’s OK to catch it and retry.

Conclusion

You should under no circumstance catch more exceptions than you can handle. Blanket except blocks are a recipe for bugs and unpredictable code. In other words: catch what you can handle.

If you write your code with the ‘catch what you can handle’ matra in mind, writing catch-all blocks is breaking all the rules. So please, stop doing it. As an exercise, you could revisit some of your existing code and see if it can be improved with this new knowledge!

Learn more

I recommend everyone to read my comprehensive tutorial on the Python try except else finally construct.