Home » Blog » How Not to Handle Exceptions in Python
A broom sweeps errors under a rug beneath a crossed-out except: pass, the catch-all you should avoid

How Not to Handle Exceptions in Python

By Published Updated

I see lots of people handling Python exceptions the wrong way. Perhaps this applies to you too. Does the following situation sound familiar?

You’re writing some code, but you know that the library you’re using might raise an exception. You don’t remember which one, exactly. At this point, it’s tempting to use so-called catch-all blocks and get on with the fun stuff.

The worst way to do it

The worst you can do is create a try-except block that catches anything. By catch-all, I mean something like:

try:
    ...
except:
    passCode language: Python (python)

Catch-all blocks like these are bad because:

  1.  You have no idea what other exceptions might be raised (more on this later). 
  2. We’re hiding the exception by silently using pass instead of logging the error.

Furthermore, an empty except will catch everything, including KeyboardInterrupt (control + c) and SystemExit, which are meant to stop your program. This means that the following code can not be stopped cleanly:

from time import sleep

while True:
    try:
        print("Try and stop me")
        sleep(1)
    except:
        print("Don't.. stop.. me now!")Code language: Python (python)

Feel free to try it. You need to close your terminal window or kill the Python process to stop this program. 

A somewhat better way to catch all exceptions

In contrast, when using except Exception, although still a quick and dirty way to catch too many exceptions, at least you’ll be able to stop the running process properly:

from time import sleep
while True:
    try:
        print("Try and stop me")
        sleep(1)
    except Exception:
        print("Ok I'll stop!")Code language: Python (python)

When catching Exception you will not catch SystemExit, KeyboardInterrupt and other such exceptions. Why’s that, you ask?

All exceptions inherit from a class called BaseException. According to the official documentation: “In a try statement with an except clause that mentions a particular class, that clause also handles any exception classes derived from that class.” An empty except is equivalent to except BaseException, hence it will catch all possible exceptions.

In contrast, the class Exception is defined as: “All built-in, non-system-exiting exceptions are derived from this class. All user-defined exceptions should also be derived from this class.”

It gets even worse

In the following example, we import the module called os to get the current working directory. However, my fatty little fingers made a typo:

import os

try:
    working_dir = os.getcdw()
    print(working_dir)
except:
    print('error')Code language: Python (python)

Because os.getcdw is not a function in the os module, Python raises an AttributeError. Without the try-except, you would see this helpful message:

AttributeError: module 'os' has no attribute 'getcdw'. Did you mean '.getcwd' instead of '.getcdw'?Code language: Python (python)

Instead of failing, the except clause will catch the error, print ‘error’ and the program will continue despite our blatant typo. Unfortunately, this one is not solvable by catching Exception either, because AttributeError is a regular exception that inherits from Exception!

Apparently, our little trick from step one is not a fix for all our problems. So what should we do?

If you must catch broadly, log it

Sometimes you do need a broad except Exception, for example at the top level of a program or a web request handler, where one failure shouldn’t bring everything down. In that case, never throw the error away. Log it, including the traceback, with logging.exception():

import logging
import os

logging.basicConfig(level=logging.INFO)

try:
    working_dir = os.getcdw()
    print(working_dir)
except Exception:
    logging.exception("Could not get the working directory")Code language: Python (python)

The program keeps running, but the full error ends up in your log:

ERROR:root:Could not get the working directory
Traceback (most recent call last):
  File "/home/erik/logex.py", line 7, in <module>
    working_dir = os.getcdw()
                  ^^^^^^^^^
AttributeError: module 'os' has no attribute 'getcdw'. Did you mean '.getcwd' instead of '.getcdw'?Code language: Python (python)

Now the typo can’t hide anymore. Call logging.exception() only inside an except block, because that’s where it finds the exception to log.

And if you really want to ignore one specific error, say so explicitly with contextlib.suppress. Here, we delete a cache file, and it’s fine if it doesn’t exist:

import os
from contextlib import suppress

with suppress(FileNotFoundError):
    os.remove('cache.tmp')Code language: Python (python)

This only ignores FileNotFoundError. Any other error, like a typo or a permission problem, still shows up.

Catch what you can handle

A phrase that is often heard about exceptions, is: catch what you can handle. Many developers are tempted to directly deal with exceptions, while it’s often better to let the exception propagate to a part of your program that can actually handle it.

For example, consider the part of a text editor that opens and loads files, let’s call it the OpenFile class. If the user requested to open a file that does not exist, you can either directly handle that error, or let it propagate. 

In this case, it’s better to propagate the exception to the caller, because OpenFile has no idea how bad this exception is for the caller. The caller can handle the situation in multiple ways:

  • It could create a new file with that name instead and go on
  • Maybe the caller needs the file to be there, in which case it can show an error dialog to inform the user that this file does not exist. 

Either way, it’s not up to the OpenFile class to decide what to do in case of a FileNotFoundError. 

So should an exception always be propagated? No. A possible exception that can be handled in the OpenFile class, is the TimeoutError. You might want to retry a few times, for example, without bothering the caller with the error. This is an exception that OpenFile can handle, so it’s OK to catch it and retry.

Conclusion

You should under no circumstance catch more exceptions than you can handle. Blanket except blocks are a recipe for bugs and unpredictable code. In other words: catch what you can handle.

If you write your code with the ‘catch what you can handle’ mantra in mind, writing catch-all blocks is breaking all the rules. So please, stop doing it. As an exercise, you could revisit some of your existing code and see if it can be improved with this new knowledge!

Learn more

I recommend everyone to read my comprehensive tutorial on the Python try except else finally construct.

Leave a Comment