Python Pipenv: Another Package Manager

Pipenv is an alternative to Python Poetry that does similar things. Like Poetry, many prefer Pipenv instead of pip install and virtualenv separately. It’s up to you to decide which method you like better. Perhaps you get to work in a team or on an open-source project where pip + virtualenv is the norm; that’s OK. After reading this, you may want to convince them to use pipenv or Poetry instead! However, if you haven’t done so already, I suggest you look at Poetry first. It’s the better option if you ask me!

Pipenv vs virtualenv

Using pipenv as a Python package manager has several advantages compared to using pip and virtualenv separately. These are the main ones:

• You no longer need to use pip and virtualenv separately. Instead, you have one tool that does it all and more!
• pipenv separates your top-level dependencies from the last tested combination (e.g., the pip freeze output). This makes dependency management more user-friendly for you as a developer.
• pipenv encourages using the latest versions of dependencies to minimize security risks. It can even scan your dependencies for known vulnerabilities.
• pipenv gives insight into your dependency graph with pipenv graph.
• pipenv hashes all dependencies. It will detect packages that have been tampered with after you initially included them as a dependency.
• pipenv can work with requirements.txt files too. If there is one, it will automatically detect it and convert it into a Pipfile.

Perhaps some of these points confuse you. Don’t worry; they will all be explained!

Pipenv install

We can use pip (or even better: pipx) to install pipenv:

$ pip install --user pipenv

The --user option installs pipenv for the local user only. If you use it, you ensure that you won’t interfere with system-wide packages. If that’s an issue you’re worried about, also read about pipx since that tool is designed to fix these issues. If you get an error that pip was not found, try pip3 or python -m pip. You may also want to review the Python installation instructions if you still have problems.

If you are on MacOS or Linux, I can also recommend Homebrew. Go to the Homebrew website to install it. After that, it’s as simple as:

$ brew install pipenv

How to use Python pipenv

Now that you have pipenv installed let’s try it!

First, create a directory. I called mine myenv. Now cd into the directory and install your first dependency with pipenv:

$ pipenv install requests

What happens next is:

• pipenv detects there’s no virtual environment yet, so it creates one.
• It installs requests.
• It creates two files: Pipfile and Pipfile.lock.

The screenshot below shows pipenv in action on my system.

After it’s done, you can optionally enter the virtual environment with:

$ pipenv shell

This is optional because you can launch any command directly in the virtual environment like this:

$ pipenv run <your command>

For example, to start a Python script inside the virtual environment:

$ pipenv run python3 myscript.py

How does Pipenv work?

I don’t like black boxes. I want to know what’s going on. So let’s first inspect Pipfile. It should look like this:

[[source]]
 name = "pypi"
 url = "https://pypi.org/simple"
 verify_ssl = true
 [dev-packages]
 [packages]
 requests = "*"
 [requires]
 python_version = "3.6"

As you can see, Pipfile contains our top-level requirements without specifying a version number because we didn’t specify it. Now take a look at Pipfile.lock. It’s a big file, so I’ll show a snippet here to illustrate:

...
"requests": {
  "hashes": [
    "sha256:43999036bfa82904....",
    "sha256:b3f43d496c6da..."
  ],
  "index": "pypi",
  "version": "==2.23.0"
},
...

requests and all of its dependencies are listed, including the version numbers. Basically, this is an advanced version of the output of pip freeze. These are the versions we worked with. It’s the combination of packages that is guaranteed to work.

But where’s the virtual environment? If you look closely at the screenshot in @fig:pipenv_in_action, you see the virtualenv is created in ~/.local/share/virtualenvs/myenv-mAvwj65b/. We could have asked pipenv too with pipenv --venv.

Since we are already familiar with the structure of a virtual environment, you’ll notice that this one is no different. It’s just located somewhere else, outside of your project structure. This way, you don’t need to exclude it from your version control system too.

It won’t be reused if you create another virtual environment with the same name in another location. The hash in the directory name is, I presume, based on the path to your virtualenv. Pipenv uses it to find the correct one despite double names.

Additional Pipenv features

Let’s look at some more features that set Pipenv apart from using plain pip and virtualenvs.

Separating Development Packages

With pip, you are only able to separate development requirements from production requirements by using a second requirements file (e.g. requirements-dev.txt). With pipenv, you can simply use the --dev option:

$ pipenv install pytest --dev

Another developer that starts working on your project can install all requirements, including the developer requirements, using:

$ pipenv install --dev

Dependency Management

pipenv attempts to install all sub-dependencies required by your core dependencies. Conflicts might arise, though. If package A requires at least version 1.10 of package B, while package C requires version 1.9, you have a problem. pipenv can’t solve it for you, but it will warn you about it and refuse to continue by default. It also offers a nice insight into your dependencies with the pipenv graph command:

$ pipenv graph
pytest==5.3.5
  - attrs [required: >=17.4.0, installed: 19.3.0]
  - importlib-metadata [required: >=0.12, installed: 1.5.0]
    - zipp [required: >=0.5, installed: 3.1.0]
  - more-itertools [required: >=4.0.0, installed: 8.2.0]
  - packaging [required: Any, installed: 20.3]
    - pyparsing [required: >=2.0.2, installed: 2.4.6]
    - six [required: Any, installed: 1.14.0]
  - pluggy [required: >=0.12,<1.0, installed: 0.13.1]
    - importlib-metadata [required: >=0.12, installed: 1.5.0]
      - zipp [required: >=0.5, installed: 3.1.0]
  - py [required: >=1.5.0, installed: 1.8.1]
  - wcwidth [required: Any, installed: 0.1.8]
requests==2.23.0
  - certifi [required: >=2017.4.17, installed: 2019.11.28]
  - chardet [required: >=3.0.2,<4, installed: 3.0.4]
  - idna [required: >=2.5,<3, installed: 2.9]
  - urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.8]

Detection of Security Vulnerabilities

pipenv can scan your packages for security vulnerabilities. It does so by using the safety package. Enter:

$ pipenv check

and pipenv will scan your dependency graph for packages with known security vulnerabilities!