What Is Concurrency

Concurrency is working on multiple things at the same time. In Python, this can be done in several ways:

  • With threading, by letting multiple threads take turns.
  • With multiprocessing we’re using multiple processes. This way we can truly do more than one thing at a time using multiple processor cores. This is called parallelism.
  • Using asynchronous IO, firing off a task and continuing to do other stuff, instead of waiting for an answer from the network or disk.
  • By using distributed computing, which basically means using multiple computers at the same time

The difference between threads and processes

A thread is an independent sequence of execution, but it shares memory with all the other threads belonging to your program. A Python program has, by default, one main thread. You can create more of them and let Python switch between them. This switching happens so fast that it appears like they are running side by side at the same time.

Single-threaded vs multithreading
Single-threaded vs multithreading

A process is also an independent sequence of execution. Unlike threads, a process has its own memory space that is not shared with other processes. A process can clone itself, creating two or more instances. The following image illustrates this:


Asynchronous IO is not threading, nor is it multiprocessing. In fact, it is a single-threaded, single-process paradigm. I will not go into asynchronous IO in this chapter.

Concurrency by using Threads

Most software is I/O bound and not CPU bound. In case these terms are new to you:

I/O bound  software
Software that is mostly waiting for input/output operations to finish, usually when fetching data from the network or from storage media
CPU bound  software
Software that uses all CPU power to produce the needed results. It maxes out the CPU.

While waiting for answers from the network or from disk, you can keep other parts running by using multiple threads. A thread is an independent sequence of execution. Your Python program has, by default, one main thread. But you can create more of them and let Python switch between them. This switching happens so fast that it appears like they are running side by side at the same time.

Unlike other languages, Python threads don’t run at the same time; they take turns instead. The reason for this is a mechanism in Python called the Global Interpreter Lock (GIL). This, together with the threading library, is explained further on in this chapter.

The takeaway is that threads will make a big difference for I/O bound software but are useless for CPU bound software. Why is that? It’s simple. While one thread is waiting for a reply from the network, other threads can continue running. If you make a lot of network requests, threads can make a tremendous difference. If your threads are doing heavy calculations instead, they are just waiting for their turn to continue. Threading would only introduce more overhead.

Concurrency by using Asyncio

Asyncio is a relatively new core library in Python. It solves the same problem as threading: it speeds up I/O bound software, but it does so differently. I’m going to admit right away I’m not a fan of asyncio in Python yet. It’s fairly complex, especially for beginners. Another problem I encountered is that the asyncio library has evolved a lot in the past years. Tutorials and example code on the web is often outdated. That doesn’t mean it’s useless, though. It’s a powerful paradigm that is used in many high-performance applications.

Concurrency by using Multiprocessing

If your software is CPU bound, you can often rewrite your code in such a way that you can use more processors at the same time. This way, you can linearly scale the execution speed. This is called parallelism.

Not all algorithms can be made to run in parallel. It is impossible to parallelize a recursive algorithm, for example. But there’s almost always an alternative algorithm that can work in parallel just fine.

There are two ways of using more processors:

  • Using multiple processors and/or cores within the same machine. In Python, this can be done with the multiprocessing library.
  • Using a network of computers to use many processors, spread over multiple machines. We call this distributed computing.

The multiprocessing library, unlike the threading library, bypasses the Python Global Interpreter Lock. It does so by actually spawning multiple instances of Python. Instead of threads taking turns within a single Python process, you now have multiple Python processes running your code at once.

The multiprocessing library is very similar to the threading library. A question that might arise is: why should you even consider threading? The answer can be guessed. Threading is ‘lighter’: it requires less memory since it only requires one running Python interpreter. Spawning new processes has its overhead as well. So if your code is I/O bound, threading is likely good enough.

Distributed Computing

Once you implemented your software to work in parallel, it’s a small step further to use distributed computing with the distributed computing platforms, like Hadoop. By leveraging cloud computing platforms, you can scale up with relative ease these days. You can process huge datasets in the cloud, and use the results locally, for example. With this hybrid way of operating, you can save considerable amounts of money since computing power in the cloud is costly.