Python Functions: Advanced Concepts

Do you know how to force keyword or positional arguments, accept any number of arguments, create a function decorator, create anonymous functions, or unpack a list or dictionary into a function’s arguments? In this article, Python functions advanced concepts, we’ll cover some of the more advanced concepts surrounding functions. I’ve put these in a separate article because I don’t want to confuse beginners with these concepts.

You may want to read our introduction to Python functions first.

Forced keyword arguments

Keyword arguments have a number of advantages:

  • You’re not forced to a particular order in which you supply your arguments—the name matters, not the position.
  • Keyword arguments provide clarity. Without looking up the function itself, you can often guess what the argument is used for by looking at the names.

That’s nice, but you probably already knew these things. What you might not know is that you can also force keyword arguments. The details are described in PEP 3102, but it comes down to using an asterisk before the arguments you want to force as keyword arguments. Or, as shown below, before everything, forcing all arguments to be keyword arguments:

>>> def f(*, a, b):
...     print(a, b)
...
>>> f(1, 2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    f(1, 2)
    ~^^^^^^
TypeError: f() takes 0 positional arguments but 2 were given
>>> f(a=1, b=2)
1 2Code language: Python (python)

This advanced function trick could come in handy when you’re developing some kind of library. Perhaps you don’t want to commit to a certain order of arguments (yet). In such cases, this is the way to force users of your library to use named arguments, making the order irrelevant.

Positional-only parameters

The opposite is possible too. Since Python 3.8, you can put a slash (/) in the parameter list. Everything before the slash can only be passed by position, not by name (see PEP 570). You can combine both in one function:

>>> def greet(name, /, greeting='Hello', *, punctuation='!'):
...     print(f'{greeting}, {name}{punctuation}')
...
>>> greet('Erik')
Hello, Erik!
>>> greet('Erik', 'Hi', punctuation='?')
Hi, Erik?
>>> greet(name='Erik')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    greet(name='Erik')
    ~~~~~^^^^^^^^^^^^^
TypeError: greet() got some positional-only arguments passed as keyword arguments: 'name'Code language: Python (python)

In this function, name is positional-only, greeting can be passed either way, and punctuation is keyword-only. Positional-only parameters are useful when the name of a parameter doesn’t mean anything to the caller, or when you want to be free to rename it later without breaking anyone’s code. Many built-in functions work like this. For example, you can’t call len(obj=[1, 2]).

Using * and ** for function arguments

Some functions require a long list of arguments. Although this should be avoided altogether, for example, by using data classes, it’s not always up to you. In such cases, the second-best option is to create a dictionary with all the named arguments and pass that to the function instead. It will generally make your code more readable. You can unpack a dictionary for use with named keywords by using the ** prefix:

>>> def f(a, b):
...     print(a, b)
...
>>> args = { "a": 1, "b": 2 }
>>> f(**args)
1 2Code language: Python (python)

Similarly, we can use a single * to unpack a list and feed its content as positional arguments to a function:

>>> def f(a, b, c):
...    print(a, b, c)
...
>>> l = [1, 2, 3]
>>> f(*l)
1 2 3Code language: Python (python)

Collecting arguments with *args and **kwargs

The same * and ** also work the other way around, in a function definition. There, they collect arguments instead of unpacking them. A parameter with a single * collects all remaining positional arguments into a tuple:

>>> def total(*numbers):
...     print(numbers)
...     return sum(numbers)
...
>>> total(1, 2, 3)
(1, 2, 3)
6Code language: Python (python)

A parameter with ** collects all remaining keyword arguments into a dictionary:

>>> def show(**options):
...     print(options)
...
>>> show(color='red', size=3)
{'color': 'red', 'size': 3}Code language: Python (python)

By convention, these parameters are called *args and **kwargs, but the names are up to you. You can combine them with regular parameters, as long as the regular ones come first:

>>> def log(message, *args, **kwargs):
...     print(message, args, kwargs)
...
>>> log('hi', 1, 2, level='debug')
hi (1, 2) {'level': 'debug'}Code language: Python (python)

You’ll use this a lot when writing decorators, as you’ll see next.

Decorating your functions

Decorators are wrappers around a function that modify the behavior of the function in a certain way. There are many use-cases for decorators, and you may have used them before when working with frameworks like Flask. 

Let’s create our own decorator; it’s simpler than you might expect and might come in handy someday:

import functools

def print_arguments(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print("Arguments for", func.__name__, "are", args, kwargs)
        return func(*args, **kwargs)

    return wrapper

@print_arguments
def add_one(x):
    """Add one to x."""
    return x + 1

@print_arguments
def greet(name, greeting="Hello"):
    return f"{greeting}, {name}!"

print(add_one(1))
print(greet("Erik", greeting="Hi"))
print(add_one.__name__, "-", add_one.__doc__)Code language: Python (python)

Inside print_arguments, we define a wrapper function. This function prints the arguments and the name of the called function. Next, it executes the actual function and returns its result as if the function was called regularly. With @print_arguments we apply our decorator to a function.

Two details make this decorator reusable. Because the wrapper accepts *args and **kwargs and passes them on, it works for any function, no matter how many arguments it takes. And functools.wraps copies the name and docstring of the original function to the wrapper. Without it, add_one.__name__ would return 'wrapper', which makes debugging and documentation tools a lot less helpful.

The output of our little script will be:

Arguments for add_one are (1,) {} 2 Arguments for greet are (‘Erik’,) {‘greeting’: ‘Hi’} Hi, Erik! add_one – Add one to x.

Anonymous functions

Sometimes, naming a function is not worth the trouble. An example is when you’re sure the function will only be used once. For such cases, Python offers us anonymous functions, also called lambda functions.

A lambda function can be assigned to a variable, creating a concise way of defining a function:

>>> add_one = lambda x: x + 1
>>> add_one(3)
4Code language: Python (python)

It gets more interesting when you need to use a function as an argument. In such cases, the function is often used only once. As you may know, map applies a function to all elements of an iterable object. We can use a lambda when calling map:

>>> numbers = [1, 2, 3, 4]
>>> times_two = map(lambda x: x * 2, numbers)
>>> list(times_two)
[2, 4, 6, 8]Code language: Python (python)

You’ll see this pattern in a lot of code. Most Python programmers would write this particular example as a list comprehension, though, because it’s easier to read:

>>> numbers = [1, 2, 3, 4]
>>> [x * 2 for x in numbers]
[2, 4, 6, 8]Code language: Python (python)

Lambdas are still handy wherever a function expects another function as its argument, for example as the key when sorting: sorted(people, key=lambda p: p['age']).

Related articles

Leave a Comment