Notes on Python Decorators
The GoF grouped some well-known design patterns under 3 categories. Under the structural patterns you can find The Decorator Pattern. A design pattern that allows us to change, or add, objects functionality at run-time, in a statically typed object-oriented programming languages.
Naming aside, Python Decorators are a different thing. Even though we get some similar things, and some times we might feel that they are the same, I think that not many people stress this enough: Python Decorators are NOT an implementation of the decorator pattern. Python decorators allow us to inject new behavior in an already existent functionality, at definition time.
Python Decorators for Functions and Methods
A.K.A. wrappers, allow us to change the functionality of a function or method without actually changing the original callable code.
To do this we need to define a callable that takes a callable, the one with the behavior we want to change, and returns a new callable with the desired behavior.
The simplest example I can come up with would be:
def greeting(name):
return f"Hello, {name}."
def html_title(func):
def wrapper(name):
return f"<h1>{func(name)}</h1>"
return wrapper
greeting = html_title(greeting)
print(greeting("Ramiro"))
# '<h1>Hello, Ramiro.</h1>'
And that’s mostly it, that is the starting point. There you have it, a Python Decorator.
Let’s break it down:
…define a callable that takes a callable…
That would be our html_title function. It’s callable it’s just a function that takes another callable, our simple function , as the parameter.greeting
…and returns a new callable…
Now, this is an important part. As you can see we changed the behavior by defining a new function that uses the function whose behavior we want to change. Note that, in order for this to happen the parameters for both functions have to match. A better way to do this would be to generalize the wrapper’s parameters using the well-known *args, **kwargs
def greeting(name):
return f"Hello, {name}."
def html_title(func):
def wrapper(*args, **kwargs):
return f"<h1>{func(*args, **kwargs)}</h1>"
return wrapper
greeting = html_title(greeting)
print(greeting('Ramiro'))
# '<h1>Hello, Ramiro.</h1>'
We can beautify this whole thing with the help of @, as follows:
def html_title(func):
def wrapper(*args, **kwargs):
return f"<h1>{func(*args, **kwargs)}</h1>"
return wrapper
@html_title
def greeting(name):
return f"Hello, {name}."
print(greeting("Ramiro"))
# '<h1>Hello, Ramiro.</h1>'
Even though I’m keeping this simple, I now want to change examples in order for you to have a better sense of what you can accomplish using python decorators.
One typical example is to enforce access control and/or authentication:
class NoPrivilege(Exception):
pass
current_user = {'name': 'Ramiro', 'profile': 'admin'}
def make_secure(func):
def wrapper(*args, **kwargs):
'''Not Eminem but still a wrapper.'''
if current_user['profile'] in ['admin', 'manager']:
return func(*args, **kwargs)
raise NoPrivilege(f"{current_user['profile']} is not allow to see this!")
return wrapper
@make_secure
def get_sensible_server_data(key='root_password'):
'''Returns super sensitive data. Deadly on the wrong hands!'''
if key == 'root_password':
return "r007_P@sw0rd!"
return "Some other sensible data"
print(get_sensible_server_data())
# r007_P@sw0rd!
current_user = {'name': 'John', 'profile': 'guest'}
print(get_sensible_server_data())
# Traceback (most recent call last):
# ...
# NoPrivilege: guest is not allow to see this!
The nature of our wrappers
When we decorate a function we lose some data from the original function. As we’ve seen the @ syntax sugar is just a shortcut for the assignment. Even though we gave it the same name, the actual function’s name is different.
get_sensible_server_data.__name__
# 'wrapper'
get_sensible_server_data.__doc__
# 'Not Eminem but still a wrapper.'
If you’re like me, you don’t like this. Luckily for us, Python has our backs: Enter the functools.wraps decorator.
We have to tweak a little bit our code. First, we need to import the required decorator, and then just use it to decorate our wrapper function.
from functools import wraps
def make_secure(func):
@wraps(func)
def wrapper(*args, **kwargs):
'''Not Eminem, but a wrapper though. '''
if current_user['profile'] in ['admin', 'manager']:
return func(*args, **kwargs)
raise NoPrivilege(f"{current_user['profile']} is not allow to see this!")
return wrapper
Now, we got that back 🙂 👍 :
get_sensible_server_data.__doc__
# 'Returns super sensitive data. Deadly on the wrong hands!'
get_sensible_server_data.__name__
# 'get_sensible_server_data'
Decorators with parameters
You may have seen this before. I bet you did when doing some TDD or just unittesting a class. This is a widely used real-life example of Python decorators using parameters:
from some_module import SomeClass
from unittest.mock import patch
@patch.object(SomeClass, 'method_to_mock', return_value=None)
def test_some_class_method(self, patched_object):
# ... some more code
So, as you can see this type of decorator is not that odd as one may think. But, how does it work? How can you define your own parameterized python decorators?
The first thing we need to understand is that a python decorator takes one argument, and one argument only, the one function we want to wrap in order to modify its behavior. There is no way to pass additional arguments. Isn’t it?
The solution trick is to use a Decorator Factory:
We will make a function that takes an arbitrary amount of arguments and returns a decorator.
Let’s say that you have two functions, both of which do the same thing, and you would like to compare them in terms of running time, in order to decide which one to use.
You could actually create a separated script, import both functions and create a little program that computes the running time of a given function; but you would have to do it or change it every time you want to measure how long does it take for each specific function to compute.
Instead, using what we have seen so far in this post, we could create a factory for a timer decorator. This factory will receive a parameter that defines how many times we would like to run this functionality that we want to time, and would create a timer decorator accordingly.
from time import perf_counter
def decorator_factory(loops_amount=1):
if loops_amount <= 0:
raise ValueError('Loops amount has to be bigger than 0.')
def decorator(func):
def wrapper(*args, **kwargs):
total_elapsed = 0
for _ in range(loops_amount):
start = perf_counter()
func(*args, **kwargs)
end = perf_counter()
total_elapsed += end - start
avg_run_time = total_elapsed/loops_amount
return avg_run_time
return wrapper
return decorator
As we can see, the decorator_factory() function will create and return an actual decorator.
The only thing that we have to be careful about is the fact that this is not a decorator but a creator of one, so in order to use it we need to explicitly invoke it by adding parentheses, as you would do with any other normal function :
@decorator_factory() # even if it has default values
def some_function_to_decorate():
pass
We will actually decorate some_function_to_decorate() with what decorator_factory() returns.
To see this in action, lets do a list comprehension vs non-pythonic for-loop, just for the sake of an example:
@decorator_factory(100)
def forloop_odds(up_to=1_000_000):
'''The c/c++ way.'''
odd_list = []
for n in range(1, up_to):
if n % 2 != 0:
odd_list.append(n)
@decorator_factory(100)
def listcomp_odds(up_to=1_000_000):
'''The Pythonic way.'''
return [n for n in range(1, up_to) if n % 2 != 0]
# Lets now compute the difference
avg1 = listcomp_odds()
avg2 = forloop_odds()
print(avg2 / avg1)
I actually ran it a few times (n=50) to compute the average difference, and it took a while but I finally got that for this particular example, the pythonic-way is approximately 25% faster.