How to use functions as Mocks in Python tests

How to use functions as Mocks in Python tests

One common question about testing is how to avoid calling “heavy services” during test execution.

If there is a need to call services like Redis, Docker, MySQL, RabbitMQ, or any other service for every test, it might be necessary to install them on the test server. However, this approach consumes a lot of resources and increases the execution time, which is not ideal for developers.

Another important consideration is to avoid calling real URLs. Since we have no control over those addresses, they might not be available at all times.

To address these issues, we can make use of Python Mocks. With mocks, we can verify that our code makes a call to an external URL and simulate edge cases, such as the URL not being available or returning a successful response.

In this example project, I will use Poetry as the package manager, the requests package to make calls to an external URL, and pytest with unittest as testing utilities.

The following code has been tested in Python 3.11

Using Poetry to setup the project

Poetry is a package manager easy to use. You can download it from https://python-poetry.org/ . Here is the list of dependencies to use.

pyproject.toml
[tool.poetry.dependencies]
python = "^3.10"
requests = "^2.31.0"


[tool.poetry.group.dev.dependencies]
black = "^23.3.0"
pytest = "^7.4.0"
pytest-mock = "^3.11.1"

We have black, a code formatter, pytest, and pytest-mock to provide a simple way to create mocks in Python. These will be included as development dependencies. We do not need to deploy them to our production code.

For our production dependencies, we are going to use Python 3.11 and requests, a library for making HTTP calls to any URL.

Creating a Python script to call an API

Like any Python script, we are doing something simple. I will proceed to make three of the most common API calls: a GET call, a POST call, and another POST call with data.

If I execute this script, it will work as long as the JsonPlaceholder URL is available.

main.py
import requests


todos_single_url = "https://jsonplaceholder.typicode.com/todos/1"
posts_url = "https://jsonplaceholder.typicode.com/posts"


def call_single_endpoint():
    request = requests.get(todos_single_url)
    return request.json()


def call_post_endpoint():
    request = requests.post(posts_url)
    return request.json()


def call_post_endpoint_with_data(data):
    request = requests.post(posts_url, data)
    return request.json()

Using mocks to override the function behaviour

Let’s frequently use the patch method from unittest.mock. With this decorator, we can patch the desired method and turn it into a mock. In the process, it will provide us with many utility functions that allow us to check if the method has been called, which parameters were passed, and even return a fake value as desired.

In this case, the function we want to mock is requests.get. Make sure you import get from requests and include the complete package path, including the package name.

The functions within the mocks are so diverse that sometimes you can choose between using assert or using the internal assertions within the mocks.

Check if a function was called once, using the decorator approach

If we want to use assert from Python. The mock comes with a function that returns true or false, named called_once .

See how we use a decorator, above our test test_call_single_endpoint_mock . Using this mode, we avoid code indentation.

tests.py
from unittest.mock import patch
from python_mocking import call_single_endpoint


@patch("requests.get")
def test_call_single_endpoint_mock(mocked_requests_get):
    call_single_endpoint()
    assert mocked_requests_get.called_once

Check if the function was called once, using the with keyword

We can also choose not to use the decorator. In this case, we don’t need a patch. Instead, we simply add an extra line with indentation. Personally, I prefer the decorator version, but it never hurts to learn the alternative approach.

tests.py
from python_mocking import call_single_endpoint


def test_call_single_endpoint_mock_without_decorator():
    with patch("requests.get") as mocked_requests_get:
        call_single_endpoint()
        assert mocked_requests_get.called_once

Check if the function was called, using the mocker parameter

You can use mocker as a parameter to create a mock of the function that you want to override. This method also helps to avoid adding another level of indentation. The advantage of using this parameter is that you can perform multiple patching of mocks within the same method, without accumulating decorators.

tests.py
from python_mocking import call_single_endpoint


def test_call_single_endpoint_mock_with_mocker_param(mocker):
    mocked_requests_get = mocker.patch("requests.get")
    call_single_endpoint()
    assert mocked_requests_get.called_once

Check if the function was called with specific parameters

We have two versions of these assertions. We can check if a function was called with specific parameters using the mock’s function called assert_called_with. There is another variation of this assertion called assert_called_once_with which ensures that the function was called only once.

tests.py
from unittest.mock import patch
from python_mocking import call_post_endpoint_with_data, posts_url


@patch("requests.post")
def test_assert_called_with(mocked_requests_post):
    data = {"userId": 3}
    call_post_endpoint_with_data(data)
    mocked_requests_post.assert_called_with(posts_url, data)


@patch("requests.post")
def test_assert_called_once_with(mocked_requests_post):
    data = {"userId": 3}
    call_post_endpoint_with_data(data)
    mocked_requests_post.assert_called_once_with(posts_url, data)

Grouping tests

If we want to have better organization for our tests, we need to group them inside a class. You can even combine grouped tests and non-grouped tests within the same file.

For the following example, we will tell the mock object what to return. This can be useful if you want to force a specific flow to check how your functions behave when they are mocked.

Remember that in a class, the first parameter of every function will always be self. So, the second parameter will become your function as a mock.

tests.py
from unittest.mock import patch
from python_mocking import call_single_endpoint


class TestRequests:
    @patch("requests.get")
    def test_call_and_stubbed_response(self, mocked_requests_get):
        mocked_requests_get.return_value.json.return_value = {"foo": "bar"}
        call_single_endpoint()
        assert mocked_requests_get.called_once
        assert mocked_requests_get.return_value.json.return_value == {
            "foo": "bar"}

Check consecutive calls from a method

Inside of the mocked function, there is a method called side_effect. This is useful when you want to test consecutive method calls and test its returns.

Also, the side_effect method is used to send Exceptions as response.

tests.py
from unittest.mock import patch
from python_mocking import call_single_endpoint


class TestRequests:
  @patch("requests.get")
    def test_call_and_response_different_values(self, mocked_requests_get):
        mocked_requests_get.return_value.json.side_effect = [
            {"fruit": "orange"},
            {"fruit": "apple"},
        ]
        first_result = call_single_endpoint()
        assert mocked_requests_get.return_value.json.call_count == 1
        assert first_result == {"fruit": "orange"}
        second_result = call_single_endpoint()
        assert second_result == {"fruit": "apple"}
        assert mocked_requests_get.return_value.json.call_count == 2

Check that a call is equal to, using call

If we prefer not to use assert_called_with or assert_called_once_with, we can instead utilize the call function provided by unittest. This function returns a call to Mock or MagicMock, allowing us to test if the calls at specific points in time match what the mock has recorded.

tests.py
from unittest.mock import patch, call
from python_mocking import call_single_endpoint, todos_single_url


class TestRequests:
    @patch("requests.get")
    def test_call_args(self, mocked_requests_get):
        call_single_endpoint()
        assert mocked_requests_get.call_args == call(todos_single_url)

Check the n parameter of a function was a specific value

This is a variation of a test where we want to know if a parameter in a specific place of a function was equals to a certain value. Since we have the parameters as a list, we can access to its index.

In this example we can obtain all parameters of a mock call inside of call_args.

test.py
from unittest.mock import patch
from python_mocking import call_post_endpoint_with_data


class TestRequests:
  @patch("requests.post")
    def test_call_post_endpoint_with_data(self, mocked_requests_post):
        mocked_requests_post.return_value.json.return_value = {"foo": "bar"}
        call_post_endpoint_with_data({"userId": 2})
        second_param_of_call = mocked_requests_post.call_args[0][1]
        assert mocked_requests_post.called_once
        assert second_param_of_call == {"userId": 2}

Final result

This is the final code containing all the tests that I have made. It doesn’t cover 100% of all application cases, but I hope those are enough to show and describe.

The link to the repository will be in the Recommendations section.

tests.py
from unittest.mock import patch, call
from python_mocking import (
    call_single_endpoint,
    call_post_endpoint,
    call_post_endpoint_with_data,
    posts_url,
    todos_single_url,
)


@patch("requests.get")
def test_call_single_endpoint_mock(mocked_requests_get):
    call_single_endpoint()
    assert mocked_requests_get.called_once


def test_call_single_endpoint_mock_without_decorator():
    with patch("requests.get") as mocked_requests_get:
        call_single_endpoint()
        assert mocked_requests_get.called_once


def test_call_single_endpoint_mock_with_mocker_param(mocker):
    mocked_requests_get = mocker.patch("requests.get")
    call_single_endpoint()
    assert mocked_requests_get.called_once


@patch("requests.post")
def test_assert_called_with(mocked_requests_post):
    data = {"userId": 3}
    call_post_endpoint_with_data(data)
    mocked_requests_post.assert_called_with(posts_url, data)


@patch("requests.post")
def test_assert_called_once_with(mocked_requests_post):
    data = {"userId": 3}
    call_post_endpoint_with_data(data)
    mocked_requests_post.assert_called_once_with(posts_url, data)


class TestRequests:
    @patch("requests.get")
    def test_call_and_stubbed_response(self, mocked_requests_get):
        mocked_requests_get.return_value.json.return_value = {"foo": "bar"}
        call_single_endpoint()
        assert mocked_requests_get.called_once_with(todos_single_url)
        assert mocked_requests_get.return_value.json.return_value == {"foo": "bar"}


    @patch("requests.get")
    def test_call_args(self, mocked_requests_get):
        call_single_endpoint()
        assert mocked_requests_get.call_args == call(todos_single_url)


    @patch("requests.get")
    def test_call_and_response_different_values(self, mocked_requests_get):
        mocked_requests_get.return_value.json.side_effect = [
            {"fruit": "orange"},
            {"fruit": "apple"},
        ]
        first_result = call_single_endpoint()
        assert mocked_requests_get.return_value.json.call_count == 1
        assert first_result == {"fruit": "orange"}
        second_result = call_single_endpoint()
        assert second_result == {"fruit": "apple"}
        assert mocked_requests_get.return_value.json.call_count == 2


    @patch("requests.post")
    def test_call_post_endpoint(self, mocked_requests_post):
        mocked_requests_post.return_value.json.return_value = {"foo": "bar"}
        call_post_endpoint()
        assert mocked_requests_post.called_once
        assert mocked_requests_post.return_value.json.return_value == {"foo": "bar"}


    @patch("requests.post")
    def test_call_post_endpoint_with_data(self, mocked_requests_post):
        mocked_requests_post.return_value.json.return_value = {"foo": "bar"}
        call_post_endpoint_with_data({"userId": 2})
        second_param_of_call = mocked_requests_post.call_args[0][1]
        assert mocked_requests_post.called_once
        assert second_param_of_call == {"userId": 2}

Recommendations

You can see the repository in this link .

For more information about unittest.mock check its documentation .

My posts are not AI generated, they might be only AI corrected. The first draft is always my creation

Tags

#python #testing #mocking #pytest

Author

Written by Helmer Davila

In other languages

Útil para evitar llamadas reales hacia API o servicios

Cómo usar funciones como Mocks para tests en Python

 C'est utile pour éviter les appels réels vers une API ou un service.

Comment utiliser des fonctions comme des mocks dans les tests de Python

Related posts

I was trying to test some global variables with Python, specially for a script which contain global variables. And after trying and failing I think that I can show you the most simple version to do it.

Mocking global variables in Python tests