Django: Testing, Factories, and Data Seeding (Pytest, Mixer)

If you're a Django developer, you know how important it is to have a solid suite of tests to ensure that your code is working as expected. But what do you do when you inherit a huge codebase that has zero tests? That was the situation I found myself in at some point in 2022.

Coming from a project that was built using Elixir/Phoenix Live view and React, I was familiar with the concept of factories and the benefits they provide for test data management. I decided to bring this approach over to my Django project and was pleased with the results.

In this blog post, I'll explain how I set up Pytest for my Django project, defined factories using the Mixer library, and used them in my tests to create test data and run assertions. I'll also cover some advanced techniques for using factories, such as seeding the database. If you're looking for a more efficient and organized way to manage test data in your Django projects, read on to learn more about using factories with Pytest and Mixer.

Setting up Pytest

Before we can start using factories in our Django tests, we need to set up Pytest as our testing framework. Pytest is a powerful, feature-rich testing tool that is well-suited for testing Django applications. It's easy to install and has a number of plugins and features that can make testing Django projects a breeze.

To get started with Pytest in a Django project, you'll need to install the pytest and pytest-django packages. You can do this using pip:

pip install pytest pytest-django

If your team uses flake8 and black you might need to add some extra configs because pytest's script can also execute flake8 and black rules to check for linting issues. To do that you first need to install flake8 and black with the following script

pip install flake8 black

And then create a file named pytest.ini

Here is an example:

[pytest]
filterwarnings =
    error
    ignore::UserWarning
    ignore:function ham\(\) is deprecated:DeprecationWarning
DJANGO_SETTINGS_MODULE = server.settings
flake8-max-line-length = 120

There we provide some configs for pytest. (Please read the official docs from the link under references to understand more about the filterwarnings setting.). And then we provide the settings.py file that Django uses for global configs, in our case, a file called settings.py that is under a directory called server . And then lastly we provide the maximum length for flake8.

In general, this should be enough. You can simply run pytest from the command line to execute your tests.

By default, Pytest will discover and run all tests within the tests directory and its subdirectories. If you want to run a specific test or group of tests, you can use the -k flag to specify a test function or method name:

pytest -k test_function_name

2. Creating factories

Now that we have Pytest set up in our Django project, let's look at how we can use factories to create test data using Pytest fixtures. A fixture is a function that returns test data and can be used in multiple tests. It allows us to easily create realistic, customizable test data without having to manually set up complex data structures or write repetitive test setup code.

To use factories as Pytest fixtures in a Django project, we can use the Mixer library. Mixer is a powerful and easy-to-use library that allows us to define factories for Django models, forms, views, and other objects. It also has a number of features for customizing factory data and creating relationships between factories.

To install Mixer, you can use pip:

pip install mixer

Once Mixer is installed, we can start defining factories for our Django models. Here is an example of a factory for a Person model with a first_name and last_name field:

from mixer.backend.django import mixer

def person_factory(**kwargs):
    return mixer.blend(
        'server.app.models.Person',
        first_name=mixer.sequence(lambda n: f'first_name_{n}'),
        last_name=mixer.sequence(lambda n: f'last_name_{n}'),
        **kwargs,
    )

This factory uses Mixer's blend function to create a new Person object with default values for the first_name and last_name fields. The mixer.sequence function generates unique values for these fields using a given lambda function. We can also pass additional keyword arguments to the factory function to override the default values for any field.

We can use this factory as a Pytest fixture by decorating it with the @pytest.fixture decorator:

import pytest

@pytest.fixture
def person(db):
    return person_factory()

This fixture returns a new Person object created by the person_factory function. The db fixture provided by Pytest-Django is used to ensure that the Person object is saved to the database. We can then use this fixture in our Pytest tests like this:

def test_person_model(person):
    assert person.first_name == 'first_name_0'
    assert person.last_name == 'last_name_0'

This test uses the person fixture to get a Person object and then runs assertions on its fields.

By using Mixer factories as Pytest fixtures, we can easily create test data for our tests and reuse it across multiple tests. In the next section, we'll look at more advanced techniques for using factories in tests.

3. Advanced techniques

In the previous sections, we looked at how to set up Pytest in a Django project and how to use Mixer factories to create test data. In this section, we'll explore some advanced techniques for using factories in tests, including how to refactor our factory code to support multiple keyword arguments and how to create custom Pytest fixtures for creating test data.

First, let's refactor our Person factory to support multiple keyword arguments. This will allow us to easily override the default values for any field in our Person model:

from mixer.backend.django import mixer

def person_factory(**kwargs):
    defaults = {
        'first_name': mixer.sequence(lambda n: f'first_name_{n}'),
        'last_name': mixer.sequence(lambda n: f'last_name_{n}'),
    }
    defaults.update(kwargs)
    return mixer.blend('server.app.models.Person', **defaults)

This updated version of the person_factory function defines default values for the first_name and last_name fields and then updates them with any keyword arguments passed to the factory. This allows us to easily create a Person object with custom values for any field by calling the factory with keyword arguments like this:

person = person_factory(first_name='John', last_name='Doe')

Next, let's create a custom Pytest fixture for creating test data using our person_factory . Let's call itinsert . This fixture will allow us to easily create multiple Person objects with different values for any field by calling it with keyword arguments.

First, we need a container for all factories: here is how you can create one:

model_factories = []

Here is an example of the insert fixture:

import pytest

@pytest.fixture
def insert(db):
    def _insert(model_name, count, persist=True, **kwargs):
        model_factory = next(f for f in model_factories if f.__name__ == f"{model_name.lower()}_factory")
        objects = [model_factory(**kwargs) for _ in range(count)]
        if persist:
            for obj in objects:
                obj.save()
        return objects
    return _insert

This fixture takes three required arguments:

model_name: The name of the model as a string.
count: The number of objects to return in a list.
persist: A boolean indicating whether the objects should be saved to the database (defaults to True).

It also supports additional keyword arguments that will be passed to the model factory.

Here is an example of how we can use the insert fixture to create multiple Person objects with different values for the first_name field:

def test_person_model(insert, db):
    persons = insert(
        'app.Person',
        count=3,
        first_name=mixer.sequence(lambda n: f'first_name_{n}'),
    )
    assert persons[0].first_name == 'first_name_0'
    assert persons[1].first_name == 'first_name_1'
    assert persons[2].first_name == 'first_name_2'

This test uses the insert fixture to create three Person objects with unique first_name values and then runs assertions on the first_name fields of the objects. The db fixture provided by Pytest-Django is used to ensure that the Person objects are saved to the database.

And that is how you can easily seed dynamic data directly in tests using pytest and mixer.

4. Bonus

As developers, we often join projects where we are not provided with a test database dump to use for local development. In these situations, it can be useful to create a Django management command that generates dummy data for use in local development.

To create a management command for generating dummy data, we can use our Pytest fixtures and the Faker library. Faker is a library that generates fake data, such as names, addresses, and phone numbers, for use in test environments. To install Faker, you can use pip:

$ pip install Faker

Here is an example of a Django management command called db_seed that generates dummy data using our Pytest fixtures and Faker:

import faker
import logging
from django.core.management.base import BaseCommand

class Command(BaseCommand):
    help = 'Seeds the database with fake data'

    def add_arguments(self, parser):
        parser.add_argument('--quantity', type=int, default=1, help='Number of items to create')
        parser.add_argument('--model', type=str, required=True, help='Model to use for creating data')
        parser.add_argument('--attributes', type=str, help='Model attributes')

    def handle(self, *args, **options):
        fake = faker.Faker()
        quantity = options['quantity']
        model_name = options['model']
        model_attributes = options['attributes']

        # Get the model factory function from the Pytest fixtures
        model_factory = pytest.fixture('model_factory')

        for _ in range(quantity):
            # Use the model factory function to create dummy data
            obj = model_factory(model_name=model_name, **model_attributes)
            obj.save()
        logging.info(f'Successfully seeded {quantity} {model_name} objects')

This management command takes three arguments:

quantity: The number of items to create
model: The name of the model to use for creating data. This argument is required.
attributes: A string of Python-like keyword arguments to pass to the model factory.

It uses the Pytest fixtures and Faker to create quantity number of dummy objects for the specified model, using the specified attributes. The objects are then saved to the database.

To use this management command, you would run the following command from the command line:

$ python manage.py db_seed --quantity=5 --model=app.Person --attributes='first_name="John", last_name="Doe"'

This command would create five Person objects with the first_name of "John" and the last_name of "Doe".

The db_seed management command uses the built-in Python logging module to log a message when the seeding is complete. You can configure the logging module to output log messages to different places, such as a file or the console, by setting up a logging configuration in your Django settings.

Furthermore, for safety, you can even handle all operations to the database in transactions. That way you keep the integrity of the data and also secure the CLI from unexpected errors.

5. Conclusion

We have explored how to use Pytest, Mixer, and Faker to create factories for generating dummy data in Django tests. We have covered how to create a simple model factory, how to use Pytest fixtures to create test data, and how to create a Django management command for generating dummy data for local development.

Using these techniques, you can easily create test data for your Django applications, making it easier to write and run tests that rely on data being present in the database. This can help you ensure that your code is working correctly and reduce the risk of regressions as you make changes to your codebase.

I hope you have found this blog post helpful, and that you have learned some useful techniques for testing Django applications. If you have any questions or suggestions, please feel free to leave a comment below.

References: