How to Run Tests Using Pytest: A Comprehensive Guide¶

In modern software development, automated testing is crucial for ensuring the reliability and functionality of your code. One of the most popular testing frameworks for Python is pytest.

This blog will provide an in-depth look at how to run tests using pytest, including testing a single file, multiple files, every file in the test repository, and providing guidelines for contributors to run tests reliably.

What is Pytest?¶

pytest is a testing framework for Python that makes it easy to write simple and scalable test cases. It supports fixtures, parameterized testing, and has a rich plugin architecture. pytest is widely used because of its ease of use and powerful features that help streamline the testing process.

Installation¶

To get started with pytest, you need to install it. You can install pytest using pip:

pip install pytest

Writing Your First Test¶

Before diving into running tests, let’s write a simple test. Create a file named test_sample.py with the following content:

def test_addition():
    assert 1 + 1 == 2

def test_subtraction():
    assert 2 - 1 == 1

In this example, we have defined two basic tests: test_addition and test_subtraction.

Running Tests¶

Running a Single Test File¶

To run a single test file, you can use the pytest command followed by the filename. For example, to run the tests in test_sample.py, use the following command:

pytest test_sample.py

The output will show the test results, including the number of tests passed, failed, or skipped.

Running Multiple Test Files¶

You can also run multiple test files by specifying their filenames separated by a space. For example:

pytest test_sample.py test_another_sample.py

If you have multiple test files in a directory, you can run all of them by specifying the directory name:

pytest tests/

Running All Tests in the Repository¶

To run all tests in the repository, navigate to the root directory of your project and simply run:

pytest

pytest will automatically discover and run all the test files that match the pattern test_*.py or *_test.py.

Test Discovery¶

pytest automatically discovers test files and test functions based on their naming conventions. By default, it looks for files that match the pattern test_*.py or *_test.py and functions or methods that start with test_.

Using Markers¶

pytest allows you to use markers to group tests or add metadata to them. Markers can be used to run specific subsets of tests. For example, you can mark a test as slow and then run only the slow tests or skip them.

import pytest

@pytest.mark.slow
def test_long_running():
    import time
    time.sleep(5)
    assert True

def test_fast():
    assert True

To run only the tests marked as slow, use the -m option:

pytest -m slow

Parameterized Tests¶

pytest supports parameterized testing, which allows you to run a test with different sets of input data. This can be done using the @pytest.mark.parametrize decorator.

import pytest

@pytest.mark.parametrize("a,b,expected", [
    (1, 2, 3),
    (2, 3, 5),
    (3, 5, 8),
])
def test_add(a, b, expected):
    assert a + b == expected

In this example, test_add will run three times with different sets of input data.

Fixtures¶

Fixtures are a powerful feature of pytest that allow you to set up some context for your tests. They can be used to provide a fixed baseline upon which tests can reliably and repeatedly execute.

import pytest

@pytest.fixture
def sample_data():
    return {"name": "John", "age": 30}

def test_sample_data(sample_data):
    assert sample_data["name"] == "John"
    assert sample_data["age"] == 30

Fixtures can be used to share setup and teardown code between tests.

Advanced Usage¶

Running Tests in Parallel¶

pytest can run tests in parallel using the pytest-xdist plugin. To install pytest-xdist, run:

pip install pytest-xdist

To run tests in parallel, use the -n option followed by the number of CPU cores you want to use:

pytest -n 4

Generating Test Reports¶

pytest can generate detailed test reports. You can use the --html option to generate an HTML report:

pip install pytest-html
pytest --html=report.html

This command will generate a file named report.html with a detailed report of the test results.

Code Coverage¶

You can use the pytest-cov plugin to measure code coverage. To install pytest-cov, run:

pip install pytest-cov

To generate a coverage report, use the --cov option followed by the module name:

pytest --cov=my_module

This command will show the coverage summary in the terminal. You can also generate an HTML report:

pytest --cov=my_module --cov-report=html

The coverage report will be generated in the htmlcov directory.

Best Practices for Writing Tests¶

Write Clear and Concise Tests: Each test should focus on a single piece of functionality.
Use Descriptive Names: Test function names should clearly describe what they are testing.
Keep Tests Independent: Tests should not depend on each other and should run in isolation.
Use Fixtures: Use fixtures to set up the context for your tests.
Mock External Dependencies: Use mocking to isolate the code under test from external dependencies.

Running Tests Reliably¶

For contributors and team members, it’s important to run tests reliably to ensure consistent results. Here are some guidelines:

Set Up a Virtual Environment: Use a virtual environment to manage dependencies and ensure a consistent testing environment.
```
python -m venv venv
source venv/bin/activate  # On Windows use `venv\Scripts\activate`
```
Install Dependencies: Install all required dependencies from the requirements.txt file.
```
pip install -r requirements.txt
```
Run Tests Before Pushing: Ensure all tests pass before pushing code to the repository.
Use Continuous Integration (CI): Set up CI pipelines to automatically run tests on each commit or pull request.

Example CI Configuration (GitHub Actions)¶

Here is an example of a GitHub Actions workflow to run tests using pytest:

name: Python package

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: '3.8'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Run tests
      run: |
        pytest

This configuration will run the tests on every push and pull request, ensuring that your codebase remains stable.

Conclusion¶

pytest is a powerful and flexible testing framework that makes it easy to write and run tests for your Python code. By following the guidelines and best practices outlined in this blog, you can ensure that your tests are reliable and your codebase is robust. Whether you are testing a single file, multiple files, or the entire repository, pytest provides the tools you need to automate and streamline your testing process.

Happy testing!