Measuring Test Coverage

Test coverage measures how much of your code is executed while running tests. It provides valuable insights into untested parts of your codebase, helping ensure robustness and reliability.

Introduction to Test Coverage

Test coverage answers questions such as:

• Which parts of the code are tested?
• Are there any untested paths or branches?
• How comprehensive are the tests?

A high test coverage percentage indicates better-tested software, but remember, 100% coverage does not guarantee bug-free code.

Using pytest-cov for Measuring Coverage

pytest-cov is a popular plugin for measuring test coverage in pytest. It integrates seamlessly and provides detailed coverage reports.

Installing pytest-cov

Install the plugin via pip:

pip install pytest-cov

Running Tests with Coverage

To measure test coverage, use the --cov option:

pytest --cov=your_module_name

This generates a coverage summary in the terminal.

Generating Coverage Reports

pytest-cov allows you to generate various types of coverage reports for deeper analysis.

HTML Report

Generate a detailed HTML report for easy visualization:

pytest --cov=your_module_name --cov-report=html

After running the command, open the generated htmlcov/index.html file in a browser.

XML Report

Useful for integrating with CI/CD tools:

pytest --cov=your_module_name --cov-report=xml

This generates a coverage.xml file in the project directory.

Terminal Report

For a detailed report in the terminal:

pytest --cov=your_module_name --cov-report=term-missing

This highlights missing lines of code directly in the terminal output.

Example: Measuring Coverage

Example Test File (test_example.py)

# example_module.py
def add(a, b):
    return a + b
 
def subtract(a, b):
    return a - b
 
# test_example.py
from example_module import add, subtract
 
def test_add():
    assert add(2, 3) == 5
 
def test_subtract():
    assert subtract(5, 3) == 2

Running Coverage

pytest --cov=example_module --cov-report=term-missing

Output

---------- coverage: ... ----------
Name               Stmts   Miss  Cover
--------------------------------------
example_module.py      4      0   100%
--------------------------------------
TOTAL                  4      0   100%

Tips for Effective Coverage Measurement

• Focus on Critical Areas: Aim for high coverage in critical parts of the application.
• Analyze Missed Lines: Review uncovered lines to identify missing tests or dead code.
• Avoid Coverage Overkill: Balance achieving high coverage with meaningful tests.

Measuring test coverage with pytest-cov provides valuable insights into the quality of your tests. Use coverage reports to improve test completeness and ensure robust code. By combining pytest-cov with other pytest features, you can enhance your testing strategy effectively.