Skip to main content

Assertions & Reporting

What's inside
  • ✅ Write clear assertions using plain Python’s assert
  • 🔍 Explore rich output: diffs, timings, paths, and full structure comparisons
  • 🧯 Catch and verify exceptions explicitly with catched()

Introduction​

Assertions describe what should happen. Reporting shows what actually happened.

Together, they form the feedback loop that makes tests valuable: you write a check, run the test, and instantly see whether the test passed, failed, or was skipped, providing just enough detail to fix what’s wrong or move on with confidence.

Assertions: Plain Python, No Surprises​

In Vedro, you write assertions using Python’s built-in assert statement: no new syntax, no learning curve. Just the assert you already know, used exactly as you expect.

import vedro

class Scenario(vedro.Scenario):
    subject = 'greet user'

    def when_greeting_user(self):
        self.greeting = greet_user('Alice')

    def then_greeting_should_be_correct(self):
        assert self.greeting == 'Hello Alice, welcome back!'

When an assertion fails, you receive immediate, precise feedback with a clean, colorized diff that highlights the difference between expected and actual values:

 ✗ greet user
   ✔ when greeting user
   ✗ then greeting should be correct
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /app/scenarios/greet_user.py:14 in then_greeting_should_be_correct           │
│                                                                              │
│   11         self.greeting = greet_user('Alice')                             │
│   12                                                                         │
│   13     def then_greeting_should_be_correct(self):                          │
│ ❱ 14         assert self.greeting == 'Hello Alice, welcome back!'            │
│   15                                                                         │
╰──────────────────────────────────────────────────────────────────────────────╯
AssertionError
>>> assert actual == expected
  - 'Hello Alice, welcome back!'
  + 'Hi Alice, welcome!'

No guesswork. No extra logging. Just the information you need, right when you need it.

Use Any Assertion You Need​

Vedro gives you the full power of Python. Use any operator or expression to craft exactly the assertion you need — whether you're comparing values, checking membership, or verifying custom logic.

import vedro

class Scenario(vedro.Scenario):
    subject = 'search user'

    def when_searching_for_user(self):
        self.result = search_users('Bob')

    def then_user_should_be_in_result(self):
        assert 'Bob' in self.result

Failures are reported clearly and with helpful context:

 ✗ search user
   ✔ when searching for user
   ✗ then user should be in result
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /app/scenarios/greet_user.py:14 in then_user_should_be_in_result             │
│                                                                              │
│   11         self.result = search_users('Bob')                               │
│   12                                                                         │
│   13     def then_user_should_be_in_result(self):                            │
│ ❱ 14         assert 'Bob' in self.result                                     │
│   15                                                                         │
╰──────────────────────────────────────────────────────────────────────────────╯
AssertionError
>>> assert member in container
  'Bob'
  ['Alice', 'Charlie']

Custom Messages in Assertions?​

Python supports custom messages in assertions:

assert user.id == 42, "Expected user ID to be 42"

This works in Vedro too — but most of the time, it’s unnecessary.

With descriptive step names, structured output, and clear diffs, you already get all the context you need when something fails. Adding messages often duplicates information or leads to less readable code.

You also avoid the common trap of writing inverted logic like:

assert not error_occurred, "An error occurred"

Instead, focus on writing clean, direct checks:

def then_user_should_have_correct_id(self):
    assert user.id == 42

This keeps your scenarios readable, and Vedro's reporter will make sure you still see exactly what went wrong.

Testing Exceptions Explicitly​

Following Python’s principle that «Explicit is better than implicit», Vedro provides a simple, explicit way to capture and verify exceptions using the catched() context manager:

import vedro
from vedro import catched

class Scenario(vedro.Scenario):
    subject = 'divide by zero'

    def when_dividing_by_zero(self):
        with catched(Exception) as self.exc_info:
            1 / 0

    def then_exception_should_be_raised(self):
        assert self.exc_info.type is ZeroDivisionError
        assert str(self.exc_info.value) == 'division by zero'

The catched() block captures the exception, and it’s up to you to assert its type and message. This keeps tests honest, readable, and fully under your control.

tip

Learn more in the Testing Exceptions guide.

Reporting: Readable and Customizable​

By default, Vedro uses the RichReporter — a powerful, highly customizable reporter that adds structure, color, and metadata to your test output.

Show Timings​

Curious how long each test takes to run? You can enable timing output to display how much time is spent in each scenario.

$ vedro run --show-timings

Output:

Scenarios
* auth / login
 ✔ login as registered user (0.21s)
 ✔ try to login as nonexisting user (0.11s)
 ✔ try to login with incorrect password (0.22s)
 
# 3 scenarios, 3 passed, 0 failed, 0 skipped (0.53s)

To include step-level timings, combine it with --show-steps. This is useful for profiling slow steps or understanding bottlenecks in longer flows:

$ vedro run --show-timings --show-steps
Scenarios
* auth / login
 ✔ login as registered user (0.36s)
   ✔ given user (0.19s)
   ✔ when user logs in (0.18s)
   ✔ then it should return success response (0.00s)
   ✔ and it should return created token (0.00s)
 
# 1 scenarios, 1 passed, 0 failed, 0 skipped (0.36s)

Running all tests with --show-steps gives you more than just verbose output, it’s a form of living documentation. Each step describes the behavior being tested in clear, structured language. Use it to validate test coverage, onboard new team members, or even review product behavior during CI runs.

Show Paths​

Want to trace exactly where each scenario is defined? Vedro always shows file paths for failed scenarios, but you can show paths for all scenarios using --show-paths.

$ vedro run --show-paths

Output:

Scenarios
* auth / login
 ✔ login as registered user
   > scenarios/auth/login/login_as_registered_user.py:5
 ✔ try to login as nonexisting user
   > scenarios/auth/login/try_to_login_as_nonexisting_user.py:8
 ✔ try to login with incorrect password
   > scenarios/auth/login/try_to_login_with_incorrect_password.py:9
 
# 3 scenarios, 3 passed, 0 failed, 0 skipped (0.81s)

Show Full Diff​

By default, Vedro trims assertion diffs to highlight just the changed lines, enough to catch most issues at a glance.

But sometimes, especially with large dictionaries or deeply nested objects, you want to see the full picture. For that, use the --show-full-diff flag:

$ vedro run --show-full-diff
AssertionError
>>> assert actual == expected
            {
  -             'task_id': 1,
  +             'task_id': 2,
                'description': 'Implement user authentication system',
  ...
                'assignee': 'Bob',
  -             'due_date': '2024-07-14'
  +             'due_date': '2024-07-15'
            }