| |
| The writing and reporting of assertions in tests |
| ================================================== |
| |
| .. _`assertfeedback`: |
| .. _`assert with the assert statement`: |
| .. _`assert`: |
| |
| |
| Asserting with the ``assert`` statement |
| --------------------------------------------------------- |
| |
| ``pytest`` allows you to use the standard python ``assert`` for verifying |
| expectations and values in Python tests. For example, you can write the |
| following:: |
| |
| # content of test_assert1.py |
| def f(): |
| return 3 |
| |
| def test_function(): |
| assert f() == 4 |
| |
| to assert that your function returns a certain value. If this assertion fails |
| you will see the return value of the function call:: |
| |
| $ pytest test_assert1.py |
| =========================== test session starts ============================ |
| platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y |
| rootdir: $REGENDOC_TMPDIR, inifile: |
| collected 1 item |
| |
| test_assert1.py F [100%] |
| |
| ================================= FAILURES ================================= |
| ______________________________ test_function _______________________________ |
| |
| def test_function(): |
| > assert f() == 4 |
| E assert 3 == 4 |
| E + where 3 = f() |
| |
| test_assert1.py:5: AssertionError |
| ========================= 1 failed in 0.12 seconds ========================= |
| |
| ``pytest`` has support for showing the values of the most common subexpressions |
| including calls, attributes, comparisons, and binary and unary |
| operators. (See :ref:`tbreportdemo`). This allows you to use the |
| idiomatic python constructs without boilerplate code while not losing |
| introspection information. |
| |
| However, if you specify a message with the assertion like this:: |
| |
| assert a % 2 == 0, "value was odd, should be even" |
| |
| then no assertion introspection takes places at all and the message |
| will be simply shown in the traceback. |
| |
| See :ref:`assert-details` for more information on assertion introspection. |
| |
| .. _`assertraises`: |
| |
| Assertions about expected exceptions |
| ------------------------------------------ |
| |
| In order to write assertions about raised exceptions, you can use |
| ``pytest.raises`` as a context manager like this:: |
| |
| import pytest |
| |
| def test_zero_division(): |
| with pytest.raises(ZeroDivisionError): |
| 1 / 0 |
| |
| and if you need to have access to the actual exception info you may use:: |
| |
| def test_recursion_depth(): |
| with pytest.raises(RuntimeError) as excinfo: |
| def f(): |
| f() |
| f() |
| assert 'maximum recursion' in str(excinfo.value) |
| |
| ``excinfo`` is a ``ExceptionInfo`` instance, which is a wrapper around |
| the actual exception raised. The main attributes of interest are |
| ``.type``, ``.value`` and ``.traceback``. |
| |
| .. versionchanged:: 3.0 |
| |
| In the context manager form you may use the keyword argument |
| ``message`` to specify a custom failure message:: |
| |
| >>> with raises(ZeroDivisionError, message="Expecting ZeroDivisionError"): |
| ... pass |
| ... Failed: Expecting ZeroDivisionError |
| |
| If you want to write test code that works on Python 2.4 as well, |
| you may also use two other ways to test for an expected exception:: |
| |
| pytest.raises(ExpectedException, func, *args, **kwargs) |
| pytest.raises(ExpectedException, "func(*args, **kwargs)") |
| |
| both of which execute the specified function with args and kwargs and |
| asserts that the given ``ExpectedException`` is raised. The reporter will |
| provide you with helpful output in case of failures such as *no |
| exception* or *wrong exception*. |
| |
| Note that it is also possible to specify a "raises" argument to |
| ``pytest.mark.xfail``, which checks that the test is failing in a more |
| specific way than just having any exception raised:: |
| |
| @pytest.mark.xfail(raises=IndexError) |
| def test_f(): |
| f() |
| |
| Using ``pytest.raises`` is likely to be better for cases where you are testing |
| exceptions your own code is deliberately raising, whereas using |
| ``@pytest.mark.xfail`` with a check function is probably better for something |
| like documenting unfixed bugs (where the test describes what "should" happen) |
| or bugs in dependencies. |
| |
| Also, the context manager form accepts a ``match`` keyword parameter to test |
| that a regular expression matches on the string representation of an exception |
| (like the ``TestCase.assertRaisesRegexp`` method from ``unittest``):: |
| |
| import pytest |
| |
| def myfunc(): |
| raise ValueError("Exception 123 raised") |
| |
| def test_match(): |
| with pytest.raises(ValueError, match=r'.* 123 .*'): |
| myfunc() |
| |
| The regexp parameter of the ``match`` method is matched with the ``re.search`` |
| function. So in the above example ``match='123'`` would have worked as |
| well. |
| |
| |
| .. _`assertwarns`: |
| |
| Assertions about expected warnings |
| ----------------------------------------- |
| |
| .. versionadded:: 2.8 |
| |
| You can check that code raises a particular warning using |
| :ref:`pytest.warns <warns>`. |
| |
| |
| .. _newreport: |
| |
| Making use of context-sensitive comparisons |
| ------------------------------------------------- |
| |
| .. versionadded:: 2.0 |
| |
| ``pytest`` has rich support for providing context-sensitive information |
| when it encounters comparisons. For example:: |
| |
| # content of test_assert2.py |
| |
| def test_set_comparison(): |
| set1 = set("1308") |
| set2 = set("8035") |
| assert set1 == set2 |
| |
| if you run this module:: |
| |
| $ pytest test_assert2.py |
| =========================== test session starts ============================ |
| platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y |
| rootdir: $REGENDOC_TMPDIR, inifile: |
| collected 1 item |
| |
| test_assert2.py F [100%] |
| |
| ================================= FAILURES ================================= |
| ___________________________ test_set_comparison ____________________________ |
| |
| def test_set_comparison(): |
| set1 = set("1308") |
| set2 = set("8035") |
| > assert set1 == set2 |
| E AssertionError: assert {'0', '1', '3', '8'} == {'0', '3', '5', '8'} |
| E Extra items in the left set: |
| E '1' |
| E Extra items in the right set: |
| E '5' |
| E Use -v to get the full diff |
| |
| test_assert2.py:5: AssertionError |
| ========================= 1 failed in 0.12 seconds ========================= |
| |
| Special comparisons are done for a number of cases: |
| |
| * comparing long strings: a context diff is shown |
| * comparing long sequences: first failing indices |
| * comparing dicts: different entries |
| |
| See the :ref:`reporting demo <tbreportdemo>` for many more examples. |
| |
| Defining your own assertion comparison |
| ---------------------------------------------- |
| |
| It is possible to add your own detailed explanations by implementing |
| the ``pytest_assertrepr_compare`` hook. |
| |
| .. autofunction:: _pytest.hookspec.pytest_assertrepr_compare |
| :noindex: |
| |
| As an example consider adding the following hook in a :ref:`conftest.py <conftest.py>` |
| file which provides an alternative explanation for ``Foo`` objects:: |
| |
| # content of conftest.py |
| from test_foocompare import Foo |
| def pytest_assertrepr_compare(op, left, right): |
| if isinstance(left, Foo) and isinstance(right, Foo) and op == "==": |
| return ['Comparing Foo instances:', |
| ' vals: %s != %s' % (left.val, right.val)] |
| |
| now, given this test module:: |
| |
| # content of test_foocompare.py |
| class Foo(object): |
| def __init__(self, val): |
| self.val = val |
| |
| def __eq__(self, other): |
| return self.val == other.val |
| |
| def test_compare(): |
| f1 = Foo(1) |
| f2 = Foo(2) |
| assert f1 == f2 |
| |
| you can run the test module and get the custom output defined in |
| the conftest file:: |
| |
| $ pytest -q test_foocompare.py |
| F [100%] |
| ================================= FAILURES ================================= |
| _______________________________ test_compare _______________________________ |
| |
| def test_compare(): |
| f1 = Foo(1) |
| f2 = Foo(2) |
| > assert f1 == f2 |
| E assert Comparing Foo instances: |
| E vals: 1 != 2 |
| |
| test_foocompare.py:11: AssertionError |
| 1 failed in 0.12 seconds |
| |
| .. _assert-details: |
| .. _`assert introspection`: |
| |
| Advanced assertion introspection |
| ---------------------------------- |
| |
| .. versionadded:: 2.1 |
| |
| |
| Reporting details about a failing assertion is achieved by rewriting assert |
| statements before they are run. Rewritten assert statements put introspection |
| information into the assertion failure message. ``pytest`` only rewrites test |
| modules directly discovered by its test collection process, so asserts in |
| supporting modules which are not themselves test modules will not be rewritten. |
| |
| .. note:: |
| |
| ``pytest`` rewrites test modules on import by using an import |
| hook to write new ``pyc`` files. Most of the time this works transparently. |
| However, if you are messing with import yourself, the import hook may |
| interfere. |
| |
| If this is the case you have two options: |
| |
| * Disable rewriting for a specific module by adding the string |
| ``PYTEST_DONT_REWRITE`` to its docstring. |
| |
| * Disable rewriting for all modules by using ``--assert=plain``. |
| |
| Additionally, rewriting will fail silently if it cannot write new ``.pyc`` files, |
| i.e. in a read-only filesystem or a zipfile. |
| |
| |
| For further information, Benjamin Peterson wrote up `Behind the scenes of pytest's new assertion rewriting <http://pybites.blogspot.com/2011/07/behind-scenes-of-pytests-new-assertion.html>`_. |
| |
| .. versionadded:: 2.1 |
| Add assert rewriting as an alternate introspection technique. |
| |
| .. versionchanged:: 2.1 |
| Introduce the ``--assert`` option. Deprecate ``--no-assert`` and |
| ``--nomagic``. |
| |
| .. versionchanged:: 3.0 |
| Removes the ``--no-assert`` and ``--nomagic`` options. |
| Removes the ``--assert=reinterp`` option. |