| .. _fixture: |
| .. _fixtures: |
| .. _`fixture functions`: |
| |
| pytest fixtures: explicit, modular, scalable |
| ======================================================== |
| |
| .. currentmodule:: _pytest.python |
| |
| .. versionadded:: 2.0/2.3/2.4 |
| |
| .. _`xUnit`: http://en.wikipedia.org/wiki/XUnit |
| .. _`purpose of test fixtures`: http://en.wikipedia.org/wiki/Test_fixture#Software |
| .. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection |
| |
| The `purpose of test fixtures`_ is to provide a fixed baseline |
| upon which tests can reliably and repeatedly execute. pytest fixtures |
| offer dramatic improvements over the classic xUnit style of setup/teardown |
| functions: |
| |
| * fixtures have explicit names and are activated by declaring their use |
| from test functions, modules, classes or whole projects. |
| |
| * fixtures are implemented in a modular manner, as each fixture name |
| triggers a *fixture function* which can itself use other fixtures. |
| |
| * fixture management scales from simple unit to complex |
| functional testing, allowing to parametrize fixtures and tests according |
| to configuration and component options, or to re-use fixtures |
| across function, class, module or whole test session scopes. |
| |
| In addition, pytest continues to support :ref:`xunitsetup`. You can mix |
| both styles, moving incrementally from classic to new style, as you |
| prefer. You can also start out from existing :ref:`unittest.TestCase |
| style <unittest.TestCase>` or :ref:`nose based <nosestyle>` projects. |
| |
| |
| .. _`funcargs`: |
| .. _`funcarg mechanism`: |
| .. _`fixture function`: |
| .. _`@pytest.fixture`: |
| .. _`pytest.fixture`: |
| |
| Fixtures as Function arguments |
| ----------------------------------------- |
| |
| Test functions can receive fixture objects by naming them as an input |
| argument. For each argument name, a fixture function with that name provides |
| the fixture object. Fixture functions are registered by marking them with |
| :py:func:`@pytest.fixture <_pytest.python.fixture>`. Let's look at a simple |
| self-contained test module containing a fixture and a test function |
| using it:: |
| |
| # content of ./test_smtpsimple.py |
| import pytest |
| |
| @pytest.fixture |
| def smtp(): |
| import smtplib |
| return smtplib.SMTP("smtp.gmail.com", 587, timeout=5) |
| |
| def test_ehlo(smtp): |
| response, msg = smtp.ehlo() |
| assert response == 250 |
| assert 0 # for demo purposes |
| |
| Here, the ``test_ehlo`` needs the ``smtp`` fixture value. pytest |
| will discover and call the :py:func:`@pytest.fixture <_pytest.python.fixture>` |
| marked ``smtp`` fixture function. Running the test looks like this:: |
| |
| $ pytest test_smtpsimple.py |
| =========================== test session starts ============================ |
| platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y |
| rootdir: $REGENDOC_TMPDIR, inifile: |
| collected 1 item |
| |
| test_smtpsimple.py F [100%] |
| |
| ================================= FAILURES ================================= |
| ________________________________ test_ehlo _________________________________ |
| |
| smtp = <smtplib.SMTP object at 0xdeadbeef> |
| |
| def test_ehlo(smtp): |
| response, msg = smtp.ehlo() |
| assert response == 250 |
| > assert 0 # for demo purposes |
| E assert 0 |
| |
| test_smtpsimple.py:11: AssertionError |
| ========================= 1 failed in 0.12 seconds ========================= |
| |
| In the failure traceback we see that the test function was called with a |
| ``smtp`` argument, the ``smtplib.SMTP()`` instance created by the fixture |
| function. The test function fails on our deliberate ``assert 0``. Here is |
| the exact protocol used by ``pytest`` to call the test function this way: |
| |
| 1. pytest :ref:`finds <test discovery>` the ``test_ehlo`` because |
| of the ``test_`` prefix. The test function needs a function argument |
| named ``smtp``. A matching fixture function is discovered by |
| looking for a fixture-marked function named ``smtp``. |
| |
| 2. ``smtp()`` is called to create an instance. |
| |
| 3. ``test_ehlo(<SMTP instance>)`` is called and fails in the last |
| line of the test function. |
| |
| Note that if you misspell a function argument or want |
| to use one that isn't available, you'll see an error |
| with a list of available function arguments. |
| |
| .. note:: |
| |
| You can always issue :: |
| |
| pytest --fixtures test_simplefactory.py |
| |
| to see available fixtures (fixtures with leading ``_`` are only shown if you add the ``-v`` option). |
| |
| Fixtures: a prime example of dependency injection |
| --------------------------------------------------- |
| |
| Fixtures allow test functions to easily receive and work |
| against specific pre-initialized application objects without having |
| to care about import/setup/cleanup details. |
| It's a prime example of `dependency injection`_ where fixture |
| functions take the role of the *injector* and test functions are the |
| *consumers* of fixture objects. |
| |
| .. _`conftest.py`: |
| .. _`conftest`: |
| |
| ``conftest.py``: sharing fixture functions |
| ------------------------------------------ |
| |
| If during implementing your tests you realize that you |
| want to use a fixture function from multiple test files you can move it |
| to a ``conftest.py`` file. |
| You don't need to import the fixture you want to use in a test, it |
| automatically gets discovered by pytest. The discovery of |
| fixture functions starts at test classes, then test modules, then |
| ``conftest.py`` files and finally builtin and third party plugins. |
| |
| You can also use the ``conftest.py`` file to implement |
| :ref:`local per-directory plugins <conftest.py plugins>`. |
| |
| Sharing test data |
| ----------------- |
| |
| If you want to make test data from files available to your tests, a good way |
| to do this is by loading these data in a fixture for use by your tests. |
| This makes use of the automatic caching mechanisms of pytest. |
| |
| Another good approach is by adding the data files in the ``tests`` folder. |
| There are also community plugins available to help managing this aspect of |
| testing, e.g. `pytest-datadir <https://github.com/gabrielcnr/pytest-datadir>`__ |
| and `pytest-datafiles <https://pypi.python.org/pypi/pytest-datafiles>`__. |
| |
| .. _smtpshared: |
| |
| Scope: sharing a fixture instance across tests in a class, module or session |
| ---------------------------------------------------------------------------- |
| |
| .. regendoc:wipe |
| |
| Fixtures requiring network access depend on connectivity and are |
| usually time-expensive to create. Extending the previous example, we |
| can add a ``scope="module"`` parameter to the |
| :py:func:`@pytest.fixture <_pytest.python.fixture>` invocation |
| to cause the decorated ``smtp`` fixture function to only be invoked once |
| per test *module* (the default is to invoke once per test *function*). |
| Multiple test functions in a test module will thus |
| each receive the same ``smtp`` fixture instance, thus saving time. |
| |
| The next example puts the fixture function into a separate ``conftest.py`` file |
| so that tests from multiple test modules in the directory can |
| access the fixture function:: |
| |
| # content of conftest.py |
| import pytest |
| import smtplib |
| |
| @pytest.fixture(scope="module") |
| def smtp(): |
| return smtplib.SMTP("smtp.gmail.com", 587, timeout=5) |
| |
| The name of the fixture again is ``smtp`` and you can access its result by |
| listing the name ``smtp`` as an input parameter in any test or fixture |
| function (in or below the directory where ``conftest.py`` is located):: |
| |
| # content of test_module.py |
| |
| def test_ehlo(smtp): |
| response, msg = smtp.ehlo() |
| assert response == 250 |
| assert b"smtp.gmail.com" in msg |
| assert 0 # for demo purposes |
| |
| def test_noop(smtp): |
| response, msg = smtp.noop() |
| assert response == 250 |
| assert 0 # for demo purposes |
| |
| We deliberately insert failing ``assert 0`` statements in order to |
| inspect what is going on and can now run the tests:: |
| |
| $ pytest test_module.py |
| =========================== test session starts ============================ |
| platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y |
| rootdir: $REGENDOC_TMPDIR, inifile: |
| collected 2 items |
| |
| test_module.py FF [100%] |
| |
| ================================= FAILURES ================================= |
| ________________________________ test_ehlo _________________________________ |
| |
| smtp = <smtplib.SMTP object at 0xdeadbeef> |
| |
| def test_ehlo(smtp): |
| response, msg = smtp.ehlo() |
| assert response == 250 |
| assert b"smtp.gmail.com" in msg |
| > assert 0 # for demo purposes |
| E assert 0 |
| |
| test_module.py:6: AssertionError |
| ________________________________ test_noop _________________________________ |
| |
| smtp = <smtplib.SMTP object at 0xdeadbeef> |
| |
| def test_noop(smtp): |
| response, msg = smtp.noop() |
| assert response == 250 |
| > assert 0 # for demo purposes |
| E assert 0 |
| |
| test_module.py:11: AssertionError |
| ========================= 2 failed in 0.12 seconds ========================= |
| |
| You see the two ``assert 0`` failing and more importantly you can also see |
| that the same (module-scoped) ``smtp`` object was passed into the two |
| test functions because pytest shows the incoming argument values in the |
| traceback. As a result, the two test functions using ``smtp`` run as |
| quick as a single one because they reuse the same instance. |
| |
| If you decide that you rather want to have a session-scoped ``smtp`` |
| instance, you can simply declare it: |
| |
| .. code-block:: python |
| |
| @pytest.fixture(scope="session") |
| def smtp(): |
| # the returned fixture value will be shared for |
| # all tests needing it |
| ... |
| |
| Finally, the ``class`` scope will invoke the fixture once per test *class*. |
| |
| |
| Higher-scoped fixtures are instantiated first |
| --------------------------------------------- |
| |
| .. versionadded:: 3.5 |
| |
| Within a function request for features, fixture of higher-scopes (such as ``session``) are instantiated first than |
| lower-scoped fixtures (such as ``function`` or ``class``). The relative order of fixtures of same scope follows |
| the declared order in the test function and honours dependencies between fixtures. |
| |
| Consider the code below: |
| |
| .. code-block:: python |
| |
| @pytest.fixture(scope="session") |
| def s1(): |
| pass |
| |
| |
| @pytest.fixture(scope="module") |
| def m1(): |
| pass |
| |
| |
| @pytest.fixture |
| def f1(tmpdir): |
| pass |
| |
| |
| @pytest.fixture |
| def f2(): |
| pass |
| |
| |
| def test_foo(f1, m1, f2, s1): |
| ... |
| |
| |
| The fixtures requested by ``test_foo`` will be instantiated in the following order: |
| |
| 1. ``s1``: is the highest-scoped fixture (``session``). |
| 2. ``m1``: is the second highest-scoped fixture (``module``). |
| 3. ``tmpdir``: is a ``function``-scoped fixture, required by ``f1``: it needs to be instantiated at this point |
| because it is a dependency of ``f1``. |
| 4. ``f1``: is the first ``function``-scoped fixture in ``test_foo`` parameter list. |
| 5. ``f2``: is the last ``function``-scoped fixture in ``test_foo`` parameter list. |
| |
| |
| .. _`finalization`: |
| |
| Fixture finalization / executing teardown code |
| ------------------------------------------------------------- |
| |
| pytest supports execution of fixture specific finalization code |
| when the fixture goes out of scope. By using a ``yield`` statement instead of ``return``, all |
| the code after the *yield* statement serves as the teardown code: |
| |
| .. code-block:: python |
| |
| # content of conftest.py |
| |
| import smtplib |
| import pytest |
| |
| |
| @pytest.fixture(scope="module") |
| def smtp(): |
| smtp = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) |
| yield smtp # provide the fixture value |
| print("teardown smtp") |
| smtp.close() |
| |
| The ``print`` and ``smtp.close()`` statements will execute when the last test in |
| the module has finished execution, regardless of the exception status of the |
| tests. |
| |
| Let's execute it:: |
| |
| $ pytest -s -q --tb=no |
| FFteardown smtp |
| |
| 2 failed in 0.12 seconds |
| |
| We see that the ``smtp`` instance is finalized after the two |
| tests finished execution. Note that if we decorated our fixture |
| function with ``scope='function'`` then fixture setup and cleanup would |
| occur around each single test. In either case the test |
| module itself does not need to change or know about these details |
| of fixture setup. |
| |
| Note that we can also seamlessly use the ``yield`` syntax with ``with`` statements: |
| |
| .. code-block:: python |
| |
| # content of test_yield2.py |
| |
| import smtplib |
| import pytest |
| |
| |
| @pytest.fixture(scope="module") |
| def smtp(): |
| with smtplib.SMTP("smtp.gmail.com", 587, timeout=5) as smtp: |
| yield smtp # provide the fixture value |
| |
| |
| The ``smtp`` connection will be closed after the test finished execution |
| because the ``smtp`` object automatically closes when |
| the ``with`` statement ends. |
| |
| Note that if an exception happens during the *setup* code (before the ``yield`` keyword), the |
| *teardown* code (after the ``yield``) will not be called. |
| |
| An alternative option for executing *teardown* code is to |
| make use of the ``addfinalizer`` method of the `request-context`_ object to register |
| finalization functions. |
| |
| Here's the ``smtp`` fixture changed to use ``addfinalizer`` for cleanup: |
| |
| .. code-block:: python |
| |
| # content of conftest.py |
| import smtplib |
| import pytest |
| |
| |
| @pytest.fixture(scope="module") |
| def smtp(request): |
| smtp = smtplib.SMTP("smtp.gmail.com", 587, timeout=5) |
| |
| def fin(): |
| print("teardown smtp") |
| smtp.close() |
| |
| request.addfinalizer(fin) |
| return smtp # provide the fixture value |
| |
| |
| Both ``yield`` and ``addfinalizer`` methods work similarly by calling their code after the test |
| ends, but ``addfinalizer`` has two key differences over ``yield``: |
| |
| 1. It is possible to register multiple finalizer functions. |
| |
| 2. Finalizers will always be called regardless if the fixture *setup* code raises an exception. |
| This is handy to properly close all resources created by a fixture even if one of them |
| fails to be created/acquired:: |
| |
| @pytest.fixture |
| def equipments(request): |
| r = [] |
| for port in ('C1', 'C3', 'C28'): |
| equip = connect(port) |
| request.addfinalizer(equip.disconnect) |
| r.append(equip) |
| return r |
| |
| In the example above, if ``"C28"`` fails with an exception, ``"C1"`` and ``"C3"`` will still |
| be properly closed. Of course, if an exception happens before the finalize function is |
| registered then it will not be executed. |
| |
| |
| .. _`request-context`: |
| |
| Fixtures can introspect the requesting test context |
| ------------------------------------------------------------- |
| |
| Fixture functions can accept the :py:class:`request <FixtureRequest>` object |
| to introspect the "requesting" test function, class or module context. |
| Further extending the previous ``smtp`` fixture example, let's |
| read an optional server URL from the test module which uses our fixture:: |
| |
| # content of conftest.py |
| import pytest |
| import smtplib |
| |
| @pytest.fixture(scope="module") |
| def smtp(request): |
| server = getattr(request.module, "smtpserver", "smtp.gmail.com") |
| smtp = smtplib.SMTP(server, 587, timeout=5) |
| yield smtp |
| print ("finalizing %s (%s)" % (smtp, server)) |
| smtp.close() |
| |
| We use the ``request.module`` attribute to optionally obtain an |
| ``smtpserver`` attribute from the test module. If we just execute |
| again, nothing much has changed:: |
| |
| $ pytest -s -q --tb=no |
| FFfinalizing <smtplib.SMTP object at 0xdeadbeef> (smtp.gmail.com) |
| |
| 2 failed in 0.12 seconds |
| |
| Let's quickly create another test module that actually sets the |
| server URL in its module namespace:: |
| |
| # content of test_anothersmtp.py |
| |
| smtpserver = "mail.python.org" # will be read by smtp fixture |
| |
| def test_showhelo(smtp): |
| assert 0, smtp.helo() |
| |
| Running it:: |
| |
| $ pytest -qq --tb=short test_anothersmtp.py |
| F [100%] |
| ================================= FAILURES ================================= |
| ______________________________ test_showhelo _______________________________ |
| test_anothersmtp.py:5: in test_showhelo |
| assert 0, smtp.helo() |
| E AssertionError: (250, b'mail.python.org') |
| E assert 0 |
| ------------------------- Captured stdout teardown ------------------------- |
| finalizing <smtplib.SMTP object at 0xdeadbeef> (mail.python.org) |
| |
| voila! The ``smtp`` fixture function picked up our mail server name |
| from the module namespace. |
| |
| .. _`fixture-factory`: |
| |
| Factories as fixtures |
| ------------------------------------------------------------- |
| |
| The "factory as fixture" pattern can help in situations where the result |
| of a fixture is needed multiple times in a single test. Instead of returning |
| data directly, the fixture instead returns a function which generates the data. |
| This function can then be called multiple times in the test. |
| |
| Factories can have have parameters as needed:: |
| |
| @pytest.fixture |
| def make_customer_record(): |
| |
| def _make_customer_record(name): |
| return { |
| "name": name, |
| "orders": [] |
| } |
| |
| return _make_customer_record |
| |
| |
| def test_customer_records(make_customer_record): |
| customer_1 = make_customer_record("Lisa") |
| customer_2 = make_customer_record("Mike") |
| customer_3 = make_customer_record("Meredith") |
| |
| If the data created by the factory requires managing, the fixture can take care of that:: |
| |
| @pytest.fixture |
| def make_customer_record(): |
| |
| created_records = [] |
| |
| def _make_customer_record(name): |
| record = models.Customer(name=name, orders=[]) |
| created_records.append(record) |
| return record |
| |
| yield _make_customer_record |
| |
| for record in created_records: |
| record.destroy() |
| |
| |
| def test_customer_records(make_customer_record): |
| customer_1 = make_customer_record("Lisa") |
| customer_2 = make_customer_record("Mike") |
| customer_3 = make_customer_record("Meredith") |
| |
| |
| .. _`fixture-parametrize`: |
| |
| Parametrizing fixtures |
| ----------------------------------------------------------------- |
| |
| Fixture functions can be parametrized in which case they will be called |
| multiple times, each time executing the set of dependent tests, i. e. the |
| tests that depend on this fixture. Test functions do usually not need |
| to be aware of their re-running. Fixture parametrization helps to |
| write exhaustive functional tests for components which themselves can be |
| configured in multiple ways. |
| |
| Extending the previous example, we can flag the fixture to create two |
| ``smtp`` fixture instances which will cause all tests using the fixture |
| to run twice. The fixture function gets access to each parameter |
| through the special :py:class:`request <FixtureRequest>` object:: |
| |
| # content of conftest.py |
| import pytest |
| import smtplib |
| |
| @pytest.fixture(scope="module", |
| params=["smtp.gmail.com", "mail.python.org"]) |
| def smtp(request): |
| smtp = smtplib.SMTP(request.param, 587, timeout=5) |
| yield smtp |
| print ("finalizing %s" % smtp) |
| smtp.close() |
| |
| The main change is the declaration of ``params`` with |
| :py:func:`@pytest.fixture <_pytest.python.fixture>`, a list of values |
| for each of which the fixture function will execute and can access |
| a value via ``request.param``. No test function code needs to change. |
| So let's just do another run:: |
| |
| $ pytest -q test_module.py |
| FFFF [100%] |
| ================================= FAILURES ================================= |
| ________________________ test_ehlo[smtp.gmail.com] _________________________ |
| |
| smtp = <smtplib.SMTP object at 0xdeadbeef> |
| |
| def test_ehlo(smtp): |
| response, msg = smtp.ehlo() |
| assert response == 250 |
| assert b"smtp.gmail.com" in msg |
| > assert 0 # for demo purposes |
| E assert 0 |
| |
| test_module.py:6: AssertionError |
| ________________________ test_noop[smtp.gmail.com] _________________________ |
| |
| smtp = <smtplib.SMTP object at 0xdeadbeef> |
| |
| def test_noop(smtp): |
| response, msg = smtp.noop() |
| assert response == 250 |
| > assert 0 # for demo purposes |
| E assert 0 |
| |
| test_module.py:11: AssertionError |
| ________________________ test_ehlo[mail.python.org] ________________________ |
| |
| smtp = <smtplib.SMTP object at 0xdeadbeef> |
| |
| def test_ehlo(smtp): |
| response, msg = smtp.ehlo() |
| assert response == 250 |
| > assert b"smtp.gmail.com" in msg |
| E AssertionError: assert b'smtp.gmail.com' in b'mail.python.org\nPIPELINING\nSIZE 51200000\nETRN\nSTARTTLS\nAUTH DIGEST-MD5 NTLM CRAM-MD5\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nSMTPUTF8' |
| |
| test_module.py:5: AssertionError |
| -------------------------- Captured stdout setup --------------------------- |
| finalizing <smtplib.SMTP object at 0xdeadbeef> |
| ________________________ test_noop[mail.python.org] ________________________ |
| |
| smtp = <smtplib.SMTP object at 0xdeadbeef> |
| |
| def test_noop(smtp): |
| response, msg = smtp.noop() |
| assert response == 250 |
| > assert 0 # for demo purposes |
| E assert 0 |
| |
| test_module.py:11: AssertionError |
| ------------------------- Captured stdout teardown ------------------------- |
| finalizing <smtplib.SMTP object at 0xdeadbeef> |
| 4 failed in 0.12 seconds |
| |
| We see that our two test functions each ran twice, against the different |
| ``smtp`` instances. Note also, that with the ``mail.python.org`` |
| connection the second test fails in ``test_ehlo`` because a |
| different server string is expected than what arrived. |
| |
| pytest will build a string that is the test ID for each fixture value |
| in a parametrized fixture, e.g. ``test_ehlo[smtp.gmail.com]`` and |
| ``test_ehlo[mail.python.org]`` in the above examples. These IDs can |
| be used with ``-k`` to select specific cases to run, and they will |
| also identify the specific case when one is failing. Running pytest |
| with ``--collect-only`` will show the generated IDs. |
| |
| Numbers, strings, booleans and None will have their usual string |
| representation used in the test ID. For other objects, pytest will |
| make a string based on the argument name. It is possible to customise |
| the string used in a test ID for a certain fixture value by using the |
| ``ids`` keyword argument:: |
| |
| # content of test_ids.py |
| import pytest |
| |
| @pytest.fixture(params=[0, 1], ids=["spam", "ham"]) |
| def a(request): |
| return request.param |
| |
| def test_a(a): |
| pass |
| |
| def idfn(fixture_value): |
| if fixture_value == 0: |
| return "eggs" |
| else: |
| return None |
| |
| @pytest.fixture(params=[0, 1], ids=idfn) |
| def b(request): |
| return request.param |
| |
| def test_b(b): |
| pass |
| |
| The above shows how ``ids`` can be either a list of strings to use or |
| a function which will be called with the fixture value and then |
| has to return a string to use. In the latter case if the function |
| return ``None`` then pytest's auto-generated ID will be used. |
| |
| Running the above tests results in the following test IDs being used:: |
| |
| $ pytest --collect-only |
| =========================== test session starts ============================ |
| platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y |
| rootdir: $REGENDOC_TMPDIR, inifile: |
| collected 10 items |
| <Module 'test_anothersmtp.py'> |
| <Function 'test_showhelo[smtp.gmail.com]'> |
| <Function 'test_showhelo[mail.python.org]'> |
| <Module 'test_ids.py'> |
| <Function 'test_a[spam]'> |
| <Function 'test_a[ham]'> |
| <Function 'test_b[eggs]'> |
| <Function 'test_b[1]'> |
| <Module 'test_module.py'> |
| <Function 'test_ehlo[smtp.gmail.com]'> |
| <Function 'test_noop[smtp.gmail.com]'> |
| <Function 'test_ehlo[mail.python.org]'> |
| <Function 'test_noop[mail.python.org]'> |
| |
| ======================= no tests ran in 0.12 seconds ======================= |
| |
| .. _`fixture-parametrize-marks`: |
| |
| Using marks with parametrized fixtures |
| -------------------------------------- |
| |
| :func:`pytest.param` can be used to apply marks in values sets of parametrized fixtures in the same way |
| that they can be used with :ref:`@pytest.mark.parametrize <@pytest.mark.parametrize>`. |
| |
| Example:: |
| |
| # content of test_fixture_marks.py |
| import pytest |
| @pytest.fixture(params=[0, 1, pytest.param(2, marks=pytest.mark.skip)]) |
| def data_set(request): |
| return request.param |
| |
| def test_data(data_set): |
| pass |
| |
| Running this test will *skip* the invocation of ``data_set`` with value ``2``:: |
| |
| $ pytest test_fixture_marks.py -v |
| =========================== test session starts ============================ |
| platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 |
| cachedir: .pytest_cache |
| rootdir: $REGENDOC_TMPDIR, inifile: |
| collecting ... collected 3 items |
| |
| test_fixture_marks.py::test_data[0] PASSED [ 33%] |
| test_fixture_marks.py::test_data[1] PASSED [ 66%] |
| test_fixture_marks.py::test_data[2] SKIPPED [100%] |
| |
| =================== 2 passed, 1 skipped in 0.12 seconds ==================== |
| |
| .. _`interdependent fixtures`: |
| |
| Modularity: using fixtures from a fixture function |
| ---------------------------------------------------------- |
| |
| You can not only use fixtures in test functions but fixture functions |
| can use other fixtures themselves. This contributes to a modular design |
| of your fixtures and allows re-use of framework-specific fixtures across |
| many projects. As a simple example, we can extend the previous example |
| and instantiate an object ``app`` where we stick the already defined |
| ``smtp`` resource into it:: |
| |
| # content of test_appsetup.py |
| |
| import pytest |
| |
| class App(object): |
| def __init__(self, smtp): |
| self.smtp = smtp |
| |
| @pytest.fixture(scope="module") |
| def app(smtp): |
| return App(smtp) |
| |
| def test_smtp_exists(app): |
| assert app.smtp |
| |
| Here we declare an ``app`` fixture which receives the previously defined |
| ``smtp`` fixture and instantiates an ``App`` object with it. Let's run it:: |
| |
| $ pytest -v test_appsetup.py |
| =========================== test session starts ============================ |
| platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 |
| cachedir: .pytest_cache |
| rootdir: $REGENDOC_TMPDIR, inifile: |
| collecting ... collected 2 items |
| |
| test_appsetup.py::test_smtp_exists[smtp.gmail.com] PASSED [ 50%] |
| test_appsetup.py::test_smtp_exists[mail.python.org] PASSED [100%] |
| |
| ========================= 2 passed in 0.12 seconds ========================= |
| |
| Due to the parametrization of ``smtp`` the test will run twice with two |
| different ``App`` instances and respective smtp servers. There is no |
| need for the ``app`` fixture to be aware of the ``smtp`` parametrization |
| as pytest will fully analyse the fixture dependency graph. |
| |
| Note, that the ``app`` fixture has a scope of ``module`` and uses a |
| module-scoped ``smtp`` fixture. The example would still work if ``smtp`` |
| was cached on a ``session`` scope: it is fine for fixtures to use |
| "broader" scoped fixtures but not the other way round: |
| A session-scoped fixture could not use a module-scoped one in a |
| meaningful way. |
| |
| |
| .. _`automatic per-resource grouping`: |
| |
| Automatic grouping of tests by fixture instances |
| ---------------------------------------------------------- |
| |
| .. regendoc: wipe |
| |
| pytest minimizes the number of active fixtures during test runs. |
| If you have a parametrized fixture, then all the tests using it will |
| first execute with one instance and then finalizers are called |
| before the next fixture instance is created. Among other things, |
| this eases testing of applications which create and use global state. |
| |
| The following example uses two parametrized fixture, one of which is |
| scoped on a per-module basis, and all the functions perform ``print`` calls |
| to show the setup/teardown flow:: |
| |
| # content of test_module.py |
| import pytest |
| |
| @pytest.fixture(scope="module", params=["mod1", "mod2"]) |
| def modarg(request): |
| param = request.param |
| print (" SETUP modarg %s" % param) |
| yield param |
| print (" TEARDOWN modarg %s" % param) |
| |
| @pytest.fixture(scope="function", params=[1,2]) |
| def otherarg(request): |
| param = request.param |
| print (" SETUP otherarg %s" % param) |
| yield param |
| print (" TEARDOWN otherarg %s" % param) |
| |
| def test_0(otherarg): |
| print (" RUN test0 with otherarg %s" % otherarg) |
| def test_1(modarg): |
| print (" RUN test1 with modarg %s" % modarg) |
| def test_2(otherarg, modarg): |
| print (" RUN test2 with otherarg %s and modarg %s" % (otherarg, modarg)) |
| |
| |
| Let's run the tests in verbose mode and with looking at the print-output:: |
| |
| $ pytest -v -s test_module.py |
| =========================== test session starts ============================ |
| platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5 |
| cachedir: .pytest_cache |
| rootdir: $REGENDOC_TMPDIR, inifile: |
| collecting ... collected 8 items |
| |
| test_module.py::test_0[1] SETUP otherarg 1 |
| RUN test0 with otherarg 1 |
| PASSED TEARDOWN otherarg 1 |
| |
| test_module.py::test_0[2] SETUP otherarg 2 |
| RUN test0 with otherarg 2 |
| PASSED TEARDOWN otherarg 2 |
| |
| test_module.py::test_1[mod1] SETUP modarg mod1 |
| RUN test1 with modarg mod1 |
| PASSED |
| test_module.py::test_2[mod1-1] SETUP otherarg 1 |
| RUN test2 with otherarg 1 and modarg mod1 |
| PASSED TEARDOWN otherarg 1 |
| |
| test_module.py::test_2[mod1-2] SETUP otherarg 2 |
| RUN test2 with otherarg 2 and modarg mod1 |
| PASSED TEARDOWN otherarg 2 |
| |
| test_module.py::test_1[mod2] TEARDOWN modarg mod1 |
| SETUP modarg mod2 |
| RUN test1 with modarg mod2 |
| PASSED |
| test_module.py::test_2[mod2-1] SETUP otherarg 1 |
| RUN test2 with otherarg 1 and modarg mod2 |
| PASSED TEARDOWN otherarg 1 |
| |
| test_module.py::test_2[mod2-2] SETUP otherarg 2 |
| RUN test2 with otherarg 2 and modarg mod2 |
| PASSED TEARDOWN otherarg 2 |
| TEARDOWN modarg mod2 |
| |
| |
| ========================= 8 passed in 0.12 seconds ========================= |
| |
| You can see that the parametrized module-scoped ``modarg`` resource caused an |
| ordering of test execution that lead to the fewest possible "active" resources. |
| The finalizer for the ``mod1`` parametrized resource was executed before the |
| ``mod2`` resource was setup. |
| |
| In particular notice that test_0 is completely independent and finishes first. |
| Then test_1 is executed with ``mod1``, then test_2 with ``mod1``, then test_1 |
| with ``mod2`` and finally test_2 with ``mod2``. |
| |
| The ``otherarg`` parametrized resource (having function scope) was set up before |
| and teared down after every test that used it. |
| |
| |
| .. _`usefixtures`: |
| |
| Using fixtures from classes, modules or projects |
| ---------------------------------------------------------------------- |
| |
| .. regendoc:wipe |
| |
| Sometimes test functions do not directly need access to a fixture object. |
| For example, tests may require to operate with an empty directory as the |
| current working directory but otherwise do not care for the concrete |
| directory. Here is how you can use the standard `tempfile |
| <http://docs.python.org/library/tempfile.html>`_ and pytest fixtures to |
| achieve it. We separate the creation of the fixture into a conftest.py |
| file:: |
| |
| # content of conftest.py |
| |
| import pytest |
| import tempfile |
| import os |
| |
| @pytest.fixture() |
| def cleandir(): |
| newpath = tempfile.mkdtemp() |
| os.chdir(newpath) |
| |
| and declare its use in a test module via a ``usefixtures`` marker:: |
| |
| # content of test_setenv.py |
| import os |
| import pytest |
| |
| @pytest.mark.usefixtures("cleandir") |
| class TestDirectoryInit(object): |
| def test_cwd_starts_empty(self): |
| assert os.listdir(os.getcwd()) == [] |
| with open("myfile", "w") as f: |
| f.write("hello") |
| |
| def test_cwd_again_starts_empty(self): |
| assert os.listdir(os.getcwd()) == [] |
| |
| Due to the ``usefixtures`` marker, the ``cleandir`` fixture |
| will be required for the execution of each test method, just as if |
| you specified a "cleandir" function argument to each of them. Let's run it |
| to verify our fixture is activated and the tests pass:: |
| |
| $ pytest -q |
| .. [100%] |
| 2 passed in 0.12 seconds |
| |
| You can specify multiple fixtures like this: |
| |
| .. code-block:: python |
| |
| @pytest.mark.usefixtures("cleandir", "anotherfixture") |
| def test(): |
| ... |
| |
| and you may specify fixture usage at the test module level, using |
| a generic feature of the mark mechanism: |
| |
| .. code-block:: python |
| |
| pytestmark = pytest.mark.usefixtures("cleandir") |
| |
| Note that the assigned variable *must* be called ``pytestmark``, assigning e.g. |
| ``foomark`` will not activate the fixtures. |
| |
| Lastly you can put fixtures required by all tests in your project |
| into an ini-file: |
| |
| .. code-block:: ini |
| |
| # content of pytest.ini |
| [pytest] |
| usefixtures = cleandir |
| |
| |
| .. _`autouse`: |
| .. _`autouse fixtures`: |
| |
| Autouse fixtures (xUnit setup on steroids) |
| ---------------------------------------------------------------------- |
| |
| .. regendoc:wipe |
| |
| Occasionally, you may want to have fixtures get invoked automatically |
| without declaring a function argument explicitly or a `usefixtures`_ decorator. |
| As a practical example, suppose we have a database fixture which has a |
| begin/rollback/commit architecture and we want to automatically surround |
| each test method by a transaction and a rollback. Here is a dummy |
| self-contained implementation of this idea:: |
| |
| # content of test_db_transact.py |
| |
| import pytest |
| |
| class DB(object): |
| def __init__(self): |
| self.intransaction = [] |
| def begin(self, name): |
| self.intransaction.append(name) |
| def rollback(self): |
| self.intransaction.pop() |
| |
| @pytest.fixture(scope="module") |
| def db(): |
| return DB() |
| |
| class TestClass(object): |
| @pytest.fixture(autouse=True) |
| def transact(self, request, db): |
| db.begin(request.function.__name__) |
| yield |
| db.rollback() |
| |
| def test_method1(self, db): |
| assert db.intransaction == ["test_method1"] |
| |
| def test_method2(self, db): |
| assert db.intransaction == ["test_method2"] |
| |
| The class-level ``transact`` fixture is marked with *autouse=true* |
| which implies that all test methods in the class will use this fixture |
| without a need to state it in the test function signature or with a |
| class-level ``usefixtures`` decorator. |
| |
| If we run it, we get two passing tests:: |
| |
| $ pytest -q |
| .. [100%] |
| 2 passed in 0.12 seconds |
| |
| Here is how autouse fixtures work in other scopes: |
| |
| - autouse fixtures obey the ``scope=`` keyword-argument: if an autouse fixture |
| has ``scope='session'`` it will only be run once, no matter where it is |
| defined. ``scope='class'`` means it will be run once per class, etc. |
| |
| - if an autouse fixture is defined in a test module, all its test |
| functions automatically use it. |
| |
| - if an autouse fixture is defined in a conftest.py file then all tests in |
| all test modules below its directory will invoke the fixture. |
| |
| - lastly, and **please use that with care**: if you define an autouse |
| fixture in a plugin, it will be invoked for all tests in all projects |
| where the plugin is installed. This can be useful if a fixture only |
| anyway works in the presence of certain settings e. g. in the ini-file. Such |
| a global fixture should always quickly determine if it should do |
| any work and avoid otherwise expensive imports or computation. |
| |
| Note that the above ``transact`` fixture may very well be a fixture that |
| you want to make available in your project without having it generally |
| active. The canonical way to do that is to put the transact definition |
| into a conftest.py file **without** using ``autouse``:: |
| |
| # content of conftest.py |
| @pytest.fixture |
| def transact(request, db): |
| db.begin() |
| yield |
| db.rollback() |
| |
| and then e.g. have a TestClass using it by declaring the need:: |
| |
| @pytest.mark.usefixtures("transact") |
| class TestClass(object): |
| def test_method1(self): |
| ... |
| |
| All test methods in this TestClass will use the transaction fixture while |
| other test classes or functions in the module will not use it unless |
| they also add a ``transact`` reference. |
| |
| Overriding fixtures on various levels |
| ------------------------------------- |
| |
| In relatively large test suite, you most likely need to ``override`` a ``global`` or ``root`` fixture with a ``locally`` |
| defined one, keeping the test code readable and maintainable. |
| |
| Override a fixture on a folder (conftest) level |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Given the tests file structure is: |
| |
| :: |
| |
| tests/ |
| __init__.py |
| |
| conftest.py |
| # content of tests/conftest.py |
| import pytest |
| |
| @pytest.fixture |
| def username(): |
| return 'username' |
| |
| test_something.py |
| # content of tests/test_something.py |
| def test_username(username): |
| assert username == 'username' |
| |
| subfolder/ |
| __init__.py |
| |
| conftest.py |
| # content of tests/subfolder/conftest.py |
| import pytest |
| |
| @pytest.fixture |
| def username(username): |
| return 'overridden-' + username |
| |
| test_something.py |
| # content of tests/subfolder/test_something.py |
| def test_username(username): |
| assert username == 'overridden-username' |
| |
| As you can see, a fixture with the same name can be overridden for certain test folder level. |
| Note that the ``base`` or ``super`` fixture can be accessed from the ``overriding`` |
| fixture easily - used in the example above. |
| |
| Override a fixture on a test module level |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Given the tests file structure is: |
| |
| :: |
| |
| tests/ |
| __init__.py |
| |
| conftest.py |
| # content of tests/conftest.py |
| @pytest.fixture |
| def username(): |
| return 'username' |
| |
| test_something.py |
| # content of tests/test_something.py |
| import pytest |
| |
| @pytest.fixture |
| def username(username): |
| return 'overridden-' + username |
| |
| def test_username(username): |
| assert username == 'overridden-username' |
| |
| test_something_else.py |
| # content of tests/test_something_else.py |
| import pytest |
| |
| @pytest.fixture |
| def username(username): |
| return 'overridden-else-' + username |
| |
| def test_username(username): |
| assert username == 'overridden-else-username' |
| |
| In the example above, a fixture with the same name can be overridden for certain test module. |
| |
| |
| Override a fixture with direct test parametrization |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Given the tests file structure is: |
| |
| :: |
| |
| tests/ |
| __init__.py |
| |
| conftest.py |
| # content of tests/conftest.py |
| import pytest |
| |
| @pytest.fixture |
| def username(): |
| return 'username' |
| |
| @pytest.fixture |
| def other_username(username): |
| return 'other-' + username |
| |
| test_something.py |
| # content of tests/test_something.py |
| import pytest |
| |
| @pytest.mark.parametrize('username', ['directly-overridden-username']) |
| def test_username(username): |
| assert username == 'directly-overridden-username' |
| |
| @pytest.mark.parametrize('username', ['directly-overridden-username-other']) |
| def test_username_other(other_username): |
| assert other_username == 'other-directly-overridden-username-other' |
| |
| In the example above, a fixture value is overridden by the test parameter value. Note that the value of the fixture |
| can be overridden this way even if the test doesn't use it directly (doesn't mention it in the function prototype). |
| |
| |
| Override a parametrized fixture with non-parametrized one and vice versa |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Given the tests file structure is: |
| |
| :: |
| |
| tests/ |
| __init__.py |
| |
| conftest.py |
| # content of tests/conftest.py |
| import pytest |
| |
| @pytest.fixture(params=['one', 'two', 'three']) |
| def parametrized_username(request): |
| return request.param |
| |
| @pytest.fixture |
| def non_parametrized_username(request): |
| return 'username' |
| |
| test_something.py |
| # content of tests/test_something.py |
| import pytest |
| |
| @pytest.fixture |
| def parametrized_username(): |
| return 'overridden-username' |
| |
| @pytest.fixture(params=['one', 'two', 'three']) |
| def non_parametrized_username(request): |
| return request.param |
| |
| def test_username(parametrized_username): |
| assert parametrized_username == 'overridden-username' |
| |
| def test_parametrized_username(non_parametrized_username): |
| assert non_parametrized_username in ['one', 'two', 'three'] |
| |
| test_something_else.py |
| # content of tests/test_something_else.py |
| def test_username(parametrized_username): |
| assert parametrized_username in ['one', 'two', 'three'] |
| |
| def test_username(non_parametrized_username): |
| assert non_parametrized_username == 'username' |
| |
| In the example above, a parametrized fixture is overridden with a non-parametrized version, and |
| a non-parametrized fixture is overridden with a parametrized version for certain test module. |
| The same applies for the test folder level obviously. |
