hypothesis - Man Page

Suppose we've written a run length encoding system and we want to test it out.

We have the following code which I took straight from the Rosetta Code wiki (OK, I removed some commented out code and fixed the formatting, but there are no functional modifications):

def encode(input_string):
    count = 1
    prev = ""
    lst = []
    for character in input_string:
        if character != prev:
            if prev:
                entry = (prev, count)
                lst.append(entry)
            count = 1
            prev = character
        else:
            count += 1
    entry = (character, count)
    lst.append(entry)
    return lst


def decode(lst):
    q = ""
    for character, count in lst:
        q += character * count
    return q

We want to write a test for this that will check some invariant of these functions.

The invariant one tends to try when you've got this sort of encoding / decoding is that if you encode something and then decode it then you get the same value back.

Let's see how you'd do that with Hypothesis:

from hypothesis import given
from hypothesis.strategies import text


@given(text())
def test_decode_inverts_encode(s):
    assert decode(encode(s)) == s

(For this example we'll just let pytest discover and run the test. We'll cover other ways you could have run it later).

The text function returns what Hypothesis calls a search strategy. An object with methods that describe how to generate and simplify certain kinds of values. The @given decorator then takes our test function and turns it into a parametrized one which, when called, will run the test function over a wide range of matching data from that strategy.

Anyway, this test immediately finds a bug in the code:

Falsifying example: test_decode_inverts_encode(s='')

UnboundLocalError: local variable 'character' referenced before assignment

Hypothesis correctly points out that this code is simply wrong if called on an empty string.

If we fix that by just adding the following code to the beginning of our encode function then Hypothesis tells us the code is correct (by doing nothing as you'd expect a passing test to).

if not input_string:
    return []

If we wanted to make sure this example was always checked we could add it in explicitly by using the @example decorator:

from hypothesis import example, given, strategies as st


@given(st.text())
@example("")
def test_decode_inverts_encode(s):
    assert decode(encode(s)) == s

This can be useful to show other developers (or your future self) what kinds of data are valid inputs, or to ensure that particular edge cases such as "" are tested every time.  It's also great for regression tests because although Hypothesis will remember failing examples, we don't recommend distributing that database.

It's also worth noting that both @example and @given support keyword arguments as well as positional. The following would have worked just as well:

@given(s=st.text())
@example(s="")
def test_decode_inverts_encode(s):
    assert decode(encode(s)) == s

Suppose we had a more interesting bug and forgot to reset the count each time. Say we missed a line in our encode method:

def encode(input_string):
    count = 1
    prev = ""
    lst = []
    for character in input_string:
        if character != prev:
            if prev:
                entry = (prev, count)
                lst.append(entry)
            # count = 1  # Missing reset operation
            prev = character
        else:
            count += 1
    entry = (character, count)
    lst.append(entry)
    return lst

Hypothesis quickly informs us of the following example:

Falsifying example: test_decode_inverts_encode(s='001')

Note that the example provided is really quite simple. Hypothesis doesn't just find any counter-example to your tests, it knows how to simplify the examples it finds to produce small easy to understand ones. In this case, two identical values are enough to set the count to a number different from one, followed by another distinct value which should have reset the count but in this case didn't.

Hypothesis is available on PyPI as "hypothesis". You can install it with:

pip install hypothesis

You can install the dependencies for optional extensions with e.g. pip install hypothesis[pandas,django].

If you want to install directly from the source code (e.g. because you want to make changes and install the changed version), check out the instructions in CONTRIBUTING.rst.

In our example above we just let pytest discover and run our tests, but we could also have run it explicitly ourselves:

if __name__ == "__main__":
    test_decode_inverts_encode()

We could also have done this as a python:unittest.TestCase:

import unittest


class TestEncoding(unittest.TestCase):
    @given(text())
    def test_decode_inverts_encode(self, s):
        self.assertEqual(decode(encode(s)), s)


if __name__ == "__main__":
    unittest.main()

A detail: This works because Hypothesis ignores any arguments it hasn't been told to provide (positional arguments start from the right), so the self argument to the test is simply ignored and works as normal. This also means that Hypothesis will play nicely with other ways of parameterizing tests. e.g it works fine if you use pytest fixtures for some arguments and Hypothesis for others.

A test in Hypothesis consists of two parts: A function that looks like a normal test in your test framework of choice but with some additional arguments, and a @given decorator that specifies how to provide those arguments.

Here are some other examples of how you could use that:

from hypothesis import given, strategies as st


@given(st.integers(), st.integers())
def test_ints_are_commutative(x, y):
    assert x + y == y + x


@given(x=st.integers(), y=st.integers())
def test_ints_cancel(x, y):
    assert (x + y) - y == x


@given(st.lists(st.integers()))
def test_reversing_twice_gives_same_list(xs):
    # This will generate lists of arbitrary length (usually between 0 and
    # 100 elements) whose elements are integers.
    ys = list(xs)
    ys.reverse()
    ys.reverse()
    assert xs == ys


@given(st.tuples(st.booleans(), st.text()))
def test_look_tuples_work_too(t):
    # A tuple is generated as the one you provided, with the corresponding
    # types in those positions.
    assert len(t) == 2
    assert isinstance(t[0], bool)
    assert isinstance(t[1], str)

Note that as we saw in the above example you can pass arguments to @given either as positional or as keywords.

You should now know enough of the basics to write some tests for your code using Hypothesis. The best way to learn is by doing, so go have a try.

If you're stuck for ideas for how to use this sort of test for your code, here are some good starting points:

1. Try just calling functions with appropriate arbitrary data and see if they crash. You may be surprised how often this works. e.g. note that the first bug we found in the encoding example didn't even get as far as our assertion: It crashed because it couldn't handle the data we gave it, not because it did the wrong thing.
2. Look for duplication in your tests. Are there any cases where you're testing the same thing with multiple different examples? Can you generalise that to a single test using Hypothesis?
3. This piece is designed for an F# implementation, but is still very good advice which you may find helps give you good ideas for using Hypothesis.

If you have any trouble getting started, don't feel shy about asking for help.

Normally the output of a failing test will look something like:

Falsifying example: test_a_thing(x=1, y="foo")

With the repr of each keyword argument being printed.

Sometimes this isn't enough, either because you have a value with a __repr__() method that isn't very descriptive or because you need to see the output of some intermediate steps of your test. That's where the note function comes in:

hypothesis.note(value)

Report this value for the minimal failing example.

>>> from hypothesis import given, note, strategies as st
>>> @given(st.lists(st.integers()), st.randoms())
... def test_shuffle_is_noop(ls, r):
...     ls2 = list(ls)
...     r.shuffle(ls2)
...     note(f"Shuffle: {ls2!r}")
...     assert ls == ls2
...
>>> try:
...     test_shuffle_is_noop()
... except AssertionError:
...     print("ls != ls2")
...
Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
Shuffle: [1, 0]
ls != ls2

The note is printed for the minimal failing example of the test in order to include any additional information you might need in your test.

If you are using pytest you can see a number of statistics about the executed tests by passing the command line argument --hypothesis-show-statistics. This will include some general statistics about the test:

For example if you ran the following with --hypothesis-show-statistics:

from hypothesis import given, strategies as st


@given(st.integers())
def test_integers(i):
    pass

You would see:

- during generate phase (0.06 seconds):
    - Typical runtimes: < 1ms, ~ 47% in data generation
    - 100 passing examples, 0 failing examples, 0 invalid examples
- Stopped because settings.max_examples=100

The final "Stopped because" line is particularly important to note: It tells you the setting value that determined when the test should stop trying new examples. This can be useful for understanding the behaviour of your tests. Ideally you'd always want this to be max_examples.

In some cases (such as filtered and recursive strategies) you will see events mentioned which describe some aspect of the data generation:

from hypothesis import given, strategies as st


@given(st.integers().filter(lambda x: x % 2 == 0))
def test_even_integers(i):
    pass

You would see something like:

test_even_integers:

  - during generate phase (0.08 seconds):
      - Typical runtimes: < 1ms, ~ 57% in data generation
      - 100 passing examples, 0 failing examples, 12 invalid examples
      - Events:
        * 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
        * 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
  - Stopped because settings.max_examples=100

You can also mark custom events in a test using the event function:

hypothesis.event(value, payload='')

Record an event that occurred during this test. Statistics on the number of test runs with each event will be reported at the end if you run Hypothesis in statistics reporting mode.

Event values should be strings or convertible to them.  If an optional payload is given, it will be included in the string for Test statistics.

from hypothesis import event, given, strategies as st


@given(st.integers().filter(lambda x: x % 2 == 0))
def test_even_integers(i):
    event(f"i mod 3 = {i%3}")

You will then see output like:

test_even_integers:

  - during generate phase (0.09 seconds):
      - Typical runtimes: < 1ms, ~ 59% in data generation
      - 100 passing examples, 0 failing examples, 32 invalid examples
      - Events:
        * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
        * 31.06%, i mod 3 = 2
        * 28.79%, i mod 3 = 0
        * 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
        * 15.91%, i mod 3 = 1
  - Stopped because settings.max_examples=100

Arguments to event can be any hashable type, but two events will be considered the same if they are the same when converted to a string with python:str.

Sometimes Hypothesis doesn't give you exactly the right sort of data you want - it's mostly of the right shape, but some examples won't work and you don't want to care about them. You can just ignore these by aborting the test early, but this runs the risk of accidentally testing a lot less than you think you are. Also it would be nice to spend less time on bad examples - if you're running 100 examples per test (the default) and it turns out 70 of those examples don't match your needs, that's a lot of wasted time.

hypothesis.assume(condition)

Calling assume is like an assert that marks the example as bad, rather than failing the test.

This allows you to specify properties that you assume will be true, and let Hypothesis try to avoid similar examples in future.

For example suppose you had the following test:

@given(floats())
def test_negation_is_self_inverse(x):
    assert x == -(-x)

Running this gives us:

Falsifying example: test_negation_is_self_inverse(x=float('nan'))
AssertionError

This is annoying. We know about NaN and don't really care about it, but as soon as Hypothesis finds a NaN example it will get distracted by that and tell us about it. Also the test will fail and we want it to pass.

So let's block off this particular example:

from math import isnan


@given(floats())
def test_negation_is_self_inverse_for_non_nan(x):
    assume(not isnan(x))
    assert x == -(-x)

And this passes without a problem.

In order to avoid the easy trap where you assume a lot more than you intended, Hypothesis will fail a test when it can't find enough examples passing the assumption.

If we'd written:

@given(floats())
def test_negation_is_self_inverse_for_non_nan(x):
    assume(False)
    assert x == -(-x)

Then on running we'd have got the exception:

Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions

Hypothesis has an adaptive exploration strategy to try to avoid things which falsify assumptions, which should generally result in it still being able to find examples in hard to find situations.

Suppose we had the following:

@given(lists(integers()))
def test_sum_is_positive(xs):
    assert sum(xs) > 0

Unsurprisingly this fails and gives the falsifying example [].

Adding assume(xs) to this removes the trivial empty example and gives us [0].

Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list of positive integers is positive.

The reason that this should be surprising is not that it doesn't find a counter-example, but that it finds enough examples at all.

In order to make sure something interesting is happening, suppose we wanted to try this for long lists. e.g. suppose we added an assume(len(xs) > 10) to it. This should basically never find an example: a naive strategy would find fewer than one in a thousand examples, because if each element of the list is negative with probability one-half, you'd have to have ten of these go the right way by chance. In the default configuration Hypothesis gives up long before it's tried 1000 examples (by default it tries 200).

Here's what happens if we try to run this:

@given(lists(integers()))
def test_sum_is_positive(xs):
    assume(len(xs) > 10)
    assume(all(x > 0 for x in xs))
    print(xs)
    assert sum(xs) > 0

In: test_sum_is_positive()

[17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
[6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
[25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2, ...]
[17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
[13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...]
[2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]

As you can see, Hypothesis doesn't find many examples here, but it finds some - enough to keep it happy.

In general if you can shape your strategies better to your tests you should - for example integers(1, 1000) is a lot better than assume(1 <= x <= 1000), but assume will take you a long way if you can't.

The type of object that is used to explore the examples given to your test function is called a SearchStrategy. These are created using the functions exposed in the hypothesis.strategies module.

Many of these strategies expose a variety of arguments you can use to customize generation. For example for integers you can specify min and max values of integers you want. If you want to see exactly what a strategy produces you can ask for an example:

>>> integers(min_value=0, max_value=10).example()
1

Many strategies are built out of other strategies. For example, if you want to define a tuple you need to say what goes in each element:

>>> from hypothesis.strategies import tuples
>>> tuples(integers(), integers()).example()
(-24597, 12566)

Further details are available in a separate document.

hypothesis.given(*_given_arguments, **_given_kwargs)

A decorator for turning a test function that accepts arguments into a randomized test.

This is the main entry point to Hypothesis.

The @given decorator may be used to specify which arguments of a function should be parametrized over. You can use either positional or keyword arguments, but not a mixture of both.

For example all of the following are valid uses:

@given(integers(), integers())
def a(x, y):
    pass


@given(integers())
def b(x, y):
    pass


@given(y=integers())
def c(x, y):
    pass


@given(x=integers())
def d(x, y):
    pass


@given(x=integers(), y=integers())
def e(x, **kwargs):
    pass


@given(x=integers(), y=integers())
def f(x, *args, **kwargs):
    pass


class SomeTest(TestCase):
    @given(integers())
    def test_a_thing(self, x):
        pass

The following are not:

@given(integers(), integers(), integers())
def g(x, y):
    pass


@given(integers())
def h(x, *args):
    pass


@given(integers(), x=integers())
def i(x, y):
    pass


@given()
def j(x, y):
    pass

The rules for determining what are valid uses of given are as follows:

1. You may pass any keyword argument to given.
2. Positional arguments to given are equivalent to the rightmost named arguments for the test function.
3. Positional arguments may not be used if the underlying test function has varargs, arbitrary keywords, or keyword-only arguments.
4. Functions tested with given may not have any defaults.

The reason for the "rightmost named arguments" behaviour is so that using @given with instance methods works: self will be passed to the function as normal and not be parametrized over.

The function returned by given has all the same arguments as the original test, minus those that are filled in by @given. Check the notes on framework compatibility to see how this affects other testing libraries you may be using.

Targeted property-based testing combines the advantages of both search-based and property-based testing.  Instead of being completely random, T-PBT uses a search-based component to guide the input generation towards values that have a higher probability of falsifying a property.  This explores the input space more effectively and requires fewer tests to find a bug or achieve a high confidence in the system being tested than random PBT. (Löscher and Sagonas)

This is not always a good idea - for example calculating the search metric might take time better spent running more uniformly-random test cases, or your target metric might accidentally lead Hypothesis away from bugs - but if there is a natural metric like "floating-point error", "load factor" or "queue length", we encourage you to experiment with targeted testing.

hypothesis.target(observation, *, label='')

Calling this function with an int or float observation gives it feedback with which to guide our search for inputs that will cause an error, in addition to all the usual heuristics.  Observations must always be finite.

Hypothesis will try to maximize the observed value over several examples; almost any metric will work so long as it makes sense to increase it. For example, -abs(error) is a metric that increases as error approaches zero.

Example metrics:

• Number of elements in a collection, or tasks in a queue
• Mean or maximum runtime of a task (or both, if you use label)
• Compression ratio for data (perhaps per-algorithm or per-level)
• Number of steps taken by a state machine

The optional label argument can be used to distinguish between and therefore separately optimise distinct observations, such as the mean and standard deviation of a dataset.  It is an error to call target() with any label more than once per test case.

NOTE:

The more examples you run, the better this technique works.

As a rule of thumb, the targeting effect is noticeable above max_examples=1000, and immediately obvious by around ten thousand examples per label used by your test.

Test statistics include the best score seen for each label, which can help avoid the threshold problem when the minimal example shrinks right down to the threshold of failure (issue #2180).

from hypothesis import given, strategies as st, target


@given(st.floats(0, 1e100), st.floats(0, 1e100), st.floats(0, 1e100))
def test_associativity_with_target(a, b, c):
    ab_c = (a + b) + c
    a_bc = a + (b + c)
    difference = abs(ab_c - a_bc)
    target(difference)  # Without this, the test almost always passes
    assert difference < 2.0

We recommend that users also skim the papers introducing targeted PBT; from ISSTA 2017 and ICST 2018. For the curious, the initial implementation in Hypothesis uses hill-climbing search via a mutating fuzzer, with some tactics inspired by simulated annealing to avoid getting stuck and endlessly mutating a local maximum.

Hypothesis provides you with a hook that lets you control how it runs examples.

This lets you do things like set up and tear down around each example, run examples in a subprocess, transform coroutine tests into normal tests, etc. For example, TransactionTestCase in the Django extra runs each example in a separate database transaction.

The way this works is by introducing the concept of an executor. An executor is essentially a function that takes a block of code and run it. The default executor is:

def default_executor(function):
    return function()

You define executors by defining a method execute_example on a class. Any test methods on that class with @given used on them will use self.execute_example as an executor with which to run tests. For example, the following executor runs all its code twice:

from unittest import TestCase


class TestTryReallyHard(TestCase):
    @given(integers())
    def test_something(self, i):
        perform_some_unreliable_operation(i)

    def execute_example(self, f):
        f()
        return f()

Note: The functions you use in map, etc. will run inside the executor. i.e. they will not be called until you invoke the function passed to execute_example.

An executor must be able to handle being passed a function which returns None, otherwise it won't be able to run normal test cases. So for example the following executor is invalid:

from unittest import TestCase


class TestRunTwice(TestCase):
    def execute_example(self, f):
        return f()()

and should be rewritten as:

from unittest import TestCase


class TestRunTwice(TestCase):
    def execute_example(self, f):
        result = f()
        if callable(result):
            result = result()
        return result

An alternative hook is provided for use by test runner extensions such as pytest-trio, which cannot use the execute_example method. This is not recommended for end-users - it is better to write a complete test function directly, perhaps by using a decorator to perform the same transformation before applying @given.

@given(x=integers())
@pytest.mark.trio
async def test(x): ...


# Illustrative code, inside the pytest-trio plugin
test.hypothesis.inner_test = lambda x: trio.run(test, x)

For authors of test runners however, assigning to the inner_test attribute of the hypothesis attribute of the test will replace the interior test.

NOTE:

If the end user has also specified a custom executor using the execute_example method, it - and all other execution-time logic - will be applied to the new inner test assigned by the test runner.

While Hypothesis' example generation can be used for nondeterministic tests, debugging anything nondeterministic is usually a very frustrating exercise. To make things worse, our example shrinking relies on the same input causing the same failure each time - though we show the un-shrunk failure and a decent error message if it doesn't.

By default, Hypothesis will handle the global random and numpy.random random number generators for you, and you can register others:

hypothesis.register_random(r)

Register (a weakref to) the given Random-like instance for management by Hypothesis.

You can pass instances of structural subtypes of random.Random (i.e., objects with seed, getstate, and setstate methods) to register_random(r) to have their states seeded and restored in the same way as the global PRNGs from the random and numpy.random modules.

All global PRNGs, from e.g. simulation or scheduling frameworks, should be registered to prevent flaky tests. Hypothesis will ensure that the PRNG state is consistent for all test runs, always seeding them to zero and restoring the previous state after the test, or, reproducibly varied if you choose to use the random_module() strategy.

register_random only makes weakrefs to r, thus r will only be managed by Hypothesis as long as it has active references elsewhere at runtime. The pattern register_random(MyRandom()) will raise a ReferenceError to help protect users from this issue. This check does not occur for the PyPy interpreter. See the following example for an illustration of this issue

def my_BROKEN_hook():
    r = MyRandomLike()

    # `r` will be garbage collected after the hook resolved
    # and Hypothesis will 'forget' that it was registered
    register_random(r)  # Hypothesis will emit a warning


rng = MyRandomLike()


def my_WORKING_hook():
    register_random(rng)

In some cases, Hypothesis can work out what to do when you omit arguments. This is based on introspection, not magic, and therefore has well-defined limits.

builds() will check the signature of the target (using signature()). If there are required arguments with type annotations and no strategy was passed to builds(), from_type() is used to fill them in. You can also pass the value ... (Ellipsis) as a keyword argument, to force this inference for arguments with a default value.

>>> def func(a: int, b: str):
...     return [a, b]
...
>>> builds(func).example()
[-6993, '']

hypothesis.infer

@given does not perform any implicit inference for required arguments, as this would break compatibility with pytest fixtures. ... (python:Ellipsis), can be used as a keyword argument to explicitly fill in an argument from its type annotation.  You can also use the hypothesis.infer alias if writing a literal ... seems too weird.

@given(a=...)  # or @given(a=infer)
def test(a: int):
    pass


# is equivalent to
@given(a=from_type(int))
def test(a):
    pass

@given(...) can also be specified to fill all arguments from their type annotations.

@given(...)
def test(a: int, b: str):
    pass


# is equivalent to
@given(a=..., b=...)
def test(a, b):
    pass

Hypothesis does not inspect PEP 484 type comments at runtime.  While from_type() will work as usual, inference in builds() and @given will only work if you manually create the __annotations__ attribute (e.g. by using @annotations(...) and @returns(...) decorators).

The python:typing module changes between different Python releases, including at minor versions.  These are all supported on a best-effort basis, but you may encounter problems.  Please report them to us, and consider updating to a newer version of Python as a workaround.

If you install Hypothesis and use mypy 0.590+, or another PEP 561-compatible tool, the type checker should automatically pick up our type hints.

NOTE:

There are known issues inferring the type of examples generated by deferred(), recursive(), one_of(), dictionaries(), and fixed_dictionaries(). We will fix these, and require correspondingly newer versions of Mypy for type hinting, as the ecosystem improves.

Projects that provide Hypothesis strategies and use type hints may wish to annotate their strategies too.  This is a supported use-case, again on a best-effort provisional basis.  For example:

def foo_strategy() -> SearchStrategy[Foo]: ...

class hypothesis.strategies.SearchStrategy

SearchStrategy is the type of all strategy objects.  It is a generic type, and covariant in the type of the examples it creates.  For example:

• integers() is of type SearchStrategy[int].
• lists(integers()) is of type SearchStrategy[List[int]].
• SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is a subtype of Animal (as seems likely).

WARNING:

SearchStrategy should only be used in type hints.  Please do not inherit from, compare to, or otherwise use it in any way outside of type hints.  The only supported way to construct objects of this type is to use the functions provided by the hypothesis.strategies module!

Hypothesis includes a tiny plugin to improve integration with pytest, which is activated by default (but does not affect other test runners). It aims to improve the integration between Hypothesis and Pytest by providing extra information and convenient access to config options.

• pytest --hypothesis-show-statistics can be used to display test and data generation statistics.
• pytest --hypothesis-profile=<profile name> can be used to load a settings profile.
• pytest --hypothesis-verbosity=<level name> can be used to override the current verbosity level.
• pytest --hypothesis-seed=<an int> can be used to reproduce a failure with a particular seed.
• pytest --hypothesis-explain can be used to temporarily enable the explain phase.

Finally, all tests that are defined with Hypothesis automatically have @pytest.mark.hypothesis applied to them.  See here for information on working with markers.

NOTE:

TIP:

Want an integrated workflow for your team's local tests, CI, and continuous fuzzing?
Use HypoFuzz to fuzz your whole test suite,
and find more bugs without more tests!

Sometimes, you might want to point a traditional fuzzer such as python-afl, pythonfuzz, or Google's atheris (for Python and native extensions) at your code. Wouldn't it be nice if you could use any of your @given tests as fuzz targets, instead of converting bytestrings into your objects by hand?

@given(st.text())
def test_foo(s): ...


# This is a traditional fuzz target - call it with a bytestring,
# or a binary IO object, and it runs the test once.
fuzz_target = test_foo.hypothesis.fuzz_one_input

# For example:
fuzz_target(b"\x00\x00\x00\x00\x00\x00\x00\x00")
fuzz_target(io.BytesIO(...))

Depending on the input to fuzz_one_input, one of three things will happen:

• If the bytestring was invalid, for example because it was too short or failed a filter or assume() too many times, fuzz_one_input returns None.
• If the bytestring was valid and the test passed, fuzz_one_input returns a canonicalised and pruned buffer which will replay that test case.  This is provided as an option to improve the performance of mutating fuzzers, but can safely be ignored.
• If the test failed, i.e. raised an exception, fuzz_one_input will add the pruned buffer to the Hypothesis example database and then re-raise that exception.  All you need to do to reproduce, minimize, and de-duplicate all the failures found via fuzzing is run your test suite!

Note that the interpretation of both input and output bytestrings is specific to the exact version of Hypothesis you are using and the strategies given to the test, just like the example database and @reproduce_failure decorator.

fuzz_one_input uses just enough of Hypothesis' internals to drive your test function with a fuzzer-provided bytestring, and most settings therefore have no effect in this mode.  We recommend running your tests the usual way before fuzzing to get the benefits of healthchecks, as well as afterwards to replay, shrink, deduplicate, and report whatever errors were discovered.

• The database setting is used by fuzzing mode - adding failures to the database to be replayed when you next run your tests is our preferred reporting mechanism and response to the 'fuzzer taming' problem.
• The verbosity and stateful_step_count settings work as usual.

The deadline, derandomize, max_examples, phases, print_blob, report_multiple_bugs, and suppress_health_check settings do not affect fuzzing mode.

As discussed in issue #2719, Hypothesis is not truly thread-safe and that's unlikely to change in the future.  This policy therefore describes what you can expect if you use Hypothesis with multiple threads.

Running tests in multiple processes, e.g. with pytest -n auto, is fully supported and we test this regularly in CI - thanks to process isolation, we only need to ensure that DirectoryBasedExampleDatabase can't tread on its own toes too badly.  If you find a bug here we will fix it ASAP.

Running separate tests in multiple threads is not something we design or test for, and is not formally supported.  That said, anecdotally it does mostly work and we would like it to keep working - we accept reasonable patches and low-priority bug reports.  The main risks here are global state, shared caches, and cached strategies.

Using multiple threads within a single test , or running a single test simultaneously in multiple threads, makes it pretty easy to trigger internal errors.  We usually accept patches for such issues unless readability or single-thread performance suffer.

Hypothesis assumes that tests are single-threaded, or do a sufficiently-good job of pretending to be single-threaded.  Tests that use helper threads internally should be OK, but the user must be careful to ensure that test outcomes are still deterministic. In particular it counts as nondeterministic if helper-thread timing changes the sequence of dynamic draws using e.g. the data().

Interacting with any Hypothesis APIs from helper threads might do weird/bad things, so avoid that too - we rely on thread-local variables in a few places, and haven't explicitly tested/audited how they respond to cross-thread API calls.  While data() and equivalents are the most obvious danger, other APIs might also be subtly affected.

class hypothesis.settings(parent=None, *, max_examples=not_set, derandomize=not_set, database=not_set, verbosity=not_set, phases=not_set, stateful_step_count=not_set, report_multiple_bugs=not_set, suppress_health_check=not_set, deadline=not_set, print_blob=not_set, backend=not_set)

A settings object configures options including verbosity, runtime controls, persistence, determinism, and more.

Default values are picked up from the settings.default object and changes made there will be picked up in newly created settings.

backend

EXPERIMENTAL AND UNSTABLE - see Alternative backends for Hypothesis. The importable name of a backend which Hypothesis should use to generate primitive types.  We aim to support heuristic-random, solver-based, and fuzzing-based backends.

default value: (dynamically calculated)

database

An instance of ExampleDatabase that will be used to save examples to and load previous examples from. May be None in which case no storage will be used.

See the example database documentation for a list of built-in example database implementations, and how to define custom implementations.

default value: (dynamically calculated)

deadline

If set, a duration (as timedelta, or integer or float number of milliseconds) that each individual example (i.e. each time your test function is called, not the whole decorated test) within a test is not allowed to exceed. Tests which take longer than that may be converted into errors (but will not necessarily be if close to the deadline, to allow some variability in test run time).

Set this to None to disable this behaviour entirely.

default value: timedelta(milliseconds=200)

derandomize

If True, seed Hypothesis' random number generator using a hash of the test function, so that every run will test the same set of examples until you update Hypothesis, Python, or the test function.

This allows you to check for regressions and look for bugs using separate settings profiles - for example running quick deterministic tests on every commit, and a longer non-deterministic nightly testing run.

default value: False

max_examples

Once this many satisfying examples have been considered without finding any counter-example, Hypothesis will stop looking.

Note that we might call your test function fewer times if we find a bug early or can tell that we've exhausted the search space; or more if we discard some examples due to use of .filter(), assume(), or a few other things that can prevent the test case from completing successfully.

The default value is chosen to suit a workflow where the test will be part of a suite that is regularly executed locally or on a CI server, balancing total running time against the chance of missing a bug.

If you are writing one-off tests, running tens of thousands of examples is quite reasonable as Hypothesis may miss uncommon bugs with default settings. For very complex code, we have observed Hypothesis finding novel bugs after several million examples while testing SymPy. If you are running more than 100k examples for a test, consider using our integration for coverage-guided fuzzing - it really shines when given minutes or hours to run.

default value: 100

phases

Control which phases should be run. See the full documentation for more details

default value: (Phase.explicit, Phase.reuse, Phase.generate, Phase.target, Phase.shrink, Phase.explain)

print_blob

If set to True, Hypothesis will print code for failing examples that can be used with @reproduce_failure to reproduce the failing example. The default is True if the CI or TF_BUILD env vars are set, False otherwise.

default value: (dynamically calculated)

report_multiple_bugs

Because Hypothesis runs the test many times, it can sometimes find multiple bugs in a single run.  Reporting all of them at once is usually very useful, but replacing the exceptions can occasionally clash with debuggers. If disabled, only the exception with the smallest minimal example is raised.

default value: True

stateful_step_count

Number of steps to run a stateful program for before giving up on it breaking.

default value: 50

suppress_health_check

A list of HealthCheck items to disable.

default value: ()

verbosity

Control the verbosity level of Hypothesis messages

default value: Verbosity.normal

Hypothesis divides tests into logically distinct phases:

1.

Running explicit examples provided with the @example decorator.

2.

Rerunning a selection of previously failing examples to reproduce a previously seen error.

3.

Generating new examples.

4.

Mutating examples for targeted property-based testing (requires generate phase).

5.

Attempting to shrink an example found in previous phases (other than phase 1 - explicit examples cannot be shrunk). This turns potentially large and complicated examples which may be hard to read into smaller and simpler ones.

6.

Attempting to explain why your test failed (requires shrink phase).

NOTE:

The explain phase has two parts, each of which is best-effort - if Hypothesis can't find a useful explanation, we'll just print the minimal failing example.

Following the first failure, Hypothesis will (usually) track which lines of code are always run on failing but never on passing inputs. This relies on python:sys.settrace(), and is therefore automatically disabled on PyPy or if you are using coverage or a debugger.  If there are no clearly suspicious lines of code, we refuse the temptation to guess.

After shrinking to a minimal failing example, Hypothesis will try to find parts of the example -- e.g. separate args to @given() -- which can vary freely without changing the result of that minimal failing example. If the automated experiments run without finding a passing variation, we leave a comment in the final report:

test_x_divided_by_y(
    x=0,  # or any other generated value
    y=0,
)

Just remember that the lack of an explanation sometimes just means that Hypothesis couldn't efficiently find one, not that no explanation (or simpler failing example) exists.

The phases setting provides you with fine grained control over which of these run, with each phase corresponding to a value on the Phase enum:

class hypothesis.Phase(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

explicit = 0

controls whether explicit examples are run.

reuse = 1

controls whether previous examples will be reused.

generate = 2

controls whether new examples will be generated.

target = 3

controls whether examples will be mutated for targeting.

shrink = 4

controls whether examples will be shrunk.

explain = 5

controls whether Hypothesis attempts to explain test failures.

The phases argument accepts a collection with any subset of these. e.g. settings(phases=[Phase.generate, Phase.shrink]) will generate new examples and shrink them, but will not run explicit examples or reuse previous failures, while settings(phases=[Phase.explicit]) will only run the explicit examples.

To see what's going on while Hypothesis runs your tests, you can turn up the verbosity setting.

>>> from hypothesis import find, settings, Verbosity
>>> from hypothesis.strategies import lists, integers
>>> @given(lists(integers()))
... @settings(verbosity=Verbosity.verbose)
... def f(x):
...     assert not any(x)
... f()
Trying example: []
Falsifying example: [-1198601713, -67, 116, -29578]
Shrunk example to [-1198601713]
Shrunk example to [-1198601600]
Shrunk example to [-1191228800]
Shrunk example to [-8421504]
Shrunk example to [-32896]
Shrunk example to [-128]
Shrunk example to [64]
Shrunk example to [32]
Shrunk example to [16]
Shrunk example to [8]
Shrunk example to [4]
Shrunk example to [3]
Shrunk example to [2]
Shrunk example to [1]
[1]

The four levels are quiet, normal, verbose and debug. normal is the default, while in quiet mode Hypothesis will not print anything out, not even the final falsifying example. debug is basically verbose but a bit more so. You probably don't want it.

If you are using pytest, you may also need to disable output capturing for passing tests.

Settings can be created by calling settings with any of the available settings values. Any absent ones will be set to defaults:

>>> from hypothesis import settings
>>> settings().max_examples
100
>>> settings(max_examples=10).max_examples
10

You can also pass a 'parent' settings object as the first argument, and any settings you do not specify as keyword arguments will be copied from the parent settings:

>>> parent = settings(max_examples=10)
>>> child = settings(parent, deadline=None)
>>> parent.max_examples == child.max_examples == 10
True
>>> parent.deadline
200
>>> child.deadline is None
True

At any given point in your program there is a current default settings, available as settings.default. As well as being a settings object in its own right, all newly created settings objects which are not explicitly based off another settings are based off the default, so will inherit any values that are not explicitly set from it.

You can change the defaults by using profiles.

Depending on your environment you may want different default settings. For example: during development you may want to lower the number of examples to speed up the tests. However, in a CI environment you may want more examples so you are more likely to find bugs.

Hypothesis allows you to define different settings profiles. These profiles can be loaded at any time.

static settings.register_profile(name, parent=None, **kwargs)

Registers a collection of values to be used as a settings profile.

Settings profiles can be loaded by name - for example, you might create a 'fast' profile which runs fewer examples, keep the 'default' profile, and create a 'ci' profile that increases the number of examples and uses a different database to store failures.

The arguments to this method are exactly as for settings: optional parent settings, and keyword arguments for each setting that will be set differently to parent (or settings.default, if parent is None).

static settings.get_profile(name)

Return the profile with the given name.

static settings.load_profile(name)

Loads in the settings defined in the profile provided.

If the profile does not exist, InvalidArgument will be raised. Any setting not defined in the profile will be the library defined default for that setting.

Loading a profile changes the default settings but will not change the behaviour of tests that explicitly change the settings.

>>> from hypothesis import settings
>>> settings.register_profile("ci", max_examples=1000)
>>> settings().max_examples
100
>>> settings.load_profile("ci")
>>> settings().max_examples
1000

Instead of loading the profile and overriding the defaults you can retrieve profiles for specific tests.

>>> settings.get_profile("ci").max_examples
1000

Optionally, you may define the environment variable to load a profile for you. This is the suggested pattern for running your tests on CI. The code below should run in a conftest.py or any setup/initialization section of your test suite. If this variable is not defined the Hypothesis defined defaults will be loaded.

>>> import os
>>> from hypothesis import settings, Verbosity
>>> settings.register_profile("ci", max_examples=1000)
>>> settings.register_profile("dev", max_examples=10)
>>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
>>> settings.load_profile(os.getenv("HYPOTHESIS_PROFILE", "default"))

If you are using the hypothesis pytest plugin and your profiles are registered by your conftest you can load one with the command line option --hypothesis-profile.

$ pytest tests --hypothesis-profile <profile-name>

Hypothesis' health checks are designed to detect and warn you about performance problems where your tests are slow, inefficient, or generating very large examples.

If this is expected, e.g. when generating large arrays or dataframes, you can selectively disable them with the suppress_health_check setting. The argument for this parameter is a list with elements drawn from any of the class-level attributes of the HealthCheck class. Using a value of list(HealthCheck) will disable all health checks.

class hypothesis.HealthCheck(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Arguments for suppress_health_check.

Each member of this enum is a type of health check to suppress.

data_too_large = 1

Checks if too many examples are aborted for being too large.

This is measured by the number of random choices that Hypothesis makes in order to generate something, not the size of the generated object. For example, choosing a 100MB object from a predefined list would take only a few bits, while generating 10KB of JSON from scratch might trigger this health check.

filter_too_much = 2

Check for when the test is filtering out too many examples, either through use of assume() or filter(), or occasionally for Hypothesis internal reasons.

too_slow = 3

Check for when your data generation is extremely slow and likely to hurt testing.

return_value = 5

Deprecated; we always error if a test returns a non-None value.

large_base_example = 7

Checks if the natural example to shrink towards is very large.

not_a_test_method = 8

Deprecated; we always error if @given is applied to a method defined by python:unittest.TestCase (i.e. not a test).

function_scoped_fixture = 9

Checks if @given has been applied to a test with a pytest function-scoped fixture. Function-scoped fixtures run once for the whole function, not once per example, and this is usually not what you want.

Because of this limitation, tests that need to set up or reset state for every example need to do so manually within the test itself, typically using an appropriate context manager.

Suppress this health check only in the rare case that you are using a function-scoped fixture that does not need to be reset between individual examples, but for some reason you cannot use a wider fixture scope (e.g. session scope, module scope, class scope).

This check requires the Hypothesis pytest plugin, which is enabled by default when running Hypothesis inside pytest.

differing_executors = 10

Checks if @given has been applied to a test which is executed by different executors. If your test function is defined as a method on a class, that class will be your executor, and subclasses executing an inherited test is a common way for things to go wrong.

The correct fix is often to bring the executor instance under the control of hypothesis by explicit parametrization over, or sampling from, subclasses, or to refactor so that @given is specified on leaf subclasses.

Functions for building strategies are all available in the hypothesis.strategies module. The salient functions from it are as follows:

hypothesis.strategies.binary(*, min_size=0, max_size=None)

Generates python:bytes.

The generated python:bytes will have a length of at least min_size and at most max_size.  If max_size is None there is no upper limit.

Examples from this strategy shrink towards smaller strings and lower byte values.

hypothesis.strategies.booleans()

Returns a strategy which generates instances of python:bool.

Examples from this strategy will shrink towards False (i.e. shrinking will replace True with False where possible).

hypothesis.strategies.builds(target, /, *args, **kwargs)

Generates values by drawing from args and kwargs and passing them to the callable (provided as the first positional argument) in the appropriate argument position.

e.g. builds(target, integers(), flag=booleans()) would draw an integer i and a boolean b and call target(i, flag=b).

If the callable has type annotations, they will be used to infer a strategy for required arguments that were not passed to builds.  You can also tell builds to infer a strategy for an optional argument by passing ... (python:Ellipsis) as a keyword argument to builds, instead of a strategy for that argument to the callable.

If the callable is a class defined with attrs, missing required arguments will be inferred from the attribute on a best-effort basis, e.g. by checking attrs standard validators. Dataclasses are handled natively by the inference from type hints.

Examples from this strategy shrink by shrinking the argument values to the callable.

hypothesis.strategies.characters(*, codec=None, min_codepoint=None, max_codepoint=None, categories=None, exclude_categories=None, exclude_characters=None, include_characters=None)

Generates characters, length-one python:strings, following specified filtering rules.

• When no filtering rules are specified, any character can be produced.
• If min_codepoint or max_codepoint is specified, then only characters having a codepoint in that range will be produced.
• If categories is specified, then only characters from those Unicode categories will be produced. This is a further restriction, characters must also satisfy min_codepoint and max_codepoint.
• If exclude_categories is specified, then any character from those categories will not be produced.  You must not pass both categories and exclude_categories; these arguments are alternative ways to specify exactly the same thing.
• If include_characters is specified, then any additional characters in that list will also be produced.
• If exclude_characters is specified, then any characters in that list will be not be produced. Any overlap between include_characters and exclude_characters will raise an exception.
• If codec is specified, only characters in the specified codec encodings will be produced.

The _codepoint arguments must be integers between zero and python:sys.maxunicode.  The _characters arguments must be collections of length-one unicode strings, such as a unicode string.

The _categories arguments must be used to specify either the one-letter Unicode major category or the two-letter Unicode general category.  For example, ('Nd', 'Lu') signifies "Number, decimal digit" and "Letter, uppercase".  A single letter ('major category') can be given to match all corresponding categories, for example 'P' for characters in any punctuation category.

We allow codecs from the codecs module and their aliases, platform specific and user-registered codecs if they are available, and python-specific text encodings (but not text or binary transforms). include_characters which cannot be encoded using this codec will raise an exception.  If non-encodable codepoints or categories are explicitly allowed, the codec argument will exclude them without raising an exception.

Examples from this strategy shrink towards the codepoint for '0', or the first allowable codepoint after it if '0' is excluded.

hypothesis.strategies.complex_numbers(*, min_magnitude=0, max_magnitude=None, allow_infinity=None, allow_nan=None, allow_subnormal=True, width=128)

Returns a strategy that generates python:complex numbers.

This strategy draws complex numbers with constrained magnitudes. The min_magnitude and max_magnitude parameters should be non-negative Real numbers; a value of None corresponds an infinite upper bound.

If min_magnitude is nonzero or max_magnitude is finite, it is an error to enable allow_nan.  If max_magnitude is finite, it is an error to enable allow_infinity.

allow_infinity, allow_nan, and allow_subnormal are applied to each part of the complex number separately, as for floats().

The magnitude constraints are respected up to a relative error of (around) floating-point epsilon, due to implementation via the system sqrt function.

The width argument specifies the maximum number of bits of precision required to represent the entire generated complex number. Valid values are 32, 64 or 128, which correspond to the real and imaginary components each having width 16, 32 or 64, respectively. Passing width=64 will still use the builtin 128-bit python:complex class, but always for values which can be exactly represented as two 32-bit floats.

Examples from this strategy shrink by shrinking their real and imaginary parts, as floats().

If you need to generate complex numbers with particular real and imaginary parts or relationships between parts, consider using builds(complex, ...) or @composite respectively.

hypothesis.strategies.composite(f)

Defines a strategy that is built out of potentially arbitrarily many other strategies.

This is intended to be used as a decorator. See the full documentation for more details about how to use this function.

Examples from this strategy shrink by shrinking the output of each draw call.

hypothesis.strategies.data()

This isn't really a normal strategy, but instead gives you an object which can be used to draw data interactively from other strategies.

See the rest of the documentation for more complete information.

Examples from this strategy do not shrink (because there is only one), but the result of calls to each data.draw() call shrink as they normally would.

class hypothesis.strategies.DataObject

This type only exists so that you can write type hints for tests using the data() strategy.  Do not use it directly!

hypothesis.strategies.dates(min_value=datetime.date.min, max_value=datetime.date.max)

A strategy for dates between min_value and max_value.

Examples from this strategy shrink towards January 1st 2000.

hypothesis.strategies.datetimes(min_value=datetime.datetime.min, max_value=datetime.datetime.max, *, timezones=none(), allow_imaginary=True)

A strategy for generating datetimes, which may be timezone-aware.

This strategy works by drawing a naive datetime between min_value and max_value, which must both be naive (have no timezone).

timezones must be a strategy that generates either None, for naive datetimes, or tzinfo objects for 'aware' datetimes. You can construct your own, though we recommend using one of these built-in strategies:

• with Python 3.9 or newer or backports.zoneinfo: hypothesis.strategies.timezones();
• with dateutil: hypothesis.extra.dateutil.timezones(); or
• with pytz: hypothesis.extra.pytz.timezones().

You may pass allow_imaginary=False to filter out "imaginary" datetimes which did not (or will not) occur due to daylight savings, leap seconds, timezone and calendar adjustments, etc.  Imaginary datetimes are allowed by default, because malformed timestamps are a common source of bugs.

Examples from this strategy shrink towards midnight on January 1st 2000, local time.

hypothesis.strategies.decimals(min_value=None, max_value=None, *, allow_nan=None, allow_infinity=None, places=None)

Generates instances of python:decimal.Decimal, which may be:

• A finite rational number, between min_value and max_value.
• Not a Number, if allow_nan is True.  None means "allow NaN, unless min_value and max_value are not None".
• Positive or negative infinity, if max_value and min_value respectively are None, and allow_infinity is not False.  None means "allow infinity, unless excluded by the min and max values".

Note that where floats have one NaN value, Decimals have four: signed, and either quiet or signalling.  See the decimal module docs for more information on special values.

If places is not None, all finite values drawn from the strategy will have that number of digits after the decimal place.

Examples from this strategy do not have a well defined shrink order but try to maximize human readability when shrinking.

hypothesis.strategies.deferred(definition)

A deferred strategy allows you to write a strategy that references other strategies that have not yet been defined. This allows for the easy definition of recursive and mutually recursive strategies.

The definition argument should be a zero-argument function that returns a strategy. It will be evaluated the first time the strategy is used to produce an example.

Example usage:

>>> import hypothesis.strategies as st
>>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
>>> x.example()
(((False, (True, True)), (False, True)), (True, True))
>>> x.example()
True

Mutual recursion also works fine:

>>> a = st.deferred(lambda: st.booleans() | b)
>>> b = st.deferred(lambda: st.tuples(a, a))
>>> a.example()
True
>>> b.example()
(False, (False, ((False, True), False)))

Examples from this strategy shrink as they normally would from the strategy returned by the definition.

hypothesis.strategies.dictionaries(keys, values, *, dict_class=<class 'dict'>, min_size=0, max_size=None)

Generates dictionaries of type dict_class with keys drawn from the keys argument and values drawn from the values argument.

The size parameters have the same interpretation as for lists().

Examples from this strategy shrink by trying to remove keys from the generated dictionary, and by shrinking each generated key and value.

class hypothesis.strategies.DrawFn

This type only exists so that you can write type hints for functions decorated with @composite.

@composite
def list_and_index(draw: DrawFn) -> Tuple[int, str]:
    i = draw(integers())  # type inferred as 'int'
    s = draw(text())  # type inferred as 'str'
    return i, s

hypothesis.strategies.emails(*, domains=domains())

A strategy for generating email addresses as unicode strings. The address format is specified in RFC 5322#section-3.4.1. Values shrink towards shorter local-parts and host domains.

If domains is given then it must be a strategy that generates domain names for the emails, defaulting to domains().

This strategy is useful for generating "user data" for tests, as mishandling of email addresses is a common source of bugs.

hypothesis.strategies.fixed_dictionaries(mapping, *, optional=None)

Generates a dictionary of the same type as mapping with a fixed set of keys mapping to strategies. mapping must be a dict subclass.

Generated values have all keys present in mapping, in iteration order, with the corresponding values drawn from mapping[key].

If optional is passed, the generated value may or may not contain each key from optional and a value drawn from the corresponding strategy. Generated values may contain optional keys in an arbitrary order.

Examples from this strategy shrink by shrinking each individual value in the generated dictionary, and omitting optional key-value pairs.

hypothesis.strategies.floats(min_value=None, max_value=None, *, allow_nan=None, allow_infinity=None, allow_subnormal=None, width=64, exclude_min=False, exclude_max=False)

Returns a strategy which generates floats.

• If min_value is not None, all values will be >= min_value (or > min_value if exclude_min).
• If max_value is not None, all values will be <= max_value (or < max_value if exclude_max).
• If min_value or max_value is not None, it is an error to enable allow_nan.
• If both min_value and max_value are not None, it is an error to enable allow_infinity.
• If inferred values range does not include subnormal values, it is an error to enable allow_subnormal.

Where not explicitly ruled out by the bounds, subnormals, infinities, and NaNs are possible values generated by this strategy.

The width argument specifies the maximum number of bits of precision required to represent the generated float. Valid values are 16, 32, or 64. Passing width=32 will still use the builtin 64-bit python:float class, but always for values which can be exactly represented as a 32-bit float.

The exclude_min and exclude_max argument can be used to generate numbers from open or half-open intervals, by excluding the respective endpoints. Excluding either signed zero will also exclude the other. Attempting to exclude an endpoint which is None will raise an error; use allow_infinity=False to generate finite floats.  You can however use e.g. min_value=-math.inf, exclude_min=True to exclude only one infinite endpoint.

Examples from this strategy have a complicated and hard to explain shrinking behaviour, but it tries to improve "human readability". Finite numbers will be preferred to infinity and infinity will be preferred to NaN.

hypothesis.strategies.fractions(min_value=None, max_value=None, *, max_denominator=None)

Returns a strategy which generates Fractions.

If min_value is not None then all generated values are no less than min_value.  If max_value is not None then all generated values are no greater than max_value.  min_value and max_value may be anything accepted by the Fraction constructor.

If max_denominator is not None then the denominator of any generated values is no greater than max_denominator. Note that max_denominator must be None or a positive integer.

Examples from this strategy shrink towards smaller denominators, then closer to zero.

hypothesis.strategies.from_regex(regex, *, fullmatch=False, alphabet=None)

Generates strings that contain a match for the given regex (i.e. ones for which python:re.search() will return a non-None result).

regex may be a pattern or compiled regex. Both byte-strings and unicode strings are supported, and will generate examples of the same type.

You can use regex flags such as python:re.IGNORECASE or python:re.DOTALL to control generation. Flags can be passed either in compiled regex or inside the pattern with a (?iLmsux) group.

Some regular expressions are only partly supported - the underlying strategy checks local matching and relies on filtering to resolve context-dependent expressions.  Using too many of these constructs may cause health-check errors as too many examples are filtered out. This mainly includes (positive or negative) lookahead and lookbehind groups.

If you want the generated string to match the whole regex you should use boundary markers. So e.g. r"\A.\Z" will return a single character string, while "." will return any string, and r"\A.$" will return a single character optionally followed by a "\n". Alternatively, passing fullmatch=True will ensure that the whole string is a match, as if you had used the \A and \Z markers.

The alphabet= argument constrains the characters in the generated string, as for text(), and is only supported for unicode strings.

Examples from this strategy shrink towards shorter strings and lower character values, with exact behaviour that may depend on the pattern.

hypothesis.strategies.from_type(thing)

Looks up the appropriate search strategy for the given type.

from_type is used internally to fill in missing arguments to builds() and can be used interactively to explore what strategies are available or to debug type resolution.

You can use register_type_strategy() to handle your custom types, or to globally redefine certain strategies - for example excluding NaN from floats, or use timezone-aware instead of naive time and datetime strategies.

The resolution logic may be changed in a future version, but currently tries these five options:

1. If thing is in the default lookup mapping or user-registered lookup, return the corresponding strategy.  The default lookup covers all types with Hypothesis strategies, including extras where possible.
2. If thing is from the python:typing module, return the corresponding strategy (special logic).
3. If thing has one or more subtypes in the merged lookup, return the union of the strategies for those types that are not subtypes of other elements in the lookup.
4. Finally, if thing has type annotations for all required arguments, and is not an abstract class, it is resolved via builds().
5. Because abstract types cannot be instantiated, we treat abstract types as the union of their concrete subclasses. Note that this lookup works via inheritance but not via register, so you may still need to use register_type_strategy().

There is a valuable recipe for leveraging from_type() to generate "everything except" values from a specified type. I.e.

def everything_except(excluded_types):
    return (
        from_type(type)
        .flatmap(from_type)
        .filter(lambda x: not isinstance(x, excluded_types))
    )

For example, everything_except(int) returns a strategy that can generate anything that from_type() can ever generate, except for instances of python:int, and excluding instances of types added via register_type_strategy().

This is useful when writing tests which check that invalid input is rejected in a certain way.

hypothesis.strategies.frozensets(elements, *, min_size=0, max_size=None)

This is identical to the sets function but instead returns frozensets.

hypothesis.strategies.functions(*, like=lambda : ..., returns=..., pure=False)

A strategy for functions, which can be used in callbacks.

The generated functions will mimic the interface of like, which must be a callable (including a class, method, or function).  The return value for the function is drawn from the returns argument, which must be a strategy.  If returns is not passed, we attempt to infer a strategy from the return-type annotation if present, falling back to none().

If pure=True, all arguments passed to the generated function must be hashable, and if passed identical arguments the original return value will be returned again - not regenerated, so beware mutable values.

If pure=False, generated functions do not validate their arguments, and may return a different value if called again with the same arguments.

Generated functions can only be called within the scope of the @given which created them.  This strategy does not support .example().

hypothesis.strategies.integers(min_value=None, max_value=None)

Returns a strategy which generates integers.

If min_value is not None then all values will be >= min_value. If max_value is not None then all values will be <= max_value

Examples from this strategy will shrink towards zero, and negative values will also shrink towards positive (i.e. -n may be replaced by +n).

hypothesis.strategies.ip_addresses(*, v=None, network=None)

Generate IP addresses - v=4 for IPv4Addresses, v=6 for IPv6Addresses, or leave unspecified to allow both versions.

network may be an IPv4Network or IPv6Network, or a string representing a network such as "127.0.0.0/24" or "2001:db8::/32".  As well as generating addresses within a particular routable network, this can be used to generate addresses from a reserved range listed in the IANA registries.

If you pass both v and network, they must be for the same version.

hypothesis.strategies.iterables(elements, *, min_size=0, max_size=None, unique_by=None, unique=False)

This has the same behaviour as lists, but returns iterables instead.

Some iterables cannot be indexed (e.g. sets) and some do not have a fixed length (e.g. generators). This strategy produces iterators, which cannot be indexed and do not have a fixed length. This ensures that you do not accidentally depend on sequence behaviour.

hypothesis.strategies.just(value)

Return a strategy which only generates value.

Note: value is not copied. Be wary of using mutable values.

If value is the result of a callable, you can use builds(callable) instead of just(callable()) to get a fresh value each time.

Examples from this strategy do not shrink (because there is only one).

hypothesis.strategies.lists(elements, *, min_size=0, max_size=None, unique_by=None, unique=False)

Returns a list containing values drawn from elements with length in the interval [min_size, max_size] (no bounds in that direction if these are None). If max_size is 0, only the empty list will be drawn.

If unique is True (or something that evaluates to True), we compare direct object equality, as if unique_by was lambda x: x. This comparison only works for hashable types.

If unique_by is not None it must be a callable or tuple of callables returning a hashable type when given a value drawn from elements. The resulting list will satisfy the condition that for i != j, unique_by(result[i]) != unique_by(result[j]).

If unique_by is a tuple of callables the uniqueness will be respective to each callable.

For example, the following will produce two columns of integers with both columns being unique respectively.

>>> twoints = st.tuples(st.integers(), st.integers())
>>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))

Examples from this strategy shrink by trying to remove elements from the list, and by shrinking each individual element of the list.

hypothesis.strategies.none()

Return a strategy which only generates None.

Examples from this strategy do not shrink (because there is only one).

hypothesis.strategies.nothing()

This strategy never successfully draws a value and will always reject on an attempt to draw.

Examples from this strategy do not shrink (because there are none).

hypothesis.strategies.one_of(*args)

Return a strategy which generates values from any of the argument strategies.

This may be called with one iterable argument instead of multiple strategy arguments, in which case one_of(x) and one_of(*x) are equivalent.

Examples from this strategy will generally shrink to ones that come from strategies earlier in the list, then shrink according to behaviour of the strategy that produced them. In order to get good shrinking behaviour, try to put simpler strategies first. e.g. one_of(none(), text()) is better than one_of(text(), none()).

This is especially important when using recursive strategies. e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will shrink well, but x = st.deferred(lambda: st.tuples(x, x) | st.none()) will shrink very badly indeed.

hypothesis.strategies.permutations(values)

Return a strategy which returns permutations of the ordered collection values.

Examples from this strategy shrink by trying to become closer to the original order of values.

hypothesis.strategies.random_module()

Hypothesis always seeds global PRNGs before running a test, and restores the previous state afterwards.

If having a fixed seed would unacceptably weaken your tests, and you cannot use a random.Random instance provided by randoms(), this strategy calls python:random.seed() with an arbitrary integer and passes you an opaque object whose repr displays the seed value for debugging. If numpy.random is available, that state is also managed, as is anything managed by hypothesis.register_random().

Examples from these strategy shrink to seeds closer to zero.

hypothesis.strategies.randoms(*, note_method_calls=False, use_true_random=False)

Generates instances of random.Random. The generated Random instances are of a special HypothesisRandom subclass.

• If note_method_calls is set to True, Hypothesis will print the randomly drawn values in any falsifying test case. This can be helpful for debugging the behaviour of randomized algorithms.
• If use_true_random is set to True then values will be drawn from their usual distribution, otherwise they will actually be Hypothesis generated values (and will be shrunk accordingly for any failing test case). Setting use_true_random=False will tend to expose bugs that would occur with very low probability when it is set to True, and this flag should only be set to True when your code relies on the distribution of values for correctness.

For managing global state, see the random_module() strategy and register_random() function.

hypothesis.strategies.recursive(base, extend, *, max_leaves=100)

base: A strategy to start from.

extend: A function which takes a strategy and returns a new strategy.

max_leaves: The maximum number of elements to be drawn from base on a given run.

This returns a strategy S such that S = extend(base | S). That is, values may be drawn from base, or from any strategy reachable by mixing applications of | and extend.

An example may clarify: recursive(booleans(), lists) would return a strategy that may return arbitrarily nested and mixed lists of booleans. So e.g. False, [True], [False, []], and [[[[True]]]] are all valid values to be drawn from that strategy.

Examples from this strategy shrink by trying to reduce the amount of recursion and by shrinking according to the shrinking behaviour of base and the result of extend.

hypothesis.strategies.register_type_strategy(custom_type, strategy)

Add an entry to the global type-to-strategy lookup.

This lookup is used in builds() and @given.

builds() will be used automatically for classes with type annotations on __init__ , so you only need to register a strategy if one or more arguments need to be more tightly defined than their type-based default, or if you want to supply a strategy for an argument with a default value.

strategy may be a search strategy, or a function that takes a type and returns a strategy (useful for generic types). The function may return NotImplemented to conditionally not provide a strategy for the type (the type will still be resolved by other methods, if possible, as if the function was not registered).

Note that you may not register a parametrised generic type (such as MyCollection[int]) directly, because the resolution logic does not handle this case correctly.  Instead, you may register a function for MyCollection and inspect the type parameters within that function.

hypothesis.strategies.runner(*, default=not_set)

A strategy for getting "the current test runner", whatever that may be. The exact meaning depends on the entry point, but it will usually be the associated 'self' value for it.

If you are using this in a rule for stateful testing, this strategy will return the instance of the RuleBasedStateMachine that the rule is running for.

If there is no current test runner and a default is provided, return that default. If no default is provided, raises InvalidArgument.

Examples from this strategy do not shrink (because there is only one).

hypothesis.strategies.sampled_from(elements)

Returns a strategy which generates any value present in elements.

Note that as with just(), values will not be copied and thus you should be careful of using mutable data.

sampled_from supports ordered collections, as well as Enum objects.  Flag objects may also generate any combination of their members.

Examples from this strategy shrink by replacing them with values earlier in the list. So e.g. sampled_from([10, 1]) will shrink by trying to replace 1 values with 10, and sampled_from([1, 10]) will shrink by trying to replace 10 values with 1.

It is an error to sample from an empty sequence, because returning nothing() makes it too easy to silently drop parts of compound strategies.  If you need that behaviour, use sampled_from(seq) if seq else nothing().

hypothesis.strategies.sets(elements, *, min_size=0, max_size=None)

This has the same behaviour as lists, but returns sets instead.

Note that Hypothesis cannot tell if values are drawn from elements are hashable until running the test, so you can define a strategy for sets of an unhashable type but it will fail at test time.

Examples from this strategy shrink by trying to remove elements from the set, and by shrinking each individual element of the set.

hypothesis.strategies.shared(base, *, key=None)

Returns a strategy that draws a single shared value per run, drawn from base. Any two shared instances with the same key will share the same value, otherwise the identity of this strategy will be used. That is:

>>> s = integers()  # or any other strategy
>>> x = shared(s)
>>> y = shared(s)

In the above x and y may draw different (or potentially the same) values. In the following they will always draw the same:

>>> x = shared(s, key="hi")
>>> y = shared(s, key="hi")

Examples from this strategy shrink as per their base strategy.

hypothesis.strategies.slices(size)

Generates slices that will select indices up to the supplied size

Generated slices will have start and stop indices that range from -size to size - 1 and will step in the appropriate direction. Slices should only produce an empty selection if the start and end are the same.

Examples from this strategy shrink toward 0 and smaller values

hypothesis.strategies.text(alphabet=characters(codec='utf-8'), *, min_size=0, max_size=None)

Generates strings with characters drawn from alphabet, which should be a collection of length one strings or a strategy generating such strings.

The default alphabet strategy can generate the full unicode range but excludes surrogate characters because they are invalid in the UTF-8 encoding.  You can use characters() without arguments to find surrogate-related bugs such as bpo-34454.

min_size and max_size have the usual interpretations. Note that Python measures string length by counting codepoints: U+00C5 Å is a single character, while U+0041 U+030A Å is two - the A, and a combining ring above.

Examples from this strategy shrink towards shorter strings, and with the characters in the text shrinking as per the alphabet strategy. This strategy does not normalize() examples, so generated strings may be in any or none of the 'normal forms'.

hypothesis.strategies.timedeltas(min_value=datetime.timedelta.min, max_value=datetime.timedelta.max)

A strategy for timedeltas between min_value and max_value.

Examples from this strategy shrink towards zero.

hypothesis.strategies.times(min_value=datetime.time.min, max_value=datetime.time.max, *, timezones=none())

A strategy for times between min_value and max_value.

The timezones argument is handled as for datetimes().

Examples from this strategy shrink towards midnight, with the timezone component shrinking as for the strategy that provided it.

hypothesis.strategies.timezone_keys(*, allow_prefix=True)

A strategy for IANA timezone names.

As well as timezone names like "UTC", "Australia/Sydney", or "America/New_York", this strategy can generate:

• Aliases such as "Antarctica/McMurdo", which links to "Pacific/Auckland".
• Deprecated names such as "Antarctica/South_Pole", which also links to "Pacific/Auckland".  Note that most but not all deprecated timezone names are also aliases.
• Timezone names with the "posix/" or "right/" prefixes, unless allow_prefix=False.

These strings are provided separately from Tzinfo objects - such as ZoneInfo instances from the timezones() strategy - to facilitate testing of timezone logic without needing workarounds to access non-canonical names.

NOTE:

The python:zoneinfo module is new in Python 3.9, so you will need to install the backports.zoneinfo module on earlier versions.

On Windows, you will also need to install the tzdata package.

pip install hypothesis[zoneinfo] will install these conditional dependencies if and only if they are needed.

On Windows, you may need to access IANA timezone data via the tzdata package.  For non-IANA timezones, such as Windows-native names or GNU TZ strings, we recommend using sampled_from() with the dateutil package, e.g. dateutil:dateutil.tz.tzwin.list().

hypothesis.strategies.timezones(*, no_cache=False)

A strategy for python:zoneinfo.ZoneInfo objects.

If no_cache=True, the generated instances are constructed using ZoneInfo.no_cache instead of the usual constructor.  This may change the semantics of your datetimes in surprising ways, so only use it if you know that you need to!

NOTE:

The python:zoneinfo module is new in Python 3.9, so you will need to install the backports.zoneinfo module on earlier versions.

On Windows, you will also need to install the tzdata package.

pip install hypothesis[zoneinfo] will install these conditional dependencies if and only if they are needed.

hypothesis.strategies.tuples(*args)

Return a strategy which generates a tuple of the same length as args by generating the value at index i from args[i].

e.g. tuples(integers(), integers()) would generate a tuple of length two with both values an integer.

Examples from this strategy shrink by shrinking their component parts.

hypothesis.strategies.uuids(*, version=None, allow_nil=False)

Returns a strategy that generates UUIDs.

If the optional version argument is given, value is passed through to UUID and only UUIDs of that version will be generated.

If allow_nil is True, generate the nil UUID much more often. Otherwise, all returned values from this will be unique, so e.g. if you do lists(uuids()) the resulting list will never contain duplicates.

Examples from this strategy don't have any meaningful shrink order.

This module contains various provisional APIs and strategies.

It is intended for internal use, to ease code reuse, and is not stable. Point releases may move or break the contents at any time!

Internet strategies should conform to RFC 3986 or the authoritative definitions it links to.  If not, report the bug!

hypothesis.provisional.domains(*, max_length=255, max_element_length=63)

Generate RFC 1035 compliant fully qualified domain names.

hypothesis.provisional.urls()

A strategy for RFC 3986, generating http/https URLs.

When using strategies it is worth thinking about how the data shrinks. Shrinking is the process by which Hypothesis tries to produce human readable examples when it finds a failure - it takes a complex example and turns it into a simpler one.

Each strategy defines an order in which it shrinks - you won't usually need to care about this much, but it can be worth being aware of as it can affect what the best way to write your own strategies is.

The exact shrinking behaviour is not a guaranteed part of the API, but it doesn't change that often and when it does it's usually because we think the new way produces nicer examples.

Possibly the most important one to be aware of is one_of(), which has a preference for values produced by strategies earlier in its argument list. Most of the others should largely "do the right thing" without you having to think about it.

Often it is the case that a strategy doesn't produce exactly what you want it to and you need to adapt it. Sometimes you can do this in the test, but this hurts reuse because you then have to repeat the adaption in every test.

Hypothesis gives you ways to build strategies from other strategies given functions for transforming the data.

map is probably the easiest and most useful of these to use. If you have a strategy s and a function f, then an example s.map(f).example() is f(s.example()), i.e. we draw an example from s and then apply f to it.

e.g.:

>>> lists(integers()).map(sorted).example()
[-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]

Note that many things that you might use mapping for can also be done with builds(), and if you find yourself indexing into a tuple within .map() it's probably time to use that instead.

filter lets you reject some examples. s.filter(f).example() is some example of s such that f(example) is truthy.

>>> integers().filter(lambda x: x > 11).example()
26126
>>> integers().filter(lambda x: x > 11).example()
23324

It's important to note that filter isn't magic and if your condition is too hard to satisfy then this can fail:

>>> integers().filter(lambda x: False).example()
Traceback (most recent call last):
    ...
hypothesis.errors.Unsatisfiable: Could not find any valid examples in 20 tries

In general you should try to use filter only to avoid corner cases that you don't want rather than attempting to cut out a large chunk of the search space.

A technique that often works well here is to use map to first transform the data and then use filter to remove things that didn't work out. So for example if you wanted pairs of integers (x,y) such that x < y you could do the following:

>>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
[-8543729478746591815, 3760495307320535691]

Finally there is flatmap. flatmap draws an example, then turns that example into a strategy, then draws an example from that strategy.

It may not be obvious why you want this at first, but it turns out to be quite useful because it lets you generate different types of data with relationships to each other.

For example suppose we wanted to generate a list of lists of the same length:

>>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
...     lambda n: lists(lists(integers(), min_size=n, max_size=n))
... )
>>> rectangle_lists.example()
[]
>>> rectangle_lists.filter(lambda x: len(x) >= 10).example()
[[], [], [], [], [], [], [], [], [], []]
>>> rectangle_lists.filter(lambda t: len(t) >= 3 and len(t[0]) >= 3).example()
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
>>> rectangle_lists.filter(lambda t: sum(len(s) for s in t) >= 10).example()
[[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]

In this example we first choose a length for our tuples, then we build a strategy which generates lists containing lists precisely of that length. The finds show what simple examples for this look like.

Most of the time you probably don't want flatmap, but unlike filter and map which are just conveniences for things you could just do in your tests, flatmap allows genuinely new data generation that you wouldn't otherwise be able to easily do.

(If you know Haskell: Yes, this is more or less a monadic bind. If you don't know Haskell, ignore everything in these parentheses. You do not need to understand anything about monads to use this, or anything else in Hypothesis).

Sometimes the data you want to generate has a recursive definition. e.g. if you wanted to generate JSON data, valid JSON is:

1. Any float, any boolean, any unicode string.
2. Any list of valid JSON data
3. Any dictionary mapping unicode strings to valid JSON data.

The problem is that you cannot call a strategy recursively and expect it to not just blow up and eat all your memory.  The other problem here is that not all unicode strings display consistently on different machines, so we'll restrict them in our doctest.

The way Hypothesis handles this is with the recursive() strategy which you pass in a base case and a function that, given a strategy for your data type, returns a new strategy for it. So for example:

>>> from string import printable
... from pprint import pprint
>>> json = recursive(
...     none() | booleans() | floats() | text(printable),
...     lambda children: lists(children) | dictionaries(text(printable), children),
... )
>>> pprint(json.example())
[[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
>>> pprint(json.example())
{'de(l': None,
 'nK': {'(Rt)': None,
        '+hoZh1YU]gy8': True,
        '8z]EIFA06^li^': 'LFE{Q',
        '9,': 'l{cA=/'}}

That is, we start with our leaf data and then we augment it by allowing lists and dictionaries of anything we can generate as JSON data.

The size control of this works by limiting the maximum number of values that can be drawn from the base strategy. So for example if we wanted to only generate really small JSON we could do this as:

>>> small_lists = recursive(booleans(), lists, max_leaves=5)
>>> small_lists.example()
True
>>> small_lists.example()
[False]

The @composite decorator lets you combine other strategies in more or less arbitrary ways. It's probably the main thing you'll want to use for complicated custom strategies.

The composite decorator works by converting a function that returns one example into a function that returns a strategy that produces such examples - which you can pass to @given, modify with .map or .filter, and generally use like any other strategy.

It does this by giving you a special function draw as the first argument, which can be used just like the corresponding method of the data() strategy within a test.  In fact, the implementation is almost the same - but defining a strategy with @composite makes code reuse easier, and usually improves the display of failing examples.

For example, the following gives you a list and an index into it:

>>> @composite
... def list_and_index(draw, elements=integers()):
...     xs = draw(lists(elements, min_size=1))
...     i = draw(integers(min_value=0, max_value=len(xs) - 1))
...     return (xs, i)
...

draw(s) is a function that should be thought of as returning s.example(), except that the result is reproducible and will minimize correctly. The decorated function has the initial argument removed from the list, but will accept all the others in the expected order. Defaults are preserved.

>>> list_and_index()
list_and_index()
>>> list_and_index().example()
([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)

>>> list_and_index(booleans())
list_and_index(elements=booleans())
>>> list_and_index(booleans()).example()
([True, False], 0)

Note that the repr will work exactly like it does for all the built-in strategies: it will be a function that you can call to get the strategy in question, with values provided only if they do not match the defaults.

You can use assume inside composite functions:

@composite
def distinct_strings_with_common_characters(draw):
    x = draw(text(min_size=1))
    y = draw(text(alphabet=x))
    assume(x != y)
    return (x, y)

This works as assume normally would, filtering out any examples for which the passed in argument is falsey.

Take care that your function can cope with adversarial draws, or explicitly rejects them using the .filter() method or assume() - our mutation and shrinking logic can do some strange things, and a naive implementation might lead to serious performance problems.  For example:

@composite
def reimplementing_sets_strategy(draw, elements=st.integers(), size=5):
    # The bad way: if Hypothesis keeps generating e.g. zero,
    # we'll keep looping for a very long time.
    result = set()
    while len(result) < size:
        result.add(draw(elements))
    # The good way: use a filter, so Hypothesis can tell what's valid!
    for _ in range(size):
        result.add(draw(elements.filter(lambda x: x not in result)))
    return result

If @composite is used to decorate a method or classmethod, the draw argument must come before self or cls. While we therefore recommend writing strategies as standalone functions and using the register_type_strategy() function to associate them with a class, methods are supported and the @composite decorator may be applied either before or after @classmethod or @staticmethod. See issue #2578 and pull request #2634 for more details.

There is also the data() strategy, which gives you a means of using strategies interactively. Rather than having to specify everything up front in @given you can draw from strategies in the body of your test.

This is similar to @composite, but even more powerful as it allows you to mix test code with example generation. The downside of this power is that data() is incompatible with explicit @example(...)s - and the mixed code is often harder to debug when something goes wrong.

If you need values that are affected by previous draws but which don't depend on the execution of your test, stick to the simpler @composite.

@given(data())
def test_draw_sequentially(data):
    x = data.draw(integers())
    y = data.draw(integers(min_value=x))
    assert x < y

If the test fails, each draw will be printed with the falsifying example. e.g. the above is wrong (it has a boundary condition error), so will print:

Falsifying example: test_draw_sequentially(data=data(...))
Draw 1: 0
Draw 2: 0

As you can see, data drawn this way is simplified as usual.

Optionally, you can provide a label to identify values generated by each call to data.draw().  These labels can be used to identify values in the output of a falsifying example.

For instance:

@given(data())
def test_draw_sequentially(data):
    x = data.draw(integers(), label="First number")
    y = data.draw(integers(min_value=x), label="Second number")
    assert x < y

will produce the output:

Falsifying example: test_draw_sequentially(data=data(...))
Draw 1 (First number): 0
Draw 2 (Second number): 0

$ hypothesis --help
Usage: hypothesis [OPTIONS] COMMAND [ARGS]...

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  codemod  `hypothesis codemod` refactors deprecated or inefficient code.
  fuzz     [hypofuzz] runs tests with an adaptive coverage-guided fuzzer.
  write    `hypothesis write` writes property-based tests for you!

This module requires the click package, and provides Hypothesis' command-line interface, for e.g. 'ghostwriting' tests via the terminal. It's also where HypoFuzz adds the hypothesis fuzz command (learn more about that here).

This module provides codemods based on the LibCST library, which can both detect and automatically fix issues with code that uses Hypothesis, including upgrading from deprecated features to our recommended style.

You can run the codemods via our CLI:

$ hypothesis codemod --help
Usage: hypothesis codemod [OPTIONS] PATH...

  `hypothesis codemod` refactors deprecated or inefficient code.

  It adapts `python -m libcst.tool`, removing many features and config
  options which are rarely relevant for this purpose.  If you need more
  control, we encourage you to use the libcst CLI directly; if not this one
  is easier.

  PATH is the file(s) or directories of files to format in place, or "-" to
  read from stdin and write to stdout.

Options:
  -h, --help  Show this message and exit.

Alternatively you can use python -m libcst.tool, which offers more control at the cost of additional configuration (adding 'hypothesis.extra' to the modules list in .libcst.codemod.yaml) and some issues on Windows.

hypothesis.extra.codemods.refactor(code)

Update a source code string from deprecated to modern Hypothesis APIs.

This may not fix all the deprecation warnings in your code, but we're confident that it will be easier than doing it all by hand.

We recommend using the CLI, but if you want a Python function here it is.

TIP:

For new projects, we recommend using either deal or icontract and icontract-hypothesis over dpcontracts. They're generally more powerful tools for design-by-contract programming, and have substantially nicer Hypothesis integration too!

This extra can be used to generate strings matching any context-free grammar, using the Lark parser library.

It currently only supports Lark's native EBNF syntax, but we plan to extend this to support other common syntaxes such as ANTLR and RFC 5234 ABNF. Lark already supports loading grammars from nearley.js, so you may not have to write your own at all.

hypothesis.extra.lark.from_lark(grammar, *, start=None, explicit=None, alphabet=characters(codec='utf-8'))

A strategy for strings accepted by the given context-free grammar.

grammar must be a Lark object, which wraps an EBNF specification. The Lark EBNF grammar reference can be found here.

from_lark will automatically generate strings matching the nonterminal start symbol in the grammar, which was supplied as an argument to the Lark class.  To generate strings matching a different symbol, including terminals, you can override this by passing the start argument to from_lark.  Note that Lark may remove unreachable productions when the grammar is compiled, so you should probably pass the same value for start to both.

Currently from_lark does not support grammars that need custom lexing. Any lexers will be ignored, and any undefined terminals from the use of %declare will result in generation errors.  To define strategies for such terminals, pass a dictionary mapping their name to a corresponding strategy as the explicit argument.

The hypothesmith project includes a strategy for Python source, based on a grammar and careful post-processing.

Example grammars, which may provide a useful starting point for your tests, can be found in the Lark repository and in this third-party collection.

This module provides pytz timezones.

You can use this strategy to make hypothesis.strategies.datetimes() and hypothesis.strategies.times() produce timezone-aware values.

hypothesis.extra.pytz.timezones()

Any timezone in the Olsen database, as a pytz tzinfo object.

This strategy minimises to UTC, or the smallest possible fixed offset, and is designed for use with hypothesis.strategies.datetimes().

This module provides dateutil timezones.

You can use this strategy to make datetimes() and times() produce timezone-aware values.

hypothesis.extra.dateutil.timezones()

Any timezone from dateutil.

This strategy minimises to UTC, or the timezone with the smallest offset from UTC as of 2000-01-01, and is designed for use with datetimes().

Note that the timezones generated by the strategy may vary depending on the configuration of your machine. See the dateutil documentation for more information.

Ghostwritten tests are intended as a starting point for human authorship, to demonstrate best practice, help novices past blank-page paralysis, and save time for experts.  They may be ready-to-run, or include placeholders and # TODO: comments to fill in strategies for unknown types.  In either case, improving tests for their own code gives users a well-scoped and immediately rewarding context in which to explore property-based testing.

By contrast, most test-generation tools aim to produce ready-to-run test suites... and implicitly assume that the current behavior is the desired behavior. However, the code might contain bugs, and we want our tests to fail if it does! Worse, tools require that the code to be tested is finished and executable, making it impossible to generate tests as part of the development process.

Fraser 2013 found that evolving a high-coverage test suite (e.g. Randoop, EvoSuite, Pynguin) "leads to clear improvements in commonly applied quality metrics such as code coverage [but] no measurable improvement in the number of bugs actually found by developers" and that "generating a set of test cases, even high coverage test cases, does not necessarily improve our ability to test software". Invariant detection (famously Daikon; in PBT see e.g. Alonso 2022, QuickSpec, Speculate) relies on code execution. Program slicing (e.g. FUDGE, FuzzGen, WINNIE) requires downstream consumers of the code to test.

Ghostwriter inspects the function name, argument names and types, and docstrings. It can be used on buggy or incomplete code, runs in a few seconds, and produces a single semantically-meaningful test per function or group of functions. Rather than detecting regressions, these tests check semantic properties such as encode/decode or save/load round-trips, for commutative, associative, and distributive operations, equivalence between methods, array shapes, and idempotence.  Where no property is detected, we simply check for 'no error on valid input' and allow the user to supply their own invariants.

Evaluations such as the SBFT24 competition measure performance on a task which the Ghostwriter is not intended to perform.  I'd love to see qualitative user studies, such as PBT in Practice for test generation, which could check whether the Ghostwriter is onto something or tilting at windmills. If you're interested in similar questions, drop me an email!

If you have a custom Django field type you can register it with Hypothesis's model deriving functionality by registering a default strategy for it:

>>> from toystore.models import CustomishField, Customish
>>> from_model(Customish).example()
hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
    customish for model Customish
>>> from hypothesis.extra.django import register_field_strategy
>>> from hypothesis.strategies import just
>>> register_field_strategy(CustomishField, just("hi"))
>>> x = from_model(Customish).example()
>>> x.customish
'hi'

Note that this mapping is on exact type. Subtypes will not inherit it.

hypothesis.extra.django.register_field_strategy(field_type, strategy)

Add an entry to the global field-to-strategy lookup used by from_field().

field_type must be a subtype of django.db.models.Field or django.forms.Field, which must not already be registered. strategy must be a SearchStrategy.

hypothesis.extra.django.from_field(field)

Return a strategy for values that fit the given field.

This function is used by from_form() and from_model() for any fields that require a value, or for which you passed ... (python:Ellipsis) to infer a strategy from an annotation.

It's pretty similar to the core from_type() function, with a subtle but important difference: from_field takes a Field instance, rather than a Field subtype, so that it has access to instance attributes such as string length and validators.

For the moment there's no explicit support in hypothesis-django for generating dependent models. i.e. a Company model will generate no Shops. However if you want to generate some dependent models as well, you can emulate this by using the flatmap function as follows:

from hypothesis.strategies import just, lists


def generate_with_shops(company):
    return lists(from_model(Shop, company=just(company))).map(lambda _: company)


company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)

Let's unpack what this is doing:

The way flatmap works is that we draw a value from the original strategy, then apply a function to it which gives us a new strategy. We then draw a value from that strategy. So in this case we're first drawing a company, and then we're drawing a list of shops belonging to that company: The just strategy is a strategy such that drawing it always produces the individual value, so from_model(Shop, company=just(company)) is a strategy that generates a Shop belonging to the original company.

So the following code would give us a list of shops all belonging to the same company:

from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))

The only difference from this and the above is that we want the company, not the shops. This is where the inner map comes in. We build the list of shops and then throw it away, instead returning the company we started for. This works because the models that Hypothesis generates are saved in the database, so we're essentially running the inner strategy purely for the side effect of creating those children in the database.

If your model includes a custom primary key that you want to generate using a strategy (rather than a default auto-increment primary key) then Hypothesis has to deal with the possibility of a duplicate primary key.

If a model strategy generates a value for the primary key field, Hypothesis will create the model instance with update_or_create(), overwriting any existing instance in the database for this test case with the same primary key.

Django forms feature the MultiValueField which allows for several fields to be combined under a single named field, the default example of this is the SplitDateTimeField.

class CustomerForm(forms.Form):
    name = forms.CharField()
    birth_date_time = forms.SplitDateTimeField()

from_form supports MultiValueField subclasses directly, however if you want to define your own strategy be forewarned that Django binds data for a MultiValueField in a peculiar way. Specifically each sub-field is expected to have its own entry in data addressed by the field name (e.g. birth_date_time) and the index of the sub-field within the MultiValueField, so form data for the example above might look like this:

{
    "name": "Samuel John",
    "birth_date_time_0": "2018-05-19",  # the date, as the first sub-field
    "birth_date_time_1": "15:18:00",  # the time, as the second sub-field
}

Thus, if you want to define your own strategies for such a field you must address your sub-fields appropriately:

from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))

Hypothesis offers a number of strategies for NumPy testing, available in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy package.

The centerpiece is the arrays() strategy, which generates arrays with any dtype, shape, and contents you can specify or give a strategy for. To make this as useful as possible, strategies are provided to generate array shapes and generate all kinds of fixed-size or compound dtypes.

hypothesis.extra.numpy.from_dtype(dtype, *, alphabet=None, min_size=0, max_size=None, min_value=None, max_value=None, allow_nan=None, allow_infinity=None, allow_subnormal=None, exclude_min=None, exclude_max=None, min_magnitude=0, max_magnitude=None)

Creates a strategy which can generate any value of the given dtype.

Compatible parameters are passed to the inferred strategy function while inapplicable ones are ignored. This allows you, for example, to customise the min and max values, control the length or contents of strings, or exclude non-finite numbers. This is particularly useful when kwargs are passed through from arrays() which allow a variety of numeric dtypes, as it seamlessly handles the width or representable bounds for you.

hypothesis.extra.numpy.arrays(dtype, shape, *, elements=None, fill=None, unique=False)

Returns a strategy for generating numpy:numpy.ndarrays.

• dtype may be any valid input to dtype (this includes dtype objects), or a strategy that generates such values.
• shape may be an integer >= 0, a tuple of such integers, or a strategy that generates such values.
• elements is a strategy for generating values to put in the array. If it is None a suitable value will be inferred based on the dtype, which may give any legal value (including eg NaN for floats). If a mapping, it will be passed as **kwargs to from_dtype()
• fill is a strategy that may be used to generate a single background value for the array. If None, a suitable default will be inferred based on the other arguments. If set to nothing() then filling behaviour will be disabled entirely and every element will be generated independently.
• unique specifies if the elements of the array should all be distinct from one another. Note that in this case multiple NaN values may still be allowed. If fill is also set, the only valid values for it to return are NaN values (anything for which numpy:numpy.isnan returns True. So e.g. for complex numbers nan+1j is also a valid fill). Note that if unique is set to True the generated values must be hashable.

Arrays of specified dtype and shape are generated for example like this:

>>> import numpy as np
>>> arrays(np.int8, (2, 3)).example()
array([[-8,  6,  3],
       [-6,  4,  6]], dtype=int8)
>>> arrays(np.float, 3, elements=st.floats(0, 1)).example()
array([ 0.88974794,  0.77387938,  0.1977879 ])

Array values are generated in two parts:

1. Some subset of the coordinates of the array are populated with a value drawn from the elements strategy (or its inferred form).
2. If any coordinates were not assigned in the previous step, a single value is drawn from the fill strategy and is assigned to all remaining places.

You can set fill=nothing() to disable this behaviour and draw a value for every element.

If fill=None, then it will attempt to infer the correct behaviour automatically. If unique is True, no filling will occur by default. Otherwise, if it looks safe to reuse the values of elements across multiple coordinates (this will be the case for any inferred strategy, and for most of the builtins, but is not the case for mutable values or strategies built with flatmap, map, composite, etc) then it will use the elements strategy as the fill, else it will default to having no fill.

Having a fill helps Hypothesis craft high quality examples, but its main importance is when the array generated is large: Hypothesis is primarily designed around testing small examples. If you have arrays with hundreds or more elements, having a fill value is essential if you want your tests to run in reasonable time.

hypothesis.extra.numpy.array_shapes(*, min_dims=1, max_dims=None, min_side=1, max_side=None)

Return a strategy for array shapes (tuples of int >= 1).

• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to min_dims + 2.
• min_side is the smallest size that a dimension can possess.
• max_side is the largest size that a dimension can possess, defaulting to min_side + 5.

hypothesis.extra.numpy.scalar_dtypes()

Return a strategy that can return any non-flexible scalar dtype.

hypothesis.extra.numpy.unsigned_integer_dtypes(*, endianness='?', sizes=(8, 16, 32, 64))

Return a strategy for unsigned integer dtypes.

endianness may be < for little-endian, > for big-endian, = for native byte order, or ? to allow either byte order. This argument only applies to dtypes of more than one byte.

sizes must be a collection of integer sizes in bits.  The default (8, 16, 32, 64) covers the full range of sizes.

hypothesis.extra.numpy.integer_dtypes(*, endianness='?', sizes=(8, 16, 32, 64))

Return a strategy for signed integer dtypes.

endianness and sizes are treated as for unsigned_integer_dtypes().

hypothesis.extra.numpy.floating_dtypes(*, endianness='?', sizes=(16, 32, 64))

Return a strategy for floating-point dtypes.

sizes is the size in bits of floating-point number.  Some machines support 96- or 128-bit floats, but these are not generated by default.

Larger floats (96 and 128 bit real parts) are not supported on all platforms and therefore disabled by default.  To generate these dtypes, include these values in the sizes argument.

hypothesis.extra.numpy.complex_number_dtypes(*, endianness='?', sizes=(64, 128))

Return a strategy for complex-number dtypes.

sizes is the total size in bits of a complex number, which consists of two floats.  Complex halves (a 16-bit real part) are not supported by numpy and will not be generated by this strategy.

hypothesis.extra.numpy.datetime64_dtypes(*, max_period='Y', min_period='ns', endianness='?')

Return a strategy for datetime64 dtypes, with various precisions from year to attosecond.

hypothesis.extra.numpy.timedelta64_dtypes(*, max_period='Y', min_period='ns', endianness='?')

Return a strategy for timedelta64 dtypes, with various precisions from year to attosecond.

hypothesis.extra.numpy.byte_string_dtypes(*, endianness='?', min_len=1, max_len=16)

Return a strategy for generating bytestring dtypes, of various lengths and byteorder.

While Hypothesis' string strategies can generate empty strings, string dtypes with length 0 indicate that size is still to be determined, so the minimum length for string dtypes is 1.

hypothesis.extra.numpy.unicode_string_dtypes(*, endianness='?', min_len=1, max_len=16)

Return a strategy for generating unicode string dtypes, of various lengths and byteorder.

While Hypothesis' string strategies can generate empty strings, string dtypes with length 0 indicate that size is still to be determined, so the minimum length for string dtypes is 1.

hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(), *, min_size=1, max_size=5, allow_subarrays=False)

Return a strategy for generating array (compound) dtypes, with members drawn from the given subtype strategy.

hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(), *, max_leaves=10, max_itemsize=None)

Return the most-general dtype strategy.

Elements drawn from this strategy may be simple (from the subtype_strategy), or several such values drawn from array_dtypes() with allow_subarrays=True. Subdtypes in an array dtype may be nested to any depth, subject to the max_leaves argument.

hypothesis.extra.numpy.valid_tuple_axes(ndim, *, min_size=0, max_size=None)

Return a strategy for generating permissible tuple-values for the axis argument for a numpy sequential function (e.g. numpy:numpy.sum()), given an array of the specified dimensionality.

All tuples will have a length >= min_size and <= max_size. The default

value for max_size is ndim.

Examples from this strategy shrink towards an empty tuple, which render most sequential functions as no-ops.

The following are some examples drawn from this strategy.

>>> [valid_tuple_axes(3).example() for i in range(4)]
[(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]

valid_tuple_axes can be joined with other strategies to generate any type of valid axis object, i.e. integers, tuples, and None:

any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)

hypothesis.extra.numpy.broadcastable_shapes(shape, *, min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for shapes that are broadcast-compatible with the provided shape.

Examples from this strategy shrink towards a shape with length min_dims. The size of an aligned dimension shrinks towards size 1. The size of an unaligned dimension shrink towards min_side.

• shape is a tuple of integers.
• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to max(len(shape), min_dims) + 2.
• min_side is the smallest size that an unaligned dimension can possess.
• max_side is the largest size that an unaligned dimension can possess, defaulting to 2 plus the size of the largest aligned dimension.

The following are some examples drawn from this strategy.

>>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
[(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]

hypothesis.extra.numpy.mutually_broadcastable_shapes(*, num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for a specified number of shapes N that are

mutually-broadcastable with one another and with the provided base shape.

• num_shapes is the number of mutually broadcast-compatible shapes to generate.
• base_shape is the shape against which all generated shapes can broadcast. The default shape is empty, which corresponds to a scalar and thus does not constrain broadcasting at all.
• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to max(len(shape), min_dims) + 2.
• min_side is the smallest size that an unaligned dimension can possess.
• max_side is the largest size that an unaligned dimension can possess, defaulting to 2 plus the size of the largest aligned dimension.

The strategy will generate a python:typing.NamedTuple containing:

• input_shapes as a tuple of the N generated shapes.
• result_shape as the resulting shape produced by broadcasting the N shapes with the base shape.

The following are some examples drawn from this strategy.

>>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
>>> for _ in range(5):
...     print(strat.example())
BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
BroadcastableShapes(input_shapes=((), (), ()), result_shape=())
BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(3,))
BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))



**Use with Generalised Universal Function signatures**

A :doc:`universal function <numpy:reference/ufuncs>` (or ufunc for short) is a function
that operates on ndarrays in an element-by-element fashion, supporting array
broadcasting, type casting, and several other standard features.
A :doc:`generalised ufunc <numpy:reference/c-api/generalized-ufuncs>` operates on
sub-arrays rather than elements, based on the "signature" of the function.
Compare e.g. :obj:`numpy.add() <numpy:numpy.add>` (ufunc) to
:obj:`numpy.matmul() <numpy:numpy.matmul>` (gufunc).

To generate shapes for a gufunc, you can pass the ``signature`` argument instead of
``num_shapes``.  This must be a gufunc signature string; which you can write by
hand or access as e.g. ``np.matmul.signature`` on generalised ufuncs.

In this case, the ``side`` arguments are applied to the 'core dimensions' as well,
ignoring any frozen dimensions.  ``base_shape``  and the ``dims`` arguments are
applied to the 'loop dimensions', and if necessary, the dimensionality of each
shape is silently capped to respect the 32-dimension limit.

The generated ``result_shape`` is the real result shape of applying the gufunc
to arrays of the generated ``input_shapes``, even where this is different to
broadcasting the loop dimensions.

gufunc-compatible shapes shrink their loop dimensions as above, towards omitting
optional core dimensions, and smaller-size core dimensions.

.. code-block:: pycon

    >>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
    >>> for _ in range(3):
    ...     mutually_broadcastable_shapes(signature=np.matmul.signature).example()
    BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
    BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
    BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))

hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0, max_dims=None, allow_newaxis=False, allow_ellipsis=True)

Return a strategy for basic indexes of arrays with the specified shape, which may include dimensions of size zero.

It generates tuples containing some mix of integers, python:slice objects, ... (an Ellipsis), and None. When a length-one tuple would be generated, this strategy may instead return the element which will index the first axis, e.g. 5 instead of (5,).

• shape is the shape of the array that will be indexed, as a tuple of positive integers. This must be at least two-dimensional for a tuple to be a valid index; for one-dimensional arrays use slices() instead.
• min_dims is the minimum dimensionality of the resulting array from use of the generated index. When min_dims == 0, scalars and zero-dimensional arrays are both allowed.
• max_dims is the the maximum dimensionality of the resulting array, defaulting to len(shape) if not allow_newaxis else max(len(shape), min_dims) + 2.
• allow_newaxis specifies whether None is allowed in the index.
• allow_ellipsis specifies whether ... is allowed in the index.

hypothesis.extra.numpy.integer_array_indices(shape, *, result_shape=array_shapes(), dtype=dtype('int64'))

Return a search strategy for tuples of integer-arrays that, when used to index into an array of shape shape, given an array whose shape was drawn from result_shape.

Examples from this strategy shrink towards the tuple of index-arrays:

len(shape) * (np.zeros(drawn_result_shape, dtype), )

• shape a tuple of integers that indicates the shape of the array, whose indices are being generated.
• result_shape a strategy for generating tuples of integers, which describe the shape of the resulting index arrays. The default is array_shapes().  The shape drawn from this strategy determines the shape of the array that will be produced when the corresponding example from integer_array_indices is used as an index.
• dtype the integer data type of the generated index-arrays. Negative integer indices can be generated if a signed integer type is specified.

Recall that an array can be indexed using a tuple of integer-arrays to access its members in an arbitrary order, producing an array with an arbitrary shape. For example:

>>> from numpy import array
>>> x = array([-0, -1, -2, -3, -4])
>>> ind = (array([[4, 0], [0, 1]]),)  # a tuple containing a 2D integer-array
>>> x[ind]  # the resulting array is commensurate with the indexing array(s)
array([[-4,  0],
       [0, -1]])

Note that this strategy does not accommodate all variations of so-called 'advanced indexing', as prescribed by NumPy's nomenclature.  Combinations of basic and advanced indexes are too complex to usefully define in a standard strategy; we leave application-specific strategies to the user. Advanced-boolean indexing can be defined as arrays(shape=..., dtype=bool), and is similarly left to the user.

Hypothesis provides strategies for several of the core pandas data types: pandas.Index, pandas.Series and pandas.DataFrame.

The general approach taken by the pandas module is that there are multiple strategies for generating indexes, and all of the other strategies take the number of entries they contain from their index strategy (with sensible defaults). So e.g. a Series is specified by specifying its numpy.dtype (and/or a strategy for generating elements for it).

hypothesis.extra.pandas.indexes(*, elements=None, dtype=None, min_size=0, max_size=None, unique=True, name=none())

Provides a strategy for producing a pandas.Index.

Arguments:

• elements is a strategy which will be used to generate the individual values of the index. If None, it will be inferred from the dtype. Note: even if the elements strategy produces tuples, the generated value will not be a MultiIndex, but instead be a normal index whose elements are tuples.
• dtype is the dtype of the resulting index. If None, it will be inferred from the elements strategy. At least one of dtype or elements must be provided.
• min_size is the minimum number of elements in the index.
• max_size is the maximum number of elements in the index. If None then it will default to a suitable small size. If you want larger indexes you should pass a max_size explicitly.
• unique specifies whether all of the elements in the resulting index should be distinct.
• name is a strategy for strings or None, which will be passed to the pandas.Index constructor.

hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None, name=none())

Provides a strategy which generates an Index whose values are 0, 1, ..., n for some n.

Arguments:

• min_size is the smallest number of elements the index can have.
• max_size is the largest number of elements the index can have. If None it will default to some suitable value based on min_size.
• name is the name of the index. If st.none(), the index will have no name.

hypothesis.extra.pandas.series(*, elements=None, dtype=None, index=None, fill=None, unique=False, name=none())

Provides a strategy for producing a pandas.Series.

Arguments:

• elements: a strategy that will be used to generate the individual values in the series. If None, we will attempt to infer a suitable default from the dtype.
• dtype: the dtype of the resulting series and may be any value that can be passed to numpy.dtype. If None, will use pandas's standard behaviour to infer it from the type of the elements values. Note that if the type of values that comes out of your elements strategy varies, then so will the resulting dtype of the series.

• index: If not None, a strategy for generating indexes for the resulting Series. This can generate either pandas.Index objects or any sequence of values (which will be passed to the Index constructor).

  You will probably find it most convenient to use the indexes() or range_indexes() function to produce values for this argument.

• name: is a strategy for strings or None, which will be passed to the pandas.Series constructor.

Usage:

>>> series(dtype=int).example()
0   -2001747478
1    1153062837

class hypothesis.extra.pandas.column(name=None, elements=None, dtype=None, fill=None, unique=False)

Data object for describing a column in a DataFrame.

Arguments:

• name: the column name, or None to default to the column position. Must be hashable, but can otherwise be any value supported as a pandas column name.
• elements: the strategy for generating values in this column, or None to infer it from the dtype.
• dtype: the dtype of the column, or None to infer it from the element strategy. At least one of dtype or elements must be provided.
• fill: A default value for elements of the column. See arrays() for a full explanation.
• unique: If all values in this column should be distinct.

hypothesis.extra.pandas.columns(names_or_number, *, dtype=None, elements=None, fill=None, unique=False)

A convenience function for producing a list of column objects of the same general shape.

The names_or_number argument is either a sequence of values, the elements of which will be used as the name for individual column objects, or a number, in which case that many unnamed columns will be created. All other arguments are passed through verbatim to create the columns.

hypothesis.extra.pandas.data_frames(columns=None, *, rows=None, index=None)

Provides a strategy for producing a pandas.DataFrame.

Arguments:

• columns: An iterable of column objects describing the shape of the generated DataFrame.

• rows: A strategy for generating a row object. Should generate either dicts mapping column names to values or a sequence mapping column position to the value in that position (note that unlike the pandas.DataFrame constructor, single values are not allowed here. Passing e.g. an integer is an error, even if there is only one column).

  At least one of rows and columns must be provided. If both are provided then the generated rows will be validated against the columns and an error will be raised if they don't match.

  Caveats on using rows:

  • In general you should prefer using columns to rows, and only use rows if the columns interface is insufficiently flexible to describe what you need - you will get better performance and example quality that way.
  • If you provide rows and not columns, then the shape and dtype of the resulting DataFrame may vary. e.g. if you have a mix of int and float in the values for one column in your row entries, the column will sometimes have an integral dtype and sometimes a float.

• index: If not None, a strategy for generating indexes for the resulting DataFrame. This can generate either pandas.Index objects or any sequence of values (which will be passed to the Index constructor).

  You will probably find it most convenient to use the indexes() or range_indexes() function to produce values for this argument.

Usage:

The expected usage pattern is that you use column and columns() to specify a fixed shape of the DataFrame you want as follows. For example the following gives a two column data frame:

>>> from hypothesis.extra.pandas import column, data_frames
>>> data_frames([
... column('A', dtype=int), column('B', dtype=float)]).example()
            A              B
0  2021915903  1.793898e+232
1  1146643993            inf
2 -2096165693   1.000000e+07

If you want the values in different columns to interact in some way you can use the rows argument. For example the following gives a two column DataFrame where the value in the first column is always at most the value in the second:

>>> from hypothesis.extra.pandas import column, data_frames
>>> import hypothesis.strategies as st
>>> data_frames(
...     rows=st.tuples(st.floats(allow_nan=False),
...                    st.floats(allow_nan=False)).map(sorted)
... ).example()
               0             1
0  -3.402823e+38  9.007199e+15
1 -1.562796e-298  5.000000e-01

You can also combine the two:

>>> from hypothesis.extra.pandas import columns, data_frames
>>> import hypothesis.strategies as st
>>> data_frames(
...     columns=columns(["lo", "hi"], dtype=float),
...     rows=st.tuples(st.floats(allow_nan=False),
...                    st.floats(allow_nan=False)).map(sorted)
... ).example()
         lo            hi
0   9.314723e-49  4.353037e+45
1  -9.999900e-01  1.000000e+07
2 -2.152861e+134 -1.069317e-73

(Note that the column dtype must still be specified and will not be inferred from the rows. This restriction may be lifted in future).

Combining rows and columns has the following behaviour:

• The column names and dtypes will be used.
• If the column is required to be unique, this will be enforced.
• Any values missing from the generated rows will be provided using the column's fill.
• Any values in the row not present in the column specification (if dicts are passed, if there are keys with no corresponding column name, if sequences are passed if there are too many items) will result in InvalidArgument being raised.

There is quite a lot of variation between pandas versions. We only commit to supporting the latest version of pandas, but older minor versions are supported on a "best effort" basis.  Hypothesis is currently tested against and confirmed working with every Pandas minor version from 1.1 through to 2.2.

Releases that are not the latest patch release of their minor version are not tested or officially supported, but will probably also work unless you hit a pandas bug.

Hypothesis offers strategies for Array API adopting libraries in the hypothesis.extra.array_api package. See issue #3037 for more details.  If you want to test with CuPy, Dask, JAX, MXNet, PyTorch, TensorFlow, or Xarray - or just NumPy - this is the extension for you!

hypothesis.extra.array_api.make_strategies_namespace(xp, *, api_version=None)

Creates a strategies namespace for the given array module.

• xp is the Array API library to automatically pass to the namespaced methods.
• api_version is the version of the Array API which the returned strategies namespace should conform to. If None, the latest API version which xp supports will be inferred from xp.__array_api_version__. If a version string in the YYYY.MM format, the strategies namespace will conform to that version if supported.

A python:types.SimpleNamespace is returned which contains all the strategy methods in this module but without requiring the xp argument. Creating and using a strategies namespace for NumPy's Array API implementation would go like this:

>>> xp.__array_api_version__  # xp is your desired array library
'2021.12'
>>> xps = make_strategies_namespace(xp)
>>> xps.api_version
'2021.12'
>>> x = xps.arrays(xp.int8, (2, 3)).example()
>>> x
Array([[-8,  6,  3],
       [-6,  4,  6]], dtype=int8)
>>> x.__array_namespace__() is xp
True

The resulting namespace contains all our familiar strategies like arrays() and from_dtype(), but based on the Array API standard semantics and returning objects from the xp module:

xps.from_dtype(dtype, *, min_value=None, max_value=None, allow_nan=None, allow_infinity=None, allow_subnormal=None, exclude_min=None, exclude_max=None)

Return a strategy for any value of the given dtype.

Values generated are of the Python scalar which is promotable to dtype, where the values do not exceed its bounds.

• dtype may be a dtype object or the string name of a valid dtype.

Compatible **kwargs are passed to the inferred strategy function for integers and floats.  This allows you to customise the min and max values, and exclude non-finite numbers. This is particularly useful when kwargs are passed through from arrays(), as it seamlessly handles the width or other representable bounds for you.

xps.arrays(dtype, shape, *, elements=None, fill=None, unique=False)

Returns a strategy for arrays.

• dtype may be a valid dtype object or name, or a strategy that generates such values.
• shape may be an integer >= 0, a tuple of such integers, or a strategy that generates such values.
• elements is a strategy for values to put in the array. If None then a suitable value will be inferred based on the dtype, which may give any legal value (including e.g. NaN for floats). If a mapping, it will be passed as **kwargs to from_dtype() when inferring based on the dtype.
• fill is a strategy that may be used to generate a single background value for the array. If None, a suitable default will be inferred based on the other arguments. If set to nothing() then filling behaviour will be disabled entirely and every element will be generated independently.
• unique specifies if the elements of the array should all be distinct from one another; if fill is also set, the only valid values for fill to return are NaN values.

Arrays of specified dtype and shape are generated for example like this:

>>> from numpy import array_api as xp
>>> xps.arrays(xp, xp.int8, (2, 3)).example()
Array([[-8,  6,  3],
       [-6,  4,  6]], dtype=int8)

Specifying element boundaries by a python:dict of the kwargs to pass to from_dtype() will ensure dtype bounds will be respected.

>>> xps.arrays(xp, xp.int8, 3, elements={"min_value": 10}).example()
Array([125, 13, 79], dtype=int8)

Refer to What you can generate and how for passing your own elements strategy.

>>> xps.arrays(xp, xp.float32, 3, elements=floats(0, 1, width=32)).example()
Array([ 0.88974794,  0.77387938,  0.1977879 ], dtype=float32)

Array values are generated in two parts:

1. A single value is drawn from the fill strategy and is used to create a filled array.
2. Some subset of the coordinates of the array are populated with a value drawn from the elements strategy (or its inferred form).

You can set fill to nothing() if you want to disable this behaviour and draw a value for every element.

By default arrays will attempt to infer the correct fill behaviour: if unique is also True, no filling will occur. Otherwise, if it looks safe to reuse the values of elements across multiple coordinates (this will be the case for any inferred strategy, and for most of the builtins, but is not the case for mutable values or strategies built with flatmap, map, composite, etc.) then it will use the elements strategy as the fill, else it will default to having no fill.

Having a fill helps Hypothesis craft high quality examples, but its main importance is when the array generated is large: Hypothesis is primarily designed around testing small examples. If you have arrays with hundreds or more elements, having a fill value is essential if you want your tests to run in reasonable time.

xps.array_shapes(*, min_dims=1, max_dims=None, min_side=1, max_side=None)

Return a strategy for array shapes (tuples of int >= 1).

• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to min_dims + 2.
• min_side is the smallest size that a dimension can possess.
• max_side is the largest size that a dimension can possess, defaulting to min_side + 5.

xps.scalar_dtypes()

Return a strategy for all valid dtype objects.

xps.boolean_dtypes()

Return a strategy for just the boolean dtype object.

xps.numeric_dtypes()

Return a strategy for all numeric dtype objects.

xps.real_dtypes()

Return a strategy for all real-valued dtype objects.

xps.integer_dtypes(*, sizes=(8, 16, 32, 64))

Return a strategy for signed integer dtype objects.

sizes contains the signed integer sizes in bits, defaulting to (8, 16, 32, 64) which covers all valid sizes.

xps.unsigned_integer_dtypes(*, sizes=(8, 16, 32, 64))

Return a strategy for unsigned integer dtype objects.

sizes contains the unsigned integer sizes in bits, defaulting to (8, 16, 32, 64) which covers all valid sizes.

xps.floating_dtypes(*, sizes=(32, 64))

Return a strategy for real-valued floating-point dtype objects.

sizes contains the floating-point sizes in bits, defaulting to (32, 64) which covers all valid sizes.

xps.complex_dtypes(*, sizes=(64, 128))

Return a strategy for complex dtype objects.

sizes contains the complex sizes in bits, defaulting to (64, 128) which covers all valid sizes.

xps.valid_tuple_axes(ndim, *, min_size=0, max_size=None)

Return a strategy for permissible tuple-values for the axis argument in Array API sequential methods e.g. sum, given the specified dimensionality.

All tuples will have a length >= min_size and <= max_size. The default

value for max_size is ndim.

Examples from this strategy shrink towards an empty tuple, which render most sequential functions as no-ops.

The following are some examples drawn from this strategy.

>>> [valid_tuple_axes(3).example() for i in range(4)]
[(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]

valid_tuple_axes can be joined with other strategies to generate any type of valid axis object, i.e. integers, tuples, and None:

any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)

xps.broadcastable_shapes(shape, *, min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for shapes that are broadcast-compatible with the provided shape.

Examples from this strategy shrink towards a shape with length min_dims. The size of an aligned dimension shrinks towards size 1. The size of an unaligned dimension shrink towards min_side.

• shape is a tuple of integers.
• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to max(len(shape), min_dims) + 2.
• min_side is the smallest size that an unaligned dimension can possess.
• max_side is the largest size that an unaligned dimension can possess, defaulting to 2 plus the size of the largest aligned dimension.

The following are some examples drawn from this strategy.

>>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
[(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]

xps.mutually_broadcastable_shapes(num_shapes, *, base_shape=(), min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for a specified number of shapes N that are mutually-broadcastable with one another and with the provided base shape.

• num_shapes is the number of mutually broadcast-compatible shapes to generate.
• base_shape is the shape against which all generated shapes can broadcast. The default shape is empty, which corresponds to a scalar and thus does not constrain broadcasting at all.
• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to max(len(shape), min_dims) + 2.
• min_side is the smallest size that an unaligned dimension can possess.
• max_side is the largest size that an unaligned dimension can possess, defaulting to 2 plus the size of the largest aligned dimension.

The strategy will generate a python:typing.NamedTuple containing:

• input_shapes as a tuple of the N generated shapes.
• result_shape as the resulting shape produced by broadcasting the N shapes with the base shape.

The following are some examples drawn from this strategy.

>>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
>>> for _ in range(5):
...     print(strat.example())
BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
BroadcastableShapes(input_shapes=((), (), ()), result_shape=())
BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(3,))
BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))

xps.indices(shape, *, min_dims=0, max_dims=None, allow_newaxis=False, allow_ellipsis=True)

Return a strategy for valid indices of arrays with the specified shape, which may include dimensions of size zero.

It generates tuples containing some mix of integers, python:slice objects, ... (an Ellipsis), and None. When a length-one tuple would be generated, this strategy may instead return the element which will index the first axis, e.g. 5 instead of (5,).

• shape is the shape of the array that will be indexed, as a tuple of integers >= 0. This must be at least two-dimensional for a tuple to be a valid index;  for one-dimensional arrays use slices() instead.
• min_dims is the minimum dimensionality of the resulting array from use of the generated index.
• max_dims is the the maximum dimensionality of the resulting array, defaulting to len(shape) if not allow_newaxis else max(len(shape), min_dims) + 2.
• allow_ellipsis specifies whether None is allowed in the index.
• allow_ellipsis specifies whether ... is allowed in the index.

The database is best thought of as a cache that you never need to invalidate: Information may be lost when you upgrade a Hypothesis version or change your test, so you shouldn't rely on it for correctness - if there's an example you want to ensure occurs each time then there's a feature for including them in your source code - but it helps the development workflow considerably by making sure that the examples you've just found are reproduced.

The database also records examples that exercise less-used parts of your code, so the database may update even when no failing examples were found.

The design of the Hypothesis database is such that you can put arbitrary data in the database and not get wrong behaviour. When you upgrade Hypothesis, old data might be invalidated, but this should happen transparently. It can never be the case that e.g. changing the strategy that generates an argument gives you data from the old strategy.

Hypothesis' default database setting creates a DirectoryBasedExampleDatabase in your current working directory, under .hypothesis/examples.  If this location is unusable, e.g. because you do not have read or write permissions, Hypothesis will emit a warning and fall back to an InMemoryExampleDatabase.

Hypothesis provides the following ExampleDatabase implementations:

class hypothesis.database.InMemoryExampleDatabase

A non-persistent example database, implemented in terms of a dict of sets.

This can be useful if you call a test function several times in a single session, or for testing other database implementations, but because it does not persist between runs we do not recommend it for general use.

class hypothesis.database.DirectoryBasedExampleDatabase(path)

Use a directory to store Hypothesis examples as files.

Each test corresponds to a directory, and each example to a file within that directory.  While the contents are fairly opaque, a DirectoryBasedExampleDatabase can be shared by checking the directory into version control, for example with the following .gitignore:

# Ignore files cached by Hypothesis...
.hypothesis/*
# except for the examples directory
!.hypothesis/examples/

Note however that this only makes sense if you also pin to an exact version of Hypothesis, and we would usually recommend implementing a shared database with a network datastore - see ExampleDatabase, and the MultiplexedDatabase helper.

class hypothesis.database.GitHubArtifactDatabase(owner, repo, artifact_name='hypothesis-example-db', cache_timeout=datetime.timedelta(days=1), path=None)

A file-based database loaded from a GitHub Actions artifact.

You can use this for sharing example databases between CI runs and developers, allowing the latter to get read-only access to the former. This is particularly useful for continuous fuzzing (i.e. with HypoFuzz), where the CI system can help find new failing examples through fuzzing, and developers can reproduce them locally without any manual effort.

NOTE:

You must provide GITHUB_TOKEN as an environment variable. In CI, Github Actions provides this automatically, but it needs to be set manually for local usage. In a developer machine, this would usually be a Personal Access Token. If the repository is private, it's necessary for the token to have repo scope in the case of a classic token, or actions:read in the case of a fine-grained token.

In most cases, this will be used through the MultiplexedDatabase, by combining a local directory-based database with this one. For example:

local = DirectoryBasedExampleDatabase(".hypothesis/examples")
shared = ReadOnlyDatabase(GitHubArtifactDatabase("user", "repo"))

settings.register_profile("ci", database=local)
settings.register_profile("dev", database=MultiplexedDatabase(local, shared))
# We don't want to use the shared database in CI, only to populate its local one.
# which the workflow should then upload as an artifact.
settings.load_profile("ci" if os.environ.get("CI") else "dev")

NOTE:

Because this database is read-only, you always need to wrap it with the ReadOnlyDatabase.

A setup like this can be paired with a GitHub Actions workflow including something like the following:

- name: Download example database
  uses: dawidd6/action-download-artifact@v2.24.3
  with:
    name: hypothesis-example-db
    path: .hypothesis/examples
    if_no_artifact_found: warn
    workflow_conclusion: completed

- name: Run tests
  run: pytest

- name: Upload example database
  uses: actions/upload-artifact@v3
  if: always()
  with:
    name: hypothesis-example-db
    path: .hypothesis/examples

In this workflow, we use dawidd6/action-download-artifact to download the latest artifact given that the official actions/download-artifact does not support downloading artifacts from previous workflow runs.

The database automatically implements a simple file-based cache with a default expiration period of 1 day. You can adjust this through the cache_timeout property.

For mono-repo support, you can provide a unique artifact_name (e.g. hypofuzz-example-db-frontend).

class hypothesis.database.ReadOnlyDatabase(db)

A wrapper to make the given database read-only.

The implementation passes through fetch, and turns save, delete, and move into silent no-ops.

Note that this disables Hypothesis' automatic discarding of stale examples. It is designed to allow local machines to access a shared database (e.g. from CI servers), without propagating changes back from a local or in-development branch.

class hypothesis.database.MultiplexedDatabase(*dbs)

A wrapper around multiple databases.

Each save, fetch, move, or delete operation will be run against all of the wrapped databases.  fetch does not yield duplicate values, even if the same value is present in two or more of the wrapped databases.

This combines well with a ReadOnlyDatabase, as follows:

local = DirectoryBasedExampleDatabase("/tmp/hypothesis/examples/")
shared = CustomNetworkDatabase()

settings.register_profile("ci", database=shared)
settings.register_profile(
    "dev", database=MultiplexedDatabase(local, ReadOnlyDatabase(shared))
)
settings.load_profile("ci" if os.environ.get("CI") else "dev")

So your CI system or fuzzing runs can populate a central shared database; while local runs on development machines can reproduce any failures from CI but will only cache their own failures locally and cannot remove examples from the shared database.

class hypothesis.extra.redis.RedisExampleDatabase(redis, *, expire_after=datetime.timedelta(days=8), key_prefix=b'hypothesis-example:')

Store Hypothesis examples as sets in the given Redis datastore.

This is particularly useful for shared databases, as per the recipe for a MultiplexedDatabase.

NOTE:

If a test has not been run for expire_after, those examples will be allowed to expire.  The default time-to-live persists examples between weekly runs.

You can define your ExampleDatabase, for example to use a shared datastore, with just a few methods:

class hypothesis.database.ExampleDatabase(*args, **kwargs)

An abstract base class for storing examples in Hypothesis' internal format.

An ExampleDatabase maps each bytes key to many distinct bytes values, like a Mapping[bytes, AbstractSet[bytes]].

abstract save(key, value)

Save value under key.

If this value is already present for this key, silently do nothing.

abstract fetch(key)

Return an iterable over all values matching this key.

abstract delete(key, value)

Remove this value from this key.

If this value is not present, silently do nothing.

move(src, dest, value)

Move value from key src to key dest. Equivalent to delete(src, value) followed by save(src, value), but may have a more efficient implementation.

Note that value will be inserted at dest regardless of whether it is currently present at src.

The basic idea of stateful testing is to make Hypothesis choose actions as well as values for your test, and state machines are a great declarative way to do just that.

For simpler cases though, you might not need them at all - a standard test with @given might be enough, since you can use data() in branches or loops.  In fact, that's how the state machine explorer works internally.  For more complex workloads though, where a higher level API comes into it's own, keep reading!

class hypothesis.stateful.RuleBasedStateMachine

A RuleBasedStateMachine gives you a structured way to define state machines.

The idea is that a state machine carries the system under test and some supporting data. This data can be stored in instance variables or divided into Bundles. The state machine has a set of rules which may read data from bundles (or just from normal strategies), push data onto bundles, change the state of the machine, or verify properties. At any given point a random applicable rule will be executed.

A rule is very similar to a normal @given based test in that it takes values drawn from strategies and passes them to a user defined test function, which may use assertions to check the system's behavior. The key difference is that where @given based tests must be independent, rules can be chained together - a single test run may involve multiple rule invocations, which may interact in various ways.

Rules can take normal strategies as arguments, but normal strategies, with the exception of  runner() and data(), cannot take into account the current state of the machine. This is where bundles come in.

A rule can, in place of a normal strategy, take a Bundle. A hypothesis.stateful.Bundle is a named collection of generated values that can be reused by other operations in the test. They are populated with the results of rules, and may be used as arguments to rules, allowing data to flow from one rule to another, and rules to work on the results of previous computations or actions.

Specifically, a rule that specifies target=a_bundle will cause its return value to be added to that bundle. A rule that specifies an_argument=a_bundle as a strategy will draw a value from that bundle.  A rule can also specify that an argument chooses a value from a bundle and removes that value by using consumes() as in an_argument=consumes(a_bundle).

NOTE:

The following rule based state machine example is a simplified version of a test for Hypothesis's example database implementation. An example database maps keys to sets of values, and in this test we compare one implementation of it to a simplified in memory model of its behaviour, which just stores the same values in a Python dict. The test then runs operations against both the real database and the in-memory representation of it and looks for discrepancies in their behaviour.

import shutil
import tempfile
from collections import defaultdict

import hypothesis.strategies as st
from hypothesis.database import DirectoryBasedExampleDatabase
from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule


class DatabaseComparison(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.tempd = tempfile.mkdtemp()
        self.database = DirectoryBasedExampleDatabase(self.tempd)
        self.model = defaultdict(set)

    keys = Bundle("keys")
    values = Bundle("values")

    @rule(target=keys, k=st.binary())
    def add_key(self, k):
        return k

    @rule(target=values, v=st.binary())
    def add_value(self, v):
        return v

    @rule(k=keys, v=values)
    def save(self, k, v):
        self.model[k].add(v)
        self.database.save(k, v)

    @rule(k=keys, v=values)
    def delete(self, k, v):
        self.model[k].discard(v)
        self.database.delete(k, v)

    @rule(k=keys)
    def values_agree(self, k):
        assert set(self.database.fetch(k)) == self.model[k]

    def teardown(self):
        shutil.rmtree(self.tempd)


TestDBComparison = DatabaseComparison.TestCase

In this we declare two bundles - one for keys, and one for values. We have two trivial rules which just populate them with data (k and v), and three non-trivial rules: save saves a value under a key and delete removes a value from a key, in both cases also updating the model of what should be in the database. values_agree then checks that the contents of the database agrees with the model for a particular key.

NOTE:

We can now integrate this into our test suite by getting a unittest TestCase from it:

TestTrees = DatabaseComparison.TestCase

# Or just run with pytest's unittest support
if __name__ == "__main__":
    unittest.main()

This test currently passes, but if we comment out the line where we call self.model[k].discard(v), we would see the following output when run under pytest:

AssertionError: assert set() == {b''}

------------ Hypothesis ------------

state = DatabaseComparison()
var1 = state.add_key(k=b'')
var2 = state.add_value(v=var1)
state.save(k=var1, v=var2)
state.delete(k=var1, v=var2)
state.values_agree(k=var1)
state.teardown()

Note how it's printed out a very short program that will demonstrate the problem. The output from a rule based state machine should generally be pretty close to Python code - if you have custom repr implementations that don't return valid Python then it might not be, but most of the time you should just be able to copy and paste the code into a test to reproduce it.

You can control the detailed behaviour with a settings object on the TestCase (this is a normal hypothesis settings object using the defaults at the time the TestCase class was first referenced). For example if you wanted to run fewer examples with larger programs you could change the settings to:

DatabaseComparison.TestCase.settings = settings(
    max_examples=50, stateful_step_count=100
)

Which doubles the number of steps each program runs and halves the number of test cases that will be run.

As said earlier, rules are the most common feature used in RuleBasedStateMachine. They are defined by applying the rule() decorator on a function. Note that RuleBasedStateMachine must have at least one rule defined and that a single function cannot be used to define multiple rules (this to avoid having multiple rules doing the same things). Due to the stateful execution method, rules generally cannot take arguments from other sources such as fixtures or pytest.mark.parametrize - consider providing them via a strategy such as sampled_from() instead.

hypothesis.stateful.rule(*, targets=(), target=None, **kwargs)

Decorator for RuleBasedStateMachine. Any Bundle present in target or targets will define where the end result of this function should go. If both are empty then the end result will be discarded.

target must be a Bundle, or if the result should be replicated to multiple bundles you can pass a tuple of them as the targets argument. It is invalid to use both arguments for a single rule.  If the result should go to exactly one of several bundles, define a separate rule for each case.

kwargs then define the arguments that will be passed to the function invocation. If their value is a Bundle, or if it is consumes(b) where b is a Bundle, then values that have previously been produced for that bundle will be provided. If consumes is used, the value will also be removed from the bundle.

Any other kwargs should be strategies and values from them will be provided.

hypothesis.stateful.consumes(bundle)

When introducing a rule in a RuleBasedStateMachine, this function can be used to mark bundles from which each value used in a step with the given rule should be removed. This function returns a strategy object that can be manipulated and combined like any other.

For example, a rule declared with

@rule(value1=b1, value2=consumes(b2), value3=lists(consumes(b3)))

will consume a value from Bundle b2 and several values from Bundle b3 to populate value2 and value3 each time it is executed.

hypothesis.stateful.multiple(*args)

This function can be used to pass multiple results to the target(s) of a rule. Just use return multiple(result1, result2, ...) in your rule.

It is also possible to use return multiple() with no arguments in order to end a rule without passing any result.

class hypothesis.stateful.Bundle(name, *, consume=False)

A collection of values for use in stateful testing.

Bundles are a kind of strategy where values can be added by rules, and (like any strategy) used as inputs to future rules.

The name argument they are passed is the they are referred to internally by the state machine; no two bundles may have the same name. It is idiomatic to use the attribute being assigned to as the name of the Bundle:

class MyStateMachine(RuleBasedStateMachine):
    keys = Bundle("keys")

Bundles can contain the same value more than once; this becomes relevant when using consumes() to remove values again.

If the consume argument is set to True, then all values that are drawn from this bundle will be consumed (as above) when requested.

Initializes are a special case of rules, which are guaranteed to be run exactly once before any normal rule is called. Note if multiple initialize rules are defined, they will all be called but in any order, and that order will vary from run to run.

Initializes are typically useful to populate bundles:

hypothesis.stateful.initialize(*, targets=(), target=None, **kwargs)

Decorator for RuleBasedStateMachine.

An initialize decorator behaves like a rule, but all @initialize() decorated methods will be called before any @rule() decorated methods, in an arbitrary order.  Each @initialize() method will be called exactly once per run, unless one raises an exception - after which only the .teardown() method will be run. @initialize() methods may not have preconditions.

import hypothesis.strategies as st
from hypothesis.stateful import Bundle, RuleBasedStateMachine, initialize, rule

name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)


class NumberModifier(RuleBasedStateMachine):
    folders = Bundle("folders")
    files = Bundle("files")

    @initialize(target=folders)
    def init_folders(self):
        return "/"

    @rule(target=folders, name=name_strategy)
    def create_folder(self, parent, name):
        return f"{parent}/{name}"

    @rule(target=files, name=name_strategy)
    def create_file(self, parent, name):
        return f"{parent}/{name}"

Initializes can also allow you to initialize the system under test in a way that depends on values chosen from a strategy. You could do this by putting an instance variable in the state machine that indicates whether the system under test has been initialized or not, and then using preconditions (below) to ensure that exactly one of the rules that initialize it get run before any rules that depend on it being initialized.

While it's possible to use assume() in RuleBasedStateMachine rules, if you use it in only a few rules you can quickly run into a situation where few or none of your rules pass their assumptions. Thus, Hypothesis provides a precondition() decorator to avoid this problem. The precondition() decorator is used on rule-decorated functions, and must be given a function that returns True or False based on the RuleBasedStateMachine instance.

hypothesis.stateful.precondition(precond)

Decorator to apply a precondition for rules in a RuleBasedStateMachine. Specifies a precondition for a rule to be considered as a valid step in the state machine, which is more efficient than using assume() within the rule.  The precond function will be called with the instance of RuleBasedStateMachine and should return True or False. Usually it will need to look at attributes on that instance.

For example:

class MyTestMachine(RuleBasedStateMachine):
    state = 1

    @precondition(lambda self: self.state != 0)
    @rule(numerator=integers())
    def divide_with(self, numerator):
        self.state = numerator / self.state

If multiple preconditions are applied to a single rule, it is only considered a valid step when all of them return True.  Preconditions may be applied to invariants as well as rules.

from hypothesis.stateful import RuleBasedStateMachine, precondition, rule


class NumberModifier(RuleBasedStateMachine):
    num = 0

    @rule()
    def add_one(self):
        self.num += 1

    @precondition(lambda self: self.num != 0)
    @rule()
    def divide_with_one(self):
        self.num = 1 / self.num

By using precondition() here instead of assume(), Hypothesis can filter the inapplicable rules before running them. This makes it much more likely that a useful sequence of steps will be generated.

Note that currently preconditions can't access bundles; if you need to use preconditions, you should store relevant data on the instance instead.

Often there are invariants that you want to ensure are met after every step in a process.  It would be possible to add these as rules that are run, but they would be run zero or multiple times between other rules. Hypothesis provides a decorator that marks a function to be run after every step.

hypothesis.stateful.invariant(*, check_during_init=False)

Decorator to apply an invariant for rules in a RuleBasedStateMachine. The decorated function will be run after every rule and can raise an exception to indicate failed invariants.

For example:

class MyTestMachine(RuleBasedStateMachine):
    state = 1

    @invariant()
    def is_nonzero(self):
        assert self.state != 0

By default, invariants are only checked after all @initialize() rules have been run. Pass check_during_init=True for invariants which can also be checked during initialization.

from hypothesis.stateful import RuleBasedStateMachine, invariant, rule


class NumberModifier(RuleBasedStateMachine):
    num = 0

    @rule()
    def add_two(self):
        self.num += 2
        if self.num > 50:
            self.num += 1

    @invariant()
    def divide_with_one(self):
        assert self.num % 2 == 0


NumberTest = NumberModifier.TestCase

Invariants can also have precondition()s applied to them, in which case they will only be run if the precondition function returns true.

Note that currently invariants can't access bundles; if you need to use invariants, you should store relevant data on the instance instead.

If you want to bypass the TestCase infrastructure you can invoke these manually. The stateful module exposes the function run_state_machine_as_test, which takes an arbitrary function returning a RuleBasedStateMachine and an optional settings parameter and does the same as the class based runTest provided.

This is not recommended as it bypasses some important internal functions, including reporting of statistics such as runtimes and event() calls.  It was originally added to support custom __init__ methods, but you can now use initialize() rules instead.

Backwards compatibility is better than backporting fixes, so we use semantic versioning and only support the most recent version of Hypothesis.  See Help and support for more information.

Documented APIs will not break except between major version bumps. All APIs mentioned in this documentation are public unless explicitly noted as provisional, in which case they may be changed in minor releases. Undocumented attributes, modules, and behaviour may include breaking changes in patch releases.

Deprecated features will emit warnings for at least six months, and then be removed in the following major release.

Note however that not all warnings are subject to this grace period; sometimes we strengthen validation by adding a warning and these may become errors immediately at a major release.

We use custom exception and warning types, so you can see exactly where an error came from, or turn only our warnings into errors.

class hypothesis.errors.HypothesisDeprecationWarning

A deprecation warning issued by Hypothesis.

Actually inherits from FutureWarning, because DeprecationWarning is hidden by the default warnings filter.

You can configure the Python python:warnings to handle these warnings differently to others, either turning them into errors or suppressing them entirely.  Obviously we would prefer the former!

Hypothesis is supported and tested on CPython 3.8+, i.e. all versions of CPython with upstream support, along with PyPy for the same versions. 32-bit builds of CPython also work, though we only test them on Windows.

In general Hypothesis does not officially support anything except the latest patch release of any version of Python it supports. Earlier releases should work and bugs in them will get fixed if reported, but they're not tested in CI and no guarantees are made.

In theory Hypothesis should work anywhere that Python does. In practice it is only known to work and regularly tested on OS X, Windows and Linux, and you may experience issues running it elsewhere.

If you're using something else and it doesn't work, do get in touch and I'll try to help, but unless you can come up with a way for me to run a CI server on that operating system it probably won't stay fixed due to the inevitable march of time.

In general Hypothesis goes to quite a lot of effort to generate things that look like normal Python test functions that behave as closely to the originals as possible, so it should work sensibly out of the box with every test framework.

If your testing relies on doing something other than calling a function and seeing if it raises an exception then it probably won't work out of the box. In particular things like tests which return generators and expect you to do something with them (e.g. nose's yield based tests) will not work. Use a decorator or similar to wrap the test to take this form, or ask the framework maintainer to support our hooks for inserting such a wrapper later.

In terms of what's actually known to work:

• Hypothesis integrates as smoothly with pytest and unittest as we can make it, and this is verified as part of the CI.
• pytest fixtures work in the usual way for tests that have been decorated with @given - just avoid passing a strategy for each argument that will be supplied by a fixture.  However, each fixture will run once for the whole function, not once per example.  Decorating a fixture function with @given is meaningless.
• The python:unittest.mock.patch() decorator works with @given, but we recommend using it as a context manager within the decorated test to ensure that the mock is per-test-case and avoid poor interactions with Pytest fixtures.
• Nose works fine with Hypothesis, and this is tested as part of the CI. yield based tests simply won't work.
• Integration with Django's testing requires use of the Hypothesis for Django users extra. The issue is that in Django's tests' normal mode of execution it will reset the database once per test rather than once per example, which is not what you want.
• Coverage works out of the box with Hypothesis; our own test suite has 100% branch coverage.

The supported versions of optional packages, for strategies in hypothesis.extra, are listed in the documentation for that extra.  Our general goal is to support all versions that are supported upstream.

Everything mentioned above as explicitly supported is checked on every commit with GitHub Actions. Our continuous delivery pipeline runs all of these checks before publishing each release, so when we say they're supported we really mean it.

The following is an example that's been extracted and simplified from a real bug that occurred in an earlier version of Hypothesis. The real bug was a lot harder to find.

Suppose we've got the following type:

class Node:
    def __init__(self, label, value):
        self.label = label
        self.value = tuple(value)

    def __repr__(self):
        return f"Node({self.label!r}, {self.value!r})"

    def sorts_before(self, other):
        if len(self.value) >= len(other.value):
            return False
        return other.value[: len(self.value)] == self.value

Each node is a label and a sequence of some data, and we have the relationship sorts_before meaning the data of the left is an initial segment of the right. So e.g. a node with value [1, 2] will sort before a node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort before the other.

We have a list of nodes, and we want to topologically sort them with respect to this ordering. That is, we want to arrange the list so that if x.sorts_before(y) then x appears earlier in the list than y. We naively think that the easiest way to do this is to extend the  partial order defined here to a total order by breaking ties arbitrarily and then using a normal sorting algorithm. So we define the following code:

from functools import total_ordering


@total_ordering
class TopoKey:
    def __init__(self, node):
        self.value = node

    def __lt__(self, other):
        if self.value.sorts_before(other.value):
            return True
        if other.value.sorts_before(self.value):
            return False

        return self.value.label < other.value.label


def sort_nodes(xs):
    xs.sort(key=TopoKey)

This takes the order defined by sorts_before and extends it by breaking ties by comparing the node labels.

But now we want to test that it works.

First we write a function to verify that our desired outcome holds:

def is_prefix_sorted(xs):
    for i in range(len(xs)):
        for j in range(i + 1, len(xs)):
            if xs[j].sorts_before(xs[i]):
                return False
    return True

This will return false if it ever finds a pair in the wrong order and return true otherwise.

Given this function, what we want to do with Hypothesis is assert that for all sequences of nodes, the result of calling sort_nodes on it is sorted.

First we need to define a strategy for Node:

import hypothesis.strategies as st

NodeStrategy = st.builds(Node, st.integers(), st.lists(st.booleans(), max_size=10))

We want to generate short lists of values so that there's a decent chance of one being a prefix of the other (this is also why the choice of bool as the elements). We then define a strategy which builds a node out of an integer and one of those short lists of booleans.

We can now write a test:

from hypothesis import given


@given(st.lists(NodeStrategy))
def test_sorting_nodes_is_prefix_sorted(xs):
    sort_nodes(xs)
    assert is_prefix_sorted(xs)

this immediately fails with the following example:

[Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]

The reason for this is that because False is not a prefix of (True, True) nor vice versa, sorting things the first two nodes are equal because they have equal labels. This makes the whole order non-transitive and produces basically nonsense results.

But this is pretty unsatisfying. It only works because they have the same label. Perhaps we actually wanted our labels to be unique. Let's change the test to do that.

def deduplicate_nodes_by_label(nodes):
    table = {node.label: node for node in nodes}
    return list(table.values())

We define a function to deduplicate nodes by labels, and can now map that over a strategy for lists of nodes to give us a strategy for lists of nodes with unique labels:

@given(st.lists(NodeStrategy).map(deduplicate_nodes_by_label))
def test_sorting_nodes_is_prefix_sorted(xs):
    sort_nodes(xs)
    assert is_prefix_sorted(xs)

Hypothesis quickly gives us an example of this still being wrong:

[Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))]

Now this is a more interesting example. None of the nodes will sort equal. What is happening here is that the first node is strictly less than the last node because (False,) is a prefix of (False, False). This is in turn strictly less than the middle node because neither is a prefix of the other and -2 < -1. The middle node is then less than the first node because -1 < 0.

So, convinced that our implementation is broken, we write a better one:

def sort_nodes(xs):
    for i in range(1, len(xs)):
        j = i - 1
        while j >= 0:
            if xs[j].sorts_before(xs[j + 1]):
                break
            xs[j], xs[j + 1] = xs[j + 1], xs[j]
            j -= 1

This is just insertion sort slightly modified - we swap a node backwards until swapping it further would violate the order constraints. The reason this works is because our order is a partial order already (this wouldn't produce a valid result for a general topological sorting - you need the transitivity).

We now run our test again and it passes, telling us that this time we've successfully managed to sort some nodes without getting it completely wrong. Go us.

This is an example of some tests for pytz which check that various timezone conversions behave as you would expect them to. These tests should all pass, and are mostly a demonstration of some useful sorts of thing to test with Hypothesis, and how the datetimes() strategy works.

from datetime import timedelta

# The datetimes strategy is naive by default, so tell it to use timezones
aware_datetimes = st.datetimes(timezones=st.timezones())


@given(aware_datetimes, st.timezones(), st.timezones())
def test_convert_via_intermediary(dt, tz1, tz2):
    """Test that converting between timezones is not affected
    by a detour via another timezone.
    """
    assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)


@given(aware_datetimes, st.timezones())
def test_convert_to_and_fro(dt, tz2):
    """If we convert to a new timezone and back to the old one
    this should leave the result unchanged.
    """
    tz1 = dt.tzinfo
    assert dt == dt.astimezone(tz2).astimezone(tz1)


@given(aware_datetimes, st.timezones())
def test_adding_an_hour_commutes(dt, tz):
    """When converting between timezones it shouldn't matter
    if we add an hour here or add an hour there.
    """
    an_hour = timedelta(hours=1)
    assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour


@given(aware_datetimes, st.timezones())
def test_adding_a_day_commutes(dt, tz):
    """When converting between timezones it shouldn't matter
    if we add a day here or add a day there.
    """
    a_day = timedelta(days=1)
    assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day

A classic paradox in voting theory, called Condorcet's paradox, is that majority preferences are not transitive. That is, there is a population and a set of three candidates A, B and C such that the majority of the population prefer A to B, B to C and C to A.

Wouldn't it be neat if we could use Hypothesis to provide an example of this?

Well as you can probably guess from the presence of this section, we can! The main trick is to decide how we want to represent the result of an election - for this example, we'll use a list of "votes", where each vote is a list of candidates in the voters preferred order. Without further ado, here is the code:

from collections import Counter

from hypothesis import given
from hypothesis.strategies import lists, permutations


# We need at least three candidates and at least three voters to have a
# paradox; anything less can only lead to victories or at worst ties.
@given(lists(permutations(["A", "B", "C"]), min_size=3))
def test_elections_are_transitive(election):
    all_candidates = {"A", "B", "C"}

    # First calculate the pairwise counts of how many prefer each candidate
    # to the other
    counts = Counter()
    for vote in election:
        for i in range(len(vote)):
            for j in range(i + 1, len(vote)):
                counts[(vote[i], vote[j])] += 1

    # Now look at which pairs of candidates one has a majority over the
    # other and store that.
    graph = {}
    for i in all_candidates:
        for j in all_candidates:
            if counts[(i, j)] > counts[(j, i)]:
                graph.setdefault(i, set()).add(j)

    # Now for each triple assert that it is transitive.
    for x in all_candidates:
        for y in graph.get(x, ()):
            for z in graph.get(y, ()):
                assert x not in graph.get(z, ())

The example Hypothesis gives me on my first run (your mileage may of course vary) is:

[["A", "B", "C"], ["B", "C", "A"], ["C", "A", "B"]]

Which does indeed do the job: The majority (votes 0 and 1) prefer B to C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1 and 2) prefer C to A. This is in fact basically the canonical example of the voting paradox.

Hypothesis's support for testing HTTP services is somewhat nascent. There are plans for some fully featured things around this, but right now they're probably quite far down the line.

But you can do a lot yourself without any explicit support! Here's a script I wrote to throw arbitrary data against the API for an entirely fictitious service called Waspfinder (this is only lightly obfuscated and you can easily figure out who I'm actually talking about, but I don't want you to run this code and hammer their API without their permission).

All this does is use Hypothesis to generate arbitrary JSON data matching the format their API asks for and check for 500 errors. More advanced tests which then use the result and go on to do other things are definitely also possible. The schemathesis package provides an excellent example of this!

import math
import os
import random
import time
import unittest
from collections import namedtuple

import requests

from hypothesis import assume, given, strategies as st

Goal = namedtuple("Goal", ("slug",))


# We just pass in our API credentials via environment variables.
waspfinder_token = os.getenv("WASPFINDER_TOKEN")
waspfinder_user = os.getenv("WASPFINDER_USER")
assert waspfinder_token is not None
assert waspfinder_user is not None

GoalData = st.fixed_dictionaries(
    {
        "title": st.text(),
        "goal_type": st.sampled_from(
            ["hustler", "biker", "gainer", "fatloser", "inboxer", "drinker", "custom"]
        ),
        "goaldate": st.one_of(st.none(), st.floats()),
        "goalval": st.one_of(st.none(), st.floats()),
        "rate": st.one_of(st.none(), st.floats()),
        "initval": st.floats(),
        "panic": st.floats(),
        "secret": st.booleans(),
        "datapublic": st.booleans(),
    }
)


needs2 = ["goaldate", "goalval", "rate"]


class WaspfinderTest(unittest.TestCase):
    @given(GoalData)
    def test_create_goal_dry_run(self, data):
        # We want slug to be unique for each run so that multiple test runs
        # don't interfere with each other. If for some reason some slugs trigger
        # an error and others don't we'll get a Flaky error, but that's OK.
        slug = hex(random.getrandbits(32))[2:]

        # Use assume to guide us through validation we know about, otherwise
        # we'll spend a lot of time generating boring examples.

        # Title must not be empty
        assume(data["title"])

        # Exactly two of these values should be not None. The other will be
        # inferred by the API.

        assume(len([1 for k in needs2 if data[k] is not None]) == 2)
        for v in data.values():
            if isinstance(v, float):
                assume(not math.isnan(v))
        data["slug"] = slug

        # The API nicely supports a dry run option, which means we don't have
        # to worry about the user account being spammed with lots of fake goals
        # Otherwise we would have to make sure we cleaned up after ourselves
        # in this test.
        data["dryrun"] = True
        data["auth_token"] = waspfinder_token
        for d, v in data.items():
            if v is None:
                data[d] = "null"
            else:
                data[d] = str(v)
        result = requests.post(
            "https://waspfinder.example.com/api/v1/users/"
            "%s/goals.json" % (waspfinder_user,),
            data=data,
        )

        # Let's not hammer the API too badly. This will of course make the
        # tests even slower than they otherwise would have been, but that's
        # life.
        time.sleep(1.0)

        # For the moment all we're testing is that this doesn't generate an
        # internal error. If we didn't use the dry run option we could have
        # then tried doing more with the result, but this is a good start.
        self.assertNotEqual(result.status_code, 500)


if __name__ == "__main__":
    unittest.main()

At Stripe we use Hypothesis to test every piece of our machine learning model training pipeline (powered by scikit). Before we migrated, our tests were filled with hand-crafted pandas Dataframes that weren't representative at all of our actual very complex data. Because we needed to craft examples for each test, we took the easy way out and lived with extremely low test coverage.

Hypothesis changed all that. Once we had our strategies for generating Dataframes of features it became trivial to slightly customize each strategy for new tests. Our coverage is now close to 90%.

Full-stop, property-based testing is profoundly more powerful - and has caught or prevented far more bugs - than our old style of example-based testing.

Hypothesis has been brilliant for expanding the coverage of our test cases, and also for making them much easier to read and understand, so we're sure we're testing the things we want in the way we want.

When I first heard about Hypothesis, I knew I had to include it in my two open-source Python libraries, natsort and fastnumbers . Quite frankly, I was a little appalled at the number of bugs and "holes" I found in the code. I can now say with confidence that my libraries are more robust to "the wild." In addition, Hypothesis gave me the confidence to expand these libraries to fully support Unicode input, which I never would have had the stomach for without such thorough testing capabilities. Thanks!

At Sixty North we use Hypothesis for testing Segpy an open source Python library for shifting data between Python data structures and SEG Y files which contain geophysical data from the seismic reflection surveys used in oil and gas exploration.

This is our first experience of property-based testing – as opposed to example-based testing.  Not only are our tests more powerful, they are also much better explanations of what we expect of the production code. In fact, the tests are much closer to being specifications.  Hypothesis has located real defects in our code which went undetected by traditional test cases, simply because Hypothesis is more relentlessly devious about test case generation than us mere humans!  We found Hypothesis particularly beneficial for Segpy because SEG Y is an antiquated format that uses legacy text encodings (EBCDIC) and even a legacy floating point format we implemented from scratch in Python.

Hypothesis is sure to find a place in most of our future Python codebases and many existing ones too.

Just found out about this excellent QuickCheck for Python implementation and ran up a few tests for my bytesize package last night. Refuted a few hypotheses in the process.

Looking forward to using it with a bunch of other projects as well.

I have written a small library to serialize dicts to MariaDB's dynamic columns binary format, mariadb-dyncol. When I first developed it, I thought I had tested it really well - there were hundreds of test cases, some of them even taken from MariaDB's test suite itself. I was ready to release.

Lucky for me, I tried Hypothesis with David at the PyCon UK sprints. Wow! It found bug after bug after bug. Even after a first release, I thought of a way to make the tests do more validation, which revealed a further round of bugs! Most impressively, Hypothesis found a complicated off-by-one error in a condition with 4095 versus 4096 bytes of data - something that I would never have found.

Long live Hypothesis! (Or at least, property-based testing).

Adopting Hypothesis improved bidict's test coverage and significantly increased our ability to make changes to the code with confidence that correct behavior would be preserved. Thank you, David, for the great testing tool.

Hypothesis is the single most powerful tool in my toolbox for working with algorithmic code, or any software that produces predictable output from a wide range of sources. When using it with Priority, Hypothesis consistently found errors in my assumptions and extremely subtle bugs that would have taken months of real-world use to locate. In some cases, Hypothesis found subtle deviations from the correct output of the algorithm that may never have been noticed at all.

When it comes to validating the correctness of your tools, nothing comes close to the thoroughness and power of Hypothesis.

One extremely satisfied user here. Hypothesis is a really solid implementation of property-based testing, adapted well to Python, and with good features such as failure-case shrinkers. I first used it on a project where we needed to verify that a vendor's Python and non-Python implementations of an algorithm matched, and it found about a dozen cases that previous example-based testing and code inspections had not. Since then I've been evangelizing for it at our firm.

I am using Hypothesis as an integral part of my Python workshops. Testing is an integral part of Python programming and whilst unittest and, better, pytest can handle example-based testing, property-based testing is increasingly far more important than example-base testing, and Hypothesis fits the bill.

We've been using Hypothesis in a variety of client projects, from testing Django-related functionality to domain-specific calculations. It both speeds up and simplifies the testing process since there's so much less tedious and error-prone work to do in identifying edge cases. Test coverage is nice but test depth is even nicer, and it's much easier to get meaningful test depth using Hypothesis.

Hypothesis is being used as the engine for random object generation with my open source function fuzzer battle_tested which maps all behaviors of a function allowing you to minimize the chance of unexpected crashes when running code in production.

With how efficient Hypothesis is at generating the edge cases that cause unexpected behavior occur, battle_tested is able to map out the entire behavior of most functions in less than a few seconds.

Hypothesis truly is a masterpiece. I can't thank you enough for building it.

Just minutes after our first use of hypothesis we uncovered a subtle bug in one of our most used library.  Since then, we have increasingly used hypothesis to improve the quality of our testing in libraries and applications as well.

At Roboception GmbH I use Hypothesis to implement fully automated stateless and stateful reliability tests for the 3D sensor rc_visard and robotic software components .

Thank you very much for creating the (probably) most powerful property-based testing framework.

With a micro-service architecture, testing between services is made easy using Hypothesis in integration testing. Ensuring everything is running smoothly is vital to help maintain a secure network of Virtual Power Plants.

It allows us to find potential bugs and edge cases with relative ease and minimal overhead. As our architecture relies on services communicating effectively, Hypothesis allows us to strictly test for the kind of data which moves around our services, particularly our backend Python applications.

I know there are many more, because I keep finding out about new people I'd never even heard of using Hypothesis. If you're looking to way to give back to a tool you love, adding your name here only takes a moment and would really help a lot. As per instructions at the top, just send me a pull request and I'll add you to the list.

Some packages provide strategies directly:

• hypothesis-fspaths - strategy to generate filesystem paths.
• hypothesis-geojson - strategy to generate GeoJson.
• hypothesis-geometry - strategies to generate geometric objects.
• hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
• hypothesis-sqlalchemy - strategies to generate SQLAlchemy objects.
• hypothesis-ros - strategies to generate messages and parameters for the Robot Operating System.
• hypothesis-csv - strategy to generate CSV files.
• hypothesis-networkx - strategy to generate networkx graphs.
• hypothesis-bio - strategies for bioinformatics data, such as DNA, codons, FASTA, and FASTQ formats.
• hypothesis-rdkit - strategies to generate RDKit molecules and representations such as SMILES and mol blocks
• hypothesmith - strategy to generate syntatically-valid Python code.

Others provide a function to infer a strategy from some other schema:

• hypothesis-jsonschema - infer strategies from JSON schemas.
• lollipop-hypothesis - infer strategies from lollipop schemas.
• hypothesis-drf - infer strategies from a djangorestframework serialiser.
• hypothesis-graphql - infer strategies from GraphQL schemas.
• hypothesis-mongoengine - infer strategies from a mongoengine model.
• hypothesis-pb - infer strategies from Protocol Buffer schemas.

Or some other custom integration, such as a "hypothesis" entry point:

• deal is a design-by-contract library with built-in Hypothesis support.
• icontract-hypothesis infers strategies from icontract code contracts.
• Pandera schemas all have a .strategy() method, which returns a strategy for matching DataFrames.
• Pydantic automatically registers constrained types - so builds() and from_type() "just work" regardless of the underlying implementation.

Tyche (source) is a VSCode extension which provides live insights into your property-based tests, including the distribution of generated inputs and the resulting code coverage.  You can read the research paper here.

schemathesis is a tool for testing web applications built with Open API / Swagger specifications. It reads the schema and generates test cases which will ensure that the application is compliant with its schema. The application under test could be written in any language, the only thing you need is a valid API schema in a supported format. Includes CLI and convenient pytest integration. Powered by Hypothesis and hypothesis-jsonschema, inspired by the earlier swagger-conformance library.

Trio is an async framework with "an obsessive focus on usability and correctness", so naturally it works with Hypothesis! pytest-trio includes a custom hook that allows @given(...) to work with Trio-style async test functions, and hypothesis-trio includes stateful testing extensions to support concurrent programs.

pymtl3 is "an open-source Python-based hardware generation, simulation, and verification framework with multi-level hardware modeling support", which ships with Hypothesis integrations to check that all of those levels are equivalent, from function-level to register-transfer level and even to hardware.

libarchimedes makes it easy to use Hypothesis in the Hy language, a Lisp embedded in Python.

battle_tested is a fuzzing tool that will show you how your code can fail - by trying all kinds of inputs and reporting whatever happens.

pytest-subtesthack functions as a workaround for issue #377.

returns uses Hypothesis to verify that Higher Kinded Types correctly implement functor, applicative, monad, and other laws; allowing a declarative approach to be combined with traditional pythonic code.

icontract-hypothesis includes a ghostwriter for test files and IDE integrations such as icontract-hypothesis-vim, icontract-hypothesis-pycharm, and icontract-hypothesis-vscode - you can run a quick 'smoke test' with only a few keystrokes for any type-annotated function, even if it doesn't have any contracts!

See CONTRIBUTING.rst for more information.

New strategies can be added to Hypothesis, or published as an external package on PyPI - either is fine for most strategies. If in doubt, ask!

It's generally much easier to get things working outside, because there's more freedom to experiment and fewer requirements in stability and API style. We're happy to review and help with external packages as well as pull requests!

If you're thinking about writing an extension, please name it hypothesis-{something} - a standard prefix makes the community more visible and searching for extensions easier.  And make sure you use the Framework :: Hypothesis trove classifier!

On the other hand, being inside gets you access to some deeper implementation features (if you need them) and better long-term guarantees about maintenance. We particularly encourage pull requests for new composable primitives that make implementing other strategies easier, or for widely used types in the standard library. Strategies for other things are also welcome; anything with external dependencies just goes in hypothesis.extra.

Tools such as assertion helpers may also need to check whether the current test is using Hypothesis:

hypothesis.currently_in_test_context()

Return True if the calling code is currently running inside an @given or stateful test, False otherwise.

This is useful for third-party integrations and assertion helpers which may be called from traditional or property-based tests, but can only use assume() or target() in the latter case.

If you would like to ship Hypothesis strategies for a custom type - either as part of the upstream library, or as a third-party extension, there's a catch: from_type() only works after the corresponding call to register_type_strategy(), and you'll have the same problem with register_random().  This means that either

• you have to try importing Hypothesis to register the strategy when your library is imported, though that's only useful at test time, or
• the user has to call a 'register the strategies' helper that you provide before running their tests

Entry points are Python's standard way of automating the latter: when you register a "hypothesis" entry point in your setup.py, we'll import and run it automatically when hypothesis is imported.  Nothing happens unless Hypothesis is already in use, and it's totally seamless for downstream users!

Let's look at an example.  You start by adding a function somewhere in your package that does all the Hypothesis-related setup work:

# mymodule.py


class MyCustomType:
    def __init__(self, x: int):
        assert x >= 0, f"got {x}, but only positive numbers are allowed"
        self.x = x


def _hypothesis_setup_hook():
    import hypothesis.strategies as st

    st.register_type_strategy(MyCustomType, st.integers(min_value=0))

and then tell setuptools that this is your "hypothesis" entry point:

# setup.py

# You can list a module to import by dotted name
entry_points = {"hypothesis": ["_ = mymodule.a_submodule"]}

# Or name a specific function too, and Hypothesis will call it for you
entry_points = {"hypothesis": ["_ = mymodule:_hypothesis_setup_hook"]}

And that's all it takes!

HYPOTHESIS_NO_PLUGINS

If set, disables automatic loading of all hypothesis plugins. This is probably only useful for our own self-tests, but documented in case it might help narrow down any particularly weird bugs in complex environments.

Because pytest does not load plugins from entrypoints in any particular order, using the Hypothesis entrypoint may import your module before pytest-cov starts.  This is a known issue, but there are workarounds.

You can use coverage run pytest ... instead of pytest --cov ..., opting out of the pytest plugin entirely.  Alternatively, you can ensure that Hypothesis is loaded after coverage measurement is started by disabling the entrypoint, and loading our pytest plugin from your conftest.py instead:

echo "pytest_plugins = ['hypothesis.extra.pytestplugin']\n" > tests/conftest.py
pytest -p "no:hypothesispytest" ...

Another alternative, which we in fact use in our CI self-tests because it works well also with parallel tests, is to automatically start coverage early for all new processes if an environment variable is set. This automatic starting is set up by the PyPi package coverage_enable_subprocess.

This means all configuration must be done in .coveragerc, and not on the command line:

[run]
parallel = True
source = ...

Then, set the relevant environment variable and run normally:

python -m pip install coverage_enable_subprocess
export COVERAGE_PROCESS_START=$PATH/.coveragerc
pytest [-n auto] ...
coverage combine
coverage report

WARNING:

EXPERIMENTAL AND UNSTABLE.

The importable name of a backend which Hypothesis should use to generate primitive types.  We aim to support heuristic-random, solver-based, and fuzzing-based backends.

See issue #3086 for details, e.g. if you're interested in writing your own backend. (note that there is no stable interface for this; you'd be helping us work out what that should eventually look like, and we're likely to make regular breaking changes for some time to come)

Using the prototype crosshair-tool backend via hypothesis-crosshair, a solver-backed test might look something like:

from hypothesis import given, settings, strategies as st


@settings(backend="crosshair")  # pip install hypothesis[crosshair]
@given(st.integers())
def test_needs_solver(x):
    assert x != 123456789

This patch fixes an issue when realizing symbolics with our experimental backend setting.

Improves internal test coverage.

This release adds strategies for Django's ModelChoiceField and ModelMultipleChoiceField (issue #4010).

Thanks to Joshua Munn for this contribution.

Fixes and reinstates full coverage of internal tests, which was accidentally disabled in pull request #3935.

Closes issue #4003.

This release prevents a race condition inside internal cache implementation.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch improves our deduplication tracking across all strategies (pull request #4007). Hypothesis is now less likely to generate the same input twice.

Account for time spent in garbage collection during tests, to avoid flaky DeadlineExceeded errors as seen in issue #3975.

Also fixes overcounting of stateful run times, a minor observability bug dating to version 6.98.9 (pull request #3890).

This release migrates the shrinker to our new internal representation, called the IR layer (pull request #3962). This improves the shrinker's performance in the majority of cases. For example, on the Hypothesis test suite, shrinking is a median of 1.38x faster.

It is possible this release regresses performance while shrinking certain strategies. If you encounter strategies which reliably shrink more slowly than they used to (or shrink slowly at all), please open an issue!

You can read more about the IR layer at issue #3921.

This patch fixes one of our shrinking passes getting into a rare O(n) case instead of O(log(n)).

This patch fixes some introspection errors new in Python 3.11.9 and 3.13.0b1, for the Ghostwriter and from_type().

Internal developer documentation, no user-visible changes.

This patch improves our shrinking of unique collections, such as  dictionaries(), sets(), and lists() with unique=True.

This patch fixes a rare internal error when generating very large elements from strategies (issue #3874).

This patch fixes an overly strict internal type assertion.

This release improves our support for the annotated-types iterable GroupedMetadata protocol.  In order to treat the elements "as if they had been unpacked", if one such element is a SearchStrategy we now resolve to that strategy.  Previously, we treated this as an unknown filter predicate.

We expect this to be useful for libraries implementing custom metadata - instead of requiring downstream integration, they can implement the protocol and yield a lazily-created strategy.  Doing so only if Hypothesis is in sys.modules gives powerful integration with no runtime overhead or extra dependencies.

The from_model() function currently tries to create a strategy for AutoField fields if they don't have auto_created set to True.  The docs say it's supposed to skip all AutoField fields, so this patch updates the code to do what the docs say (issue #3978).

This patch adds some internal type annotations (issue #3074). Thanks to Andrew Sansom for his contribution!

This patch fixes a rare internal error when using integers() with a high max_examples setting (issue #3974).

This patch improves our internal caching logic. We don't expect it to result in any performance improvements (yet!).

This patch turns off a check in register_random() for possibly unreferenced RNG instances on the free-threaded build of CPython 3.13 because this check has a much higher false positive rate in the free-threaded build (issue #3965).

Thanks to Nathan Goldbaum for this patch.

This patch turns off a warning for functions decorated with typing.overload() and then composite(), although only in that order (issue #3970).

This patch fixes a significant slowdown when using the precondition() decorator in some cases, due to expensive repr formatting internally (issue #3963).

Explicitly cast numpy.finfo.smallest_normal to builtin float in preparation for the numpy==2.0 release (issue #3950)

This patch improve a rare error message for flaky tests (issue #3940).

The from_dtype() function no longer generates NaT ("not-a-time") values for the datetime64 or timedelta64 dtypes if passed allow_nan=False (issue #3943).

This patch includes the backend setting in the how_generated field of our observability output.

If you were running Python 3.13 (currently in alpha) with pytest-xdist and then attempted to pretty-print a lambda functions which was created using the eval() builtin, it would have raised an AssertionError. Now you'll get "lambda ...: <unknown>", as expected.

This release improves an internal invariant.

This patch fixes Hypothesis sometimes raising a Flaky error when generating collections of unique floats containing nan. See issue #3926 for more details.

This patch continues our work on refactoring the shrinker (issue #3921).

This patch continues our work on refactoring shrinker internals (issue #3921).

This release resolves PermissionError that come from creating databases on inaccessible paths.

This patch starts work on refactoring our shrinker internals. There is no user-visible change.

This patch fixes a longstanding performance problem in stateful testing (issue #3618), where state machines which generated a substantial amount of input for each step would hit the maximum amount of entropy and then fail with an Unsatisfiable error.

We now stop taking additional steps when we're approaching the entropy limit, which neatly resolves the problem without touching unaffected tests.

Fix regression caused by using PEP 696 default in TypeVar with Python 3.13.0a3.

This patch further improves the type annotations in hypothesis.extra.numpy.

Simplify the type annotation of column() and columns() by using PEP 696 to avoid overloading.

This patch implements type annotations for column().

This release adds the experimental and unstable backend setting.  See Alternative backends for Hypothesis for details.

This patch fixes issue #3900, a performance regression for arrays() due to the interaction of 6.98.12 - 2024-02-25 and 6.97.1 - 2024-01-27.

This patch improves the type annotations in hypothesis.extra.numpy, which makes inferred types more precise for both mypy and pyright, and fixes some strict-mode errors on the latter.

Thanks to Jonathan Plasse for reporting and fixing this in pull request #3889!

This patch paves the way for future shrinker improvements. There is no user-visible change.

This release adds support for the Array API's 2023.12 release via the api_version argument in make_strategies_namespace(). The API additions and modifications in the 2023.12 spec do not necessitate any changes in the Hypothesis strategies, hence there is no distinction between a 2022.12 and 2023.12 strategies namespace.

This patch adjusts the printing of bundle values to correspond with their names when using stateful testing.

This patch implements filter-rewriting for text() and binary() with the search(), match(), or fullmatch() method of a re.compile()d regex.

This patch implements filter-rewriting for most length filters on some additional collection types (issue #3795), and fixes several latent bugs where unsatisfiable or partially-infeasible rewrites could trigger internal errors.

This patch makes stateful testing somewhat less likely to get stuck when there are only a few possible rules.

This patch adds a note to errors which occur while drawing from a strategy, to make it easier to tell why your test failed in such cases.

This patch ensures that observability outputs include an informative repr for RuleBasedStateMachine stateful tests, along with more detailed timing information.

This patch improves the Ghostwriter for binary operators.

This patch improves import-detection in the Ghostwriter (issue #3884), particularly for from_type() and strategies from hypothesis.extra.*.

This patch clarifies the documentation on stateful testing (issue #3511).

This patch improves argument-to-json conversion for observability output.  Checking for a .to_json() method on the object before a few other options like dataclass support allows better user control of the process (issue #3880).

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch fixes an error when generating observability reports involving large (n > 1e308) integers.

This patch refactors some internals. There is no user-visible change.

This release improves our distribution of generated values for all strategies, by doing a better job of tracking which values we have generated before and avoiding generating them again.

For example, st.lists(st.integers()) previously generated ~5 each of [] [0] in 100 examples. In this release, each of [] and [0] are generated ~1-2 times each.

This release deprecates use of the global random number generator while drawing from a strategy, because this makes test cases less diverse and prevents us from reporting minimal counterexamples (issue #3810).

If you see this new warning, you can get a quick fix by using randoms(); or use more idiomatic strategies sampled_from(), floats(), integers(), and so on.

Note that the same problem applies to e.g. numpy.random, but for performance reasons we only check the stdlib random module - ignoring even other sources passed to register_random().

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch adds some observability information about how many times predicates in assume() or precondition() were satisfied, so that downstream tools can warn you if some were never satisfied by any test case.

This patch improves formatting and adds some cross-references to our docs.

Internal test refactoring.

This patch slightly changes how we replay examples from the database: if the behavior of the saved example has changed, we now keep running the test case instead of aborting at the size of the saved example.  While we know it's not the same example, we might as well continue running the test!

Because we now finish running a few more examples for affected tests, this might be a slight slowdown - but correspondingly more likely to find a bug.

We've also applied similar tricks to the target phase, where they are a pure performance improvement for affected tests.

Improves the performance of the arrays() strategy when generating unique values.

Changes the distribution of sampled_from() when sampling from a Flag. Previously, no-flags-set values would never be generated, and all-flags-set values would be unlikely for large enums. With this change, the distribution is more uniform in the number of flags set.

This patch slightly refactors some internals. There is no user-visible change.

This patch fixes a spurious warning about slow imports when HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY was set.

This patch refactors some more internals, continuing our work on supporting alternative backends (issue #3086). There is no user-visible change.

Fix a spurious warning seen when running pytest's test suite, caused by never realizing we got out of initialization due to imbalanced hook calls.

Warns when constructing a repr that is overly long. This can happen by accident if stringifying arbitrary strategies, and is expensive in time and memory. The associated deferring of these long strings in sampled_from() should also lead to improved performance.

This release adds the ability to pass any object to note(), instead of just strings. The pretty-printed representation of the object will be used.

See also issue #3843.

This release avoids creating a .hypothesis directory when using register_type_strategy() (issue #3836), and adds warnings for plugins which do so by other means or have other unintended side-effects.

This patch improves observability reports by moving timing information from metadata to a new timing key, and supporting conversion of additional argument types to json rather than string reprs via a .to_json() method (including e.g. Pandas dataframes).

Additionally, the too_slow health check will now report which strategies were slow, e.g. for strategies a, b, c, ...:

    count | fraction |    slowest draws (seconds)
a |    3  |     65%  |      --      --      --   0.357,  2.000
b |    8  |     16%  |   0.100,  0.100,  0.100,  0.111,  0.123
c |    3  |      8%  |      --      --   0.030,  0.050,  0.200
(skipped 2 rows of fast draws)

This patch refactors some internals, continuing our work on supporting alternative backends (issue #3086). There is no user-visible change.

The from_lark() strategy now accepts an alphabet= argument, which is passed through to from_regex(), so that you can e.g. constrain the generated strings to a particular codec.

In support of this feature, from_regex() will avoid generating optional parts which do not fit the alphabet.  For example, from_regex(r"abc|def", alphabet="abcd") was previously an error, and will now generate only 'abc'.  Cases where there are no valid strings remain an error.

This patch refactors some internals, continuing our work on supporting alternative backends (issue #3086). There is no user-visible change.

This patch adds a test statistics event when a generated example is rejected via assume.

This may also help with distinguishing gave_up examples in observability (issue #3827).

This introduces the rewriting of length filters on some collection strategies (issue #3791).

Thanks to Reagan Lee for implementing this feature!

If a test uses sampled_from() on a sequence of strategies, and raises a TypeError, we now add a note asking whether you meant to use one_of().

Thanks to Vince Reuter for suggesting and implementing this hint!

This patch registers explicit strategies for a handful of builtin types, motivated by improved introspection in PyPy 7.3.14 triggering existing internal warnings. Thanks to Carl Friedrich Bolz-Tereick for helping us work out what changed!

This patch fixes an error when writing observability reports without a pre-existing .hypothesis directory.

This patch adds a new environment variable HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVER, which turns on observability data collection without collecting code coverage data, which may be faster on Python 3.11 and earlier.

Thanks to Harrison Goldstein for reporting and fixing issue #3821.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch fixes a bug introduced in version 6.92.0, where using the data() strategy would fail to draw a dataclass() with a defaultdict field.  This was due to a bug in the standard library which was fixed in 3.12, so we've vendored the fix.

This release adds an experimental observability mode.  You can read the docs about it here.

This patch refactors some more internals, continuing our work on supporting alternative backends (issue #3086). There is no user-visible change.

This patch fixes an issue where builds() could not be used with attrs objects that defined private attributes (i.e. attributes with a leading underscore). See also issue #3791.

This patch also adds support more generally for using builds() with attrs' alias parameter, which was previously unsupported.

This patch increases the minimum required version of attrs to 22.2.0.

This release adds an optional payload argument to hypothesis.event(), so that you can clearly express the difference between the label and the value of an observation.  Test statistics will still summarize it as a string, but future observability options can preserve the distinction.

This patch supports assigning settings = settings(...) as a class attribute on a subclass of a .TestCase attribute of a RuleBasedStateMachine. Previously, this did nothing at all.

Thanks to Joey Tran for reporting these settings-related edge cases in stateful testing.

This release makes it an error to assign settings = settings(...) as a class attribute on a RuleBasedStateMachine. This has never had any effect, and it should be used as a decorator instead:

This patch refactors some internals.  There is no user-visible change, but we hope to improve performance and unlock support for alternative backends such as symbolic execution with crosshair in future (issue #3086).

Thanks to Liam DeVoe for this fantastic contribution!

This release teaches from_type() to handle constraints implied by the annotated-types package - as used by e.g. Pydantic. This is usually efficient, but falls back to filtering in a few remaining cases.

Thanks to Viicos for pull request #3780!

This patch adds a warning when @st.composite wraps a function annotated as returning a SearchStrategy, since this is usually an error (issue #3786).  The function should return a value, and the decorator will convert it to a function which returns a strategy.

This patch refactors from_type(typing.Tuple), allowing register_type_strategy() to take effect for tuples instead of being silently ignored (issue #3750).

Thanks to Nick Collins for reporting and extensive work on this issue.

This patch improves the speed of the explain phase on python 3.12+, by using the new sys.monitoring module to collect coverage, instead of sys.settrace.

Thanks to Liam DeVoe for pull request #3776!

This patch improves register_type_strategy() when used with tuple subclasses, by preventing them from being interpreted as generic and provided to strategies like st.from_type(Sequence[int]) (issue #3767).

This release allows strategy-generating functions registered with register_type_strategy() to conditionally not return a strategy, by returning NotImplemented (issue #3767).

When randoms() was called with use_true_randoms=False, calling r.sample([], 0) would result in an error, when it should have returned an empty sequence to agree with the normal behaviour of random.sample(). This fixes that discrepancy (issue #3765).

This patch ensures that the hypothesis codemod CLI will print a warning instead of stopping with an internal error if one of your files contains invalid syntax (issue #3759).

This patch makes some small changes to our NumPy integration to ensure forward compatibility.  Thanks to Mateusz Sokół for pull request #3761.

Fixes issue #3755, where an internal condition turns out to be reachable after all.

This release deprecates use of assume() and reject() outside of property-based tests, because these functions work by raising a special exception (issue #3743).  It also fixes some type annotations (issue #3753).

Hotfix for issue #3747, a bug in explain mode which is so rare that we missed it in six months of dogfooding.  Thanks to mygrad for discovering and promptly reporting this!

This patch improves the documentation of @example(...).xfail() by adding a note about PEP 614, similar to @example(...).via(), and adds a warning when a strategy generates a test case which seems identical to one provided by an xfailed example.

This release enables the explain phase by default.  We hope it helps you to understand why your failing tests have failed!

This patch switches some of our type annotations to use typing.Literal when only a few specific values are allowed, such as UUID or IP address versions.

This release deprecates the old whitelist/blacklist arguments to characters(), in favor of include/exclude arguments which more clearly describe their effects on the set of characters which can be generated.

You can use Hypothesis' codemods to automatically upgrade to the new argument names.  In a future version, the old names will start to raise a DeprecationWarning.

This patch automatically disables the differing_executors health check for methods which are also pytest parametrized tests, because those were mostly false alarms (issue #3733).

Building on recent releases, characters() now accepts _any_ codec=, not just "utf-8" and "ascii".

This includes standard codecs from the codecs module and their aliases, platform specific and user-registered codecs if they are available, and python-specific text encodings (but not text transforms or binary transforms).

This patch by Reagan Lee makes st.text(...).filter(str.isidentifier) return an efficient custom strategy (issue #3480).

The from_regex() strategy now takes an optional alphabet=characters(codec="utf-8") argument for unicode strings, like text().

This offers more and more-consistent control over the generated strings, removing previously-hard-coded limitations.  With fullmatch=False and alphabet=characters(), surrogate characters are now possible in leading and trailing text as well as the body of the match.  Negated character classes such as [^A-Z] or \S had a hard-coded exclusion of control characters and surrogate characters; now they permit anything in alphabet= consistent with the class, and control characters are permitted by default.

Add a health check that detects if the same test is executed several times by different executors. This can lead to difficult-to-debug problems such as issue #3446.

Pretty-printing of failing examples can now use functions registered with IPython.lib.pretty.for_type() or for_type_by_name(), as well as restoring compatibility with _repr_pretty_ callback methods which were accidentally broken in version 6.61.2 (issue #3721).

Adds a new codec= option in characters(), making it convenient to produce only characters which can be encoded as ascii or utf-8 bytestrings.

Support for other codecs will be added in a future release.

This patch updates our autoformatting tools, improving our code style without any API changes.

This patch enables and fixes many more of ruff's lint rules.

Fixes the error message for missing [cli] extra.

This patch ensures that we always close the download connection in GitHubArtifactDatabase.

We can now pretty-print combinations of zero enum.Flag values, like SomeFlag(0), which has never worked before.

This patch fixes pretty-printing of combinations of enum.Flag values, which was previously an error (issue #3709).

Improve shrinking of floats in narrow regions that don't cross an integer boundary. Closes issue #3357.

from_regex() now supports the atomic grouping ((?>...)) and possessive quantifier (*+, ++, ?+, {m,n}+) syntax added in Python 3.11.

Thanks to Cheuk Ting Ho for implementing this!

If the HYPOTHESIS_NO_PLUGINS environment variable is set, we'll avoid loading plugins such as the old Pydantic integration or HypoFuzz' CLI options.

This is probably only useful for our own self-tests, but documented in case it might help narrow down any particularly weird bugs in complex environments.

Fixes some lingering issues with inference of recursive types in from_type(). Closes issue #3525.

This release further improves our .patch-file support from version 6.75, skipping duplicates, tests which use data() (and don't support @example()), and various broken edge-cases.

Because libCST has released version 1.0 which uses the native parser by default, we no longer set the LIBCST_PARSER_TYPE=native environment variable.  If you are using an older version, you may need to upgrade or set this envvar for yourself.

This patch updates some internal code for selftests. There is no user-visible change.

This release drops support for Python 3.7, which reached end of life on 2023-06-27.

Fixes occasional recursion-limit-exceeded errors when validating deeply nested strategies. Closes: issue #3671

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

Improve the type rendered in from_type(), which improves the coverage of Ghostwriter.

We now test against Python 3.12 beta in CI, and this patch fixes some new deprecations.

This release changes register_type_strategy() for compatibility with PEP 585: we now store only a single strategy or resolver function which is used for both the builtin and the typing module version of each type (issue #3635).

If you previously relied on registering separate strategies for e.g. list vs typing.List, you may need to use explicit strategies rather than inferring them from types.

This release ensures that Ghostwriter does not use the deprecated aliases for the collections.abc classes in collections.

This patch improves Ghostwriter's use of qualified names for re-exported functions and classes, and avoids importing useless TypeVars.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

New input validation for recursive() will raise an error rather than hanging indefinitely if passed invalid max_leaves= arguments.

from_type() now handles numpy array types: np.typing.ArrayLike, np.typing.NDArray, and parameterized versions including np.ndarray[shape, elem_type].

Warn in from_type() if the inferred strategy has no variation (always returning default instances). Also handles numpy data types by calling from_dtype() on the corresponding dtype, thus ensuring proper variation for these types.

from_type() now works in cases where we use builds() to create an instance and the constructor has an argument which would lead to recursion.  Previously, this would raise an error if the argument had a default value.

Thanks to Joachim B Haga for reporting and fixing this problem.

In preparation for supporting JAX in hypothesis.extra.array_api, this release supports immutable arrays being generated via xps.arrays(). In particular, we internally removed an instance of in-place array modification, which isn't possible for an immutable array.

This release fixes some .patch-file bugs from version 6.75, and adds automatic support for writing @hypothesis.example() or @example() depending on the current style in your test file - defaulting to the latter.

Note that this feature requires libcst to be installed, and black is strongly recommended.  You can ensure you have the dependencies with pip install "hypothesis[cli,codemods]".

This patch continues the work started in pull request #3651 by adding ruff linter rules for pyflakes, flake8-comprehensions, and flake8-implicit-str-concat.

This patch updates our linter stack to use ruff, and fixes some previously-ignored lints.  Thanks to Christian Clauss for his careful review and pull request #3651!

Hypothesis will now record an event for more cases where data is marked invalid, including for exceeding the internal depth limit.

This patch fixes complex_numbers() accidentally invalidating itself when passed magnitude arguments for 32 and 64-bit widths, i.e. 16- and 32-bit floats, due to not internally down-casting numbers (issue #3573).

Improved the documentation regarding how to use GitHubArtifactDatabase and fixed a bug that occurred in repositories with no existing artifacts.

Thanks to Agustín Covarrubias for this contribution.

hypothesis.errors will now raise AttributeError when attempting to access an undefined attribute, rather than returning None.

Sick of adding @example()s by hand? Our Pytest plugin now writes .patch files to insert them for you, making this workflow easier than ever before.

Note that you'll need LibCST (via hypothesis[codemods]), and that @example().via() requires PEP 614 (Python 3.9 or later).

This patch provides better error messages for datetime- and timedelta-related invalid dtypes in our Pandas extra (issue #3518). Thanks to Nick Muoh at the PyCon Sprints!

This release adds support for nullable pandas dtypes in pandas() (issue #3604). Thanks to Cheuk Ting Ho for implementing this at the PyCon sprints!

This patch updates our minimum Numpy version to 1.16, and restores compatibility with versions before 1.20, which were broken by a mistake in Hypothesis 6.72.4 (issue #3625).

This release upgrades the explain phase (issue #3411).

• Following the first failure, Hypothesis will (usually) track which lines of code were executed by passing and failing examples, and report where they diverged - with some heuristics to drop unhelpful reports.  This is an existing feature, now upgraded and newly enabled by default.

• After shrinking to a minimal failing example, Hypothesis will try to find parts of the example -- e.g. separate args to @given() -- which can vary freely without changing the result of that minimal failing example. If the automated experiments run without finding a passing variation, we leave a comment in the final report:

  test_x_divided_by_y(
      x=0,  # or any other generated value
      y=0,
  )

Just remember that the lack of an explanation sometimes just means that Hypothesis couldn't efficiently find one, not that no explanation (or simpler failing example) exists.

This patch fixes type annotations for the arrays() strategy.  Thanks to Francesc Elies for pull request #3602.

This patch fixes a bug with from_type() with dict[tuple[int, int], str] (issue #3527).

This patch refactors our internals to facilitate an upcoming feature.

This patch fixes some documentation and prepares for future features.

This release deprecates Healthcheck.all(), and adds a codemod to automatically replace it with list(Healthcheck) (issue #3596).

This release adds GitHubArtifactDatabase, a new database backend that allows developers to access the examples found by a Github Actions CI job. This is particularly useful for workflows that involve continuous fuzzing, like HypoFuzz.

Thanks to Agustín Covarrubias for this feature!

This patch clarifies the reporting of time spent generating data. A simple arithmetic mean of the percentage of time spent can be misleading; reporting the actual time spent avoids misunderstandings.

Thanks to Andrea Reina for reporting and fixing issue #3598!

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This release adds an optional domains= parameter to the emails() strategy, and excludes the special-use .arpa domain from the default strategy (issue #3567).

Thanks to Jens Tröger for reporting and fixing this bug!

This release turns HealthCheck.return_value and HealthCheck.not_a_test_method into unconditional errors.  Passing them to suppress_health_check= is therefore a deprecated no-op. (issue #3568).  Thanks to Reagan Lee for the patch!

Separately, GraalPy can now run and pass most of the hypothesis test suite (issue #3587).

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch fixes missing imports of the re module, when ghostwriting tests which include compiled patterns or regex flags. Thanks to Jens Heinrich for reporting and promptly fixing this bug!

This patch adds some private hooks for use in research on Schemathesis (see our preprint here).

This release adds support for the Array API's 2022.12 release via the api_version argument in make_strategies_namespace(). Concretely this involves complex support in its existing strategies, plus an introduced xps.complex_dtypes() strategy.

Additionally this release now treats hypothesis.extra.array_api as stable, meaning breaking changes should only happen with major releases of Hypothesis.

This patch updates our autoformatting tools, improving our code style without any API changes.

This release allows for more precise generation of complex numbers using from_dtype(), by supporting the width, min_magnitude, and min_magnitude arguments (issue #3468).

Thanks to Felix Divo for this feature!

This patch fixes a rare RecursionError when pretty-printing a multi-line object without type-specific printer, which was passed to a function which returned the same object by .map() or builds() and thus recursed due to the new pretty reprs in Hypothesis 6.65.0 - 2023-01-24 (issue #3560).  Apologies to all those affected.

This makes from_dtype() pass through the parameter allow_subnormal for complex dtypes.

This release adds a width parameter to complex_numbers(), analogously to floats().

Thanks to Felix Divo for the new feature!

This patch fixes invalid annotations detected for the tests generated by Ghostwritter. It will now correctly generate Optional types with just one type argument and handle union expressions inside of type arguments correctly. Additionally, it now supports code with the from __future__ import annotations marker for Python 3.10 and newer.

This release improves the pretty-printing of enums in falsifying examples, so that they print as their full identifier rather than their repr.

Hypothesis now reports some failing inputs by showing the call which constructed an object, rather than the repr of the object.  This can be helpful when the default repr does not include all relevant details, and will unlock further improvements in a future version.

For now, we capture calls made via builds(), and via SearchStrategy.map().

The Ghostwritter will now include type annotations on tests for type-annotated code.  If you want to force this to happen (or not happen), pass a boolean to the new annotate= argument to the Python functions, or the --[no-]annotate CLI flag.

Thanks to Nicolas Ganz for this new feature!

range_indexes() now accepts a name= argument, to generate named pandas.RangeIndex objects.

Thanks to Sam Watts for this new feature!

This patch tweaks xps.arrays() internals to improve PyTorch compatibility. Specifically, torch.full() does not accept integers as the shape argument (n.b. technically "size" in torch), but such behaviour is expected in internal code, so we copy the torch module and patch in a working full() function.

A classic error when testing is to write a test function that can never fail, even on inputs that aren't allowed or manually provided.  By analogy to the design pattern of:

@pytest.mark.parametrize("arg", [
    ...,  # passing examples
    pytest.param(..., marks=[pytest.mark.xfail])  # expected-failing input
])

we now support @example(...).xfail(), with the same (optional) condition, reason, and raises arguments as pytest.mark.xfail().

Naturally you can also write .via(...).xfail(...), or .xfail(...).via(...), if you wish to note the provenance of expected-failing examples.

This patch teaches our enhanced get_type_hints() function to 'see through' partial application, allowing inference from type hints to work in a few more cases which aren't (yet!) supported by the standard-library version.

This patch improves our pretty-printing of failing examples, including some refactoring to prepare for exciting future features.

This patch brings our domains() and emails() strategies into compliance with RFC 5890 §2.3.1: we no longer generate parts-of-domains where the third and fourth characters are -- ("R-LDH labels"), though future versions may deliberately generate xn-- punycode labels.  Thanks to python-email-validator for the report!

This release improves our treatment of database keys, which based on (among other things) the source code of your test function.  We now post-process this source to ignore decorators, comments, trailing whitespace, and blank lines - so that you can add @example()s or make some small no-op edits to your code without preventing replay of any known failing or covering examples.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This release improves Hypothesis' ability to resolve forward references in type annotations. It fixes a bug that prevented builds() from being used with pydantic models that possess updated forward references. See issue #3519.

The @example(...) decorator now has a .via() method, which future tools will use to track automatically-added covering examples (issue #3506).

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch shifts hypothesis[lark] from depending on the old lark-parser package to the new lark package.  There are no code changes in Hypothesis, it's just that Lark got a new name on PyPI for version 1.0 onwards.

register_random() has used weakref since 6.27.1 - 2021-11-22, allowing the Random-compatible objects to be garbage-collected when there are no other references remaining in order to avoid memory leaks. We now raise an error or emit a warning when this seems likely to happen immediately.

The type annotation of register_random() was also widened so that structural subtypes of Random are accepted by static typecheckers.

This patch updates some internal type annotations and fixes a formatting bug in the explain phase reporting.

Hypothesis now raises an error if you passed a strategy as the alphabet= argument to text(), and it generated something which was not a length-one string.  This has never been supported, we're just adding explicit validation to catch cases like this StackOverflow question.

This patch updates some docs, and depends on exceptiongroup 1.0.0 final to avoid a bug in the previous version.

This patch teaches text() to rewrite a few more filter predicates (issue #3134).  You're unlikely to notice any change.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy, and fixes some incorrect examples in the docs for mutually_broadcastable_shapes().

This patch improves the error message when Hypothesis detects "flush to zero" mode for floating-point: we now report which package(s) enabled this, which can make debugging much easier.  See issue #3458 for details.

This release defines __bool__() on SearchStrategy. It always returns True, like before, but also emits a warning to help with cases where you intended to draw a value (issue #3463).

In preparation for future versions of the Array API standard, make_strategies_namespace() now accepts an optional api_version argument, which determines the version conformed to by the returned strategies namespace. If None, the version of the passed array module xp is inferred.

This release also introduces xps.real_dtypes(). This is currently equivalent to the existing xps.numeric_dtypes() strategy, but exists because the latter is expected to include complex numbers in the next version of the standard.

If multiple explicit examples (from @example()) raise a Skip exception, for consistency with generated examples we now re-raise the first instead of collecting them into an ExceptionGroup (issue #3453).

This patch updates our autoformatting tools, improving our code style without any API changes.

This patch fixes some type annotations for Python 3.9 and earlier (issue #3397), and teaches explain mode about certain locations it should not bother reporting (issue #3439).

This patch teaches the Ghostwriter an additional check for function and class locations that should make it use public APIs more often.

This patch fixes our workaround for a pytest bug where the inner exceptions in an ExceptionGroup are not displayed (issue #3430).

This patch makes FailedHealthCheck and DeadlineExceeded exceptions picklable, for compatibility with Django's parallel test runner (issue #3426).

Reporting of multiple failing examples now uses the PEP 654 ExceptionGroup type, which is provided by the exceptiongroup backport on Python 3.10 and earlier (issue #3175). hypothesis.errors.MultipleFailures is therefore deprecated.

Failing examples and other reports are now stored as PEP 678 exception notes, which ensures that they will always appear together with the traceback and other information about their respective error.

from_field() now supports UsernameField from django.contrib.auth.forms.

Thanks to Afonso Silva for reporting and working on issue #3417.

This patch improves the error message when you pass filenames to the hypothesis write CLI, which takes the name of a module or function (e.g. hypothesis write gzip or hypothesis write package.some_function rather than hypothesis write script.py).

Thanks to Ed Rogers for implementing this as part of the SciPy 2022 sprints!

This patch ensures that the warning for non-interactive .example() points to your code instead of Hypothesis internals (issue #3403).

Thanks to @jameslamb for this fix.

This patch makes integers() more likely to generate boundary values for large two-sided intervals (issue #2942).

This patch adds filter rewriting for math.isfinite(), math.isinf(), and math.isnan() on integers() or floats() (issue #2701).

Thanks to Sam Clamons at the SciPy Sprints!

This release adds the allow_subnormal argument to complex_numbers() by applying it to each of the real and imaginary parts separately. Closes issue #3390.

Thanks to Evan Tey for this fix.

Issue a deprecation warning if a function decorated with @composite does not draw any values (issue #3384).

Thanks to Grzegorz Zieba, Rodrigo Girão, and Thomas Ball for working on this at the EuroPython sprints!

This patch improves the error messages in @example() argument validation following the recent release of 6.49.1.

This release allows from_dtype() to generate Unicode strings which cannot be encoded in UTF-8, but are valid in Numpy arrays (which use UTF-32).

This logic will only be used with Numpy >= 1.19, because earlier versions have an issue which led us to revert Hypothesis 5.2 last time!

This patch fixes some inconsistency between argument handling for @example and @given (2706).

This release uses PEP 612 python:typing.ParamSpec (or the typing_extensions backport) to express the first-argument-removing behaviour of @st.composite and signature-preservation of functions() to IDEs, editor plugins, and static type checkers such as mypy.

hypothesis.event() now works for hashable objects which do not support weakrefs, such as integers and tuples.

This patch tidies up some internal introspection logic, which will improve support for positional-only arguments in a future release (issue #2706).

This release automatically rewrites some simple filters, such as floats().filter(lambda x: x >= 10) to the more efficient floats(min_value=10), based on the AST of the predicate.

We continue to recommend using the efficient form directly wherever possible, but this should be useful for e.g. pandera "Checks" where you already have a simple predicate and translating manually is really annoying.  See issue #2701 for details.

This release raises SkipTest for tests which never executed any examples, for example because the phases setting excluded the explicit, reuse, and generate phases.  This helps to avoid cases where broken tests appear to pass, because they didn't actually execute (issue #3328).

This patch fixes type annotations that had caused the signature of @given to be partially-unknown to type-checkers for Python versions before 3.10.

This patch fixes from_type() on Python 3.11, following python/cpython#93754.

This patch makes the too_slow health check more consistent with long deadline tests (issue #3367) and fixes an install issue under pipenv which was introduced in Hypothesis 6.47.2 (issue #3374).

We now use the PEP 654 ExceptionGroup type - provided by the exceptiongroup backport on older Pythons - to ensure that if multiple errors are raised in teardown, they will all propagate.

Our pretty-printer no longer sorts dictionary keys, since iteration order is stable in Python 3.7+ and this can affect reproducing examples (issue #3370). This PR was kindly supported by Ordina Pythoneers.

The Ghostwritter can now write tests for @classmethod or @staticmethod methods, in addition to the existing support for functions and other callables (issue #3318).  Thanks to Cheuk Ting Ho for the patch.

Mention hypothesis.strategies.timezones() in the documentation of hypothesis.strategies.datetimes() for completeness.

Thanks to George Macon for this addition.

This release contains some small improvements to our documentation. Thanks to Felix Divo for his contribution!

This patch by Adrian Garcia Badaracco adds type annotations to some private internals (issue #3074).

This patch by Phillip Schanely makes changes to the floats() strategy when min_value or max_value is present. Hypothesis will now be capable of generating every representable value in the bounds. You may notice that hypothesis is more likely to test values near boundaries, and values that are very close to zero.

These changes also support future integrations with symbolic execution tools and fuzzers (issue #3086).

This patch updates the type annotations for tuples() and one_of() so that type-checkers require its arguments to be positional-only, and so that it no longer fails under pyright-strict mode (see issue #3348). Additional changes are made to Hypothesis' internals improve pyright scans.

This patch by Cheuk Ting Ho adds support for PEP 655 Required and NotRequired as attributes of TypedDict in from_type() (issue #3339).

This patch fixes from_dtype() with long-precision floating-point datatypes (typecode g; see numpy:numpy.typename()).

This patch improves some error messages for custom signatures containing invalid parameter names (issue #3317).

This patch by Cheuk Ting Ho makes it an explicit error to call from_type() or register_type_strategy() with types that have no runtime instances (issue #3280).

This patch fixes silently dropping examples when the @example decorator is applied to itself (issue #3319).  This was always a weird pattern, but now it works.  Thanks to Ray Sogata, Keeri Tramm, and Kevin Khuong for working on this patch!

This patch fixes a rare bug where we could incorrectly treat empty as a type annotation, if the callable had an explicitly assigned __signature__.

This release adds an allow_nil argument to uuids(), which you can use to... generate the nil UUID.  Thanks to Shlok Gandhi for the patch!

This patch fixes some missing imports for certain Ghostwritten tests.  Thanks to Mel Seto for fixing issue #3316.

This patch teaches the Ghostwriter to recognize many more common argument names (issue #3311).

This patch fixes issue #3314, where Hypothesis would raise an internal error from domains() or (only on Windows) from timezones() in some rare circumstances where the installation was subtly broken.

Thanks to Munir Abdinur for this contribution.

This release fixes deprecation warnings about sre_compile and sre_parse imports and importlib.resources usage when running Hypothesis on Python 3.11.

Thanks to Florian Bruhin for this contribution.

This release updates xps.indices() by introducing an allow_newaxis argument, defaulting to False. If allow_newaxis=True, indices can be generated that add dimensions to arrays, which is achieved by the indexer containing None. This change is to support a specification change that expand dimensions via indexing (data-apis/array-api#408).

This release adds a names argument to indexes() and series(), so that you can create Pandas objects with specific or varied names.

Contributed by Sam Watts.

This patch updates the type annotations for @given so that type-checkers will warn on mixed positional and keyword arguments, as well as fixing issue #3296.

Fixed a type annotation for pyright --strict (issue #3287).

This patch makes it an explicit error to call register_type_strategy() with a Pydantic GenericModel and a callable, because GenericModel isn't actually a generic type at runtime and so you have to register each of the "parametrized versions" (actually subclasses!) manually.  See issue #2940 for more details.

This release makes it an explicit error to apply @pytest.fixture to a function which has already been decorated with @given().  Previously, pytest would convert your test to a fixture, and then never run it.

This patch fixes from_type() on a TypedDict with complex annotations, defined in a file using from __future__ import annotations. Thanks to Katelyn Gigante for identifying and fixing this bug!

The Hypothesis pytest plugin was not outputting valid xunit2 nodes when --junit-xml was specified. This has been broken since Pytest 5.4, which changed the internal API for adding nodes to the junit report.

This also fixes the issue when using hypothesis with --junit-xml and pytest-xdist where the junit xml report would not be xunit2 compatible. Now, when using with pytest-xdist, the junit report will just omit the <properties> node.

For more details, see this pytest issue, this pytest issue, and issue #1935

Thanks to Brandon Chinn for this bug fix!

This patch fixes pretty-printing of regular expressions in Python 3.11.0a7, and updates our vendored list of top-level domains,.

This release makes st.functions(pure=True) less noisy (issue #3253), and generally improves pretty-printing of functions.

This release changes the implementation of infer to be an alias for python:Ellipsis. E.g. @given(a=infer) is now equivalent to @given(a=...). Furthermore, @given(...) can now be specified so that @given will infer the strategies for all arguments of the decorated function based on its annotations.

This patch simplifies the repr of the strategies namespace returned in make_strategies_namespace(), e.g.

>>> from hypothesis.extra.array_api import make_strategies_namespace
>>> from numpy import array_api as xp
>>> xps = make_strategies_namespace(xp)
>>> xps
make_strategies_namespace(numpy.array_api)

Fixed from_type() support for PEP 604 union types, like int | None (issue #3255).

Fixed an internal error when given() was passed a lambda.

The Ghostwriter can now write tests which check that two or more functions are equivalent on valid inputs, or raise the same type of exception for invalid inputs (issue #3267).

This patch makes some quality-of-life improvements to the Ghostwriter: we guess the text() strategy for arguments named text (...obvious in hindsight, eh?); and improved the error message if you accidentally left in a nothing() or broke your rich install.

This patch improves our error detection and message when Hypothesis is run on a Python implementation without support for -0.0, which is required for the floats() strategy but can be disabled by unsafe compiler options (issue #3265).

This patch tweaks some internal formatting.  There is no user-visible change.

If the shrink phase is disabled, we now stop the generate phase as soon as an error is found regardless of the value of the report_multiple_examples setting, since that's probably what you wanted (issue #3244).

This patch clarifies rare error messages in builds() (issue #3225) and floats() (issue #3207).

This patch fixes a regression where the bound inner function (your_test.hypothesis.inner_test) would be invoked with positional arguments rather than passing them by name, which broke pytest-asyncio (issue #3245).

This release improves Hypothesis' handling of positional-only arguments, which are now allowed @st.composite strategies.

On Python 3.8 and later, the first arguments to builds() and from_model() are now natively positional-only. In cases which were already errors, the TypeError from incorrect usage will therefore be raises immediately when the function is called, rather than when the strategy object is used.

This release makes floats() error consistently when your floating-point hardware has been configured to violate IEEE-754 for subnormal numbers, instead of only when an internal assertion was tripped (issue #3092).

If this happens to you, passing allow_subnormal=False will suppress the explicit error.  However, we strongly recommend fixing the root cause by disabling global-effect unsafe-math compiler options instead, or at least consulting e.g. Simon Byrne's Beware of fast-math explainer first.

This patch fixes a bug in stateful testing, where returning a single value wrapped in multiple() would be printed such that the assigned variable was a tuple rather than the single element (issue #3236).

This patch fixes a warning under pytest 7 relating to our rich traceback display logic (issue #3223).

When distinguishing multiple errors, Hypothesis now looks at the inner exceptions of PEP 654 ExceptionGroups.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch fixes some deprecation warnings from pytest 7.0, along with some code formatting and docs updates.

This release disallows using python:typing.Final with from_type() and register_type_strategy().

Why? Because Final can only be used during class definition. We don't generate class attributes.

It also does not make sense as a runtime type on its own.

This patch fixes hypothesis write output highlighting with rich version 12.0 and later.

This release disallows using python:typing.ClassVar with from_type() and register_type_strategy().

Why? Because ClassVar can only be used during class definition. We don't generate class attributes.

It also does not make sense as a runtime type on its own.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch fixes issue #3169, an extremely rare bug which would trigger if an internal least-recently-reused cache dropped a newly added entry immediately after it was added.

This release fixes issue #3133 and issue #3144, where attempting to generate Pandas series of lists or sets would fail with confusing errors if you did not specify dtype=object.

This release disallows using python:typing.TypeAlias with from_type() and register_type_strategy().

Why? Because TypeAlias is not really a type, it is a tag for type checkers that some expression is a type alias, not something else.

It does not make sense for Hypothesis to resolve it as a strategy. References issue #2978.

This patch updates our autoformatting tools, improving our code style without any API changes.

This release drops support for Python 3.6, which reached end of life upstream on 2021-12-23.

This patch adds a temporary hook for a downstream tool, which is not part of the public API.

This release updates our copyright headers to use a general authorship statement and omit the year.

This patch makes the .example() method more representative of test-time data generation, albeit often at a substantial cost to readability (issue #3182).

This patch improves annotations on some of Hypothesis' internal functions, in order to deobfuscate the signatures of some strategies. In particular, strategies shared between hypothesis.extra.numpy and the hypothesis.extra.array_api extra will benefit from this patch.

This patch fix invariants display in stateful falsifying examples (issue #3185).

This patch updates xps.indices() so no flat indices are generated, i.e. generated indices will now always explicitly cover each axes of an array if no ellipsis is present. This is to be consistent with a specification change that dropped support for flat indexing (#272).

This release makes us compatible with Django 4.0, in particular by adding support for use of zoneinfo timezones (though we respect the new USE_DEPRECATED_PYTZ setting if you need it).

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This release adds an allow_subnormal argument to the floats() strategy, which can explicitly toggle the generation of subnormal floats (issue #3155). Disabling such generation is useful when testing flush-to-zero builds of libraries.

nps.from_dtype() and xps.from_dtype() can also accept the allow_subnormal argument, and xps.from_dtype() or xps.arrays() will disable subnormals by default if the array module xp is detected to flush-to-zero (like is typical with CuPy).

This patch fixes a bug in mutually_broadcastable_shapes(), which restricted the patterns of singleton dimensions that could be generated for dimensions that extended beyond base_shape (issue #3170).

This patch clarifies our pretty-printing of DataFrames (issue #3114).

This patch documents timezones() Windows-only requirement for the tzdata package, and ensures that pip install hypothesis[zoneinfo] will install the latest version.

This release teaches builds() to use deferred() when resolving unrecognised type hints, so that you can conveniently register strategies for recursive types with constraints on some arguments (issue #3026):

class RecursiveClass:
    def __init__(self, value: int, next_node: typing.Optional["SomeClass"]):
        assert value > 0
        self.value = value
        self.next_node = next_node


st.register_type_strategy(
    RecursiveClass, st.builds(RecursiveClass, value=st.integers(min_value=1))
)

This release fixes some internal calculations related to collection sizes (issue #3143).

This release modifies our pytest plugin, to avoid importing Hypothesis and therefore triggering Hypothesis' entry points for test suites where Hypothesis is installed but not actually used (issue #3140).

This release fixes issue #3080, where from_type() failed on unions containing PEP 585 builtin generic types (like list[int]) in Python 3.9 and later.

This patch makes the hypothesis codemod command somewhat faster.

This patch changes the backing datastructures of register_random() and a few internal caches to use weakref.WeakValueDictionary.  This reduces memory usage and may improve performance when registered Random instances are only used for a subset of your tests (issue #3131).

This release teaches Hypothesis' multiple-error reporting to format tracebacks using pytest or better-exceptions, if they are installed and enabled (issue #3116).

Did you know that of the 2⁶⁴ possible floating-point numbers, 2⁵³ of them are nan - and Python prints them all the same way?

While nans usually have all zeros in the sign bit and mantissa, this isn't always true, and 'signaling' nans might trap or error. To help distinguish such errors in e.g. CI logs, Hypothesis now prints -nan for negative nans, and adds a comment like # Saw 3 signaling NaNs if applicable.

This release adds special filtering logic to make a few special cases like s.map(lambda x: x) and lists().filter(len) more efficient (issue #2701).

This patch makes floats() generate "subnormal" floating point numbers more often, as these rare values can have strange interactions with unsafe compiler optimisations like -ffast-math (issue #2976).

This patch fixes a rare internal error in the datetimes() strategy, where the implementation of allow_imaginary=False crashed when checking a time during the skipped hour of a DST transition if the DST offset is negative - only true of Europe/Dublin, who we presume have their reasons - and the tzinfo object is a pytz timezone (which predates PEP 495).

This patch gives Hypothesis it's own internal Random instance, ensuring that test suites which reset the global random state don't induce weird correlations between property-based tests (issue #2135).

This patch updates documentation of note() (issue #3147).

This patch updates internal testing for the Array API extra to be consistent with new specification changes: sum() not accepting boolean arrays (#234), unique() split into separate functions (#275), and treating NaNs as distinct (#310). It has no user visible impact.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

This patch updates our vendored list of top-level domains, which is used by the provisional domains() strategy.

(did you know that gTLDs can be both added and removed?)

This patch adds an error for when shapes in xps.arrays() is not passed as either a valid shape or strategy.

This patch updates our formatting with shed.

This patch replaces external links to NumPy API docs with sphinx.ext.intersphinx cross-references. It is purely a documentation improvement.

This patch cleans up internal logic for xps.arrays(). There is no user-visible change.

This release follows pytest in considering SystemExit and GeneratorExit exceptions to be test failures, meaning that we will shink to minimal examples and check for flakiness even though they subclass BaseException directly (issue #2223).

KeyboardInterrupt continues to interrupt everything, and will be re-raised immediately.

This release adds LiveServerTestCase and StaticLiveServerTestCase for django test. Thanks to Ivan Tham for this feature!

This patch fixes some new linter warnings such as flake8-bugbear's B904 for explicit exception chaining, so tracebacks might be a bit nicer.

This release fixes None being inferred as the float64 dtype in from_dtype() and arrays() from the Array API extra.

This release fixes the type hint for the @given() decorator when decorating an async function (issue #3099).

This release improves Ghostwritten tests for builtins (issue #2977).

This release deprecates use of both min_dims > len(shape) and max_dims > len(shape) when allow_newaxis == False in basic_indices() (issue #3091).

This release improves the behaviour of builds() and from_type() in certain situations involving decorators (issue #2495 and issue #3029).

This release introduces strategies for array/tensor libraries adopting the Array API standard (issue #3037). They are available in the hypothesis.extra.array_api extra, and work much like the existing strategies for NumPy.

This patch fixes issue #961, where calling given() inline on a bound method would fail to handle the self argument correctly.

This release allows slices() to generate step=None, and fixes an off-by-one error where the start index could be equal to size. This works fine for all Python sequences and Numpy arrays, but is undefined behaviour in the Array API standard (see pull request #3065).

This release makes stateful testing more likely to tell you if you do something unexpected and unsupported:

• The return_value health check now applies to rule() and initialize() rules, if they don't have target bundles, as well as invariant().
• Using a consumes() bundle as a target is deprecated, and will be an error in a future version.

If existing code triggers these new checks, check for related bugs and misunderstandings - these patterns never had any effect.

This release teaches from_type() a neat trick: when resolving an python:typing.Annotated type, if one of the annotations is a strategy object we use that as the inferred strategy.  For example:

PositiveInt = Annotated[int, st.integers(min_value=1)]

If there are multiple strategies, we use the last outer-most annotation. See issue #2978 and pull request #3082 for discussion.

Requires Python 3.9 or later for get_type_hints(..., include_extras=False).

This patch makes unique arrays() much more efficient, especially when there are only a few valid elements - such as for eight-bit integers (issue #3066).

This patch fixes the repr of array_shapes().

This patch wraps some internal helper code in our proxies decorator to prevent mutations of method docstrings carrying over to other instances of the respective methods.

This patch moves some internal helper code in preparation for issue #3065. There is no user-visible change, unless you depended on undocumented internals.

This release adds type annotations to the stateful testing API.

Thanks to Ruben Opdebeeck for this contribution!

This release adds the DrawFn type as a reusable type hint for the draw argument of @composite functions.

Thanks to Ruben Opdebeeck for this contribution!

This release emits a more useful error message when @given() is applied to a coroutine function, i.e. one defined using async def (issue #3054).

This was previously only handled by the generic return_value health check, which doesn't direct you to use either a custom executor or a library such as pytest-trio or pytest-asyncio to handle it for you.

This patch fixes a regression in Hypothesis 6.14.8, where from_type() failed to resolve types which inherit from multiple parametrised generic types, affecting the returns package (issue #3060).

This patch ensures that registering a strategy for a subclass of a a parametrised generic type such as class Lines(Sequence[str]): will not "leak" into unrelated strategies such as st.from_type(Sequence[int]) (issue #2951). Unfortunately this fix requires PEP 560, meaning Python 3.7 or later.

This patch fixes issue #3050, where attrs classes could cause an internal error in the ghostwriter.

This patch improves the error message for issue #3016, where PEP 585 builtin generics with self-referential forward-reference strings cannot be resolved to a strategy by from_type().

This patch fixes hypothesis.strategies._internal.types.is_a_new_type. It was failing on Python 3.10.0b4, where NewType is a function.

This patch fixes from_type() and register_type_strategy() for python:typing.NewType on Python 3.10, which changed the underlying implementation (see bpo-44353 for details).

This patch updates our autoformatting tools, improving our code style without any API changes.

This patch ensures that we shorten tracebacks for tests which fail due to inconsistent data generation between runs (i.e. raise Flaky).

This patch updates some internal type annotations. There is no user-visible change.

The explain phase now requires shrinking to be enabled, and will be automatically skipped for deadline-exceeded errors.

This patch improves the tuples() strategy type annotations, to preserve the element types for up to length-five tuples (issue #3005).

As for one_of(), this is the best we can do before a planned extension to PEP 646 is released, hopefully in Python 3.11.

This patch teaches the Ghostwriter how to find custom ufuncs from any module that defines them, and that yaml.unsafe_load() does not undo yaml.safe_load().

This patch reduces the amount of internal code excluded from our test suite's code coverage checks.

There is no user-visible change.

This patch removes some old internal helper code that previously existed to make Python 2 compatibility easier.

There is no user-visible change.

This release adjusts some internal code to help make our test suite more reliable.

There is no user-visible change.

This patch cleans up some internal code related to filtering strategies.

There is no user-visible change.

This patch slightly improves the performance of some internal code for generating integers.

This patch fixes a bug in from_regex() that caused from_regex("", fullmatch=True) to unintentionally generate non-empty strings (issue #4982).

The only strings that completely match an empty regex pattern are empty strings.

This patch fixes a bug that caused integers() to shrink towards negative values instead of positive values in some cases.

This patch fixes rare cases where hypothesis write --binary-op could print reproducing instructions from the internal search for an identity element.

This patch removes some unnecessary intermediate list-comprehensions, using the latest versions of pyupgrade and shed.

This patch adds a .hypothesis property to invalid test functions, bringing them inline with valid tests and fixing a bug where pytest-asyncio would swallow the real error message and mistakenly raise a version incompatibility error.

Some of Hypothesis's numpy/pandas strategies use a fill argument to speed up generating large arrays, by generating a single fill value and sharing that value among many array slots instead of filling every single slot individually.

When no fill argument is provided, Hypothesis tries to detect whether it is OK to automatically use the elements argument as a fill strategy, so that it can still use the faster approach.

This patch fixes a bug that would cause that optimization to trigger in some cases where it isn't 100% guaranteed to be OK.

If this makes some of your numpy/pandas tests run more slowly, try adding an explicit fill argument to the relevant strategies to ensure that Hypothesis always uses the faster approach.

This patch strengthens some internal import-time consistency checks for the built-in strategies.

There is no user-visible change.

This release adds URL fragment generation to the urls() strategy (issue #2908). Thanks to Pax (R. Margret) for contributing this patch at the PyCon US Mentored Sprints!

This patch fixes issue #2964, where .map() and .filter() methods were omitted from the repr() of just() and sampled_from() strategies, since version 5.43.7.

This release automatically rewrites some simple filters, such as integers().filter(lambda x: x > 9) to the more efficient integers(min_value=10), based on the AST of the predicate.

We continue to recommend using the efficient form directly wherever possible, but this should be useful for e.g. pandera "Checks" where you already have a simple predicate and translating manually is really annoying.  See issue #2701 for ideas about floats and simple text strategies.

hypothesis.target() now returns the observation value, allowing it to be conveniently used inline in expressions such as assert target(abs(a - b)) < 0.1.

This patch fixes a deprecation warning if you're using recent versions of importlib-metadata (issue #2934), which we use to load third-party plugins such as Pydantic's integration. On older versions of importlib-metadata, there is no change and you don't need to upgrade.

This release teaches the Ghostwriter to read parameter types from Sphinx, Google, or Numpy-style structured docstrings, and improves some related heuristics about how to test scientific and numerical programs.

This release improves the Ghostwriter's handling of exceptions, by reading :raises ...: entries in function docstrings and ensuring that we don't suppresss the error raised by test assertions.

This patch updates our autoformatting tools, improving our code style without any API changes.

This release teaches from_type() how to see through python:typing.Annotated.  Thanks to Vytautas Strimaitis for reporting and fixing issue #2919!

If rich is installed, the hypothesis write command will use it to syntax-highlight the Ghostwritten code.

This patch improves an error message from builds() when from_type() would be more suitable (issue #2930).

This patch updates the type annotations for arrays() to reflect that shape: SearchStrategy[int] is supported.

This patch fixes from_type() with abstract types which have either required but non-type-annotated arguments to __init__, or where from_type() can handle some concrete subclasses but not others.

This patch teaches hypothesis write to check for possible roundtrips in several more cases, such as by looking for an inverse in the module which defines the function to test.

This patch adds a more helpful error message if you try to call sampled_from() on an Enum which has no members, but does have dataclass()-style annotations (issue #2923).

The fixed_dictionaries() strategy now preserves dict iteration order instead of sorting the keys.  This also affects the pretty-printing of keyword arguments to @given() (issue #2913).

This patch teaches hypothesis write to default to ghostwriting tests with --style=pytest only if pytest is installed, or --style=unittest otherwise.

This patch adds type annotations for the settings decorator, to avoid an error when running mypy in strict mode.

This patch improves the Ghostwriter's handling of strategies to generate various fiddly types including frozensets, keysviews, valuesviews, regex matches and patterns, and so on.

This patch fixes some internal typos.  There is no user-visible change.

This patch lays more groundwork for filter rewriting (issue #2701). There is no user-visible change... yet.

This release registers the remaining builtin types, and teaches from_type() to try resolving ForwardRef and Type references to built-in types.

This release teaches RuleBasedStateMachine to avoid checking invariant()s until all initialize() rules have been run.  You can enable checking of specific invariants for incompletely initialized machines by using @invariant(check_during_init=True) (issue #2868).

In previous versions, it was possible if awkward to implement this behaviour using precondition() and an auxiliary variable.

This patch improves the error message when from_type() fails to resolve a forward-reference inside a python:typing.Type such as Type["int"] (issue #2565).

This release makes it an explicit error to apply invariant() to a rule() or initialize() rule in stateful testing.  Such a combination had unclear semantics, especially in combination with precondition(), and was never meant to be allowed (issue #2681).

This release adds the explain phase, in which Hypothesis attempts to explain why your test failed by pointing to suspicious lines of code (i.e. those which were always, and only, run on failing inputs). We plan to include "generalising" failing examples in this phase in a future release (issue #2192).

This patch fixes issue #2794, where nesting deferred() strategies within recursive() strategies could trigger an internal assertion.  While it was always possible to get the same results from a more sensible strategy, the convoluted form now works too.

This patch fixes several problems with mypy when --no-implicit-reexport was activated in user projects.

Thanks to Nikita Sobolev for fixing issue #2884!

This patch fixes an exception that occurs when using type unions of the typing_extensions Literal backport on Python 3.6.

Thanks to Ben Anhalt for identifying and fixing this bug.

This release fixes stateful testing methods with multiple precondition() decorators.  Previously, only the outer-most precondition was checked (issue #2681).

This patch refactors some internals of RuleBasedStateMachine. There is no change to the public API or behaviour.

This patch moves some internal code, so that future work can avoid creating import cycles.  There is no user-visible change.

This patch enables register_type_strategy() for subclasses of python:typing.TypedDict.  Previously, from_type() would ignore the registered strategy (issue #2872).

Thanks to Ilya Lebedev for identifying and fixing this bug!

This release lays the groundwork for automatic rewriting of simple filters, for example converting integers().filter(lambda x: x > 9) to integers(min_value=10).

Note that this is not supported yet, and we will continue to recommend writing the efficient form directly wherever possible - predicate rewriting is provided mainly for the benefit of downstream libraries which would otherwise have to implement it for themselves (e.g. pandera and icontract-hypothesis).  See issue #2701 for details.

The Hypothesis pytest plugin now requires pytest version 4.6 or later. If the plugin detects an earlier version of pytest, it will automatically deactivate itself.

(4.6.x is the earliest pytest branch that still accepts community bugfixes.)

Hypothesis-based tests should continue to work in earlier versions of pytest, but enhanced integrations provided by the plugin (such as --hypothesis-show-statistics and other command-line flags) will no longer be available in obsolete pytest versions.

If you use pytest-html, Hypothesis now includes the summary statistics for each test in the HTML report, whether or not the --hypothesis-show-statistics argument was passed to show them in the command-line output.

This patch updates our automatic code formatting to use shed, which includes autoflake, black, isort, and pyupgrade (issue #2780).

This release teaches Hypothesis to distinguish between errors based on the __cause__ or __context__ of otherwise identical exceptions, which is particularly useful when internal errors can be wrapped by a library-specific or semantically appropriate exception such as:

try:
    do_the_thing(foo, timeout=10)
except Exception as err:
    raise FooError("Failed to do the thing") from err

Earlier versions of Hypothesis only see the FooError, while we can now distinguish a FooError raised because of e.g. an internal assertion from one raised because of a TimeoutExceeded exception.

This release prevents a race condition inside recursive() strategies. The race condition occurs when the same recursive() strategy is shared among tests that are running in multiple threads (issue #2717).

This patch improves the type annotations for one_of(), by adding overloads to handle up to five distinct arguments as Union before falling back to Any, as well as annotating the | (__or__) operator for strategies (issue #2765).

This release makes some small improvements to how filtered strategies work. It should improve the performance of shrinking filtered strategies, and may under some (probably rare) circumstances improve the diversity of generated examples.

This patch fixes an interaction where our test statistics handling made Pytest's --junit-xml output fail to validate against the strict xunit2 schema (issue #1975).

Welcome to the next major version of Hypothesis!

There are no new features here, as we release those in minor versions. Instead, 6.0 is a chance for us to remove deprecated features (many already converted into no-ops), and turn a variety of warnings into errors.

If you were running on the last version of Hypothesis 5.x without any Hypothesis deprecation warnings, this will be a very boring upgrade. In fact, nothing will change for you at all.

• Many functions now use PEP 3102 keyword-only arguments where passing positional arguments was deprecated since 5.5.
• hypothesis.extra.django.from_model() no longer accepts model as a keyword argument, where it could conflict with fields named "model".
• randoms() now defaults to use_true_random=False.
• complex_numbers() no longer accepts min_magnitude=None; either use min_magnitude=0 or just omit the argument.
• hypothesis.provisional.ip4_addr_strings and ip6_addr_strings are removed in favor of ip_addresses(v=...).map(str).
• register_type_strategy() no longer accepts generic types with type arguments, which were always pretty badly broken.
• Using function-scoped pytest fixtures is now a health-check error, instead of a warning.

TIP:

The hypothesis codemod command can automatically refactor your code, particularly to convert positional to keyword arguments where those are now required.

This release adds the function_scoped_fixture health check value, which can be used to suppress the existing warning that appears when @given is applied to a test that uses pytest function-scoped fixtures.

(This warning exists because function-scoped fixtures only run once per function, not once per example, which is usually unexpected and can cause subtle problems.)

When this warning becomes a health check error in a future release, suppressing it via Python warning settings will no longer be possible. In the rare case that once-per-function behaviour is intended, it will still be possible to use function_scoped_fixture to opt out of the health check error for specific tests.

This release adds hypothesis.currently_in_test_context(), which can be used to check whether the calling code is currently running inside an @given or stateful test.

This is most useful for third-party integrations and assertion helpers which may wish to use assume() or target(), without also requiring that the helper only be used from property-based tests (issue #2581).

This release upgrades the import logic for ghostwritten tests, handling many cases where imports would previously be missing or from unexpected locations.

This release upgrades from_type(), to infer strategies for type-annotated arguments even if they have defaults when it otherwise falls back to builds() (issue #2708).

This release adds the hypothesis[codemods] extra, which you can use to check for and automatically fix issues such as use of deprecated Hypothesis APIs (issue #2705).

This patch fixes from_type() with the typing_extensions Literal backport on Python 3.6.

This patch fixes issue #2722, where certain orderings of register_type_strategy(), ForwardRef, and from_type() could trigger an internal error.

This patch makes some strategies for collections with a uniqueness constraint much more efficient, including dictionaries(keys=sampled_from(...), values=..) and lists(tuples(sampled_from(...), ...), unique_by=lambda x: x[0]). (related to issue #2036)

This patch extends our faster special case for sampled_from() elements in unique lists() to account for chains of .map(...) and .filter(...) calls (issue #2036).

This patch improves the type annotations on assume() and @reproduce_failure().

This patch updates our copyright headers to include 2021.  Happy new year!

This change fixes a documentation error in the database setting.

The previous documentation suggested that callers could specify a database path string, or the special string ":memory:", but this setting has never actually allowed string arguments.

Permitted values are None, and instances of ExampleDatabase.

This patch fixes issue #2696, an internal error triggered when the @example decorator was used and the verbosity setting was quiet.

This patch improves the error message from the data_frames() strategy when both the rows and columns arguments are given, but there is a missing entry in rows and the corresponding column has no fill value (issue #2678).

This patch improves the error message if builds() is passed an Enum which cannot be called without arguments, to suggest using sampled_from() (issue #2693).

This release adds new timezones() and timezone_keys() strategies (issue #2630) based on the new python:zoneinfo module in Python 3.9.

pip install hypothesis[zoneinfo] will ensure that you have the appropriate backports installed if you need them.

This patch fixes an internal error in datetimes() with allow_imaginary=False where the timezones argument can generate tzinfo=None (issue #2662).

This patch teaches hypothesis.extra.django.from_field() to infer more efficient strategies by inspecting (not just filtering by) field validators for numeric and string fields (issue #1116).

This patch refactors hypothesis.settings to use type-annotated keyword arguments instead of **kwargs, which makes tab-completion much more useful - as well as type-checkers like mypy.

This patch teaches the magic() ghostwriter to recognise "en/de" function roundtrips other than the common encode/decode pattern, such as encrypt/decrypt or, encipher/decipher.

This patch adds a performance optimisation to avoid saving redundant seeds when using the .fuzz_one_input hook.

This patch fixes issue #2657, where passing unicode patterns compiled with python:re.IGNORECASE to from_regex() could trigger an internal error when casefolding a character creates a longer string (e.g. "\u0130".lower() -> "i\u0370").

This patch adds a final fallback clause to our plugin logic to fail with a warning rather than error on Python < 3.8 when neither the importlib_metadata (preferred) or setuptools (fallback) packages are available.

This patch fixes urls() strategy ensuring that ~ (tilde) is treated as one of the url-safe characters (issue #2658).

This patch improves our CLI help and documentation.

Hypothesis now shrinks examples where the error is raised while drawing from a strategy.  This makes complicated custom strategies much easier to debug, at the cost of a slowdown for use-cases where you catch and ignore such errors.

This release teaches from_type() how to handle ChainMap, Counter, Deque, Generator, Match, OrderedDict, Pattern, and Set (issue #2654).

from_type() now knows how to resolve PEP 585 parameterized standard collection types, which are new in Python 3.9 (issue #2629).

This patch fixes builds(), so that when passed infer for an argument with a non-Optional type annotation and a default value of None to build a class which defines an explicit __signature__ attribute, either None or that type may be generated.

This is unlikely to happen unless you are using pydantic (issue #2648).

This release improves our support for @st.composite on a python:classmethod or python:staticmethod (issue #2578).

This patch fixes from_type() with Iterable[T] (issue #2645).

This patch teaches the magic() ghostwriter to recognise that pairs of functions like rgb_to_hsv() and hsv_to_rgb() should roundtrip().

This patch improves builds() and from_type() support for explicitly defined __signature__ attributes, from version 5.8.3, to support generic types from the python:typing module.

Thanks to Rónán Carrigan for identifying and fixing this problem!

This patch fixes from_lark() with version 0.10.1+ of the lark-parser package.

This patch fixes some broken links in the lark extra documentation.

This release adds a new RedisExampleDatabase, along with the ReadOnlyDatabase and MultiplexedDatabase helpers, to support team workflows where failing examples can be seamlessly shared between everyone on the team - and your CI servers or buildbots.

This patch ensures that if the "hypothesis" entry point is callable, we call it after importing it.  You can still use non-callable entry points (like modules), which are only imported.

We also prefer importlib.metadata or the backport over pkg_resources, which makes import hypothesis around 200 milliseconds faster (issue #2571).

This patch adds some helpful suggestions to error messages you might see while learning to use the @example() decorator (issue #2611) or the one_of() strategy.

This release upgrades the from_dtype() strategy to pass optional **kwargs to the inferred strategy, and upgrades the arrays() strategy to accept an elements=kwargs dict to pass through to from_dtype().

arrays(floating_dtypes(), shape, elements={"min_value": -10, "max_value": 10}) is a particularly useful pattern, as it allows for any floating dtype without triggering the roundoff warning for smaller types or sacrificing variety for larger types (issue #2552).

This patch reformats our code with the latest black to take advantage of the support for magic trailing commas.

This release significantly improves the performance of Hypothesis's internal implementation of automaton learning. However this code does not run as part of the user-accessible API so this has no user-visible impact.

This patch ensures that, when the generate phases is disabled, we can replay up to max_examples examples from the database - which is very useful when using Hypothesis with a fuzzer.

Thanks to Afrida Tabassum for fixing issue #2585!

This patch changes some internal python:struct.Struct.format strings from bytes to str, to avoid python:BytesWarning when running python -bb.

Thanks to everyone involved in pytest-xdist issue 596, bpo-16349, bpo-21071, and bpo-41777 for their work on this - it was a remarkably subtle issue!

The target() function now accepts integers as well as floats.

This patch adds explicit Optional annotations to our public API, to better support users who run mypy with --strict or no_implicit_optional=True.

Thanks to Krzysztof Przybyła for bringing this to our attention and writing the patch!

This release drops support for Python 3.5, which reached end of life upstream on 2020-09-13.

This patch fixes a problem with builds() that was not able to generate valid data for annotated classes with constructors.

Thanks to Nikita Sobolev for fixing issue #2603!

This patch improves the error message from the hypothesis write command if black (required for the ghostwriter) is not installed.

Thanks to Nikita Sobolev for fixing issue #2604!

When reporting failing examples, or tried examples in verbose mode, Hypothesis now identifies which were from @example(...) explicit examples.

This patch contains some internal refactoring. Thanks to Felix Sheldon for fixing issue #2516!

An array drawn from arrays() will own its own memory; previously most arrays returned by this strategy were views.

builds() will use the __signature__ attribute of the target, if it exists, to retrieve type hints. Previously python:typing.get_type_hints(), was used by default. If argument names varied between the __annotations__ and __signature__, they would not be supplied to the target.

This was particularly an issue for pydantic models which use an alias generator.

This patch makes the ghostwriter much more robust when passed unusual modules.

• improved support for non-resolvable type annotations
• magic() can now write equivalent() tests
• running magic() on modules where some names in __all__ are undefined skips such names, instead of raising an error
• magic() now knows to skip mocks
• improved handling of import-time errors found by the ghostwriter CLI

register_type_strategy() now supports python:typing.TypeVar, which was previously hard-coded, and allows a variety of types to be generated for an unconstrained TypeVar instead of just text().

Thanks again to Nikita Sobolev for all your work on advanced types!

This release fixes some hard to trigger bugs in Hypothesis's automata learning code. This code is only run as part of the Hypothesis build process, and not for user code, so this release has no user visible impact.

This patch adds type annotations to the hypothesis.database module.  There is no runtime change, but your typechecker might notice.

This patch tracks some additional information in Hypothesis internals, and has no user-visible impact.

This release fixes a bug in some Hypothesis internal support code for learning automata. This mostly doesn't have any user visible impact, although it slightly affects the learned shrink passes so shrinking may be subtly different.

This release adds support for Hypothesis integration via setuptools entry points, which allows for smoother integration of third-party Hypothesis extensions and external libraries. Unless you're publishing a library with Hypothesis integration, you'll probably only ever use this indirectly!

from_type() can now resolve TypeVar instances when the bound is a ForwardRef, so long as that name is in fact defined in the same module as the typevar (no TYPE_CHECKING tricks, sorry). This feature requires Python 3.7 or later.

Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!

This patch adds two new ghostwriters to test binary operations, like python:operator.add(), and Numpy ufuncs and gufuncs like np.matmul().

This release improves the performance of some methods in Hypothesis's internal automaton library. These are currently only lightly used by user code, but this may result in slightly faster shrinking.

register_type_strategy() no longer accepts parametrised user-defined generic types, because the resolution logic was quite badly broken (issue #2537).

Instead of registering a strategy for e.g. MyCollection[int], you should register a function for MyCollection and inspect the type parameters within that function.

Thanks to Nikita Sobolev for the bug report, design assistance, and pull request to implement this feature!

Tired of writing tests?  Or new to Hypothesis and not sure where to start?

This release is for you!  With our new Ghostwriter functions and hypothesis write ... command-line interface, you can stop writing tests entirely... or take the source code Hypothesis writes for you as a starting point.

This has been in the works for months, from issue #2118 to versions 5.18.3, 5.23.5, and 5.23.5 - particular thanks to the many people who reviewed pull requests or commented on demos, and to Timothy Crosley's hypothesis-auto project for inspiration.

This patch adds yet more internal functions to support a new feature we're working on, like version 5.18.3 and version 5.23.6.  We promise it's worth the wait!

This release fixes a small internal bug in Hypothesis's internal automaton library. Fortunately this bug was currently impossible to hit in user facing code, so this has no user visible impact.

This release improves shrink quality by allowing Hypothesis to automatically learn new shrink passes for difficult to shrink tests.

The automatic learning is not currently accessible in user code (it still needs significant work on robustness and performance before it is ready for that), but this release includes learned passes that should improve shrinking quality for tests which use any of the text(), floats(), datetimes(), emails(), and complex_numbers() strategies.

This patch updates some docstrings, without changing runtime behaviour.

The functions() strategy has a new argument pure=True, which ensures that the same return value is generated for identical calls to the generated function (issue #2538).

Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!

This release removes a number of Hypothesis's internal "shrink passes" - transformations it makes to a generated test case during shrinking - which appeared to be redundant with other transformations.

It is unlikely that you will see much impact from this. If you do, it will likely show up as a change in shrinking performance (probably slower, maybe faster), or possibly in worse shrunk results. If you encounter the latter, please let us know.

This release fixes a bug in some internal Hypothesis support code. It has no user visible impact.

This release improves the quality of shrunk test cases in some special cases. Specifically, it should get shrinking unstuck in some scenarios which require simultaneously changing two parts of the generated test case.

This release improves the performance of some internal support code. It has no user visible impact, as that code is not currently run during normal Hypothesis operation.

This release adds a heuristic to detect when shrinking has finished despite the fact that there are many more possible transformations to try. This will be particularly useful for tests where the minimum failing test case is very large despite there being many smaller test cases possible, where it is likely to speed up shrinking dramatically.

In some cases it is likely that this will result in worse shrunk test cases. In those cases rerunning the test will result in further shrinking.

This release makes some performance improvements to shrinking. They should only be noticeable for tests that are currently particularly slow to shrink.

This patch adds some more internal functions to support a new feature we're working on, like version 5.18.3. There is still no user-visible change... yet.

This release makes some changes to internal support code that is not currently used in production Hypothesis. It has no user visible effect at present.

This release improves shrinking quality in some special cases.

This release fixes issue #2507, where lazy evaluation meant that the values drawn from a sampled_from() strategy could depend on mutations of the sampled sequence that happened after the strategy was constructed.

This patch fixes issue #2462, a bug in our handling of unittest.TestCase.subTest(). Thanks to Israel Fruchter for fixing this at the EuroPython sprints!

This release improves the behaviour of the characters() strategy when shrinking, by changing which characters are considered smallest to prefer more "normal" ascii characters where available.

The default print_blob setting is now smarter. It defaults to True in CI and False for local development.

Thanks to Hugo van Kemenade for implementing this feature at the EuroPython sprints!

The slices() strategy can now generate slices for empty sequences, slices with negative start and stop indices (from the end of the sequence), and step=None in place of step=1.

Thanks to Sangarshanan for implementing this feature at the EuroPython sprints!

This release ensures that tests which raise RecursionError are not reported as flaky simply because we run them from different initial stack depths (issue #2494).

This release improves the performance of the sample method on objects obtained from randoms() when use_true_random=False. This should mostly only be noticeable when the sample size is a large fraction of the population size, but may also help avoid health check failures in some other cases.

This release makes some internal changes for testing purposes and should have no user visible effect.

This release fixes a small caching bug in Hypothesis internals that may under some circumstances have resulted in a less diverse set of test cases being generated than was intended.

Fixing this problem revealed some performance problems that could occur during targeted property based testing, so this release also fixes those. Targeted property-based testing should now be significantly faster in some cases, but this may be at the cost of reduced effectiveness.

This patch updates our formatting to use isort 5. There is no user-visible change.

The basic_indices() strategy can now generate bare indexers in place of length-one tuples. Thanks to Andrea for this patch!

This patch removes an internal use of distutils in order to avoid this setuptools warning for some users.

This patch contains a small internal refactoring with no user-visible impact.

Thanks to Andrea for writing this at the SciPy 2020 Sprints!

This release slightly improves shrinking behaviour. This should mainly only impact stateful tests, but may have some minor positive impact on shrinking collections (lists, sets, etc).

This release improves the randoms() strategy by adding support for Random instances where Hypothesis generates the random values rather than having them be "truly" random.

This patch adds some internal functions to support a new feature we're working on.  There is no user-visible change... yet.

This patch improves our docs for the derandomize setting.

This release consists of some internal refactoring to the shrinker in preparation for future work. It has no user visible impact.

This release teaches Hypothesis to shorten tracebacks for explicit examples, as we already do for generated examples, so that you can focus on your code rather than ours.

If you have multiple failing explicit examples, they will now all be reported. To report only the first failure, you can use the report_multiple_bugs=False setting as for generated examples.

This patch adds strategy inference for the Literal, NewType, Type, DefaultDict, and TypedDict types from the typing_extensions backport on PyPI.

This patch precomputes some of the setup logic for our external fuzzer integration and sets deadline=None in fuzzing mode, saving around 150us on each iteration.

This is around two-thirds the runtime to fuzz an empty test with @given(st.none()), and nice to have even as a much smaller fraction of the runtime for non-trivial tests.

This patch fixes an internal error when warning about the use of function-scoped fixtures for parametrised tests where the parametrised value contained a % character. Thanks to Bryant for reporting and fixing this bug!

If you pass a python:list or python:tuple where a strategy was expected, the error message now mentions sampled_from() as an example strategy.

Thanks to the enthusiastic participants in the PyCon Mentored Sprints who suggested adding this hint.

functions() can now infer the appropriate returns strategy if you pass a like function with a return-type annotation.  Before, omitting the returns argument would generate functions that always returned None.

Fix from_type() with generic types under Python 3.9.

This patch fixes an error that happens when multiple threads create new strategies.

Passing min_magnitude=None to complex_numbers() is now deprecated - you can explicitly pass min_magnitude=0, or omit the argument entirely.

This patch fixes an internal error in from_type() for python:typing.NamedTuple in Python 3.9.  Thanks to Michel Salim for reporting and fixing issue #2427!

This release upgrades the test statistics available via the --hypothesis-show-statistics option to include separate information on each of the phases (issue #1555).

This patch teaches the from_type() internals to return slightly more efficient strategies for some generic sets and mappings.

This patch adds a # noqa comment for flake8 3.8.0, which disagrees with mypy about how to write the type of ....