The doctest24 module is accessible via the formencode.util module.
In simplest use, end each module M to be tested with:
Then running the module as a script will cause the examples in the docstrings to get executed and verified:
python M.py
This won't display anything unless an example fails, in which case the failing example(s) and the cause(s) of the failure(s) are printed to stdout (why not stderr? because stderr is a lame hack <0.2 wink>), and the final line of output is "Test failed.".
Run it with the -v switch instead:
python M.py -v
and a detailed report of all examples tried is printed to stdout, along with assorted summaries at the end.
You can force verbose mode by passing "verbose=True" to testmod, or prohibit it by passing "verbose=False". In either of those cases, sys.argv is not examined by testmod.
There are a variety of other ways to run doctests, including integration with the unittest framework, and support for running non-Python text files containing doctests. There are also many ways to override parts of doctest's default behaviors. See the Library Reference Manual for details.
prefix, base -> true iff name prefix + "." + base is "private".
Prefix may be an empty string, and base does not contain a period. Prefix is ignored (although functions you write conforming to this protocol may make use of it). Return true iff base begins with an (at least one) underscore, but does not both begin and end with (at least) two underscores.
>>> is_private("a.b", "my_func")
False
>>> is_private("____", "_my_func")
True
>>> is_private("someclass", "__init__")
False
>>> is_private("sometypo", "__init_")
True
>>> is_private("x.y.z", "_")
True
>>> is_private("_x.y.z", "__")
False
>>> is_private("", "") # senseless but consistent
False
Test examples in docstrings in functions and classes reachable from module m (or the current module if m is not supplied), starting with m.__doc__. Unless isprivate is specified, private names are not skipped.
Also test examples reachable from dict m.__test__ if it exists and is not None. m.__test__ maps names to functions, classes and strings; function and class docstrings are tested even if the name is private; strings are tested directly, as if they were docstrings.
Return (#failures, #tests).
See doctest.__doc__ for an overview.
Optional keyword arg "name" gives the name of the module; by default use m.__name__.
Optional keyword arg "globs" gives a dict to be used as the globals when executing examples; by default, use m.__dict__. A copy of this dict is actually used for each docstring, so that each docstring's examples start with a clean slate.
Optional keyword arg "extraglobs" gives a dictionary that should be merged into the globals that are used to execute examples. By default, no extra globals are used. This is new in 2.4.
Optional keyword arg "verbose" prints lots of stuff if true, prints only failures if false; by default, it's true iff "-v" is in sys.argv.
Optional keyword arg "report" prints a summary at the end when true, else prints nothing at the end. In verbose mode, the summary is detailed, else very brief (in fact, empty if all tests passed).
Optional keyword arg "optionflags" or's together module constants, and defaults to 0. This is new in 2.3. Possible values (see the docs for details):
DONT_ACCEPT_TRUE_FOR_1 DONT_ACCEPT_BLANKLINE NORMALIZE_WHITESPACE ELLIPSIS IGNORE_EXCEPTION_DETAIL REPORT_UDIFF REPORT_CDIFF REPORT_NDIFF REPORT_ONLY_FIRST_FAILURE
Optional keyword arg "raise_on_error" raises an exception on the first unexpected exception or failure. This allows failures to be post-mortem debugged.
Deprecated in Python 2.4: Optional keyword arg "isprivate" specifies a function used to determine whether a name is private. The default function is treat all functions as public. Optionally, "isprivate" can be set to doctest.is_private to skip over functions marked as private using the underscore naming convention; see its docs for details.
Advanced tomfoolery: testmod runs methods of a local instance of class doctest.Tester, then merges the results into (or creates) global Tester instance doctest.master. Methods of doctest.master can be called directly too, if you want to do something unusual. Passing report=0 to testmod is especially useful then, to delay displaying a summary. Invoke doctest.master.summarize(verbose) when you're done fiddling.
Test examples in the given file. Return (#failures, #tests).
Optional keyword arg "module_relative" specifies how filenames should be interpreted:
- If "module_relative" is True (the default), then "filename"
specifies a module-relative path. By default, this path is relative to the calling module's directory; but if the "package" argument is specified, then it is relative to that package. To ensure os-independence, "filename" should use "/" characters to separate path segments, and should not be an absolute path (i.e., it may not begin with "/").
If "module_relative" is False, then "filename" specifies an os-specific path. The path may be absolute or relative (to the current working directory).
Optional keyword arg "name" gives the name of the test; by default use the file's basename.
Optional keyword argument "package" is a Python package or the name of a Python package whose directory should be used as the base directory for a module relative filename. If no package is specified, then the calling module's directory is used as the base directory for module relative filenames. It is an error to specify "package" if "module_relative" is False.
Optional keyword arg "globs" gives a dict to be used as the globals when executing examples; by default, use {}. A copy of this dict is actually used for each docstring, so that each docstring's examples start with a clean slate.
Optional keyword arg "extraglobs" gives a dictionary that should be merged into the globals that are used to execute examples. By default, no extra globals are used.
Optional keyword arg "verbose" prints lots of stuff if true, prints only failures if false; by default, it's true iff "-v" is in sys.argv.
Optional keyword arg "report" prints a summary at the end when true, else prints nothing at the end. In verbose mode, the summary is detailed, else very brief (in fact, empty if all tests passed).
Optional keyword arg "optionflags" or's together module constants, and defaults to 0. Possible values (see the docs for details):
DONT_ACCEPT_TRUE_FOR_1 DONT_ACCEPT_BLANKLINE NORMALIZE_WHITESPACE ELLIPSIS IGNORE_EXCEPTION_DETAIL REPORT_UDIFF REPORT_CDIFF REPORT_NDIFF REPORT_ONLY_FIRST_FAILURE
Optional keyword arg "raise_on_error" raises an exception on the first unexpected exception or failure. This allows failures to be post-mortem debugged.
Optional keyword arg "parser" specifies a DocTestParser (or subclass) that should be used to extract tests from the files.
Advanced tomfoolery: testmod runs methods of a local instance of class doctest.Tester, then merges the results into (or creates) global Tester instance doctest.master. Methods of doctest.master can be called directly too, if you want to do something unusual. Passing report=0 to testmod is especially useful then, to delay displaying a summary. Invoke doctest.master.summarize(verbose) when you're done fiddling.
Test examples in the given object's docstring ( f ), using globs as globals. Optional argument name is used in failure messages. If the optional argument verbose is true, then generate output even if there are no failures.
compileflags gives the set of flags that should be used by the Python compiler when running the examples. If not specified, then it will default to the set of future-import flags that apply to globs .
Optional keyword arg optionflags specifies options for the testing and output. See the documentation for testmod for more information.
Sets the unittest option flags.
The old flag is returned so that a runner could restore the old value if it wished to:
>>> old = _unittest_reportflags >>> set_unittest_reportflags(REPORT_NDIFF | ... REPORT_ONLY_FIRST_FAILURE) == old True>>> import doctest >>> doctest._unittest_reportflags == (REPORT_NDIFF | ... REPORT_ONLY_FIRST_FAILURE) True
Only reporting flags can be set:
>>> set_unittest_reportflags(ELLIPSIS) Traceback (most recent call last): ... ValueError: ('Only reporting flags allowed', 8)>>> set_unittest_reportflags(old) == (REPORT_NDIFF | ... REPORT_ONLY_FIRST_FAILURE) True
Convert doctest tests for a module to a unittest test suite.
This converts each documentation string in a module that contains doctest tests to a unittest test case. If any of the tests in a doc string fail, then the test case fails. An exception is raised showing the name of the file containing the test and a (sometimes approximate) line number.
The module argument provides the module to be tested. The argument can be either a module or a module name.
If no argument is given, the calling module is used.
A number of options may be provided as keyword arguments:
A unittest suite for one or more doctest files.
The path to each doctest file is given as a string; the interpretation of that string depends on the keyword argument "module_relative".
A number of options may be provided as keyword arguments:
If "module_relative" is True, then the given file paths are interpreted as os-independent module-relative paths. By default, these paths are relative to the calling module's directory; but if the "package" argument is specified, then they are relative to that package. To ensure os-independence, "filename" should use "/" characters to separate path segments, and may not be an absolute path (i.e., it may not begin with "/").
If "module_relative" is False, then the given file paths are interpreted as os-specific paths. These paths may be absolute or relative (to the current working directory).
Extract script from text with examples.
Converts text with examples to a Python script. Example input is converted to regular code. Example output and all other words are converted to comments:
>>> text = ''' ... Here are examples of simple math. ... ... Python has super accurate integer addition ... ... >>> 2 + 2 ... 5 ... ... And very friendly error messages: ... ... >>> 1/0 ... To Infinity ... And ... Beyond ... ... You can use logic if you want: ... ... >>> if 0: ... ... blah ... ... blah ... ... ... ... Ho hum ... '''
>>> print script_from_examples(text) # Here are examples of simple math. # # Python has super accurate integer addition # 2 + 2 # Expected: ## 5 # # And very friendly error messages: # 1/0 # Expected: ## To Infinity ## And ## Beyond # # You can use logic if you want: # if 0: blah blah # # Ho hum
Extract the test sources from a doctest docstring as a script.
Provide the module (or dotted name of the module) containing the test to be debugged and the name (within the module) of the object with the doc string with tests to be debugged.
Debug a single doctest docstring.
Provide the module (or dotted name of the module) containing the test to be debugged and the name (within the module) of the object with the docstring with tests to be debugged.
A single doctest example, consisting of source code and expected output. Example defines the following attributes:
The constructor adds a newline if needed.
This class contains 1 member.
A collection of doctest examples that should be run in a single namespace. Each DocTest defines the following attributes:
This class contains 2 members.
A class used to parse strings containing doctest examples.
This class contains 3 members.
A class used to extract the DocTests that are relevant to a given object, from its docstring and the docstrings of its contained objects. Doctests can currently be extracted from the following object types: modules, functions, classes, methods, staticmethods, classmethods, and properties.
This class contains 2 members.
A class used to run DocTest test cases, and accumulate statistics. The run method is used to process a single DocTest case. It returns a tuple (f, t) , where t is the number of test cases tried, and f is the number of test cases that failed.
>>> tests = DocTestFinder().find(_TestClass)
>>> runner = DocTestRunner(verbose=False)
>>> for test in tests:
... print runner.run(test)
(0, 2)
(0, 1)
(0, 2)
(0, 2)
The summarize method prints a summary of all the test cases that have been run by the runner, and returns an aggregated (f, t) tuple:
>>> runner.summarize(verbose=1) 4 items passed all tests: 2 tests in _TestClass 2 tests in _TestClass.__init__ 2 tests in _TestClass.get 1 tests in _TestClass.square 7 tests in 4 items. 7 passed and 0 failed. Test passed. (0, 7)
The aggregated number of tried examples and failed examples is also available via the tries and failures attributes:
>>> runner.tries 7 >>> runner.failures 0
The comparison between expected outputs and actual outputs is done by an OutputChecker . This comparison may be customized with a number of option flags; see the documentation for testmod for more information. If the option flags are insufficient, then the comparison may also be customized by passing a subclass of OutputChecker to the constructor.
The test runner's display output can be controlled in two ways. First, an output function ( out) can be passed to `TestRunner.run ; this function will be called with strings that should be displayed. It defaults to sys.stdout.write . If capturing the output is not sufficient, then the display output can be also customized by subclassing DocTestRunner, and overriding the methods report_start , report_success , report_unexpected_exception , and report_failure .
This class contains 9 members.
A class used to check the whether the actual output from a doctest example matches the expected output. OutputChecker defines two methods: check_output , which compares a given pair of outputs, and returns true if they match; and output_difference , which returns a string describing the differences between two outputs.
This class contains 2 members.
A DocTest example has failed in debugging mode.
The exception instance has variables:
This class contains 2 members.
A DocTest example has encountered an unexpected exception
The exception instance has variables:
This class contains 2 members.
Run doc tests but raise an exception as soon as there is a failure.
If an unexpected exception occurs, an UnexpectedException is raised. It contains the test, the example, and the original exception:
>>> runner = DebugRunner(verbose=False) >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42', ... {}, 'foo', 'foo.py', 0) >>> try: ... runner.run(test) ... except UnexpectedException, failure: ... pass>>> failure.test is test True>>> failure.example.want '42\n'>>> exc_info = failure.exc_info >>> raise exc_info[0], exc_info[1], exc_info[2] Traceback (most recent call last): ... KeyError
We wrap the original exception to give the calling application access to the test and example information.
If the output doesn't match, then a DocTestFailure is raised:
>>> test = DocTestParser().get_doctest(''' ... >>> x = 1 ... >>> x ... 2 ... ''', {}, 'foo', 'foo.py', 0)>>> try: ... runner.run(test) ... except DocTestFailure, failure: ... pass
DocTestFailure objects provide access to the test:
>>> failure.test is test True
As well as to the example:
>>> failure.example.want '2\n'
and the actual output:
>>> failure.got '1\n'
If a failure or error occurs, the globals are left intact:
>>> del test.globs['__builtins__'] >>> test.globs {'x': 1}>>> test = DocTestParser().get_doctest(''' ... >>> x = 2 ... >>> raise KeyError ... ''', {}, 'foo', 'foo.py', 0)>>> runner.run(test) Traceback (most recent call last): ... UnexpectedException: <DocTest foo from foo.py:0 (2 examples)>>>> del test.globs['__builtins__'] >>> test.globs {'x': 2}
But the globals are cleared if there is no error:
>>> test = DocTestParser().get_doctest(''' ... >>> x = 2 ... ''', {}, 'foo', 'foo.py', 0)>>> runner.run(test) (0, 1)>>> test.globs {}
This class contains 9 members.
See the source for more information.