2.27.3.4 Test results

Running a test sets various status properties in the current test-runner. This can be examined by a custom test-runner, or (more rarely) in a test-suite.

Result kind

Running a test may yield one of the following status symbols:

pass

The passed, as expected.

fail

The test failed (and was not expected to).

xfail

The test failed and was expected to.

xpass

The test passed, but was expected to fail.

skip

The test was skipped.

Function: test-result-kind

Function: test-result-kind runner

Return one of the above result codes from the most recent tests. Return #f if no tests have been run yet. If we’ve started on a new test, but don’t have a result yet, then the result kind is xfail is the test is expected to fail, skip is the test is supposed to be skipped, or #f otherwise.

Function: test-passed?

Function: test-passed? runner

True if the value of:

(test-result-kind)
(test-result-kind runner)

is one of pass or xpass. This is a convenient shorthand that might be useful in a test suite to only run certain tests if the previous test passed.

Test result properties

A test runner also maintains a set of more detailed “result properties” associated with the current or most recent test. (I.e. the properties of the most recent test are available as long as a new test hasn’t started.) Each property has a name (a symbol) and a value (any value). Some properties are standard or set by the implementation; implementations can add more.

Function: test-result-ref runner pname

Function: test-result-ref runner pname default

Return the property value associated with the pname property name. If there is no value associated with pname return default, or #f if default isn’t specified.

Function: test-result-set! runner pname value

Set the property value associated with the pname property name to value. Usually implementation code should call this function, but it may be useful for a custom test-runner to add extra properties.

Function: test-result-remove runner pname

Remove the property with the name pname.

Function: test-result-clear runner

Remove all result properties. The implementation automatically calls test-result-clear at the start of a test-assert and similar procedures.

Function: test-result-alist runner

Return an association list of the current result properties. It is unspecified if the result shares state with the test-runner. The result should not be modified, on the other hand the result may be implicitly modified by future test-result-set! or test-result-remove calls. However, a test-result-clear does not modify the returned alist. Thus you can “archive” result objects from previous runs.

Standard result properties

The set of available result properties is implementation–specific. However, it is suggested that the following might be provided:

result-kind

The result kind, as defined previously. This is the only mandatory result property.

(test-result-kind runner)
≡ (test-result-ref runner 'result-kind)

source-file

source-line

If known, the location of test statements (such as test-assert) in test suite source code.

source-form

The source form, if meaningful and known.

expected-value

The expected non–error result, if meaningful and known.

expected-error

The error–type specified in a test-error, if it meaningful and known.

actual-value

The actual non–error result value, if meaningful and known.

actual-error

The error value, if an error was signalled and it is known. The actual error value is implementation–defined.