test.reporter

import test.reporter

test lifts part of this module out to its own top level; each name below is shown with the path that reaches it. Anything still spelled test.reporter.* needs import test.reporter.

How a run is shown.

The runner knows nothing about output. It walks the tree and calls the methods below as things happen, and a reporter decides what, if anything, that looks like. Swapping one for another changes the report and nothing else.

The built-in reporters

test.run({ reporter: 'dot' }) picks one by name, and test.run({ reporter: MyReporter() }) uses your own.

Writing one

Subclass Reporter and override what you care about. Every method has a do-nothing default, so a reporter that only reacts to failures is six lines:

import test.reporter { Reporter }

class Quiet < Reporter {
  test_finished(test) {
    if test.status == 'failed' {
      echo test.full_name()
    }
  }
}

Constants

PREFIX

test.reporter.PREFIX = '@@zuri-test@@'

What marks a line of ndjson output as protocol rather than as something the test file happened to print.

Functions

create()

test.reporter.create(name: string, options: ?dict) -> Reporter

Builds a reporter by name.

Parameters

• name (string) — one of spec, dot, tap, junit, json, ndjson, silent.
• options (?dict) — passed to the reporter’s constructor.

Returns Reporter

Raises TestSetupError when there is no such reporter.

Classes

Reporter

class test.Reporter

The interface the runner talks to, with every method doing nothing.

Subclass it and override what you want; there is no need to call parent() from an override, since nothing here does anything.

Reporter.run_started()

test.Reporter.run_started(plan)

Called once, before anything runs.

Parameters

• plan (dict) — { cases, suites, files, options }, where cases is how many tests are going to run.

Reporter.suite_started()

test.Reporter.suite_started(suite)

Called when a suite is entered, before any of its tests.

Parameters

• suite (Suite)

Reporter.suite_finished()

test.Reporter.suite_finished(suite)

Called when a suite is left, after every one of its tests.

Parameters

• suite (Suite)

Reporter.test_started()

test.Reporter.test_started(test)

Called before a test’s body runs. Not called for a test that is skipped or is a todo.

Parameters

• test (Case)

Reporter.test_finished()

test.Reporter.test_finished(test)

Called once per test, whatever became of it, including the ones that never ran.

Parameters

• test (Case)

Reporter.run_finished()

test.Reporter.run_finished(summary, root)

Called once, after everything.

Parameters

• summary (Summary)
• root (Suite)

Spec

class test.reporter.Spec < Reporter

The default: the suite tree, one line per test, and every failure written out in full at the end.

Constructor

test.reporter.Spec(options)

Parameters

• options (?dict) — slow is the millisecond threshold past which a test’s time is highlighted (300 by default, 0 to never highlight). verbose prints a passing test’s captured output as well as a failing one’s.

Spec.run_started()

test.reporter.Spec.run_started(plan)

Spec.test_finished()

test.reporter.Spec.test_finished(test)

Spec.run_finished()

test.reporter.Spec.run_finished(summary, root)

Dot

class test.reporter.Dot < Reporter

One character per test, wrapped to the terminal, then the same failure detail and summary the spec reporter gives.

For a run long enough that a line per test is more scrolling than information.

Constructor

test.reporter.Dot(options)

Dot.run_started()

test.reporter.Dot.run_started(plan)

Dot.test_finished()

test.reporter.Dot.test_finished(test)

Dot.run_finished()

test.reporter.Dot.run_finished(summary, root)

Tap

class test.reporter.Tap < Reporter

TAP version 14: one ok/not ok line per test, with failure detail in a YAML block underneath.

The output carries no colour, whatever the terminal supports, since something else is going to parse it.

Constructor

test.reporter.Tap(options)

Tap.run_started()

test.reporter.Tap.run_started(plan)

Tap.test_finished()

test.reporter.Tap.test_finished(test)

Tap.run_finished()

test.reporter.Tap.run_finished(summary, root)

Junit

class test.reporter.Junit < Reporter

JUnit XML, which is the format nearly every CI system knows how to turn into a test report page.

Each describe becomes a <testsuite> and each test a <testcase>. Written to standard output; redirect it to the file your CI is configured to collect.

Constructor

test.reporter.Junit(options)

Junit.run_finished()

test.reporter.Junit.run_finished(summary, root)

Json

class test.reporter.Json < Reporter

One JSON document at the end, holding the summary and every test.

For a tool of your own that wants the whole run rather than a stream of it.

Constructor

test.reporter.Json(options)

Json.test_finished()

test.reporter.Json.test_finished(test)

Json.run_finished()

test.reporter.Json.run_finished(summary, root)

Ndjson

class test.reporter.Ndjson < Reporter

One JSON object per line, emitted as each thing happens.

This is what a test file writes when it is being run by the conductor in a process of its own: the parent reads the stream as it arrives rather than waiting for the child to finish.

Every line carries the prefix below, so anything else the file prints can be told apart from the protocol and passed through.

Constructor

test.reporter.Ndjson(options)

Ndjson.run_started()

test.reporter.Ndjson.run_started(plan)

Ndjson.test_finished()

test.reporter.Ndjson.test_finished(test)

Ndjson.run_finished()

test.reporter.Ndjson.run_finished(summary, root)

Silent

class test.reporter.Silent < Reporter

No output at all, for a caller reading the returned Summary instead.

Constructor

test.reporter.Silent(options)