test.conduct

import test.conduct

test does not re-export this module, so it is reached only by importing it directly.

Running a directory of test files, each in a process of its own.

import test

test.conduct('tests')

zuri test is this call behind a command line, with every option below as a flag, so a project gets a test runner without writing one. Call conduct() directly when the run wants to be under the project’s own control.

One process per file

A test file is an ordinary Zuri script, and each one is run as one:

• A file that loops forever is killed, and the rest still run.
• A file that segfaults, or calls os.exit() halfway through, is reported as a crashed file rather than taking the run with it.
• Global state, a module loaded with a side effect, a signal handler, a changed working directory: none of it leaks from one file into the next.
• timeout is enforced by kill, so it holds for a file that never returns.

The index is not a test file

index.zu is left out of discovery. A directory handed to zuri run runs its index.zu, so a suite that wants to start itself puts one there:

# tests/index.zu
import os
import test

test.conduct(os.dir_name(__file__))

$ zuri run tests

The script that is currently running is left out as well, whatever it is called, so no arrangement of files makes a run start itself again.

What a test file has to do

Nothing special. It declares its tests. The conductor sets ZURI_TEST_CONDUCTED in the child, and the run switches to a machine-readable reporter when it sees it, so the same file is equally runnable on its own.

A file that declares no tests at all, or exits before it can report, is called out rather than silently counting as zero tests.

Ordering

Files are reported in the order they were discovered, whatever order they finish in, so a run is reproducible with jobs turned up.

Constants

CONDUCTED

test.conduct.CONDUCTED = 'ZURI_TEST_CONDUCTED'

Functions

discover()

test.conduct.discover(directory: string, options: ?dict)

Every test file under directory, in the order they will run.

index.zu is never one of them. A directory handed to zuri run runs its index.zu, so that file is the thing that starts a suite rather than a part of one, and conducting a directory from inside its own index is the shape this is written for. Pass an ignore list of your own when a file of that name really is a test.

The script that is currently running is left out too, whatever it is called, so no arrangement of files can make a run start itself again.

Parameters

• directory (string)
• options (?dict) — match, ignore and recursive, as conduct() takes them.

Returns — list: paths, relative to directory.

conduct()

test.conduct.conduct(directory: ?string, options: ?dict)

Runs every test file under directory, each in its own process, and reports them together.

import test

test.conduct('tests', { timeout: 30000, jobs: 4 })

The process ends with status 1 when anything failed, through os.set_exit_code(), so nothing has to be added for CI. The returned dictionary is there for a caller that wants to look at the run rather than just be told about it.

• match: filename patterns to run, ['*.zu'] by default. * and ? are the only wildcards.
  • ignore: patterns for files and directories to skip, ['_*', '.*', 'index.zu'] by default, which is what keeps helper files, __snapshots__ and the index that starts the suite out of it.
  • recursive: whether to look in subdirectories. true.
  • jobs: how many files to run at once. 1, because a test file that binds a port or writes a fixture was probably not written to share the machine. Reporting order is by discovery whatever this is set to.
  • timeout: milliseconds to give one file before killing it. 0, meaning no limit.
  • bail: stop after this many failing files. 0 never stops.
  • env: extra environment variables for every child.

Parameters

• directory (?string) — 'tests' by default.
• options (?dict)

Returns — dict: { files, totals, duration, exit_code, ok }, where files is a list of FileResult.

Raises TestSetupError when directory does not exist.

Note: index.zu is never one of the files it runs, and neither is the script that called it, so a tests/index.zu conducting its own directory works.

is_conducted()

test.conduct.is_conducted() -> bool

Whether this process was started by the conductor.

run() uses it to switch to the machine-readable reporter. A test file has no reason to care.

Returns bool

Classes

FileResult

class test.conduct.FileResult

What became of one file.

Fields

Constructor

test.conduct.FileResult(path)

FileResult.ok()

test.conduct.FileResult.ok()

Whether the file ran cleanly and everything in it passed. @returns bool