[#](#) Inquest

## Overview

Inquest provides a database of test results which can be used as part of developer workflow to ensure/check things like:

* No commits without having had a test failure, test fixed cycle.
* No commits without new tests being added.
* What tests have failed since the last commit (to run just a subset).
* What tests are currently failing and need work.

Test results are inserted using subunit (and thus anything that can output subunit or be converted into a subunit stream can be accepted).

Inquest started as a Rust port of the Python [testrepository](https://github.com/testing-cabal/testrepository) tool, originally written by Robert Collins. It has since grown its own features and on-disk format.

**Features:**
- Fast, native binary with no Python runtime required
- SQLite-backed repository with rich run metadata
- Timeout support (per-test, overall, no-output)
- Auto-detection of project type
- Optional backward compatibility with testrepository's on-disk format
- Configuration via `inquest.toml` (TOML) or `.testr.conf` (legacy INI)

## Installation

Build from source:

```sh
cargo build --release
```

The binary will be available at `target/release/inq`.

## Quick Start

Create a config file `inquest.toml`:

```toml
test_command = "cargo test $IDOPTION"
test_id_option = "--test $IDFILE"
```

Create a repository:

```sh
inq init
```

Run tests and load results:

```sh
inq run
```

Query the repository:

```sh
inq stats
inq last
inq failing
inq slowest
```

Re-run only failing tests:

```sh
inq run --failing
```

List available tests:

```sh
inq list-tests
```

Delete a repository:

```sh
rm -rf .inquest
```

## Commands

### `inq auto`

Auto-detect the project type and generate an `inquest.toml` configuration file.

### `inq init`

Initialize a new test repository in the current directory. Creates a `.inquest/` directory with a SQLite metadata database and runs directory.

### `inq run`

Execute tests using the command defined in `inquest.toml` and load the results into the repository.

Options:
- `--failing`: Run only the tests that failed in the last run
- `--partial`: Partial run mode (update failing tests additively)
- `--force-init`: Create repository if it doesn't exist
- `--auto`: Auto-detect project type and generate `inquest.toml` if missing
- `--load-list <FILE>`: Run only tests listed in the file (one per line)
- `-j, --parallel <N>`: Run tests in parallel across N workers
- `--until-failure`: Run tests repeatedly until they fail
- `--max-iterations <N>`: Maximum iterations for `--until-failure`
- `--isolated`: Run each test in a separate process
- `--subunit`: Show output as a subunit stream
- `--all-output`: Show output from all tests (not just failures)
- `--test-timeout <TIMEOUT>`: Per-test timeout (`5m`, `auto`, or `disabled`)
- `--max-duration <DURATION>`: Overall run timeout
- `--no-output-timeout <DURATION>`: Kill test process if no output for this duration
- `TESTFILTER`: Regex patterns to filter which tests to run
- `-- TESTARGS`: Additional arguments passed through to the test command

### `inq ci`

Run tests with output formatted for a CI provider. Wraps `inq run` with
CI-friendly defaults and emits structured output that GitHub Actions or
GitLab CI can render natively: foldable log groups per failing test,
inline annotations on the changed lines, and a markdown job summary.

Options:
- `--format <auto|github|gitlab|plain>`: Output format. `auto` (default)
  detects `GITHUB_ACTIONS=true` or `GITLAB_CI=true` from the environment
  and falls back to `plain` otherwise.
- `--retry <N>`: Re-run failing tests up to N times. Tests that pass on
  retry are reported as warnings (the run still succeeds) rather than
  errors. Opt-in; defaults to `0` so the first run is the honest signal.
- `--order <ORDER>`: Test ordering. Defaults to `frequent-failing-first`
  when the repository has run history (so known-bad tests fail fast),
  otherwise `discovery`. Accepts the same vocabulary as `inq run --order`.
- `-j, --parallel <N>`: Parallel worker count.
- `--test-timeout <TIMEOUT>` / `--max-duration <DURATION>`: Timeout
  overrides, same syntax as `inq run`.
- `-s, --starting-with <PREFIX>`: Restrict to tests with the given prefix.
- `TESTFILTER`: Regex patterns to filter which tests to run.
- `-- TESTARGS`: Additional arguments passed through to the test command.

See [README.gh-action.md](README.gh-action.md) for a workflow example
using the bundled `jelmer/inquest` composite action, plus how to cache
the `.inquest` directory across runs so the ordering heuristics have
history to work with.

### `inq rerun`

Re-run exactly the tests of a previous run, in the same order, forwarding the same `--`-style test arguments captured in the run's metadata. Without an argument, re-runs the latest run.

```sh
inq rerun 7    # repeat run 7
inq rerun -1   # repeat the latest run
```

### `inq load`

Load test results from stdin in subunit format.

```sh
my-test-runner | inq load
```

Options:
- `--partial`: Partial run mode (update failing tests additively)
- `--force-init`: Create repository if it doesn't exist

### `inq last`

Show results from a test run, including timestamp, counts, and list of failing tests.

Options:
- `-r, --run <ID>`: Run ID to show (defaults to latest; supports negative indices like `-1`, `-2`)
- `--subunit`: Output results as a subunit stream
- `--no-output`: Don't show test output/tracebacks for failed tests

### `inq info`

Show detailed information about a test run, including git commit, command, duration, exit code, and concurrency.

Options:
- `-r, --run <ID>`: Run ID to show (defaults to latest; supports negative indices)

### `inq failing`

Show only the failing tests from the last run. Exits with code 0 if no failures, 1 if there are failures.

Options:
- `--list`: List test IDs only, one per line (for scripting)
- `--subunit`: Output results as a subunit stream

### `inq running`

Show currently in-progress test runs. Tails the in-progress subunit stream
on disk to report live test counts (passed/failed), percent complete versus
the historical test count, elapsed wall-clock time, and an ETA derived from
each test's historical duration. Uses lock files to track active runs.

### `inq config`

Print the resolved effective configuration. Combines values from the config file
(`inquest.toml`, `.inquest.toml`, or `.testr.conf`), CLI overrides, and built-in
defaults. Each value is annotated with its source (`[config]`, `[cli]`, or
`[default]`), making it easy to see what `inq run` would actually use.

Options accept the same overrides as `inq run`: `--test-timeout`, `--max-duration`,
`--no-output-timeout`, `--order`, and `-j/--parallel`.

### `inq stats`

Show repository statistics including total test runs, latest run details, and total tests executed.

### `inq slowest`

Show the slowest tests from the last run, sorted by duration.

Options:
- `-n, --count <N>`: Number of tests to show (default: 10)
- `--all`: Show all tests (not just top N)

### `inq flaky`

Rank tests by flakiness across recorded runs. Flakiness is measured by
pass↔fail transitions in consecutive runs in which the test was recorded,
so chronically broken tests rank low and genuinely flapping tests rank high.

Options:
- `-n, --count <N>`: Number of tests to show (default: 10)
- `--all`: Show all candidate tests (not just top N)
- `--min-runs <N>`: Minimum runs a test must appear in to be ranked (default: 5)

Each row reports `flake%` (transitions / max(1, runs - 1)), `fail%`
(failures / runs), the raw transition count, and the number of recorded runs.

### `inq summarize`

Group a run's failing tests by their common failure cause. Each failure's
traceback is normalized (file paths, line numbers, hex addresses, temp dirs,
durations are replaced with placeholders), anchored at the last line that
names the failure (e.g. `AssertionError:`, `panicked at`, `FAILED`), and
clustered with other failures whose normalized tails share a common suffix.
Each cluster is reported with the longest line sequence shared by every
member, so you see the unique failure once and the count of tests it covers.

Options:
- `-r, --run <ID>`: Run ID to summarize (defaults to latest; supports negative indices)
- `--samples <N>`: Maximum sample test IDs to list per group (default: 5)

### `inq log <TESTPATTERN>`

Show logs and tracebacks for specific tests, matched by glob-style patterns.

Options:
- `-r, --run <ID>`: Run ID to show logs from (defaults to latest)

Example:
```sh
inq log 'test.module.TestCase.*'
```

### `inq shard <N/M>`

Print the test IDs assigned to one shard of an `M`-way balanced split.
Distributed CI nodes can each call `inq shard N/M` to claim a disjoint,
load-balanced slice of the suite. Historical durations from the repository
are used to balance shards when available; without history, the split
degrades to round-robin.

The partition is deterministic given the same suite and history, so two
nodes calling `inq shard 1/4` and `inq shard 2/4` produce disjoint shards
whose union is the full suite.

Options:
- `--group-regex <REGEX>`: Override the config's `group_regex`. Pass an
  empty string to disable grouping for this command only.
- `--zero-indexed`: Treat the shard index as 0-based (i.e. `0/4..3/4`).

Example (GitHub Actions matrix):

```yaml
strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: inq shard ${{ matrix.shard }}/4 > shard.txt
  - run: inq run --load-list shard.txt
```

### `inq prune`

Drop older test runs from the repository. Exactly one selection mode must be
given:

- `--keep <N>`: Keep the N most recent runs and prune the rest
- `--older-than <DURATION>`: Prune runs older than the given duration. Accepts
  `s`, `m`, `h`, `d`, and `w` suffixes (e.g. `30d`, `2w`, `12h`)
- `--run <ID>`: Prune the named run by ID. May be given multiple times
- `--all`: Prune every run in the repository

Pass `--dry-run` to see which runs would be pruned without modifying the
repository. In-progress runs are always skipped.

### `inq list-tests`

List all available tests by querying the test command with the list option from configuration.

### `inq analyze-isolation <TEST>`

Analyze test isolation issues using bisection to find which tests cause a target test to fail when run together but pass in isolation.

This command:
1. Runs the target test in isolation to verify it passes alone
2. Runs all tests together to verify the failure reproduces
3. Uses binary search to find the minimal set of tests causing the failure
4. Reports which tests interact with the target test

Example:
```sh
inq analyze-isolation test.module.TestCase.test_flaky
```

### `inq upgrade`

Upgrade a legacy `.testrepository/` directory to the new `.inquest/` format. Only available when built with the `testr` feature.

### `inq completions <SHELL>`

Generate shell completions for the given shell. Supported shells: `bash`, `zsh`, `fish`, `elvish`, `powershell`.

To enable completions:

```sh
# Zsh (add to ~/.zshrc)
eval "$(inq completions zsh)"

# Bash (add to ~/.bashrc)
eval "$(inq completions bash)"

# Fish (add to ~/.config/fish/config.fish)
inq completions fish | source
```

## Global Options

All commands support:
- `-C, --directory <PATH>`: Specify repository path (defaults to current directory)

## Configuration

inq looks for configuration in the following files (in order of priority):
`inquest.toml`, `.inquest.toml`, `.testr.conf`. The TOML format is preferred;
`.testr.conf` (INI format with a `[DEFAULT]` section) is supported for backward
compatibility.

Key options:

- `test_command`: Command to run tests (required)
- `test_id_option`: Option format for running specific tests (e.g., `--test $IDFILE`)
- `test_list_option`: Option to list all available tests
- `test_id_list_default`: Default value for $IDLIST when no specific tests
- `group_regex`: Regex to group related tests together during parallel execution
- `test_run_concurrency`: Command to determine concurrency level (e.g., `nproc`)
- `filter_tags`: Tags to filter test results by (for parallel execution)
- `instance_provision`: Command to provision test instances (receives `$INSTANCE_COUNT`)
- `instance_execute`: Command template for running tests in an instance (receives `$INSTANCE_ID`)
- `instance_dispose`: Command to clean up test instances (receives `$INSTANCE_ID`)

### Variable Substitution

The following variables are available for use in `test_command`:

- `$IDOPTION`: Expands to the `test_id_option` with actual test IDs
- `$IDFILE`: Path to a temporary file containing test IDs (one per line)
- `$IDLIST`: Space-separated list of test IDs
- `$LISTOPT`: Expands to the `test_list_option`

### Example Configurations

#### Rust with Cargo

```toml
test_command = "cargo test $IDOPTION"
test_id_option = "--test $IDFILE"
test_list_option = "--list"
```

#### Python with pytest

```toml
test_command = "pytest $IDOPTION"
test_id_option = "--test-id-file=$IDFILE"
test_list_option = "--collect-only -q"
```

#### Go

Use the `gotest-run` wrapper (ships with python-subunit):

```toml
test_command = "gotest-run $LISTOPT $IDOPTION"
test_id_option = "--id-file $IDFILE"
test_list_option = "--list"
```

`inq auto` generates this automatically when it sees a `go.mod`. The
wrapper enumerates tests via `go test -json -list ...` for `--list`,
fans out one `go test -json -run <regex>` invocation per affected
package for `--id-file`, and otherwise runs the whole tree. Subtests
are created at runtime by `t.Run` and aren't statically discoverable,
so they're absent from listings — but executing them by ID works
(e.g. `inq run --failing` correctly re-runs `pkg::TestX/sub_one`).

#### JVM (Maven / Gradle)

Drive the build tool through `jvmtest-subunit` (ships with
python-subunit):

```toml
test_command = "jvmtest-subunit $LISTOPT $IDOPTION"
test_id_option = "--id-file $IDFILE"
test_list_option = "--list"
```

`inq auto` generates this from `pom.xml` or
`build.gradle{,.kts}`/`settings.gradle{,.kts}`. The wrapper:

* auto-detects Maven vs Gradle from the working directory;
* spawns the build tool and watches its reports directory live so
  per-class results stream into `inq run` as each test class finishes;
* enumerates tests for `--list` by walking `src/test/java` and
  `src/test/kotlin` for conventionally-named test classes
  (`*Test`/`*Tests`/`*TestCase`/`*IT`);
* translates `--id-file` IDs (`Class` or `Class::method`) to the
  build tool's selection syntax — `-Dtest=Class1,Class2#methodA+methodB`
  for Maven, `--tests Class1 --tests Class2.methodA` for Gradle;
* forwards build-tool stdout/stderr so users still see compile errors
  and progress.

`inq run --failing`, `inq run --isolated`, and `inq list-tests` all
work — methods created at runtime (`@ParameterizedTest`,
`@TestFactory` dynamic tests) aren't statically discoverable and so
are absent from listings, but executing them by ID works.

The wrapper takes optional flags for non-default layouts:
`jvmtest-subunit --tool gradle --reports-dir build/custom-reports`,
or pass extra build args after `--`:
`jvmtest-subunit -- -Dmaven.test.skip=false`.

#### Node.js with Vitest

Vitest ships a built-in TAP reporter; pipe it through `tap2subunit`
(ships with python-subunit):

```toml
test_command = "vitest run --reporter=tap | tap2subunit"
```

`inq auto` generates this when it finds a `vitest.config.*` file or
`vitest` in `package.json` dependencies.

#### Node.js with Jest

Jest has no built-in machine-readable reporter, so the standard
approach is to install `jest-junit` as a devDependency and convert
its XML output:

```toml
test_command = "jest --ci --reporters=jest-junit; junitxml2subunit junit.xml"
```

`inq auto` generates this when it finds a `jest.config.*` file, a
`jest` block in `package.json`, or `jest` in `package.json`
dependencies. The `;` (rather than `&&`) ensures `junitxml2subunit`
still runs when tests fail.

#### Advanced Configuration with Parallel Execution

```toml
test_command = "cargo test --quiet $IDOPTION"
test_id_option = "$IDLIST"
test_list_option = "--list"

# Use system CPU count for parallel execution
test_run_concurrency = "nproc"

# Group tests by module (keeps related tests together)
group_regex = "^(.*)::[^:]+$"
```

## GitHub Actions

The bundled `jelmer/inquest` composite action installs `inq`, restores
and saves the `.inquest` history cache, and runs `inq ci`:

```yaml
- uses: actions/checkout@v4
- uses: jelmer/inquest@main
```

See [README.gh-action.md](README.gh-action.md) for the full reference,
including inputs and outputs, a manual setup recipe (no action
dependency), test-history caching, flake retries, sharding across
matrix jobs, and GitLab CI.

## Repository Format

Inquest has its own on-disk format, stored in a `.inquest/` directory:

- `metadata.db`: SQLite database storing run metadata (git commit, command, duration, exit code, concurrency)
- `runs/`: Directory containing individual test run files in subunit v2 binary format
- `runs/N.lock`: Lock files for tracking in-progress test runs

When built with the `testr` feature, inquest can also read and write the legacy `.testrepository/` format used by the Python testrepository tool. Use `inq upgrade` to migrate.

## Licensing

Inquest is under BSD / Apache 2.0 licences. See the file COPYING in the source for details.

## History

Inquest started as a Rust port of the Python [testrepository](https://github.com/testing-cabal/testrepository) tool, originally written by Robert Collins. It has since diverged with its own features:

- **New on-disk format** (`.inquest/`) using SQLite for metadata, with richer run information (git commit, command, duration, exit code, concurrency)
- **Run lock files** and `inq running` command for tracking in-progress test runs
- **Auto-detection** of project type (`inq auto`)
- **Timeout support**: per-test timeouts, overall run timeouts, and no-output timeouts with automatic process killing
- **`inq log`** command for viewing logs/tracebacks of individual tests by glob pattern
- **`inq info`** command for detailed run metadata
- **Test filtering** by regex patterns on the command line
- **Pass-through arguments** to the underlying test command via `--`

## Links

- Original Python version: https://github.com/testing-cabal/testrepository
- Subunit: http://subunit.readthedocs.io/
