<SYSTEM>This is the full developer documentation for Runsight</SYSTEM>

# Runsight

> YAML-first workflow engine for AI agents. Workflows are files on disk. Your repo is the database. Git is your version control.

## Get started

[Section titled “Get started”](#get-started)

[Quickstart ](/docs/getting-started/quickstart/)Install Runsight and run your first workflow in under 5 minutes.

[Key Concepts ](/docs/getting-started/key-concepts/)Workflows, blocks, souls, tools, and dispatch.

## Core documentation

[Section titled “Core documentation”](#core-documentation)

[YAML Schema ](/docs/workflows/yaml-schema/)Top-level sections, fields, and structure of a workflow file.

[Block Types ](/docs/workflows/block-types/)Linear, gate, code, loop, workflow — the five block types.

[Souls ](/docs/souls/overview/)Agent identities — role, prompt, model, tools. Reusable library files or inline.

[Custom Tools ](/docs/tools/custom-tools/)Define tools as YAML files with canonical IDs. HTTP, Python, and file-based executors.

## Execution and quality

[Section titled “Execution and quality”](#execution-and-quality)

[Git Integration ](/docs/execution/git-integration/)Save, commit, simulation branches, and fork recovery.

[Assertions and Eval ](/docs/evaluation/assertions/)Block-level assertions, transform hooks, and the offline eval test harness.

# Fallback Model

> Per-provider fallback targets and strict one-hop failover in Runsight.

When an LLM provider goes down or returns an error, you do not want your workflow to fail immediately. Runsight’s fallback system lets you define a single backup provider and model for each configured provider. If the primary fails, Runsight retries once on the fallback target. There is no chain — one hop, then fail.

## Why per-provider, not global

[Section titled “Why per-provider, not global”](#why-per-provider-not-global)

Early versions of Runsight had a global fallback chain: a ranked list of providers tried in sequence. This was replaced with per-provider fallback targets for three reasons:

1. **Predictability.** A global chain makes it hard to reason about which model will actually run a given soul. With per-provider targets, each provider maps to exactly one fallback — you always know the backup.
2. **Cost control.** A global chain can silently route traffic to an expensive provider. Per-provider mapping gives explicit control over where traffic lands.
3. **No implicit defaults.** The global chain had a “first available” fallback that could pick a provider you did not intend. The current system has no implicit behavior — if no fallback is configured, the call fails.

## How fallback targets work

[Section titled “How fallback targets work”](#how-fallback-targets-work)

Each provider can have at most one fallback target, which consists of:

* **Fallback provider** — a different active provider
* **Fallback model** — a specific model available on that provider

The rules are strict:

* A provider **cannot** fall back to itself.
* The fallback provider must be **active** (enabled).
* The fallback model must be in the fallback provider’s **discovered model list**.
* Both `fallback_provider_id` and `fallback_model_id` must be set together, or both omitted.

```plaintext
OpenAI (gpt-4o) ──fails──> Anthropic (claude-sonnet-4-20250514)     ✓ valid
Anthropic       ──fails──> OpenAI (gpt-4o-mini)                     ✓ valid
Google          ──fails──> (none configured)                        ✓ valid — fails on error
OpenAI          ──fails──> OpenAI (gpt-4-turbo)                     ✗ cannot self-reference
```

## Enabling fallback

[Section titled “Enabling fallback”](#enabling-fallback)

Fallback is disabled by default. To turn it on:

1. Open **Settings** and go to the **Fallback** tab.
2. Toggle **Enable fallback** on. This requires at least two active providers.
3. For each provider row, select a fallback provider and then a fallback model from its available models.
4. The selection saves automatically when you pick a model.

The enable/disable toggle is a global switch stored in app settings as `fallback_enabled`. When disabled, all fallback targets are preserved but inactive — no runtime retry occurs.

## What happens at runtime

[Section titled “What happens at runtime”](#what-happens-at-runtime)

When a soul’s LLM call fails and fallback is enabled:

1. Runsight looks up the soul’s `provider` field to find the primary provider.
2. It checks whether that provider has a fallback target configured.
3. If yes, it retries the call once using the fallback provider and model.
4. If the retry also fails, the block fails with the retry error.

If fallback is disabled or no target is configured for the provider, the original error propagates immediately.

Note

Fallback is **one hop only**. If OpenAI falls back to Anthropic and Anthropic also fails, the call does not chain further to a third provider. This is by design — chains hide failures and make debugging harder.

## How settings are stored

[Section titled “How settings are stored”](#how-settings-are-stored)

Fallback configuration lives in `.runsight/settings.yaml` alongside other app settings. This file is gitignored.

.runsight/settings.yaml

```yaml
fallback_enabled: true
fallback_map:
  - provider_id: openai
    fallback_provider_id: anthropic
    fallback_model_id: claude-sonnet-4-20250514
  - provider_id: anthropic
    fallback_provider_id: openai
    fallback_model_id: gpt-4o-mini
```

Each entry in `fallback_map` is a `FallbackTargetEntry` with three fields:

| Field                  | Type  | Description                                      |
| ---------------------- | ----- | ------------------------------------------------ |
| `provider_id`          | `str` | The primary provider’s ID                        |
| `fallback_provider_id` | `str` | The backup provider’s ID                         |
| `fallback_model_id`    | `str` | The specific model to use on the backup provider |

## Fallback API endpoints

[Section titled “Fallback API endpoints”](#fallback-api-endpoints)

| Method | Endpoint                            | Description                                    |
| ------ | ----------------------------------- | ---------------------------------------------- |
| `GET`  | `/settings/fallbacks`               | List all fallback targets for active providers |
| `PUT`  | `/settings/fallbacks/{provider_id}` | Set or clear a provider’s fallback target      |

To clear a fallback target, send both `fallback_provider_id` and `fallback_model_id` as empty strings:

```json
{
  "fallback_provider_id": "",
  "fallback_model_id": ""
}
```

## Soul resolution and fallback

[Section titled “Soul resolution and fallback”](#soul-resolution-and-fallback)

Souls specify their provider via the `provider` field (e.g., `provider: openai`). When a workflow runs:

1. The engine resolves the soul reference via `soul_ref` and looks it up in `custom/souls/`.
2. The soul’s `provider` and `model_name` determine which LLM backend handles the call.
3. If the call fails and fallback is enabled, the fallback target for that provider type is used for a single retry.

If a soul does not set a `provider`, it uses the runner’s default. Fallback only applies to providers that have an explicit fallback target configured — there is no implicit “pick the next available provider” behavior.

<!-- Linear: RUN-233, RUN-240 — last verified against codebase 2026-04-07 -->

# First-Time Setup

> What Runsight creates on first launch and how to configure your API keys.

## Project scaffolding

[Section titled “Project scaffolding”](#project-scaffolding)

When Runsight starts for the first time, the API server creates a blank workspace automatically:

1. **Empty workspace directories.** Runsight creates empty `custom/workflows/`, `custom/souls/`, and `custom/tools/` directories, plus a `.gitignore` that excludes `.runsight/` and `.canvas/`.

2. **Git initialization.** If no `.git` directory exists, Runsight runs `git init`, sets a local git user (`runsight@localhost`), stages the scaffolded files, and creates an initial commit.

3. **Settings file.** `.runsight/settings.yaml` is created with `onboarding_completed: false`. See [Settings](/docs/configuration/settings) for the full reference.

## API key setup

[Section titled “API key setup”](#api-key-setup)

Runsight needs an API key for at least one LLM provider before you can run workflows.

**Environment variables are checked first.** If you already have a key exported in your shell, Runsight picks it up automatically — no further configuration needed:

```bash
export OPENAI_API_KEY=sk-...
```

If no environment variable is found, Runsight checks `.runsight/secrets.env`. This file is created when you add a provider through the Settings page. The resolution order is:

1. `os.environ` (real environment variable)
2. `.runsight/secrets.env` (managed by Runsight, gitignored)

Provider YAML files in `custom/providers/` store references like `${OPENAI_API_KEY}`, never raw keys.

## Provider configuration

[Section titled “Provider configuration”](#provider-configuration)

Once your API key is available, configure a provider through **Settings > Providers > Add Provider** in the sidebar. Runsight auto-tests the connection and populates the model list on success.

See [Providers](/docs/configuration/providers) for supported providers, storage format, and how souls reference them.

## Project directory after setup

[Section titled “Project directory after setup”](#project-directory-after-setup)

```plaintext
your-project/
├── .git/                    # Auto-initialized git repo
├── .gitignore               # Excludes .runsight/ and .canvas/
├── .runsight/               # Gitignored runtime data
│   ├── settings.yaml        # App settings
│   └── secrets.env          # API keys (gitignored)
└── custom/
    ├── providers/            # Provider YAML files
    ├── souls/                # Empty until you add your own souls
    ├── tools/                # Empty until you add your own tools
    └── workflows/            # Empty until you create your first workflow
```

## Next steps

[Section titled “Next steps”](#next-steps)

Runsight does not ship sample workflow, soul, or tool YAML into your runtime workspace. After first launch, create your first workflow through onboarding or by adding files under `custom/`.

* [Providers](/docs/configuration/providers) — add and manage LLM providers
* [Quickstart](/docs/getting-started/quickstart) — create and run your first workflow
* [Settings](/docs/configuration/settings) — full settings.yaml reference

# Providers

> How to add and manage LLM providers in Runsight — OpenAI, Anthropic, Google, and more.

Providers are the LLM backends that power your workflows. Each provider represents a connection to an AI service — OpenAI, Anthropic, Google, Ollama, or any OpenAI-compatible endpoint. Before you can run a workflow, at least one provider must be configured with a valid API key.

## Supported providers

[Section titled “Supported providers”](#supported-providers)

Runsight supports the following providers out of the box:

| Provider     | Type key       | Models                          | API key required |
| ------------ | -------------- | ------------------------------- | ---------------- |
| OpenAI       | `openai`       | GPT-4o, GPT-4, o-series         | Yes              |
| Anthropic    | `anthropic`    | Claude Haiku, Sonnet, Opus      | Yes              |
| Google       | `google`       | Gemini Pro, Gemini Flash        | Yes              |
| Azure OpenAI | `azure_openai` | GPT models via Azure            | Yes              |
| AWS Bedrock  | `aws_bedrock`  | Claude, Titan via AWS           | Yes              |
| Mistral      | `mistral`      | Mistral Large, Codestral        | Yes              |
| Cohere       | `cohere`       | Command R+, Embed               | Yes              |
| Groq         | `groq`         | LLaMA, Mixtral (fast inference) | Yes              |
| Together AI  | `together`     | Open-source models              | Yes              |
| Ollama       | `ollama`       | Local models (LLaMA, etc.)      | No               |
| Custom       | `custom`       | Any OpenAI-compatible endpoint  | Yes              |

Provider type is selected explicitly when you create a provider. The provider’s identity is its embedded `id`; the display `name` is only a label.

## Adding a provider via the Settings page

[Section titled “Adding a provider via the Settings page”](#adding-a-provider-via-the-settings-page)

1. Open **Settings** from the sidebar.
2. On the **Providers** tab, click **Add Provider**.
3. In the dialog, select a provider from the dropdown.
4. Enter your API key. Runsight auto-tests the connection after a short debounce.
5. For Ollama or custom providers, enter a **Base URL** (Ollama defaults to `http://localhost:11434`).
6. Once the connection test shows success, click **Save**.

After saving, Runsight tests the connection again and populates the provider’s model list from the remote API.

Tip

You can also add a provider during your first run. If no providers are configured when you try to execute a workflow, Runsight shows an API key modal so you can set one up on the spot.

## How providers are stored

[Section titled “How providers are stored”](#how-providers-are-stored)

Providers are persisted as individual YAML files in `custom/providers/`. The provider ID is the embedded `id`, and the filename stem must match that id. For example, `custom/providers/openai.yaml` must contain `id: openai`.

custom/providers/openai.yaml

```yaml
id: openai
kind: provider
name: OpenAI
type: openai
api_key: ${OPENAI_API_KEY}
base_url: null
status: connected
is_active: true
models:
  - gpt-4o
  - gpt-4o-mini
  - gpt-4-turbo
```

API keys are stored as environment variable references (`${PROVIDER_API_KEY}`) pointing to entries in `.runsight/secrets.env`. The secrets file is gitignored and never committed. You can also reference real environment variables — `os.environ` takes precedence over the secrets file.

## How souls reference providers

[Section titled “How souls reference providers”](#how-souls-reference-providers)

Every soul **must** set both `provider` and `model_name`. The soul schema marks these fields as optional, but the runner hard-fails at execution time if either is missing:

* Neither set → `ValueError: "Soul '{id}' must define an explicit provider and model_name"`
* `model_name` set but no `provider` → `ValueError: "Soul '{id}' must define an explicit provider"`
* `provider` set but no `model_name` → `ValueError: "Soul '{id}' must define an explicit model_name"`

There is no global fallback or default provider. If you see a soul without these fields, it will fail on run.

custom/souls/researcher.yaml

```yaml
id: researcher
kind: soul
name: Researcher
role: Senior Researcher
system_prompt: "Research the given topic thoroughly."
provider: openai
model_name: gpt-4o
temperature: 0.7
```

The `provider` field value must match the `type` of a configured provider (e.g., `openai`, `anthropic`, `google`), not the provider’s display name or ID.

## Model catalog

[Section titled “Model catalog”](#model-catalog)

Runsight includes a built-in model catalog powered by LiteLLM’s model cost dictionary. The catalog provides metadata for every known model across all providers:

* Model ID and provider
* Max tokens and max input tokens
* Input and output cost per token
* Capability flags: vision support, function calling support, streaming support

By default, the `/models` API endpoint returns only models whose provider matches a configured provider. Pass `?all=true` to see the full catalog regardless of configuration.

## Provider API endpoints

[Section titled “Provider API endpoints”](#provider-api-endpoints)

All provider endpoints live under `/settings/providers`:

| Method   | Endpoint                        | Description                        |
| -------- | ------------------------------- | ---------------------------------- |
| `GET`    | `/settings/providers`           | List all providers                 |
| `GET`    | `/settings/providers/{id}`      | Get a single provider              |
| `POST`   | `/settings/providers`           | Create a provider                  |
| `PUT`    | `/settings/providers/{id}`      | Update a provider                  |
| `DELETE` | `/settings/providers/{id}`      | Delete a provider                  |
| `POST`   | `/settings/providers/{id}/test` | Test a saved provider’s connection |
| `POST`   | `/settings/providers/test`      | Test credentials before saving     |

The model catalog has its own endpoints:

| Method | Endpoint            | Description                                               |
| ------ | ------------------- | --------------------------------------------------------- |
| `GET`  | `/models`           | List models (filtered to configured providers by default) |
| `GET`  | `/models/providers` | List provider summaries with `is_configured` flag         |

## Managing providers

[Section titled “Managing providers”](#managing-providers)

### Testing a connection

[Section titled “Testing a connection”](#testing-a-connection)

Click **Test** on any provider row in the Providers tab. The test hits the provider’s model-listing endpoint and reports success or failure along with latency. On success, the provider’s model list is refreshed.

### Enabling and disabling

[Section titled “Enabling and disabling”](#enabling-and-disabling)

Each provider has an **Enabled** toggle. Disabled providers are excluded from model catalog queries and fallback target lists but are not deleted — you can re-enable them at any time.

### Editing

[Section titled “Editing”](#editing)

Click **Edit** on a provider row to update the API key or base URL. Leave the API key field empty to keep the existing key.

### Deleting

[Section titled “Deleting”](#deleting)

Click **Delete** to remove a provider. This deletes the YAML file from `custom/providers/` and removes the API key from `.runsight/secrets.env`.

Caution

Deleting a provider that souls reference by type will cause those souls to fail at runtime. Check soul files before deleting a provider.

<!-- Linear: RUN-151, RUN-233 — last verified against codebase 2026-04-07 -->

# Settings

> Reference for the settings.yaml file and the settings API.

Application settings are stored in `.runsight/settings.yaml`. This file is created automatically during [project scaffolding](/docs/configuration/first-time-setup) and is gitignored.

## Settings fields

[Section titled “Settings fields”](#settings-fields)

| Field                  | Type   | Default | Description                                                                                                   |
| ---------------------- | ------ | ------- | ------------------------------------------------------------------------------------------------------------- |
| `onboarding_completed` | `bool` | `false` | Whether the first-time setup flow has been completed. Controls whether the app redirects to the setup screen. |
| `fallback_enabled`     | `bool` | `false` | Whether runtime fallback is active. See [Fallback](/docs/configuration/fallback).                             |

## Fallback map

[Section titled “Fallback map”](#fallback-map)

The same file stores per-provider fallback targets under a `fallback_map` key:

.runsight/settings.yaml

```yaml
onboarding_completed: true
fallback_enabled: true
fallback_map:
  - provider_id: anthropic
    fallback_provider_id: openai
    fallback_model_id: gpt-4.1-mini
```

Each entry in `fallback_map` has three fields:

| Field                  | Type  | Description                                        |
| ---------------------- | ----- | -------------------------------------------------- |
| `provider_id`          | `str` | The provider this fallback applies to              |
| `fallback_provider_id` | `str` | The provider to fall back to                       |
| `fallback_model_id`    | `str` | The specific model to use on the fallback provider |

See [Fallback](/docs/configuration/fallback) for how to configure fallback targets.

## Storage locations

[Section titled “Storage locations”](#storage-locations)

| Data                          | Location                  | Git tracked     |
| ----------------------------- | ------------------------- | --------------- |
| Providers                     | `custom/providers/*.yaml` | Yes             |
| API keys                      | `.runsight/secrets.env`   | No (gitignored) |
| App settings and fallback map | `.runsight/settings.yaml` | No (gitignored) |

Provider YAML files live inside `custom/providers/` and are committed to git. API keys are stored as environment variable references (`${OPENAI_API_KEY}`) that resolve against the real environment first, then `.runsight/secrets.env`. See [Providers](/docs/configuration/providers) for details.

## Settings API endpoints

[Section titled “Settings API endpoints”](#settings-api-endpoints)

### App settings

[Section titled “App settings”](#app-settings)

| Method | Endpoint        | Description                         |
| ------ | --------------- | ----------------------------------- |
| `GET`  | `/settings/app` | Get current app settings            |
| `PUT`  | `/settings/app` | Update app settings (partial merge) |

### Providers

[Section titled “Providers”](#providers)

| Method   | Endpoint                        | Description                      |
| -------- | ------------------------------- | -------------------------------- |
| `GET`    | `/settings/providers`           | List all providers               |
| `GET`    | `/settings/providers/{id}`      | Get a single provider            |
| `POST`   | `/settings/providers`           | Create a provider                |
| `PUT`    | `/settings/providers/{id}`      | Update a provider                |
| `DELETE` | `/settings/providers/{id}`      | Delete a provider                |
| `POST`   | `/settings/providers/{id}/test` | Test a saved provider connection |
| `POST`   | `/settings/providers/test`      | Test credentials before saving   |

### Fallback

[Section titled “Fallback”](#fallback)

| Method | Endpoint                            | Description                                    |
| ------ | ----------------------------------- | ---------------------------------------------- |
| `GET`  | `/settings/fallbacks`               | List fallback targets for all active providers |
| `PUT`  | `/settings/fallbacks/{provider_id}` | Set or clear a provider’s fallback target      |

### Budgets

[Section titled “Budgets”](#budgets)

| Method | Endpoint            | Description                                        |
| ------ | ------------------- | -------------------------------------------------- |
| `GET`  | `/settings/budgets` | List budgets (placeholder — returns an empty list) |

<!-- Linear: RUN-151, RUN-233, RUN-240 — last verified against codebase 2026-04-07 -->

# Assertions

> Built-in and custom block-level assertions, their config fields, and how Runsight computes eval results.

Assertions are quality checks that run after a block completes. You define them directly in workflow YAML, and each assertion produces a pass/fail result plus a numeric score from `0.0` to `1.0`.

Runsight ships 15 built-in deterministic assertion types. You can also add your own scanner-discovered Python assertions under `custom/assertions/`. See [Custom Assertions](/docs/evaluation/custom-assertions) for the custom plugin workflow.

## Where assertions live

[Section titled “Where assertions live”](#where-assertions-live)

Assertions are defined on the `assertions` field of any block definition. The field accepts a list of assertion config objects:

custom/workflows/research.yaml

```yaml
version: "1.0"
id: research
kind: workflow
blocks:
  analyze:
    type: code
    code: |
      def main(data):
          return "analysis ready"
    assertions:
      - type: contains
        value: "analysis"
      - type: cost
        threshold: 0.05
workflow:
  name: assertions_demo
  entry: analyze
  transitions:
    - from: analyze
      to: null
```

The `assertions` field is declared on `BaseBlockDef` as `Optional[List[Dict[str, Any]]]` and defaults to `None`.

## Assertion config fields

[Section titled “Assertion config fields”](#assertion-config-fields)

Each assertion in the list is a dict with these fields:

| Field       | Type    | Required | Default | Description                                                                                            |
| ----------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------ |
| `type`      | `str`   | yes      | —       | Assertion type identifier. For custom assertions, use `custom:<id>`                                    |
| `value`     | `any`   | depends  | `""`    | Comparison value. Required for most string and linguistic assertions                                   |
| `threshold` | `float` | no       | varies  | Numeric threshold. Meaning depends on assertion type                                                   |
| `config`    | `any`   | no       | `None`  | Per-assertion config payload. Built-ins ignore it. Custom assertions receive it as `context["config"]` |
| `weight`    | `float` | no       | `1.0`   | Weight in the aggregate score calculation                                                              |
| `metric`    | `str`   | no       | `None`  | Named metric label. When set, the score is stored in `named_scores` under this key                     |
| `transform` | `str`   | no       | `None`  | Pre-process output before evaluation. See [Transform Hooks](/docs/evaluation/transform-hooks)          |

## Built-in assertion types

[Section titled “Built-in assertion types”](#built-in-assertion-types)

Runsight ships 15 deterministic assertion types across four categories.

### String assertions

[Section titled “String assertions”](#string-assertions)

| Type           | Value                 | Behavior                                                                   |
| -------------- | --------------------- | -------------------------------------------------------------------------- |
| `equals`       | `str`                 | Exact string match. Use `config: {mode: json}` to opt into JSON deep-equal |
| `contains`     | `str`                 | Case-sensitive substring check                                             |
| `icontains`    | `str`                 | Case-insensitive substring check                                           |
| `contains-all` | `list[str]`           | All items must appear as substrings                                        |
| `contains-any` | `list[str]`           | At least one item must appear as a substring                               |
| `starts-with`  | `str`                 | String prefix check                                                        |
| `regex`        | `str`                 | Regex search (uses `re.search`, not full match)                            |
| `word-count`   | `int` or `{min, max}` | Exact count or range check on whitespace-split words                       |

#### Examples

[Section titled “Examples”](#examples)

String assertion examples

```yaml
assertions:
  # Exact match
  - type: equals
    value: "approved"


  # Case-insensitive substring
  - type: icontains
    value: "conclusion"


  # All keywords must appear
  - type: contains-all
    value: ["summary", "recommendation", "next steps"]


  # Any keyword is acceptable
  - type: contains-any
    value: ["approve", "accept", "pass"]


  # Regex pattern
  - type: regex
    value: "\\d{4}-\\d{2}-\\d{2}"


  # Word count range
  - type: word-count
    value:
      min: 50
      max: 500
```

### Structural assertions

[Section titled “Structural assertions”](#structural-assertions)

| Type            | Value                         | Behavior                                                                                           |
| --------------- | ----------------------------- | -------------------------------------------------------------------------------------------------- |
| `is-json`       | `dict` (optional JSON Schema) | Validates output is valid JSON. If `value` is provided, validates against a JSON Schema            |
| `contains-json` | `dict` (optional JSON Schema) | Finds a JSON substring in the output. Scans for `{` and `[` delimiters. Optional schema validation |

#### Examples

[Section titled “Examples”](#examples-1)

Structural assertion examples

```yaml
assertions:
  # Output must be valid JSON
  - type: is-json


  # Output must contain a JSON object matching a schema
  - type: contains-json
    value:
      type: object
      required: ["name", "score"]
      properties:
        name:
          type: string
        score:
          type: number
```

### Performance assertions

[Section titled “Performance assertions”](#performance-assertions)

| Type      | Value | Threshold     | Behavior                                                                     |
| --------- | ----- | ------------- | ---------------------------------------------------------------------------- |
| `cost`    | —     | `float` (USD) | Passes if `cost_usd` from the execution context is at or below `threshold`   |
| `latency` | —     | `float` (ms)  | Passes if `latency_ms` from the execution context is at or below `threshold` |

Performance assertions read from the `AssertionContext`, not from the block output string. In live API runs that context is populated with run metrics such as cost, latency, and tokens; offline eval uses zeroed metric fields.

Performance assertion examples

```yaml
assertions:
  - type: cost
    threshold: 0.10
  - type: latency
    threshold: 5000
```

### Linguistic assertions

[Section titled “Linguistic assertions”](#linguistic-assertions)

| Type          | Value                  | Default threshold | Behavior                                                                    |
| ------------- | ---------------------- | ----------------- | --------------------------------------------------------------------------- |
| `levenshtein` | `str` (reference text) | `5`               | Edit distance between output and reference. Passes if distance <= threshold |
| `bleu`        | `str` (reference text) | `0.5`             | BLEU-4 score with smoothing. Passes if score >= threshold                   |
| `rouge-n`     | `str` (reference text) | `0.75`            | ROUGE-1 F-measure. Passes if score >= threshold                             |

Linguistic assertion examples

```yaml
assertions:
  - type: levenshtein
    value: "The capital of France is Paris."
    threshold: 10
  - type: bleu
    value: "Machine learning models process data to find patterns."
    threshold: 0.3
```

## Custom assertions

[Section titled “Custom assertions”](#custom-assertions)

Beyond the 15 built-in types, you can create your own assertions in Python. The contract is **promptfoo-compatible** — existing promptfoo assertion functions work with minimal changes. Custom assertions are discovered from manifest files under `custom/assertions/*.yaml` and are referenced by embedded assertion id:

Custom assertion usage

```yaml
assertions:
  - type: custom:tone_check
    config:
      prefix: calm
```

Important details:

* The runtime key is always `custom:<id>`.
* The manifest `name` is display-only.
* Custom assertions can be used alongside built-in assertions in the same list.
* Custom assertions support the same `not-` negation prefix as built-in assertions.

See [Custom Assertions](/docs/evaluation/custom-assertions) for the manifest format, Python contract, params schema validation, context keys, and migration guidance.

## Negation with not- prefix

[Section titled “Negation with not- prefix”](#negation-with-not--prefix)

Any assertion type can be negated by prefixing it with `not-`. The engine inverts both the pass/fail boolean and the score (1.0 - original):

Negated assertion

```yaml
assertions:
  - type: not-contains
    value: "error"
  - type: not-contains-json
  - type: not-custom:blocked_word
    config:
      blocked: storm
```

Negation works for both built-in and custom assertion types.

## Weighted scoring

[Section titled “Weighted scoring”](#weighted-scoring)

When a block has multiple assertions, the aggregate score is a weighted average. Each assertion’s `weight` (default `1.0`) determines its contribution:

Weighted assertions

```yaml
assertions:
  - type: contains
    value: "recommendation"
    weight: 2.0
  - type: word-count
    value:
      min: 100
    weight: 1.0
  - type: cost
    threshold: 0.05
    weight: 0.5
```

The `AssertionsResult` class accumulates weighted results. Its `aggregate_score` property returns the weighted average: `total_score / total_weight`. The `passed()` method without a threshold returns `True` only if every individual assertion passed.

## Execution model

[Section titled “Execution model”](#execution-model)

Assertions do not share state with each other. Each configured assertion is evaluated independently against the same block result.

Runsight evaluates assertions in two contexts:

* **Offline eval** — assertions run concurrently for each block
* **Live API runs** — assertions run sequentially after each block completes

In both contexts:

* Every configured assertion produces its own result
* A transform failure on one assertion does not prevent the rest from running
* Aggregate scoring follows the same weighted scoring rules

## Assertion chaining

[Section titled “Assertion chaining”](#assertion-chaining)

This is especially useful with `transform` hooks. Each assertion can target a different field in the block output using its own `json_path` transform:

Multiple assertions with different transforms

```yaml
assertions:
  - type: equals
    value: "completed"
    transform: "json_path:$.status"
  - type: contains
    value: "success"
    transform: "json_path:$.message"
  - type: cost
    threshold: 0.02
```

In this example, the first assertion extracts `$.status` and checks for an exact match, the second extracts `$.message` and checks for a substring, and the third checks execution cost without any transform.

If a transform fails on one assertion (e.g., the output is not valid JSON, or the path does not exist), that assertion returns `passed=False` with a descriptive reason. The other assertions still run normally and produce their own results. The aggregate score and pass/fail are then computed across all of them using the standard [weighted scoring](#weighted-scoring) rules.

## How assertions fire during execution

[Section titled “How assertions fire during execution”](#how-assertions-fire-during-execution)

When a workflow runs via the API, assertions fire automatically — no extra configuration needed beyond defining them on your blocks:

1. The engine reads each block’s `assertions` list from the workflow YAML
2. After a block completes, the engine evaluates all configured assertions against the block output
3. Each assertion receives the block’s output text plus execution context (cost, latency, soul info, tokens)
4. Results are persisted on the run record and pushed to the UI via SSE

The `RunNode` entity stores three eval fields:

| Field          | Type              | Description                                                                                          |
| -------------- | ----------------- | ---------------------------------------------------------------------------------------------------- |
| `eval_score`   | `Optional[float]` | Weighted average score across all assertions on the block                                            |
| `eval_passed`  | `Optional[bool]`  | `True` when every individual assertion passed                                                        |
| `eval_results` | `Optional[Dict]`  | Per-assertion results including pass/fail, score, reason, and handler type when the handler sets one |

These fields are `None` when a block has no assertions configured.

<!-- Linear: RUN-555, RUN-685, RUN-693, RUN-705, RUN-769, RUN-800, RUN-801 -- last verified against codebase 2026-04-10 -->

# Custom Assertions

> Create project-local Python assertions under custom/assertions and use them in offline evals and live workflow runs.

Custom assertions let you add workspace-local checks alongside Runsight’s 15 built-in assertions. Add a YAML manifest, drop your Python file next to it, and reference it as `custom:<id>` in your workflow.

Promptfoo compatible

The Python contract is the same as promptfoo’s `get_assert(output, context)`. If you already have promptfoo assertion functions, they work in Runsight with minimal changes — see [Migrating from Promptfoo](#migrating-from-promptfoo).

Runsight discovers custom assertions from `custom/assertions/*.yaml`, registers each one under `custom:{id}`, and runs them in both offline evals and live API workflow runs. The embedded `id` must match the YAML filename stem.

For built-in assertion types and shared assertion config fields, see [Assertions](/docs/evaluation/assertions).

## Quick Start

[Section titled “Quick Start”](#quick-start)

This example creates a custom assertion named `tone_check`, then uses it in an offline eval case with a built-in assertion alongside it.

custom/assertions/tone\_check.yaml

```yaml
version: "1.0"
id: tone_check
kind: assertion
name: "Tone Check"
description: "Passes when output starts with a configured prefix."
returns: "grading_result"
source: "tone_check.py"
params:
  type: object
  properties:
    prefix:
      type: string
  required: ["prefix"]
```

custom/assertions/tone\_check.py

```python
def get_assert(output, context):
    config = context.get("config", {})
    return {
        "pass": output.startswith(config.get("prefix", "")),
        "score": 0.9,
        "reason": f"prefix={config.get('prefix', '')}",
    }
```

custom/workflows/custom-assertions-demo.yaml

```yaml
version: "1.0"
id: custom-assertions-demo
kind: workflow
config:
  model_name: gpt-4o
blocks:
  analyze:
    type: code
    code: |
      def main(data):
          return "unused in fixture mode"
workflow:
  name: custom_assertions_demo
  entry: analyze
  transitions:
    - from: analyze
      to: null
eval:
  threshold: 0.5
  cases:
    - id: tone_case
      fixtures:
        analyze: "calm response"
      expected:
        analyze:
          - type: custom:tone_check
            config:
              prefix: calm
          - type: contains
            value: "response"
```

What this does:

* The assertion’s canonical ID is `tone_check` because the manifest file is `tone_check.yaml`.
* The runtime type is `custom:tone_check`.
* The `config` object is validated against `params` before the plugin runs.
* The same custom assertion can also be used under a block’s normal `assertions:` list during API workflow runs.

Note

Custom assertion discovery requires a workflow file on disk — the scanner reads your project’s `custom/assertions/` directory relative to the workflow file path. If you pass raw YAML strings programmatically instead of file paths, custom assertions won’t be discovered automatically.

## YAML Manifest Reference

[Section titled “YAML Manifest Reference”](#yaml-manifest-reference)

Each custom assertion is defined by a YAML manifest in `custom/assertions/<id>.yaml`.

custom/assertions/example.yaml

```yaml
version: "1.0"
id: example
kind: assertion
name: "Example Assertion"
description: "Checks something about the block output."
returns: "bool"
source: "example.py"
params:
  type: object
  properties:
    enabled:
      type: boolean
```

| Field         | Type                           | Required | Description                                                              |
| ------------- | ------------------------------ | -------- | ------------------------------------------------------------------------ |
| `version`     | `string`                       | yes      | Manifest version string                                                  |
| `id`          | `string`                       | yes      | Embedded assertion id. Must match the filename stem                      |
| `kind`        | `"assertion"`                  | yes      | Entity kind                                                              |
| `name`        | `string`                       | yes      | Display name for humans. This is not the runtime ID                      |
| `description` | `string`                       | yes      | Short description of the assertion                                       |
| `returns`     | `"bool"` or `"grading_result"` | yes      | Declares the plugin return contract                                      |
| `source`      | `string`                       | yes      | Python file to load, relative to the manifest file                       |
| `params`      | JSON Schema object             | no       | Schema used to validate the assertion’s `config` before plugin execution |

Important details:

* The canonical runtime ID is the embedded `id`, not `name`.
* `custom/assertions/tone_check.yaml` always registers as `custom:tone_check`.
* Extra top-level manifest fields are rejected.
* Built-in assertion name collisions are rejected at scan time.
* The source file must exist relative to the manifest file.

## Python Contract

[Section titled “Python Contract”](#python-contract)

A custom assertion source file must define exactly this function:

custom/assertions/example.py

```python
def get_assert(output, context):
    return True
```

Rules:

* The function name must be `get_assert`.
* The parameter list must be exactly `(output, context)`.
* The function must be synchronous.
* Runsight validates the function contract before registration.
* The plugin runs in a separate subprocess with a minimal environment.
* Plugin execution times out after 30 seconds.

The plugin receives:

* `output`: the block output string being checked
* `context`: a plain Python dict with assertion metadata and per-assertion config

Runsight does not forward API keys into the plugin subprocess environment.

## Return Types

[Section titled “Return Types”](#return-types)

The manifest `returns` field controls how Runsight interprets the plugin result.

### `bool`

[Section titled “bool”](#bool)

Use `returns: "bool"` when the assertion is a simple pass/fail check.

custom/assertions/contains\_calm.py

```python
def get_assert(output, context):
    return "calm" in output
```

`True` becomes a passing result with score `1.0`. `False` becomes a failing result with score `0.0`.

### `grading_result`

[Section titled “grading\_result”](#grading_result)

Use `returns: "grading_result"` when you need to control the score or reason.

custom/assertions/tone\_check.py

```python
def get_assert(output, context):
    config = context.get("config", {})
    return {
        "pass": output.startswith(config.get("prefix", "")),
        "score": 0.9,
        "reason": f"prefix={config.get('prefix', '')}",
    }
```

Accepted fields:

| Field                         | Required | Notes                                                |
| ----------------------------- | -------- | ---------------------------------------------------- |
| `passed` or `pass_` or `pass` | yes      | Runsight accepts these aliases in that precedence    |
| `score`                       | yes      | Must be numeric and between `0.0` and `1.0`          |
| `reason`                      | no       | Optional. Non-string values are coerced with `str()` |

Notes:

* `score` may be an `int` or `float`; Runsight converts it to `float`.
* If the returned shape does not match the declared contract, the assertion fails with a runtime error message instead of crashing the run.

## Config & Params

[Section titled “Config & Params”](#config--params)

Each assertion entry in workflow YAML can include a `config` field:

Block assertion config

```yaml
assertions:
  - type: custom:tone_check
    config:
      prefix: calm
```

For custom assertions, Runsight passes that value through two stages:

1. If the manifest defines `params`, Runsight validates `config` against that JSON Schema.
2. If validation succeeds, the exact value is exposed to the plugin as `context["config"]`.

If the manifest does not define `params`, Runsight skips config validation.

Note

Custom plugins receive their per-assertion input through `config`. The generic `value` and `threshold` fields still exist on assertion objects, but Runsight does not inject them into the plugin context dict.

### Schema-validated config

[Section titled “Schema-validated config”](#schema-validated-config)

custom/assertions/budget\_guard.yaml

```yaml
version: "1.0"
id: budget_guard
kind: assertion
name: "Budget Guard"
description: "Requires a numeric budget."
returns: "bool"
source: "budget_guard.py"
params:
  type: object
  properties:
    budget:
      type: number
  required: ["budget"]
```

custom/assertions/budget\_guard.py

```python
def get_assert(output, context):
    return True
```

Workflow usage

```yaml
assertions:
  - type: custom:budget_guard
    config:
      budget: 0.05
```

If `config` is invalid, Runsight returns a failing result whose reason starts with `Config validation failed:` and skips plugin execution.

### Generic assertion features still work

[Section titled “Generic assertion features still work”](#generic-assertion-features-still-work)

Custom assertions use the same outer assertion config object as built-ins, so these features still apply:

* `weight`
* `metric`
* `transform`
* `not-` negation, for example `not-custom:blocked_word`

## Context Dict Reference

[Section titled “Context Dict Reference”](#context-dict-reference)

Custom assertions receive a plain dict, not an `AssertionContext` object.

| Key            | Type     | Description                                                                                                                                     |
| -------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `vars`         | `dict`   | Workflow variables for the assertion context                                                                                                    |
| `config`       | `any`    | The per-assertion `config` value from workflow YAML                                                                                             |
| `prompt`       | `string` | Prompt text in the current assertion context                                                                                                    |
| `prompt_hash`  | `string` | Prompt hash for the current run                                                                                                                 |
| `soul_id`      | `string` | Soul ID for the block being evaluated                                                                                                           |
| `soul_version` | `string` | Soul version hash or identifier                                                                                                                 |
| `block_id`     | `string` | Block ID                                                                                                                                        |
| `block_type`   | `string` | Block type                                                                                                                                      |
| `cost_usd`     | `float`  | Execution cost in USD                                                                                                                           |
| `total_tokens` | `int`    | Total tokens used                                                                                                                               |
| `latency_ms`   | `float`  | Block latency in milliseconds                                                                                                                   |
| `run_id`       | `string` | Run ID                                                                                                                                          |
| `workflow_id`  | `string` | Workflow identifier from the current assertion context. In live API runs this is currently the workflow name; offline eval uses an empty string |

Notes:

* The key is `vars`, not `variables`.
* The key is `config`, even when the config value is `None`.
* Offline eval populates most context fields with empty strings or zeros.
* Live API execution fills prompt, soul, cost, token, latency, run, and workflow fields from the run context.

## Migrating from Promptfoo

[Section titled “Migrating from Promptfoo”](#migrating-from-promptfoo)

Runsight’s custom assertion contract is intentionally close to promptfoo-style Python assertions.

A promptfoo-style function body like this works unchanged:

custom/assertions/tone\_check.py

```python
def get_assert(output, context):
    config = context.get("config", {})
    return {
        "pass": output.startswith(config.get("prefix", "")),
        "score": 0.9,
        "reason": f"prefix={config.get('prefix', '')}",
    }
```

The main Runsight-specific additions are:

1. Put the code in `custom/assertions/<id>.py`
2. Add a matching manifest in `custom/assertions/<id>.yaml`
3. Reference it in workflow YAML as `custom:<id>`

Example:

Workflow assertion entry

```yaml
assertions:
  - type: custom:tone_check
    config:
      prefix: calm
```

Remember that `tone_check` comes from the filename, not the manifest `name`.

## Limitations

[Section titled “Limitations”](#limitations)

Current custom assertion support is intentionally narrow:

* Discovery only looks for YAML manifests in `custom/assertions/*.yaml`.
* The Python contract is only `def get_assert(output, context)`.
* Supported return contracts are only `bool` and `grading_result`.
* Manifest fields are fixed to `version`, `name`, `description`, `returns`, `source`, and optional `params`.
* Plugins run in a separate subprocess with a minimal environment and a 30 second timeout.
* API keys are not forwarded into that subprocess environment.
* Offline eval auto-discovers custom assertions only from workflow file paths, not raw YAML strings.
* Simple custom assertions (return `bool` or `grading_result`) run in a minimal subprocess with no API keys and no IPC access.

## LLM-Graded Assertions (`llm_judge`)

[Section titled “LLM-Graded Assertions (llm\_judge)”](#llm-graded-assertions-llm_judge)

For assertions that need to call an LLM to grade output (e.g., rubric-based evaluation, factual consistency checks), use the `llm_judge` assertion type. These assertions run through the same [process isolation](/docs/execution/process-isolation) path as regular LLM blocks --- the judge LLM call is proxied through the IPC channel with full budget enforcement and observability.

llm\_judge assertion example

```yaml
assertions:
  - type: llm_judge
    config:
      model: claude-haiku
      rubric: "Grade the output for factual accuracy and completeness."
```

Key differences from simple custom assertions:

* The `llm_judge` assertion runs inside an isolated subprocess with IPC access (not `_minimal_subprocess_env`)
* The judge’s LLM call goes through the `BudgetInterceptor` --- assertion costs count toward the block’s and workflow’s budget
* A `judge_soul` is constructed from the assertion config and used to call the LLM
* The `GradingResult` includes `assertion_type: "llm_judge"` and `metadata.judge_model` for traceability
* Simple `get_assert()` custom plugins still use the minimal subprocess with no IPC --- only `llm_judge` type assertions use the full isolation path

## Error Messages

[Section titled “Error Messages”](#error-messages)

These are the most common failure modes you will see.

### Manifest and discovery errors

[Section titled “Manifest and discovery errors”](#manifest-and-discovery-errors)

These happen before the assertion is registered:

* Missing or invalid required manifest fields
* Unsupported extra manifest fields
* Invalid `returns` value
* Built-in ID collision such as `contains.yaml`
* Invalid Python signature such as anything other than `def get_assert(output, context)`

### Runtime assertion errors

[Section titled “Runtime assertion errors”](#runtime-assertion-errors)

These return failing results instead of crashing the run:

* `Config validation failed: ...`
* `Custom assertion 'name' failed: plugin exploded`
* `Custom assertion 'name' declares returns: bool but get_assert returned 'dict'`
* `custom assertion plugin timed out after 30s`

### What happens on failure

[Section titled “What happens on failure”](#what-happens-on-failure)

In both offline eval and live API execution:

* The assertion result is recorded as failed
* The run continues
* Other assertions on the same block still produce their own results
* Live API runs still persist `eval_score`, `eval_passed`, and `eval_results`, and still emit `node_eval_complete` SSE events

<!-- Linear: RUN-769, RUN-794, RUN-795, RUN-796, RUN-797, RUN-798, RUN-799, RUN-800, RUN-801, RUN-391 -- last verified against codebase 2026-04-11 -->

# Eval Test Harness

> Write offline test cases in YAML and run them with fixture mode -- no LLM calls needed.

The eval test harness lets you define test cases directly in your workflow YAML file. Each case specifies inputs, optional fixture outputs, and expected assertions per block. Fixture mode skips all LLM calls, making tests fast, free, and deterministic.

## The eval YAML section

[Section titled “The eval YAML section”](#the-eval-yaml-section)

Add an `eval:` section at the top level of your workflow file, alongside `blocks:` and `workflow:`:

custom/workflows/research.yaml

```yaml
version: "1.0"
id: research
kind: workflow
blocks:
  analyze:
    type: code
    code: |
      def main(data):
          return "unused in fixture mode"
    assertions:
      - type: contains
        value: "analysis"


workflow:
  name: research
  entry: analyze
  transitions:
    - from: analyze
      to: null


eval:
  threshold: 0.8
  cases:
    - id: basic_research
      inputs:
        task_instruction: "Research LLMs"
      fixtures:
        analyze: "LLMs have transformed software development. This analysis covers key trends."
      expected:
        analyze:
          - type: contains
            value: "analysis"
```

## EvalSectionDef fields

[Section titled “EvalSectionDef fields”](#evalsectiondef-fields)

The `eval:` section is parsed as an `EvalSectionDef` model:

| Field       | Type                | Required | Default                                                        | Description                                                      |
| ----------- | ------------------- | -------- | -------------------------------------------------------------- | ---------------------------------------------------------------- |
| `threshold` | `float`             | no       | `None` in the schema, treated as `1.0` at runtime when omitted | Minimum aggregate score for the suite to pass. Range: 0.0 to 1.0 |
| `cases`     | `list[EvalCaseDef]` | yes      | —                                                              | At least one test case required (`min_length=1`)                 |

Case IDs must be unique within the eval section. Duplicates cause a validation error.

## EvalCaseDef fields

[Section titled “EvalCaseDef fields”](#evalcasedef-fields)

Each entry in `cases:` is an `EvalCaseDef`:

| Field         | Type                    | Required | Default | Description                                                                                 |
| ------------- | ----------------------- | -------- | ------- | ------------------------------------------------------------------------------------------- |
| `id`          | `str`                   | yes      | —       | Unique identifier for this test case                                                        |
| `description` | `str`                   | no       | `None`  | Human-readable description of what this case tests                                          |
| `inputs`      | `dict[str, any]`        | no       | `None`  | Input values passed to the executor                                                         |
| `fixtures`    | `dict[str, str]`        | no       | `None`  | Block ID to output string mapping. Skips LLM calls for these blocks                         |
| `expected`    | `dict[str, list[dict]]` | no       | `None`  | Block ID to list of assertion configs. These assertions are evaluated against block outputs |

## Fixture mode

[Section titled “Fixture mode”](#fixture-mode)

When a case provides `fixtures` that cover every block listed in `expected`, the eval runner operates in **fixture mode**:

* No executor is called (no LLM calls, no API calls)
* A `WorkflowState` is built directly from fixture strings
* Assertions run against the fixture values

This makes tests instant and free. You can run hundreds of cases without spending tokens.

Fixture mode -- all expected blocks have fixtures

```yaml
eval:
  cases:
    - id: fixture_only
      fixtures:
        analyze: "The LLM landscape is evolving rapidly."
        summarize: "Summary: LLMs are improving."
      expected:
        analyze:
          - type: contains
            value: "LLM"
        summarize:
          - type: starts-with
            value: "Summary"
```

If a case has `expected` blocks without matching fixtures, the runner requires an executor callback. If no executor is provided, it raises a `RuntimeError`.

Note

Custom assertions are supported in `expected` just like built-in assertions, but scanner-based auto-discovery only happens when `run_eval()` is given a workflow file path. Passing raw YAML text does not scan `custom/assertions`.

## Running offline evals

[Section titled “Running offline evals”](#running-offline-evals)

When you run evals, the harness loads your workflow YAML, finds the `eval:` section, and executes each case. Fixture-only cases run instantly with no LLM calls. The result tells you whether the suite passed and gives per-case scores.

### Result types

[Section titled “Result types”](#result-types)

The suite result contains:

| Field          | Type                   | Description                                                          |
| -------------- | ---------------------- | -------------------------------------------------------------------- |
| `passed`       | `bool`                 | `True` if `score >= threshold`                                       |
| `score`        | `float`                | Average of all case scores                                           |
| `threshold`    | `float`                | From `eval.threshold`, or `1.0` at runtime when the field is omitted |
| `case_results` | `list[EvalCaseResult]` | Per-case breakdown                                                   |

Each `EvalCaseResult` contains:

| Field           | Type                          | Description                                 |
| --------------- | ----------------------------- | ------------------------------------------- |
| `case_id`       | `str`                         | Matches `id` from the YAML case             |
| `passed`        | `bool`                        | `True` if all block assertion suites passed |
| `score`         | `float`                       | Average of block aggregate scores           |
| `block_results` | `dict[str, AssertionsResult]` | Block ID to assertion results               |

### Score computation

[Section titled “Score computation”](#score-computation)

1. Each assertion produces a score between `0.0` and `1.0`
2. Block score = weighted average of assertion scores
3. Case score = average of block scores
4. Suite score = average of case scores
5. Suite passes if `suite_score >= threshold`

## Executor mode

[Section titled “Executor mode”](#executor-mode)

For cases that need live execution (no fixtures for some blocks), `run_eval()` awaits a caller-supplied `executor(raw, inputs)` function that must return a `WorkflowState`. That executor can call the real workflow runtime, a test double, or any other harness you provide. `run_eval()` itself does not automatically run the full execution pipeline.

## Mixing fixture and executor cases

[Section titled “Mixing fixture and executor cases”](#mixing-fixture-and-executor-cases)

A single eval section can contain both fixture-only and executor-required cases. The runner decides per-case:

Mixed cases

```yaml
eval:
  threshold: 0.5
  cases:
    - id: fast_fixture_test
      fixtures:
        analyze: "LLMs are powerful language models."
      expected:
        analyze:
          - type: contains
            value: "LLM"
    - id: live_execution_test
      inputs:
        task_instruction: "Research transformers"
      expected:
        analyze:
          - type: contains
            value: "transformer"
```

The fixture case runs without an executor. The live case requires one. If no executor is provided, `run_eval()` raises a `RuntimeError` when it reaches the first executor-required case; it does not return partial suite results.

<!-- Linear: RUN-694, RUN-695, RUN-699, RUN-700, RUN-769, RUN-800, RUN-801 -- last verified against codebase 2026-04-10 -->

# Regressions

> How Runsight detects quality regressions by comparing runs -- assertion failures, cost spikes, and score drops.

Regressions are quality problems that appear when comparing consecutive runs of the same workflow. Runsight automatically detects three types of regression by comparing each production run against its predecessor.

## What counts as a regression

[Section titled “What counts as a regression”](#what-counts-as-a-regression)

The `EvalService._detect_node_regressions` method compares two matching nodes (same `node_id` and `soul_version`) across consecutive production runs. It flags three conditions:

| Type                   | Condition                                    | Delta payload                                              |
| ---------------------- | -------------------------------------------- | ---------------------------------------------------------- |
| `assertion_regression` | `eval_passed` changed from `True` to `False` | `{eval_passed: false, baseline_eval_passed: true}`         |
| `cost_spike`           | Cost increased more than 20% vs previous run | `{cost_pct: <percentage>, baseline_cost: <previous cost>}` |
| `quality_drop`         | `eval_score` dropped by more than 0.1        | `{score_delta: <negative number>}`                         |

Note

Only **production runs** are compared — runs on the `main` branch with a source of `manual`, `api`, `webhook`, or `schedule`. Direct API runs use the shipped `api` source. Webhook and schedule sources are reserved and are not part of RUN-85. Simulation branches are excluded.

## How eval\_score and eval\_passed work

[Section titled “How eval\_score and eval\_passed work”](#how-eval_score-and-eval_passed-work)

These fields live on the `RunNode` entity and are populated by the `EvalObserver` when a block with [assertions](/docs/evaluation/assertions) completes:

* **`eval_score`** (`Optional[float]`): The weighted average of all assertion scores for that node. A value of `1.0` means every assertion scored perfectly; `0.0` means all failed.
* **`eval_passed`** (`Optional[bool]`): `True` only if every individual assertion passed. A node can have a high score but still fail if one assertion did not pass.
* **`eval_results`** (`Optional[Dict]`): Detailed breakdown with per-assertion `passed`, `score`, and `reason`, plus `type` when the assertion handler sets one. Custom assertions do; many built-in deterministic assertions leave it as `null`.

When a block has no assertions, all three fields are `None` and the node is excluded from regression detection.

## Run-to-run comparison

[Section titled “Run-to-run comparison”](#run-to-run-comparison)

Regression detection works at the run level via the `EvalService`:

**For a single run** (`GET /api/runs/{run_id}/regressions`):

1. Find all production runs for the same workflow, ordered by `created_at`
2. Identify the previous production run before the target run
3. Match nodes between the two runs by `(node_id, soul_version)`
4. For each matching pair, check the three regression conditions
5. Return `{count: N, issues: [...]}` — or `{count: 0, issues: []}` if this is the first run

**For a workflow** (`GET /api/workflows/{id}/regressions`):

1. Get all production runs for the workflow, ordered by `created_at`
2. For each consecutive pair, compare matching nodes
3. Each issue includes `run_id` and `run_number` to identify which run introduced it
4. Return the aggregate across all run pairs

The first production run of a workflow always has zero regressions since there is no baseline to compare against.

## Regression issue structure

[Section titled “Regression issue structure”](#regression-issue-structure)

Each regression issue in the response contains:

| Field        | Type   | Description                                                                     |
| ------------ | ------ | ------------------------------------------------------------------------------- |
| `node_id`    | `str`  | The block that regressed                                                        |
| `node_name`  | `str`  | Display label for the node when available; otherwise it falls back to `node_id` |
| `type`       | `str`  | One of `assertion_regression`, `cost_spike`, `quality_drop`                     |
| `delta`      | `dict` | Type-specific comparison data (see table above)                                 |
| `run_id`     | `str`  | (workflow endpoint only) Which run introduced this regression                   |
| `run_number` | `int`  | (workflow endpoint only) Sequential run number                                  |

## Pass rate tracking

[Section titled “Pass rate tracking”](#pass-rate-tracking)

Run-level pass rate is tracked via `eval_pass_pct` on the `RunResponse` schema. This field represents the percentage of eval-bearing nodes that passed in a run. The `RunResponse` also includes:

| Field              | Type              | Description                                                                            |
| ------------------ | ----------------- | -------------------------------------------------------------------------------------- |
| `eval_pass_pct`    | `Optional[float]` | Percentage of nodes with `eval_passed = True`                                          |
| `eval_score_avg`   | `Optional[float]` | Average `eval_score` across all eval-bearing nodes                                     |
| `regression_count` | `Optional[int]`   | Number of regression issues detected for this run                                      |
| `regression_types` | `list[str]`       | List of regression type strings found (e.g., `["assertion_regression", "cost_spike"]`) |

These fields appear in the runs list (`GET /api/runs`) and single run detail (`GET /api/runs/{id}`), enabling dashboards to show pass rate trends across runs.

## Regressions in the UI

[Section titled “Regressions in the UI”](#regressions-in-the-ui)

The Run Detail view displays a priority banner when regressions are detected. The banner shows the regression count (e.g., “3 regressions found”) and appears at the top of the run detail page.

The frontend fetches regression data via `useRunRegressions(runId)` and displays the count. The workflow-level regression query (`useWorkflowRegressions(workflowId)`) provides cross-run regression history.

The regression response schema on the frontend validates three regression types: `assertion_regression`, `cost_spike`, and `quality_drop`. Unknown types are rejected by the `WorkflowRegressionSchema` validator.

## Attention items

[Section titled “Attention items”](#attention-items)

The `EvalService.get_attention_items` method scans production runs from the last 24 hours and surfaces regressions as attention items on the dashboard. It flags the same three conditions as the regression endpoints, plus a `new_baseline` info item for the first production run of a soul version. Items are sorted by severity (warnings before info) and recency.

<!-- Linear: RUN-555, RUN-558, RUN-769, RUN-801, RUN-943 -- last verified against codebase 2026-04-26 -->

# Transform Hooks

> Pre-process block output before assertion evaluation using json_path transforms.

When a block produces structured output (like JSON), you often want to assert on a specific field rather than the entire output string. Transform hooks let you extract a sub-field before the assertion runs.

## Add a transform to an assertion

[Section titled “Add a transform to an assertion”](#add-a-transform-to-an-assertion)

Set the `transform` field on any assertion config. The transform runs before handler lookup and before the assertion evaluator sees the output, so it works with built-in assertions, `custom:*` assertions, and `not-custom:*` assertions:

custom/workflows/extract-check.yaml

```yaml
blocks:
  classify:
    type: code
    code: |
      def main(data):
          return '{"sentiment": "positive"}'
    assertions:
      - type: contains-any
        value: ["positive", "negative", "neutral"]
        transform: "json_path:$.sentiment"
```

Without the transform, the `contains-any` assertion would check the entire LLM output string. With `transform: "json_path:$.sentiment"`, it checks only the extracted value of the `sentiment` field.

## json\_path transform

[Section titled “json\_path transform”](#json_path-transform)

The only supported transform type is `json_path`. The syntax is:

```plaintext
json_path:<JSONPath expression>
```

The engine:

1. Parses the block output as JSON (using `json.loads`)
2. Evaluates the JSONPath expression (using the `jsonpath_ng` library)
3. Extracts the first match
4. Converts the result to a string if it is not already one
5. Passes the extracted string to the assertion evaluator

### Supported JSONPath syntax

[Section titled “Supported JSONPath syntax”](#supported-jsonpath-syntax)

Runsight uses the `jsonpath_ng` library. Common patterns:

| Expression        | Selects                                                  |
| ----------------- | -------------------------------------------------------- |
| `$.field`         | Top-level field                                          |
| `$.nested.field`  | Nested field                                             |
| `$.items[0]`      | First array element                                      |
| `$.items[*].name` | `name` field from every array element (first match used) |
| `$..field`        | Recursive descent — finds `field` at any depth           |

## Examples

[Section titled “Examples”](#examples)

### Check a sentiment label in a JSON response

[Section titled “Check a sentiment label in a JSON response”](#check-a-sentiment-label-in-a-json-response)

Extract and assert on a JSON field

```yaml
blocks:
  analyze:
    type: code
    code: |
      def main(data):
          return '{"outlook": "bullish", "confidence": 0.92}'
    assertions:
      - type: contains-any
        value: ["bullish", "bearish", "neutral"]
        transform: "json_path:$.outlook"
```

If the block output is `{"outlook": "bullish", "confidence": 0.92}`, the assertion receives `"bullish"` and passes.

### Validate a nested score

[Section titled “Validate a nested score”](#validate-a-nested-score)

Check a numeric field with equals

```yaml
blocks:
  scorer:
    type: code
    code: |
      def main(data):
          return '{"rating": 4, "explanation": "Good quality"}'
    assertions:
      - type: regex
        value: "^[1-5]$"
        transform: "json_path:$.rating"
```

If the block output is `{"rating": 4, "explanation": "Good quality"}`, the transform extracts `4`, converts it to the string `"4"`, and the regex assertion checks it.

### Combine with negation

[Section titled “Combine with negation”](#combine-with-negation)

Transforms work with negated assertion types:

Negated assertion with transform

```yaml
assertions:
  - type: not-contains
    value: "error"
    transform: "json_path:$.status"
```

## Error handling

[Section titled “Error handling”](#error-handling)

The transform returns a failing `GradingResult` (score 0.0, passed false) in these cases:

| Condition                                   | Reason message                                             |
| ------------------------------------------- | ---------------------------------------------------------- |
| Output is not valid JSON                    | `"Transform json_path failed: output is not valid JSON"`   |
| JSONPath expression matches nothing         | `"Transform json_path: path '<path>' not found in output"` |
| Unknown transform format (no `:` separator) | `"Unknown transform format: '<value>'"`                    |
| Unknown transform type (not `json_path`)    | `"Unknown transform type: '<type>'"`                       |

When a transform fails, the assertion itself does not run. The failing `GradingResult` from the transform is used directly.

Caution

A single transform extracts only the **first match** from the JSONPath expression. If your path matches multiple values (e.g., `$.items[*].name`), only the first one is used for that assertion. To check multiple fields, add separate assertions each with its own `transform` — see [Assertion chaining](/docs/evaluation/assertions#assertion-chaining).

<!-- Linear: RUN-685, RUN-695, RUN-769, RUN-801 -- last verified against codebase 2026-04-10 -->

# Budget & Limits

> Cost caps, token caps, timeouts, and warn/kill modes — the limits YAML section for workflow and block-level budget enforcement.

The `limits` section controls how much a workflow or individual block is allowed to spend in cost, tokens, and wall-clock time. When a limit is breached, the engine either kills execution immediately or emits a warning and continues, depending on the `on_exceed` mode.

Caution

Budget enforcement is backend-only. There is no frontend UI for configuring or visualizing limits yet. You set limits directly in the workflow YAML.

## Workflow-level limits

[Section titled “Workflow-level limits”](#workflow-level-limits)

Add a `limits` section at the top level of your workflow YAML:

custom/workflows/research-pipeline.yaml

```yaml
version: "1.0"
id: research-pipeline
kind: workflow
limits:
  cost_cap_usd: 2.50
  token_cap: 100000
  max_duration_seconds: 300
  on_exceed: fail
  warn_at_pct: 0.8
workflow:
  name: research-pipeline
  entry: step_one
blocks:
  step_one:
    type: linear
    soul_ref: analyst
```

### WorkflowLimitsDef fields

[Section titled “WorkflowLimitsDef fields”](#workflowlimitsdef-fields)

| Field                  | Type               | Default  | Constraints  | Description                                   |
| ---------------------- | ------------------ | -------- | ------------ | --------------------------------------------- |
| `cost_cap_usd`         | `float?`           | `None`   | `>= 0.0`     | Maximum total LLM cost in USD                 |
| `token_cap`            | `int?`             | `None`   | `>= 1`       | Maximum total tokens (prompt + completion)    |
| `max_duration_seconds` | `int?`             | `None`   | `1 -- 86400` | Wall-clock timeout for the entire workflow    |
| `on_exceed`            | `"warn" \| "fail"` | `"fail"` | ---          | What happens when a cap is breached           |
| `warn_at_pct`          | `float`            | `0.8`    | `0.0 -- 1.0` | Percentage threshold for early warning events |

All fields are optional. If you omit `limits` entirely, no budget enforcement is applied.

## Per-block limits

[Section titled “Per-block limits”](#per-block-limits)

Individual blocks can have their own `limits` section:

custom/workflows/research-pipeline.yaml

```yaml
blocks:
  expensive_step:
    type: linear
    soul_ref: analyst
    limits:
      cost_cap_usd: 1.00
      token_cap: 50000
      max_duration_seconds: 120
      on_exceed: fail
```

### BlockLimitsDef fields

[Section titled “BlockLimitsDef fields”](#blocklimitsdef-fields)

| Field                  | Type               | Default  | Constraints  | Description                         |
| ---------------------- | ------------------ | -------- | ------------ | ----------------------------------- |
| `cost_cap_usd`         | `float?`           | `None`   | `>= 0.0`     | Maximum LLM cost for this block     |
| `token_cap`            | `int?`             | `None`   | `>= 1`       | Maximum tokens for this block       |
| `max_duration_seconds` | `int?`             | `None`   | `1 -- 86400` | Wall-clock timeout for this block   |
| `on_exceed`            | `"warn" \| "fail"` | `"fail"` | ---          | What happens when a cap is breached |

Note

Block-level limits do **not** have a `warn_at_pct` field. Only workflow-level limits support the warning threshold.

## Warn mode vs kill mode

[Section titled “Warn mode vs kill mode”](#warn-mode-vs-kill-mode)

The `on_exceed` field controls what happens when a budget cap is breached:

### Kill mode (`on_exceed: "fail"`)

[Section titled “Kill mode (on\_exceed: "fail")”](#kill-mode-on_exceed-fail)

The default. When any cap is exceeded, the engine raises a `BudgetKilledException` immediately. The exception includes structured metadata:

* `scope` --- `"block"` or `"workflow"`
* `block_id` --- which block triggered the breach (if block-scoped)
* `limit_kind` --- `"cost_usd"`, `"token_cap"`, or `"timeout"`
* `limit_value` --- the configured cap
* `actual_value` --- the value that exceeded the cap

**Error route interaction:** If the block that triggered the exception has an `error_route` configured, the `BudgetKilledException` is caught by the workflow’s generic exception handler. The block result is written with `exit_handle: "error"` and error metadata, and execution **continues** on the error route target block. Only when no `error_route` exists does the exception propagate and terminate the run with `status: failed`.

**Flow-level timeouts are the exception:** When a workflow-level `max_duration_seconds` fires, the resulting `BudgetKilledException` is raised **outside** the block execution loop. It cannot be caught by any block’s `error_route` and always terminates the run unconditionally.

### Warn mode (`on_exceed: "warn"`)

[Section titled “Warn mode (on\_exceed: "warn")”](#warn-mode-on_exceed-warn)

When a cap is exceeded, the engine logs a warning but execution continues. The run can finish normally even after exceeding a budget cap. Use this for soft budgets where you want visibility but not hard stops.

Warn mode example

```yaml
limits:
  cost_cap_usd: 1.00
  on_exceed: warn
  warn_at_pct: 0.8
```

With this configuration, a warning event is emitted at 80% of the cost cap ($0.80), and another when the cap is exceeded ($1.00+), but execution continues.

## How enforcement works

[Section titled “How enforcement works”](#how-enforcement-works)

Budget enforcement tracks the active budget session per async task. Every LLM call passes through a single chokepoint where cost and token limits are checked --- whether the block runs in the engine process or in an isolated subprocess.

### The enforcement chain

[Section titled “The enforcement chain”](#the-enforcement-chain)

1. **Workflow start:** If the workflow has `limits`, a budget session is created and set as the active budget for the run.

2. **Block start:** If a block has `limits`, a child budget session is created with the workflow session as its parent. The child replaces the active budget for the duration of that block.

3. **LLM call returns:** After each LLM call, the session records the cost and tokens. If the session has a parent, costs propagate up the chain automatically.

4. **Cap check:** After recording, the engine walks the entire parent chain. If any session (block or workflow) has exceeded its cap with `on_exceed: "fail"`, execution is killed immediately.

5. **Block end:** The block’s budget session is removed and the workflow session is restored.

### Enforcement across process isolation

[Section titled “Enforcement across process isolation”](#enforcement-across-process-isolation)

LLM blocks run in isolated subprocesses that have no direct access to API keys or budget state. Budget enforcement still works because every LLM call from the subprocess is proxied through an IPC channel back to the engine, where a `BudgetInterceptor` enforces caps:

* **Before each LLM call:** The interceptor checks the active budget session. If the budget is exceeded, the call is rejected before the LLM provider is contacted --- no money spent.
* **After each LLM call:** The interceptor accrues the reported cost and tokens to the budget session. Costs propagate up the parent chain automatically.
* **Budget kill propagation:** When a budget cap is breached, a `BudgetKilledException` is serialized back to the subprocess, which writes it into the `ResultEnvelope`. The engine deserializes and re-raises it in the main process.

Block-level and workflow-level caps both work across the isolation boundary. The subprocess never sees or manipulates budget state directly --- the engine owns it entirely.

See [Process Isolation](/docs/execution/process-isolation) for the full architecture.

### Parent propagation

[Section titled “Parent propagation”](#parent-propagation)

When a block has its own limits, a child budget session is created with the workflow session as its parent. Every cost recorded on the child **also increments the parent’s counters** recursively up the chain. After recording, the engine walks the entire parent chain, so both the block’s caps and the workflow’s caps are enforced on every LLM call:

```plaintext
Block accrues $0.50, 1000 tokens
  → Block session: cost=$0.50, tokens=1000
  → Parent (workflow) session: cost=$0.50, tokens=1000 (propagated)
```

This means a workflow with `cost_cap_usd: 2.00` will kill the run even if the individual block has no limit, as long as the workflow’s total cost exceeds $2.00. Sub-flow budgets are **not** independent --- the parent workflow’s caps are always enforced because costs propagate upward through the parent chain.

### Timeout enforcement

[Section titled “Timeout enforcement”](#timeout-enforcement)

Timeouts work differently from cost and token caps:

* **Workflow timeout:** The engine wraps the main execution loop with the configured `max_duration_seconds`. If the timeout fires, execution is killed immediately with a budget exception (`limit_kind="timeout"`).

* **Block timeout:** Each block is individually wrapped with its own `max_duration_seconds`. Block timeouts are independent of the workflow timeout.

### Dispatch branch isolation

[Section titled “Dispatch branch isolation”](#dispatch-branch-isolation)

When a `dispatch` block fans out to multiple exit branches, each branch gets an **isolated child session** created via `create_isolated_child(branch_id=exit_id)`. The child inherits the parent’s exact caps (`cost_cap_usd`, `token_cap`, `max_duration_seconds`, `on_exceed`) but has **no parent pointer** --- it accumulates costs independently. This prevents concurrent branches from sharing mutable budget state during execution.

Each child is individually capped at the parent’s full cap value. For example, if the workflow cap is $5.00, each branch is independently allowed up to $5.00 --- not $5.00 divided by the number of branches. This means a single branch can hit the cap on its own before the others finish.

After all branches complete via `asyncio.gather`, each child’s totals are **reconciled** back to the parent session via `reconcile_child()`, which adds the child’s cost and token totals to the parent. Then the parent session’s caps are checked with `check_or_raise()`.

```plaintext
Parent session: cost_cap=$5.00
  ├── Branch A (isolated, cap=$5.00): cost=$1.00
  ├── Branch B (isolated, cap=$5.00): cost=$2.00
  ├── Reconciliation: parent cost += $1.00 + $2.00 = $3.00
  └── Parent cap check: $3.00 ≤ $5.00 → passes
```

## Complete example

[Section titled “Complete example”](#complete-example)

custom/workflows/budget-example.yaml

```yaml
version: "1.0"
id: budget-example
kind: workflow
limits:
  cost_cap_usd: 5.00
  token_cap: 200000
  max_duration_seconds: 600
  on_exceed: fail
  warn_at_pct: 0.8
workflow:
  name: budget-example
  entry: research
  transitions:
    - from: research
      to: summarize
    - from: summarize
blocks:
  research:
    type: linear
    soul_ref: researcher
    limits:
      cost_cap_usd: 3.00
      max_duration_seconds: 300
      on_exceed: fail
  summarize:
    type: linear
    soul_ref: writer
    limits:
      cost_cap_usd: 1.00
      on_exceed: warn
```

In this example:

* The workflow will hard-stop at $5.00 total or 200k tokens or 10 minutes.
* The `research` block will hard-stop at $3.00 or 5 minutes.
* The `summarize` block will warn at $1.00 but continue running.
* Both blocks’ costs propagate to the workflow total.

<!-- Linear: RUN-708, RUN-717, RUN-732, RUN-391 — last verified against codebase 2026-04-11 -->

# Error Handling

> error_route, on_error modes, retry_config, and how errors propagate through the workflow execution lifecycle.

Runsight provides three mechanisms for handling errors in workflows: `error_route` for redirecting execution after a block failure, `on_error` for controlling sub-workflow failure behavior, and `retry_config` for automatic retry with backoff.

## error\_route

[Section titled “error\_route”](#error_route)

The `error_route` field on any block redirects execution to a specific block when an error occurs, instead of failing the entire workflow.

custom/workflows/error-example.yaml

```yaml
blocks:
  risky_step:
    type: linear
    soul_ref: analyst
    error_route: fallback_step
  fallback_step:
    type: linear
    soul_ref: writer
```

### How error\_route works

[Section titled “How error\_route works”](#how-error_route-works)

When a block with `error_route` raises an exception:

1. The exception is caught by the workflow’s main execution loop.
2. A `BlockResult` is created for the failed block with `exit_handle: "error"` and the error details stored in metadata (`error_type`, `error_message`, `block_id`).
3. The error info is also written to `shared_memory` under `__error__{block_id}`.
4. The execution queue is cleared and replaced with the error route target.
5. Execution continues from the error route block.

The error route block can read the error details from shared memory:

Accessing error context in the fallback block

```yaml
blocks:
  risky_step:
    type: linear
    soul_ref: analyst
    error_route: handle_error
  handle_error:
    type: linear
    soul_ref: error_handler
```

The `handle_error` block’s soul can access `__error__risky_step` in shared memory, which contains `{"type": "...", "message": "..."}`.

### Soft-error routing

[Section titled “Soft-error routing”](#soft-error-routing)

Error routing also activates for **soft errors** --- blocks that complete normally but produce a `BlockResult` with `exit_handle: "error"`. This happens when a `workflow` block uses `on_error: "catch"` and the child workflow fails. The parent block completes (no exception raised), but its result signals an error. If the parent block has an `error_route`, execution is redirected.

Note

The `error_route` field must reference a block that exists in the workflow. The parser validates this during workflow construction, and the runtime validates it again before execution.

## on\_error (workflow blocks)

[Section titled “on\_error (workflow blocks)”](#on_error-workflow-blocks)

The `on_error` field is specific to `workflow` blocks (type: `"workflow"`). It controls what happens when a child workflow fails.

| Value     | Behavior                                                                                                                                                                           |
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `"raise"` | **Default.** The child’s exception propagates to the parent. The parent block fails, and normal error handling applies (error\_route if set, otherwise the parent workflow fails). |
| `"catch"` | The exception is swallowed. The parent block completes with `exit_handle: "error"` and a `BlockResult` containing the child error details. Execution continues from the parent.    |

Catch mode example

```yaml
blocks:
  optional_enrichment:
    type: workflow
    workflow_ref: enrichment-pipeline
    on_error: catch
    error_route: skip_enrichment
    inputs:
      topic: "shared_memory.topic"
    outputs:
      "shared_memory.enriched": summary
  skip_enrichment:
    type: linear
    soul_ref: writer
```

In this example:

1. If `enrichment-pipeline` fails, the exception is caught (`on_error: "catch"`).
2. The `optional_enrichment` block completes with `exit_handle: "error"`.
3. Because `error_route` is set, execution redirects to `skip_enrichment`.
4. The workflow continues instead of failing.

### Soft error detection in catch mode

[Section titled “Soft error detection in catch mode”](#soft-error-detection-in-catch-mode)

When `on_error: "catch"` is set, the workflow block also detects soft errors in the child’s results. If any child block produced a `BlockResult` with `exit_handle: "error"` (even without raising an exception), the parent block treats this as a failure and returns an error `BlockResult`. This prevents silently swallowing errors that were caught by error\_route within the child workflow.

## retry\_config

[Section titled “retry\_config”](#retry_config)

The `retry_config` field adds automatic retry with configurable backoff to any block.

Retry example

```yaml
blocks:
  flaky_api_call:
    type: linear
    soul_ref: analyst
    retry_config:
      max_attempts: 3
      backoff: exponential
      backoff_base_seconds: 2.0
      non_retryable_errors:
        - AuthenticationError
```

### RetryConfig fields

[Section titled “RetryConfig fields”](#retryconfig-fields)

| Field                  | Type                       | Default   | Constraints   | Description                                     |
| ---------------------- | -------------------------- | --------- | ------------- | ----------------------------------------------- |
| `max_attempts`         | `int`                      | `3`       | `1 -- 20`     | Total attempts (1 = no retry)                   |
| `backoff`              | `"fixed" \| "exponential"` | `"fixed"` | ---           | Backoff strategy between attempts               |
| `backoff_base_seconds` | `float`                    | `1.0`     | `0.1 -- 60.0` | Base delay between retries                      |
| `non_retryable_errors` | `list[str]?`               | `None`    | ---           | Exception type names that should not be retried |

### Backoff strategies

[Section titled “Backoff strategies”](#backoff-strategies)

**Fixed backoff** (`backoff: "fixed"`): Waits `backoff_base_seconds` between every retry attempt.

```plaintext
Attempt 1 → fail → wait 1.0s → Attempt 2 → fail → wait 1.0s → Attempt 3
```

**Exponential backoff** (`backoff: "exponential"`): Doubles the delay each attempt using `backoff_base_seconds * 2^(attempt-1)`.

```plaintext
Attempt 1 → fail → wait 2.0s → Attempt 2 → fail → wait 4.0s → Attempt 3
```

### Retry behavior

[Section titled “Retry behavior”](#retry-behavior)

* Each retry starts from the **original pre-retry state**. Failed-attempt messages are never carried over to the next attempt.
* On success after retries, the block result includes retry metadata in `shared_memory` under `__retry__{block_id}`:

```json
{
  "attempt": 2,
  "max_attempts": 3,
  "last_error": "Connection timeout",
  "last_error_type": "TimeoutError",
  "total_retries": 1
}
```

* If an error’s type name matches an entry in `non_retryable_errors`, the error is raised immediately without further retry attempts.
* `KeyboardInterrupt` and `SystemExit` are never retried.

## Error propagation lifecycle

[Section titled “Error propagation lifecycle”](#error-propagation-lifecycle)

Here is how errors flow through the execution engine:

1. **Block raises an exception** during `execute_block()`.

2. **Retry check:** If `retry_config` is set and `max_attempts > 1`, the block is retried according to the backoff strategy. Non-retryable errors skip this step.

3. **Error route check:** If `error_route` is set on the block, the exception is caught, a `BlockResult` with error metadata is written, and execution redirects to the error route target.

4. **Workflow-level propagation:** If no error\_route is set, the exception propagates up to the workflow runner. If the workflow is a child (running inside a `workflow` block), the parent’s `on_error` determines what happens next.

5. **Terminal state:** If the exception reaches the top-level workflow, the run is marked as `failed` with the error message and traceback stored on the `Run` record.

### Combining error\_route with retry\_config

[Section titled “Combining error\_route with retry\_config”](#combining-error_route-with-retry_config)

You can use both on the same block. Retries are attempted first. If all retry attempts are exhausted and the block still fails, the error\_route takes over:

Retry then redirect

```yaml
blocks:
  api_call:
    type: linear
    soul_ref: analyst
    retry_config:
      max_attempts: 3
      backoff: exponential
      backoff_base_seconds: 1.0
    error_route: handle_api_failure
  handle_api_failure:
    type: linear
    soul_ref: error_handler
```

## Block timeout

[Section titled “Block timeout”](#block-timeout)

Every block has a `timeout_seconds` field (default: `300`, range: `1 -- 3600`) separate from the budget `limits.max_duration_seconds`. If the block execution exceeds this timeout, a `BudgetKilledException` is raised with `limit_kind="timeout"`. This exception can be caught by `error_route` or `retry_config` like any other error.

See [Budget & Limits](/docs/execution/budget-and-limits) for the full budget enforcement model.

<!-- Linear: RUN-554, RUN-590, RUN-708 — last verified against codebase 2026-04-07 -->

# Fork Recovery

> Fork from a failed run to iterate on your workflow without losing the original run history.

Fork recovery lets you take a failed (or completed) run, read the exact YAML that executed, and create a new draft workflow from it. You iterate on the draft, fix the issue, and run again --- all without modifying the original workflow or losing the run history.

## When to fork

[Section titled “When to fork”](#when-to-fork)

Fork is the primary recovery path when a run fails. Instead of editing the live workflow and hoping you remember what changed, forking gives you:

* The **exact YAML snapshot** from the failed run’s `commit_sha`.
* A **new draft workflow** that is independent of the original.
* The original run history and the original workflow remain untouched.

## How to fork a run

[Section titled “How to fork a run”](#how-to-fork-a-run)

### From the GUI

[Section titled “From the GUI”](#from-the-gui)

1. Open any completed or failed run from the **Runs** page.
2. The run detail view is read-only --- you cannot edit the YAML directly.
3. Click the **Fork** button in the topbar header.
4. Runsight creates a new draft workflow named `drft-{slug}-{short-id}` (e.g., `drft-research-pipeline-a3f1`).
5. The editor opens the new draft, ready for you to modify and re-run.

Caution

The Fork button is **disabled** while a run is still active (`pending` or `running`). Wait for the run to finish before forking. It is also disabled if the run has no `commit_sha` (snapshot unavailable).

### What happens under the hood

[Section titled “What happens under the hood”](#what-happens-under-the-hood)

1. The frontend reads the workflow YAML at the run’s commit SHA via `GET /api/git/file?ref={commit_sha}&path=custom/workflows/{workflow_id}.yaml`.
2. It parses the YAML and sets `enabled: false` so the draft does not appear as a live workflow.
3. It calls `POST /api/workflows` with `commit: false` to create the draft without auto-committing it. The draft appears as an uncommitted workflow.
4. The browser navigates to the new draft in edit mode.

No dedicated fork endpoint exists on the backend. The fork flow is composed entirely from existing primitives: git file read, workflow creation, and navigation.

### From the API

[Section titled “From the API”](#from-the-api)

You can replicate the fork flow manually:

Step 1: Read the YAML from the failed run's commit

```bash
curl "http://localhost:8321/api/git/file?ref=abc1234&path=custom/workflows/research-pipeline.yaml"
```

Step 2: Create a new draft workflow

```bash
curl -X POST http://localhost:8321/api/workflows \
  -H "Content-Type: application/json" \
  -d '{
    "name": "drft-research-pipeline-x9k2",
    "yaml": "... modified YAML ...",
    "commit": false
  }'
```

## Fork naming convention

[Section titled “Fork naming convention”](#fork-naming-convention)

Forked drafts follow the pattern:

```plaintext
drft-{slugified-workflow-name}-{4-char-random}
```

The slug is lowercase, non-alphanumeric characters replaced with hyphens, consecutive hyphens collapsed. The random suffix uses a 4-character alphanumeric string from `Math.random().toString(36)`.

Examples:

* `drft-research-pipeline-a3f1`
* `drft-customer-onboarding-zk92`

The `drft-` prefix signals that this workflow is a draft fork, not a production workflow.

## Priority banner

[Section titled “Priority banner”](#priority-banner)

The canvas uses a **PriorityBanner** component to surface contextual alerts. The banner supports three condition types in priority order:

1. **explore** --- informational banner (info styling, dismissible with localStorage persistence).
2. **uncommitted** --- warning banner when the workflow has unsaved changes (warning styling, session-scoped dismiss).
3. **regressions** --- warning banner when eval regressions are detected (warning styling, session-scoped dismiss).

Only the highest-priority active banner is shown at a time. If you dismiss the top-priority banner, lower-priority banners do **not** cascade into view --- the dismiss is sticky for that session.

## Fork recovery workflow

[Section titled “Fork recovery workflow”](#fork-recovery-workflow)

The typical fork recovery loop:

1. A production run **fails** on the main branch.
2. You open the failed run from the Runs page.
3. You review the run detail --- block outputs, errors, eval results --- in read-only mode.
4. You click **Fork** to create a draft from the exact YAML that failed.
5. You edit the draft to fix the issue (adjust prompts, change transitions, modify limits).
6. You run the draft as a **simulation run** (since it has uncommitted changes).
7. If the simulation succeeds, you **save** the draft (commit to main) or copy the changes back to the original workflow.

This preserves the full audit trail: the original run, the fork point, and the iteration history are all distinct records.

<!-- Linear: RUN-554, RUN-557, RUN-564, RUN-590, RUN-595 — last verified against codebase 2026-04-07 -->

# Git Integration

> Save = commit to main. Dirty runs create simulation branches. Every run snapshots the YAML that executed.

Runsight is git-native by design. Git is not optional --- it is required for the workflow persistence model to function. Every save is a commit, every run records a commit SHA, and simulation branches capture dirty state for reproducible execution.

## Git auto-initialization

[Section titled “Git auto-initialization”](#git-auto-initialization)

When Runsight starts and no git repository exists at the project root, it automatically initializes one:

1. Runs `git init` in the project base path.
2. Configures a local git identity (`Runsight <runsight@localhost>`).
3. Stages all project files and creates an initial commit: `"Initial Runsight project"`.

This happens inside `scaffold_project()` during project detection. The result is that every Runsight project is always inside a git repository from the first launch.

## Save = commit to main

[Section titled “Save = commit to main”](#save--commit-to-main)

When you click **Save** in the canvas topbar, the workflow YAML is committed directly to the `main` branch. There is no separate “save” concept --- saving and committing are the same operation.

The commit flow:

1. The GUI calls `POST /api/git/commit` with the changed files and a commit message.
2. The git service stages the specified files (or all files if none are specified).
3. A commit is created on the current branch.
4. The response returns the new commit hash.

API — commit changes

```bash
curl -X POST http://localhost:8321/api/git/commit \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Update research-pipeline workflow",
    "files": ["custom/workflows/research-pipeline.yaml"]
  }'
```

Note

Files matched by `.gitignore` patterns are silently skipped during staging. The commit proceeds with only the versionable files.

## Simulation branches

[Section titled “Simulation branches”](#simulation-branches)

When you run a workflow with unsaved changes, a simulation branch captures the exact state:

**Branch naming convention:**

```plaintext
sim/{workflow-slug}/{YYYYMMDD}/{5-char-hex}
```

Example: `sim/research-pipeline/20260407/a3f1b`

**How it works:**

1. The GUI sends `POST /api/git/sim-branch` with the workflow ID and current YAML content.
2. The git service creates a temporary index starting from `HEAD`.
3. It stages the entire worktree into that index (so parent/child workflow files are included).
4. It force-overrides the target workflow file with the in-memory YAML draft.
5. A commit is created from this tree, and a branch is pointed at it.
6. The response returns the branch name and commit SHA.

API — create a simulation branch

```bash
curl -X POST http://localhost:8321/api/git/sim-branch \
  -H "Content-Type: application/json" \
  -d '{
    "workflow_id": "research-pipeline",
    "yaml_content": "version: \"1.0\"\nworkflow:\n  name: research-pipeline\n  entry: step_one\n..."
  }'
```

The simulation branch is a real git branch pointing to a real commit. It includes the full worktree snapshot, not just the modified workflow file. This matters for workflows that reference other workflows via `workflow_ref` --- those child workflow files are resolved from the same branch.

## Run-to-commit tracking

[Section titled “Run-to-commit tracking”](#run-to-commit-tracking)

Every run records two git coordinates:

| Field        | Source                                           | Purpose                                           |
| ------------ | ------------------------------------------------ | ------------------------------------------------- |
| `branch`     | Request parameter (default: `"main"`)            | Which branch the YAML was read from               |
| `commit_sha` | `git log -1 --format=%H -- {path}` on the branch | Exact commit that last touched this workflow file |

When git is configured, the execution service always reads the workflow YAML from the requested branch using `git show {branch}:{path}`, not from the filesystem. This guarantees the run executes the committed version, not whatever is in the working tree.

## Historical YAML snapshots

[Section titled “Historical YAML snapshots”](#historical-yaml-snapshots)

The run detail view shows the YAML **as it was when the run executed**, not the current version. This uses the `commit_sha` stored on the run record:

API — read historical YAML

```bash
curl "http://localhost:8321/api/git/file?ref=abc1234&path=custom/workflows/research-pipeline.yaml"
```

This powers the fork recovery flow --- when you fork a failed run, the fork reads the YAML from the run’s `commit_sha` to create a new draft based on the exact version that failed. See [Fork Recovery](/docs/execution/fork-recovery) for details.

## Git endpoints

[Section titled “Git endpoints”](#git-endpoints)

The API exposes these git operations:

| Method | Path                  | Description                                       |
| ------ | --------------------- | ------------------------------------------------- |
| `GET`  | `/api/git/status`     | Current branch, uncommitted files, is\_clean flag |
| `POST` | `/api/git/commit`     | Stage files and commit with a message             |
| `GET`  | `/api/git/diff`       | Diff of working tree against HEAD                 |
| `GET`  | `/api/git/log`        | Last 50 commits (hash, message, date, author)     |
| `GET`  | `/api/git/file`       | Read a file at a specific ref (branch or SHA)     |
| `POST` | `/api/git/sim-branch` | Create a simulation branch from a YAML draft      |

All endpoints validate paths against the project root to prevent directory traversal. Absolute paths outside the project root, paths starting with `-`, and symlinks escaping the project boundary are rejected.

## The unsaved indicator

[Section titled “The unsaved indicator”](#the-unsaved-indicator)

The canvas topbar shows a small dot and a **Save** button when the workflow has unsaved changes. This indicator is driven by local `isDirty` state in the editor --- it tracks whether the in-memory YAML differs from the last saved version, not the git status endpoint. Clicking **Save** opens a `CommitDialog` which commits the changes via `POST /api/workflows/{id}/commits`, clears the dirty state, and the next run will execute from `main` instead of creating a sim branch.

<!-- Linear: RUN-557, RUN-747, RUN-749 — last verified against codebase 2026-04-07 -->

# Workspace Isolation

> How Runsight isolates LLM block execution with per-run workspaces, mediated host capabilities, and Unix-local workers.

Runsight isolates LLM block execution around a **workspace** rather than around a provider or a container backend. The durable contract is:

1. The parser wraps LLM blocks with `IsolatedBlockWrapper`.
2. At execution time, the wrapper builds a `WorkspaceRunRequest`.
3. A workspace harness executes the request and validates a `ResultEnvelope`.
4. The wrapper converts the validated result back into normal `BlockOutput`.

The current local implementation of that contract is `UnixLocalHarness`. It creates a fresh workspace session, starts a local Unix worker process inside that session’s runtime directory, mediates host capabilities through IPC, validates the worker result, and cleans up according to the workspace policy.

Note

Workspace isolation is a credential, state, and workspace boundary. The current Unix-local backend is not a hard OS security sandbox: it does not add container namespaces, cgroups, or seccomp. It is the architecture layer that lets stricter backends be added later without changing workflow YAML, parser contracts, or block wrappers.

## Deployment hardening

[Section titled “Deployment hardening”](#deployment-hardening)

Workspace isolation and container hardening are separate layers. The workspace boundary governs each isolated block run; the Docker deployment also applies service-level process controls around Runsight itself.

* **Layer 1 — Container hardening:** Docker deployments run as a non-root/unprivileged user, drop Linux capabilities, prevent privilege escalation, and apply container CPU and memory limits.

## Why workspace isolation

[Section titled “Why workspace isolation”](#why-workspace-isolation)

LLM blocks accept arbitrary prompts and may request model calls, tools, HTTP access, or file operations. Runsight treats those requests as work that must cross an explicit boundary:

* Model provider keys stay in the host runtime.
* HTTP credentials and URL allowlists stay in host-only bindings.
* Executable tools stay registered on the host.
* Worker-visible state is serialized into a manifest and policy.
* File operations mediated by the host are scoped to the run workspace.

This design keeps the public execution model provider-neutral. The worker asks the host to perform model and tool work through the workspace IPC layer. The host decides which provider credentials, HTTP credentials, allowlists, and executable tool handlers apply to that run.

## The workspace boundary

[Section titled “The workspace boundary”](#the-workspace-boundary)

When workflow YAML is parsed, LLM blocks are wrapped with `IsolatedBlockWrapper`. At execution time the wrapper sends a `WorkspaceRunRequest` to `UnixLocalHarness`.

The request contains serializable data the worker is allowed to know:

| Data                 | Purpose                                                        |
| -------------------- | -------------------------------------------------------------- |
| Block manifest       | Describes the block, inputs, and runtime-visible configuration |
| Workspace manifest   | Lists files to materialize into the run workspace              |
| `WorkspacePolicy`    | Runtime policy for the session                                 |
| Worker tool metadata | Serializable declarations for tools the worker may request     |

Host-only bindings are intentionally separate. `WorkspaceHostBindings` carries provider API keys, HTTP credentials, URL allowlists, and executable host tools. Those bindings are consumed by the harness to build host-side IPC handlers and are not serialized into the worker manifest or injected as worker environment secrets.

For HTTP tools, request-backed custom tools seed the allowlist from their host-side request URL. Dynamic built-in `http` calls use the host-only `RUNSIGHT_HTTP_URL_ALLOWLIST` setting, a comma- or whitespace-separated list of allowed hostnames or URLs. Empty allowlists deny mediated HTTP before any network request is made.

## Workspace materialization

[Section titled “Workspace materialization”](#workspace-materialization)

`UnixLocalHarness` owns the local workspace base by default. For each session it creates a fresh child workspace and:

* Validates that manifest paths are relative.
* Materializes declared files under the harness-owned workspace root.
* Creates the runtime working directory.
* Starts the worker with `cwd` set to that runtime directory.
* Scopes host-mediated file reads and writes to the same canonical workspace root.
* Applies cleanup according to the workspace policy.

The important detail is that process `cwd` and mediated file I/O share one canonical root. A block can reason about its workspace consistently, while the host still validates mediated file paths against the session root.

## IPC and authentication

[Section titled “IPC and authentication”](#ipc-and-authentication)

The worker discovers its IPC configuration from one environment variable:

```text
RUNSIGHT_IPC_CONFIG_B64=<base64-json IPCClientConfig>
```

That encoded `IPCClientConfig` is the current worker discovery contract. The worker uses it to connect to the host-side IPC handlers for model calls, tool execution, HTTP access, and file access.

Worker IPC handlers are for model calls, tool execution, HTTP access, and file access. Heartbeats are emitted as stderr JSON lines and monitored by the harness. The final `ResultEnvelope` is written to stdout as JSON and validated by the harness before the wrapper converts it back into `BlockOutput` for the normal workflow execution path.

## Policy and capabilities

[Section titled “Policy and capabilities”](#policy-and-capabilities)

`WorkspacePolicy` is runtime policy, not just documentation. It validates modes and controls the workspace session behavior the harness can enforce. The serializable request shape, including the worker manifest and policy data, is validated before execution.

`PolicyCapabilityReport.from_policy` reports capability entries conditionally from the policy:

| Capability area             | Unix-local behavior                                  |
| --------------------------- | ---------------------------------------------------- |
| Raw network deny            | Advisory in Unix-local when requested by policy      |
| Raw filesystem deny         | Advisory in Unix-local when requested by policy      |
| Credential host binding     | Enforced when credential host-binding policy applies |
| Mediated file constraints   | Enforced when mediated file policy applies           |
| Materialization size limits | Enforced when workspace materialization limits apply |

Because Unix-local workers are local Unix processes, direct process-level network and filesystem restrictions are advisory. Runsight’s enforced controls apply to host-mediated capabilities: provider calls, credentialed HTTP calls, registered tool execution, materialized workspace files, and mediated file operations.

## Tool execution

[Section titled “Tool execution”](#tool-execution)

Tool execution is split across two registries:

| Registry                    | Visibility     | Contents                               |
| --------------------------- | -------------- | -------------------------------------- |
| `WorkerToolRegistry`        | Worker-visible | Serializable tool names and metadata   |
| `HostToolExecutionRegistry` | Host-only      | Executable tool references and secrets |

A worker can request a tool only by using worker-visible metadata. The host executes the tool only when the same name exists in the host execution registry for that run. This keeps tool discovery serializable while keeping executable references and secrets out of the worker manifest.

## Provider-neutral model calls

[Section titled “Provider-neutral model calls”](#provider-neutral-model-calls)

The workspace harness does not depend on a specific model provider. The worker uses a proxied runner/client over IPC. The host-side handler resolves the provider from the requested model name and the API keys available in the run’s host bindings.

Budget enforcement and tracing stay on the host side, so model calls made from isolated blocks still participate in normal Runsight accounting. See [Budget & Limits](/docs/execution/budget-and-limits) for budget configuration.

## Assertions

[Section titled “Assertions”](#assertions)

Assertions that use model calls for grading, such as `llm_judge`, use the same host-mediated model path. Their model calls are still budgeted and observed through the workspace boundary.

See [Custom Assertions](/docs/evaluation/custom-assertions#llm-graded-assertions-llm_judge) for assertion configuration details.

## Trade-offs

[Section titled “Trade-offs”](#trade-offs)

Workspace isolation gives Runsight a stable execution boundary without making the local backend pretend to be stronger than it is.

| Property             | What Unix-local provides                                                           |
| -------------------- | ---------------------------------------------------------------------------------- |
| Fresh workspace      | Each session gets a harness-owned workspace root and runtime working directory     |
| Credential isolation | Provider keys, HTTP credentials, allowlists, and executable tools stay host-only   |
| File mediation       | Host-mediated reads and writes are scoped to the session root                      |
| Supervision          | Worker heartbeat, timeout, result validation, and cleanup are owned by the harness |
| Backend portability  | Parser, wrapper, request, and result contracts are backend-neutral                 |
| OS sandboxing        | Not a container-grade boundary in the current local implementation                 |

Future backends can provide stronger OS-level enforcement behind the same `WorkspaceRunRequest` to `ResultEnvelope` contract. The current docs describe the shipped Unix-local runtime only.

<!-- Linear: RUN-999, RUN-1000, RUN-1001, RUN-1002, RUN-1003, RUN-1004, RUN-1005, RUN-1006, RUN-1007 - last verified against codebase 2026-05-09 -->

# Running Workflows

> Production runs vs simulation runs - how Runsight executes workflows and tracks run history.

Runsight has two execution modes: **production runs** on the main branch and **simulation runs** on disposable branches. Every run is persisted as a database record with per-block node tracking, parent-child linkage for sub-workflows, and a commit SHA tying the run back to the workflow YAML that executed.

## Run lifecycle

[Section titled “Run lifecycle”](#run-lifecycle)

A run moves through a fixed state machine:

| State       | Meaning                                                          |
| ----------- | ---------------------------------------------------------------- |
| `pending`   | Created in the database, waiting for a concurrency slot          |
| `running`   | Actively executing blocks                                        |
| `completed` | All blocks finished successfully                                 |
| `failed`    | A block raised an unhandled error or a budget limit was exceeded |
| `cancelled` | Stopped by user action or server shutdown                        |

Terminal states (`completed`, `failed`, `cancelled`) are final --- no further transitions are allowed. The valid transitions are:

* `pending` → `running`, `cancelled`, `failed`
* `running` → `completed`, `failed`, `cancelled`

## Production runs

[Section titled “Production runs”](#production-runs)

A production run executes the workflow YAML as committed on the **main branch**.

Production runs can be started from the GUI when there are no unsaved workflow changes, or through the external Direct API. Direct API invocation uses `POST /api/workflows/{workflow_id}/runs`, always records `source: "api"`, always resolves `branch: "main"`, and only runs workflows committed on `main` and explicitly enabled with `enabled: true`. Callers cannot supply `source`, `branch`, provenance, delivery, simulation, or idempotency fields in the request body.

When a production run starts:

1. The API resolves the committed `main` workflow YAML for the requested workflow and verifies that the committed snapshot has `enabled: true`.
2. A `Run` record is created with `status: pending` and `branch: "main"`.
3. The execution service acquires a concurrency slot (default: 5 concurrent runs), then transitions the run to `running`.
4. The engine parses the YAML, builds the workflow graph, and wraps every LLM block in an `IsolatedBlockWrapper` backed by `UnixLocalHarness`.
5. Each block executes sequentially through the transition graph. A `RunNode` record is created per block.
6. For LLM blocks (linear, gate, synthesize, dispatch), the wrapper creates a `WorkspaceRunRequest`. `UnixLocalHarness` creates a workspace session, materializes declared files, and starts a local Unix worker inside the session workspace.
7. The worker discovers host IPC through `RUNSIGHT_IPC_CONFIG_B64=<base64-json IPCClientConfig>`. Model calls, HTTP access, file access, and tool execution are mediated by host-side handlers built from per-run host bindings and registries.
8. The harness validates the worker `ResultEnvelope`, cleans up the workspace according to policy, and returns the result to the normal block wrapper path.
9. On completion, the observer writes `status: completed` with final cost and token totals.

Direct API runs ignore dirty working tree edits to workflow YAML and nested workflow YAML. Referenced workflow blocks are resolved through the same committed snapshot. Parser and discovery paths receive the resolved git ref for snapshot-capable workflow assets, while provider settings, server settings, and API keys remain live runtime configuration from the running server environment. Omitted `enabled` is treated as disabled for Direct API invocation.

For the exact external HTTP contract, request body, success response, and error codes, see [Direct API Invocation](/docs/reference/direct-api-invocation).

## Simulation runs

[Section titled “Simulation runs”](#simulation-runs)

When the workflow has **unsaved changes** (the canvas shows an uncommitted badge), clicking **Run** creates a simulation run:

1. The GUI calls `POST /api/git/sim-branch` with the current YAML draft.
2. A simulation branch is created with the naming convention `sim/{workflow-slug}/{YYYYMMDD}/{short-id}` (e.g., `sim/research-pipeline/20260407/a3f1b`).
3. The run is created with `branch: "sim/research-pipeline/20260407/a3f1b"` and `source: "simulation"`.
4. Execution proceeds identically to a production run, but reads the YAML from the sim branch snapshot.

Simulation branches are disposable --- they capture the exact state of the workflow at the time of the run, including any unsaved edits. See [Git Integration](/docs/execution/git-integration) for details on how sim branches work.

## The Run record

[Section titled “The Run record”](#the-run-record)

Each run is stored as a `Run` row in the SQLite database. Important fields include:

| Field            | Type        | Description                                                                  |
| ---------------- | ----------- | ---------------------------------------------------------------------------- |
| `id`             | `str`       | Primary key (UUID)                                                           |
| `workflow_id`    | `str`       | Which workflow was executed                                                  |
| `workflow_name`  | `str`       | Human-readable workflow name                                                 |
| `status`         | `RunStatus` | Current state (`pending` / `running` / `completed` / `failed` / `cancelled`) |
| `branch`         | `str`       | Git branch this run executed against; production launch paths set `"main"`   |
| `source`         | `str`       | How the run was triggered, such as `manual`, `simulation`, or `api`          |
| `commit_sha`     | `str?`      | Git commit SHA of the YAML that executed                                     |
| `total_cost_usd` | `float`     | Accumulated LLM cost                                                         |
| `total_tokens`   | `int`       | Accumulated token count                                                      |
| `error`          | `str?`      | Error message if the run failed                                              |
| `fail_reason`    | `str?`      | Structured failure category (e.g., `"budget_exceeded"`)                      |
| `parent_run_id`  | `str?`      | Parent run ID for sub-workflow runs                                          |
| `root_run_id`    | `str?`      | Top-level ancestor run ID                                                    |
| `depth`          | `int`       | Nesting depth (0 for top-level runs)                                         |

## RunNode - per-block tracking

[Section titled “RunNode - per-block tracking”](#runnode---per-block-tracking)

Each block execution within a run creates a `RunNode` record:

| Field          | Type         | Description                                                                         |
| -------------- | ------------ | ----------------------------------------------------------------------------------- |
| `id`           | `str`        | Composite key: `{run_id}:{node_id}`                                                 |
| `node_id`      | `str`        | Block ID from the workflow YAML                                                     |
| `block_type`   | `str`        | Block type (`linear`, `gate`, `synthesize`, `dispatch`, `code`, `loop`, `workflow`) |
| `status`       | `NodeStatus` | `pending` / `running` / `completed` / `failed`                                      |
| `cost_usd`     | `float`      | Cost for this block’s LLM calls                                                     |
| `tokens`       | `dict`       | Token breakdown: `{"prompt": N, "completion": N, "total": N}`                       |
| `output`       | `str?`       | Block output text                                                                   |
| `eval_score`   | `float?`     | Assertion evaluation score                                                          |
| `eval_passed`  | `bool?`      | Whether all assertions passed                                                       |
| `child_run_id` | `str?`       | For workflow blocks: the child run’s ID                                             |
| `exit_handle`  | `str?`       | Which exit port this block took                                                     |

## Sub-workflow runs

[Section titled “Sub-workflow runs”](#sub-workflow-runs)

When a workflow contains a `workflow` block, the child workflow executes as a nested run. The parent `RunNode` records the `child_run_id`, and the child `Run` stores `parent_run_id`, `root_run_id`, and `depth`. This creates a tree of runs that you can query via the API:

API - list child runs

```bash
curl http://localhost:8000/api/runs/{parent_run_id}/children
```

The child run response includes `parent_run_id`, `root_run_id`, and `depth` fields so you can reconstruct the full execution tree.

## Ghost run recovery

[Section titled “Ghost run recovery”](#ghost-run-recovery)

If the server restarts while runs are in `pending` or `running` status, those runs become “ghosts” --- they will never complete because the asyncio task is gone. On startup, the execution service calls `fail_ghost_runs()` to mark `pending` and `running` runs as `failed` with `error: "API server restarted during execution"`.

## Triggering runs

[Section titled “Triggering runs”](#triggering-runs)

| Method         | How                                                                                                                                                                                                             |
| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **GUI**        | Click the **Run** button on the canvas topbar. Committed clean workflows run on `main`; dirty workflows create a simulation branch.                                                                             |
| **Direct API** | `POST /api/workflows/{workflow_id}/runs` with a required body containing only `inputs`. Direct API runs require a committed `main` workflow with `enabled: true`, and use `source: "api"` and `branch: "main"`. |

Note

Webhook and schedule triggers are not part of RUN-85. Runs can be triggered manually through the GUI or externally through the Direct API.

See [Direct API Invocation](/docs/reference/direct-api-invocation) for the copyable curl example and full request reference.

## Workspace isolation

[Section titled “Workspace isolation”](#workspace-isolation)

Every LLM block runs through the workspace isolation path. API keys, HTTP credentials, URL allowlists, and executable tool references stay in host-only `WorkspaceHostBindings` and host execution registries. The worker receives only serializable manifests, policy, worker-visible tool metadata, and its config-based IPC connection details.

Request-backed custom HTTP tools seed allowed hosts from their host-side request URL. Dynamic built-in `http` calls require `RUNSIGHT_HTTP_URL_ALLOWLIST` on the host; otherwise mediated HTTP is denied before network access.

`UnixLocalHarness` is the current local workspace harness. It gives each run a fresh workspace root, starts the worker with `cwd` inside that workspace, scopes mediated file I/O to the same root, validates the result envelope, and cleans up afterward. Unix-local isolation protects host-mediated credentials and workspace access, but it is not a container-grade OS sandbox.

See [Workspace Isolation](/docs/execution/process-isolation) for the full architecture.

<!-- Linear: RUN-554, RUN-590, RUN-607, RUN-717, RUN-391, RUN-943, RUN-999, RUN-1000, RUN-1001, RUN-1002, RUN-1003, RUN-1004, RUN-1005, RUN-1006, RUN-1007 - last verified against codebase 2026-05-09 -->

# Installation

> Install Runsight via uvx, Docker, or from source for development.

Caution

Runsight’s self-hosted API is unauthenticated today. By default, `uvx runsight` binds to `127.0.0.1` for local-only access. For Docker, keep host port publishing on loopback, for example `-p 127.0.0.1:8000:8000`, unless you add your own proxy and auth controls.

## uvx (recommended)

[Section titled “uvx (recommended)”](#uvx-recommended)

The fastest way to run Runsight. Requires Python 3.11+ and [uv](https://docs.astral.sh/uv/).

```bash
uvx runsight
```

This downloads and runs the `runsight` package in an isolated environment. Open <http://localhost:8000>.

To bind loopback explicitly:

```bash
uvx runsight --host 127.0.0.1
```

Don’t have `uv`? Install it first:

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

### Options

[Section titled “Options”](#options)

```plaintext
runsight [--host HOST] [--port PORT]
```

| Option   | Default     | Description  |
| -------- | ----------- | ------------ |
| `--host` | `127.0.0.1` | Bind address |
| `--port` | `8000`      | Bind port    |

## Docker

[Section titled “Docker”](#docker)

Run Runsight in a container with the current directory mounted as the workspace.

```bash
docker run -p 127.0.0.1:8000:8000 -v "$(pwd)":/workspace ghcr.io/runsight-ai/runsight
```

Or use Docker Compose:

```bash
docker compose up
```

The included `docker-compose.yml` publishes `127.0.0.1:8000:8000`, stores the workspace in the named volume `workspace_data`, and adds a healthcheck at `/health`. If you want your current directory to be the workspace instead, use the `docker run` command above or edit `docker-compose.yml` to replace the named volume with a bind mount.

The Dockerfile may bind to `0.0.0.0` inside the container. Host port publishing should still use `127.0.0.1` unless the service is intentionally exposed behind your own network and authentication controls.

### What the container does

[Section titled “What the container does”](#what-the-container-does)

* **Multi-stage build**: Node 20 builds the frontend, Python 3.12 runs the API server
* **System dependencies**: git (required for GitOps features) and curl (healthcheck)
* **Image runtime**: runs as a non-root `runsight` user
* **Compose hardening**: the included `docker-compose.yml` drops Linux capabilities and sets `no-new-privileges`
* **Workspace**: `RUNSIGHT_BASE_PATH` defaults to `/workspace`
* **Healthcheck**: `curl -f http://localhost:8000/health` every 30 seconds

### Environment variables

[Section titled “Environment variables”](#environment-variables)

| Variable              | Default       | Purpose                                                |
| --------------------- | ------------- | ------------------------------------------------------ |
| `RUNSIGHT_BASE_PATH`  | `/workspace`  | Root directory for workflow, soul, and tool YAML files |
| `RUNSIGHT_STATIC_DIR` | `/app/static` | Path to built frontend assets                          |
| `RUNSIGHT_LOG_FORMAT` | `text`        | Log format                                             |

## From source (development)

[Section titled “From source (development)”](#from-source-development)

For contributing or running the full development environment with hot-reload.

### Prerequisites

[Section titled “Prerequisites”](#prerequisites)

* **Python 3.11+** with [uv](https://docs.astral.sh/uv/)
* **Node.js 20+** with [pnpm](https://pnpm.io/) (v10)
* **Git**

### Setup

[Section titled “Setup”](#setup)

```bash
git clone https://github.com/runsight-ai/runsight.git
cd runsight


# Install Python dependencies (workspace: apps/api + packages/core)
uv sync


# Install Node dependencies (workspace: apps/gui + packages/shared + packages/ui)
pnpm install
```

### Start the development environment

[Section titled “Start the development environment”](#start-the-development-environment)

Open two terminals:

```bash
# Terminal 1 — API server (port 8000)
uv run runsight
```

```bash
# Terminal 2 — GUI dev server with hot-reload (port 3000)
pnpm -C apps/gui dev
```

In development, the frontend dev server runs on <http://localhost:3000> with Vite hot-reload. In production (Docker/uvx), the API server serves the built frontend directly on port 8000.

### Project structure

[Section titled “Project structure”](#project-structure)

```plaintext
runsight/
├── apps/
│   ├── api/        # FastAPI server (Python) — runsight_api
│   ├── gui/        # React 19 + Vite frontend — visual builder
│   └── site/       # Astro + Starlight documentation site
├── packages/
│   ├── core/       # Pure Python engine — runsight_core
│   ├── shared/     # Shared TypeScript utilities
│   └── ui/         # Shared UI components
├── custom/         # User workspace (auto-discovered)
│   ├── workflows/  # Workflow YAML files
│   ├── souls/      # Soul YAML files
│   └── tools/      # Custom tool YAML files
└── testing/
    └── gui-e2e/    # Playwright end-to-end tests
```

### Running tests

[Section titled “Running tests”](#running-tests)

```bash
# Engine tests (target specific files — full suite is heavy)
uv run python -m pytest packages/core/tests/test_specific_file.py -v


# API tests
uv run python -m pytest apps/api/tests/test_specific_file.py -v


# Frontend unit tests
pnpm -C apps/gui test:unit


# Linting
pnpm run lint
```

## Git requirement

[Section titled “Git requirement”](#git-requirement)

Runsight requires git in the environment. When running for the first time, Runsight auto-initializes a git repository in the workspace if one doesn’t exist. Under the workspace root, it scaffolds `custom/workflows`, `custom/workflows/.canvas`, `custom/souls`, `custom/tools`, and `.runsight`. The default SQLite database lives at `.runsight/runsight.db`.

All workflow saves commit to git, simulation runs create branches, and run history is tied to commit SHAs.

If git is not available, the API server will start but git-dependent features (save, commit, simulation branches, fork recovery) will fail.

<!-- Linear: RUN-821, RUN-847, RUN-848, RUN-944, RUN-943 — last verified against codebase 2026-04-26 -->

# Key Concepts

> Core primitives in Runsight — workflows, blocks, souls, tools, and dispatch.

Runsight has five core primitives: **workflows**, **blocks**, **souls**, **tools**, and **dispatch**. Everything is defined in YAML files on your filesystem. Git is the version control layer.

## Workflows

[Section titled “Workflows”](#workflows)

A workflow is a YAML file in `custom/workflows/` that defines a directed graph of blocks. The engine discovers workflow files automatically.

```yaml
version: "1.0"
id: research-pipeline
kind: workflow
blocks:
  research:
    type: linear
    soul_ref: researcher
  summarize:
    type: linear
    soul_ref: writer
    depends: research
workflow:
  name: Research Pipeline
  entry: research
```

A workflow file has these top-level sections:

| Section     | Default      | Purpose                                                   |
| ----------- | ------------ | --------------------------------------------------------- |
| `version`   | `"1.0"`      | Schema version                                            |
| `blocks`    | `{}`         | Block definitions keyed by block ID                       |
| `workflow`  | **required** | Graph metadata — `name`, `entry` block, and `transitions` |
| `souls`     | `{}`         | Inline soul definitions (optional shorthand)              |
| `tools`     | `[]`         | Tool IDs available to this workflow                       |
| `limits`    | none         | Budget constraints (cost caps, timeouts)                  |
| `eval`      | none         | Test cases for offline evaluation                         |
| `enabled`   | `false`      | Whether the workflow is active                            |
| `interface` | none         | Public input/output contract for callable sub-workflows   |
| `config`    | `{}`         | Arbitrary workflow configuration                          |

Only `workflow` is required — all other sections have defaults.

Workflows are **nestable** — a `workflow` block can execute another workflow file as a child, with parent-child run linkage and independent error handling.

## Blocks

[Section titled “Blocks”](#blocks)

Blocks are the execution units inside a workflow. Each block has a `type` that determines its behavior. Runsight ships with six block types:

| Type       | What it does                                                                                                   |
| ---------- | -------------------------------------------------------------------------------------------------------------- |
| `linear`   | Single LLM call through a soul. The most common block type.                                                    |
| `gate`     | LLM quality gate — evaluates another block’s output and routes on pass/fail.                                   |
| `code`     | Runs Python code. Define a `def main(data)` function in the `code` field.                                      |
| `loop`     | Iterates inner blocks for up to `max_rounds` rounds with optional break conditions.                            |
| `workflow` | Executes a child workflow via `workflow_ref`. Parent-child run linkage, `on_error` modes.                      |
| `dispatch` | Parallel branching — each exit port gets its own soul and task instruction. All branches execute concurrently. |

All blocks share a unified execution lifecycle via `execute_block()`:

1. Observer notified (`on_block_start`)
2. Block-scoped budget session created (if `limits` defined)
3. Retry wrapper applied (if `retry_config` defined)
4. Timeout enforced via `asyncio.wait_for`
5. Block executes
6. Exit conditions evaluated — sets the `exit_handle` for downstream routing
7. Observer notified (`on_block_complete`)

Blocks can define `assertions` for quality evaluation, `exits` for multi-path routing, `depends` for dependency ordering, and `error_route` for error-specific branching.

## Souls

[Section titled “Souls”](#souls)

A soul is an agent identity — it defines who an LLM is and how it behaves. Souls are YAML files in `custom/souls/`:

custom/souls/researcher.yaml

```yaml
id: researcher
kind: soul
name: Researcher
role: Senior Researcher
system_prompt: >
  You are an expert researcher. Given a topic, provide a concise,
  well-structured summary of key findings, trends, and insights.
provider: openai
model_name: gpt-4.1-mini
temperature: 0.7
max_tokens: 2048
```

Soul fields:

| Field                 | Required | Purpose                                                                     |
| --------------------- | -------- | --------------------------------------------------------------------------- |
| `id`                  | Yes      | Unique identifier                                                           |
| `kind`                | Yes      | Entity kind, always `soul`                                                  |
| `name`                | Yes      | Display name                                                                |
| `role`                | Yes      | Agent role label                                                            |
| `system_prompt`       | Yes      | LLM system instructions                                                     |
| `provider`            | No       | LLM provider (e.g., `openai`, `anthropic`)                                  |
| `model_name`          | No       | Model name (e.g., `gpt-4.1-mini`). Falls back to runner default if omitted. |
| `temperature`         | No       | Sampling temperature                                                        |
| `max_tokens`          | No       | Output token limit                                                          |
| `tools`               | No       | Tool names this soul can use                                                |
| `avatar_color`        | No       | UI display color                                                            |
| `max_tool_iterations` | No       | Max tool-use loops per execution (default: 5)                               |

Blocks reference souls via `soul_ref`:

```yaml
blocks:
  research:
    type: linear
    soul_ref: researcher  # resolves by embedded soul id
```

Souls can also be defined inline in the workflow YAML as optional shorthand — see [Inline Souls](/docs/souls/inline-souls).

**One soul per step.** Each block references at most one soul. The exception is `dispatch`, where each exit port has its own soul.

## Tools

[Section titled “Tools”](#tools)

Tools are capabilities that souls can use during execution. They come in three types:

* **Built-in tools**: Ship with Runsight — `delegate` (exit port routing), `http` (outbound requests), `file_io` (file operations)
* **Custom tools**: YAML files in `custom/tools/` with a Python or HTTP executor
* **HTTP tools**: Declarative HTTP request definitions

Custom tool example:

custom/tools/slack\_payload\_builder.yaml

```yaml
version: "1.0"
id: slack_payload_builder
kind: tool
type: custom
executor: python
name: Slack Payload Builder
description: Builds a Slack message payload from plain text.
parameters:
  type: object
  properties:
    text:
      type: string
  required: [text]
code: |
  import json
  def main(args):
      return {"payload_json": json.dumps({"text": args["text"]})}
```

Custom tools are identified by their embedded `id`, and the YAML filename stem must match that id. Built-in tools use reserved IDs: `delegate`, `http`, `file_io`.

Tools are discovered automatically from `custom/tools/`.

**Tool governance** is enforced at the workflow level: a soul only gets access to tools that the workflow explicitly lists in its `tools` section. Both the workflow and the soul must declare the tool for it to be available during execution.

## Dispatch

[Section titled “Dispatch”](#dispatch)

Dispatch is the branching mechanism. There are two ways to branch in Runsight:

### Exit ports on any block

[Section titled “Exit ports on any block”](#exit-ports-on-any-block)

Any block can define `exits`. When a soul lists `delegate` in its tools and the workflow enables it, the soul can call `delegate(port="...", task="...")` to pick which exit port to route to — making the LLM the decision-maker.

```yaml
blocks:
  triage:
    type: linear
    soul_ref: triager
    exits:
      - id: urgent
        label: Urgent — needs immediate attention
      - id: normal
        label: Normal — standard processing
  handle_urgent:
    type: linear
    soul_ref: responder
    depends: triage
  handle_normal:
    type: linear
    soul_ref: processor
    depends: triage
```

The `delegate` tool requires two parameters: `port` (constrained to the declared exit IDs) and `task` (instruction for the downstream block). The soul must list `delegate` in its `tools` array, and the workflow must include `delegate` in its top-level `tools` list.

### Dispatch block (parallel branching)

[Section titled “Dispatch block (parallel branching)”](#dispatch-block-parallel-branching)

The `dispatch` block type runs all branches concurrently — each exit port gets its own soul and task instruction. Unlike exit ports on a `linear` block (where the LLM picks one), dispatch executes every branch.

See [Dispatch & Delegate](/docs/tools/dispatch-and-delegate) for details.

## Process Isolation

[Section titled “Process Isolation”](#process-isolation)

Every LLM block runs in an isolated subprocess. The subprocess has no API keys, no access to engine memory, and no credentials. All LLM calls are proxied through a Unix socket IPC channel where the engine holds the real keys, enforces budget limits, and records observability traces.

This is transparent --- you don’t configure it. Workflow YAML and block behavior are identical whether isolation is enabled or not. The isolation layer protects against prompt injection, model misbehavior, and credential leakage.

See [Process Isolation](/docs/execution/process-isolation) for the full architecture.

## YAML-First, Git-Native

[Section titled “YAML-First, Git-Native”](#yaml-first-git-native)

Everything in Runsight is a file:

| Primitive | Location                  | Format |
| --------- | ------------------------- | ------ |
| Workflows | `custom/workflows/*.yaml` | YAML   |
| Souls     | `custom/souls/*.yaml`     | YAML   |
| Tools     | `custom/tools/*.yaml`     | YAML   |

Git is the version control layer:

* **Save** writes the workflow YAML to disk and commits to the `main` branch
* **Dirty runs** (unsaved changes) automatically create **simulation branches** (`sim/{workflow-slug}/{YYYYMMDD}/{short-id}`)
* Every run records the **commit SHA** of the workflow that executed
* **Fork recovery** lets you branch from a failed run to iterate without losing history
* The run detail view shows the **historical YAML snapshot** — the exact file content that ran, not the current version

There is no database for workflow definitions. Your repo is the database. Diff your workflows, review them in PRs, roll back with git.

# Quickstart

> Install Runsight and run your first AI agent workflow in under 5 minutes.

1. **Install and start Runsight**

   For local-only access with `uvx`:

   ```bash
   uvx runsight --host 127.0.0.1
   ```

   Or use Docker:

   ```bash
   docker run -p 127.0.0.1:8000:8000 -v "$(pwd)":/workspace ghcr.io/runsight-ai/runsight
   ```

   Don’t have `uv`? Install it first: `curl -LsSf https://astral.sh/uv/install.sh | sh`

   Open <http://localhost:8000>. The onboarding flow walks you through API key setup.

2. **Create a workflow file**

   Create `custom/workflows/my-first-flow.yaml`:

   custom/workflows/my-first-flow\.yaml

   ```yaml
   version: "1.0"
   id: my-first-flow
   kind: workflow
   blocks:
     research:
       type: linear
       soul_ref: researcher
     write_summary:
       type: linear
       soul_ref: writer
       depends: research
     quality_review:
       type: gate
       soul_ref: reviewer
       eval_key: write_summary
       depends: write_summary
   workflow:
     name: Research & Review
     entry: research
   ```

   Three blocks: **research** calls an LLM, **write\_summary** runs after it, and **quality\_review** evaluates the summary.

3. **Define your souls**

   Each `soul_ref` in the workflow points to a soul — the agent identity behind a block.

   #### Inline (single file, fastest)

   [Section titled “Inline (single file, fastest)”](#inline-single-file-fastest)

   Add a `souls:` section directly in the workflow file:

   custom/workflows/my-first-flow\.yaml

   ```yaml
   version: "1.0"
   id: my-first-flow
   kind: workflow
   souls:
     researcher:
       id: researcher
       kind: soul
       name: Researcher
       role: Senior Researcher
       system_prompt: >
         You are an expert researcher. Given a topic, provide a concise,
         well-structured summary of the key findings, trends, and insights.
       provider: openai
       model_name: gpt-4.1-mini
   blocks:
     research:
       type: linear
       soul_ref: researcher
   workflow:
     name: My First Flow
     entry: research
   ```

   #### Soul file (reusable across workflows)

   [Section titled “Soul file (reusable across workflows)”](#soul-file-reusable-across-workflows)

   Extract the soul to its own file in `custom/souls/`. Any workflow can reference it by embedded id (`soul_ref: researcher`), and the file stem must match that id:

   custom/souls/researcher.yaml

   ```yaml
   id: researcher
   kind: soul
   name: Researcher
   role: Senior Researcher
   system_prompt: >
     You are an expert researcher. Given a topic, provide a concise,
     well-structured summary of the key findings, trends, and insights.
   provider: openai
   model_name: gpt-4.1-mini
   ```

   See [Inline Souls](/docs/souls/inline-souls) and [Soul Files](/docs/souls/soul-files) for details.

4. **Run it**

   1. Open the GUI at <http://localhost:8000>
   2. Your workflow appears on the **Flows** page (auto-discovered from `custom/workflows/`)
   3. Click the workflow to open it on the canvas
   4. Click **Run**

## What’s next

[Section titled “What’s next”](#whats-next)

* [Key Concepts](/docs/getting-started/key-concepts) — blocks, souls, tools, and how they fit together
* [Block Types](/docs/workflows/block-types) — the block types and when to use each
* [Custom Tools](/docs/tools/custom-tools) — give your agents tools defined as YAML files
* [Assertions & Eval](/docs/evaluation/assertions) — add quality checks to your workflows
* [Git Integration](/docs/execution/git-integration) — save, commit, and simulation branches

# Assertion Reference

> All 15 assertion types with their parameters, plus the not- prefix and json_path transform hook.

<!-- RUN-695 -->

Assertions validate block outputs against expected values. They are used in the `assertions` field on any block and in the `eval` section’s `expected` entries. Each assertion is a dict with at minimum a `type` field.

## Assertion config fields

[Section titled “Assertion config fields”](#assertion-config-fields)

Every assertion config supports these fields:

| Field       | Type    | Default | Description                                                                                                       |
| ----------- | ------- | ------- | ----------------------------------------------------------------------------------------------------------------- |
| `type`      | `str`   | —       | Assertion type name (see sections below). Prefix with `not-` to negate.                                           |
| `value`     | `Any`   | `""`    | Comparison value (type-specific)                                                                                  |
| `threshold` | `float` | varies  | Pass threshold (type-specific)                                                                                    |
| `config`    | `Any`   | none    | Optional per-assertion config. Built-ins usually ignore it; `equals` uses `config.mode: json` for JSON deep-equal |
| `weight`    | `float` | `1.0`   | Weight for aggregated scoring                                                                                     |
| `metric`    | `str`   | none    | Named score key for tracking                                                                                      |
| `transform` | `str`   | none    | Pre-processing transform (see [Transform hooks](#transform-hooks))                                                |

***

## String assertions

[Section titled “String assertions”](#string-assertions)

### equals

[Section titled “equals”](#equals)

Exact string match by default. To compare parsed JSON values explicitly, set `config.mode: json`.

| Parameter | Type  | Description                     |
| --------- | ----- | ------------------------------- |
| `value`   | `Any` | Expected value (string or JSON) |

```yaml
- type: equals
  value: "expected output"
```

```yaml
- type: equals
  value: '{"a": 1, "b": 2}'
  config:
    mode: json
```

### contains

[Section titled “contains”](#contains)

Case-sensitive substring check.

| Parameter | Type  | Description       |
| --------- | ----- | ----------------- |
| `value`   | `str` | Substring to find |

```yaml
- type: contains
  value: "machine learning"
```

### icontains

[Section titled “icontains”](#icontains)

Case-insensitive substring check.

| Parameter | Type  | Description                          |
| --------- | ----- | ------------------------------------ |
| `value`   | `str` | Substring to find (case-insensitive) |

```yaml
- type: icontains
  value: "Machine Learning"
```

### contains-all

[Section titled “contains-all”](#contains-all)

All items in the value list must be present as substrings.

| Parameter | Type        | Description                                 |
| --------- | ----------- | ------------------------------------------- |
| `value`   | `List[str]` | List of substrings that must all be present |

```yaml
- type: contains-all
  value: ["introduction", "methodology", "conclusion"]
```

### contains-any

[Section titled “contains-any”](#contains-any)

At least one item in the value list must be present as a substring.

| Parameter | Type        | Description                  |
| --------- | ----------- | ---------------------------- |
| `value`   | `List[str]` | List of candidate substrings |

```yaml
- type: contains-any
  value: ["approved", "accepted", "passed"]
```

### starts-with

[Section titled “starts-with”](#starts-with)

String prefix check.

| Parameter | Type  | Description     |
| --------- | ----- | --------------- |
| `value`   | `str` | Expected prefix |

```yaml
- type: starts-with
  value: "Summary:"
```

### regex

[Section titled “regex”](#regex)

Regex search match (uses `re.search`, not full match).

| Parameter | Type  | Description                |
| --------- | ----- | -------------------------- |
| `value`   | `str` | Regular expression pattern |

```yaml
- type: regex
  value: "\\d{4}-\\d{2}-\\d{2}"
```

### word-count

[Section titled “word-count”](#word-count)

Word count check. Supports exact count or min/max range.

| Parameter | Type                  | Description                                                      |
| --------- | --------------------- | ---------------------------------------------------------------- |
| `value`   | `int` or `{min, max}` | Exact word count, or a dict with optional `min` and `max` bounds |

```yaml
# Exact count
- type: word-count
  value: 100


# Range
- type: word-count
  value:
    min: 50
    max: 200
```

***

## Structural assertions

[Section titled “Structural assertions”](#structural-assertions)

### is-json

[Section titled “is-json”](#is-json)

Validates that the output is valid JSON. Optionally validates against a JSON Schema.

| Parameter | Type                           | Description                              |
| --------- | ------------------------------ | ---------------------------------------- |
| `value`   | `Dict` (JSON Schema) or `null` | Optional JSON Schema to validate against |

```yaml
# Just check valid JSON
- type: is-json


# With schema validation
- type: is-json
  value:
    type: object
    required: ["title", "body"]
    properties:
      title:
        type: string
      body:
        type: string
```

### contains-json

[Section titled “contains-json”](#contains-json)

Finds a valid JSON substring in the output. Scans for `{` and `[` delimiters and attempts to parse. Optionally validates the extracted JSON against a JSON Schema.

| Parameter | Type                           | Description                                             |
| --------- | ------------------------------ | ------------------------------------------------------- |
| `value`   | `Dict` (JSON Schema) or `null` | Optional JSON Schema to validate extracted JSON against |

```yaml
- type: contains-json
  value:
    type: object
    required: ["status"]
```

***

## Linguistic assertions

[Section titled “Linguistic assertions”](#linguistic-assertions)

### levenshtein

[Section titled “levenshtein”](#levenshtein)

Edit distance between the output and a reference string must be at or below the threshold.

| Parameter   | Type    | Default | Description                         |
| ----------- | ------- | ------- | ----------------------------------- |
| `value`     | `str`   | `""`    | Reference string to compare against |
| `threshold` | `float` | `5`     | Maximum allowed edit distance       |

```yaml
- type: levenshtein
  value: "expected text"
  threshold: 10
```

### bleu

[Section titled “bleu”](#bleu)

BLEU-4 score against a reference string must be at or above the threshold. Uses smoothing method 1 (add-one).

| Parameter   | Type    | Default | Description        |
| ----------- | ------- | ------- | ------------------ |
| `value`     | `str`   | `""`    | Reference text     |
| `threshold` | `float` | `0.5`   | Minimum BLEU score |

```yaml
- type: bleu
  value: "The expected reference text for comparison."
  threshold: 0.4
```

### rouge-n

[Section titled “rouge-n”](#rouge-n)

ROUGE-1 F-measure against a reference string must be at or above the threshold. Uses the `rouge-score` library.

| Parameter   | Type    | Default | Description           |
| ----------- | ------- | ------- | --------------------- |
| `value`     | `str`   | `""`    | Reference text        |
| `threshold` | `float` | `0.75`  | Minimum ROUGE-N score |

```yaml
- type: rouge-n
  value: "The reference summary text."
  threshold: 0.6
```

***

## Performance assertions

[Section titled “Performance assertions”](#performance-assertions)

### cost

[Section titled “cost”](#cost)

Checks that the block’s `cost_usd` is within the threshold. Reads from `AssertionContext.cost_usd`, not from the output string.

| Parameter   | Type    | Default | Description         |
| ----------- | ------- | ------- | ------------------- |
| `threshold` | `float` | `0.0`   | Maximum cost in USD |

```yaml
- type: cost
  threshold: 0.50
```

### latency

[Section titled “latency”](#latency)

Checks that the block’s `latency_ms` is within the threshold. Reads from `AssertionContext.latency_ms`, not from the output string.

| Parameter   | Type    | Default | Description                     |
| ----------- | ------- | ------- | ------------------------------- |
| `threshold` | `float` | `0.0`   | Maximum latency in milliseconds |

```yaml
- type: latency
  threshold: 5000
```

***

## Negation prefix

[Section titled “Negation prefix”](#negation-prefix)

Any assertion type can be negated by prefixing with `not-`. The result is inverted: `passed` becomes `not passed`, and `score` becomes `1.0 - score`.

```yaml
- type: not-contains
  value: "error"


- type: not-is-json
```

***

## Transform hooks

[Section titled “Transform hooks”](#transform-hooks)

Transforms pre-process the block output before the assertion evaluates it. Specified via the `transform` field on any assertion config.

### json\_path

[Section titled “json\_path”](#json_path)

Extracts a value from JSON output using JSONPath syntax (via the `jsonpath-ng` library). The extracted value is converted to a string and passed to the assertion.

Format: `json_path:<expression>`

```yaml
- type: contains
  value: "active"
  transform: "json_path:$.status"


- type: equals
  value: "42"
  transform: "json_path:$.data.count"
```

If the output is not valid JSON or the path is not found, the assertion fails with a descriptive error before the actual assertion runs.

***

## Scoring and aggregation

[Section titled “Scoring and aggregation”](#scoring-and-aggregation)

Assertions are scored and aggregated using weighted averages:

* Each assertion produces a `GradingResult` with `passed` (bool) and `score` (0.0—1.0).
* Weights default to `1.0` and are used for the weighted average (`aggregate_score`).
* In eval mode, the suite passes if `aggregate_score >= threshold` (default threshold: `1.0`).
* Without a threshold, all individual assertions must pass.

### Using in eval cases

[Section titled “Using in eval cases”](#using-in-eval-cases)

eval section with assertions

```yaml
eval:
  threshold: 0.8
  cases:
    - id: test_summary
      fixtures:
        summarize: "Machine learning is a subset of AI that enables systems to learn."
      expected:
        summarize:
          - type: contains
            value: "machine learning"
          - type: word-count
            value:
              min: 5
              max: 50
          - type: not-contains
            value: "error"
```

### Using on blocks

[Section titled “Using on blocks”](#using-on-blocks)

block-level assertions

```yaml
blocks:
  research:
    type: linear
    soul_ref: researcher
    assertions:
      - type: is-json
      - type: word-count
        value:
          min: 100
```

# Block Type Reference

> Quick-reference card for every block type — all fields, defaults, and minimal YAML examples.

<!-- RUN-110 -->

Compact lookup reference for all six block types. For detailed explanations and common patterns, see [Block Types](/docs/workflows/block-types).

## Common fields (BaseBlockDef)

[Section titled “Common fields (BaseBlockDef)”](#common-fields-baseblockdef)

Every block inherits these fields regardless of type.

| Field               | Type                   | Default | Constraints                                                               | Description                                                                                       |
| ------------------- | ---------------------- | ------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| `type`              | `str`                  | —       | required                                                                  | Block type discriminator (`linear`, `gate`, `code`, `loop`, `workflow`, `dispatch`, `synthesize`) |
| `stateful`          | `bool`                 | `false` | —                                                                         | Maintain conversation history across re-invocations                                               |
| `depends`           | `str` or `List[str]`   | none    | non-blank                                                                 | Upstream block dependencies                                                                       |
| `error_route`       | `str`                  | none    | non-blank                                                                 | Target block on error                                                                             |
| `inputs`            | `Dict[str, InputRef]`  | none    | —                                                                         | Explicit upstream data references (each entry has a `from` field)                                 |
| `outputs`           | `Dict[str, str]`       | none    | —                                                                         | Output field name to type string mapping                                                          |
| `output_conditions` | `List[CaseDef]`        | none    | mutually exclusive with `routes`                                          | Named output branches                                                                             |
| `routes`            | `List[RouteDef]`       | none    | mutually exclusive with `output_conditions`, requires exactly one default | Shorthand routing                                                                                 |
| `exits`             | `List[ExitDef]`        | none    | —                                                                         | Named exit ports for branching                                                                    |
| `exit_conditions`   | `List[ExitCondition]`  | none    | —                                                                         | Output pattern to exit handle mapping                                                             |
| `assertions`        | `List[Dict[str, Any]]` | none    | —                                                                         | Block-level quality assertions                                                                    |
| `retry_config`      | `RetryConfig`          | none    | —                                                                         | Retry on failure                                                                                  |
| `timeout_seconds`   | `int`                  | `300`   | 1—3600                                                                    | Block execution timeout in seconds                                                                |
| `stall_thresholds`  | `Dict[str, int]`       | none    | —                                                                         | Per-phase stall detection thresholds                                                              |
| `limits`            | `BlockLimitsDef`       | none    | —                                                                         | Per-block budget constraints                                                                      |

***

## linear

[Section titled “linear”](#linear)

Single LLM call through a soul.

| Field      | Type                | Default    | Required | Description                   |
| ---------- | ------------------- | ---------- | -------- | ----------------------------- |
| `type`     | `Literal["linear"]` | `"linear"` | **yes**  | Type discriminator            |
| `soul_ref` | `str`               | —          | **yes**  | Soul ID to use for this block |

minimal linear block

```yaml
blocks:
  research:
    type: linear
    soul_ref: researcher
```

***

## gate

[Section titled “gate”](#gate)

LLM quality gate — evaluates another block’s output and routes on pass/fail.

| Field           | Type              | Default  | Required | Description                                                |
| --------------- | ----------------- | -------- | -------- | ---------------------------------------------------------- |
| `type`          | `Literal["gate"]` | `"gate"` | **yes**  | Type discriminator                                         |
| `soul_ref`      | `str`             | —        | **yes**  | Soul ID for the gate evaluator                             |
| `eval_key`      | `str`             | —        | **yes**  | Block ID whose output is being evaluated                   |
| `extract_field` | `str`             | none     | no       | JSON field to extract before evaluation                    |
| `pass`          | `str`             | none     | no       | Target block on pass (shorthand). Must be set with `fail`. |
| `fail`          | `str`             | none     | no       | Target block on fail (shorthand). Must be set with `pass`. |

When `pass` and `fail` are omitted, the gate auto-creates two `ExitDef` entries with IDs `"pass"` and `"fail"`.

minimal gate block

```yaml
blocks:
  quality_check:
    type: gate
    soul_ref: reviewer
    eval_key: draft
    pass: publish
    fail: revise
```

***

## code

[Section titled “code”](#code)

Sandboxed Python code execution. No LLM calls.

| Field             | Type              | Default                                                                                                       | Required | Description                                       |
| ----------------- | ----------------- | ------------------------------------------------------------------------------------------------------------- | -------- | ------------------------------------------------- |
| `type`            | `Literal["code"]` | `"code"`                                                                                                      | **yes**  | Type discriminator                                |
| `code`            | `str`             | —                                                                                                             | **yes**  | Python source code containing `def main(data)`    |
| `timeout_seconds` | `int`             | `30`                                                                                                          | no       | Execution timeout (overrides base default of 300) |
| `allowed_imports` | `List[str]`       | `["json", "re", "math", "datetime", "collections", "itertools", "hashlib", "base64", "time", "urllib.parse"]` | no       | Whitelist of importable modules                   |

The `main` function receives a `data` dict containing only the local names declared in the block’s `inputs` map. It must return a JSON-serializable value. If the return value is a dict containing `exit_handle`, that value is extracted and used as the block’s exit handle.

declared-input code block

```yaml
blocks:
  transform:
    type: code
    depends: draft
    inputs:
      draft_text:
        from: draft.text
    code: |
      def main(data):
          return {"word_count": len(data.get("draft_text", "").split())}
```

***

## loop

[Section titled “loop”](#loop)

Iterates inner blocks for multiple rounds with optional break conditions.

| Field              | Type                                  | Default  | Required | Constraints | Description                                           |
| ------------------ | ------------------------------------- | -------- | -------- | ----------- | ----------------------------------------------------- |
| `type`             | `Literal["loop"]`                     | `"loop"` | **yes**  | —           | Type discriminator                                    |
| `inner_block_refs` | `List[str]`                           | —        | **yes**  | min 1 item  | Block IDs to execute each round                       |
| `max_rounds`       | `int`                                 | `5`      | no       | 1—50        | Maximum iterations                                    |
| `break_condition`  | `ConditionDef` or `ConditionGroupDef` | none     | no       | —           | Condition evaluated against last inner block’s output |
| `carry_context`    | `CarryContextConfig`                  | none     | no       | —           | Context propagation between rounds                    |
| `break_on_exit`    | `str`                                 | none     | no       | —           | Exit handle value that triggers loop break            |
| `retry_on_exit`    | `str`                                 | none     | no       | —           | Exit handle value that triggers another round         |

### CarryContextConfig

[Section titled “CarryContextConfig”](#carrycontextconfig)

| Field           | Type        | Default                    | Description                                                                                      |
| --------------- | ----------- | -------------------------- | ------------------------------------------------------------------------------------------------ |
| `enabled`       | `bool`      | `true`                     | Enable context carrying                                                                          |
| `mode`          | `str`       | `"last"`                   | `"last"` (previous round only) or `"all"` (accumulate all rounds)                                |
| `source_blocks` | `List[str]` | none                       | Specific blocks to carry from (default: all inner blocks). Must be subset of `inner_block_refs`. |
| `inject_as`     | `str`       | `"previous_round_context"` | Key name for injected context in `shared_memory`                                                 |

minimal loop block

```yaml
blocks:
  refine:
    type: loop
    inner_block_refs: [draft, review]
    max_rounds: 3
    break_on_exit: pass
```

***

## workflow

[Section titled “workflow”](#workflow)

Executes a child workflow as a sub-step (Hierarchical State Machine pattern).

| Field          | Type                  | Default      | Required | Description                                                              |
| -------------- | --------------------- | ------------ | -------- | ------------------------------------------------------------------------ |
| `type`         | `Literal["workflow"]` | `"workflow"` | **yes**  | Type discriminator                                                       |
| `workflow_ref` | `str`                 | —            | **yes**  | Embedded workflow id of the child workflow to execute                    |
| `inputs`       | `Dict[str, str]`      | none         | no       | Interface name to parent state path mapping. Keys must not contain dots. |
| `outputs`      | `Dict[str, str]`      | none         | no       | Parent path to interface name mapping. Values must not contain dots.     |
| `max_depth`    | `int`                 | none         | no       | Maximum nesting depth limit (default engine limit is 10)                 |
| `on_error`     | `str`                 | `"raise"`    | no       | `"raise"` (propagate error) or `"catch"` (absorb error, continue parent) |

minimal workflow block

```yaml
blocks:
  sub_pipeline:
    type: workflow
    workflow_ref: analysis-pipeline
    inputs:
      topic: results.research
    outputs:
      results.summary: analysis_result
```

***

## synthesize

[Section titled “synthesize”](#synthesize)

Combines outputs from multiple upstream blocks into a single result via an LLM.

| Field             | Type                    | Default        | Required | Description                                              |
| ----------------- | ----------------------- | -------------- | -------- | -------------------------------------------------------- |
| `type`            | `Literal["synthesize"]` | `"synthesize"` | **yes**  | Type discriminator                                       |
| `soul_ref`        | `str`                   | —              | **yes**  | Soul ID for the synthesizer                              |
| `input_block_ids` | `List[str]`             | —              | **yes**  | Block IDs whose outputs are combined (must be non-empty) |

minimal synthesize block

```yaml
blocks:
  combine:
    type: synthesize
    soul_ref: synthesizer
    input_block_ids: [research, code_review, design]
```

***

## dispatch

[Section titled “dispatch”](#dispatch)

Parallel branching — each exit port gets its own soul and task instruction. All branches execute concurrently.

| Field   | Type                    | Default      | Required | Description                                         |
| ------- | ----------------------- | ------------ | -------- | --------------------------------------------------- |
| `type`  | `Literal["dispatch"]`   | `"dispatch"` | **yes**  | Type discriminator                                  |
| `exits` | `List[DispatchExitDef]` | —            | **yes**  | Exit port definitions with per-branch soul and task |

### DispatchExitDef

[Section titled “DispatchExitDef”](#dispatchexitdef)

| Field      | Type  | Required | Description                             |
| ---------- | ----- | -------- | --------------------------------------- |
| `id`       | `str` | **yes**  | Unique exit port ID                     |
| `label`    | `str` | **yes**  | Human-readable label                    |
| `soul_ref` | `str` | **yes**  | Soul ID for this branch                 |
| `task`     | `str` | **yes**  | Task instruction for this branch’s soul |

Results are stored per-exit at `state.results["{block_id}.{exit_id}"]` and combined at `state.results[block_id]` as a JSON array.

minimal dispatch block

```yaml
blocks:
  analyze:
    type: dispatch
    exits:
      - id: sentiment
        label: Sentiment Analysis
        soul_ref: sentiment_analyst
        task: Analyze the sentiment of the input text.
      - id: entities
        label: Entity Extraction
        soul_ref: entity_extractor
        task: Extract all named entities from the input.
```

# CLI Reference

> Command-line interface for the Runsight server — runsight command, options, and Docker usage.

<!-- RUN-110 -->

The `runsight` command starts the Runsight server (FastAPI + Uvicorn). The CLI is intentionally minimal — it launches the server and serves the bundled GUI.

## Running with uvx

[Section titled “Running with uvx”](#running-with-uvx)

The recommended way to run Runsight is via `uvx`, which handles Python package resolution automatically:

```bash
uvx runsight
```

This installs the `runsight` package (if not already cached) and runs the `runsight` entry point, which is registered in `pyproject.toml` as:

```plaintext
[project.scripts]
runsight = "runsight_api.cli:main"
```

## Command syntax

[Section titled “Command syntax”](#command-syntax)

```plaintext
runsight [--host HOST] [--port PORT]
```

## Options

[Section titled “Options”](#options)

| Flag           | Type  | Default     | Description                      |
| -------------- | ----- | ----------- | -------------------------------- |
| `--host`       | `str` | `127.0.0.1` | Bind address for the server      |
| `--port`       | `int` | `8000`      | Bind port for the server         |
| `--help`, `-h` | —     | —           | Print usage information and exit |

Unknown arguments cause the CLI to print an error and exit with code 1.

## Examples

[Section titled “Examples”](#examples)

```bash
# Start with defaults (127.0.0.1:8000)
uvx runsight


# Custom port
uvx runsight --port 3000


# Bind to localhost only
uvx runsight --host 127.0.0.1 --port 9000


# Bind to all interfaces only when protected by your own network and auth controls
uvx runsight --host 0.0.0.0 --port 8000
```

On startup, the CLI prints:

```plaintext
  Runsight running at http://localhost:8000
  Press Ctrl+C to stop
```

## Docker

[Section titled “Docker”](#docker)

Runsight ships a multi-stage Dockerfile that bundles the frontend and backend into a single image.

```bash
# Build the image
docker build -t runsight .


# Run for local-only access
docker run -p 127.0.0.1:8000:8000 runsight


# Mount the whole workspace root so .runsight state persists too
docker run -p 127.0.0.1:8000:8000 -v "$(pwd)":/workspace runsight
```

### Environment variables

[Section titled “Environment variables”](#environment-variables)

| Variable              | Default       | Description                         |
| --------------------- | ------------- | ----------------------------------- |
| `RUNSIGHT_BASE_PATH`  | `/workspace`  | Root path for workflow discovery    |
| `RUNSIGHT_STATIC_DIR` | `/app/static` | Path to the bundled frontend assets |
| `RUNSIGHT_LOG_FORMAT` | `text`        | Log output format                   |

The container exposes port `8000` and includes a health check at `/health`.

Caution

Publishing the container on a non-loopback interface exposes an unauthenticated API unless you add your own proxy or authentication controls.

### Passing CLI flags in Docker

[Section titled “Passing CLI flags in Docker”](#passing-cli-flags-in-docker)

The default Docker `CMD` is `["runsight"]`. Override it to pass flags:

```bash
docker run -p 127.0.0.1:3000:3000 runsight runsight --host 0.0.0.0 --port 3000
```

<!-- Linear: RUN-821, RUN-847, RUN-848, RUN-944, RUN-943 — last verified against codebase 2026-04-26 -->

## Requirements

[Section titled “Requirements”](#requirements)

* Python >= 3.11
* The `runsight` package installs FastAPI, Uvicorn, SQLModel, and all other dependencies automatically.

# Direct API Invocation

> Reference for invoking Runsight workflows through the external Direct API.

Direct API invocation creates a production workflow run through an external HTTP request. The requested workflow must exist in the committed `main` workflow snapshot and requires `enabled: true`. Omitting `enabled` is treated the same as `enabled: false`, so a user must explicitly enable the workflow before it is externally invokable.

## Endpoint

[Section titled “Endpoint”](#endpoint)

```http
POST /api/workflows/{workflow_id}/runs
```

| Path parameter | Type   | Required | Description                                                                                                                                                 |
| -------------- | ------ | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `workflow_id`  | string | Yes      | Workflow ID to run. The server resolves the committed `main` workflow snapshot for this workflow and only invokes it when the snapshot has `enabled: true`. |

## Request body

[Section titled “Request body”](#request-body)

The request body is required and strict. It must contain only `inputs`.

```json
{
  "inputs": {
    "topic": "quantum computing trends"
  }
}
```

| Field    | Type   | Required | Description                                                                   |
| -------- | ------ | -------- | ----------------------------------------------------------------------------- |
| `inputs` | object | Yes      | Workflow input values. May be empty when the workflow has no required inputs. |

No other request body fields are accepted.

## Example request

[Section titled “Example request”](#example-request)

Invoke a workflow through the Direct API

```bash
curl -X POST http://localhost:8000/api/workflows/research-pipeline/runs \
  -H "Content-Type: application/json" \
  -H "x-request-id: req_01HTZK7V9QK4K4Z8Z9J4Z4Z4Z4" \
  -d '{
    "inputs": {
      "topic": "quantum computing trends"
    }
  }'
```

The `x-request-id` header is optional. When present, Runsight records it as the run source correlation ID.

## Rejected request body fields

[Section titled “Rejected request body fields”](#rejected-request-body-fields)

Direct API callers must not send privileged execution, provenance, delivery, simulation, or idempotency fields in the JSON body.

Examples of rejected fields:

| Field                   |
| ----------------------- |
| `workflow_id`           |
| `source`                |
| `branch`                |
| `commit_sha`            |
| `debug`                 |
| `simulation`            |
| `simulation_id`         |
| `simulation_branch`     |
| `idempotency_key`       |
| `idempotency_token`     |
| `trigger`               |
| `trigger_id`            |
| `delivery`              |
| `delivery_id`           |
| `provenance`            |
| `source_metadata`       |
| `source_correlation_id` |
| `caller`                |

Unsupported or privileged request body fields are rejected with `422 WORKFLOW_INPUT_VALIDATION_ERROR`.

## Success response

[Section titled “Success response”](#success-response)

A successful Direct API invocation returns `200` with a `RunResponse`.

`200` means run creation or launch was accepted. Workflow execution continues in the background, and the run may later be pending, running, failed, or completed.

The response uses the server run response schema and includes the created run details, including server-authored values such as `source: "api"` and `branch: "main"`.

## Server-authored fields

[Section titled “Server-authored fields”](#server-authored-fields)

Direct API always sets these run properties on the server.

| Property                       | Value                | Caller-controlled |
| ------------------------------ | -------------------- | ----------------- |
| `source`                       | `api`                | No                |
| `branch`                       | `main`               | No                |
| `source_metadata.entry_path`   | `direct_api`         | No                |
| `source_metadata.request_path` | Request path         | No                |
| `source_correlation_id`        | From request headers | Headers only      |

## Correlation headers

[Section titled “Correlation headers”](#correlation-headers)

Runsight reads correlation IDs from request headers.

| Header             | Description                      |
| ------------------ | -------------------------------- |
| `x-request-id`     | Request correlation ID           |
| `x-correlation-id` | Alternate request correlation ID |

`source_correlation_id` cannot be supplied in the request body.

## Workflow snapshot resolution

[Section titled “Workflow snapshot resolution”](#workflow-snapshot-resolution)

Direct API always resolves the committed `main` workflow snapshot and requires that snapshot to be enabled.

| Runtime asset                             | Resolution behavior                                      |
| ----------------------------------------- | -------------------------------------------------------- |
| Workflow YAML                             | Committed `main` snapshot with `enabled: true`           |
| Nested workflow YAML                      | Committed `main` snapshot through the resolved git ref   |
| Snapshot-capable workflow asset discovery | Receives the resolved git ref from the execution runtime |
| Provider settings                         | Live server runtime configuration                        |
| Server settings                           | Live server runtime configuration                        |
| API keys and credentials                  | Live server environment or configuration                 |

Dirty working tree edits to workflow YAML and nested workflow YAML are ignored by Direct API runs. Do not rely on dirty working tree edits for Direct API workflow definitions.

## Configuration

[Section titled “Configuration”](#configuration)

Environment variables use the `RUNSIGHT_` prefix.

| Setting                                         | Default   | Description                                                |
| ----------------------------------------------- | --------- | ---------------------------------------------------------- |
| `RUNSIGHT_EXTERNAL_INVOCATION_ENABLED`          | `true`    | Enables external Direct API invocation.                    |
| `RUNSIGHT_PUBLIC_BASE_URL`                      | unset     | Optional public base URL for external invocation contexts. |
| `RUNSIGHT_EXTERNAL_INVOCATION_BODY_LIMIT_BYTES` | `1048576` | Maximum Direct API request body size in bytes.             |
| `RUNSIGHT_MAX_CONCURRENT_RUNS`                  | `5`       | Maximum concurrent workflow runs.                          |
| `RUNSIGHT_MAX_PENDING_EXTERNAL_INVOCATIONS`     | `32`      | Maximum queued pending external invocations.               |

## Host binding defaults

[Section titled “Host binding defaults”](#host-binding-defaults)

The default host is `127.0.0.1`.

Caution

Runsight’s self-hosted API is unauthenticated today. Keep host binding and Docker port publishing on loopback unless you intentionally expose the service behind your own proxy and authentication controls.

Docker compose publishes the API on host loopback:

docker-compose.yml

```yaml
ports:
  - "127.0.0.1:8000:8000"
```

The Dockerfile command may bind to `0.0.0.0` inside the container. Host publishing examples should still bind to loopback unless you intentionally expose the service.

## Error responses

[Section titled “Error responses”](#error-responses)

| Status | Code                              | Description                                                                                                         |
| ------ | --------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| `404`  | `WORKFLOW_NOT_FOUND`              | The workflow ID does not resolve to a workflow, or the committed `main` workflow snapshot is not enabled.           |
| `413`  | `REQUEST_BODY_TOO_LARGE`          | The request body exceeds `RUNSIGHT_EXTERNAL_INVOCATION_BODY_LIMIT_BYTES`.                                           |
| `422`  | `WORKFLOW_INPUT_VALIDATION_ERROR` | The body shape is invalid, privileged fields are present, required inputs are missing, or input types do not match. |
| `429`  | `ADMISSION_SATURATED`             | Runtime admission limits are saturated.                                                                             |
| `503`  | `RUNTIME_UNAVAILABLE`             | The workflow runtime is unavailable or external invocation is disabled.                                             |

Direct API request and workflow input validation failures use `422`, not `400`.

## Idempotency

[Section titled “Idempotency”](#idempotency)

Direct API invocation in RUN-85 does not provide idempotency support.

| Input                           | Behavior                                            |
| ------------------------------- | --------------------------------------------------- |
| `x-idempotency-key` header      | Not persisted, exposed, or used for deduplication   |
| Request body idempotency fields | Rejected with `422 WORKFLOW_INPUT_VALIDATION_ERROR` |

## Redaction and metadata filtering

[Section titled “Redaction and metadata filtering”](#redaction-and-metadata-filtering)

Runsight redacts sensitive workflow input values by omitting the value from stored and returned `workflow_inputs` payloads.

`source_metadata` is server-authored for Direct API runs. Unsafe metadata keys are rejected or dropped. Accepted responses, detail responses, list responses, and stored payloads must not leak authorization data, raw sensitive input values, or unsupported idempotency fields.

## Availability of other external sources

[Section titled “Availability of other external sources”](#availability-of-other-external-sources)

Webhook and schedule invocation are not part of RUN-85.

| Source     | Availability              |
| ---------- | ------------------------- |
| `api`      | Shipped production source |
| `manual`   | Shipped production source |
| `webhook`  | Not part of RUN-85        |
| `schedule` | Not part of RUN-85        |

<!-- Linear: RUN-943 — last verified against codebase 2026-04-26 -->

# Unified Entity Identity

> Reference for entity ids, YAML identity fields, filename rules, and workflow references.

## Identity Model

[Section titled “Identity Model”](#identity-model)

Runsight uses one identity model for persisted YAML-backed entities.

| Concept          | Value                                               | Notes                                                   |
| ---------------- | --------------------------------------------------- | ------------------------------------------------------- |
| Entity kinds     | `soul`, `workflow`, `tool`, `provider`, `assertion` | There is no `step` kind.                                |
| Entity reference | `{kind}:{id}`                                       | Used in identity-related messages, such as `tool:http`. |
| Embedded id      | `id`                                                | Canonical machine identity.                             |
| Embedded kind    | `kind`                                              | Must match the entity family.                           |
| Display name     | `name` or `workflow.name`                           | Human label, not identity.                              |
| Filename stem    | `{id}`                                              | Must match the embedded id exactly.                     |

entity ref

```python
from runsight_core.identity import EntityKind, EntityRef


ref = EntityRef(kind=EntityKind.WORKFLOW, id="research-review")
str(ref)  # "workflow:research-review"
```

## ID Rules

[Section titled “ID Rules”](#id-rules)

Every entity id must satisfy the shared entity id validator.

| Rule              | Constraint                                                                   |
| ----------------- | ---------------------------------------------------------------------------- |
| Length            | 3 to 100 characters                                                          |
| Start             | Lowercase letter                                                             |
| End               | Lowercase letter or digit                                                    |
| Middle characters | Lowercase letters, digits, `_`, or `-`                                       |
| Reserved ids      | `pause`, `resume`, `kill`, `cancel`, `status`, `http`, `file_io`, `delegate` |

Valid examples:

* `researcher`
* `research-review`
* `slack_webhook`
* `openai`

Invalid examples:

* `AI-review`
* `99-review`
* `ab`
* `tool/evil`
* `http`

## YAML Examples

[Section titled “YAML Examples”](#yaml-examples)

### Workflow

[Section titled “Workflow”](#workflow)

custom/workflows/summarizer.yaml

```yaml
version: "1.0"
id: summarizer
kind: workflow


interface:
  inputs:
    - name: topic
      target: shared_memory.topic
      type: string
      required: true
  outputs:
    - name: summary
      source: results.summarize
      type: string


souls:
  writer:
    id: writer
    kind: soul
    name: Writer
    role: Summarizer
    system_prompt: "Write a concise summary of shared_memory.topic."


blocks:
  summarize:
    type: linear
    soul_ref: writer


workflow:
  name: Summarizer
  entry: summarize
```

### Soul

[Section titled “Soul”](#soul)

custom/souls/researcher.yaml

```yaml
id: researcher
kind: soul
name: Senior Researcher
role: Researcher
system_prompt: "Find the relevant facts and cite the evidence."
provider: openai
model_name: gpt-4o
```

### Tool

[Section titled “Tool”](#tool)

custom/tools/slack\_payload\_builder.yaml

```yaml
version: "1.0"
id: slack_payload_builder
kind: tool
type: custom
executor: python
name: Slack Payload Builder
description: Build a JSON payload string for Slack.
parameters:
  type: object
  properties:
    text:
      type: string
      description: Message text to encode for Slack.
  required:
    - text
code: |
  import json


  def main(args):
      text = str(args.get("text", ""))
      return {"payload_json": json.dumps({"text": text})}
```

### Assertion

[Section titled “Assertion”](#assertion)

custom/assertions/budget\_guard.yaml

```yaml
version: "1.0"
id: budget_guard
kind: assertion
name: Budget Guard
description: Keeps cost under budget.
returns: bool
source: budget_guard.py
```

custom/assertions/budget\_guard.py

```python
def get_assert(output, context):
    return True
```

### Provider

[Section titled “Provider”](#provider)

custom/providers/openai.yaml

```yaml
id: openai
kind: provider
name: OpenAI
type: openai
is_active: true
```

## Filename Convention

[Section titled “Filename Convention”](#filename-convention)

Use the embedded id as the filename stem.

| Kind        | Directory            | Example                                 |
| ----------- | -------------------- | --------------------------------------- |
| `workflow`  | `custom/workflows/`  | `custom/workflows/research-review.yaml` |
| `soul`      | `custom/souls/`      | `custom/souls/researcher.yaml`          |
| `tool`      | `custom/tools/`      | `custom/tools/slack_webhook.yaml`       |
| `provider`  | `custom/providers/`  | `custom/providers/openai.yaml`          |
| `assertion` | `custom/assertions/` | `custom/assertions/budget_guard.yaml`   |

The embedded `id` must match the filename stem exactly. `custom/workflows/summarizer.yaml` must contain `id: summarizer`.

## Scanner Behavior

[Section titled “Scanner Behavior”](#scanner-behavior)

Scanners extract identity from YAML content.

| Scanner           | Required identity          | Rejection behavior                                                         |
| ----------------- | -------------------------- | -------------------------------------------------------------------------- |
| Workflow scanner  | `id`, `kind: workflow`     | Missing fields, invalid ids, duplicate ids, and id/stem mismatches fail.   |
| Soul scanner      | `id`, `kind: soul`, `name` | Missing fields, invalid ids, duplicate ids, and id/stem mismatches fail.   |
| Tool scanner      | `id`, `kind: tool`         | Reserved builtin ids, invalid ids, and id/stem mismatches fail.            |
| Assertion scanner | `id`, `kind: assertion`    | Builtin assertion id collisions, invalid ids, and id/stem mismatches fail. |

Use `ScanIndex.ids()` and `without_ids()` when working with discovered entities. Stem-based helpers are not part of the final identity API.

## Repository Behavior

[Section titled “Repository Behavior”](#repository-behavior)

Repositories persist by embedded id.

| Repository          | Behavior                                                                                                          |
| ------------------- | ----------------------------------------------------------------------------------------------------------------- |
| Workflow repository | Creates and updates `custom/workflows/{id}.yaml`. Duplicate create fails before write. Update cannot change `id`. |
| Soul repository     | Uses `custom/souls/{id}.yaml` through the shared YAML repository behavior.                                        |
| Provider repository | Creates and updates `custom/providers/{id}.yaml`, with provider ids validated by the shared rules.                |

Repositories reject malformed or mismatched identity. They do not repair a missing id from the filename, and they do not derive ids from display names.

## Workflow References

[Section titled “Workflow References”](#workflow-references)

`workflow_ref` resolves by embedded workflow id only.

Accepted:

```yaml
blocks:
  summarize:
    type: workflow
    workflow_ref: summarizer
```

Rejected:

* `custom/workflows/summarizer.yaml`
* `summarizer.yaml`
* `Summarizer`
* relative paths
* workflow display names

The resolver does not apply path, stem, relative-path, or display-name aliases.

## Runs, Simulations, and Forks

[Section titled “Runs, Simulations, and Forks”](#runs-simulations-and-forks)

New run records store the embedded workflow id in `workflow_id`. Historical run records that contain old ids are not backfilled.

A simulation branch keeps the same embedded workflow id. The simulation is identified by the branch plus the workflow id, not by mutating the workflow YAML.

A fork is different from a simulation. Forking creates a disabled draft workflow with a new embedded workflow id. Nested `workflow_ref` values remain unchanged.

## GUI Create Contract

[Section titled “GUI Create Contract”](#gui-create-contract)

Workflow creation in the GUI submits YAML that already contains `id` and `kind: workflow`.

workflow draft

```yaml
version: "1.0"
id: research-review
kind: workflow
workflow:
  name: Research Review
  entry: start
blocks:
  start:
    type: code
    code: |
      def main(data):
          return {"ok": True}
```

The create UI exposes an editable workflow id. The default id is derived from the workflow name, but the user can change it before creation. Backend validation remains the final authority.

Provider creation also carries embedded identity in the request body:

POST /api/settings/providers

```json
{
  "id": "openai",
  "kind": "provider",
  "name": "OpenAI",
  "api_key_env": "OPENAI_API_KEY"
}
```

## Adding a YAML-Backed Entity Kind

[Section titled “Adding a YAML-Backed Entity Kind”](#adding-a-yaml-backed-entity-kind)

Use this checklist when adding a future YAML-backed entity kind:

1. Add the kind to `EntityKind`.
2. Require a literal `kind` field in the schema.
3. Require `id` and validate it with the shared id rules.
4. Enforce `id == filename stem`.
5. Extract ids from YAML in the scanner.
6. Reject duplicate embedded ids.
7. Resolve references by embedded id only.
8. Add repository create/update validation.
9. Update GUI create flows if users can create the entity from the UI.
10. Add targeted tests and docs.

## Scope Exclusions

[Section titled “Scope Exclusions”](#scope-exclusions)

Unified entity identity does not define:

* `EntityKind.STEP`
* display-name, path, or filename aliases for `workflow_ref`
* a separate simulation entity id
* historical run backfills
* stem-based APIs such as `ScanIndex.stems()` or `without_stems()`

Runtime step wrappers still exist in execution code, but they are Python/plugin execution concepts, not persisted YAML entity identity.

<!-- Linear: RUN-773, RUN-835, RUN-846 — last verified against codebase 2026-04-13 -->

# ADR: Unified Entity Identity

> Why Runsight uses embedded kind and id fields for YAML-backed entities.

## Status

[Section titled “Status”](#status)

Accepted.

YAML-backed Runsight entities use embedded `kind` and `id` fields as their source of truth. Filenames still matter, but only as a storage invariant: the filename stem must equal the embedded id.

## Context

[Section titled “Context”](#context)

Runsight stores workflows, souls, tools, providers, and assertions as YAML-backed assets. Before unified identity, different paths could treat a filename stem, display name, file path, or embedded field as the entity identity.

That made reference resolution ambiguous. A `workflow_ref` could look like a file path in one path and a workflow name in another. Create flows could produce files whose storage name did not match the identity used by scanners or execution. Error messages also used bare ids, which made it unclear whether a message referred to a soul, workflow, tool, provider, or assertion.

The identity model now gives every YAML-backed entity the same shape and the same validation boundary.

## Decision

[Section titled “Decision”](#decision)

Runsight identifies YAML-backed entities by `kind + id`.

| Decision                | Result                                                          |
| ----------------------- | --------------------------------------------------------------- |
| Entity kinds            | `soul`, `workflow`, `tool`, `provider`, `assertion`             |
| Entity reference format | `{kind}:{id}`, such as `soul:researcher`                        |
| YAML identity           | `kind` and `id` are required in every YAML-backed entity        |
| Filename convention     | Store YAML at `custom/<kind plural>/<id>.yaml`                  |
| Filename validation     | Reject YAML when embedded `id` does not equal the filename stem |
| Workflow references     | Resolve `workflow_ref` by embedded workflow id only             |
| Display names           | Human labels only; never canonical identity                     |

There is no defaulted `kind` in the final contract. YAML must carry the field explicitly.

## Scope

[Section titled “Scope”](#scope)

This decision applies to YAML-backed entities:

* souls in `custom/souls/`
* workflows in `custom/workflows/`
* tools in `custom/tools/`
* providers in `custom/providers/`
* assertions in `custom/assertions/`

It also applies to scanners, repositories, workflow reference resolution, identity-related messages, GUI create payloads, forked workflows, and simulation/run contracts.

This decision does not add a `step` entity kind. Runtime step discovery is Python plugin discovery and is outside this YAML identity model.

## Consequences

[Section titled “Consequences”](#consequences)

The runtime has one lookup story. It no longer guesses whether a value is a path, a filename stem, a display name, or an embedded id.

The trade-off is stricter validation:

* ids must follow the shared entity id rules
* reserved ids are rejected
* id/filename mismatches fail instead of being repaired
* duplicate embedded ids fail instead of silently overwriting
* old path/name aliases for `workflow_ref` are not accepted

Compatibility shims such as `ScanIndex.stems()` and `without_stems()` are not part of the final state.

## Migration Contracts

[Section titled “Migration Contracts”](#migration-contracts)

New YAML must use the embedded id contract. Existing production YAML was migrated to the same shape during the RUN-773 work.

| Contract            | Required behavior                                                 |
| ------------------- | ----------------------------------------------------------------- |
| Entity id           | Embedded `id` is the canonical id                                 |
| Entity kind         | Embedded `kind` identifies the entity family                      |
| Filename            | Filename stem equals embedded id                                  |
| Workflow ref        | `workflow_ref` is the embedded workflow id                        |
| Run records         | New records store the embedded workflow id                        |
| Historical runs     | Old run records are not backfilled                                |
| Simulation branches | Simulation YAML must not mutate workflow id                       |
| Forked workflows    | Forks get a new embedded workflow id                              |
| GUI create          | Create flows submit YAML with `kind: workflow` and an editable id |

Simulation identity is the pair `branch + workflow_id`; there is no separate simulation entity id.

## Future YAML-Backed Kinds

[Section titled “Future YAML-Backed Kinds”](#future-yaml-backed-kinds)

Adding a future YAML-backed entity kind requires the same model:

1. Add an `EntityKind` value.
2. Require a literal `kind` field in the schema.
3. Require and validate `id`.
4. Enforce `id == filename stem`.
5. Index and resolve by embedded id.
6. Reject duplicates.
7. Add targeted tests and docs.

Do not add path, stem, or display-name aliases as compatibility fallbacks.

<!-- Linear: RUN-773, RUN-835, RUN-846 — last verified against codebase 2026-04-13 -->

# YAML Schema Reference

> Complete annotated reference for all Runsight YAML file types — workflow files, soul files, and custom tool files.

<!-- RUN-110, RUN-490 -->

Runsight uses YAML files for workflows, souls, tools, providers, and assertions. This page focuses on workflow, soul, and custom tool schemas. For a guided walkthrough of workflow files, see [YAML Schema](/docs/workflows/yaml-schema).

## Workflow files

[Section titled “Workflow files”](#workflow-files)

Workflow files live in `custom/workflows/` and define the full execution graph. The root model is `RunsightWorkflowFile`.

### Top-level fields

[Section titled “Top-level fields”](#top-level-fields)

| Field       | Type                   | Default | Required | Description                                                                |
| ----------- | ---------------------- | ------- | -------- | -------------------------------------------------------------------------- |
| `id`        | `str`                  | —       | **yes**  | Embedded workflow id. Must match the filename stem.                        |
| `kind`      | `"workflow"`           | —       | **yes**  | Entity kind. Must be `"workflow"`.                                         |
| `version`   | `str`                  | `"1.0"` | no       | Schema version                                                             |
| `enabled`   | `bool`                 | `false` | no       | Whether the workflow is active                                             |
| `config`    | `Dict[str, Any]`       | `{}`    | no       | Arbitrary workflow configuration                                           |
| `interface` | `WorkflowInterfaceDef` | none    | no       | Public input/output contract for callable sub-workflows                    |
| `tools`     | `List[str]`            | `[]`    | no       | Tool IDs available to souls in this workflow. Duplicates are rejected.     |
| `souls`     | `Dict[str, SoulDef]`   | `{}`    | no       | Inline soul definitions. The dict key must match the soul’s `id` field.    |
| `blocks`    | `Dict[str, BlockDef]`  | `{}`    | no       | Block definitions keyed by block ID. Uses a discriminated union on `type`. |
| `workflow`  | `WorkflowDef`          | —       | **yes**  | Graph metadata (name, entry point, transitions)                            |
| `limits`    | `WorkflowLimitsDef`    | none    | no       | Workflow-level budget constraints                                          |
| `eval`      | `EvalSectionDef`       | none    | no       | Embedded test cases for offline evaluation                                 |

### WorkflowDef

[Section titled “WorkflowDef”](#workflowdef)

| Field                     | Type                             | Default | Required | Description                            |
| ------------------------- | -------------------------------- | ------- | -------- | -------------------------------------- |
| `name`                    | `str`                            | —       | **yes**  | Workflow name                          |
| `entry`                   | `str`                            | —       | **yes**  | Block ID to start execution            |
| `transitions`             | `List[TransitionDef]`            | `[]`    | no       | Simple A to B transitions              |
| `conditional_transitions` | `List[ConditionalTransitionDef]` | `[]`    | no       | Multi-path transitions based on output |

### TransitionDef

[Section titled “TransitionDef”](#transitiondef)

| Field  | Type            | Required | Description                             |
| ------ | --------------- | -------- | --------------------------------------- |
| `from` | `str`           | **yes**  | Source block ID                         |
| `to`   | `str` or `null` | no       | Target block ID, or `null` for terminal |

### ConditionalTransitionDef

[Section titled “ConditionalTransitionDef”](#conditionaltransitiondef)

| Field          | Type            | Required | Description                            |
| -------------- | --------------- | -------- | -------------------------------------- |
| `from`         | `str`           | **yes**  | Source block ID                        |
| `default`      | `str` or `null` | no       | Fallback target if no key matches      |
| *(extra keys)* | `str`           | no       | Decision key mapped to target block ID |

### WorkflowInterfaceDef

[Section titled “WorkflowInterfaceDef”](#workflowinterfacedef)

Public callable contract for sub-workflows.

| Field     | Type                               | Default | Description                              |
| --------- | ---------------------------------- | ------- | ---------------------------------------- |
| `inputs`  | `List[WorkflowInterfaceInputDef]`  | `[]`    | Input parameters. Names must be unique.  |
| `outputs` | `List[WorkflowInterfaceOutputDef]` | `[]`    | Output parameters. Names must be unique. |

#### WorkflowInterfaceInputDef

[Section titled “WorkflowInterfaceInputDef”](#workflowinterfaceinputdef)

| Field         | Type   | Default | Required | Description                           |
| ------------- | ------ | ------- | -------- | ------------------------------------- |
| `name`        | `str`  | —       | **yes**  | Input parameter name (must be unique) |
| `target`      | `str`  | —       | **yes**  | Dot-notation path to child state key  |
| `type`        | `str`  | none    | no       | Type hint                             |
| `required`    | `bool` | `true`  | no       | Whether input must be provided        |
| `default`     | `Any`  | none    | no       | Default value if not provided         |
| `description` | `str`  | none    | no       | Human-readable description            |

#### WorkflowInterfaceOutputDef

[Section titled “WorkflowInterfaceOutputDef”](#workflowinterfaceoutputdef)

| Field         | Type  | Default | Required | Description                            |
| ------------- | ----- | ------- | -------- | -------------------------------------- |
| `name`        | `str` | —       | **yes**  | Output parameter name (must be unique) |
| `source`      | `str` | —       | **yes**  | Dot-notation path to child result      |
| `type`        | `str` | none    | no       | Type hint                              |
| `description` | `str` | none    | no       | Human-readable description             |

### WorkflowLimitsDef

[Section titled “WorkflowLimitsDef”](#workflowlimitsdef)

| Field                  | Type    | Default  | Constraints          | Description                               |
| ---------------------- | ------- | -------- | -------------------- | ----------------------------------------- |
| `max_duration_seconds` | `int`   | none     | 1—86400              | Maximum wall-clock time                   |
| `cost_cap_usd`         | `float` | none     | >= 0.0               | Maximum cost in USD                       |
| `token_cap`            | `int`   | none     | >= 1                 | Maximum total tokens                      |
| `on_exceed`            | `str`   | `"fail"` | `"warn"` or `"fail"` | Action when limit is exceeded             |
| `warn_at_pct`          | `float` | `0.8`    | 0.0—1.0              | Threshold percentage to trigger a warning |

### BlockLimitsDef

[Section titled “BlockLimitsDef”](#blocklimitsdef)

Same as `WorkflowLimitsDef` but without `warn_at_pct`. Applied per block via the `limits` field.

| Field                  | Type    | Default  | Constraints          | Description                   |
| ---------------------- | ------- | -------- | -------------------- | ----------------------------- |
| `max_duration_seconds` | `int`   | none     | 1—86400              | Maximum wall-clock time       |
| `cost_cap_usd`         | `float` | none     | >= 0.0               | Maximum cost in USD           |
| `token_cap`            | `int`   | none     | >= 1                 | Maximum total tokens          |
| `on_exceed`            | `str`   | `"fail"` | `"warn"` or `"fail"` | Action when limit is exceeded |

### Block types

[Section titled “Block types”](#block-types)

The `blocks` dict uses a discriminated union on the `type` field. Each block type extends `BaseBlockDef` and adds its own fields. The following sections document block types with non-trivial additional fields.

#### WorkflowBlockDef (`type: "workflow"`)

[Section titled “WorkflowBlockDef (type: "workflow")”](#workflowblockdef-type-workflow)

Calls a child workflow as a sub-workflow (hierarchical state machine). The parent block binds values into the child’s declared `interface` and reads results back out after the child completes.

| Field          | Type                   | Default                   | Required | Description                                                                                      |
| -------------- | ---------------------- | ------------------------- | -------- | ------------------------------------------------------------------------------------------------ |
| `type`         | `"workflow"`           | —                         | **yes**  | Discriminator                                                                                    |
| `workflow_ref` | `str`                  | —                         | **yes**  | Embedded workflow id of the child workflow to call                                               |
| `inputs`       | `Dict[str, str]`       | none                      | no       | Maps child interface input names to parent dotted paths (e.g. `topic: shared_memory.topic`)      |
| `outputs`      | `Dict[str, str]`       | none                      | no       | Maps parent dotted paths to child interface output names (e.g. `shared_memory.summary: summary`) |
| `max_depth`    | `int`                  | none (runtime default 10) | no       | Maximum HSM recursion depth                                                                      |
| `on_error`     | `"raise"` or `"catch"` | `"raise"`                 | no       | `"catch"` swallows child failure and returns an error exit handle instead of propagating         |

All inherited `BaseBlockDef` fields (`stateful`, `routes`, `depends`, `error_route`, `retry_config`, `exits`, `exit_conditions`, `timeout_seconds`, `limits`, etc.) are also available.

**Validation rules:**

* `inputs` keys must be plain interface names (no dots). They reference the child workflow’s `interface.inputs[].name`.
* `outputs` values must be plain interface names (no dots). They reference the child workflow’s `interface.outputs[].name`.

#### Sub-workflow example

[Section titled “Sub-workflow example”](#sub-workflow-example)

The parent workflow defines a `workflow` block that calls a child. The child declares its callable contract via the top-level `interface` section.

**Parent workflow** — calls the child and wires data in and out:

custom/workflows/research\_pipeline.yaml

```yaml
version: "1.0"
id: research_pipeline
kind: workflow
enabled: true


blocks:
  run_analysis:
    type: workflow
    workflow_ref: analysis_subworkflow
    inputs:
      topic: shared_memory.topic
      depth: shared_memory.analysis_depth
    outputs:
      shared_memory.summary: summary
      shared_memory.citations: sources
    max_depth: 5
    on_error: catch
    timeout_seconds: 600


workflow:
  name: Research Pipeline
  entry: run_analysis
  transitions:
    - from: run_analysis
      to: null
```

**Child workflow** — declares the interface contract the parent binds to:

custom/workflows/analysis\_subworkflow\.yaml

```yaml
version: "1.0"
id: analysis_subworkflow
kind: workflow
enabled: true


interface:
  inputs:
    - name: topic
      target: shared_memory.topic
      type: string
      required: true
      description: The research topic to analyze
    - name: depth
      target: shared_memory.depth
      type: integer
      required: false
      default: 3
      description: How many layers deep to research
  outputs:
    - name: summary
      source: shared_memory.final_summary
      type: string
      description: Completed analysis summary
    - name: sources
      source: shared_memory.collected_sources
      type: list
      description: List of cited sources


souls:
  analyst:
    id: analyst
    kind: soul
    name: Research Analyst
    role: Research Analyst
    system_prompt: "Analyze the topic in shared_memory.topic."


blocks:
  analyze:
    type: soul
    soul_ref: analyst
    task: "Research the topic and write a summary."


workflow:
  name: Analysis Sub-Workflow
  entry: analyze
  transitions:
    - from: analyze
      to: null
```

In the parent, `inputs` keys (`topic`, `depth`) match the child’s `interface.inputs[].name` values. The parent values (`shared_memory.topic`, `shared_memory.analysis_depth`) are dotted paths into the parent’s own state.

In the parent, `outputs` values (`summary`, `sources`) match the child’s `interface.outputs[].name` values. The parent keys (`shared_memory.summary`, `shared_memory.citations`) are dotted paths where results are written in the parent’s state.

### EvalSectionDef

[Section titled “EvalSectionDef”](#evalsectiondef)

| Field       | Type                | Default | Constraints            | Description                       |
| ----------- | ------------------- | ------- | ---------------------- | --------------------------------- |
| `threshold` | `float`             | none    | 0.0—1.0                | Pass rate threshold for the suite |
| `cases`     | `List[EvalCaseDef]` | —       | min 1 item, unique IDs | Test case definitions             |

#### EvalCaseDef

[Section titled “EvalCaseDef”](#evalcasedef)

| Field         | Type                              | Default | Description                               |
| ------------- | --------------------------------- | ------- | ----------------------------------------- |
| `id`          | `str`                             | —       | Unique test case ID (strict string)       |
| `description` | `str`                             | none    | Human-readable description                |
| `inputs`      | `Dict[str, Any]`                  | none    | Input data for the workflow               |
| `fixtures`    | `Dict[str, str]`                  | none    | Block ID to mock output (skips LLM calls) |
| `expected`    | `Dict[str, List[Dict[str, Any]]]` | none    | Block ID to list of assertion configs     |

### Supporting models

[Section titled “Supporting models”](#supporting-models)

#### ConditionDef

[Section titled “ConditionDef”](#conditiondef)

| Field      | Type  | Default | Description                                                           |
| ---------- | ----- | ------- | --------------------------------------------------------------------- |
| `eval_key` | `str` | —       | Dot-notation path into block’s own result                             |
| `operator` | `str` | —       | Comparison operator                                                   |
| `value`    | `Any` | none    | Comparison value (none for unary operators like `is_empty`, `exists`) |

#### ConditionGroupDef

[Section titled “ConditionGroupDef”](#conditiongroupdef)

| Field        | Type                 | Default | Description                   |
| ------------ | -------------------- | ------- | ----------------------------- |
| `combinator` | `str`                | `"and"` | `"and"` or `"or"`             |
| `conditions` | `List[ConditionDef]` | —       | List of conditions to combine |

#### CaseDef

[Section titled “CaseDef”](#casedef)

| Field             | Type                | Default | Description                                          |
| ----------------- | ------------------- | ------- | ---------------------------------------------------- |
| `case_id`         | `str`               | —       | Unique case identifier                               |
| `condition_group` | `ConditionGroupDef` | none    | Conditions for this case (none when `default: true`) |
| `default`         | `bool`              | `false` | Whether this is the default/fallback case            |

#### RouteDef

[Section titled “RouteDef”](#routedef)

Shorthand route definition that compiles into output conditions and transitions.

| Field     | Type                | Default | Description                                |
| --------- | ------------------- | ------- | ------------------------------------------ |
| `case`    | `str`               | —       | Case identifier (YAML alias for `case_id`) |
| `when`    | `ConditionGroupDef` | none    | Conditions for this route                  |
| `goto`    | `str`               | —       | Target block ID                            |
| `default` | `bool`              | `false` | Whether this is the default route          |

Exactly one route must have `default: true`. Route `case` values must be unique.

#### InputRef

[Section titled “InputRef”](#inputref)

| Field  | Type  | Description                                                                     |
| ------ | ----- | ------------------------------------------------------------------------------- |
| `from` | `str` | Dot-notation reference to upstream block output (e.g. `"step_id.output_field"`) |

#### ExitDef

[Section titled “ExitDef”](#exitdef)

| Field   | Type  | Description          |
| ------- | ----- | -------------------- |
| `id`    | `str` | Unique exit port ID  |
| `label` | `str` | Human-readable label |

#### DispatchExitDef

[Section titled “DispatchExitDef”](#dispatchexitdef)

Extends `ExitDef` with per-exit soul and task instruction.

| Field      | Type  | Description                             |
| ---------- | ----- | --------------------------------------- |
| `id`       | `str` | Unique exit port ID                     |
| `label`    | `str` | Human-readable label                    |
| `soul_ref` | `str` | Soul ID for this branch                 |
| `task`     | `str` | Task instruction for this branch’s soul |

#### ExitCondition

[Section titled “ExitCondition”](#exitcondition)

| Field         | Type  | Default | Description                        |
| ------------- | ----- | ------- | ---------------------------------- |
| `contains`    | `str` | none    | Substring match against output     |
| `regex`       | `str` | none    | Regex pattern match against output |
| `exit_handle` | `str` | —       | Exit handle value to set on match  |

#### RetryConfig

[Section titled “RetryConfig”](#retryconfig)

| Field                  | Type        | Default   | Constraints                  | Description                            |
| ---------------------- | ----------- | --------- | ---------------------------- | -------------------------------------- |
| `max_attempts`         | `int`       | `3`       | 1—20                         | Maximum retry attempts                 |
| `backoff`              | `str`       | `"fixed"` | `"fixed"` or `"exponential"` | Backoff strategy                       |
| `backoff_base_seconds` | `float`     | `1.0`     | 0.1—60.0                     | Base delay between retries             |
| `non_retryable_errors` | `List[str]` | none      | —                            | Error types that should not be retried |

***

## Soul files

[Section titled “Soul files”](#soul-files)

Soul files live in `custom/souls/` as standalone YAML files (one soul per file). The embedded `id` becomes the soul key, and for external files it must match the filename stem. Soul files are flat — no wrapper object, just the fields directly.

### SoulDef fields

[Section titled “SoulDef fields”](#souldef-fields)

| Field                 | Type        | Default | Required | Description                                               |
| --------------------- | ----------- | ------- | -------- | --------------------------------------------------------- |
| `id`                  | `str`       | —       | **yes**  | Embedded soul id                                          |
| `kind`                | `"soul"`    | —       | **yes**  | Entity kind                                               |
| `name`                | `str`       | —       | **yes**  | Display name                                              |
| `role`                | `str`       | —       | **yes**  | The role of the agent (e.g. `"Senior Researcher"`)        |
| `system_prompt`       | `str`       | —       | **yes**  | System instructions defining the agent’s behavior         |
| `tools`               | `List[str]` | none    | no       | Tool name references available to this soul               |
| `required_tool_calls` | `List[str]` | none    | no       | Tool function names that must be called before completion |
| `max_tool_iterations` | `int`       | `5`     | no       | Maximum tool-use iterations per execution                 |
| `model_name`          | `str`       | none    | no       | Model override (uses runner default if not set)           |
| `provider`            | `str`       | none    | no       | Provider override for the selected model                  |
| `temperature`         | `float`     | none    | no       | Sampling temperature override                             |
| `max_tokens`          | `int`       | none    | no       | Output token limit override                               |
| `avatar_color`        | `str`       | none    | no       | UI color hint for displaying the soul                     |
| `modified_at`         | `str`       | none    | no       | Timestamp of last modification                            |

### Example soul file

[Section titled “Example soul file”](#example-soul-file)

custom/souls/researcher.yaml

```yaml
id: researcher
kind: soul
name: Researcher
role: Senior Researcher
system_prompt: >
  You are a senior researcher. Analyze the given topic thoroughly
  and produce a structured research report with citations.
tools: null
max_tool_iterations: 5
model_name: gpt-4.1-mini
provider: openai
temperature: 0.7
max_tokens: null
avatar_color: primary
modified_at: null
```

Souls can also be defined inline in a workflow file under the `souls:` section. When inline, the dict key must match the soul’s `id` field:

workflow with inline soul

```yaml
souls:
  my_analyst:
    id: my_analyst
    kind: soul
    name: Analyst
    role: Analyst
    system_prompt: "Analyze the data."
```

If a soul file and an inline soul share the same key, the inline definition takes precedence and a warning is logged.

***

## Custom tool files

[Section titled “Custom tool files”](#custom-tool-files)

Custom tool files live in `custom/tools/` as standalone YAML files (one tool per file). The embedded `id` becomes the tool ID, and it must match the filename stem. Reserved builtin tool IDs (`http`, `file_io`, `delegate`) cannot be used.

### Tool file fields

[Section titled “Tool file fields”](#tool-file-fields)

| Field             | Type     | Default | Required    | Description                                                                                          |
| ----------------- | -------- | ------- | ----------- | ---------------------------------------------------------------------------------------------------- |
| `version`         | `str`    | —       | **yes**     | Schema version (e.g. `"1.0"`)                                                                        |
| `id`              | `str`    | —       | **yes**     | Embedded tool id                                                                                     |
| `kind`            | `"tool"` | —       | **yes**     | Entity kind                                                                                          |
| `type`            | `str`    | —       | **yes**     | Must be `"custom"`                                                                                   |
| `executor`        | `str`    | —       | **yes**     | `"python"` or `"request"`                                                                            |
| `name`            | `str`    | —       | **yes**     | Human-readable tool name                                                                             |
| `description`     | `str`    | —       | **yes**     | Description of what the tool does                                                                    |
| `parameters`      | `Dict`   | —       | **yes**     | JSON Schema object describing the tool’s input parameters                                            |
| `code`            | `str`    | none    | conditional | Python source code with `def main(args)`. Required for `executor: python` unless `code_file` is set. |
| `code_file`       | `str`    | none    | conditional | Path to external Python file (relative to tool YAML). Mutually exclusive with `code`.                |
| `request`         | `Dict`   | none    | conditional | HTTP request configuration. Required for `executor: request`.                                        |
| `timeout_seconds` | `int`    | none    | no          | Request timeout in seconds. Only valid for `executor: request`. Must be a positive integer.          |

### Request configuration (executor: request)

[Section titled “Request configuration (executor: request)”](#request-configuration-executor-request)

| Field           | Type             | Default | Required | Description                                           |
| --------------- | ---------------- | ------- | -------- | ----------------------------------------------------- |
| `method`        | `str`            | `"GET"` | **yes**  | HTTP method                                           |
| `url`           | `str`            | —       | **yes**  | Request URL. Supports `${ENV_VAR}` substitution.      |
| `headers`       | `Dict[str, str]` | `{}`    | no       | Request headers                                       |
| `body_template` | `str`            | none    | no       | Request body template with `{{ param }}` substitution |
| `response_path` | `str`            | none    | no       | JSONPath to extract from response                     |

### Example: Python executor

[Section titled “Example: Python executor”](#example-python-executor)

custom/tools/slack\_payload\_builder.yaml

```yaml
version: "1.0"
id: slack_payload_builder
kind: tool
type: custom
executor: python
name: Slack Payload Builder
description: Build a JSON payload string for the Slack incoming webhook.
parameters:
  type: object
  properties:
    text:
      type: string
      description: Message text to encode for Slack.
  required:
    - text
code: |
  import json


  def main(args):
      text = str(args.get("text", ""))
      return {"payload_json": json.dumps({"text": text})}
```

### Example: Request executor

[Section titled “Example: Request executor”](#example-request-executor)

custom/tools/slack\_webhook.yaml

```yaml
version: "1.0"
id: slack_webhook
kind: tool
type: custom
executor: request
name: Slack Webhook
description: Send a message to the configured Slack incoming webhook.
parameters:
  type: object
  properties:
    payload_json:
      type: string
      description: Complete JSON payload to send to Slack.
  required:
    - payload_json
request:
  method: POST
  url: "${SLACK_WEBHOOK_URL}"
  headers:
    Content-type: application/json
  body_template: "{{ payload_json }}"
timeout_seconds: 10
```

***

## JSON schema for editors

[Section titled “JSON schema for editors”](#json-schema-for-editors)

A JSON schema is auto-generated from the Pydantic models for Monaco editor autocomplete:

```bash
python packages/core/scripts/generate_schema.py          # generate to disk
python packages/core/scripts/generate_schema.py --check   # CI mode: exit 1 if out of sync
```

The generated schema lives at `packages/core/runsight-workflow-schema.json`.

# Inline Souls

> How to define souls directly inside a workflow YAML file for quick prototyping.

Inline souls let you define an agent identity directly inside a workflow YAML file, without creating a separate file in `custom/souls/`. This is useful for quick prototyping, throwaway experiments, or self-contained example workflows.

## When to use inline souls

[Section titled “When to use inline souls”](#when-to-use-inline-souls)

Use inline souls when:

* You are **prototyping** a workflow and want to iterate fast without switching between files.
* You need a **one-off soul** that does not need to be reused across workflows.
* You want a **self-contained workflow file** that someone can run without additional soul files.

Use external soul files (in `custom/souls/`) when:

* The soul is **shared across multiple workflows**.
* You want the soul to appear in the **Soul Library UI** for visual management.
* You need **dependency tracking** — the UI shows which workflows use a given soul.
* The soul is a **production identity** that should be managed, versioned, and reviewed independently.

## Defining an inline soul

[Section titled “Defining an inline soul”](#defining-an-inline-soul)

Add a `souls:` section at the top level of your workflow YAML file. Each key in the dictionary is the soul’s lookup key, and its value contains the full soul definition:

custom/workflows/prototype.yaml

```yaml
version: "1.0"
id: prototype
kind: workflow
souls:
  drafter:
    id: drafter
    kind: soul
    name: Drafter
    role: Quick Drafter
    system_prompt: Draft a short summary of the input topic.
    model_name: gpt-4o
    provider: openai
blocks:
  draft:
    type: linear
    soul_ref: drafter
workflow:
  name: prototype
  entry: draft
  transitions:
    - from: draft
      to: null
```

The block references the inline soul using `soul_ref: drafter` — the same syntax as referencing an external soul file.

## Key must match id

[Section titled “Key must match id”](#key-must-match-id)

The dictionary key and the soul’s `id` field must be identical. The parser validates this and raises a `ValueError` if they differ:

```yaml
# This will fail validation
souls:
  drafter:
    id: draft_soul  # ERROR: key 'drafter' does not match id 'draft_soul'
    kind: soul
    name: Drafter
    role: Drafter
    system_prompt: Draft content.
```

The error message is:

```plaintext
Inline soul key/id mismatch: key 'drafter' must match id 'draft_soul'
```

## Override behavior

[Section titled “Override behavior”](#override-behavior)

When an inline soul has the same key as an external soul file in `custom/souls/`, the inline definition wins. The parser logs a warning:

```plaintext
Inline soul 'researcher' overrides external soul file
```

This lets you temporarily override a library soul for testing without modifying the shared file. The external file is not changed — the override only applies to this workflow’s parse.

* Workflow (inline override)

  custom/workflows/experiment.yaml

  ```yaml
  version: "1.0"
  id: experiment
  kind: workflow
  souls:
    researcher:
      id: researcher
      kind: soul
      name: Researcher
      role: Experimental Researcher
      system_prompt: Use a more creative approach to research.
      model_name: gpt-4o
      provider: openai
      temperature: 1.2
  blocks:
    analyze:
      type: linear
      soul_ref: researcher
  workflow:
    name: experiment
    entry: analyze
    transitions:
      - from: analyze
        to: null
  ```

* External file (overridden)

  custom/souls/researcher.yaml

  ```yaml
  id: researcher
  kind: soul
  name: Researcher
  role: Senior Researcher
  system_prompt: |
    You are a senior research analyst. Produce structured reports
    with findings, sources, and confidence levels.
  provider: openai
  model_name: gpt-4o
  temperature: 0.3
  ```

In this example, the workflow uses the inline definition (temperature 1.2, creative prompt) instead of the external file (temperature 0.3, structured prompt). Other workflows that reference `soul_ref: researcher` continue to use the external file.

## All soul fields are supported

[Section titled “All soul fields are supported”](#all-soul-fields-are-supported)

Inline souls accept every field that external soul files accept. The same Pydantic validation rules apply:

* `id`, `kind`, `name`, `role`, and `system_prompt` are required.
* `tools`, `model_name`, `provider`, `temperature`, `max_tokens`, `max_tool_iterations`, `required_tool_calls`, and `avatar_color` are optional.
* Unknown fields raise a validation error (`extra="forbid"`).

See [Soul Files](/docs/souls/soul-files) for the complete field reference.

## Limitations compared to external souls

[Section titled “Limitations compared to external souls”](#limitations-compared-to-external-souls)

Inline souls have three limitations:

1. **No Soul Library visibility.** Inline souls do not appear in the Soul Library page in the GUI. Only external files in `custom/souls/` are listed.

2. **No cross-workflow reuse.** An inline soul is scoped to the workflow file that contains it. Another workflow cannot reference it. If you need the same soul in two workflows, extract it to `custom/souls/`.

3. **No dependency tracking.** The “Used In” column in the Soul Library counts references from workflow `soul_ref` fields to external soul files. Inline souls bypass this system entirely.

Tip

Start with inline souls during prototyping, then extract to `custom/souls/` when the soul stabilizes. The syntax is identical — move the soul definition into its own file and remove the `souls:` section from the workflow.

## Tool governance still applies

[Section titled “Tool governance still applies”](#tool-governance-still-applies)

Inline souls follow the same tool governance rules as external souls. If an inline soul declares `tools: [http]`, the workflow must also include `http` in its top-level `tools:` section. The parser raises a `ValueError` if a tool is missing from the workflow whitelist.

custom/workflows/fetch.yaml

```yaml
version: "1.0"
id: fetch
kind: workflow
tools:
  - http
souls:
  fetcher:
    id: fetcher
    kind: soul
    name: Fetcher
    role: Fetcher
    system_prompt: Fetch data from the given URL.
    model_name: gpt-4o
    provider: openai
    tools:
      - http
blocks:
  fetch:
    type: linear
    soul_ref: fetcher
workflow:
  name: fetch_pipeline
  entry: fetch
  transitions:
    - from: fetch
      to: null
```

## What’s next

[Section titled “What’s next”](#whats-next)

* [Soul Files](/docs/souls/soul-files) — complete field reference for external soul files
* [Souls Overview](/docs/souls/overview) — concepts and architecture
* [Soul Library](/docs/souls/soul-library) — managing souls through the GUI

# Souls Overview

> What souls are in Runsight — agent identities with role, prompt, provider, model, and tool bindings.

Every AI agent needs an identity: who it is, how it thinks, what model powers it, and which tools it can use. In Runsight, that identity is called a **soul**.

A soul is a standalone YAML file that defines an agent’s persona, behavior constraints, model configuration, and tool access. Souls live in `custom/souls/` and are referenced by workflow blocks using the `soul_ref` field. This separation keeps agent identity independent from workflow logic — the same soul can power steps across many workflows, and changing a soul’s prompt or model updates every workflow that uses it.

## Why souls exist

[Section titled “Why souls exist”](#why-souls-exist)

Workflow engines typically embed agent configuration inline — the prompt, model, and temperature live inside each workflow step. This creates three problems:

1. **Duplication.** The same “Senior Researcher” prompt appears in every workflow that uses it. A prompt revision means editing every copy.
2. **Coupling.** Changing which model an agent uses requires editing every workflow, not just the agent’s configuration.
3. **No management surface.** There is no central place to see all agents, compare prompts, or track which workflows depend on a given agent.

Souls solve these by extracting agent identity into a standalone, reusable artifact. A soul file is the single source of truth for how an agent behaves, regardless of where it is used.

## Anatomy of a soul

[Section titled “Anatomy of a soul”](#anatomy-of-a-soul)

A soul file is a YAML document with a flat structure. Here is a complete example:

custom/souls/researcher.yaml

```yaml
id: researcher
kind: soul
name: Researcher
role: Senior Researcher
system_prompt: |
  You are a senior research analyst. Given a topic, you produce
  a structured report with findings, sources, and confidence levels.
  Always cite your sources and flag low-confidence claims.
provider: openai
model_name: gpt-4o
temperature: 0.3
max_tokens: 4096
tools:
  - http
max_tool_iterations: 5
avatar_color: "#4f46e5"
```

### Required fields

[Section titled “Required fields”](#required-fields)

| Field           | Type     | Description                                                             |
| --------------- | -------- | ----------------------------------------------------------------------- |
| `id`            | `str`    | Embedded soul id. Must match the filename stem for external soul files. |
| `kind`          | `"soul"` | Entity kind. Must be `"soul"`.                                          |
| `name`          | `str`    | Display name for this soul.                                             |
| `role`          | `str`    | The agent’s role, displayed in the Soul Library UI.                     |
| `system_prompt` | `str`    | The system instructions that define behavior and constraints.           |

### Optional fields

[Section titled “Optional fields”](#optional-fields)

| Field                 | Type        | Default | Description                                                                      |
| --------------------- | ----------- | ------- | -------------------------------------------------------------------------------- |
| `model_name`          | `str`       | `None`  | Model to use (e.g., `gpt-4o`, `claude-sonnet-4`). Required for execution.        |
| `provider`            | `str`       | `None`  | Provider for the model (e.g., `openai`, `anthropic`). Required for execution.    |
| `temperature`         | `float`     | `None`  | Sampling temperature override.                                                   |
| `max_tokens`          | `int`       | `None`  | Output token limit override.                                                     |
| `tools`               | `list[str]` | `None`  | Tool IDs this soul can use. Must be declared in the workflow’s `tools:` section. |
| `required_tool_calls` | `list[str]` | `None`  | Tool function names the LLM must call before completing.                         |
| `max_tool_iterations` | `int`       | `5`     | Maximum number of tool-use iterations per execution.                             |
| `avatar_color`        | `str`       | `None`  | UI color hint for displaying the soul (hex or HSL).                              |

Caution

Both `provider` and `model_name` must be set for a soul to execute at runtime. A soul with one but not the other will fail. There are no implicit default models.

## How blocks reference souls

[Section titled “How blocks reference souls”](#how-blocks-reference-souls)

Workflow blocks reference souls through the `soul_ref` field. The value is the soul’s embedded `id`.

custom/workflows/research.yaml

```yaml
version: "1.0"
id: research
kind: workflow
blocks:
  analyze:
    type: linear
    soul_ref: researcher
workflow:
  name: research_pipeline
  entry: analyze
  transitions:
    - from: analyze
      to: null
```

In this example, `soul_ref: researcher` resolves to the soul whose embedded id is `researcher`. For an external soul file, that means `custom/souls/researcher.yaml` must also contain `id: researcher`.

The following block types use `soul_ref`:

| Block Type | How it uses the soul                                                         |
| ---------- | ---------------------------------------------------------------------------- |
| `linear`   | Single LLM call using the soul’s prompt and model.                           |
| `gate`     | Quality-gate LLM call that evaluates output against criteria.                |
| `dispatch` | Per-exit `soul_ref` on each exit definition — each branch gets its own soul. |

## Soul discovery and resolution

[Section titled “Soul discovery and resolution”](#soul-discovery-and-resolution)

When the parser processes a workflow YAML file, it resolves souls in three steps:

1. **Discover external souls.** The parser scans `custom/souls/` for `.yaml` files, loads each into a `Soul` object, and rejects files whose embedded `id` does not match the filename stem. Only `.yaml` files are discovered — `.yml` files are ignored.

2. **Merge inline souls (if present).** If the workflow YAML contains an optional `souls:` section, those inline definitions are merged over the discovered external souls. When an inline soul has the same key as an external file, the inline definition wins and a warning is logged.

3. **Resolve `soul_ref` on each block.** Every block’s `soul_ref` is looked up in the merged souls map. If the reference is not found, the parser raises a `ValueError` listing the available souls and suggesting the file to create.

Discovery runs exactly once per workflow parse, regardless of how many blocks reference souls. Multiple blocks can share the same `soul_ref`.

## One soul per step

[Section titled “One soul per step”](#one-soul-per-step)

Each workflow step is powered by exactly one soul. There are no multi-soul nodes or agent-debate patterns in the current architecture. If a workflow needs different perspectives, use separate steps with different souls connected by transitions:

custom/workflows/review\.yaml

```yaml
version: "1.0"
id: review
kind: workflow
blocks:
  draft:
    type: linear
    soul_ref: writer
  review:
    type: gate
    soul_ref: editor
    eval_key: quality
workflow:
  name: draft_review
  entry: draft
  transitions:
    - from: draft
      to: review
    - from: review
      to: null
```

## Tool governance

[Section titled “Tool governance”](#tool-governance)

Souls can declare which tools they need via the `tools` field. However, every tool a soul references must also appear in the workflow’s top-level `tools:` whitelist. This two-layer design is intentional — it gives workflow authors explicit control over which tools are available in a given workflow, regardless of what individual souls request.

custom/souls/fetcher.yaml

```yaml
id: fetcher
kind: soul
name: Fetcher
role: Data Fetcher
system_prompt: Fetch and summarize data from URLs.
provider: openai
model_name: gpt-4o
tools:
  - http
```

custom/workflows/fetch\_pipeline.yaml

```yaml
version: "1.0"
id: fetch_pipeline
kind: workflow
tools:
  - http
blocks:
  fetch:
    type: linear
    soul_ref: fetcher
workflow:
  name: fetch_pipeline
  entry: fetch
  transitions:
    - from: fetch
      to: null
```

If the soul declares `tools: [http]` but the workflow does not include `http` in its `tools:` section, the parser raises an error naming both the soul and the missing tool.

## Inline souls (DX shorthand)

[Section titled “Inline souls (DX shorthand)”](#inline-souls-dx-shorthand)

For quick prototyping, souls can be defined inline within a workflow YAML file under the `souls:` section. The dictionary key must match the soul’s `id` field:

custom/workflows/prototype.yaml

```yaml
version: "1.0"
id: prototype
kind: workflow
souls:
  drafter:
    id: drafter
    kind: soul
    name: Drafter
    role: Quick Drafter
    system_prompt: Draft a short summary.
    model_name: gpt-4o
    provider: openai
blocks:
  draft:
    type: linear
    soul_ref: drafter
workflow:
  name: prototype
  entry: draft
  transitions:
    - from: draft
      to: null
```

Inline souls are convenience sugar for iteration. The external library model (`custom/souls/` files) is the primary approach — it enables the Soul Library UI, dependency tracking, and reuse across workflows.

Tip

When an inline soul has the same key as an external soul file, the inline definition takes precedence and a warning is logged. This lets you temporarily override a library soul for testing without modifying the shared file.

## File structure

[Section titled “File structure”](#file-structure)

```plaintext
custom/
└── souls/
    ├── researcher.yaml
    ├── editor.yaml
    ├── fetcher.yaml
    └── summarizer.yaml
```

Soul files are plain YAML with no wrapper structure — the fields sit at the top level of the file. There is no `version:` or `soul:` envelope; the file content maps directly to the soul definition.

## What’s next

[Section titled “What’s next”](#whats-next)

* [Soul Files](/docs/souls/soul-files) — detailed reference for every field, constraints, and validation rules
* [Inline Souls](/docs/souls/inline-souls) — when and how to use inline definitions
* [Block Types](/docs/workflows/block-types) — which blocks use `soul_ref` and how

<!-- Linear: RUN-467, RUN-569, RUN-438, RUN-439, RUN-490, RUN-572, RUN-585 — last verified against codebase 2026-04-07 -->

# Soul Files

> Complete field reference for soul YAML files — every field, type, default, and constraint.

Soul files are standalone YAML documents stored in `custom/souls/`. Each file defines one agent identity. There is no envelope or wrapper structure — the soul fields sit at the top level of the file.

## File location and naming

[Section titled “File location and naming”](#file-location-and-naming)

* custom/

  * souls/

    * researcher.yaml
    * editor.yaml
    * fetcher.yaml

The embedded `id` is the soul’s lookup key. When a workflow block sets `soul_ref: researcher`, the parser resolves the soul whose embedded id is `researcher`; external soul files must use the same filename stem, such as `custom/souls/researcher.yaml`.

Only `.yaml` files are discovered. Files with the `.yml` extension are ignored. Files whose names start with `_` are not excluded — all `.yaml` files in the directory are loaded.

## File format

[Section titled “File format”](#file-format)

A soul file is flat YAML with no version field and no wrapping key. The file content maps directly to the soul definition:

custom/souls/researcher.yaml

```yaml
id: researcher
kind: soul
name: Researcher
role: Senior Researcher
system_prompt: |
  You are a senior research analyst. Given a topic, you produce
  a structured report with findings, sources, and confidence levels.
  Always cite your sources and flag low-confidence claims.
provider: openai
model_name: gpt-4o
temperature: 0.3
max_tokens: 4096
tools:
  - http
max_tool_iterations: 5
avatar_color: accent
```

## Field reference

[Section titled “Field reference”](#field-reference)

### Required fields

[Section titled “Required fields”](#required-fields)

These fields must be present in every soul file. The parser raises a `ValidationError` if any are missing.

| Field           | Type     | Description                                                                         |
| --------------- | -------- | ----------------------------------------------------------------------------------- |
| `id`            | `str`    | Embedded soul id. Must match the filename stem for external soul files.             |
| `kind`          | `"soul"` | Entity kind. Must be `"soul"`.                                                      |
| `name`          | `str`    | Display name for this soul.                                                         |
| `role`          | `str`    | The agent’s role (e.g., “Senior Researcher”). Displayed as the soul name in the UI. |
| `system_prompt` | `str`    | System instructions defining the agent’s behavior and constraints.                  |

### Optional fields

[Section titled “Optional fields”](#optional-fields)

| Field                 | Type        | Default | Description                                                                                                                      |
| --------------------- | ----------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `model_name`          | `str`       | `None`  | Model identifier (e.g., `gpt-4o`, `claude-sonnet-4`). Falls back to the runner’s default if not set.                             |
| `provider`            | `str`       | `None`  | Provider for the model (e.g., `openai`, `anthropic`). Falls back to the runner’s default if not set.                             |
| `temperature`         | `float`     | `None`  | Sampling temperature override. When `None`, uses the model’s default.                                                            |
| `max_tokens`          | `int`       | `None`  | Output token limit override. When `None`, uses the model’s default.                                                              |
| `tools`               | `list[str]` | `None`  | Tool IDs this soul can use. Each must also be declared in the workflow’s top-level `tools:` section.                             |
| `required_tool_calls` | `list[str]` | `None`  | LLM-facing tool function names that must be called before the agent completes.                                                   |
| `max_tool_iterations` | `int`       | `5`     | Maximum number of tool-use iterations per execution.                                                                             |
| `avatar_color`        | `str`       | `None`  | UI color for displaying the soul. Accepts one of six preset tokens: `accent`, `info`, `success`, `warning`, `danger`, `neutral`. |

Caution

Both `provider` and `model_name` should be set for a soul to execute at runtime. A soul with one but not the other will fall back to the runner’s defaults for the missing value.

## `id` and filename stem

[Section titled “id and filename stem”](#id-and-filename-stem)

The `id` field is the identity. For external soul files, it must match the filename stem exactly. A file named `researcher.yaml` must contain `id: researcher`; a mismatch is rejected during discovery.

Workflow blocks reference the embedded id with `soul_ref: researcher`.

## Validation rules

[Section titled “Validation rules”](#validation-rules)

Soul files are validated using Pydantic’s `model_validate`. The schema model (`SoulDef` in `schema.py`) enforces:

* **`extra="forbid"`** — unknown fields raise a validation error. Only the fields listed above are accepted.
* **Required fields** — `id`, `kind`, `name`, `role`, and `system_prompt` must be present. A missing field produces a Pydantic `ValidationError`.
* **Type checking** — each field must match its declared type. A string where an integer is expected raises a validation error.

There are no `ge`, `le`, or `min_length` constraints on soul fields. The `max_tool_iterations` field accepts any integer.

## `soul_ref` resolution

[Section titled “soul\_ref resolution”](#soul_ref-resolution)

When the parser encounters a `soul_ref` on a block, it resolves the reference through these steps:

1. **Discover external souls.** The parser scans `custom/souls/` for `.yaml` files. Each file is loaded, validated against the Soul schema, checked for `id == filename stem`, and stored in a dictionary keyed by embedded id.

2. **Merge inline souls.** If the workflow YAML contains a `souls:` section, inline definitions are merged over the external map. When keys overlap, the inline soul wins and a warning is logged: `"Inline soul 'X' overrides external soul file"`.

3. **Look up `soul_ref`.** The block’s `soul_ref` value is looked up in the merged map. If not found, the parser raises a `ValueError` listing available souls.

## Tool governance

[Section titled “Tool governance”](#tool-governance)

A soul’s `tools` list declares which tools the agent needs, but every listed tool must also appear in the workflow’s top-level `tools:` section. If a soul references a tool not declared in the workflow, the parser raises a `ValueError`:

```plaintext
Soul 'fetcher' (custom/souls/fetcher.yaml) references undeclared tool 'http'.
Declared tools: []
```

This two-layer design gives workflow authors explicit control over which tools are available in a given workflow, regardless of what individual souls request.

## `modified_at` field

[Section titled “modified\_at field”](#modified_at-field)

The `modified_at` field is set by the API server when a soul is created or updated through the GUI. It stores a Unix timestamp (float). This field is not part of the engine’s runtime `Soul` model — it is metadata tracked by the API layer for display in the Soul Library’s “Modified” column.

## What’s next

[Section titled “What’s next”](#whats-next)

* [Souls Overview](/docs/souls/overview) — concepts, architecture, and design rationale
* [Inline Souls](/docs/souls/inline-souls) — defining souls directly inside workflow YAML
* [Soul Library](/docs/souls/soul-library) — managing souls through the GUI

# Soul Library

> Managing souls in the GUI — browsing, creating, editing, and deleting agent identities.

The Soul Library is the visual management surface for souls. It lives at `/souls` in the Runsight GUI and provides a searchable, sortable table of all soul files in `custom/souls/`. From the library, you can create new souls, edit existing ones, and delete souls with dependency awareness.

## Browsing the library

[Section titled “Browsing the library”](#browsing-the-library)

The Soul Library page displays all souls in a data table with six columns:

| Column       | What it shows                                                                                                                                                                  |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Name**     | The soul’s `role` field, with an avatar circle showing the first letter and the soul’s `avatar_color`. A warning icon appears if the `system_prompt` is empty.                 |
| **Model**    | The `model_name` value (e.g., `gpt-4o`). Shows a dash if not set.                                                                                                              |
| **Provider** | The `provider` value. Shows a warning if the provider is disabled in Settings.                                                                                                 |
| **Tools**    | Badge chips for each tool the soul declares (excluding `delegate`, which is hidden). Each badge shows the tool name and metadata labels for origin and executor when relevant. |
| **Used In**  | The number of workflows that reference this soul via `soul_ref`.                                                                                                               |
| **Modified** | Relative time since the soul was last saved (e.g., “2h ago”, “3d ago”).                                                                                                        |

The table supports **client-side search** (filters by name) and **column sorting** (click any sortable column header). Clicking a row navigates to the edit form.

## Creating a soul

[Section titled “Creating a soul”](#creating-a-soul)

1. Click the **New Soul** button in the top-right corner of the library page. This navigates to `/souls/new`.

2. Fill in the form. The form is organized into five sections:

   **Identity** — The soul’s display name (maps to the `role` field in YAML) and avatar color. The name field is required.

   **Model** — Provider and model dropdowns. The provider dropdown shows all active providers from Settings. Selecting a provider populates the model dropdown with that provider’s available models. If no providers are configured, the dropdown is disabled with a message to add one in Settings.

   **Prompt** — A large text area for the `system_prompt`. This is the core behavioral definition of the soul. The field is required — an empty prompt shows a warning in the library table.

   **Tools** — A collapsible section (collapsed by default) showing all available tools as toggle cards. Each card displays the tool name, description, origin (Built-in or Custom), and executor type (Native, Python, or Request). Click a card to enable or disable the tool on this soul.

   **Advanced** — A collapsible section (collapsed by default) containing:

   * **Temperature** — a slider from 0.0 to 2.0, defaulting to 0.7
   * **Max Tokens** — a number input, empty by default (uses model default)
   * **Max Tool Iterations** — a number input, defaulting to 5

3. Click **Create Soul**. The API writes a YAML file to `custom/souls/` and auto-commits it to Git. You are returned to the library page.

Note

The form requires both a name and a system prompt to enable the submit button. Provider and model are not required to save — you can create a soul definition and configure its model later.

## Editing a soul

[Section titled “Editing a soul”](#editing-a-soul)

Click any row in the library table to open the edit form at `/souls/:id/edit`. The form is pre-filled with the soul’s current values.

The same five sections (Identity, Model, Prompt, Tools, Advanced) are available. Modify any field, then click **Save Changes**. The API updates the YAML file and auto-commits to Git.

The **Save Changes** button is disabled until you modify at least one field. If you navigate away with unsaved changes, a confirmation dialog asks whether to discard or keep editing.

### Canvas return flow

[Section titled “Canvas return flow”](#canvas-return-flow)

When editing a soul from the workflow canvas (via the soul picker’s “Create new soul” link), the URL includes a `?return=` parameter. In this mode, the save button reads **Save & Return to Canvas** and navigates back to the canvas after saving.

## Deleting a soul

[Section titled “Deleting a soul”](#deleting-a-soul)

Deleting a soul involves a dependency check. When you open the delete dialog, the UI fetches `GET /api/souls/:id/usages` to determine which workflows reference this soul.

**No dependencies:** The dialog shows a simple confirmation: “Are you sure you want to delete \[soul name]? This action cannot be undone.”

**Has dependencies:** The dialog shows a warning: “This soul is currently used in active workflows.” Below the warning, it lists the total count and shows workflow name badges for up to five affected workflows. If more than five workflows are affected, a “+N more” badge appears.

In both cases, clicking **Delete** (or **Delete anyway** when dependencies exist) removes the soul’s YAML file and auto-commits the deletion to Git. The operation uses `force: true` — it does not block on dependencies. The dependency warning is informational, giving you the chance to update affected workflows first.

Caution

Deleting a soul that is referenced by workflows will cause those workflows to fail at parse time. The parser raises a `ValueError` when it cannot resolve a `soul_ref` to a soul file.

## Avatar color picker

[Section titled “Avatar color picker”](#avatar-color-picker)

The Identity section includes a color picker with six preset options. These map to design system tokens:

| Value     | Description                                 |
| --------- | ------------------------------------------- |
| `accent`  | The default. Uses the primary accent color. |
| `info`    | Blue informational tone.                    |
| `success` | Green success tone.                         |
| `warning` | Yellow/amber warning tone.                  |
| `danger`  | Red danger tone.                            |
| `neutral` | Gray neutral tone.                          |

The selected color is stored in the `avatar_color` field of the soul’s YAML file and displayed as the avatar circle in the library table.

## How the library reads souls

[Section titled “How the library reads souls”](#how-the-library-reads-souls)

The Soul Library page calls `GET /api/souls`, which scans all `.yaml` files in `custom/souls/`. For each soul, the API also scans all workflow files to count `soul_ref` references, populating the `workflow_count` field. The `modified_at` timestamp is set by the API when a soul is created or updated through the GUI.

This scan-based approach means the library always reflects the current state of the filesystem. If you add or modify a soul file directly (outside the GUI), the changes appear on the next page load.

## What’s next

[Section titled “What’s next”](#whats-next)

* [Soul Files](/docs/souls/soul-files) — complete field reference for soul YAML files
* [Inline Souls](/docs/souls/inline-souls) — defining souls directly inside workflow YAML
* [Souls Overview](/docs/souls/overview) — concepts and architecture

<!-- Linear: RUN-467, RUN-437, RUN-443, RUN-575 — Soul Management project (bb749057) — last verified against codebase 2026-04-07 -->

# Built-in Tools

> Reference for the three tools that ship with Runsight — delegate, http, and file_io.

Runsight ships with three built-in tools. They are always available and do not require any files in `custom/tools/`. Use their canonical ID directly in your workflow and soul `tools` lists.

## `delegate`

[Section titled “delegate”](#delegate)

Routes execution to an exit port. When a soul calls `delegate`, it picks which branch of the workflow to follow next. This is the mechanism behind LLM-driven branching.

See [Dispatch & Delegate](/docs/tools/dispatch-and-delegate) for the full explanation of how delegate and dispatch work together.

### Parameters

[Section titled “Parameters”](#parameters)

| Parameter | Type  | Required | Description                                                                                  |
| --------- | ----- | -------- | -------------------------------------------------------------------------------------------- |
| `port`    | `str` | Yes      | The exit port ID to delegate to. Must match one of the block’s declared `exits[].id` values. |
| `task`    | `str` | Yes      | The task instruction to delegate to this port.                                               |

When the block has exits defined, the `port` parameter is constrained to an enum of valid exit IDs. The LLM sees the valid options and picks one.

### How it works

[Section titled “How it works”](#how-it-works)

1. A block declares `exits` — a list of named ports
2. The soul assigned to that block has `delegate` in its `tools` list
3. At runtime, the LLM calls the `delegate` tool with `{"port": "some_exit", "task": "..."}`
4. The runner captures the `port` value as the block’s `exit_handle`
5. The workflow engine uses `exit_handle` to look up the next block via `conditional_transitions`

### Return value

[Section titled “Return value”](#return-value)

The delegate tool returns a JSON string `{"port": "<port>", "task": "<task>"}` on success. If the port is not in the valid set, it returns an error message listing valid ports.

### Usage example

[Section titled “Usage example”](#usage-example)

custom/workflows/triage.yaml

```yaml
version: "1.0"
id: triage
kind: workflow
tools:
  - delegate
souls:
  router:
    id: router
    kind: soul
    name: Router
    role: Triage Router
    system_prompt: >
      Read the incoming request and route it to the correct team
      using the delegate tool.
    tools:
      - delegate
blocks:
  triage:
    type: linear
    soul_ref: router
    exits:
      - id: billing
        label: Billing Team
      - id: technical
        label: Technical Support
  handle_billing:
    type: linear
    soul_ref: billing_agent
  handle_technical:
    type: linear
    soul_ref: tech_agent
workflow:
  name: Support Triage
  entry: triage
  conditional_transitions:
    - from: triage
      billing: handle_billing
      technical: handle_technical
```

## `http`

[Section titled “http”](#http)

Makes outbound HTTP requests to external URLs. Use this when a soul needs to fetch data from or send data to an external API at runtime, without defining a custom tool.

### Parameters

[Section titled “Parameters”](#parameters-1)

| Parameter       | Type     | Required | Description                                         |
| --------------- | -------- | -------- | --------------------------------------------------- |
| `method`        | `str`    | Yes      | HTTP method (`GET`, `POST`, `PUT`, `DELETE`, etc.). |
| `url`           | `str`    | Yes      | The target URL.                                     |
| `headers`       | `object` | No       | Key-value mapping of HTTP headers.                  |
| `body`          | `str`    | No       | Request body content.                               |
| `response_path` | `str`    | No       | Dot-notation path to extract from a JSON response.  |

### Response handling

[Section titled “Response handling”](#response-handling)

The response is processed based on the `Content-Type` header:

| Content-Type       | Handling                                                                        |
| ------------------ | ------------------------------------------------------------------------------- |
| `application/json` | Parsed as JSON. If `response_path` is set, the value at that path is extracted. |
| `text/html`        | Converted to readable plain text (scripts and styles stripped).                 |
| `text/plain`       | Returned as-is.                                                                 |

Responses larger than 64 KB are truncated.

Note

The `http` tool validates URLs against SSRF (Server-Side Request Forgery) before making requests.

### Usage example

[Section titled “Usage example”](#usage-example-1)

custom/workflows/lookup.yaml

```yaml
version: "1.0"
id: lookup
kind: workflow
tools:
  - http
souls:
  fetcher:
    id: fetcher
    kind: soul
    name: Fetcher
    role: Data Fetcher
    system_prompt: >
      Fetch the requested data using the http tool and summarize the results.
    tools:
      - http
blocks:
  fetch_data:
    type: linear
    soul_ref: fetcher
workflow:
  name: API Lookup
  entry: fetch_data
```

The LLM decides at runtime what URL to call, what method to use, and what headers to send. The tool result is returned to the LLM as part of the agentic tool loop.

## `file_io`

[Section titled “file\_io”](#file_io)

Reads and writes files within a sandboxed base directory. Path traversal and absolute paths are rejected.

### Parameters

[Section titled “Parameters”](#parameters-2)

| Parameter | Type                          | Required | Description                                           |
| --------- | ----------------------------- | -------- | ----------------------------------------------------- |
| `action`  | `str` (enum: `read`, `write`) | Yes      | Whether to read or write.                             |
| `path`    | `str`                         | Yes      | Relative file path within the sandbox directory.      |
| `content` | `str`                         | No       | Content to write. Only used when `action` is `write`. |

### Security constraints

[Section titled “Security constraints”](#security-constraints)

* **No absolute paths** — paths like `/etc/passwd` are rejected with a `PermissionError`
* **No path traversal** — paths containing `..` are rejected
* **Sandbox boundary** — the resolved path must stay within the configured base directory
* On write, parent directories are created automatically if they do not exist

### Return value

[Section titled “Return value”](#return-value-1)

* **Read**: returns the file content as a string
* **Write**: returns a confirmation message with the byte count (e.g., `"Written 42 bytes to output/report.txt"`)

### Usage example

[Section titled “Usage example”](#usage-example-2)

custom/workflows/report.yaml

```yaml
version: "1.0"
id: report
kind: workflow
tools:
  - file_io
souls:
  writer:
    id: writer
    kind: soul
    name: Writer
    role: Report Writer
    system_prompt: >
      Write the analysis report to a file using the file_io tool.
    tools:
      - file_io
blocks:
  write_report:
    type: linear
    soul_ref: writer
workflow:
  name: Report Generation
  entry: write_report
```

<!-- Linear: RUN-273 (Tool Registry epic) — last verified against codebase 2026-04-07 -->

# Custom Tools

> Complete YAML reference for custom tools — Python executor, HTTP request executor, parameters schema, discovery rules, and reserved IDs.

Custom tools are YAML files in `custom/tools/` that define capabilities a soul can invoke during execution. Each file describes one tool with its executor type, parameters, and implementation.

## File format

[Section titled “File format”](#file-format)

Every custom tool YAML file has these required fields:

| Field         | Type     | Description                                                      |
| ------------- | -------- | ---------------------------------------------------------------- |
| `version`     | `str`    | Schema version. Must be `"1.0"`.                                 |
| `id`          | `str`    | Embedded tool id. Must match the filename stem.                  |
| `kind`        | `str`    | Must be `"tool"`.                                                |
| `type`        | `str`    | Must be `"custom"`.                                              |
| `executor`    | `str`    | `"python"` or `"request"`.                                       |
| `name`        | `str`    | Human-readable tool name.                                        |
| `description` | `str`    | What the tool does. Sent to the LLM as the function description. |
| `parameters`  | `object` | JSON Schema object describing the tool’s input parameters.       |

Additionally, depending on the executor:

| Executor  | Required                            | Optional          |
| --------- | ----------------------------------- | ----------------- |
| `python`  | `code` or `code_file` (exactly one) | —                 |
| `request` | `request` mapping                   | `timeout_seconds` |

The canonical tool ID is the embedded `id`. A file at `custom/tools/sentiment.yaml` must contain `id: sentiment`.

Caution

The `description` field is required. Discovery will reject any tool file that omits it or provides an empty string.

## Python executor

[Section titled “Python executor”](#python-executor)

The Python executor runs your code in a sandboxed subprocess. The tool receives arguments as JSON on stdin and returns the result on stdout.

Your code must define a `def main(args)` function. The `args` parameter is a dictionary matching the `parameters` schema.

### Inline code

[Section titled “Inline code”](#inline-code)

custom/tools/sentiment.yaml

```yaml
version: "1.0"
id: sentiment
kind: tool
type: custom
executor: python
name: sentiment_analyzer
description: Analyze the sentiment of the given text and return a score.
parameters:
  type: object
  properties:
    text:
      type: string
      description: The text to analyze.
  required:
    - text
code: |
  def main(args):
      text = args["text"]
      positive_words = ["good", "great", "excellent", "love"]
      score = sum(1 for w in text.lower().split() if w in positive_words)
      return {"sentiment": "positive" if score > 0 else "neutral", "score": score}
```

### External code file

[Section titled “External code file”](#external-code-file)

For longer implementations, use `code_file` to reference a Python file in the same directory as the YAML:

custom/tools/data\_processor.yaml

```yaml
version: "1.0"
id: data_processor
kind: tool
type: custom
executor: python
name: data_processor
description: Process and transform structured data records.
parameters:
  type: object
  properties:
    records:
      type: array
      items:
        type: object
  required:
    - records
code_file: data_processor.py
```

custom/tools/data\_processor.py

```python
def main(args):
    records = args["records"]
    processed = [{"id": r.get("id"), "status": "done"} for r in records]
    return {"processed": processed, "count": len(processed)}
```

Note

You cannot declare both `code` and `code_file` on the same tool. The parser rejects this as an error.

### Python executor constraints

[Section titled “Python executor constraints”](#python-executor-constraints)

* The `code` (or contents of `code_file`) must define exactly `def main(args)` with a single parameter named `args`
* The function’s return value is serialized as JSON and sent back to the LLM
* The subprocess runs with a minimal environment (`PATH`, `HOME` only)
* Python executor tools cannot declare `request`, `timeout_seconds`, or any HTTP-related fields

## HTTP request executor

[Section titled “HTTP request executor”](#http-request-executor)

The request executor makes an outbound HTTP call. Use this for integrating with external APIs without writing Python code.

custom/tools/slack\_notify.yaml

```yaml
version: "1.0"
id: slack_notify
kind: tool
type: custom
executor: request
name: slack_notify
description: Send a notification message to a Slack channel via webhook.
parameters:
  type: object
  properties:
    message:
      type: string
      description: The message to send.
  required:
    - message
timeout_seconds: 10
request:
  method: POST
  url: "${SLACK_WEBHOOK_URL}"
  headers:
    Content-Type: application/json
  body_template: '{"text": "{{ message }}"}'
```

### Request fields

[Section titled “Request fields”](#request-fields)

| Field           | Type     | Required | Description                                                                                   |
| --------------- | -------- | -------- | --------------------------------------------------------------------------------------------- |
| `method`        | `str`    | Yes      | HTTP method (`GET`, `POST`, `PUT`, `DELETE`, etc.). Defaults to `GET` if omitted.             |
| `url`           | `str`    | Yes      | The target URL. Supports `${ENV_VAR}` for environment variable substitution.                  |
| `headers`       | `object` | No       | Key-value mapping of HTTP headers. Values support `${ENV_VAR}` substitution.                  |
| `body_template` | `str`    | No       | Request body. Supports `{{ param }}` for parameter substitution and `${ENV_VAR}` for secrets. |
| `response_path` | `str`    | No       | Dot-notation path to extract from a JSON response (e.g., `data.result`).                      |

### Template syntax

[Section titled “Template syntax”](#template-syntax)

Two template syntaxes are available in `url`, `headers`, `body_template`, and `response_path`:

* **`{{ param }}`** — substitutes a value from the tool’s `args` dictionary
* **`${ENV_VAR}`** — substitutes an environment variable value (raises an error if the variable is not set)

### Request executor constraints

[Section titled “Request executor constraints”](#request-executor-constraints)

* Request executor tools cannot declare `code` or `code_file`
* The `request` mapping is required and must contain at least `url`
* Only these fields are allowed in the `request` mapping: `method`, `url`, `headers`, `body_template`, `response_path`
* `timeout_seconds` is optional and must be a positive integer

### Response handling

[Section titled “Response handling”](#response-handling)

The response is processed based on the `Content-Type` header:

| Content-Type       | Handling                                                                        |
| ------------------ | ------------------------------------------------------------------------------- |
| `application/json` | Parsed as JSON. If `response_path` is set, the value at that path is extracted. |
| `text/html`        | Converted to readable text (scripts, styles, and hidden tags are stripped).     |
| `text/plain`       | Returned as-is.                                                                 |
| Other              | Returned as text.                                                               |

Responses larger than 64 KB are truncated with a `[truncated]` suffix.

## Parameters schema

[Section titled “Parameters schema”](#parameters-schema)

The `parameters` field uses JSON Schema format. This schema is sent to the LLM as the function parameter definition and is used to validate arguments at call time.

```yaml
parameters:
  type: object
  properties:
    query:
      type: string
      description: The search query.
    limit:
      type: integer
      description: Maximum number of results.
  required:
    - query
```

The `description` on each property helps the LLM understand what to pass. Always include descriptions for non-obvious parameters.

## Discovery rules

[Section titled “Discovery rules”](#discovery-rules)

1. Only `.yaml` files in `custom/tools/` are scanned (not subdirectories)
2. The canonical ID is the embedded `id`
3. The embedded `id` must match the filename stem
4. Duplicate embedded IDs are rejected
5. All required fields must be present and non-empty strings
6. Unknown fields are rejected (the allowed set is: `id`, `kind`, `version`, `type`, `executor`, `name`, `description`, `parameters`, `code`, `code_file`, `request`, `timeout_seconds`)
7. The `kind` field must be `"tool"` and the `type` field must be `"custom"`

## Reserved IDs

[Section titled “Reserved IDs”](#reserved-ids)

The following IDs are reserved for builtin tools and cannot be used as custom tool ids:

* `http`
* `file_io`
* `delegate`

Creating a file named `custom/tools/http.yaml` or declaring `id: http` will cause a parse error.

## Using a custom tool in a workflow

[Section titled “Using a custom tool in a workflow”](#using-a-custom-tool-in-a-workflow)

To make a custom tool available, declare it in both the workflow’s `tools` list and the soul’s `tools` list:

custom/workflows/analysis.yaml

```yaml
version: "1.0"
id: analysis
kind: workflow
tools:
  - sentiment
souls:
  analyst:
    id: analyst
    kind: soul
    name: Analyst
    role: Sentiment Analyst
    system_prompt: Analyze the sentiment of the provided text using the sentiment tool.
    tools:
      - sentiment
blocks:
  analyze:
    type: linear
    soul_ref: analyst
workflow:
  name: Sentiment Analysis
  entry: analyze
```

Both layers are required. See [Tools Overview](/docs/tools/overview) for details on the governance model.

<!-- Linear: RUN-273 (Tool Registry epic) — last verified against codebase 2026-04-07 -->

# Dispatch & Delegate

> How branching works in Runsight — the delegate tool for LLM-driven routing and the dispatch block for parallel multi-agent execution.

Runsight has two mechanisms for branching a workflow into multiple paths. Both use exit ports, but they work differently: one lets the LLM choose a path, the other runs all paths in parallel.

## Two branching mechanisms

[Section titled “Two branching mechanisms”](#two-branching-mechanisms)

| Mechanism                        | Block type                                  | Who decides                                        | Execution                                |
| -------------------------------- | ------------------------------------------- | -------------------------------------------------- | ---------------------------------------- |
| **Delegate** (exit port routing) | Any block with `exits` (typically `linear`) | The LLM calls the `delegate` tool to pick one port | Sequential — one branch runs             |
| **Dispatch**                     | `dispatch` block                            | The workflow definition                            | Parallel — all branches run concurrently |

## Delegate: LLM-driven routing

[Section titled “Delegate: LLM-driven routing”](#delegate-llm-driven-routing)

The delegate pattern puts the branching decision in the hands of the LLM. A block declares named exit ports, the soul uses the `delegate` tool to pick one, and the workflow engine routes to the corresponding downstream block.

### How it works

[Section titled “How it works”](#how-it-works)

1. Define a block with `exits` — each exit has an `id` and `label`
2. Assign a soul that has `delegate` in its `tools` list
3. At runtime, the LLM reads the available exit ports (they appear as an enum in the tool schema) and calls `delegate` with `{"port": "<exit_id>", "task": "..."}`
4. The runner captures the chosen port as the block’s `exit_handle`
5. The workflow engine matches `exit_handle` against `conditional_transitions` to find the next block

### Complete example

[Section titled “Complete example”](#complete-example)

custom/workflows/code-review\.yaml

```yaml
version: "1.0"
id: code-review
kind: workflow
tools:
  - delegate
souls:
  reviewer:
    id: reviewer
    kind: soul
    name: Reviewer
    role: Code Reviewer
    system_prompt: >
      Review the code change. If it passes quality checks, delegate to
      the "approve" port. If it needs changes, delegate to "request_changes".
      If it has critical issues, delegate to "reject".
    tools:
      - delegate
  merger:
    id: merger
    kind: soul
    name: Merger
    role: Merge Handler
    system_prompt: Merge the approved change.
  feedback_writer:
    id: feedback_writer
    kind: soul
    name: Feedback Writer
    role: Feedback Writer
    system_prompt: Write specific feedback for the requested changes.
  escalation_handler:
    id: escalation_handler
    kind: soul
    name: Escalation Handler
    role: Escalation Handler
    system_prompt: Escalate the critical issue to the team lead.
blocks:
  review:
    type: linear
    soul_ref: reviewer
    exits:
      - id: approve
        label: Approved
      - id: request_changes
        label: Changes Requested
      - id: reject
        label: Rejected
  merge:
    type: linear
    soul_ref: merger
  write_feedback:
    type: linear
    soul_ref: feedback_writer
  escalate:
    type: linear
    soul_ref: escalation_handler
workflow:
  name: Code Review
  entry: review
  conditional_transitions:
    - from: review
      approve: merge
      request_changes: write_feedback
      reject: escalate
```

In this workflow, the `reviewer` soul analyzes the code and makes a judgment call. The LLM sees three exit ports (`approve`, `request_changes`, `reject`) and calls the delegate tool to pick one. The workflow engine then routes to the corresponding block.

### Exit port definition

[Section titled “Exit port definition”](#exit-port-definition)

Exit ports are defined on the block, not the soul:

```yaml
exits:
  - id: approve       # The canonical exit ID (used in delegate calls and transitions)
    label: Approved    # Human-readable label (shown in UI, included in tool description)
  - id: reject
    label: Rejected
```

The `id` is what the LLM uses when calling `delegate(port="approve", task="...")`. The `label` is metadata for display.

### Wiring exits to transitions

[Section titled “Wiring exits to transitions”](#wiring-exits-to-transitions)

Every exit port must have a corresponding entry in `conditional_transitions`:

```yaml
conditional_transitions:
  - from: review          # The block with exits
    approve: merge        # exit_id -> downstream block_id
    reject: escalate
```

If the LLM picks a port that has no matching transition, the workflow engine raises an error with the available options.

## Dispatch: parallel multi-agent execution

[Section titled “Dispatch: parallel multi-agent execution”](#dispatch-parallel-multi-agent-execution)

The dispatch block runs multiple branches in parallel, each with its own soul and task instruction. Unlike delegate, there is no LLM decision — all branches execute concurrently.

### How it works

[Section titled “How it works”](#how-it-works-1)

1. Define a block with `type: dispatch`
2. Each exit has `soul_ref` and `task` in addition to `id` and `label`
3. At runtime, all branches execute in parallel via `asyncio.gather`
4. Results are stored per-exit and combined

### Complete example

[Section titled “Complete example”](#complete-example-1)

custom/workflows/research-analysis.yaml

```yaml
version: "1.0"
id: research-analysis
kind: workflow
souls:
  web_analyst:
    id: web_analyst
    kind: soul
    name: Web Analyst
    role: Web Research Analyst
    system_prompt: Research the topic from web sources and provide key findings.
  risk_reviewer:
    id: risk_reviewer
    kind: soul
    name: Risk Reviewer
    role: Risk Analyst
    system_prompt: Identify risks, caveats, and concerns about the topic.
  synthesizer:
    id: synthesizer
    kind: soul
    name: Synthesizer
    role: Report Synthesizer
    system_prompt: Combine the research findings into a coherent summary.
blocks:
  parallel_research:
    type: dispatch
    exits:
      - id: web_scan
        label: Web Research
        soul_ref: web_analyst
        task: >
          Research the topic from available sources. Provide compact
          bullet points that can be merged downstream.
      - id: risk_scan
        label: Risk Analysis
        soul_ref: risk_reviewer
        task: >
          Identify caveats, contradictions, and cost concerns
          the final report must address.
  synthesize:
    type: linear
    soul_ref: synthesizer
    depends: parallel_research
workflow:
  name: Research Analysis
  entry: parallel_research
  transitions:
    - from: parallel_research
      to: synthesize
```

### Dispatch exit definition

[Section titled “Dispatch exit definition”](#dispatch-exit-definition)

Dispatch exits have two additional required fields compared to regular exits:

| Field      | Type  | Required | Description                           |
| ---------- | ----- | -------- | ------------------------------------- |
| `id`       | `str` | Yes      | The exit port ID.                     |
| `label`    | `str` | Yes      | Human-readable label.                 |
| `soul_ref` | `str` | Yes      | Which soul runs this branch.          |
| `task`     | `str` | Yes      | The task instruction for this branch. |

### How results are stored

[Section titled “How results are stored”](#how-results-are-stored)

Dispatch blocks store results in two ways:

**Per-exit results** — each branch’s output is stored at `state.results["{block_id}.{exit_id}"]`:

```plaintext
state.results["parallel_research.web_scan"]   -> BlockResult(output="...", exit_handle="web_scan")
state.results["parallel_research.risk_scan"]  -> BlockResult(output="...", exit_handle="risk_scan")
```

**Combined result** — all branch outputs are combined as a JSON array at `state.results["{block_id}"]`:

```json
[
  {"exit_id": "web_scan", "output": "..."},
  {"exit_id": "risk_scan", "output": "..."}
]
```

Downstream blocks that use `depends` or `input_block_ids` referencing the dispatch block receive the combined result.

## When to use which

[Section titled “When to use which”](#when-to-use-which)

| Scenario                                                   | Use          | Why                                       |
| ---------------------------------------------------------- | ------------ | ----------------------------------------- |
| Route to one of several handlers based on content          | **Delegate** | The LLM decides which path is appropriate |
| Run multiple analyses in parallel and merge results        | **Dispatch** | All paths execute; no decision needed     |
| Triage incoming requests to different teams                | **Delegate** | Classification is an LLM judgment         |
| Gather perspectives from multiple agents on the same input | **Dispatch** | Every perspective is needed               |
| Implement approval gates with pass/fail/escalate           | **Delegate** | The gate outcome determines the path      |

### Key differences

[Section titled “Key differences”](#key-differences)

* **Delegate** runs one branch. The LLM picks which one. The soul needs the `delegate` tool. The block uses `conditional_transitions` for routing.
* **Dispatch** runs all branches in parallel. No tool call is involved. Each branch has its own soul and task instruction defined in the exit. Results are combined automatically.

Note

Both mechanisms use exit ports (`exits` on the block definition), but dispatch exits require `soul_ref` and `task` fields that regular exits do not have.

<!-- Linear: RUN-265 (Named Ports & Exit Strategy), RUN-283 (Fan-Out v2) — last verified against codebase 2026-04-07 -->

# Tools Overview

> What tools are in Runsight, the three types (builtin, custom, HTTP), canonical IDs, discovery, and the two-layer governance model.

Tools give souls the ability to take action beyond generating text. When a soul has tools, the LLM enters an agentic tool loop: it can call tools, read their results, and decide what to do next — all within a single block execution.

## Three tool types

[Section titled “Three tool types”](#three-tool-types)

Runsight supports three kinds of tools:

| Type                | Where it lives        | How it runs           |
| ------------------- | --------------------- | --------------------- |
| **Builtin**         | Ships with Runsight   | Native async Python   |
| **Custom (Python)** | `custom/tools/*.yaml` | Sandboxed subprocess  |
| **Custom (HTTP)**   | `custom/tools/*.yaml` | Outbound HTTP request |

**Builtin tools** are part of the Runsight engine. Three ship today: `delegate`, `http`, and `file_io`. See [Built-in Tools](/docs/tools/built-in-tools) for details on each.

**Custom tools** are YAML files you create in `custom/tools/`. Each file defines a tool with either a `python` executor (inline code or a `code_file` reference) or a `request` executor (outbound HTTP). See [Custom Tools](/docs/tools/custom-tools) for the full YAML format.

## Canonical IDs

[Section titled “Canonical IDs”](#canonical-ids)

Every tool has a **canonical ID** — a plain string used everywhere: workflow YAML, soul definitions, resolution, and runtime. The canonical ID is:

* For **builtin tools**: the reserved string itself — `delegate`, `http`, or `file_io`
* For **custom tools**: the embedded `id`; a file at `custom/tools/sentiment.yaml` must contain `id: sentiment`

IDs are plain strings with no prefix. You write `delegate` in your YAML, not `builtin/delegate`.

custom/workflows/example.yaml

```yaml
id: example
kind: workflow
tools:
  - delegate
  - http
  - sentiment
```

Caution

Custom tool filenames must not collide with reserved builtin IDs. If you create `custom/tools/http.yaml`, the parser will reject it with an error.

The three reserved builtin tool IDs are: `http`, `file_io`, and `delegate`.

## Discovery

[Section titled “Discovery”](#discovery)

Custom tools are discovered automatically from `custom/tools/*.yaml` at parse time. The discovery engine:

1. Scans `custom/tools/` for `.yaml` files
2. Reads the canonical ID from each file’s embedded `id`
3. Validates that no custom ID collides with reserved builtin IDs
4. Validates that each embedded `id` matches its filename stem
5. Validates all required fields (`id`, `kind`, `version`, `type`, `executor`, `name`, `description`, `parameters`)
6. For Python executors, validates that the code defines `def main(args)`
7. For request executors, validates the `request` mapping (method, url, etc.)

No registration step is needed. Drop a valid YAML file into `custom/tools/` and it is available to any workflow that declares it.

## Two-layer governance

[Section titled “Two-layer governance”](#two-layer-governance)

Tools in Runsight use a two-layer governance model. Both layers must agree before a soul can use a tool at runtime.

### Layer 1: Soul declares tools

[Section titled “Layer 1: Soul declares tools”](#layer-1-soul-declares-tools)

Each soul lists the tools it wants to use in its `tools` field:

custom/souls/router.yaml

```yaml
id: router
kind: soul
name: Router
role: Router Agent
system_prompt: Route the task to the correct team.
tools:
  - delegate
```

This is the soul’s declaration of intent. It says “I need access to the `delegate` tool.”

### Layer 2: Workflow whitelists tools

[Section titled “Layer 2: Workflow whitelists tools”](#layer-2-workflow-whitelists-tools)

The workflow file has a top-level `tools` list that acts as a whitelist:

custom/workflows/triage.yaml

```yaml
version: "1.0"
id: triage
kind: workflow
tools:
  - delegate
  - http
```

Only tools listed here can be used by any soul in this workflow.

### Governance validation

[Section titled “Governance validation”](#governance-validation)

At parse time, Runsight checks every soul referenced by the workflow. For each tool in a soul’s `tools` list, it verifies that the tool appears in the workflow’s `tools` whitelist. If a soul references a tool that the workflow does not declare, parsing fails with an error:

```plaintext
Soul 'router' (custom/souls/router.yaml) references undeclared tool 'http'.
Declared tools: ['delegate']
```

This two-layer model ensures that workflow authors control which tools are available, while soul authors declare which tools they need.

## How tools bind to souls at parse time

[Section titled “How tools bind to souls at parse time”](#how-tools-bind-to-souls-at-parse-time)

During workflow parsing (step 6.6 in the parser), tools are resolved and attached to souls:

1. **Governance validation** runs first — every soul’s tool references are checked against the workflow whitelist
2. **Definition validation** confirms that every declared tool ID is resolvable (either a known builtin or a discovered custom tool with valid metadata)
3. **Tool resolution** creates `ToolInstance` objects for each tool. For the `delegate` tool, the parser finds the block that references the soul and passes the block’s `exits` list so the delegate tool knows which exit ports are valid
4. **Binding** attaches the resolved `ToolInstance` objects to `soul.resolved_tools`, making them available to the agentic tool loop at runtime

The tool loop in the runner then uses `soul.resolved_tools` to build the OpenAI function-calling schema and execute tool calls as the LLM requests them.

<!-- Linear: RUN-273 (Tool Registry epic) — last verified against codebase 2026-04-07 -->

# Canvas Modes

> The three WorkflowSurface modes — edit, readonly, and sim — and how they control every panel in the UI.

The `WorkflowSurface` component operates in one of three modes: **edit**, **readonly**, and **sim** (simulation). Each mode configures every panel in the surface — topbar, canvas, inspector, bottom panel, and status bar — through a pure-data contract defined in `workflowSurfaceContract.ts`. This means mode behavior is declarative: changing a mode flips a set of flags, and every component reads its rules from the contract.

## The three modes

[Section titled “The three modes”](#the-three-modes)

### Edit mode

[Section titled “Edit mode”](#edit-mode)

Edit mode is the primary authoring surface. This is the mode you see when you open a workflow from the Flows page.

| Panel            | Behavior                                                                                                                                            |
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Topbar**       | Workflow name is editable (click to rename). Save button appears, enabled when there are unsaved changes. Run button visible. No execution metrics. |
| **Canvas**       | Nodes are draggable. Connections can be created. Nodes can be deleted (Backspace key). Cost badges show estimated values.                           |
| **Inspector**    | Opens on double-click. Fields are editable. Tabs: Overview, Prompt, Conditions.                                                                     |
| **Bottom panel** | Collapsed by default. Tabs: Logs, Runs.                                                                                                             |
| **Status bar**   | Shows block and edge counts (e.g., “3 blocks, 2 edges”). No execution metrics.                                                                      |
| **Tab toggle**   | Both Canvas and YAML tabs are available.                                                                                                            |

### Readonly mode

[Section titled “Readonly mode”](#readonly-mode)

Readonly mode is used when viewing a completed run. Nothing is editable — the surface is a read-only inspection view.

| Panel            | Behavior                                                                                                                                                       |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Topbar**       | Workflow name is not editable (links back to the workflow editor). Save button hidden. Fork button visible. Static execution metrics (duration, tokens, cost). |
| **Canvas**       | Nodes are not draggable. No connections or deletions. Cost badges show final values.                                                                           |
| **Inspector**    | Opens on single-click. Fields are read-only. Tabs: Overview, Output, Eval, Error.                                                                              |
| **Bottom panel** | Expanded by default. Tabs: Logs, Runs, Regressions.                                                                                                            |
| **Status bar**   | Shows progress format (e.g., “5/5 steps”). Duration and cost metrics visible.                                                                                  |
| **Tab toggle**   | Neither Canvas nor YAML tabs are togglable — the view is fixed.                                                                                                |

### Sim mode

[Section titled “Sim mode”](#sim-mode)

Sim mode is the surface during a live simulation run. It is read-only but shows live-updating metrics and node statuses.

| Panel            | Behavior                                                                                                                                          |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Topbar**       | Workflow name is not editable. Save button hidden. Cancel button visible (to abort the run). Live execution metrics (elapsed time, running cost). |
| **Canvas**       | Nodes are draggable (to rearrange while watching). Connections and deletions are allowed. Cost badges show live values.                           |
| **Inspector**    | Opens on single-click. Fields are read-only. Tabs: Overview, Results, Conditions.                                                                 |
| **Bottom panel** | Expanded by default. Tabs: Logs, Runs.                                                                                                            |
| **Status bar**   | Shows progress format. Elapsed time and cost metrics visible.                                                                                     |
| **Tab toggle**   | Canvas tab only — YAML tab is hidden during simulation.                                                                                           |

Caution

Readonly and sim modes exist in the contract and are fully specified, but **edit mode is the primary shipped mode** today. The readonly and sim mode contracts are wired into the `WorkflowSurface` component, but some panels (such as the inspector) are not yet fully connected in these modes. The `RunDetail` page implements its own readonly view separately from the `WorkflowSurface` contract.

## Contract architecture

[Section titled “Contract architecture”](#contract-architecture)

The mode contract is defined as a pure TypeScript data structure with no React dependencies. Each mode maps to a `PanelContract` object:

workflowSurfaceContract.ts (simplified)

```typescript
interface PanelContract {
  topbar: {
    nameEditable: boolean;
    metricsVisible: boolean;
    metricsStyle: "live" | "static" | "none";
    saveButton: "dirty-dependent" | "disabled" | "hidden";
  };
  palette: {
    visible: boolean;
    dimmed: boolean;
    searchEditable: boolean;
  };
  canvas: {
    draggable: boolean;
    connectionsAllowed: boolean;
    deletionAllowed: boolean;
    costBadgeStyle: "estimated" | "live" | "final";
  };
  inspector: {
    trigger: "double-click" | "single-click";
    fieldsEditable: boolean;
  };
  bottomPanel: {
    defaultState: "collapsed" | "expanded";
  };
  statusBar: {
    stepCountFormat: "steps-and-edges" | "progress";
    metricsVisibility: "hidden" | "elapsed-and-cost" | "duration-and-cost";
  };
}
```

Helper functions derive specific UI flags from the mode:

| Function                              | Returns                                                                  |
| ------------------------------------- | ------------------------------------------------------------------------ |
| `isEditable(mode)`                    | Whether the mode allows any editing (fields or dragging)                 |
| `isDraggable(mode)`                   | Whether nodes can be repositioned                                        |
| `canCreateConnections(mode)`          | Whether new edges can be drawn                                           |
| `canDeleteNodes(mode)`                | Whether the delete key works                                             |
| `getAvailableTabs(mode, panel)`       | Which tabs appear in the inspector or bottom panel                       |
| `getActionButton(mode)`               | The primary action: “Save+Run” (edit), “Cancel” (sim), “Fork” (readonly) |
| `getSaveButtonState(mode, isDirty)`   | Save button state: “enabled”, “disabled”, or “hidden”                    |
| `getCostBadgeStyle(mode)`             | How cost is displayed on nodes                                           |
| `getCanvasYamlToggleVisibility(mode)` | Which tabs (Canvas, YAML) are available in the topbar                    |

## Mode transitions

[Section titled “Mode transitions”](#mode-transitions)

Mode transitions happen through React state in `WorkflowSurface`:

* **Edit to sim** — when a run starts, the mode can switch to `sim` to show live execution
* **Readonly to edit** — the Fork button creates a new workflow from the run’s historical YAML snapshot and navigates to edit mode via `handleForkTransition`
* **Sim to edit** — when a run completes or is cancelled, the mode returns to `edit`

The fork transition deserves special attention: it reads the YAML from the run’s git commit (not the current workflow file), creates a new disabled workflow, and navigates to its edit surface. This ensures the fork reflects exactly the YAML that executed, not whatever the workflow looks like now.

## Per-mode tab maps

[Section titled “Per-mode tab maps”](#per-mode-tab-maps)

Each mode defines which tabs are available in the inspector and bottom panel:

| Mode     | Inspector tabs                | Bottom panel tabs       |
| -------- | ----------------------------- | ----------------------- |
| Edit     | Overview, Prompt, Conditions  | Logs, Runs              |
| Sim      | Overview, Results, Conditions | Logs, Runs              |
| Readonly | Overview, Output, Eval, Error | Logs, Runs, Regressions |

The Regressions tab only appears in readonly mode, where you can compare a completed run against previous runs to detect quality or cost regressions.

<!-- Linear: RUN-649, Visual Workflow Builder project — last verified against codebase 2026-04-07 -->

# Canvas Overview

> How the Runsight visual canvas works — ReactFlow rendering, sidecar coordinate storage, and the component architecture.

The Runsight visual canvas displays workflows as nodes and edges on an infinite-pan canvas, powered by [XY Flow](https://xyflow.com/) (the library behind ReactFlow). Workflow **logic** lives in YAML files, while **visual layout** (node positions, viewport, selection) is stored in a separate JSON sidecar file.

Caution

The visual canvas is under active development. The YAML editor is the primary authoring surface today. See [YAML Editor](/docs/visual-builder/yaml-editor).

## Storage architecture

[Section titled “Storage architecture”](#storage-architecture)

A Runsight workflow has two persistent representations:

| Layer  | Format | Storage                              | Contains                                                           |
| ------ | ------ | ------------------------------------ | ------------------------------------------------------------------ |
| Logic  | YAML   | `custom/workflows/{id}.yaml`         | Blocks, transitions, config, version                               |
| Layout | JSON   | Canvas sidecar (JSON alongside YAML) | Node positions, edge routing, viewport, selected node, canvas mode |

The YAML file is the source of truth for execution. The canvas sidecar stores visual-only data that the engine never reads. This separation means you can edit YAML in any text editor without losing your canvas layout, and canvas rearrangements never touch the workflow logic.

Note

The sidecar JSON is written atomically alongside the YAML file. If the sidecar write fails, the YAML save still succeeds — layout data is treated as non-critical.

## YAML-to-graph conversion

[Section titled “YAML-to-graph conversion”](#yaml-to-graph-conversion)

Two modules handle the conversion between YAML text and the ReactFlow graph.

### yamlParser — YAML to graph

[Section titled “yamlParser — YAML to graph”](#yamlparser--yaml-to-graph)

`yamlParser.parseWorkflowYamlToGraph()` takes raw YAML text and an optional persisted canvas state, then returns ReactFlow nodes and edges:

1. Parses the YAML string into a JavaScript object
2. Iterates over the `blocks` dictionary, building a `StepNodeData` object for each block
3. Looks up each node’s position from the persisted canvas state; if not found, assigns a grid position (280px horizontal spacing, 160px vertical spacing, 4 columns)
4. Builds edges from `workflow.transitions` (plain edges) and `workflow.conditional_transitions` (edges with source handles keyed to decision names)
5. Returns `{ nodes, edges, viewport, error }` — errors are returned as data, never thrown

All block fields are converted from `snake_case` (YAML) to `camelCase` (JavaScript) recursively for nested objects. The conversion is generic — there is no hardcoded list of fields per block type.

### yamlCompiler — graph to YAML

[Section titled “yamlCompiler — graph to YAML”](#yamlcompiler--graph-to-yaml)

`yamlCompiler.compileGraphToWorkflowYaml()` takes ReactFlow nodes, edges, and metadata, then produces three outputs:

1. **YAML string** — clean workflow YAML with `version`, `config` (if present), `blocks`, and `workflow` sections. Runtime-only fields (`stepId`, `name`, `status`, `cost`, `executionCost`) are stripped. All `camelCase` keys are converted back to `snake_case`.
2. **Canvas state** — minimal persisted state containing only `{ id, position }` per node, plus edges, viewport, selected node ID, and canvas mode.
3. **Workflow document** — the compiled JavaScript object before YAML serialization.

Nodes with `outputConditions` produce `conditional_transitions` in the compiled YAML. The source handle on each edge maps to the decision key; a `null` source handle maps to the `default` key.

## Nodes and edges

[Section titled “Nodes and edges”](#nodes-and-edges)

### Node types

[Section titled “Node types”](#node-types)

The canvas uses a single node component type (`canvasNode`) for all blocks. Each node displays:

* An icon derived from the block type
* The block name (truncated to fit)
* A cost badge (estimated, live, or final depending on the surface mode)
* A status indicator (idle, pending, running, completed, failed)

The `StepNodeData` interface carries all block fields from the YAML schema. The `stepType` field determines which icon and behavior apply. Supported step types include `linear`, `dispatch`, `gate`, `code`, `loop`, and `workflow`, among others.

### Edge types

[Section titled “Edge types”](#edge-types)

Edges connect source blocks to target blocks. There are two categories:

* **Plain transitions** — from blocks without `outputConditions`. These map to `workflow.transitions` entries (`from` / `to`).
* **Conditional transitions** — from blocks with `outputConditions`. Each edge carries a `sourceHandle` that maps to a decision key. These map to `workflow.conditional_transitions` entries.

Edges render as straight lines with arrow markers by default, using the `--border-default` CSS variable for color.

## Canvas store

[Section titled “Canvas store”](#canvas-store)

The canvas state is managed by a Zustand store (`useCanvasStore`) that holds:

| Field            | Type                       | Description                          |
| ---------------- | -------------------------- | ------------------------------------ |
| `nodes`          | `Node[]`                   | ReactFlow node array                 |
| `edges`          | `Edge[]`                   | ReactFlow edge array                 |
| `viewport`       | `Viewport`                 | Pan/zoom state (`x`, `y`, `zoom`)    |
| `isDirty`        | `boolean`                  | Whether unsaved changes exist        |
| `selectedNodeId` | `string \| null`           | Currently selected node              |
| `canvasMode`     | `"dag" \| "state-machine"` | Layout mode                          |
| `yamlContent`    | `string`                   | Current YAML text                    |
| `blockCount`     | `number`                   | Number of blocks (parsed from YAML)  |
| `edgeCount`      | `number`                   | Number of transitions                |
| `activeRunId`    | `string \| null`           | ID of the currently executing run    |
| `runCost`        | `number`                   | Accumulated cost from the active run |

The store provides actions for node/edge changes, selection, status updates during execution, hydration from persisted state, and serialization back to persisted state.

## Canvas controls

[Section titled “Canvas controls”](#canvas-controls)

The canvas includes standard ReactFlow controls:

* **Pan and zoom** — scroll to zoom, drag to pan the background
* **Fit view** — automatically enabled on load with 0.3 padding
* **Controls panel** — zoom in, zoom out, fit view buttons
* **MiniMap** — shows a bird’s-eye view with nodes colored by type
* **Dot grid background** — 28px gap, subtle dots for spatial orientation

Tip

The YAML editor tab is the primary authoring surface today. See [YAML Editor](/docs/visual-builder/yaml-editor) for details.

## Component architecture

[Section titled “Component architecture”](#component-architecture)

The canvas is composed from these main components:

* **`WorkflowSurface`** — the top-level orchestrator. Manages mode (edit/readonly/sim), loads workflow data, coordinates the topbar, editor, bottom panel, and status bar. See [Canvas Modes](/docs/visual-builder/canvas-modes).
* **`WorkflowCanvas`** — the ReactFlow wrapper. Renders nodes and edges, delegates to the canvas store.
* **`YamlEditor`** — Monaco-based YAML editing. See [YAML Editor](/docs/visual-builder/yaml-editor).
* **`CanvasTopbar`** — workflow name (editable in edit mode), canvas/YAML tab toggle, save button, run button, and execution metrics.
* **`CanvasBottomPanel`** — collapsible panel with Logs, Runs, and Regressions tabs. Connects to SSE for real-time log streaming during execution.
* **`CanvasStatusBar`** — footer showing block/edge counts and the active tab indicator.

<!-- Linear: RUN-649, Visual Workflow Builder project — last verified against codebase 2026-04-07 -->

# Run Detail View

> Inspecting completed and running workflows — per-block status, logs, regressions, historical YAML, cost and token metrics, and fork recovery.

The Run Detail view is a read-only inspection surface for individual workflow runs. It shows per-block execution results on a canvas, structured logs, regression analysis, and the exact YAML that was executed. You reach it by clicking a run row in the Runs page or the Runs tab in the bottom panel.

## Layout

[Section titled “Layout”](#layout)

The Run Detail view is a full-page layout with four regions:

1. **Header** — workflow name, run status badge, aggregate metrics, and action buttons
2. **Center** — either the execution canvas (block nodes with status coloring) or the historical YAML editor, toggled via tabs
3. **Inspector** — a slide-in right panel showing per-block details when you click a node
4. **Bottom panel** — collapsible panel with Logs, Runs, and Regressions tabs

## Header

[Section titled “Header”](#header)

The header displays:

| Element                      | Description                                                                          |
| ---------------------------- | ------------------------------------------------------------------------------------ |
| **Back link**                | Navigates to the Runs list                                                           |
| **Workflow name**            | The name of the workflow that produced this run                                      |
| **Status badge**             | Colored badge: green for completed, red for failed/error, yellow for running/pending |
| **”Read-only review” badge** | Indicates this is a non-editable inspection view                                     |
| **Duration**                 | Total wall-clock time for the run                                                    |
| **Token count**              | Total tokens consumed across all blocks (formatted as “1.2k tok” for large numbers)  |
| **Cost**                     | Total cost in USD                                                                    |

### Action buttons

[Section titled “Action buttons”](#action-buttons)

| Button            | When visible                                | Behavior                                              |
| ----------------- | ------------------------------------------- | ----------------------------------------------------- |
| **Cancel**        | Run is active (running or pending)          | Sends a cancel request to abort the run               |
| **Open Workflow** | Run is not active                           | Navigates to the workflow’s edit surface              |
| **Fork**          | Run is not active and has a commit snapshot | Creates a new workflow from the run’s historical YAML |

Note

The Fork button is disabled in two cases: when the run is still active (wait for it to finish) and when no commit snapshot is available (the run’s YAML was never committed). A tooltip explains the reason when you hover over the disabled button.

## Execution canvas

[Section titled “Execution canvas”](#execution-canvas)

The default center view is a ReactFlow canvas showing each block as a node. Nodes are **not draggable and not connectable** — this is a read-only visualization of the execution graph.

Each node displays:

* **Block name** — the block ID from the workflow
* **Status badge** — idle, pending, running, completed, or failed
* **Cost** — actual execution cost in USD (e.g., “$0.003”)
* **Duration and tokens** — shown in a footer row when available (e.g., “1.2s, 450 tokens”)
* **Error message** — shown in a red banner at the bottom of failed nodes

Node borders change color based on status:

* **Green** (`--success-9`) — completed
* **Red** (`--danger-9`) with a glow shadow — failed
* **Muted** with reduced opacity — pending
* **Default** — idle

The MiniMap in the corner color-codes nodes: green for completed, red for failed, muted for pending, blue for running.

### Error states

[Section titled “Error states”](#error-states)

When the run graph cannot be loaded (API error), a retry card is shown instead of the canvas. When the run failed before any blocks executed (e.g., YAML validation error), a card displays the pre-execution error message.

## Block inspector

[Section titled “Block inspector”](#block-inspector)

Click any node on the canvas to open the right inspector panel. It slides in from the right with two tabs:

### Execution tab

[Section titled “Execution tab”](#execution-tab)

Shows the block’s runtime results:

* **Status banner** — colored banner (green/red/neutral) with the status label and duration
* **Cost** — the block’s execution cost in USD
* **Token usage** — prompt tokens, completion tokens, and total, each with a progress bar showing the proportion of total
* **Error** — if the block failed, the error message appears in a red box
* **Configuration** — soul reference and model used by this block

### Overview tab

[Section titled “Overview tab”](#overview-tab)

Shows the block’s identity:

* **Name** — the block ID
* **Status** — as a badge
* **Configuration** — soul reference and model

Close the inspector by clicking the X button or clicking on empty canvas space.

## Historical YAML snapshot

[Section titled “Historical YAML snapshot”](#historical-yaml-snapshot)

Switch to the YAML tab in the header to see the exact YAML that was executed for this run. This is retrieved from git using the run’s `commit_sha`:

1. The Run Detail component reads `run.commit_sha` and `run.workflow_id`
2. It calls the git API to fetch the file at `custom/workflows/{workflow_id}.yaml` at that specific commit
3. The YAML is displayed in a read-only Monaco editor

This matters because the workflow may have been modified after the run completed. The historical snapshot shows you exactly what executed, not the current state of the file.

If the run has no commit SHA (e.g., it was a simulation run from uncommitted changes), the editor falls back to showing the current workflow YAML. If no YAML is available at all, a placeholder message is shown.

## Bottom panel

[Section titled “Bottom panel”](#bottom-panel)

The bottom panel is expanded by default in the Run Detail view. It has three tabs:

### Logs tab

[Section titled “Logs tab”](#logs-tab)

Displays structured log entries for the run. Each log line shows:

* **Timestamp** — when the event occurred
* **Level** — INFO, WARN, ERROR, or DEBUG, each with its own color styling
* **Node ID** — which block produced the log (when applicable)
* **Message** — the log content

A summary banner appears at the top when the run is complete:

* Green banner with checkmark: “Run completed in X.Xs”
* Red banner with X icon: “Run failed”

### Runs tab

[Section titled “Runs tab”](#runs-tab)

Shows all runs for the same workflow in a table, sorted by most recent first. The current run is highlighted. Click a different run row to navigate to its detail view.

### Regressions tab

[Section titled “Regressions tab”](#regressions-tab)

Compares this run against historical runs for the same workflow to detect quality or cost regressions. Each regression entry shows:

* **Node name** — which block regressed
* **Type** — the kind of regression (e.g., cost increase, score decrease)
* **Delta** — the magnitude of the change (e.g., “+15%” for cost, “-0.3” for score)

When regressions are detected, a `PriorityBanner` appears above the canvas with the count (e.g., “3 regressions found”), drawing attention before you even look at the tab.

If no regressions are detected, the tab shows “No regressions detected for this run.”

## Fork recovery

[Section titled “Fork recovery”](#fork-recovery)

The Fork button enables a recovery workflow for failed runs:

1. Click Fork on a completed or failed run
2. Runsight reads the YAML from the run’s commit snapshot (the exact YAML that executed)
3. It creates a new workflow with `enabled: false` and a generated name based on the original workflow
4. You are navigated to the new workflow’s edit surface
5. Fix the issue in the YAML, save, and run again

The fork is created as an uncommitted draft — it will not auto-execute. The `enabled: false` flag prevents accidental runs before you have reviewed and fixed the YAML.

Caution

Fork requires a commit snapshot. If the run was triggered from uncommitted (dirty) YAML via a simulation branch, no snapshot is available and the Fork button is disabled.

## Live updates for active runs

[Section titled “Live updates for active runs”](#live-updates-for-active-runs)

When viewing an active run (status: running or pending), the Run Detail polls for updates:

* The run metadata refreshes every 2 seconds while the run is active
* Node statuses update as blocks complete or fail
* Logs appear in real time
* Once the run reaches a terminal state (completed/failed), polling stops

<!-- Linear: RUN-649, RUN-749, RUN-746, Visual Workflow Builder project, Run Inspection & Fork Recovery epic — last verified against codebase 2026-04-07 -->

# YAML Editor

> How to use the Monaco-based YAML editor for workflow authoring — syntax highlighting, live validation, and canvas sync.

The YAML editor is the primary authoring surface for Runsight workflows. It embeds the Monaco editor (the same engine behind VS Code) with YAML syntax highlighting, a custom color theme, and live validation that flags syntax errors as you type.

## Opening the editor

[Section titled “Opening the editor”](#opening-the-editor)

When you open a workflow from the Flows page, the editor loads in the YAML tab by default. The Canvas tab shows a placeholder — the drag-and-drop builder is under active development, so the YAML editor is where authoring happens today.

Toggle between Canvas and YAML using the tab switcher in the topbar. In edit mode, both tabs are available. In sim mode, only the Canvas tab is shown. In readonly mode, neither tab is togglable.

## Writing workflow YAML

[Section titled “Writing workflow YAML”](#writing-workflow-yaml)

Type your workflow definition directly in the editor. A minimal workflow looks like this:

custom/workflows/my-workflow\.yaml

```yaml
version: "1.0"
id: my-workflow
kind: workflow
blocks:
  summarize:
    type: linear
    soul_ref: analyst
workflow:
  name: My Workflow
  entry: summarize
  transitions: []
```

The editor provides standard code editing features:

* **Syntax highlighting** — keys, strings, numbers, and comments are colored using the Runsight YAML theme, which reads from CSS variables (`--syntax-key`, `--syntax-string`, `--syntax-value`, `--syntax-comment`, `--syntax-punct`) to match your current theme
* **Line numbers** — always visible
* **Keyboard shortcuts** — Cmd+S / Ctrl+S triggers the save action (opens the commit dialog)
* **Undo/redo** — standard Monaco undo history

## Live validation

[Section titled “Live validation”](#live-validation)

The editor validates your YAML on every keystroke with a 500ms debounce. When the YAML contains a syntax error, the `useYamlValidation` hook:

1. Parses the YAML using the `yaml` library
2. Extracts the error position (line and column)
3. Sets a Monaco error marker at that position — a red squiggly underline appears on the offending line

When the error is fixed, the marker clears immediately.

Note

Live validation checks for YAML **syntax** errors only (malformed YAML that cannot be parsed). It does not validate against the Runsight workflow schema — for example, it will not catch a misspelled block type or a missing required field.

## Syncing with the canvas store

[Section titled “Syncing with the canvas store”](#syncing-with-the-canvas-store)

Every edit in the YAML editor updates the canvas Zustand store in real time:

1. You type in the editor
2. The `onChange` handler fires
3. The new YAML content is written to `useCanvasStore.setYamlContent()`
4. The store parses block counts and edge counts from the YAML
5. The status bar updates to show the current block and edge counts

This means the store always reflects the latest YAML content, even before you save. The `isDirty` flag is set to `true` on the first edit, and the topbar shows an unsaved-changes indicator (a small dot next to the save button).

## Saving your work

[Section titled “Saving your work”](#saving-your-work)

Saving a workflow commits it to git. When you click the Save button (or press Cmd+S / Ctrl+S):

1. The commit dialog opens
2. You enter a commit message
3. Runsight writes the YAML file to `custom/workflows/{id}.yaml`
4. If canvas state exists, the sidecar JSON is written alongside it
5. The changes are committed to the main branch

After a successful commit, the `isDirty` flag resets and the unsaved indicator disappears.

## Read-only mode

[Section titled “Read-only mode”](#read-only-mode)

When viewing a run’s historical YAML (in the Run Detail view), the editor opens in read-only mode. The `readOnly` option is passed to Monaco, which disables all editing. The editor still provides syntax highlighting and scrolling, but the cursor cannot modify content.

The historical YAML shown in a run is retrieved from the git commit that was active when the run executed — not the current workflow file. This means you always see exactly the YAML that produced the run’s results, even if the workflow has been modified since.

## Editor configuration

[Section titled “Editor configuration”](#editor-configuration)

The YAML editor uses these Monaco settings:

| Setting   | Value                                              |
| --------- | -------------------------------------------------- |
| Language  | `yaml`                                             |
| Theme     | `runsight-yaml` (custom, reads from CSS variables) |
| Read-only | `false` in edit mode, `true` in readonly/sim       |
| Height    | `100%` (fills the available surface area)          |

The Monaco editor is lazy-loaded — the bundle is split into a separate chunk and loaded on demand when you first open the YAML tab. This keeps the initial page load fast.

## Working alongside the canvas

[Section titled “Working alongside the canvas”](#working-alongside-the-canvas)

The YAML editor and canvas are two views of the same workflow state. When both are wired:

* Editing YAML updates the canvas store, which drives the canvas node rendering
* Moving nodes on the canvas updates the sidecar coordinates but does not touch the YAML (layout is stored separately)
* The compiler can regenerate YAML from the canvas graph, preserving execution semantics while stripping visual metadata

Currently, the primary authoring flow is YAML-first: you write YAML in the editor, and the canvas store parses it for counts and metadata. The reverse direction (canvas edits generating YAML) is built in the compiler but the visual canvas is not yet the primary editing surface.

<!-- Linear: RUN-649, RUN-748, RUN-749, Visual Workflow Builder project — last verified against codebase 2026-04-07 -->

# Block Types

> The six block types in Runsight — linear, gate, code, loop, workflow, and dispatch.

Every block in a workflow has a `type` field that determines its behavior. All blocks share the [common fields](/docs/workflows/yaml-schema#blocks) from `BaseBlockDef` — this page covers the type-specific fields.

## linear

[Section titled “linear”](#linear)

Single LLM call through a soul. The most common block type.

```yaml
blocks:
  research:
    type: linear
    soul_ref: researcher
```

| Field      | Type  | Default  | Description                   |
| ---------- | ----- | -------- | ----------------------------- |
| `soul_ref` | `str` | required | Soul ID to use for this block |

The soul receives the current workflow state as context and its `system_prompt` as the system message. The LLM response becomes the block’s output, stored at `state.results[block_id]`.

If the block has `exits` defined, the soul can use the `delegate` tool to pick an exit port — see [Dispatch & Delegate](/docs/tools/dispatch-and-delegate).

## gate

[Section titled “gate”](#gate)

LLM quality gate — evaluates another block’s output and routes on pass/fail.

```yaml
blocks:
  quality_check:
    type: gate
    soul_ref: reviewer
    eval_key: draft_step
    pass: publish
    fail: revise
```

| Field           | Type  | Default  | Description                                                            |
| --------------- | ----- | -------- | ---------------------------------------------------------------------- |
| `soul_ref`      | `str` | required | Soul ID for the gate evaluator                                         |
| `eval_key`      | `str` | required | Block ID whose output is being evaluated                               |
| `extract_field` | `str` | none     | JSON field to extract from the target block’s output before evaluation |
| `pass`          | `str` | none     | Target block on pass (shorthand for exit routing)                      |
| `fail`          | `str` | none     | Target block on fail (shorthand for exit routing)                      |

The gate soul receives the output of `eval_key` and makes a pass/fail judgment. If `extract_field` is set, only that JSON field is extracted before the soul sees it.

`pass` and `fail` are shorthand for exit ports — they automatically create two `ExitDef` entries with IDs `"pass"` and `"fail"`. Both must be set together or both omitted. When omitted, the gate result is determined by the soul’s output and standard exit conditions.

## code

[Section titled “code”](#code)

Runs Python code in a sandboxed environment. The code must define a `def main(data)` function.

```yaml
blocks:
  transform:
    type: code
    code: |
      import json


      def main(data):
          raw = data.get("research", "")
          parsed = json.loads(raw) if raw.startswith("{") else {"text": raw}
          return {"structured": parsed}
    timeout_seconds: 15
    allowed_imports: [json, re]
```

| Field             | Type        | Default        | Description                                                                                                                                                                                                |
| ----------------- | ----------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `code`            | `str`       | required       | Python source code with a `def main(data)` function                                                                                                                                                        |
| `timeout_seconds` | `int`       | `30`           | Execution timeout in seconds (overrides base default of 300)                                                                                                                                               |
| `allowed_imports` | `List[str]` | safe whitelist | Whitelist of importable modules. If omitted, defaults to: `json`, `re`, `math`, `datetime`, `collections`, `itertools`, `hashlib`, `base64`, `time`, `urllib.parse`. Set explicitly to expand or restrict. |

The `main` function receives a `data` dict containing all upstream block results. It must return a dict — the return value becomes the block’s output.

## loop

[Section titled “loop”](#loop)

Iterates inner blocks for multiple rounds with optional break conditions.

```yaml
blocks:
  refine:
    type: loop
    inner_block_refs: [draft, review]
    max_rounds: 3
    break_condition:
      eval_key: review.verdict
      operator: equals
      value: approved
    carry_context:
      enabled: true
      mode: last
      inject_as: previous_feedback
```

| Field              | Type                                  | Default  | Description                             |
| ------------------ | ------------------------------------- | -------- | --------------------------------------- |
| `inner_block_refs` | `List[str]`                           | required | Block IDs to execute each round (min 1) |
| `max_rounds`       | `int`                                 | `5`      | Maximum iterations (1–50)               |
| `break_condition`  | `ConditionDef` or `ConditionGroupDef` | none     | Condition to exit the loop early        |
| `carry_context`    | `CarryContextConfig`                  | none     | Pass context between rounds             |
| `break_on_exit`    | `str`                                 | none     | Exit handle that triggers loop break    |
| `retry_on_exit`    | `str`                                 | none     | Exit handle that triggers another round |

### CarryContextConfig

[Section titled “CarryContextConfig”](#carrycontextconfig)

Controls how context flows between loop rounds:

| Field           | Type        | Default                    | Description                                                       |
| --------------- | ----------- | -------------------------- | ----------------------------------------------------------------- |
| `enabled`       | `bool`      | `true`                     | Enable context carrying                                           |
| `mode`          | `str`       | `"last"`                   | `"last"` (only previous round) or `"all"` (accumulate all rounds) |
| `source_blocks` | `List[str]` | none                       | Specific blocks to carry from (default: all inner blocks)         |
| `inject_as`     | `str`       | `"previous_round_context"` | Key name for injected context                                     |

If `stateful: true` is set on inner blocks, they maintain conversation history across rounds — the soul remembers prior iterations.

## workflow

[Section titled “workflow”](#workflow)

Executes a child workflow as a sub-step. Parent-child run linkage, independent error handling.

```yaml
blocks:
  sub_pipeline:
    type: workflow
    workflow_ref: analysis-pipeline
    inputs:
      topic: research.output
    outputs:
      summary: analysis.result
    on_error: catch
```

| Field          | Type             | Default   | Description                                                        |
| -------------- | ---------------- | --------- | ------------------------------------------------------------------ |
| `workflow_ref` | `str`            | required  | ID of the child workflow file to execute                           |
| `inputs`       | `Dict[str, str]` | none      | Parent state key → child state key mapping                         |
| `outputs`      | `Dict[str, str]` | none      | Parent path → child dotted path mapping                            |
| `max_depth`    | `int`            | none      | Maximum nesting depth limit                                        |
| `on_error`     | `str`            | `"raise"` | `"raise"` (propagate) or `"catch"` (absorb error, continue parent) |

The child workflow runs as a separate execution with its own run record linked to the parent. The child’s blocks execute independently — they do not see the parent’s state unless explicitly mapped via `inputs`.

When `on_error: catch` is set, the parent workflow continues execution even if the child fails.

## dispatch

[Section titled “dispatch”](#dispatch)

Parallel branching — each exit port gets its own soul and task instruction. All branches execute concurrently.

```yaml
blocks:
  analyze:
    type: dispatch
    exits:
      - id: sentiment
        label: Sentiment Analysis
        soul_ref: sentiment_analyst
        task: Analyze the sentiment of the input text.
      - id: entities
        label: Entity Extraction
        soul_ref: entity_extractor
        task: Extract all named entities from the input.
```

The dispatch block uses `DispatchExitDef` (not the standard `ExitDef`):

| Field      | Type  | Description                             |
| ---------- | ----- | --------------------------------------- |
| `id`       | `str` | Unique exit port ID                     |
| `label`    | `str` | Human-readable label                    |
| `soul_ref` | `str` | Soul ID for this branch                 |
| `task`     | `str` | Task instruction for this branch’s soul |

All branches run concurrently via `asyncio.gather`. Each branch gets its own budget session for cost isolation. Results are stored per-exit at `state.results["{block_id}.{exit_id}"]` and combined at `state.results[block_id]` as a JSON array.

dispatch vs delegate

`dispatch` is a block type — it runs all exit branches in parallel with no tool involvement. `delegate` is a builtin tool that a soul can call on any block with exits (typically `linear`) to pick one exit port for LLM-driven routing. Dispatch does not use delegate.

## Common patterns

[Section titled “Common patterns”](#common-patterns)

### Sequential pipeline

[Section titled “Sequential pipeline”](#sequential-pipeline)

```yaml
blocks:
  step_a:
    type: linear
    soul_ref: researcher
  step_b:
    type: linear
    soul_ref: writer
    depends: step_a
  step_c:
    type: gate
    soul_ref: reviewer
    eval_key: step_b
    depends: step_b
workflow:
  name: Pipeline
  entry: step_a
```

### Loop with quality gate

[Section titled “Loop with quality gate”](#loop-with-quality-gate)

```yaml
blocks:
  draft:
    type: linear
    soul_ref: writer
    stateful: true
  review:
    type: gate
    soul_ref: reviewer
    eval_key: draft
    pass: done
    fail: draft
  refine_loop:
    type: loop
    inner_block_refs: [draft, review]
    max_rounds: 3
    break_on_exit: pass
    carry_context:
      enabled: true
      mode: last
  done:
    type: code
    code: |
      def main(data):
          return {"final": data.get("draft", "")}
    depends: refine_loop
workflow:
  name: Iterative Refinement
  entry: refine_loop
```

# Context Governance

> Declared context inputs, audit events, and context-resolution behavior for Runsight workflows.

Context governance controls which workflow state each block can read. Blocks receive only the values named in their `inputs` map. A block with no `inputs` receives empty inputs.

The same declarations are used for normal execution, isolated subprocess envelopes, audit events, historical API responses, live Server-Sent Events, and the GUI audit panel.

## Declare Inputs

[Section titled “Declare Inputs”](#declare-inputs)

Use `inputs` to map local input names to context references.

custom/workflows/context-example.yaml

```yaml
version: "1.0"
kind: workflow


workflow:
  name: Context example
  entry: draft


blocks:
  draft:
    type: linear
    soul_ref: writer
    inputs:
      request:
        from: workflow.request
      safe_to_publish:
        from: shared_memory.flags.safe
      branch:
        from: metadata.runtime.branch


  review:
    type: linear
    soul_ref: reviewer
    depends: draft
    inputs:
      draft_summary:
        from: draft.summary
```

| Local input       | Reference                  | Meaning                                                 |
| ----------------- | -------------------------- | ------------------------------------------------------- |
| `request`         | `workflow.request`         | Reads workflow-seeded input named `request`.            |
| `safe_to_publish` | `shared_memory.flags.safe` | Reads `safe` from shared memory source `flags`.         |
| `branch`          | `metadata.runtime.branch`  | Reads `branch` from metadata source `runtime`.          |
| `draft_summary`   | `draft.summary`            | Reads `summary` from the upstream `draft` block result. |

Runsight does not backfill legacy state into blocks that do not declare inputs. Code blocks follow the same rule as every other block: their subprocess receives exactly `ctx.inputs`.

## YAML Reference

[Section titled “YAML Reference”](#yaml-reference)

Every block definition supports the `inputs` field. The effective context mode is always `declared`.

| Field                | Type                  | Default  | Description                                        |
| -------------------- | --------------------- | -------- | -------------------------------------------------- |
| `inputs`             | `Dict[str, InputRef]` | none     | Local input name to context reference mapping.     |
| `inputs.<name>.from` | `str`                 | required | Context reference to resolve for this local input. |

Constraints:

| Rule                                  | Behavior                         |
| ------------------------------------- | -------------------------------- |
| Block with `inputs`                   | Resolves only the listed inputs. |
| Block without `inputs`                | Provides empty inputs.           |
| Unsupported context mode field values | Invalid workflow YAML.           |
| Reserved local input names            | Invalid workflow YAML.           |

The following local input names are reserved:

| Reserved input      | Use                                       |
| ------------------- | ----------------------------------------- |
| `workflow`          | Workflow-seeded run input reference root. |
| `results`           | Explicit block result namespace root.     |
| `shared_memory`     | Shared memory namespace root.             |
| `metadata`          | Runtime metadata namespace root.          |
| `blocks`            | Engine orchestration context.             |
| `ctx`               | Engine orchestration context.             |
| `call_stack`        | Engine orchestration context.             |
| `workflow_registry` | Engine orchestration context.             |
| `observer`          | Engine orchestration context.             |

The following block IDs are reserved and cannot be used as normal block IDs:

| Reserved ID     | Use                                       |
| --------------- | ----------------------------------------- |
| `workflow`      | Workflow-seeded run input reference root. |
| `results`       | Explicit block result namespace root.     |
| `shared_memory` | Shared memory namespace root.             |
| `metadata`      | Runtime metadata namespace root.          |

## Reference Syntax

[Section titled “Reference Syntax”](#reference-syntax)

A `from` reference identifies a namespace, source, and optional field path.

| Reference                  | Namespace       | Source     | Field path | Description                               |
| -------------------------- | --------------- | ---------- | ---------- | ----------------------------------------- |
| `draft.summary`            | `results`       | `draft`    | `summary`  | Shorthand for an upstream block result.   |
| `results.draft.summary`    | `results`       | `draft`    | `summary`  | Explicit upstream block result reference. |
| `workflow.request`         | `results`       | `workflow` | `request`  | Workflow-seeded input named `request`.    |
| `shared_memory.flags.safe` | `shared_memory` | `flags`    | `safe`     | Shared memory value.                      |
| `metadata.runtime.branch`  | `metadata`      | `runtime`  | `branch`   | Runtime metadata value.                   |

Workflow-seeded input is represented in audit records as `namespace: "results"` and `source: "workflow"`. It is not a separate namespace.

Allowed audit namespaces:

| Namespace       | Description                            |
| --------------- | -------------------------------------- |
| `results`       | Workflow-seeded input or block output. |
| `shared_memory` | Shared memory state.                   |
| `metadata`      | Runtime metadata.                      |

## Governance Modes

[Section titled “Governance Modes”](#governance-modes)

Runsight resolves context in one of two governance modes.

| Mode     | Default | Missing or denied read                    | Data exposure                             |
| -------- | ------- | ----------------------------------------- | ----------------------------------------- |
| `strict` | yes     | Fails the block and emits an audit event. | Denied or missing values are not exposed. |
| `dev`    | no      | Emits a warning audit record.             | Denied or missing values are not exposed. |

Dev mode is for development feedback. It does not grant undeclared data, backfill missing values, or create implicit broad state access.

## Audit Event Model

[Section titled “Audit Event Model”](#audit-event-model)

Historical API responses and live `context_resolution` SSE events use the same event shape.

| Field            | Type                     | Description                                                    |
| ---------------- | ------------------------ | -------------------------------------------------------------- |
| `schema_version` | `str`                    | Audit event schema version. Current value: `context_audit.v1`. |
| `event`          | `str`                    | Event name for the payload.                                    |
| `run_id`         | `str`                    | Run that emitted the event.                                    |
| `workflow_name`  | `str`                    | Workflow name.                                                 |
| `node_id`        | `str`                    | Block ID from the workflow YAML.                               |
| `block_type`     | `str`                    | Block type, such as `linear`, `code`, or `workflow`.           |
| `access`         | `"declared"`             | Context mode used for the block.                               |
| `mode`           | `"strict"` or `"dev"`    | Governance mode used for resolution.                           |
| `sequence`       | `int` or `null`          | Event sequence when available.                                 |
| `records`        | `ContextAuditRecordV1[]` | Zero or more per-input resolution records.                     |
| `resolved_count` | `int`                    | Number of resolved records.                                    |
| `denied_count`   | `int`                    | Number of denied records.                                      |
| `warning_count`  | `int`                    | Number of warning records.                                     |
| `emitted_at`     | `str`                    | Event timestamp.                                               |

Example event:

ContextAuditEventV1

```json
{
  "schema_version": "context_audit.v1",
  "event": "context_resolution",
  "run_id": "8f7a23d2-3a2d-4e9a-8a3e-4df9a37c5a91",
  "workflow_name": "Context example",
  "node_id": "draft",
  "block_type": "linear",
  "access": "declared",
  "mode": "strict",
  "sequence": 12,
  "records": [
    {
      "input_name": "request",
      "from_ref": "workflow.request",
      "namespace": "results",
      "source": "workflow",
      "field_path": "request",
      "status": "resolved",
      "severity": "allow",
      "value_type": "str",
      "preview": "Summarize the launch notes",
      "reason": null,
      "internal": false
    }
  ],
  "resolved_count": 1,
  "denied_count": 0,
  "warning_count": 0,
  "emitted_at": "2026-04-17T09:30:00Z"
}
```

A declared block with no `inputs` emits an event with empty records and zero counts.

Declared block with no inputs

```json
{
  "schema_version": "context_audit.v1",
  "event": "context_resolution",
  "run_id": "8f7a23d2-3a2d-4e9a-8a3e-4df9a37c5a91",
  "workflow_name": "Context example",
  "node_id": "start",
  "block_type": "linear",
  "access": "declared",
  "mode": "strict",
  "sequence": 1,
  "records": [],
  "resolved_count": 0,
  "denied_count": 0,
  "warning_count": 0,
  "emitted_at": "2026-04-17T09:30:00Z"
}
```

`preview` is for diagnostics only. It can be redacted or truncated and is not the full payload.

## Audit Record Model

[Section titled “Audit Record Model”](#audit-record-model)

Each event contains zero or more `ContextAuditRecordV1` records.

| Field        | Type                                            | Description                                                                         |
| ------------ | ----------------------------------------------- | ----------------------------------------------------------------------------------- |
| `input_name` | `str` or `null`                                 | Local input name.                                                                   |
| `from_ref`   | `str` or `null`                                 | Original YAML `from` reference.                                                     |
| `namespace`  | `"results"`, `"shared_memory"`, or `"metadata"` | Context namespace used for the read.                                                |
| `source`     | `str` or `null`                                 | Source inside the namespace, such as a block ID, `workflow`, `flags`, or `runtime`. |
| `field_path` | `str` or `null`                                 | Nested field path inside the source.                                                |
| `status`     | `str`                                           | Resolution outcome.                                                                 |
| `severity`   | `str`                                           | Audit severity.                                                                     |
| `value_type` | `str` or `null`                                 | Type name for the resolved value when available.                                    |
| `preview`    | `str` or `null`                                 | Redacted or truncated diagnostic preview.                                           |
| `reason`     | `str` or `null`                                 | Explanation for missing, denied, or warning records.                                |
| `internal`   | `bool`                                          | Whether the record is internal runtime context.                                     |

Status values:

| Status     | Meaning                                                |
| ---------- | ------------------------------------------------------ |
| `resolved` | The declared reference resolved successfully.          |
| `missing`  | The declared reference did not exist.                  |
| `denied`   | The read was not allowed by the declaration or policy. |
| `empty`    | No declared reads were needed.                         |

Severity values:

| Severity | Meaning                                                         |
| -------- | --------------------------------------------------------------- |
| `allow`  | Resolution succeeded without warning.                           |
| `warn`   | Resolution produced a development warning or non-fatal warning. |
| `error`  | Resolution failed in strict mode or was denied.                 |

Access values:

| Access     | Meaning                                  |
| ---------- | ---------------------------------------- |
| `declared` | The block received only declared inputs. |

## API and SSE Consumption

[Section titled “API and SSE Consumption”](#api-and-sse-consumption)

### Historical endpoint

[Section titled “Historical endpoint”](#historical-endpoint)

```http
GET /api/runs/{run_id}/context-audit
```

Response shape:

| Field           | Type                    | Description                        |
| --------------- | ----------------------- | ---------------------------------- |
| `items`         | `ContextAuditEventV1[]` | Audit events for the run.          |
| `page_size`     | `int`                   | Number of events requested.        |
| `has_next_page` | `bool`                  | Whether another page is available. |
| `end_cursor`    | `str` or `null`         | Cursor for the next page.          |

### Live SSE

[Section titled “Live SSE”](#live-sse)

Live streams use event name `context_resolution`.

SSE event

```text
event: context_resolution
data: {"schema_version":"context_audit.v1","event":"context_resolution","run_id":"8f7a23d2-3a2d-4e9a-8a3e-4df9a37c5a91","workflow_name":"Context example","node_id":"draft","block_type":"linear","access":"declared","mode":"strict","sequence":12,"records":[],"resolved_count":0,"denied_count":0,"warning_count":0,"emitted_at":"2026-04-17T09:30:00Z"}
```

Historical replay and live SSE use the same `ContextAuditEventV1` payload shape.

## GUI Interpretation

[Section titled “GUI Interpretation”](#gui-interpretation)

The GUI merges historical audit events with live SSE events and deduplicates records for the run.

The audit panel displays:

| Column      | Meaning                                                                 |
| ----------- | ----------------------------------------------------------------------- |
| `Node`      | Block ID that resolved context.                                         |
| `Input`     | Local input name.                                                       |
| `Reference` | Original `from` reference.                                              |
| `Status`    | Resolution status, such as `resolved`, `missing`, `denied`, or `empty`. |
| `Severity`  | Resolution severity: `allow`, `warn`, or `error`.                       |

Badges:

| Badge             | Source                                | Meaning                                  |
| ----------------- | ------------------------------------- | ---------------------------------------- |
| `Access declared` | `access: "declared"`                  | The block received only declared inputs. |
| `Resolved`        | `denied_count: 0`, `warning_count: 0` | No denied or warning records.            |
| `Denied N`        | `denied_count`                        | One or more denied records.              |
| `Warning N`       | `warning_count`                       | One or more warning records.             |

## Edge Cases And Constraints

[Section titled “Edge Cases And Constraints”](#edge-cases-and-constraints)

| Case                             | Behavior                                                                                                                                     |
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| No `inputs`                      | The block receives empty inputs and emits an audit event with `records: []`, `resolved_count: 0`, `denied_count: 0`, and `warning_count: 0`. |
| Missing reference in strict mode | The block fails and an audit event is emitted.                                                                                               |
| Missing reference in dev mode    | The block continues with a warning, but the missing value is not exposed.                                                                    |
| Denied read in strict mode       | The block fails and an audit event is emitted.                                                                                               |
| Denied read in dev mode          | The block continues with a warning, but the denied value is not exposed.                                                                     |
| `workflow.request`               | Audited as `namespace: "results"`, `source: "workflow"`, `field_path: "request"`.                                                            |
| Audit `preview`                  | Redacted or truncated diagnostic text, not the full value.                                                                                   |

Context governance is least-privilege by default. YAML declarations are the source of truth for what a block can read, and the same declarations are used for isolated execution and audit reporting.

# Loops

> Iterate with the loop block — inner block refs, break conditions, carry context, and stateful rounds.

<!-- Linear: RUN-155 (Loop Refactor), RUN-703 (Loop E2E), RUN-666 (YAML DX Sugar) -->

The `loop` block runs a set of inner blocks for multiple rounds. It is the primary mechanism for iterative patterns like writer-critic refinement, progressive summarization, or retry-until-success flows.

## Basic loop

[Section titled “Basic loop”](#basic-loop)

A loop block references other blocks in the same workflow by ID. Each round executes them in sequence.

simple 3-round loop

```yaml
blocks:
  draft:
    type: linear
    soul_ref: writer
  review:
    type: gate
    soul_ref: critic
    eval_key: draft
    pass: done
    fail: draft
  refine:
    type: loop
    inner_block_refs: [draft, review]
    max_rounds: 3
    break_on_exit: pass
  done:
    type: code
    code: |
      def main(data):
          return {"final": data.get("draft", "")}
    depends: refine


workflow:
  name: Iterative Refinement
  entry: refine
```

The loop runs `draft` then `review` each round. If `review` produces an exit handle of `"pass"`, the loop breaks early. Otherwise it continues up to 3 rounds.

## Loop block fields

[Section titled “Loop block fields”](#loop-block-fields)

| Field              | Type                                  | Default  | Constraints | Description                                               |
| ------------------ | ------------------------------------- | -------- | ----------- | --------------------------------------------------------- |
| `inner_block_refs` | `List[str]`                           | required | min 1 item  | Block IDs to execute each round, in order                 |
| `max_rounds`       | `int`                                 | `5`      | 1—50        | Maximum number of iterations                              |
| `break_condition`  | `ConditionDef` or `ConditionGroupDef` | none     |             | Condition evaluated against the last inner block’s output |
| `carry_context`    | `CarryContextConfig`                  | none     |             | How to pass context between rounds                        |
| `break_on_exit`    | `str`                                 | none     |             | Exit handle value that stops the loop                     |
| `retry_on_exit`    | `str`                                 | none     |             | Exit handle value that restarts the current round         |

Caution

A loop block cannot reference itself in `inner_block_refs`. Self-references raise a validation error at build time. The inner blocks must be defined in the same workflow file.

## Breaking out of a loop

[Section titled “Breaking out of a loop”](#breaking-out-of-a-loop)

There are three ways to exit a loop early.

### break\_on\_exit

[Section titled “break\_on\_exit”](#break_on_exit)

Set `break_on_exit` to an exit handle string. After each inner block executes, the engine checks the block’s result. If the `exit_handle` matches, the loop stops immediately.

```yaml
refine:
  type: loop
  inner_block_refs: [draft, review]
  max_rounds: 5
  break_on_exit: pass
```

This is the most common pattern — pair it with a gate block whose `pass` exit handle triggers the break.

### break\_condition

[Section titled “break\_condition”](#break_condition)

A condition evaluated against the **last** inner block’s output at the end of each round. Uses the same condition engine as output conditions.

```yaml
refine:
  type: loop
  inner_block_refs: [draft, review]
  max_rounds: 5
  break_condition:
    eval_key: verdict
    operator: equals
    value: approved
```

You can also use a `ConditionGroupDef` with multiple conditions:

```yaml
break_condition:
  combinator: and
  conditions:
    - eval_key: score
      operator: gte
      value: 8
    - eval_key: verdict
      operator: equals
      value: approved
```

### retry\_on\_exit

[Section titled “retry\_on\_exit”](#retry_on_exit)

Set `retry_on_exit` to an exit handle string. When a block’s exit handle matches, the loop **restarts the current round** from the first inner block instead of advancing. This skips the break condition check for that round.

```yaml
refine:
  type: loop
  inner_block_refs: [draft, review]
  max_rounds: 5
  retry_on_exit: needs_revision
  break_on_exit: approved
```

## Carrying context between rounds

[Section titled “Carrying context between rounds”](#carrying-context-between-rounds)

By default, each round starts fresh — inner blocks do not see the output of previous rounds. The `carry_context` configuration changes this by injecting prior round outputs into `shared_memory`.

```yaml
refine:
  type: loop
  inner_block_refs: [draft, review]
  max_rounds: 3
  carry_context:
    enabled: true
    mode: last
    inject_as: previous_feedback
```

### CarryContextConfig fields

[Section titled “CarryContextConfig fields”](#carrycontextconfig-fields)

| Field           | Type                | Default                    | Description                                                                                                                                    |
| --------------- | ------------------- | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `enabled`       | `bool`              | `true`                     | Enable context carrying                                                                                                                        |
| `mode`          | `"last"` or `"all"` | `"last"`                   | `"last"`: inject only the previous round’s outputs. `"all"`: inject an accumulating list of all rounds.                                        |
| `source_blocks` | `List[str]`         | none                       | Specific inner blocks whose outputs are carried between rounds. If omitted, all inner blocks are used. Must be a subset of `inner_block_refs`. |
| `inject_as`     | `str`               | `"previous_round_context"` | Key name in `shared_memory` where the carried context is stored                                                                                |

### Mode: last

[Section titled “Mode: last”](#mode-last)

Injects a dict mapping source block IDs to their outputs from the previous round:

```json
{"draft": "The revised paragraph...", "review": "PASS"}
```

### Mode: all

[Section titled “Mode: all”](#mode-all)

Injects a list of all rounds’ outputs, oldest first:

```json
[
  {"draft": "First attempt...", "review": "FAIL: too short"},
  {"draft": "Revised version...", "review": "PASS"}
]
```

When using `mode: all`, the engine applies budget-aware truncation to prevent context from growing unbounded. Older entries are pruned first when the carried context exceeds 3% of the model’s context window.

## Stateful inner blocks

[Section titled “Stateful inner blocks”](#stateful-inner-blocks)

Setting `stateful: true` on inner blocks enables **conversation history persistence** across loop rounds. The soul remembers what it said in previous rounds, which is useful for iterative refinement where the model should build on its own prior attempts.

```yaml
blocks:
  draft:
    type: linear
    soul_ref: writer
    stateful: true    # remembers prior drafts across rounds
  review:
    type: gate
    soul_ref: critic
    eval_key: draft
    pass: done
    fail: draft
  refine:
    type: loop
    inner_block_refs: [draft, review]
    max_rounds: 3
    break_on_exit: pass
```

Tip

Combine `stateful: true` with `carry_context` for best results. `stateful` gives the soul its own conversation memory, while `carry_context` gives it structured access to other blocks’ outputs from prior rounds.

## Loop metadata

[Section titled “Loop metadata”](#loop-metadata)

After the loop completes, the engine stores metadata in `shared_memory` under the key `__loop__{block_id}`:

```json
{
  "rounds_completed": 2,
  "broke_early": true,
  "break_reason": "exit_handle 'pass' matched break_on_exit"
}
```

The `break_reason` values are:

* `"exit_handle '{handle}' matched break_on_exit"` — broke via `break_on_exit`
* `"condition met"` — broke via `break_condition`
* `"max_rounds reached"` — ran all rounds without breaking

The loop also stores the current round number during execution at `shared_memory["{block_id}_round"]`, so inner blocks can access it.

## Loop with all block types

[Section titled “Loop with all block types”](#loop-with-all-block-types)

Loop blocks work with any block type as inner blocks, including other loops, workflow blocks, gate blocks, and code blocks. The unified block execution lifecycle (`execute_block`) handles dispatch for all types inside the loop.

loop with code and gate inner blocks

```yaml
blocks:
  generate:
    type: code
    code: |
      def main(data):
          round_num = data.get("improve_loop_round", 1)
          return {"draft": f"Attempt {round_num}"}
  check:
    type: gate
    soul_ref: quality_checker
    eval_key: generate
    pass: done
    fail: generate
  improve_loop:
    type: loop
    inner_block_refs: [generate, check]
    max_rounds: 5
    break_on_exit: pass
```

# Sub-Workflows

> Compose workflows by nesting one inside another — parent-child run linkage, input/output mapping, error handling, and callable contracts.

<!-- Linear: RUN-598 (Sub-Workflow Execution), RUN-610 (Post-MVP artifact governance — cut) -->

The `workflow` block type lets you execute an entire child workflow as a single step in a parent workflow. The child runs in an isolated state, with explicit input/output mapping as the only channel between parent and child. This implements a Hierarchical State Machine (HSM) pattern.

## When to use sub-workflows

[Section titled “When to use sub-workflows”](#when-to-use-sub-workflows)

Use sub-workflows when you have a reusable pipeline that multiple parent workflows call, or when you want to encapsulate a complex sequence behind a clean interface. Common patterns:

* A “summarize” sub-workflow called by different analysis pipelines
* A “review and revise” loop packaged as a reusable unit
* Breaking a large workflow into composable, testable pieces

## Define the child workflow

[Section titled “Define the child workflow”](#define-the-child-workflow)

The child workflow is a standard YAML workflow file with an `interface` section that declares its public inputs and outputs. The interface is required — the engine rejects workflow blocks that reference children without one.

custom/workflows/summarizer.yaml

```yaml
version: "1.0"
id: summarizer
kind: workflow


interface:
  inputs:
    - name: topic
      target: shared_memory.topic
      type: string
      required: true
    - name: max_words
      target: shared_memory.max_words
      type: integer
      required: false
      default: 500
  outputs:
    - name: summary
      source: results.summarize
      type: string


blocks:
  research:
    type: linear
    soul_ref: researcher
  summarize:
    type: linear
    soul_ref: writer
    depends: research


workflow:
  name: Summarizer
  entry: research
```

### Interface input fields

[Section titled “Interface input fields”](#interface-input-fields)

| Field         | Type   | Default  | Description                                                           |
| ------------- | ------ | -------- | --------------------------------------------------------------------- |
| `name`        | `str`  | required | Input parameter name (must be unique)                                 |
| `target`      | `str`  | required | Dot-notation path into the child’s state (e.g. `shared_memory.topic`) |
| `type`        | `str`  | none     | Type hint for documentation                                           |
| `required`    | `bool` | `true`   | Whether the parent must provide this input                            |
| `default`     | `Any`  | none     | Default value when the parent omits this input                        |
| `description` | `str`  | none     | Human-readable description                                            |

### Interface output fields

[Section titled “Interface output fields”](#interface-output-fields)

| Field         | Type  | Default  | Description                                                         |
| ------------- | ----- | -------- | ------------------------------------------------------------------- |
| `name`        | `str` | required | Output parameter name (must be unique)                              |
| `source`      | `str` | required | Dot-notation path in child’s final state (e.g. `results.summarize`) |
| `type`        | `str` | none     | Type hint for documentation                                         |
| `description` | `str` | none     | Human-readable description                                          |

## Call the child from a parent

[Section titled “Call the child from a parent”](#call-the-child-from-a-parent)

In the parent workflow, add a `workflow` block with `workflow_ref` pointing to the child, and map inputs and outputs using interface names.

custom/workflows/analysis-pipeline.yaml

```yaml
version: "1.0"
id: analysis-pipeline
kind: workflow


blocks:
  gather:
    type: linear
    soul_ref: collector
  run_summary:
    type: workflow
    workflow_ref: summarizer
    inputs:
      topic: results.gather          # interface name → parent state path
    outputs:
      results.final_summary: summary  # parent state path → interface name
    on_error: catch
    depends: gather
  present:
    type: linear
    soul_ref: presenter
    depends: run_summary


workflow:
  name: Analysis Pipeline
  entry: gather
```

### Workflow block fields

[Section titled “Workflow block fields”](#workflow-block-fields)

| Field          | Type             | Default   | Description                                                                     |
| -------------- | ---------------- | --------- | ------------------------------------------------------------------------------- |
| `workflow_ref` | `str`            | required  | Embedded workflow id of the child workflow                                      |
| `inputs`       | `Dict[str, str]` | none      | Interface name mapped to parent state path                                      |
| `outputs`      | `Dict[str, str]` | none      | Parent state path mapped to interface output name                               |
| `max_depth`    | `int`            | none      | Maximum nesting depth (falls back to `config.max_workflow_depth`, default `10`) |
| `on_error`     | `str`            | `"raise"` | `"raise"` or `"catch"`                                                          |

### Input mapping

[Section titled “Input mapping”](#input-mapping)

Input keys are **interface names** (plain strings like `topic`), not child state paths. The engine resolves the interface name to the child’s `target` path. The values are parent state paths using dot-notation: `results.gather`, `shared_memory.topic`.

### Output mapping

[Section titled “Output mapping”](#output-mapping)

Output keys are **parent state paths** where values get written. Output values are **interface names** from the child’s interface. The engine resolves the interface name to the child’s `source` path and copies the value into the parent state.

## Error handling

[Section titled “Error handling”](#error-handling)

The `on_error` field controls what happens when the child workflow fails.

### on\_error: raise (default)

[Section titled “on\_error: raise (default)”](#on_error-raise-default)

The child’s exception propagates to the parent. The parent workflow fails at the workflow block. If the parent block has an `error_route`, the engine routes there.

### on\_error: catch

[Section titled “on\_error: catch”](#on_error-catch)

The parent workflow continues even if the child fails. The workflow block produces a `BlockResult` with:

* `exit_handle: "error"`
* `output`: an error description string
* `metadata`: includes `child_status: "failed"`, `child_error`, `child_cost_usd`, `child_tokens`, `child_duration_s`

This also catches **soft errors** — if any child block completed with `exit_handle: "error"`, the parent treats the entire child run as failed.

catch pattern with error routing

```yaml
blocks:
  risky_sub:
    type: workflow
    workflow_ref: experimental-pipeline
    on_error: catch
    error_route: fallback
  fallback:
    type: linear
    soul_ref: fallback_handler
```

## Parent-child run linkage

[Section titled “Parent-child run linkage”](#parent-child-run-linkage)

Each sub-workflow execution creates a separate run record linked to the parent run. The child gets its own observer for independent monitoring. The `BlockResult.metadata` on the parent includes `child_run_id` for drill-down.

Cost and token usage from the child are propagated back to the parent — `total_cost_usd` and `total_tokens` accumulate across the hierarchy.

## State isolation

[Section titled “State isolation”](#state-isolation)

The child workflow receives a **clean** `WorkflowState`. It does not inherit the parent’s results, shared memory, or execution log. The only data the child sees is what the parent explicitly passes through `inputs`.

Similarly, the parent only receives data from the child through the `outputs` mapping. No results leak from child to parent outside of the declared interface.

## Depth limits and cycle detection

[Section titled “Depth limits and cycle detection”](#depth-limits-and-cycle-detection)

The engine tracks a **call stack** of workflow names during execution. Two safety mechanisms prevent runaway recursion:

* **Cycle detection**: if a workflow name already appears in the call stack, the engine raises a `RecursionError`. A workflow cannot call itself, directly or indirectly.
* **Depth limit**: the call stack depth is checked against `max_depth` before each child execution. The default limit is `10`, configurable per-block or via `config.max_workflow_depth` in the workflow file.

## Workflow ref resolution

[Section titled “Workflow ref resolution”](#workflow-ref-resolution)

The `workflow_ref` value resolves only to the embedded workflow id of the child workflow. Path, filename, relative-path, and display-name aliases are not accepted.

Use `workflow_ref: summarizer` when the child file is `custom/workflows/summarizer.yaml` and the YAML contains `id: summarizer`.

# Transitions & Routing

> How blocks connect and route in a Runsight workflow — transitions, conditional branching, exit ports, and output conditions.

<!-- Linear: RUN-666 (YAML DX Sugar), RUN-643 (Dispatch Unification), RUN-598 (Sub-Workflow) -->

Runsight workflows are directed graphs. Blocks connect through **transitions** that tell the engine which block to run next. This page explains every connection mechanism — from simple A-to-B transitions through conditional branching — and how the execution engine resolves the next block at runtime.

## Plain transitions

[Section titled “Plain transitions”](#plain-transitions)

The simplest connection. A `TransitionDef` maps one block to the next.

workflow section

```yaml
workflow:
  name: Pipeline
  entry: research
  transitions:
    - from: research
      to: draft
    - from: draft
      to: publish
    - from: publish
      to: null    # terminal — workflow ends here
```

| Field  | Type            | Description                             |
| ------ | --------------- | --------------------------------------- |
| `from` | `str`           | Source block ID                         |
| `to`   | `str` or `null` | Target block ID, or `null` for terminal |

Setting `to: null` marks the block as terminal — the workflow ends after it executes. If a block has no transition at all (not listed in `transitions` and no `depends` pointing to it), it is also terminal.

Each block can have **at most one** plain transition. Attempting to add a second raises a validation error.

## Conditional transitions

[Section titled “Conditional transitions”](#conditional-transitions)

When a block needs to route to different targets based on its output, use `conditional_transitions`. Extra keys beyond `from` and `default` map decision strings to target block IDs.

workflow section

```yaml
workflow:
  name: Review Pipeline
  entry: classifier
  conditional_transitions:
    - from: classifier
      urgent: handle_urgent
      normal: handle_normal
      default: handle_normal
```

| Field          | Type            | Description                            |
| -------------- | --------------- | -------------------------------------- |
| `from`         | `str`           | Source block ID                        |
| `default`      | `str` or `null` | Fallback target if no key matches      |
| *(extra keys)* | `str`           | Decision key mapped to target block ID |

The engine resolves which key to use via the block’s **exit handle** — see the resolution order below.

A block cannot have both a plain transition and a conditional transition. The engine enforces mutual exclusivity at build time.

## Exit ports

[Section titled “Exit ports”](#exit-ports)

Exit ports declare the named outputs a block can produce. Any block type can have exits, but they are most commonly used with `linear` blocks (via the `delegate` tool) and `gate` blocks (automatic `pass`/`fail`).

block with exits

```yaml
blocks:
  reviewer:
    type: linear
    soul_ref: review_soul
    exits:
      - id: approve
        label: Approved by reviewer
      - id: reject
        label: Rejected — needs revision
```

Each `ExitDef` has:

| Field   | Type  | Description          |
| ------- | ----- | -------------------- |
| `id`    | `str` | Unique exit port ID  |
| `label` | `str` | Human-readable label |

When a block has exits and the workflow has `conditional_transitions` for that block, the transition keys should match the exit IDs. The engine validates this at build time — a transition key that does not match any declared exit (or `"default"`) produces a validation error.

## Exit conditions

[Section titled “Exit conditions”](#exit-conditions)

Exit conditions let you map output **content patterns** to exit handles without requiring the LLM to call a tool. The engine checks them after block execution.

pattern-based exit routing

```yaml
blocks:
  classifier:
    type: linear
    soul_ref: classifier_soul
    exit_conditions:
      - contains: "APPROVED"
        exit_handle: approve
      - regex: "reject|deny"
        exit_handle: reject
    exits:
      - id: approve
        label: Approved
      - id: reject
        label: Rejected
```

Each `ExitCondition` has:

| Field         | Type            | Description                                   |
| ------------- | --------------- | --------------------------------------------- |
| `contains`    | `str` or `null` | Substring match against block output          |
| `regex`       | `str` or `null` | Regex match against block output              |
| `exit_handle` | `str`           | Exit handle to set when the condition matches |

Conditions are evaluated in order. The first match wins. If no condition matches, the exit handle remains `null` and the engine falls through to plain transitions.

## Output conditions

[Section titled “Output conditions”](#output-conditions)

Output conditions evaluate structured data from a block’s result to pick a named branch. They use the condition engine with operators like `equals`, `contains`, `gt`, and more.

output\_conditions on a block

```yaml
blocks:
  analyze:
    type: code
    code: |
      def main(data):
          score = len(data.get("text", ""))
          return {"quality": "high" if score > 100 else "low"}
    output_conditions:
      - case_id: high_quality
        condition_group:
          conditions:
            - eval_key: quality
              operator: equals
              value: high
      - case_id: low_quality
        default: true
```

Each `CaseDef` has:

| Field             | Type                | Default  | Description                                        |
| ----------------- | ------------------- | -------- | -------------------------------------------------- |
| `case_id`         | `str`               | required | Decision string emitted when this case matches     |
| `condition_group` | `ConditionGroupDef` | none     | Conditions to evaluate (omit when `default: true`) |
| `default`         | `bool`              | `false`  | Whether this is the fallback case                  |

A `ConditionGroupDef` contains:

| Field        | Type                 | Default  | Description                                |
| ------------ | -------------------- | -------- | ------------------------------------------ |
| `combinator` | `str`                | `"and"`  | `"and"` or `"or"` — how conditions combine |
| `conditions` | `List[ConditionDef]` | required | Individual conditions to evaluate          |

Each `ConditionDef` has:

| Field      | Type            | Description                                 |
| ---------- | --------------- | ------------------------------------------- |
| `eval_key` | `str`           | Dot-notation path into the block’s result   |
| `operator` | `str`           | One of the supported operators (see below)  |
| `value`    | `Any` or `null` | Comparison value (omit for unary operators) |

### Supported operators

[Section titled “Supported operators”](#supported-operators)

| Category  | Operators                                                                                                        |
| --------- | ---------------------------------------------------------------------------------------------------------------- |
| String    | `equals`, `not_equals`, `contains`, `not_contains`, `starts_with`, `ends_with`, `is_empty`, `not_empty`, `regex` |
| Numeric   | `eq`, `neq`, `gt`, `lt`, `gte`, `lte`                                                                            |
| Universal | `exists`, `not_exists`                                                                                           |

## Routes (shorthand for output conditions + transitions)

[Section titled “Routes (shorthand for output conditions + transitions)”](#routes-shorthand-for-output-conditions--transitions)

`routes` combine output conditions and conditional transitions in a single, compact block. They are **mutually exclusive** with `output_conditions` — you cannot use both on the same block.

routes shorthand

```yaml
blocks:
  review:
    type: code
    code: |
      def main(data):
          return {"status": "approved"}
    routes:
      - case: publish
        when:
          conditions:
            - eval_key: status
              operator: equals
              value: approved
        goto: publish
      - case: archive
        default: true
        goto: archive
```

Each `RouteDef` has:

| Field     | Type                | Default  | Description                                 |
| --------- | ------------------- | -------- | ------------------------------------------- |
| `case`    | `str`               | required | Case ID for this route                      |
| `when`    | `ConditionGroupDef` | none     | Condition group (ignored on default routes) |
| `goto`    | `str`               | required | Target block ID                             |
| `default` | `bool`              | `false`  | Whether this is the fallback route          |

Routes require **exactly one** default route. At parse time, the engine expands routes into output conditions and conditional transitions — they are pure sugar.

## How the engine resolves the next block

[Section titled “How the engine resolves the next block”](#how-the-engine-resolves-the-next-block)

After a block finishes executing, the engine follows this resolution order in `_resolve_next`:

1. **Read exit handle** — check `state.results[block_id].exit_handle`. If the block set one (via the delegate tool, gate pass/fail, or exit conditions), use it.
2. **Evaluate output conditions** — if no exit handle was set and the block has `output_conditions`, evaluate them against the block’s output. The winning `case_id` becomes the exit handle.
3. **Conditional transition lookup** — if the block has `conditional_transitions`, use the exit handle as a lookup key in the condition map.
4. **Default fallback** — if no key matches, fall back to the `"default"` key in the condition map. If no default exists, the engine raises a `KeyError`.
5. **Plain transition** — if no conditional transitions exist, follow the plain transition (if any). If none, the block is terminal.

## Error routing

[Section titled “Error routing”](#error-routing)

Any block can specify an `error_route` — a target block that runs when the block fails with an exception. See [YAML DX Shortcuts](/docs/workflows/yaml-dx-shortcuts#error_route) for syntax details.

Error routing also handles **soft errors**: if a block completes but its exit handle is `"error"` (for example, a `workflow` block with `on_error: catch` that caught a child failure), the engine routes to the error target instead of the normal transition.

## depends shorthand

[Section titled “depends shorthand”](#depends-shorthand)

Instead of writing explicit transitions, use `depends` on individual blocks. The engine auto-generates transitions from the dependency to the dependent block. See [YAML DX Shortcuts](/docs/workflows/yaml-dx-shortcuts#depends) for details and examples.

# YAML DX Shortcuts

> Reference for all YAML authoring shortcuts: depends, error_route, gate pass/fail, routes, and inline souls.

<!-- Linear: RUN-666 (YAML DX Sugar Layer) -->

Runsight provides several shorthand features that reduce boilerplate when writing workflow YAML by hand. Each shorthand expands into the full canonical form at parse time — the engine sees the same structures either way.

## depends

[Section titled “depends”](#depends)

**Syntax:** `str` or `List[str]` on any block

Instead of writing explicit transitions in the `workflow` section, declare `depends` on a block to auto-generate a transition from the dependency to the dependent block.

with depends

```yaml
blocks:
  research:
    type: linear
    soul_ref: researcher
  draft:
    type: linear
    soul_ref: writer
    depends: research


workflow:
  name: Pipeline
  entry: research
```

This is equivalent to:

without depends (expanded form)

```yaml
blocks:
  research:
    type: linear
    soul_ref: researcher
  draft:
    type: linear
    soul_ref: writer


workflow:
  name: Pipeline
  entry: research
  transitions:
    - from: research
      to: draft
```

### Multiple dependencies

[Section titled “Multiple dependencies”](#multiple-dependencies)

Use a list to depend on multiple blocks:

```yaml
blocks:
  final:
    type: code
    code: |
      def main(data):
          return {"combined": True}
    depends:
      - step_a
      - step_b
```

Each entry generates a separate transition: `step_a -> final` and `step_b -> final`.

### Conflict detection

[Section titled “Conflict detection”](#conflict-detection)

If a dependency already has a transition defined (either explicit or from another `depends`), the parser raises a `ValueError`. Each block can only have one outgoing plain transition.

this will fail

```yaml
blocks:
  step_a:
    type: linear
    soul_ref: s1
  step_b:
    type: linear
    soul_ref: s2
    depends: step_a
  step_c:
    type: linear
    soul_ref: s3
    depends: step_a    # error: step_a already transitions to step_b
```

## error\_route

[Section titled “error\_route”](#error_route)

**Syntax:** `str` on any block

Specifies a target block that runs when the current block fails with an exception. The engine catches the error, stores error information in `shared_memory` under `__error__{block_id}`, and routes to the target block.

```yaml
blocks:
  risky_step:
    type: linear
    soul_ref: experimental
    error_route: handle_error
  handle_error:
    type: code
    code: |
      def main(data):
          error_info = data.get("__error__risky_step", {})
          return {"recovered": True, "error": error_info.get("message", "")}


workflow:
  name: Error Recovery
  entry: risky_step
  transitions:
    - from: risky_step
      to: next_step
    - from: handle_error
      to: next_step
```

Error routes also catch **soft errors**: if a block completes normally but its `exit_handle` is `"error"` (for example, a workflow block with `on_error: catch`), the engine routes to the error target instead of the normal transition.

The error information stored in `shared_memory` includes:

| Key       | Description                                |
| --------- | ------------------------------------------ |
| `type`    | Exception class name (e.g. `"ValueError"`) |
| `message` | Exception message string                   |

## Gate pass/fail shorthand

[Section titled “Gate pass/fail shorthand”](#gate-passfail-shorthand)

**Syntax:** `pass` and `fail` fields on `gate` blocks

Gate blocks produce a `"pass"` or `"fail"` exit handle based on the LLM’s judgment. The `pass` and `fail` fields are shorthand for declaring exit ports and conditional transitions.

with shorthand

```yaml
blocks:
  quality_check:
    type: gate
    soul_ref: reviewer
    eval_key: draft
    pass: publish
    fail: revise
```

This expands to:

expanded form

```yaml
blocks:
  quality_check:
    type: gate
    soul_ref: reviewer
    eval_key: draft
    exits:
      - id: pass
        label: Pass
      - id: fail
        label: Fail


workflow:
  conditional_transitions:
    - from: quality_check
      pass: publish
      fail: revise
      default: revise
```

Both `pass` and `fail` must be set together, or both omitted. Setting only one raises a validation error. When the shorthand is used, the `default` conditional transition target is set to the `fail` target.

## Routes shorthand

[Section titled “Routes shorthand”](#routes-shorthand)

**Syntax:** `routes` list on any block

Routes combine output conditions and conditional transitions in a single declaration. They are mutually exclusive with `output_conditions` — using both on the same block raises a validation error.

routes shorthand

```yaml
blocks:
  review:
    type: code
    code: |
      def main(data):
          return {"status": "approved"}
    routes:
      - case: publish
        when:
          conditions:
            - eval_key: status
              operator: equals
              value: approved
        goto: publish
      - case: archive
        default: true
        goto: archive
```

Each route has:

| Field     | Type                | Default  | Description                                        |
| --------- | ------------------- | -------- | -------------------------------------------------- |
| `case`    | `str`               | required | Case identifier                                    |
| `when`    | `ConditionGroupDef` | none     | Conditions to evaluate (ignored on default routes) |
| `goto`    | `str`               | required | Target block ID                                    |
| `default` | `bool`              | `false`  | Whether this is the fallback route                 |

Routes require **exactly one** default route. Duplicate `case` values raise a validation error.

At parse time, routes expand into:

1. `output_conditions` with `CaseDef` entries for each route
2. A `conditional_transition` mapping each case ID to its `goto` target

### Condition operators

[Section titled “Condition operators”](#condition-operators)

The `when` block uses the same condition engine as `output_conditions`. See [Transitions & Routing](/docs/workflows/transitions-and-routing#supported-operators) for the full operator list.

## Inline souls

[Section titled “Inline souls”](#inline-souls)

**Syntax:** `souls` section at the workflow top level

Souls are primarily defined as external library files under `custom/souls/`. The inline `souls:` section lets you define souls directly in the workflow YAML for convenience. Inline souls override external ones with the same ID (with a warning logged).

inline soul definition

```yaml
souls:
  quick_reviewer:
    id: quick_reviewer
    kind: soul
    name: Quick Reviewer
    role: Reviewer
    system_prompt: "Review the content for clarity and accuracy."
    model_name: gpt-4.1-mini
    temperature: 0.3


blocks:
  review:
    type: gate
    soul_ref: quick_reviewer
    eval_key: draft
    pass: done
    fail: revise
```

The key in the `souls` dict **must match** the soul’s `id` field. A mismatch raises a validation error.

### Soul fields

[Section titled “Soul fields”](#soul-fields)

| Field                 | Type        | Default  | Description                        |
| --------------------- | ----------- | -------- | ---------------------------------- |
| `id`                  | `str`       | required | Must match the dict key            |
| `kind`                | `"soul"`    | required | Entity kind                        |
| `name`                | `str`       | required | Human-readable display name        |
| `role`                | `str`       | required | Soul’s role name                   |
| `system_prompt`       | `str`       | required | System prompt for the LLM          |
| `model_name`          | `str`       | none     | Model to use (e.g. `gpt-4.1-mini`) |
| `provider`            | `str`       | none     | Provider name                      |
| `temperature`         | `float`     | none     | Sampling temperature               |
| `max_tokens`          | `int`       | none     | Max output tokens                  |
| `tools`               | `List[str]` | none     | Tool IDs this soul can use         |
| `required_tool_calls` | `List[str]` | none     | Tools the soul must call           |
| `max_tool_iterations` | `int`       | `5`      | Max tool call rounds               |
| `avatar_color`        | `str`       | none     | Color for UI rendering             |

Tip

Use inline souls for quick prototyping or one-off workflows. For souls shared across multiple workflows, define them as external files in `custom/souls/` so they stay in sync.

## Combining shortcuts

[Section titled “Combining shortcuts”](#combining-shortcuts)

All shortcuts can be used together. Here is a compact workflow using `depends`, `error_route`, gate `pass`/`fail`, inline `souls`, and `routes`:

all shortcuts combined

```yaml
version: "1.0"
id: compact-pipeline
kind: workflow


souls:
  writer:
    id: writer
    kind: soul
    name: Writer
    role: Writer
    system_prompt: "Write a concise summary."
    model_name: gpt-4.1-mini
  reviewer:
    id: reviewer
    kind: soul
    name: Reviewer
    role: Reviewer
    system_prompt: "Evaluate the summary for accuracy."
    model_name: gpt-4.1-mini


blocks:
  research:
    type: linear
    soul_ref: writer
    error_route: fallback
  draft:
    type: linear
    soul_ref: writer
    depends: research
  quality_gate:
    type: gate
    soul_ref: reviewer
    eval_key: draft
    pass: publish
    fail: draft
    depends: draft
  publish:
    type: code
    code: |
      def main(data):
          return {"published": True}
  fallback:
    type: code
    code: |
      def main(data):
          return {"error": "research failed"}


workflow:
  name: Compact Pipeline
  entry: research
```

# YAML Schema

> Structure of a Runsight workflow YAML file — all top-level sections and their fields.

A Runsight workflow is a YAML file with a defined schema. The engine validates every file against Pydantic models before parsing. A JSON schema is auto-generated for Monaco editor autocomplete.

## File structure

[Section titled “File structure”](#file-structure)

```yaml
version: "1.0"
id: example-workflow
kind: workflow


souls:                    # optional — inline soul definitions
  my_soul:
    id: my_soul
    kind: soul
    name: My Soul
    role: Analyst
    system_prompt: "..."
    model_name: gpt-4.1-mini


tools: [delegate, http]   # optional — tool IDs available to this workflow


blocks:
  step_one:
    type: linear
    soul_ref: my_soul
    exits:
      - id: done
        label: Done
      - id: retry
        label: Retry
  step_two:
    type: code
    code: |
      def main(data):
          return {"result": data["step_one"].upper()}
    depends: step_one


workflow:
  name: Example Workflow
  entry: step_one


limits:                   # optional — budget constraints
  cost_cap_usd: 1.0
  max_duration_seconds: 300
  on_exceed: fail


eval:                     # optional — test cases
  cases:
    - id: basic_test
      fixtures:
        step_one: "mock output"
```

## Top-level sections

[Section titled “Top-level sections”](#top-level-sections)

| Section     | Type                   | Default      | Description                                              |
| ----------- | ---------------------- | ------------ | -------------------------------------------------------- |
| `version`   | `str`                  | `"1.0"`      | Schema version. Currently only `"1.0"` is supported.     |
| `workflow`  | `WorkflowDef`          | **required** | Graph metadata — the only required section.              |
| `blocks`    | `Dict[str, BlockDef]`  | `{}`         | Block definitions keyed by block ID.                     |
| `souls`     | `Dict[str, SoulDef]`   | `{}`         | Inline soul definitions. Key must match `soul.id`.       |
| `tools`     | `List[str]`            | `[]`         | Tool IDs available to souls in this workflow.            |
| `limits`    | `WorkflowLimitsDef`    | none         | Workflow-level budget constraints.                       |
| `eval`      | `EvalSectionDef`       | none         | Embedded test cases for offline evaluation.              |
| `enabled`   | `bool`                 | `false`      | Whether the workflow is active.                          |
| `config`    | `Dict[str, Any]`       | `{}`         | Arbitrary workflow configuration.                        |
| `interface` | `WorkflowInterfaceDef` | none         | Public input/output contract for callable sub-workflows. |

## workflow

[Section titled “workflow”](#workflow)

The graph definition. This is the only required top-level section.

```yaml
workflow:
  name: My Workflow
  entry: step_one
  transitions:
    - from: step_one
      to: step_two
    - from: step_two
      to: null  # terminal
```

| Field                     | Type                             | Default  | Description                            |
| ------------------------- | -------------------------------- | -------- | -------------------------------------- |
| `name`                    | `str`                            | required | Workflow name                          |
| `entry`                   | `str`                            | required | Block ID to start execution            |
| `transitions`             | `List[TransitionDef]`            | `[]`     | Simple A→B transitions                 |
| `conditional_transitions` | `List[ConditionalTransitionDef]` | `[]`     | Multi-path transitions based on output |

### TransitionDef

[Section titled “TransitionDef”](#transitiondef)

```yaml
transitions:
  - from: step_a
    to: step_b
  - from: step_b
    to: null    # terminal — workflow ends after this block
```

| Field  | Type            | Description                             |
| ------ | --------------- | --------------------------------------- |
| `from` | `str`           | Source block ID                         |
| `to`   | `str` or `null` | Target block ID, or `null` for terminal |

### ConditionalTransitionDef

[Section titled “ConditionalTransitionDef”](#conditionaltransitiondef)

```yaml
conditional_transitions:
  - from: classifier
    urgent: handle_urgent
    normal: handle_normal
    default: handle_normal
```

| Field          | Type            | Description                       |
| -------------- | --------------- | --------------------------------- |
| `from`         | `str`           | Source block ID                   |
| `default`      | `str` or `null` | Fallback target if no key matches |
| *(extra keys)* | `str`           | Decision key → target block ID    |

DX shorthand: `depends`

Instead of writing explicit transitions, you can use `depends` on individual blocks:

```yaml
blocks:
  step_a:
    type: linear
    soul_ref: analyst
  step_b:
    type: linear
    soul_ref: writer
    depends: step_a      # equivalent to transition from: step_a, to: step_b
```

See [YAML DX Shortcuts](/docs/workflows/yaml-dx-shortcuts) for more.

## blocks

[Section titled “blocks”](#blocks)

Block definitions are keyed by block ID and use a discriminated union on the `type` field. See [Block Types](/docs/workflows/block-types) for the complete reference.

All blocks share these common fields from `BaseBlockDef`:

| Field               | Type                  | Default  | Description                                                     |
| ------------------- | --------------------- | -------- | --------------------------------------------------------------- |
| `type`              | `str`                 | required | Block type discriminator                                        |
| `depends`           | `str` or `List[str]`  | none     | Upstream block dependencies                                     |
| `error_route`       | `str`                 | none     | Target block on error                                           |
| `exits`             | `List[ExitDef]`       | none     | Named exit ports for branching                                  |
| `exit_conditions`   | `List[ExitCondition]` | none     | Output pattern → exit handle mapping                            |
| `assertions`        | `List[Dict]`          | none     | Block-level quality assertions                                  |
| `retry_config`      | `RetryConfig`         | none     | Retry on failure                                                |
| `timeout_seconds`   | `int`                 | `300`    | Block execution timeout (1–3600 seconds)                        |
| `limits`            | `BlockLimitsDef`      | none     | Per-block budget constraints                                    |
| `stateful`          | `bool`                | `false`  | Maintain conversation history across re-invocations             |
| `inputs`            | `Dict[str, InputRef]` | none     | Explicit upstream data references                               |
| `outputs`           | `Dict[str, str]`      | none     | Output field name → type string                                 |
| `output_conditions` | `List[CaseDef]`       | none     | Named output branches (mutually exclusive with `routes`)        |
| `routes`            | `List[RouteDef]`      | none     | Shorthand routing (mutually exclusive with `output_conditions`) |

### ExitDef

[Section titled “ExitDef”](#exitdef)

```yaml
exits:
  - id: approve
    label: Approved by reviewer
  - id: reject
    label: Rejected — needs revision
```

### ExitCondition

[Section titled “ExitCondition”](#exitcondition)

Maps output patterns to exit handles:

```yaml
exit_conditions:
  - contains: "APPROVED"
    exit_handle: approve
  - regex: "reject|deny"
    exit_handle: reject
```

### RetryConfig

[Section titled “RetryConfig”](#retryconfig)

```yaml
retry_config:
  max_attempts: 3
  backoff: exponential
  backoff_base_seconds: 2.0
```

| Field                  | Type        | Default   | Constraints                  | Description                            |
| ---------------------- | ----------- | --------- | ---------------------------- | -------------------------------------- |
| `max_attempts`         | `int`       | `3`       | 1–20                         | Maximum retry attempts                 |
| `backoff`              | `str`       | `"fixed"` | `"fixed"` or `"exponential"` | Backoff strategy                       |
| `backoff_base_seconds` | `float`     | `1.0`     | 0.1–60.0                     | Base delay between retries             |
| `non_retryable_errors` | `List[str]` | none      | —                            | Error types that should not be retried |

### InputRef

[Section titled “InputRef”](#inputref)

Explicit reference to upstream block output:

```yaml
inputs:
  summary:
    from: research_step.output
```

## limits

[Section titled “limits”](#limits)

Budget constraints at the workflow or block level.

### WorkflowLimitsDef

[Section titled “WorkflowLimitsDef”](#workflowlimitsdef)

```yaml
limits:
  cost_cap_usd: 5.0
  max_duration_seconds: 600
  token_cap: 50000
  on_exceed: fail        # "warn" or "fail"
  warn_at_pct: 0.8       # 0.0–1.0, triggers warning at this threshold
```

| Field                  | Type    | Default  | Constraints          |
| ---------------------- | ------- | -------- | -------------------- |
| `cost_cap_usd`         | `float` | none     | >= 0.0               |
| `max_duration_seconds` | `int`   | none     | 1–86400              |
| `token_cap`            | `int`   | none     | >= 1                 |
| `on_exceed`            | `str`   | `"fail"` | `"warn"` or `"fail"` |
| `warn_at_pct`          | `float` | `0.8`    | 0.0–1.0              |

### BlockLimitsDef

[Section titled “BlockLimitsDef”](#blocklimitsdef)

Same fields as `WorkflowLimitsDef` minus `warn_at_pct`. Applied per block via the `limits` field:

```yaml
blocks:
  expensive_step:
    type: linear
    soul_ref: analyst
    limits:
      cost_cap_usd: 0.50
      max_duration_seconds: 60
      on_exceed: fail
```

## eval

[Section titled “eval”](#eval)

Embedded test cases for offline evaluation. Run without LLM calls using fixture mode.

```yaml
eval:
  threshold: 0.8         # 0.0–1.0, optional pass rate threshold
  cases:
    - id: test_summary
      description: Verify summary output
      inputs:
        topic: "machine learning"
      fixtures:
        research: "ML is a subset of AI..."
      expected:
        summarize:
          - type: contains
            value: "machine learning"
          - type: word-count
            min: 50
            max: 200
```

### EvalCaseDef

[Section titled “EvalCaseDef”](#evalcasedef)

| Field         | Type                    | Default  | Description                              |
| ------------- | ----------------------- | -------- | ---------------------------------------- |
| `id`          | `str`                   | required | Unique test case ID                      |
| `description` | `str`                   | none     | Human-readable description               |
| `inputs`      | `Dict[str, Any]`        | none     | Input data for the workflow              |
| `fixtures`    | `Dict[str, str]`        | none     | Block ID → mock output (skips LLM calls) |
| `expected`    | `Dict[str, List[Dict]]` | none     | Block ID → list of assertions            |

## interface

[Section titled “interface”](#interface)

Public contract for sub-workflows called via `workflow` blocks.

```yaml
interface:
  inputs:
    - name: topic
      target: research.instruction
      type: string
      required: true
    - name: max_words
      target: config.max_words
      type: integer
      required: false
      default: 500
  outputs:
    - name: summary
      source: summarize.output
      type: string
```

### WorkflowInterfaceInputDef

[Section titled “WorkflowInterfaceInputDef”](#workflowinterfaceinputdef)

| Field         | Type   | Default  | Description                           |
| ------------- | ------ | -------- | ------------------------------------- |
| `name`        | `str`  | required | Input parameter name (must be unique) |
| `target`      | `str`  | required | Dot-notation path to child state key  |
| `type`        | `str`  | none     | Type hint                             |
| `required`    | `bool` | `true`   | Whether input must be provided        |
| `default`     | `Any`  | none     | Default value if not provided         |
| `description` | `str`  | none     | Human-readable description            |

### WorkflowInterfaceOutputDef

[Section titled “WorkflowInterfaceOutputDef”](#workflowinterfaceoutputdef)

| Field         | Type  | Default  | Description                            |
| ------------- | ----- | -------- | -------------------------------------- |
| `name`        | `str` | required | Output parameter name (must be unique) |
| `source`      | `str` | required | Dot-notation path to child result      |
| `type`        | `str` | none     | Type hint                              |
| `description` | `str` | none     | Human-readable description             |

## JSON schema for editors

[Section titled “JSON schema for editors”](#json-schema-for-editors)

A JSON schema is auto-generated from the Pydantic models for Monaco editor autocomplete:

```bash
python packages/core/scripts/generate_schema.py          # generate
python packages/core/scripts/generate_schema.py --check   # verify in sync
```

The generated schema lives at `packages/core/runsight-workflow-schema.json`.