Dry-run mode

cannectors run --dry-run <pipeline.yaml> runs everything up to (but not including) the output's side effects. It's the safest way to validate a fresh pipeline against a real source before you let it write to production.

What dry-run does

1. Validates the YAML against the JSON Schema. Same as cannectors validate.
2. Resolves env vars the same way a normal run does.
3. Executes the input — for real. The source API gets hit, the database query runs, the webhook listener starts.
4. Runs every filter — for real. HTTP enrichments, SQL enrichments, script transforms all execute against real services.
5. Prepares the output — resolves the endpoint or renders the statement, formats the records, masks the credentials.
6. Prints a preview — what would have been sent or executed is printed to stdout instead of being performed.

Steps 3 and 4 are unchanged from a real run. Only the output side effect is suppressed.

The destination is not contacted at all: an output that holds a connection does not open it in dry-run mode. A database output whose DSN points at a host you cannot reach — or have no credentials for — still produces a full preview.

Why steps 3–4 still run

A pipeline isn't just YAML — it depends on the actual data shape coming out of the source. Real-running the input and filters is the only way to surface "this field is missing", "the API returns 401", "the condition expression doesn't compile against this record".

If you don't want the source API hit either, point the pipeline at a fixture or a local mock. The test lab exists for exactly this case.

Previewable outputs

An output module that cannot describe itself says so rather than printing nothing — an empty preview would read as "nothing would happen":

📋 Dry-Run Preview: the "customSink" output module cannot describe what it would do.
   The records above went through input and filters; the output was skipped.

Same rule when a module tries and fails — a parameter expression that cannot be evaluated, for instance. The dry-run still succeeds (nothing was written) and you get the reason instead of a blank preview:

📋 Dry-Run Preview: the output module failed to describe what it would do.
   Reason: evaluating parameter 1 ("record.id"): unknown name record
   Nothing was sent or written to the target system (dry-run mode).

Tuning the preview

The optional top-level dryRunOptions block has one setting:

dryRunOptions:
  showCredentials: false   # default: mask tokens, passwords, and DSN credentials

Set it to true only when you are debugging an auth problem in a private environment — it prints real tokens and the real password in the connection string.

Masking covers every shape a DSN takes: the password of a postgres://user:pass@host/db URL, a password= (or passwd=, pwd=) pair of a libpq key=value DSN, the same keys passed as query parameters, and a sqlite _pragma_key= encryption key. A DSN carrying no password is printed as it is — the preview never shows a redacted password the connection string does not have.

--verbose controls volume rather than secrecy: it prints every operation, the full request body and every bound row it was given, instead of a compact excerpt. Without it a pipeline in requestMode: single — or a database output whose statement renders one operation per record — prints the first 10 operations and counts the rest, and shows the first 5 bound rows of each with long values cut at 60 characters.

One cap --verbose does not lift: a database preview collects at most 100 executions per statement, and reports the others as a count. The statement, its parameters and a hundred sample rows are what you check before a write; the remaining rows are the same shape.

Sample output — HTTP output

$ cannectors run --dry-run sync-product-lines.yaml

✓ Pipeline executed successfully
  Status: success
  Records processed: 4

📋 Dry-Run Preview (what would have been done):

  Endpoint: POST https://destination.example.com/api/product-lines
  Records: 4
  Headers:
    Authorization: Bearer [MASKED-TOKEN]
    Content-Type: application/json
    User-Agent: Cannectors-Runtime/1.0
  Body (truncated, use --verbose for full):
    [
      {
        "line_id": "L-1",
        "qty": 2,
        "sku": "SKU-001"
      },
    ... (12 more lines)

ℹ️  Nothing was sent or written to the target system (dry-run mode)

Sample output — database output

The statement is printed once and the bound values follow, one line per execution: a 1000-record batch runs one statement a thousand times, and printing the SQL a thousand times would bury the thing you are checking.

$ cannectors run --dry-run load-events.yaml

✓ Pipeline executed successfully
  Status: success
  Records processed: 1000

📋 Dry-Run Preview (what would have been done):

  Operation: INSERT INTO events (postgres)
  Target: postgres://lab_user:[REDACTED]@warehouse.internal:5432/analytics
  Records: 1000
  Transaction: all statements in one transaction
  Statement:
    INSERT INTO events (id, seq) VALUES ($1, $2)
  Parameters: record.id, record.seq
  Values (truncated, use --verbose for all):
    1: "VOL-00001", 1
    2: "VOL-00002", 2
    3: "VOL-00003", 3
    4: "VOL-00004", 4
    5: "VOL-00005", 5
    ... (995 more executions)

ℹ️  Nothing was sent or written to the target system (dry-run mode)

When the query is templated and renders differently per record — an {% if %} picking a table, for instance — you get one block per rendered statement, each with its own record count. Reporting a single statement would show you SQL that half the records never run.

Parameters describes the rendered statement, not the declared parameters list, and lines up positionally with each Values row: a conditional clause left out binds fewer values, and a placeholder used twice appears twice on mysql/sqlite, which bind one value per occurrence.

When NOT to use dry-run

Dry-run isn't a substitute for testing. The input and filters all run for real, which means:

• The source API sees a real request (and may rate-limit you).
• http_call enrichments are billed/quota-consumed.
• script filters run and can mutate external state if you've written them to.
• A sql_call filter queries its database for real — only the output connection is withheld.

For deterministic testing, use the local test lab which spins up WireMock + PostgreSQL containers.

Cross-references