Dry-run mode

cannectors run --dry-run <pipeline.yaml> runs everything up to (but not including) the output's side effects. It's the safest way to validate a fresh pipeline against a real source before you let it write to production.

What dry-run does

1. Validates the YAML against the JSON Schema. Same as cannectors validate.
2. Resolves env vars the same way a normal run does.
3. Executes the input — for real. The source API gets hit, the database query runs, the webhook listener starts.
4. Runs every filter — for real. HTTP enrichments, SQL enrichments, script transforms all execute against real services.
5. Prepares the output — formats the records as they would be sent, serializes them, etc.
6. Prints a preview — the prepared payloads are printed to stdout instead of being sent to the destination.

Steps 3 and 4 are unchanged from a real run. Only the output side effect is suppressed.

Why steps 3–4 still run

A pipeline isn't just YAML — it depends on the actual data shape coming out of the source. Real-running the input and filters is the only way to surface "this field is missing", "the API returns 401", "the condition expression doesn't compile against this record".

If you don't want the source API hit either, point the pipeline at a fixture or a local mock. The test lab exists for exactly this case.

Tuning the preview

The optional top-level dryRunOptions block controls what gets printed:

dryRunOptions:
  previewLimit: 5         # max number of preview entries to print
  includeHeaders: true    # include the request headers in the preview
  includeBody: true       # include the request body in the preview

Default is previewLimit: 10, both inclusions on. Lower the limit when the source returns large batches and you only need a sanity check.

Sample output

$ cannectors run --dry-run sync-orders.yaml

· loading pipeline sync-orders v1.0.0
✓ validated   1 input · 3 filters · 1 output
· httpPolling  GET https://source.example.com/api/orders
· fetched      48 records
· mapping      48 → 48
· condition    37 kept · 11 skipped
· dry-run      output preview (first 5 of 37):

  POST https://destination.example.com/api/orders/import
  Content-Type: application/json
  Authorization: Bearer ******

  [
    {"id": "ord_001", "email": "alice@example.com", "amount": 42.00},
    {"id": "ord_002", "email": "bob@example.com",   "amount": 19.50},
    …
  ]

✓ dry-run complete — 0 destination writes

The body is the actual JSON that would be POSTed. Auth headers are redacted.

When NOT to use dry-run

Dry-run isn't a substitute for testing. The input and filters all run for real, which means:

• The source API sees a real request (and may rate-limit you).
• http_call enrichments are billed/quota-consumed.
• script filters run and can mutate external state if you've written them to.

For deterministic testing, use the local test lab which spins up WireMock + PostgreSQL containers.

Cross-references