Environment variables

Cannectors substitutes ${VAR_NAME} references in the YAML with the corresponding environment variable at startup. This is the only way to pass secrets into a pipeline — credentials never live in the YAML, and the resolved values are never logged.

Syntax

authentication:
  type: bearer
  credentials:
    token: ${SOURCE_BEARER_TOKEN}

The expression is ${NAME}. Curly braces are required. The variable name must match [A-Z_][A-Z0-9_]* (uppercase letters, digits, underscores; can't start with a digit).

When substitution happens

Substitution runs once, at pipeline startup — on cannectors run, after the config passes schema validation and before the runtime is built or the first input fetch happens. The resolved configuration is held in memory for the lifetime of the process.

cannectors validate deliberately skips substitution, so validating a config never requires the secrets to be present.

Implication: if you rotate a secret, you have to restart the process for Cannectors to pick up the new value. There's no hot-reload.

Where it works

Substitution happens on string values in the YAML, regardless of which module they belong to. The most common slots:

You can use ${VAR} anywhere a string is expected, including in endpoints if you really want to:

endpoint: ${SOURCE_BASE_URL}/api/orders

connectionStringRef is the one exception: it is resolved by the database layer and must be exactly one reference, so connectionStringRef: ${WAREHOUSE_DATABASE_URL} works but ${DB_HOST}/warehouse does not. Everywhere else a reference can sit inside a larger string, as in the endpoint above.

Where substitution applies

Every string in the configuration, at any depth, inside maps and lists alike — not just credential fields. That includes fields you may think of as inert: name, description, SQL statements, script bodies. Writing ${TOKEN} in prose inside a description therefore resolves it (and fails the run if the variable is unset), so escape the intent differently — for example $TOKEN without braces.

What isn't substituted

• Numbers, booleans, arrays: only strings get substituted. Don't write maxAttempts: ${MAX_ATTEMPTS} — keep numeric tuning inside the YAML.
• Templates: the {{ record.x }} syntax is not env-var substitution. Those are record placeholders, resolved per record at runtime — see Records.
• Defaults: if ${VAR} is missing or empty, Cannectors fails fast at startup with a clear error. There's no implicit "default if unset" mechanism.

Loading vars

Cannectors reads os.Environ() — nothing else. Use whatever pattern your shell/runtime already uses to populate env:

# direct
export SOURCE_BEARER_TOKEN=…
cannectors run pipeline.yaml

# inline (one-shot)
SOURCE_BEARER_TOKEN=… cannectors run pipeline.yaml

# via .env file (your shell, not Cannectors)
set -a; source .env; set +a
cannectors run pipeline.yaml

For systemd, set Environment= or EnvironmentFile= in the unit. For Kubernetes, use envFrom: secretRef or env: valueFrom: secretKeyRef.

What gets logged

The runtime logs the structure of the pipeline (module types, counts, schedule expression) but never the values resolved from env vars. If you see ${VAR} in logs, it's because substitution failed — the value was never resolved.

cannectors validate never resolves ${VAR} — substitution belongs to cannectors run, so you can validate a config on a machine that holds none of the secrets. Validation reports the pipeline name and version, not credential values.

Cross-references