Records

A record is the unit of data that moves through a pipeline. It's a JSON object — string, number, boolean, null, array, or nested object as values.

{
  "id": "ord_1Q2W3E",
  "status": "paid",
  "customer": {
    "email": "Hello@Example.com",
    "country": "FR"
  },
  "items": [
    { "sku": "SKU-1", "qty": 2 },
    { "sku": "SKU-2", "qty": 1 }
  ]
}

There's no schema requirement. Cannectors doesn't enforce field presence or types between filters — that's your call, per pipeline.

Where records come from

Field paths

Most modules accept dot-notated paths to point at nested fields:

filters:
  - type: mapping
    mappings:
      - source: customer.email         # reads customer.email
        target: email                  # writes top-level email
        transforms:
          - op: lowercase

Paths work for reading and writing. Writing to a missing intermediate object creates it.

- type: set
  target: metadata.source
  value: cannectors                    # creates `metadata` if absent

Array indexing isn't supported in paths — work on arrays inside a script filter if you need to iterate.

Templates

Request fields (HTTP endpoints, headers, bodies, SOAP envelopes, cache keys) are Jinja templates evaluated once per record, with escaping matched to the field's format applied automatically:

output:
  type: httpRequest
  endpoint: https://api.example.com/customers/{{ record.customerId }}
  method: GET

SQL queries use a dedicated model: the query text carries $1, $2, … placeholders and a parameters list of expressions supplies the values, bound by the driver — never string-spliced into the SQL.

See Templating for the full syntax (filters, {% if %} blocks, contexts, escaping rules).

Mutation between filters

Each filter receives a record, can mutate it, and returns it. The next filter sees the mutated version. Inputs are not consulted again once they've handed off a batch.

This is important for filters like http_call, soap_call, and sql_call with mergeStrategy: merge — they overlay the fetched fields onto the current record before passing it down the chain.

Cross-references