Records
The shape of the data that flows through a pipeline.
A record is the unit of data that moves through a pipeline. It's a
JSON object — string, number, boolean, null, array, or nested
object as values.
{
"id": "ord_1Q2W3E",
"status": "paid",
"customer": {
"email": "Hello@Example.com",
"country": "FR"
},
"items": [
{ "sku": "SKU-1", "qty": 2 },
{ "sku": "SKU-2", "qty": 1 }
]
}There's no schema requirement. Cannectors doesn't enforce field presence or types between filters — that's your call, per pipeline.
Where records come from
| Input | Record shape |
|---|---|
httpPolling | One element of the array pulled from dataField. |
soapPolling | One object or array element pulled from the parsed SOAP response dataField; received MTOM parts are exposed under _soapAttachments. |
webhook | The POST body (or one element of dataField if configured). |
database | One row of the query result, with column names as keys. |
Field paths
Most modules accept dot-notated paths to point at nested fields:
filters:
- type: mapping
mappings:
- source: customer.email # reads customer.email
target: email # writes top-level email
transforms:
- op: lowercasePaths work for reading and writing. Writing to a missing intermediate object creates it.
- type: set
target: metadata.source
value: cannectors # creates `metadata` if absentArray indexing isn't supported in paths — work on arrays inside a
script filter if you need to iterate.
Templates
A few modules (HTTP endpoints, body templates, SQL queries) accept
{{record.<path>}} placeholders. These are substituted once per
record before the request fires.
output:
type: httpRequest
endpoint: https://api.example.com/customers/{{record.customerId}}
method: GETFor sql_call and database outputs, placeholders are bound as
positional parameters ($1, $2, …) — they are never
string-spliced into the SQL. That makes them safe by construction.
Never wrap a {{record.x}} placeholder in quotes inside a SQL query.
It's already a parameter, not a string.
Mutation between filters
Each filter receives a record, can mutate it, and returns it. The next filter sees the mutated version. Inputs are not consulted again once they've handed off a batch.
This is important for filters like http_call, soap_call, and sql_call with
mergeStrategy: merge — they overlay the fetched fields onto the
current record before passing it down the chain.