loop

Iterate over an array field on each record and run a nested filter chain per item.

The loop filter walks an array field on each record and runs a nested filter chain for every item. The current item is exposed under the configured itemName alias; the root record stays available as record. Use it to flatten records that carry an array of "cells", "rows", or "lines" into top-level fields without dropping to a script filter.

Minimal example

Extract a displayValue keyed by columnId into a flat record.eventId:

filters:
  - type: loop
    field: cells
    itemName: cell
    filters:
      - type: condition
        expression: "cell.columnId == 8150579298996100"
        then:
          - type: mapping
            mappings:
              - source: cell.displayValue
                target: record.eventId

Options

property	type	default	description
typerequired	"loop"	—	Module type discriminator. Must be `loop` for this module.
id	string	—	Unique identifier within the pipeline.
name	string	—	Human-readable name.
description	string	—	—
enabled	boolean	true	Whether module is active.
tags	array<string>	—	—
onError	string	"fail"	Default error action. Case-insensitive; normalized to lowercase by the runtime.
fieldrequired	string	—	Path to the array field, relative to the input record (root record for the outermost loop, parent scope for nested loops). Supports dot notation.
itemNamerequired	string	—	Alias exposed to nested filters for the current item, both as a template variable and as a field path root. Cannot be 'record', '_metadata', 'loop', 'meta', 'state' or 'pagination' — those are template variables the alias would shadow — and must not duplicate an active parent loop alias.
filtersrequired	array<object>	—	Nested filters executed for every item.

Scope inside the loop

Nested filters operate on a scope — a synthetic record built once per item:

Key	What it is
`record`	The root record. Writes to `record.*` mutate the original record.
`<itemName>`	The current item. Writes to `<itemName>.*` are persisted back into the array at the same position.
Parent aliases	Every active parent loop alias when loops are nested.
`_metadata`	The root metadata. Loop iteration state lives at `_metadata.loop.<itemName>.index` (read-only).

The scope is built fresh per item, but record and _metadata are shared by reference — mutations are visible to subsequent items and filters after the loop.

One addressing root

These names work the same way in a template and in a field path. There is nothing to prefix, and nothing to translate between the two:

- type: loop
  field: lines
  itemName: line
  filters:
    - type: http_call
      # template
      endpoint: https://api.example.com/orders/{{ record.orderId }}/lines/{{ line.number }}
      keys:
        # field path — the very same two values
        - field: line.sku
          paramType: query
          paramName: sku
        - field: record.customerId
          paramType: query
          paramName: customer

To reach	Write
The current item	`line.number`
The root record	`record.orderId`
A parent loop's item	`<parentAlias>.field`
The iteration index	`_metadata.loop.line.index`
A field written at scope level	`price` — see below

An itemName may not be record, _metadata, loop, meta, state or pagination — each of those already names something at this level, and an alias taking one would shadow it. The pipeline is rejected when it starts.

Fields written at scope level

A nested filter that writes without the record. prefix writes into the scope, alongside the aliases rather than inside the record. That covers more than an explicit set: an http_call merges its response there, sql_call puts its resultKey there, a mapping target without a prefix lands there.

Such a field is read back by its bare name, in a template exactly as in a field path — {{ price }}, never {{ record.price }}, because record is the root record and the write never reached it:

- type: loop
  field: lines
  itemName: line
  filters:
    - type: http_call
      endpoint: https://catalog.example.com/skus/{{ line.sku }}
      # the response is merged into the scope: `price`, not `record.price`
    - type: mapping
      mappings:
        - source: price          # field path
          target: line.price
    - type: http_call
      endpoint: https://audit.example.com/{{ price }}   # template — same name

Scope-level fields live for one iteration. They are visible to the filters that follow inside the same item, and discarded when that item finishes: the pipeline succeeds and the field simply never appears downstream, which is easy to miss. Write to record.<field> when the value must outlive the iteration, or to <itemName>.<field> when it belongs to the item.

One consequence worth knowing: a scope-level field named record, meta, state or pagination is unreachable from a template — those names always mean the context variable. Unlike itemName, which is rejected at startup, this one cannot be caught in advance since it depends on the response shape. Map such a field to another name before reading it.

When `field` is absent or not an array

Case	Behaviour
`field` missing from the record, or `null`	No-op: the loop runs no inner filter and the record passes through unchanged.
`field` present but empty array	Same no-op, no inner filter runs.
`field` present but not an array (string, number, object)	Error — `loop field is not an array`. Handle it with the module's `onError` if a mixed-shape source is expected.

A missing field is deliberately tolerated so an optional array does not need a guard condition in front of the loop.

Loop metadata

The current index is exposed at _metadata.loop.<itemName>.index:

- type: loop
  field: rows
  itemName: row
  filters:
    - type: mapping
      mappings:
        - source: _metadata.loop.row.index
          target: row.position

_metadata.loop is read-only for nested filters. Any attempt to write under that path is rejected with loop nested filters wrote to _metadata.loop which is read-only, including replacement or deletion of _metadata itself.

Nested loops

Loops can be nested. Each inner loop adds its alias to the scope and keeps every parent alias available:

- type: loop
  field: cells
  itemName: cell
  filters:
    - type: loop
      field: cell.children
      itemName: x
      filters:
        - type: condition
          expression: >-
            _metadata.loop.cell.index == 2 &&
            _metadata.loop.x.index == 0
          then:
            - type: mapping
              mappings:
                - source: x.label
                  target: record.first

Constraints on itemName:

Cannot be record, _metadata, or loop (reserved scope keys).
Cannot duplicate an active parent loop alias.

Item removal and expansion

Nested filters return …	Effect on the array
Zero records for an item	Item is removed from the array.
Exactly one record	Item is updated with the result.
More than one record	Pipeline fails — item expansion is out of scope in v1.

Use a nested condition + drop to remove items conditionally:

- type: loop
  field: items
  itemName: item
  filters:
    - type: condition
      expression: "item.keep == false"
      then:
        - type: drop

Non-object items

Scalar / array / null items pass through unchanged when nested filters don't touch the alias. Sub-path writes such as item.foo = ... on a non-object item are rejected to prevent silent map auto-creation by the path engine.

Examples

examples/25-loop-cells-extraction.yaml— iterate cells[] and extract values by columnIdview source ↗

25-loop-cells-extraction.yaml

name: loop-cells-extraction
version: 1.0.0
description: Iterate over a cells[] array and extract values into flat record fields by columnId.
tags:
  - loop
  - smartsheet
input:
  type: httpPolling
  schedule: "*/15 * * * *"
  endpoint: https://source.example.com/api/rows
  dataField: rows
filters:
  - type: loop
    field: cells
    itemName: cell
    filters:
      - type: condition
        expression: cell.columnId == 1
        then:
          - type: mapping
            mappings:
              - source: cell.displayValue
                target: record.eventId
      - type: condition
        expression: cell.columnId == 2
        then:
          - type: mapping
            mappings:
              - source: cell.displayValue
                target: record.coordinates
      - type: condition
        expression: cell.columnId == 3
        then:
          - type: mapping
            mappings:
              - source: cell.displayValue
                target: record.customerName
  - type: remove
    target:
      - cells
output:
  type: httpRequest
  endpoint: https://destination.example.com/api/events
  method: POST
  requestMode: single

examples/27-loop-addressing-roots.yaml— item and root record, in templates and field pathsview source ↗

27-loop-addressing-roots.yaml

name: loop-addressing-roots
version: 1.0.0
description: Address the current item and the root record from inside a loop, in templates and in field paths.
tags:
  - loop
  - templating
input:
  type: httpPolling
  schedule: "*/15 * * * *"
  endpoint: https://source.example.com/api/orders
  dataField: orders
filters:
  # Inside a loop there is one addressing root, and both mechanisms use it:
  #
  #   line.<field>                    the current item
  #   record.<field>                  the root record
  #   _metadata.loop.line.index       the iteration index (read-only)
  #
  # The same names work in a template ({{ line.number }}) and in a field path
  # (keys[].field: line.number).
  - type: loop
    field: lines
    itemName: line
    filters:
      # Templates: the alias and the root, side by side in one endpoint.
      - type: http_call
        endpoint: https://catalog.example.com/api/orders/{{ record.orderId }}/lines/{{ line.number }}
        method: GET
        # Field paths: the very same two values, addressed identically.
        keys:
          - field: line.sku
            paramType: query
            paramName: sku
          - field: record.customerId
            paramType: query
            paramName: customer
        onError: log

      # The iteration index is available to nested filters too.
      - type: mapping
        mappings:
          - source: _metadata.loop.line.index
            target: line.position

      # Writing to record.<field> from inside the loop updates the root record.
      - type: mapping
        mappings:
          - source: line.sku
            target: record.lastSku
output:
  type: httpRequest
  endpoint: https://destination.example.com/api/orders/enriched
  method: POST
  requestMode: batch

examples/30-http-call-datafield-list-loop.yaml— iterate a list returned by an http_call enrichmentview source ↗

30-http-call-datafield-list-loop.yaml

name: http-call-datafield-list-loop
version: 1.0.0
description: Enrich each record with a list-shaped HTTP response, then iterate the list with the loop filter.
tags:
  - http-call
  - data-field
  - loop
input:
  type: httpPolling
  schedule: "*/10 * * * *"
  endpoint: https://source.example.com/api/orders
  dataField: orders
filters:
  - type: http_call
    endpoint: https://catalog.example.com/api/matches
    method: GET
    dataField: results
    keys:
      - field: reference
        paramType: query
        paramName: reference
    mergeStrategy: append
    resultKey: matches
    cache:
      enabled: true
      maxSize: 1000
      ttlSeconds: 300
      key: "matches:{{record.reference}}"
  - type: loop
    field: matches
    itemName: match
    filters:
      - type: mapping
        mappings:
          - source: match.productCode
            target: match.sku
      - type: remove
        target:
          - match.productCode
output:
  type: httpRequest
  endpoint: https://destination.example.com/api/orders/matched
  method: POST
  requestMode: batch

Minimal example

Options

Scope inside the loop

One addressing root

Fields written at scope level

When `field` is absent or not an array

Loop metadata

Nested loops

Item removal and expansion

Non-object items

Examples

Cross-references

condition filter

drop filter

mapping filter

script filter

On this page