httpRequest

The httpRequest output ships records to an HTTP destination. Two request modes: batch (everything that reached the output in one request) or single (one request per record). Supports retries, authentication, and custom success conditions.

Minimal example

output:
  type: httpRequest
  endpoint: https://destination.example.com/api/orders/import
  method: POST
  requestMode: batch

Options

Request mode

requestMode: batch       # one request with all records (default)
requestMode: single      # one request per record

In single mode, you can template the URL with {{ record.x }}:

endpoint: https://destination.example.com/api/orders/{{ record.id }}
method: PUT
requestMode: single

The URL is resolved once per record, then the runtime fires one request for each.

Batch size

By default requestMode: batch sends every record in a single request, as one JSON array. Set batchSize to cap how many records each request carries — useful when the destination limits payload size or the number of items per call.

requestMode: batch
batchSize: 50        # 120 records -> 3 requests of 50, 50, 20

Records are split into consecutive batches in order, and one request is sent per batch. batchSize is only valid with requestMode: batch — combining it with single is rejected when the pipeline starts.

Three consequences worth planning for:

• No atomicity. Without batchSize the send is all-or-nothing. With it, a failure on the third batch leaves the first two already applied on the destination. Cannectors does not compensate them.
• onError applies per batch. fail stops at the failing batch and reports how many records were sent before it; skip and log carry on with the remaining batches and exclude the failed one from the sent count.
• Retries replay a whole batch. The retry policy applies per request, so a batch that timed out but was actually applied may be sent again. Idempotency is the destination's responsibility.

A templated endpoint or header in batch mode resolves against the first record of each batch, so with batchSize set, different batches can target different URLs.

--dry-run previews one request per batch, so the preview count matches what a real run would send. The previewed body, however, is always the default JSON array of the batch's records — if you configured a body template, the dry-run body will not match what a real run sends.

Success conditions

By default, Cannectors considers status codes 200, 201, 202, 203, and 204 as success. Override via the success block:

success:
  statusCodes: [200, 201, 202]

You can also evaluate an expr expression against the response. The variables statusCode (int), headers (map), and body (parsed JSON if applicable) are exposed:

success:
  expression: 'statusCode == 200 && body.status == "ok"'

When both statusCodes and expression are set, both must hold for the request to count as successful.

Templating

The body, headers, and URL path (in single mode) are all Jinja templates — variables, filters, and {% if %} blocks — rendered at request time, once per record. Substituted values are escaped for the field they land in: URL-escaped in the endpoint, JSON-escaped in a JSON body (a value containing " or a newline cannot break the payload).

output:
  type: httpRequest
  endpoint: https://destination.example.com/api/customers/{{ record.id }}
  method: PUT
  requestMode: single
  body: |
    {
      "id": "{{ record.id }}",
      "email": "{{ record.email | lower }}",
      "tier": "{{ record.tier | default('standard') }}"
    }

Or, for larger templates, externalize the body:

bodyTemplateFile: ./templates/customer-import.json

An endpoint that does not survive template evaluation — no scheme, no host — fails the record rather than being sent as-is. onError then decides whether the pipeline stops, drops the record, or marks it.

Query parameters

queryParams adds parameters to every request. Their values accept templates, so a parameter can carry a per-record value without being spelled into the endpoint:

output:
  type: httpRequest
  endpoint: https://destination.example.com/api/customers
  queryParams:
    source: cannectors
    tenant: "{{ record.tenantId }}"

Precedence, from weakest to strongest: the endpoint's own query string, then queryParams, then a key with paramType: query.

Values are percent-encoded exactly once — writing a template here does not double-encode it.

Examples

name: http-output-single-template
version: 1.0.0
description: Send one templated HTTP request per record with path, query, and header keys.
tags:
  - http-output
  - body-template
input:
  type: httpPolling
  schedule: "*/10 * * * *"
  endpoint: https://source.example.com/api/tasks
  dataField: tasks
filters:
  - type: remove
    target:
      - internal
      - debug
output:
  type: httpRequest
  endpoint: https://destination.example.com/api/projects/{projectId}/tasks/{{record.id}}
  method: PUT
  requestMode: single
  bodyTemplateFile: examples/assets/templates/task_update.json
  queryParams:
    source: cannectors
  keys:
    - field: project.id
      paramType: path
      paramName: projectId
    - field: updatedAt
      paramType: query
      paramName: updated_at
    - field: tenantId
      paramType: header
      paramName: X-Tenant-Id
  headers:
    Content-Type: application/json
    X-Trace-Source: "{{record.source | default('unknown')}}"
  success:
    statusCodes:
      - 200
      - 204

name: http-output-retry-auth-api-key
version: 1.0.0
description: Send records with API key authentication and output retry handling.
tags:
  - http-output
  - retry
input:
  type: httpPolling
  schedule: "*/5 * * * *"
  endpoint: https://source.example.com/api/shipments
  dataField: shipments
filters:
  - type: condition
    expression: id != nil
    else:
      - type: drop
output:
  type: httpRequest
  endpoint: https://destination.example.com/api/shipments
  method: POST
  requestMode: batch
  authentication:
    type: api-key
    credentials:
      key: ${DESTINATION_API_KEY}
      location: header
      headerName: X-API-Key
  retry:
    maxAttempts: 3
    delayMs: 500
    backoffMultiplier: 2
    maxDelayMs: 5000
    retryableStatusCodes:
      - 408
      - 429
      - 500
      - 502
      - 503
      - 504
    useRetryAfterHeader: true
    retryHintFromBody: body.retryable == true
  success:
    statusCodes:
      - 200
      - 201
      - 202

Cross-references