Scheduling in production

cannectors run is a single long-running process. Use whatever process supervisor you already use to keep it alive, restart on crash, and stop it cleanly on shutdown.

Single-replica rule

The runtime fires on every CRON tick of its own scheduler. Two copies of the same scheduled pipeline both fire on the same tick → records get double-processed.

Run exactly one replica of any given scheduled pipeline:

• Kubernetes: Deployment with replicas: 1 and a StrategyType: Recreate (not RollingUpdate — that briefly runs two pods).
• ECS / Fly.io: 1 task per pipeline.
• systemd: 1 unit per pipeline.

For high availability, use the supervisor's restart mechanism rather than running multiple replicas. The runtime persists state on every successful batch, so a restart resumes where it left off.

systemd

[Unit]
Description=Cannectors — orders sync
After=network-online.target

[Service]
Type=simple
User=cannectors
WorkingDirectory=/var/lib/cannectors
EnvironmentFile=/etc/cannectors/orders.env
ExecStart=/usr/local/bin/cannectors run /etc/cannectors/orders.yaml
Restart=on-failure
RestartSec=10s
KillSignal=SIGTERM
TimeoutStopSec=30s

[Install]
WantedBy=multi-user.target

Key points:

• EnvironmentFile holds secrets — outside the YAML, outside the service file. See Secrets management.
• KillSignal=SIGTERM — Cannectors handles SIGTERM gracefully.
• TimeoutStopSec=30s — give the runtime time to finish the current batch and persist state before systemd sends SIGKILL.
• Restart=on-failure — restart on exit code 3 (runtime error), not on exit code 1/2 (YAML problem you have to fix anyway).

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-sync
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels: { app: orders-sync }
  template:
    metadata:
      labels: { app: orders-sync }
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: cannectors
          image: ghcr.io/your-org/cannectors:v0.1.0
          args: ["run", "/etc/cannectors/orders.yaml"]
          envFrom:
            - secretRef:
                name: orders-sync-secrets
          volumeMounts:
            - name: state
              mountPath: /state
            - name: config
              mountPath: /etc/cannectors
      volumes:
        - name: state
          persistentVolumeClaim:
            claimName: orders-sync-state
        - name: config
          configMap:
            name: orders-sync-config

Key points:

• Recreate strategy avoids running two pods during a rollout.
• terminationGracePeriodSeconds: 60 — at least as long as your longest expected batch.
• PersistentVolumeClaim for state — without it, restarting the pod loses the cursor and the next batch re-fetches everything.

CronJob is the wrong shape

A Kubernetes CronJob looks tempting, but it doesn't fit Cannectors:

• Cannectors does its own CRON scheduling.
• CronJob would spawn a new process per tick — the runtime would re-validate, re-acquire OAuth tokens, re-open DB pools every time.
• State persistence works across ticks within the same process; starting fresh each time loses warm caches and connection pools.

Use a Deployment and let the runtime tick itself.

Webhook inputs

Webhook inputs are long-running HTTP listeners with no schedule. Same deployment patterns apply, except expose a Service (or ingress) on the port listenAddress binds to.

input:
  type: webhook
  path: /webhooks/orders
  listenAddress: "0.0.0.0:8080"

Set a livenessProbe / readinessProbe against the listen address — Cannectors handles GET /healthz out of the box.

See also