JSON Schemas

Cannectors uses JSON Schema Draft 2020-12 to validate pipelines. The schemas live under internal/config/schema/ in the main repository and are the single source of truth for what a pipeline can declare.

The six files

Files reference each other via $ref to keep the common pieces shared. The validator resolves these cross-file references during cannectors validate and the run startup path.

Composition pattern

Each module config is structured the same way:

{
  "httpPollingInputConfig": {
    "allOf": [
      { "$ref": "common-schema.json#/$defs/httpRequestBase" },
      {
        "type": "object",
        "properties": {
          "schedule": { ... },
          "pagination": { ... },
          "statePersistence": { ... },
          "dataField": { ... },
          "retry": { ... }
        }
      }
    ]
  }
}

The allOf lists shared bases first (here httpRequestBase contributes endpoint, method, headers, queryParams, body, authentication, timeoutMs), then adds module-specific properties.

The wrapper (inputModule) uses oneOf over { type: const } to pick the right config:

{
  "inputModule": {
    "allOf": [
      { "$ref": "common-schema.json#/$defs/moduleBase" },
      {
        "oneOf": [
          { "allOf": [
            { "$ref": "#/$defs/httpPollingInputConfig" },
            { "properties": { "type": { "const": "httpPolling" } } }
          ] },
          ...
        ]
      }
    ]
  }
}

This pattern keeps every module's options discoverable through one known path: <group>-schema.json#/$defs/<type>InputConfig.

How this site uses them

The <ModuleOptionsTable> you see on every module reference page is built by walking these schemas at build time. The vendored copy lives under cannectors-doc/schemas/cannectors/ and is synced from the upstream repo via pnpm sync-schemas.

pnpm check-schemas asserts the vendored manifest matches the disk — a CI hook fails the build if the docs drift from the binary.

Adding a property

To make a new option visible:

1. Add it under the relevant properties in the right schema file.
2. Bump the schema's $id (the version string) if the change is non-backward-compatible.
3. Re-run the cannectors test suite — schema-driven tests will catch missing field handling in the runtime.
4. Sync the docs site: pnpm sync-schemas ../cannectors then pnpm generate-subpages.

The same JSON object now drives validation in the binary and documentation on this site — no risk of the two going out of sync.

See also