tiny

API Integration Flow

A complete example flow that integrates multiple external APIs with data enrichment, built entirely from shipped components.

Overview

This flow demonstrates:

  • Multi-API orchestration (http_request, http-module)
  • Sequential enrichment with context accumulation on edges
  • Response caching (redis_get / redis_set, database-module)
  • Cache-hit routing (router, common-module)
  • Bounded retry on transient failures (retry, common-module)

Use Case

Build a customer profile by aggregating data from:

  • CRM system (customer details)
  • Payment provider (transaction history)
  • Support system (ticket history)

Flow Architecture

            +--------------+
signal ---->| http_server  |  GET /profile?id=cust_123
            |   request    |
            +------+-------+
                   v
            +--------------+     +----------+  out_hit
            |  redis_get   |---->|  router  |-----------------------------+
            | (cache read) |     +----+-----+                             |
            +--------------+          | default (miss)                    |
                                      v                                   |
                              +--------------+                            |
                              | http_request |  CRM API                   |
                              +------+-------+                            |
                                     v                                    |
                              +--------------+                            |
                              | http_request |  Payments API              |
                              +------+-------+                            |
                                     v                                    |
                              +--------------+                            |
                              | http_request |  Support API               |
                              +------+-------+                            |
                                     v                                    v
                              +--------------+                   +--------------+
                              |  redis_set   |------------------>| http_server  |
                              | (cache write)|                   |   response   |
                              +--------------+                   +--------------+

The enrichment chain is sequential: every edge blocks until its subtree completes, each hop folds new data into context, and errors surface deterministically. (A single source port may fan out to several edges, but there is no join component — so accumulate through the chain instead of fanning out and hoping to merge.)

Node Configurations

1. HTTP Server (API Endpoint)

Component: http-module/http_server

Wire a signal node into start to launch the server (a cron would re-launch on every tick). Read the public URL from the _control port's listenAddr.

Edge signal.out -> api-server.start configuration:

{
  "autoHostName": true,
  "readTimeout": 60,
  "writeTimeout": 30
}

Callers hit GET /profile?id=cust_123; the customer id arrives as $.requestParams.id[0] on the request output.

2. Cache Check

Component: database-module/redis_get

Edge api-server.request -> cache-check.request configuration:

{
  "context": {
    "customerId": "{{$.requestParams.id[0]}}"
  },
  "url": "redis://redis.internal:6379",
  "key": "customer-profile:{{$.requestParams.id[0]}}"
}

redis_get responds with {context, found, value} — found=false with an empty value when the key does not exist (its error port is reserved for actual Redis failures).

3. Cache Router

Component: common-module/router

_settings port configuration:

{
  "routes": ["HIT"],
  "enableDefaultPort": true
}

Edge cache-check.response -> cache-router.input configuration:

{
  "context": {
    "customerId": "{{$.context.customerId}}",
    "cached": "{{$.value}}"
  },
  "conditions": [
    { "route": "HIT", "condition": "{{$.found}}" }
  ]
}
  • out_hit → straight to the server's response port (step 7)
  • default → the enrichment chain (cache miss)

4. CRM API Client

Component: http-module/http_request

_settings: { "enableErrorPort": true }

Edge cache-router.default -> crm-client.request configuration (remember: router output nests the payload under context, so read $.context.<field>):

{
  "context": {
    "customerId": "{{$.context.customerId}}"
  },
  "method": "GET",
  "timeout": 10,
  "url": "https://api.crm-system.com/v2/contacts/{{$.context.customerId}}",
  "contentType": "application/json",
  "headers": [
    { "key": "Authorization", "value": "Bearer [[secret:crm/token]]" }
  ]
}

5. Payments API Client

Component: http-module/http_request

Edge crm-client.response -> payments-client.request configuration. This is where accumulation happens: the CRM response body (a JSON string at $.response.body) is carried forward inside context:

{
  "context": {
    "customerId": "{{$.context.customerId}}",
    "crm": "{{$.response.body}}"
  },
  "method": "GET",
  "timeout": 10,
  "url": "https://api.payment-provider.com/v1/customers/{{$.context.customerId}}/summary",
  "contentType": "application/json",
  "headers": [
    { "key": "Authorization", "value": "Basic [[secret:payments/apiKey]]" }
  ]
}

6. Support API Client

Component: http-module/http_request

Edge payments-client.response -> support-client.request configuration:

{
  "context": {
    "customerId": "{{$.context.customerId}}",
    "crm": "{{$.context.crm}}",
    "payments": "{{$.response.body}}"
  },
  "method": "GET",
  "timeout": 10,
  "url": "https://support.company.com/api/users/by-external-id/{{$.context.customerId}}/stats",
  "contentType": "application/json",
  "headers": [
    { "key": "X-API-Key", "value": "[[secret:support/apiKey]]" }
  ]
}

7. Cache Store + Response

Component: database-module/redis_set

Edge support-client.response -> cache-store.request configuration. The profile document is assembled by string interpolation — each accumulated body is itself JSON, so splicing them into a JSON template yields a valid document:

{
  "context": {
    "customerId": "{{$.context.customerId}}"
  },
  "url": "redis://redis.internal:6379",
  "key": "customer-profile:{{$.context.customerId}}",
  "value": "{\"id\": \"{{$.context.customerId}}\", \"fetchedAt\": \"{{RFC3339(now())}}\", \"crm\": {{$.context.crm}}, \"payments\": {{$.context.payments}}, \"support\": {{$.response.body}}}",
  "ttlSeconds": 300
}

Edge cache-store.response -> api-server.response configuration (fresh path):

{
  "statusCode": 200,
  "contentType": "application/json",
  "body": "{\"cached\": false, \"profile\": {\"id\": \"{{$.context.customerId}}\"}}"
}

Edge cache-router.out_hit -> api-server.response configuration (cache hit — cached holds the stored JSON document):

{
  "statusCode": 200,
  "contentType": "application/json",
  "body": "{{$.context.cached}}"
}

8. Retry on Transient API Failures

Component: common-module/retry

http_request emits the canonical {context, error, retryable} shape on its error port — network failures, 429 and 5xx are marked retryable; 4xx is not. Wire each client's error port through one retry loop per client:

  • crm-client.error -> retry-crm.request
  • retry-crm.retry -> crm-client.request (re-map the request from $.context)
  • retry-crm.failed -> api-server.response with a 502:
{
  "statusCode": 502,
  "contentType": "application/json",
  "body": "{\"error\": \"upstream unavailable\", \"detail\": \"{{$.error}}\"}"
}

_settings:

{
  "maxAttempts": 3,
  "initialDelayMs": 500,
  "maxDelayMs": 10000,
  "backoff": "exponential",
  "jitter": true
}

Alternatively, opt individual edges into scheduler-level retry via the edge's retryPolicy ({maxAttempts, initialDelayMs, ...}) — but only component-marked-transient failures are ever re-dispatched; unmarked errors are never retried.

Complete Response Example

{
  "id": "cust_12345",
  "fetchedAt": "2026-08-03T10:30:00Z",
  "crm": {
    "email": "jane.doe@company.com",
    "name": "Jane Doe",
    "company": "Acme Corp"
  },
  "payments": {
    "lifetime_value": 15420.0,
    "currency": "USD",
    "subscription": { "status": "active" }
  },
  "support": {
    "total_tickets": 12,
    "open_tickets": 1,
    "csat_score": 4.8
  }
}

Key Patterns Used

1. Context Accumulation Through a Chain

Each hop's edge folds the previous response into context, so the last node holds everything:

"context": {
  "customerId": "{{$.context.customerId}}",
  "crm": "{{$.context.crm}}",
  "payments": "{{$.response.body}}"
}

2. Cache-Aside with Real Components

redis_get → router (found?) → ... → redis_set. The found boolean routes; the stored value returns verbatim on a hit.

3. Secrets as Placeholders

Every credential is a [[secret:<name>/<key>]] placeholder resolved by the platform — secrets never appear in flow definitions or traces.

4. Expressions Do the Glue Work

Mapping happens on edges with ajson JSONPath, ternaries, &&/||, and functions like length(), upper(), now() (unix seconds), RFC3339(). There are no JavaScript methods — no .split(), no .filter(), no JSON.stringify — so keep documents whole (raw body strings) and let string interpolation assemble output JSON.

5. Explicit Retry, Bounded and Visible

Each retry loop is drawn in the graph and honors the error's own retryable flag. Failures that exhaust the loop return a real 502 to the caller instead of hanging the request.

Error Handling

Individual API Failures

Each client's error port carries {context, error, retryable, response} — the retry component sleeps and loops on retryable errors, short-circuits to failed otherwise.

Fail the Request Honestly

Because http_server blocks until its response port is fed, every terminal branch (success, cache hit, retry exhaustion) must end at response — an unterminated branch means the HTTP caller waits for the write timeout.

Timeouts

Set per-request timeout on every http_request edge; the server's writeTimeout (in the Start payload) caps the whole flow.

Monitoring Metrics

Track for observability (see common-module/flow_telemetry and platform traces):

  • Response time per upstream: identify slow APIs
  • Cache hit rate: measure caching effectiveness
  • Retry counts and failed emissions: track API reliability
  • 502 rate by source: identify problematic integrations