Internal Routing
Internal routing handles message flow within a single module. Messages between components in the same module pod are direct Go function calls through the scheduler — no network, no serialization round trip.
Overview
+-----------------------------------------------------------------------+
| INTERNAL ROUTING |
+-----------------------------------------------------------------------+
+---------------------------------------------+
| MODULE POD |
| |
| +-----------------------------------------+ |
| | SCHEDULER | |
| | | |
| | instancesMap: | |
| | node-abc123 -> Runner A | |
| | node-def456 -> Runner B | |
| | node-ghi789 -> Runner C | |
| | | |
| +------------------+----------------------+ |
| | |
| +------------+------------+ |
| | | | |
| v v v |
| +------+ +------+ +------+ |
| |Runner| |Runner| |Runner| |
| | A |--->| B |--->| C | |
| +------+ +------+ +------+ |
| direct function calls |
+---------------------------------------------+
The Scheduler
The scheduler manages all component instances within a module (internal/scheduler/scheduler.go):
type Scheduler interface {
// Install makes a component available to run instances
Install(component module.Component) error
// Update creates or updates an instance from its TinyNode spec
Update(ctx context.Context, node *v1alpha1.TinyNode) error
// Handle processes a sync incoming call; the `any` return is the
// synchronous response for blocking I/O callers
Handle(ctx context.Context, msg *runner.Msg) (any, error)
// Destroy stops the instance and deletes it
Destroy(name string) error
// HasInstance checks if an instance exists for the given node name
HasInstance(name string) bool
}
Messages address a full port name — <node-name>:<port> (colon-separated; node names themselves are dotted: {project-prefix}.{module}.{component}-...). Handle:
- Skips
_reconcile(system port, nothing to route). - Looks up the target instance. If it isn't there yet — the node may still be starting — it retries with exponential backoff (100ms → 5s, up to 30s) before giving up.
- Waits with backoff for the target port to become ready, then dispatches to the instance's
MsgHandler. - Rejects messages whose depth exceeds
MaxMessageDepth(256) with a permanent error — the cycle guard for the blocking execution model.
Message Flow
Step 1: Component Emits
func (c *Component) Handle(ctx context.Context, handler module.Handler, port string, msg any) module.Result {
result := c.process(msg)
return handler(ctx, "output", result) // triggers routing; chain the Result
}
Step 2: Edge Resolution
The runner resolves the source port to its edges (from the node's spec.edges) and forwards each message back through the scheduler with the depth incremented:
// inside the scheduler's dispatch wrapper (simplified)
outMsg.Depth = msg.Depth + 1
return s.msgHandler(outCtx, outMsg) // local -> Handle(), remote -> transport
Step 3: Delivery
For local nodes, delivery is a direct call into the target runner — no network. For nodes owned by another module, the message goes to the cross-module wire (see Cross-Module Communication).
Edges
Edges live on the source node's spec. The real shape (v1alpha1.TinyNodeEdge):
spec:
edges:
- id: edge-abc123 # edge ID
port: output # source port on this node
to: myproj.common.node-def456:input # target full port name
flowID: flow-1
retryPolicy: # optional — single-shot by default
maxAttempts: 3
initialDelayMs: 1000
type TinyNodeEdge struct {
ID string
Port string // source port name
To string // target full port name
FlowID string
RetryPolicy *EdgeRetryPolicy // optional
}
Note what is not here: edges carry no data mapping. Transformations live on the target node.
Port Configurations (Transformations)
Data mapping is stored in the target node's spec.ports as TinyNodePortConfig entries, keyed by sender:
type TinyNodePortConfig struct {
From string // sender's full port name ("" = node's own settings)
Port string // this node's port the config applies to
Schema []byte // JSON schema of the port (with configurable overlays)
Configuration []byte // JSON configuration — may contain expressions
FlowID string
}
When a message arrives, the runner picks the config matching (From, Port), evaluates its expressions against the incoming data, and decodes the result into the port's Go type:
spec:
ports:
- from: myproj.common.node-abc123:output
port: input
flowID: flow-1
configuration: '{"field": "{{$.value}}", "computed": "{{$.a + $.b}}"}'
Config keys that don't match the target struct's JSON tags are logged as orphaned and silently dropped during deserialization — watch for that log line when a mapping "does nothing".
Edge Retry Policy
Default (no retryPolicy, or maxAttempts ≤ 1) is single-shot: the scheduler dispatches once and surfaces the error. Retry is a per-edge opt-in:
type EdgeRetryPolicy struct {
MaxAttempts int // total dispatch attempts (1 = no retry)
InitialDelayMs int // initial backoff, default 1s
BackoffCoefficient string // per-attempt multiplier, default 2.0
MaxDelayMs int // cap on a single delay, default 30s
NonRetryableErrorCodes []string // codes that skip retry (via module.NonRetryable)
TimeoutMs int // per-attempt handler timeout; 0 = transport default
}
On error the scheduler checks the error's code against nonRetryableErrorCodes (components stamp codes with module.NonRetryable(code, err)); a match surfaces immediately, otherwise it re-dispatches after backoff up to maxAttempts.
Blocking Semantics
Internal routing maintains blocking semantics — a component's emit does not return until the downstream subtree finishes:
Component A Scheduler Component B Component C
| | | |
| handler("out") | | |
| ================>| | |
| | Handle(B:input) | |
| | ==================+================+ |
| | | Handle() | |
| | | ==============>| |
| | | | |
| | | handler() | |
| | | ===============+==>|
| | | | | Handle()
| | | | | return
| | | |<==+
| | | return |
| |<==================+ |
| return | | |
|<=================+ | |
v v v v
The synchronous response (the any from Handle) travels back up the same chain — this is how a blocking component like http-server returns a response computed several nodes downstream.
Concurrency
The instance map is a lock-free concurrent map (cmap.ConcurrentMap[string, *runner.Runner]) — no global scheduler lock. Multiple instances process messages concurrently, and the transports dispatch each incoming message in its own goroutine (sequential processing would deadlock the moment a handler's downstream target lives on the same module).
Performance Characteristics
| Aspect | Internal | External (NATS/gRPC) |
|---|---|---|
| Latency | Microseconds | Milliseconds |
| Serialization | None | JSON payload on the wire |
| Network | None | NATS subject / HTTP2 |
Common Issues
Missing Instance
Error: instance node-xyz123 not found after retry
Cause: TinyNode not yet reconciled, deleted, or owned by a different module.
Solution: Check the TinyNode exists in Kubernetes and its spec.module matches.
Dangling Edge
An edge whose to points at a deleted node stalls the emit until the backoff window (30s) expires. Check spec.edges of source nodes after deleting a node.
Orphaned Config Keys
"edge config has keys that don't match target port struct" orphanedKeys=[...]
Cause: The port configuration maps to fields the target type doesn't have (schema drift after a component update).
Solution: Re-open the edge in the editor and re-apply the mapping.
Expression Error
expression_error: undefined variable $.missing
Expression failures are recorded as events on the message's trace span. Check the source schema or guard the path.
Next Steps
- Cross-Module Communication - Between modules
- gRPC Fundamentals - The fallback wire
- Message Flow - Complete flow