llm-module

• llm_chatStateless multi-turn conversation primitive. Caller supplies the full Messages history per Set Output JSON Schema in settings to force a schema-conforming reply on response.structured.call; component makes the API call and emits the updated history (with the assistant turn appended) on Response.Messages. Persist via document_store or kv around llm_chat: load → llm_chat → save. Defaults to Anthropic's Messages API; switch Provider to 'openai' for OpenAI Chat Completions or any OpenAI-compatible endpoint (Ollama, vLLM, OpenRouter, Azure OpenAI) via BaseURL. For tool-using agents, llm_tools is the right primitive; llm_chat is for pure conversation. Caches the system prompt when CacheSystem=true (Anthropic only) so long system prompts amortise across turns.
• llm_completeSingle-turn completion. Defaults to Anthropic's Messages API (with prompt caching on the system prompt); switch Provider to 'openai' for OpenAI Chat Completions or any OpenAI-compatible endpoint (Ollama, vLLM, OpenRouter, Azure OpenAI) via BaseURL. Emits text, model, usage, and stop reason on success; routes 429/529/5xx errors with retryable=true so upstream can decide whether to retry. Set Output JSON Schema in settings to force a schema-conforming reply: the parsed object lands on response.structured (read $.structured.).
• llm_routerRoute a message to one of N output ports based on LLM judgement. Configure Settings.Routes with {name, description} pairs — the model picks the best match per incoming message. Each route becomes an out_ output port. When EnableDefaultPort is true and confidence is below ConfidenceThreshold, routes to 'default'. Use for fuzzy intent classification, ticket triage, content moderation — anywhere boolean conditions would be too many to enumerate. The reasoning + confidence land in trace span attributes for observability. Output ports emit Context only — same shape as the deterministic router so downstream edges treat both identically.
• llm_tool_resultAppends a tool's output to a conversation so a tool-using loop can continue. Wire it between a tool's output and llm_tools' request port: take messages and toolUseId from the ToolCall that llm_tools emitted, put the tool's output in result, and send the response straight back to llm_tools. Objects and arrays are JSON-encoded, a string is passed through unchanged. This exists because an edge configuration can build an array but cannot append to one — without it every loop needs a js_eval doing the same six lines. Carry an apiKey through context if the next call needs one; this component never reads it.
• llm_toolsReAct / function-calling primitive. Declare tools in Settings; each becomes an out_ source port emitting {toolUseId, input, messages} when the model picks it. Multi-provider — Anthropic Messages tool_use (default) or OpenAI Chat Completions function calling via Provider=openai; BaseURL targets any OpenAI-compatible endpoint (Ollama, vLLM, OpenRouter). Component is stateless: caller supplies the full Messages history. To build a ReAct loop, wire out_ → handler → another llm_tools.request with the previous response's Messages plus a {role: tool, toolCallId, content} entry. Loop until the response port fires.
• mcp_callInvokes a tool on a remote MCP server. Request takes {tool, arguments}; Result emits {text, structured, isError} plus the context unchanged. To use remote tools in a ReAct loop: declare them in llm_tools' Tools setting (discover them with mcp_tools), wire out_ here mapping tool and {{$.input}} to arguments, then fold Result.text back into llm_tools.request as a {role: tool, toolCallId, content} message. A tool that reports its own failure arrives on Result with isError=true; transport failures go to the Error port and are marked retryable.
• mcp_toolsLists the tools a remote MCP server offers. Emits them in llm_tools' declaration shape ({name, description, inputSchema}) so they can be copied into that component's Tools setting — llm_tools needs its tools declared in settings to derive out_ ports, so discovery is a build-time step. Use mcp_call to actually invoke a tool. Provide the bearer token per-request rather than in settings.

Release notes