> ## Documentation Index > Fetch the complete documentation index at: https://gomodel-fix-guardrail-enforcement-gaps.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Plugins > One plugin contract for guardrails, response and stream filters, header edits, and routing strategies: built in, compiled in, or loaded from a .so file. ## Overview A plugin is a Go value that implements `pluginapi.Plugin` plus one or more optional hook interfaces. Every [guardrail](/advanced/guardrails) is an instance of a plugin, and so is a custom routing strategy for a [virtual model](/features/virtual-models). There is one contract, in the `github.com/enterpilot/gomodel/pluginapi` package, and three ways to ship a plugin: | How | When to use it | Registered by | | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | **Built in** | Ships with GoModel: `system_prompt`, `llm_based_altering`, `string_replace`, `header_edit`, `llm_judge`, and the `cheapest_healthy` routing strategy | nothing to do | | **Compiled in** | You build your own binary from the `run` package | `ext.RegisterPlugin(func() pluginapi.Plugin { ... })` before `run.Run` | | **Shared object** (`.so`) | You run the stock binary and want to add code at startup | `plugins.load` in `config.yaml` (needs the cgo build, see [Shared-object plugins](#shared-object-plugins)) | Whichever way it arrives, a plugin type shows up under its manifest name on the dashboard's Guardrails page, in `GET /admin/plugins`, and as a `type` in `guardrails.rules`. `pluginapi` depends on the standard library only. A plugin imports it and nothing else from GoModel. ## Phases A plugin declares which hooks it implements in `Manifest.Kinds`. GoModel runs the content phases from the matched [workflow](/advanced/workflows): | Phase | Hook | Runs | Sees | | ---------- | ------------------------------------------ | --------------------------------------------------------------------------------------- | ----------------------------------------------- | | `prompt` | `PromptHook.OnPrompt` | after routing, before the provider call | `Prompt`, resolved provider and model in `Meta` | | `response` | `ResponseHook.OnResponse` | on a complete non-streaming response, or on the assembled response of a buffered stream | `Prompt`, `Response`, `Meta.Attempts` | | `stream` | `StreamHook.OnStreamEvent` / `OnStreamEnd` | per streamed event, then once at the end | `Prompt`, `Stream` (text so far), each event | Which instance runs in which phase, and in what order, is a workflow decision (`steps[].phase` and `steps[].step`). The same instance may appear once per phase. Two more hook kinds exist in the contract but are not run by this release: `request` (before model resolution) and `complete` (after the client response is written). A manifest may declare them; GoModel validates that the interfaces are implemented and ignores them at runtime. The `route` kind is used by virtual models, see [Routing strategy plugins](#routing-strategy-plugins). Phases apply to `/v1/chat/completions`, `/v1/responses`, and `/v1/messages`. Anthropic requests are translated to the chat form before any hook runs, so a plugin sees one shape; `Meta.Dialect` reports `anthropic_messages` when it matters. Inline `/v1/batches` items go through the `prompt` phase when `guardrails.enable_for_batch_processing` is on. ## Chain execution Within one phase, instances are grouped by `step`: * Steps run in ascending order. A later step sees the edits of an earlier one. * Within a step, instances whose manifest says `Mutates: false` run **concurrently** on a shallow copy of the exchange (their `Values` and response headers are merged back). The step may hold **at most one** instance with `Mutates: true`, which runs after the readers. Two mutating instances at the same step is a configuration error. * Decisions of a step merge by severity: `block` > `respond` > `warn` > `allow`. The first blocking decision ends the chain after its step. ## Decisions Every hook returns a `pluginapi.Decision`: | Action | Effect | Defaults | | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | `allow` | continue with the edits already made to the exchange | zero value of `Decision` | | `block` | reject with `Status`, `Code`, and `Message` in the endpoint's native error format | `Status` 400 in the prompt phase, 502 in the response and stream phases; `Code` `guardrail_blocked` | | `respond` | answer the request with `Response` as an ordinary assistant turn, HTTP 200 (a synthesized one-turn stream for streaming requests); in the response and stream phases the provider's `usage` is kept on the reply, since the inference was already performed | `pluginapi.Respond(text)` builds a one-choice completion | | `warn` | continue; record `Code`, `Message`, and `Detail` in the audit trail and add `X-GoModel-Guardrail: warn; code=` to the client response | `Code` `guardrail_warning` | `respond` is the "block with a safe message" most guardrail products offer: agent loops keep running instead of surfacing a 4xx. Use `block` when the caller should see an error. `Detail` is stored in the audit record and must not contain secrets. `NoStore` on any decision keeps the response out of the [response cache](/features/cache), exact and semantic, so the next request with the same or a similar body runs the plugins again instead of replaying a stored answer. Set it when the reply carries request-specific data a plugin restores on the way out, for example a de-anonymizer that puts the caller's PII back into the assistant text: a cached copy would hand that text to whoever sends a matching prompt next. It is honoured from any phase and any instance of the request, whatever the merged decision was. ### Audit trail Every instance that ran leaves an outcome in the audit entry's `data.guardrails` list, in execution order across the prompt, response and stream phases, silent allows included: | Field | Meaning | | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | `phase`, `step`, `instance`, `type` | Where the instance ran: the workflow phase and step, the instance name (the workflow's `ref`) and its plugin type. | | `action` | `allow`, `warn`, `block`, `respond`, or `failure` when the hook errored instead of deciding. | | `code`, `message`, `detail` | The decision's fields. The dashboard list and live views omit `detail`; the entry detail carries it. | | `error`, `fail_mode` | With `failure`: the error text and whether the chain carried on (`open`) or the request failed (`closed`). | | `edited`, `target` | Whether the instance changed the `request` (prompt phase) or the `response` (response and stream phases). | | `replaced_events`, `dropped_events` | For an in-flight stream instance, how many events it rewrote or withheld. | | `duration_ns` | The hook's run time. | A configured step without an outcome did not run: an earlier block or answer ended the request, or a cache hit skipped the model call. The dashboard colors the workflow chart of an audit entry from this list, and its "Guardrails" tab lists the outcomes. The list is the decision trail; the request revisions below are the body trail. Prompt-phase decisions are also appended to the request's revision chain (the same place request rewrites are recorded), one entry per instance in step order. An instance that edited the prompt is a changed revision carrying the request as it stood right after that step, sizes included, so a chain of edits reads step by step and the last one is what was forwarded; the request bodies are stored under the same `LOGGING_LOG_BODIES` and `LOGGING_LOG_REVISION_BODIES` gates as request rewrites. Each editing step leaves only a copy of the prompt behind on the request path; applying those copies to the request and encoding the bodies runs afterwards, alongside the provider call. `LOGGING_LOG_GUARDRAIL_STEPS=false` skips them and records a chain's edits as one revision carrying the request as forwarded. A warning, block, answer or failure without an edit is a no-change entry carrying the decision; a silent allow leaves no revision. Response and stream decisions are recorded in `data.guardrails` only, and the objections among them are logged with the request id. ## Fail modes and timeouts Each instance has a `fail_mode` and a `timeout_ms`: | Setting | Values | Default | | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------- | | `fail_mode` | `closed`: an error, panic, or timeout rejects the request with HTTP 500 and code `plugin_failure`; `open`: log and continue as if the instance allowed | `closed` for `prompt`, `response`, and `stream` | | `timeout_ms` | upper bound for every hook call of the instance | `0` (no per-instance timeout) | The instance name of a fail-closed failure goes to the logs and the audit record, never to the client. Panics inside a plugin are recovered and treated as errors. **Timeouts are enforced at the deadline.** A hook receives a context that ends at `timeout_ms` (and when the client disconnects); GoModel stops waiting at that moment whether or not the hook has returned. A hook that ignores its context keeps running in the background with no effect on the request, so honour `ctx.Done()` in anything that blocks. An abandoned hook counts as a failure. It fails open only when it could not touch what the request continues with: a non-mutating hook runs on its own copy of the exchange, which is discarded. A mutating hook, or an in-flight stream hook, that outlives its deadline always fails the request, even with `fail_mode: open`, because its edits can no longer be trusted. `Init` has a fixed 10 s deadline handled the same way: an `Init` that has not returned by then fails the instance. Keep the default. Switch an instance to `open` only when it is an observer whose absence is acceptable (a header tagger, a warn-only classifier). ## Instance lifecycle GoModel builds one plugin value per guardrail definition and calls `Init` once. Definitions are reloaded from storage periodically (`workflows.refresh_interval`, default 1 minute) and on every admin change; an instance whose type, config, `user_path`, `fail_mode`, and `timeout_ms` are unchanged survives the reload, so state a plugin accumulates (counters, caches, connections) is kept. A replaced or deleted instance is retired and closed later, no sooner than two refresh intervals after retirement and only once nothing holds it: compiled workflows hold their instances until the workflow snapshot drops them, and requests hold theirs for the duration of a phase (a long stream included). A hook is never called after `Close`; a call that reaches a closed instance fails under the instance's `fail_mode` instead. Every instance is closed on shutdown. ## Streaming Bytes already sent to the client cannot be recalled, so a stream plugin picks one of three modes in `StreamHook.StreamPolicy()`: | Mode | What the plugin can do | Latency cost | | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | | `observe` | read every event; decisions are ignored except `terminate` | none | | `transform` | `pass`, `drop`, `replace` (text, reasoning, and tool-call argument deltas), or `terminate` each event | `LookbehindChars` characters of delay per window, plus up to `MinChunkChars` while a run is collected | | `buffer` | nothing per event; GoModel collects the whole stream and runs the plugin's `OnResponse` on the assembled completion (a `buffer` plugin must implement `ResponseHook`; loading fails otherwise) | time-to-first-token becomes time-to-last-token | **Lookbehind** (`transform` only): GoModel withholds the last `LookbehindChars` characters per window and presents each delta together with that tail, so a pattern spanning two chunks (a phone number, an API key) is visible in one event before any of it reaches the client. A choice's text is one window and the arguments of each of its tool calls (`StreamEvent.Call`) another, so a placeholder split across two argument deltas is rewritten the same way. A delta of another kind for the same choice flushes that choice's windows of other kinds first, so text and tool calls keep their order, while parallel tool calls stream independently. A pattern of up to `LookbehindChars + 1` characters is always caught. `0` disables it. The tail comes back already carrying the plugin's earlier edits, and `StreamEvent.Overlap` says how many leading characters it spans: a plugin must only edit matches that extend past `Overlap`, since a match inside it was handled the previous time (the built-in `string_replace` does this, so a rule like `a => aa` is applied once, not on its own output). A match that touches the end of the window may still grow with the next delta, so a plugin can leave it unedited: the tail shows it again next time. `StreamEvent.Final` marks the last event of a window (the stream ended or a delta of another kind flushed it), after which nothing is withheld, so an edit put off that way is due then. `string_replace` does this for matches that fit in its lookbehind, so an open-ended pattern covers the whole key. A chat chunk that carries text together with `finish_reason` or `usage` is re-segmented too; those members go out once, with the chunk's last text, or on a chunk with empty text when the plugin removed that text. **Coalescing** (`transform` only): a hook whose per-call cost is high, such as a classifier or a named-entity detector calling a sidecar, cannot afford a call per token. `MinChunkChars` makes GoModel collect the text deltas of a choice until at least that many new characters are pending and present them as one text event; a non-text event and the end of the stream flush a shorter run. It composes with lookbehind: the withheld tail leads the window and `Overlap` still counts it, so the same edit contract applies. The client waits for up to `MinChunkChars` characters at a time, and the largest value among the transform instances of a stream applies to all of them, capped at 16384 characters. `0`, the default, presents deltas as they arrive. **Chat chunks with several choices** are split into one chunk per choice before plugins see them, so every choice is inspected; provider `usage` stays on the last of them. **Restated text** (Responses API): after a delta was replaced or dropped, the events that repeat a part's full text (`response.output_text.done`, `response.content_part.done`, `response.output_item.done`, and the terminal `response.completed`) are rewritten to the text that was actually emitted, so clients reading the completion instead of the deltas see the same result. **Event size** (`transform` and `observe`): one SSE event larger than 4 MiB cannot be inspected in flight, so the stream ends fail-closed with code `event_too_large` rather than relaying it past the plugins. Comments and keep-alive lines are never affected. **Buffering**: the upstream is drained into a bounded buffer (`MaxBufferBytes`, default 4 MiB; exceeding it fails closed with code `response_too_large`). The buffer is shared by every plugin buffering the stream and by the response chain, so the largest cap asked for applies, and the default whenever the response chain runs or a plugin sets no cap. While draining, GoModel sends the SSE comment `: gomodel-buffering` every 15 s so proxies and clients do not time out; SDKs ignore comment lines. After the response chain runs, the original bytes are replayed unchanged (`allow`, `warn`), a stream is synthesized from the edited completion (`allow` with edits, `respond`), or a single error chunk is sent (`block`). **Mixed chains**: if any stream instance asks for `buffer`, or the workflow has any `response` step, the whole stream is buffered and `transform` instances run over the replay. **Warnings on streams**: when response or stream plugins run, GoModel commits the HTTP headers with the first bytes of the body rather than up front, so a `warn` decided over a buffered response reaches the client as the `X-GoModel-Guardrail` header as long as buffering finished before the first keep-alive comment (15 s). A warn decided later (in `OnStreamEnd` of a `transform` stream, or after a keep-alive went out) cannot change headers already sent; it is still recorded in the audit trail and the logs. **Cutting a stream**: a `terminate` decision in `transform` mode, or a `block`/`respond` from `OnStreamEnd`, ends the stream with `finish_reason: "content_filter"` (`stop_reason: "end_turn"` for Anthropic) followed by `[DONE]`. The client keeps what it already received. If nothing may leak before the decision, the plugin must use `buffer` mode; the built-in `string_replace` and `llm_judge` do exactly that for `block` and `respond`. ## The Exchange Every hook receives one `*pluginapi.Exchange`, the same object in every phase of a request: | Field | Type | Contents | | ---------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `Meta` | `Meta` | read-only request facts: request id, dialect, endpoint, user path, key id, labels, session id, requested and resolved model, provider, workflow version and features, `Stream`, provider `Attempts` (response phases), prompt-cache planning | | `Prompt` | `*Prompt` | the conversation (`Messages` with stable IDs and typed `Parts`: text, image, audio, file, tool call, tool result, reasoning, refusal, opaque), `Tools`, `Params`, raw body. Edit through `SetText`, `SetMedia`, `SetToolArguments`, `SetToolResult`, `Insert`, `Append`, `Remove`, `SetParam`; `TextTargets(roles...)` lists every text part, tool-result text included, and `SetTargetText` writes one back | | `Response` | `*Completion` | the response, `nil` until the response phase: `Choices` with the same `Part` model, `Usage`, raw body. Edit through `SetText`, `ReplaceText`, `SetFinishReason`, `SetToolArguments`; `TextTargets` and `SetTargetText` work as on `Prompt` | | `Stream` | `*StreamState` | text accumulated so far per choice and the event count; `nil` for non-streaming requests | | `Headers` | `*Headers` | `Request` (inbound, credentials redacted; prompt-phase edits apply to the live request for later plugins and request logging, not to the provider call), `Response` (applied to the client response; a header set to the empty string is removed, including ones the gateway adds such as `X-Request-Id`), `Upstream` (recorded, not forwarded in this release) | | `Values` | `Values` | per-request bag shared by every hook of the request; prefix keys with the plugin name | Edits go through methods, not field assignment, so GoModel re-encodes only what changed and untouched messages keep every provider-specific field (`cache_control`, multi-part content, extra fields). Removing a message that carries a tool call whose result is still present (or the reverse) returns a `DanglingToolError` naming the partner message; the request is rejected if the pair stays broken. A plugin that scans or rewrites text does not need to walk parts itself: `Prompt.TextTargets(roles...)` returns one `TextTarget` per text part (plain text, and text inside tool results) with the message ID, role, part indexes, and current text, and `Prompt.SetTargetText(target, text)` writes the edit back. `Completion.TextTargets()` and `Completion.SetTargetText` do the same for the choices of a response. The built-in `string_replace` and `llm_based_altering` are written this way. Media can be edited too. `Part.DecodeMedia()` returns the inline bytes and media type of an image sent as a data URI or of chat input audio, and `Prompt.SetMedia(msgID, partIdx, mediaType, data)` replaces the payload of an image or audio part: the part goes back out as a data URI (images) or base64 input audio, and the other members of the wire part, such as an image's `detail`, are kept. An image referenced by URL has nothing inline to decode; a plugin that wants to redact it fetches it itself. Media inside a tool result is replaced with `SetToolResult`. `Meta.Cache.PlannedPrefixMessages` tells a prompt plugin how many leading messages the provider cache planner will mark as the cached prefix. Appending keeps the cache; editing inside the prefix invalidates it for the session. ## Configuration ### Enabling the plugin system ```yaml theme={null} plugins: enabled: true # PLUGINS_ENABLED; default: false ``` The plugin system is off by default. With it off, nothing plugin-related exists: no built-in or loaded plugin types, no `.so` loading, no guardrails, and no routing-strategy plugins (a virtual model with `strategy: plugin` is rejected). `GET /admin/plugins` and the `/admin/guardrails` endpoints answer `503` with code `feature_unavailable`, and the dashboard hides the Plugins & Guardrails page. Guardrails are plugin instances, so `GUARDRAILS_ENABLED=true` turns the plugin system on implicitly; set `PLUGINS_ENABLED=true` on its own to use routing-strategy plugins or manage guardrail definitions without running them. ### Instances Instances are declared in `guardrails.rules` or created on the dashboard; both land in the same store. See [Guardrails](/advanced/guardrails) for the per-type settings. ```yaml theme={null} guardrails: enabled: true rules: - name: "mask-keys" type: "string_replace" # a built-in, or the manifest name of a loaded plugin phase: "response" # prompt (default) | response | stream order: 10 # step within the phase fail_mode: "open" # closed (default) | open timeout_ms: 500 # 0 = no per-instance timeout (default) config: # validated against the plugin's ConfigSchema mode: "regex" rules: | sk-[A-Za-z0-9]{20,} => [redacted] ``` Rules from `config.yaml` are seeded into the managed default workflow at `order` in `phase`. `type` accepts an optional `plugin:` prefix. ### Loading `.so` files ```yaml theme={null} plugins: search_paths: ["/etc/gomodel/plugins"] # PLUGINS_SEARCH_PATHS, comma-separated; default: empty load: - file: acme_guard.so # relative to a search path, or absolute sha256: "3b1f..." # optional pin; a mismatch fails startup ``` A relative `file` must resolve inside one of the `search_paths` (symlinks are followed and checked). A file that cannot be resolved, verified, or opened is a startup error naming the file. `GUARDRAILS_ENABLED` remains the switch that decides whether configured instances run on traffic. ## Dashboard * **Plugins & Guardrails page**: *Create Guardrail* lists every plugin that implements a `prompt`, `response`, or `stream` hook. The form is generated from the plugin's `ConfigSchema`: `text`, `textarea`, `number`, `select`, `checkboxes`, `bool` (a toggle), `list` (free-text entries, one per line, stored as a string array), `secret` (masked; `GET` returns `********` and sending that literal back keeps the stored value), and `model` (the same searchable model picker as the playground and virtual-model editors, listing the gateway's models; an alias or virtual model is typed in). The **Advanced** section holds *Fail mode* (Default, Closed, Open) and *Timeout (ms)*; they apply to the instance in every workflow that references it. * **Plugins list**: at the bottom of the Plugins & Guardrails page, every loaded plugin type with its title and name (`Header Edit` then `header_edit`, the name being what configuration refers to), version, hooks, source (built-in, registered, or the `.so` path), and health. Guardrail plugins (those whose instances apply a policy to prompts, responses, or streams) carry a shield and are listed first; routing strategies are plugins too, but not guardrails. A plugin that failed to load is listed with its error. The page is shown whenever the plugin system is on. * **Workflows page**: each step has a *Phase* selector; the instance dropdown only offers instances whose plugin implements that phase. ## Shared-object plugins Go's `plugin` package sets the rules, and they are strict: * **Platform**: Linux, macOS, and FreeBSD, and only in a binary built with `CGO_ENABLED=1`. The default GoModel binary and image are static and refuse `.so` files with an error that says so. Use `make build-plugins` (`bin/gomodel-plugins`) or the `gomodel:-plugins` image (`make image-plugins`, built from `Dockerfile.plugins`, glibc runtime). * **Exact toolchain**: the plugin must be built with the **same Go version**, the **same build flags** (`-trimpath`, `-race`, `-tags`), and **identical sources of every shared package**. Only the standard library and `pluginapi` are shared, so internal GoModel changes never affect a plugin, but every GoModel release and every Go toolchain update (patch releases included) needs a rebuild. Make it a CI step. * **No unload**: a `.so` stays loaded until restart. Changing a loaded file takes effect on restart; editing an instance's config re-runs `Init` on a fresh value without a restart. * **Trusted code**: loading a `.so` is equivalent to changing the binary. Keep `search_paths` root-owned and pin `sha256` in production. ### Build and inspect ```sh theme={null} gomodel plugin build ./docs/example_plugins/keywordblock -o plugins/keyword_block.so gomodel plugin inspect plugins/keyword_block.so shasum -a 256 plugins/keyword_block.so # value for plugins.load[].sha256 ``` `plugin build` runs `go build -buildmode=plugin` with the flags recorded in the `gomodel` binary that runs it, forces `CGO_ENABLED=1`, pins `GOTOOLCHAIN` to the host's Go version, stamps a `GoModelBuildInfo` variable into the plugin, and refuses an output whose Go version differs from the host's. Always build with the binary that will load the plugin. `plugin inspect` opens the file and prints its manifest, config schema, and build info. A refused load names both sides: ``` plugin file /app/plugins/x.so was built with a different toolchain, flags, or pluginapi sources: it was built with go1.27.1, gomodel v0.1.90, flags -trimpath; this binary was built with go1.27.1, gomodel v0.1.91, flags (none). Rebuild it with `gomodel plugin build` from this GoModel version ``` In Docker, build the `plugin-builder` target of `Dockerfile.plugins` and run it against your plugin directory: ```sh theme={null} docker build -f Dockerfile.plugins --target plugin-builder -t gomodel-plugin-builder . docker run --rm -v "$PWD/my-plugin:/src" -v "$PWD/plugins:/out" \ gomodel-plugin-builder -o /out/my-plugin.so /src ``` `gomodel --version` prints the Go and `pluginapi` versions of the binary. `docs/example_plugins/README.md` covers building a plugin in its own Go module. ## Writing a plugin A `package main` that exports `func GoModelPlugin() pluginapi.Plugin` is a shared-object plugin; the same type registered with `ext.RegisterPlugin` is a compiled-in one. This one blocks a request when the last user message contains a configured word: ```go theme={null} package main import ( "context" "encoding/json" "strings" "github.com/enterpilot/gomodel/pluginapi" ) type wordBlock struct { words []string enforce pluginapi.Enforcement } func (p *wordBlock) Manifest() pluginapi.Manifest { return pluginapi.Manifest{ Name: "word_block", Version: "1.0.0", Kinds: []pluginapi.Kind{pluginapi.KindPrompt}, Guardrail: true, ConfigSchema: []pluginapi.Field{ { Key: "words", Label: "Blocked words", Input: pluginapi.InputList, Required: true, Help: "One word per line.", }, { Key: "action", Label: "On match", Input: pluginapi.InputSelect, Default: "block", Help: "Block rejects, respond answers with the message, warn records the match.", Options: []pluginapi.Option{{Value: "block", Label: "Block"}, {Value: "respond", Label: "Respond"}, {Value: "warn", Label: "Warn"}}, }, {Key: "message", Label: "Message", Input: pluginapi.InputText, Default: "request blocked by policy", Help: "Error, reply, or audit note."}, pluginapi.BlockStatusField(), }, } } func (p *wordBlock) Init(_ context.Context, raw json.RawMessage, _ pluginapi.Host) error { cfg, err := pluginapi.ParseConfig("word_block", p.Manifest().ConfigSchema, raw) if err != nil { return err } for _, w := range cfg.List("words") { p.words = append(p.words, strings.ToLower(w)) } p.enforce = pluginapi.Enforcement{ Action: pluginapi.Action(cfg.Choice("action", "block", "block", "respond", "warn")), Message: cfg.String("message", "request blocked by policy"), BlockStatus: cfg.BlockStatus("block_status"), } return cfg.Err() } func (p *wordBlock) Close(context.Context) error { return nil } func (p *wordBlock) OnPrompt(_ context.Context, x *pluginapi.Exchange) (pluginapi.Decision, error) { if x.Prompt == nil || x.Prompt.LastUser() == nil { return pluginapi.Allow(), nil } text := strings.ToLower(x.Prompt.LastUser().Text()) for _, w := range p.words { if strings.Contains(text, w) { return p.enforce.Enforce("word_block", map[string]any{"word": w}), nil } } return pluginapi.Allow(), nil } func GoModelPlugin() pluginapi.Plugin { return &wordBlock{} } func main() {} ``` Notes for authors: * `Init` receives the config after validation against `ConfigSchema`: defaults applied, numbers, booleans, and lists coerced, unknown keys rejected, required keys checked. Line-oriented `textarea` fields accept a YAML list of strings as well as a block scalar; `list` fields accept a list or one comma-separated line and arrive as a JSON array; `bool` fields accept `true`/`false` as booleans or text and arrive as a JSON boolean. It is called once per configured instance and again when the instance is edited. * Read the config with `pluginapi.ParseConfig` and its typed readers (`String`, `Choice`, `Bool`, `Int`, `Float`, `OptionalFloat`, `List`, `Lines`, `Roles`, `BlockStatus`) instead of a struct: each reader applies the same coercions the dashboard and config.yaml need, returns its default on a problem, and `Err()` reports the first one with the key named, so a decoder is a straight list of assignments and one check. `RoleOptions()` is the checkbox list `Roles` understands. * Render findings through `pluginapi.Enforcement`: read `action`, `message`, `block_status`, and optionally `respond_text` into it, then `Enforce(code, detail)` gives the block error, the respond completion, or the warning, and `Reject(code, detail)` gives block or respond even when the action is warn (for a finding that must not pass). `BlockStatusField()` is the conventional `block_status` field. The built-in guardrails share this, so their block, respond, and warn behave alike. * Export a **constructor** (`func GoModelPlugin() pluginapi.Plugin`) so one file can back several instances. A `var GoModelPlugin pluginapi.Plugin` works too but limits the file to a single instance. * Set `Mutates: true` when the plugin edits `Prompt`, `Response`, or the stream. Non-mutating plugins may share a step and run concurrently. * Set `Guardrail: true` when the plugin's instances are guardrails: policies applied to prompts, responses, or streams, whether they block, answer, warn, or rewrite. Every built-in traffic plugin sets it (`llm_judge`, `string_replace`, `header_edit`, `system_prompt`, `llm_based_altering`); a routing strategy such as `cheapest_healthy` never touches traffic and leaves it false. The runtime ignores the flag; the dashboard marks guardrail plugins and their instances with a shield and lists them first. * The `Host` passed to `Init` offers `Logger()` (pre-tagged with plugin and instance), `Inference().Complete(...)` (a chat completion through the gateway with origin `plugin`, scoped to `/guardrails/` for budgets and audit), `Metrics()` (names prefixed `plugin__`), and `HTTPClient()` for calls to external services such as a classifier or a PII detector. The client is shared by every instance, uses the gateway's proxy environment and connection pool, and has a 60 s timeout as a backstop; build requests with `http.NewRequestWithContext` so the instance `timeout_ms` bounds each call. `History()` returns an error in this release. * Add `Summarize(config json.RawMessage) string` to render a one-line summary in the guardrails list, and `Normalize(config) (json.RawMessage, error)` to canonicalize a config before it is stored. Both are optional. * Implement `pluginapi.HealthChecker` (`Health(ctx) error`) when an instance depends on something outside the process, such as a classifier sidecar. GoModel probes it off the request path once the instance is built and on every guardrail refresh (`workflows.refresh_interval`, one minute by default), with a 5 s deadline. A failing probe marks the instance `degraded` in `GET /admin/guardrails` and on the dashboard, with the error as the reason, and is logged once per transition; the error text must not contain secrets and is truncated to 256 characters. Health is informational: traffic still runs through the instance under its `fail_mode`. * Field inputs `secret` and `model` exist for API keys of external classifiers and for model pickers; a `secret` value is masked in every admin response. Use `bool` for a switch and `list` for an open-ended list of strings (entity types, an allow list) instead of encoding them in a `select` or a `textarea`. * Add `Scope: pluginapi.ScopeRoute` to fields that belong to a virtual model rather than to the instance (routing strategies only). `docs/example_plugins/keywordblock/main.go` is a fuller, commented example with `prompt` and `response` hooks, and the built-ins under `internal/plugins/builtin/` are reference implementations of every hook kind. ### Testing a plugin `pluginapi/plugintest` tests a plugin without GoModel: * `plugintest.NewHost(replies...)` is a fake `Host`: scripted inference replies (or a `Reply` function, or an `Err`), recorded `Requests()`, recorded `Recorded()` metrics, and a settable HTTP client for a fake sidecar started with `httptest`. * `plugintest.Init`, `Text`, `Prompt`, `Completion`, and `Exchange` build the fixtures a hook call needs, with clean change tracking and a Values bag. * `plugintest.RunStream(ctx, hook, x, events)` drives a `StreamHook` the way the host does under its `StreamPolicy`: chunk coalescing, the withheld lookbehind tail shown again with `Overlap`, pass, replace, drop, and terminate applied to the whole window, and the buffered mode assembling the completion for `OnResponse`. The result holds the text a client would have received per choice, the delivered events, the terminate decision, and the end decision. From the `string_replace` tests, where `New` is that plugin's constructor: ```go theme={null} p := plugintest.Init(t, New, `{"rules": "secret => [x]", "stream_lookbehind": 4}`, nil) res, err := plugintest.RunStream(ctx, p.(pluginapi.StreamHook), plugintest.Exchange(nil, nil), []*pluginapi.StreamEvent{ plugintest.TextDelta("my se"), plugintest.TextDelta("cret is safe"), }) // res.Text[0] == "my [x] is safe": the match split across two deltas was rewritten once. ``` Buffered mode assembles the completion from the text and reasoning deltas alone; set `x.Response` yourself before calling `RunStream` when the hook must see tool calls, usage, or a finish reason. ## Routing strategy plugins A plugin that implements `pluginapi.RouteStrategy` is a load-balancing strategy for [virtual models](/features/virtual-models#load-balance-across-models): ```go theme={null} type RouteStrategy interface { Select(ctx context.Context, req RouteRequest) (RouteChoice, error) OnAttemptEnd(outcome RouteOutcome) } ``` `RouteRequest` carries the virtual model `Source`, the `Candidates` in configured order (provider, model, `provider/model`, weight, and prices per million tokens when known), the session id and sticky `SessionTarget`, `Meta`, and the virtual model's `strategy_config` as JSON (`Prompt` is `nil` in this release). `Select` returns the chosen `Qualified` target and an optional `Reason` (debug logs only); `OnAttemptEnd` reports success, status, latency, and timeouts so the strategy can adapt. Failover chains, capacity probes, and session pinning stay GoModel's job. Select the strategy on the virtual model: ```yaml theme={null} virtual_models: - source: smart-router strategy: plugin # round_robin | cost | failover | adaptive | plugin strategy_plugin: cheapest_healthy # plugin name; required with strategy: plugin strategy_config: # the plugin's Scope: route fields; defaults apply when omitted prefer: cheapest # cheapest | fastest max_error_rate: 0.2 targets: - openai/gpt-4o-mini - anthropic/claude-haiku-4-5 ``` * `strategy_config` is validated against the plugin's route-scoped fields (`route_fields` in `GET /admin/plugins`). `strategy: plugin` without `strategy_plugin` is a validation error, and `PUT /admin/virtual-models` rejects a name that is not a loaded plugin with a `route` hook. * Target `weight` is ignored under the plugin strategy. * Instance-scoped fields of a route plugin (an endpoint, an API key) come from a guardrail definition whose **name and type both equal the plugin name**; without one the plugin is initialized with `{}`. Instance-scoped `secret` values currently reach a route plugin redacted. Saving that definition rebuilds the instance; the previous one is closed once its in-flight `Select` calls return. * GoModel falls back to weighted round robin, logging one warning per virtual model, when the plugin is missing, is not a route plugin, failed to initialize, `strategy_config` is invalid, or `Select` errors, panics, takes longer than 250 ms, or returns a target outside the viable pool. On the dashboard, the virtual model editor's strategy dropdown lists one `plugin:` entry per loaded route plugin (`VIRTUAL_MODEL_STRATEGIES` in the runtime config is `round_robin,cost,failover[,adaptive],plugin:,...`); choosing it renders the plugin's route fields under the targets table. `cheapest_healthy` is the built-in reference strategy (`internal/plugins/builtin/routeexample`). It keeps the last 50 outcomes per target and picks the cheapest (`prefer: cheapest`, the default) or the lowest-median-latency (`prefer: fastest`) target whose error rate is at or below `max_error_rate` (default `0.2`), keeps a healthy `SessionTarget`, and considers every candidate when none is healthy. ## Admin endpoints All endpoints require the same admin credentials as the rest of `/admin`. ### GET /admin/plugins Every plugin type known to the gateway, including ones that failed to load: ```json theme={null} [ { "name": "string_replace", "label": "String Replace", "version": "1.0.0", "description": "Replaces, flags, or blocks text matching literal or regex rules in prompts, responses, and streams.", "kinds": ["prompt", "response", "stream"], "mutates": true, "guardrail": true, "source": "builtin", "fields": [{ "key": "rules", "label": "Rules", "input": "textarea", "required": true }], "route_fields": [], "health": "ok", "error": "" } ] ``` `source` is `builtin`, `registered` (compiled in through `ext`), or the absolute path of the `.so`. `fields` are the instance-scoped schema fields, `route_fields` the route-scoped ones. A `.so` built with `gomodel plugin build` also reports `built_with` (Go and `pluginapi` versions). ### GET /admin/guardrails/types One entry per plugin that implements a `prompt`, `response`, or `stream` hook, in the shape the guardrail editor renders: ```json theme={null} { "type": "llm_judge", "label": "LLM Judge", "description": "Asks a judge model whether prompts and responses violate a policy, then blocks, answers, or flags them.", "defaults": { "action": "block", "max_tokens": 256, "target": "auto" }, "fields": [{ "key": "model", "label": "Judge model", "input": "model", "required": true }], "phases": ["prompt", "response", "stream"], "source": "builtin", "mutates": false, "guardrail": true } ``` `phases`, `source`, `mutates`, and `guardrail` are new (instance views from `GET /admin/guardrails` carry `guardrail` as well); `fields[].input` may now be `secret`, `model`, `bool`, or `list`, and `fields[].scope` is `""` (instance) or `"route"`. ### GET, PUT, DELETE /admin/guardrails Definitions gain `fail_mode` (`closed` | `open`, empty for the phase default) and `timeout_ms` (`0` for none); views additionally carry `phases` (from the plugin manifest), `summary`, and `health` (`ok` | `degraded`, with `health_error` and `health_checked_at` for plugins that implement a health probe). `PUT` accepts both new fields. A stored `secret` value is returned as `********`; send it back unchanged to keep it, or `""` to clear it. ### GET /admin/workflows/guardrails The instances available to workflow steps, with the phases each supports: ```json theme={null} [{ "name": "pii-redact", "type": "llm_based_altering", "phases": ["prompt", "response"], "summary": "openai/gpt-4o-mini • user • default prompt" }] ``` ### Workflow payload version 2 `POST /admin/workflows` accepts `schema_version: 2` with `steps[]` carrying a `phase`; version 1 payloads (`guardrails: [{ref, step}]`) stay valid and are returned unchanged. Workflow views expose the per-phase chain hashes as `chain_hashes`. See [Workflows](/advanced/workflows#guardrail-steps).