Plugins

Overview

A plugin is a Go value that implements pluginapi.Plugin plus one or more optional hook interfaces. Every guardrail is an instance of a plugin, and so is a custom routing strategy for a virtual model. There is one contract, in the github.com/enterpilot/gomodel/pluginapi package, and three ways to ship a plugin:

How	When to use it	Registered by
Built in	Ships with GoModel: `system_prompt`, `llm_based_altering`, `string_replace`, `header_edit`, `llm_judge`, and the `cheapest_healthy` routing strategy	nothing to do
Compiled in	You build your own binary from the `run` package	`ext.RegisterPlugin(func() pluginapi.Plugin { ... })` before `run.Run`
Shared object (`.so`)	You run the stock binary and want to add code at startup	`plugins.load` in `config.yaml` (needs the cgo build, see Shared-object plugins)

Whichever way it arrives, a plugin type shows up under its manifest name on the dashboard’s Guardrails page, in GET /admin/plugins, and as a type in guardrails.rules. pluginapi depends on the standard library only. A plugin imports it and nothing else from GoModel.

Phases

A plugin declares which hooks it implements in Manifest.Kinds. GoModel runs the content phases from the matched workflow:

Phase	Hook	Runs	Sees
`prompt`	`PromptHook.OnPrompt`	after routing, before the provider call	`Prompt`, resolved provider and model in `Meta`
`response`	`ResponseHook.OnResponse`	on a complete non-streaming response, or on the assembled response of a buffered stream	`Prompt`, `Response`, `Meta.Attempts`
`stream`	`StreamHook.OnStreamEvent` / `OnStreamEnd`	per streamed event, then once at the end	`Prompt`, `Stream` (text so far), each event

Which instance runs in which phase, and in what order, is a workflow decision (steps[].phase and steps[].step). The same instance may appear once per phase. Two more hook kinds exist in the contract but are not run by this release: request (before model resolution) and complete (after the client response is written). A manifest may declare them; GoModel validates that the interfaces are implemented and ignores them at runtime. The route kind is used by virtual models, see Routing strategy plugins. Phases apply to /v1/chat/completions, /v1/responses, and /v1/messages. Anthropic requests are translated to the chat form before any hook runs, so a plugin sees one shape; Meta.Dialect reports anthropic_messages when it matters. Inline /v1/batches items go through the prompt phase when guardrails.enable_for_batch_processing is on.

Chain execution

Within one phase, instances are grouped by step:

Steps run in ascending order. A later step sees the edits of an earlier one.
Within a step, instances whose manifest says Mutates: false run concurrently on a shallow copy of the exchange (their Values and response headers are merged back). The step may hold at most one instance with Mutates: true, which runs after the readers. Two mutating instances at the same step is a configuration error.
Decisions of a step merge by severity: block > respond > warn > allow. The first blocking decision ends the chain after its step.

Decisions

Every hook returns a pluginapi.Decision:

Action	Effect	Defaults
`allow`	continue with the edits already made to the exchange	zero value of `Decision`
`block`	reject with `Status`, `Code`, and `Message` in the endpoint’s native error format	`Status` 400 in the prompt phase, 502 in the response and stream phases; `Code` `guardrail_blocked`
`respond`	answer the request with `Response` as an ordinary assistant turn, HTTP 200 (a synthesized one-turn stream for streaming requests); in the response and stream phases the provider’s `usage` is kept on the reply, since the inference was already performed	`pluginapi.Respond(text)` builds a one-choice completion
`warn`	continue; record `Code`, `Message`, and `Detail` in the audit trail and add `X-GoModel-Guardrail: warn; code=<code>` to the client response	`Code` `guardrail_warning`

respond is the “block with a safe message” most guardrail products offer: agent loops keep running instead of surfacing a 4xx. Use block when the caller should see an error. Detail is stored in the audit record and must not contain secrets. NoStore on any decision keeps the response out of the response cache, exact and semantic, so the next request with the same or a similar body runs the plugins again instead of replaying a stored answer. Set it when the reply carries request-specific data a plugin restores on the way out, for example a de-anonymizer that puts the caller’s PII back into the assistant text: a cached copy would hand that text to whoever sends a matching prompt next. It is honoured from any phase and any instance of the request, whatever the merged decision was.

Audit trail

Every instance that ran leaves an outcome in the audit entry’s data.guardrails list, in execution order across the prompt, response and stream phases, silent allows included:

Field	Meaning
`phase`, `step`, `instance`, `type`	Where the instance ran: the workflow phase and step, the instance name (the workflow’s `ref`) and its plugin type.
`action`	`allow`, `warn`, `block`, `respond`, or `failure` when the hook errored instead of deciding.
`code`, `message`, `detail`	The decision’s fields. The dashboard list and live views omit `detail`; the entry detail carries it.
`error`, `fail_mode`	With `failure`: the error text and whether the chain carried on (`open`) or the request failed (`closed`).
`edited`, `target`	Whether the instance changed the `request` (prompt phase) or the `response` (response and stream phases).
`replaced_events`, `dropped_events`	For an in-flight stream instance, how many events it rewrote or withheld.
`duration_ns`	The hook’s run time.

A configured step without an outcome did not run: an earlier block or answer ended the request, or a cache hit skipped the model call. The dashboard colors the workflow chart of an audit entry from this list, and its “Guardrails” tab lists the outcomes. The list is the decision trail; the request revisions below are the body trail. Prompt-phase decisions are also appended to the request’s revision chain (the same place request rewrites are recorded), one entry per instance in step order. An instance that edited the prompt is a changed revision carrying the request as it stood right after that step, sizes included, so a chain of edits reads step by step and the last one is what was forwarded; the request bodies are stored under the same LOGGING_LOG_BODIES and LOGGING_LOG_REVISION_BODIES gates as request rewrites. Each editing step leaves only a copy of the prompt behind on the request path; applying those copies to the request and encoding the bodies runs afterwards, alongside the provider call. LOGGING_LOG_GUARDRAIL_STEPS=false skips them and records a chain’s edits as one revision carrying the request as forwarded. A warning, block, answer or failure without an edit is a no-change entry carrying the decision; a silent allow leaves no revision. Response and stream decisions are recorded in data.guardrails only, and the objections among them are logged with the request id.

Fail modes and timeouts

Each instance has a fail_mode and a timeout_ms:

Setting	Values	Default
`fail_mode`	`closed`: an error, panic, or timeout rejects the request with HTTP 500 and code `plugin_failure`; `open`: log and continue as if the instance allowed	`closed` for `prompt`, `response`, and `stream`
`timeout_ms`	upper bound for every hook call of the instance	`0` (no per-instance timeout)

The instance name of a fail-closed failure goes to the logs and the audit record, never to the client. Panics inside a plugin are recovered and treated as errors. Timeouts are enforced at the deadline. A hook receives a context that ends at timeout_ms (and when the client disconnects); GoModel stops waiting at that moment whether or not the hook has returned. A hook that ignores its context keeps running in the background with no effect on the request, so honour ctx.Done() in anything that blocks. An abandoned hook counts as a failure. It fails open only when it could not touch what the request continues with: a non-mutating hook runs on its own copy of the exchange, which is discarded. A mutating hook, or an in-flight stream hook, that outlives its deadline always fails the request, even with fail_mode: open, because its edits can no longer be trusted. Init has a fixed 10 s deadline handled the same way: an Init that has not returned by then fails the instance. Keep the default. Switch an instance to open only when it is an observer whose absence is acceptable (a header tagger, a warn-only classifier).

Instance lifecycle

GoModel builds one plugin value per guardrail definition and calls Init once. Definitions are reloaded from storage periodically (workflows.refresh_interval, default 1 minute) and on every admin change; an instance whose type, config, user_path, fail_mode, and timeout_ms are unchanged survives the reload, so state a plugin accumulates (counters, caches, connections) is kept. A replaced or deleted instance is retired and closed later, no sooner than two refresh intervals after retirement and only once nothing holds it: compiled workflows hold their instances until the workflow snapshot drops them, and requests hold theirs for the duration of a phase (a long stream included). A hook is never called after Close; a call that reaches a closed instance fails under the instance’s fail_mode instead. Every instance is closed on shutdown.

Streaming

Bytes already sent to the client cannot be recalled, so a stream plugin picks one of three modes in StreamHook.StreamPolicy():

Mode	What the plugin can do	Latency cost
`observe`	read every event; decisions are ignored except `terminate`	none
`transform`	`pass`, `drop`, `replace` (text, reasoning, and tool-call argument deltas), or `terminate` each event	`LookbehindChars` characters of delay per window, plus up to `MinChunkChars` while a run is collected
`buffer`	nothing per event; GoModel collects the whole stream and runs the plugin’s `OnResponse` on the assembled completion (a `buffer` plugin must implement `ResponseHook`; loading fails otherwise)	time-to-first-token becomes time-to-last-token

Lookbehind (transform only): GoModel withholds the last LookbehindChars characters per window and presents each delta together with that tail, so a pattern spanning two chunks (a phone number, an API key) is visible in one event before any of it reaches the client. A choice’s text is one window and the arguments of each of its tool calls (StreamEvent.Call) another, so a placeholder split across two argument deltas is rewritten the same way. A delta of another kind for the same choice flushes that choice’s windows of other kinds first, so text and tool calls keep their order, while parallel tool calls stream independently. A pattern of up to LookbehindChars + 1 characters is always caught. 0 disables it. The tail comes back already carrying the plugin’s earlier edits, and StreamEvent.Overlap says how many leading characters it spans: a plugin must only edit matches that extend past Overlap, since a match inside it was handled the previous time (the built-in string_replace does this, so a rule like a => aa is applied once, not on its own output). A match that touches the end of the window may still grow with the next delta, so a plugin can leave it unedited: the tail shows it again next time. StreamEvent.Final marks the last event of a window (the stream ended or a delta of another kind flushed it), after which nothing is withheld, so an edit put off that way is due then. string_replace does this for matches that fit in its lookbehind, so an open-ended pattern covers the whole key. A chat chunk that carries text together with finish_reason or usage is re-segmented too; those members go out once, with the chunk’s last text, or on a chunk with empty text when the plugin removed that text. Coalescing (transform only): a hook whose per-call cost is high, such as a classifier or a named-entity detector calling a sidecar, cannot afford a call per token. MinChunkChars makes GoModel collect the text deltas of a choice until at least that many new characters are pending and present them as one text event; a non-text event and the end of the stream flush a shorter run. It composes with lookbehind: the withheld tail leads the window and Overlap still counts it, so the same edit contract applies. The client waits for up to MinChunkChars characters at a time, and the largest value among the transform instances of a stream applies to all of them, capped at 16384 characters. 0, the default, presents deltas as they arrive. Chat chunks with several choices are split into one chunk per choice before plugins see them, so every choice is inspected; provider usage stays on the last of them. Restated text (Responses API): after a delta was replaced or dropped, the events that repeat a part’s full text (response.output_text.done, response.content_part.done, response.output_item.done, and the terminal response.completed) are rewritten to the text that was actually emitted, so clients reading the completion instead of the deltas see the same result. Event size (transform and observe): one SSE event larger than 4 MiB cannot be inspected in flight, so the stream ends fail-closed with code event_too_large rather than relaying it past the plugins. Comments and keep-alive lines are never affected. Buffering: the upstream is drained into a bounded buffer (MaxBufferBytes, default 4 MiB; exceeding it fails closed with code response_too_large). The buffer is shared by every plugin buffering the stream and by the response chain, so the largest cap asked for applies, and the default whenever the response chain runs or a plugin sets no cap. While draining, GoModel sends the SSE comment : gomodel-buffering every 15 s so proxies and clients do not time out; SDKs ignore comment lines. After the response chain runs, the original bytes are replayed unchanged (allow, warn), a stream is synthesized from the edited completion (allow with edits, respond), or a single error chunk is sent (block). Mixed chains: if any stream instance asks for buffer, or the workflow has any response step, the whole stream is buffered and transform instances run over the replay. Warnings on streams: when response or stream plugins run, GoModel commits the HTTP headers with the first bytes of the body rather than up front, so a warn decided over a buffered response reaches the client as the X-GoModel-Guardrail header as long as buffering finished before the first keep-alive comment (15 s). A warn decided later (in OnStreamEnd of a transform stream, or after a keep-alive went out) cannot change headers already sent; it is still recorded in the audit trail and the logs. Cutting a stream: a terminate decision in transform mode, or a block/respond from OnStreamEnd, ends the stream with finish_reason: "content_filter" (stop_reason: "end_turn" for Anthropic) followed by [DONE]. The client keeps what it already received. If nothing may leak before the decision, the plugin must use buffer mode; the built-in string_replace and llm_judge do exactly that for block and respond.

The Exchange

Every hook receives one *pluginapi.Exchange, the same object in every phase of a request:

Field	Type	Contents
`Meta`	`Meta`	read-only request facts: request id, dialect, endpoint, user path, key id, labels, session id, requested and resolved model, provider, workflow version and features, `Stream`, provider `Attempts` (response phases), prompt-cache planning
`Prompt`	`*Prompt`	the conversation (`Messages` with stable IDs and typed `Parts`: text, image, audio, file, tool call, tool result, reasoning, refusal, opaque), `Tools`, `Params`, raw body. Edit through `SetText`, `SetMedia`, `SetToolArguments`, `SetToolResult`, `Insert`, `Append`, `Remove`, `SetParam`; `TextTargets(roles...)` lists every text part, tool-result text included, and `SetTargetText` writes one back
`Response`	`*Completion`	the response, `nil` until the response phase: `Choices` with the same `Part` model, `Usage`, raw body. Edit through `SetText`, `ReplaceText`, `SetFinishReason`, `SetToolArguments`; `TextTargets` and `SetTargetText` work as on `Prompt`
`Stream`	`*StreamState`	text accumulated so far per choice and the event count; `nil` for non-streaming requests
`Headers`	`*Headers`	`Request` (inbound, credentials redacted; prompt-phase edits apply to the live request for later plugins and request logging, not to the provider call), `Response` (applied to the client response; a header set to the empty string is removed, including ones the gateway adds such as `X-Request-Id`), `Upstream` (recorded, not forwarded in this release)
`Values`	`Values`	per-request bag shared by every hook of the request; prefix keys with the plugin name

Edits go through methods, not field assignment, so GoModel re-encodes only what changed and untouched messages keep every provider-specific field (cache_control, multi-part content, extra fields). Removing a message that carries a tool call whose result is still present (or the reverse) returns a DanglingToolError naming the partner message; the request is rejected if the pair stays broken. A plugin that scans or rewrites text does not need to walk parts itself: Prompt.TextTargets(roles...) returns one TextTarget per text part (plain text, and text inside tool results) with the message ID, role, part indexes, and current text, and Prompt.SetTargetText(target, text) writes the edit back. Completion.TextTargets() and Completion.SetTargetText do the same for the choices of a response. The built-in string_replace and llm_based_altering are written this way. Media can be edited too. Part.DecodeMedia() returns the inline bytes and media type of an image sent as a data URI or of chat input audio, and Prompt.SetMedia(msgID, partIdx, mediaType, data) replaces the payload of an image or audio part: the part goes back out as a data URI (images) or base64 input audio, and the other members of the wire part, such as an image’s detail, are kept. An image referenced by URL has nothing inline to decode; a plugin that wants to redact it fetches it itself. Media inside a tool result is replaced with SetToolResult. Meta.Cache.PlannedPrefixMessages tells a prompt plugin how many leading messages the provider cache planner will mark as the cached prefix. Appending keeps the cache; editing inside the prefix invalidates it for the session.

Configuration

Enabling the plugin system

plugins:
  enabled: true    # PLUGINS_ENABLED; default: false

The plugin system is off by default. With it off, nothing plugin-related exists: no built-in or loaded plugin types, no .so loading, no guardrails, and no routing-strategy plugins (a virtual model with strategy: plugin is rejected). GET /admin/plugins and the /admin/guardrails endpoints answer 503 with code feature_unavailable, and the dashboard hides the Plugins & Guardrails page. Guardrails are plugin instances, so GUARDRAILS_ENABLED=true turns the plugin system on implicitly; set PLUGINS_ENABLED=true on its own to use routing-strategy plugins or manage guardrail definitions without running them.

Instances

Instances are declared in guardrails.rules or created on the dashboard; both land in the same store. See Guardrails for the per-type settings.

guardrails:
  enabled: true
  rules:
    - name: "mask-keys"
      type: "string_replace"      # a built-in, or the manifest name of a loaded plugin
      phase: "response"           # prompt (default) | response | stream
      order: 10                   # step within the phase
      fail_mode: "open"           # closed (default) | open
      timeout_ms: 500             # 0 = no per-instance timeout (default)
      config:                     # validated against the plugin's ConfigSchema
        mode: "regex"
        rules: |
          sk-[A-Za-z0-9]{20,} => [redacted]

Rules from config.yaml are seeded into the managed default workflow at order in phase. type accepts an optional plugin: prefix.

Loading `.so` files

plugins:
  search_paths: ["/etc/gomodel/plugins"]   # PLUGINS_SEARCH_PATHS, comma-separated; default: empty
  load:
    - file: acme_guard.so                  # relative to a search path, or absolute
      sha256: "3b1f..."                    # optional pin; a mismatch fails startup

A relative file must resolve inside one of the search_paths (symlinks are followed and checked). A file that cannot be resolved, verified, or opened is a startup error naming the file. GUARDRAILS_ENABLED remains the switch that decides whether configured instances run on traffic.

Dashboard

Plugins & Guardrails page: Create Guardrail lists every plugin that implements a prompt, response, or stream hook. The form is generated from the plugin’s ConfigSchema: text, textarea, number, select, checkboxes, bool (a toggle), list (free-text entries, one per line, stored as a string array), secret (masked; GET returns ******** and sending that literal back keeps the stored value), and model (the same searchable model picker as the playground and virtual-model editors, listing the gateway’s models; an alias or virtual model is typed in). The Advanced section holds Fail mode (Default, Closed, Open) and Timeout (ms); they apply to the instance in every workflow that references it.
Plugins list: at the bottom of the Plugins & Guardrails page, every loaded plugin type with its title and name (Header Edit then header_edit, the name being what configuration refers to), version, hooks, source (built-in, registered, or the .so path), and health. Guardrail plugins (those whose instances apply a policy to prompts, responses, or streams) carry a shield and are listed first; routing strategies are plugins too, but not guardrails. A plugin that failed to load is listed with its error. The page is shown whenever the plugin system is on.
Workflows page: each step has a Phase selector; the instance dropdown only offers instances whose plugin implements that phase.

Shared-object plugins

Go’s plugin package sets the rules, and they are strict:

Platform: Linux, macOS, and FreeBSD, and only in a binary built with CGO_ENABLED=1. The default GoModel binary and image are static and refuse .so files with an error that says so. Use make build-plugins (bin/gomodel-plugins) or the gomodel:<version>-plugins image (make image-plugins, built from Dockerfile.plugins, glibc runtime).
Exact toolchain: the plugin must be built with the same Go version, the same build flags (-trimpath, -race, -tags), and identical sources of every shared package. Only the standard library and pluginapi are shared, so internal GoModel changes never affect a plugin, but every GoModel release and every Go toolchain update (patch releases included) needs a rebuild. Make it a CI step.
No unload: a .so stays loaded until restart. Changing a loaded file takes effect on restart; editing an instance’s config re-runs Init on a fresh value without a restart.
Trusted code: loading a .so is equivalent to changing the binary. Keep search_paths root-owned and pin sha256 in production.

Build and inspect

gomodel plugin build ./docs/example_plugins/keywordblock -o plugins/keyword_block.so
gomodel plugin inspect plugins/keyword_block.so
shasum -a 256 plugins/keyword_block.so     # value for plugins.load[].sha256

plugin build runs go build -buildmode=plugin with the flags recorded in the gomodel binary that runs it, forces CGO_ENABLED=1, pins GOTOOLCHAIN to the host’s Go version, stamps a GoModelBuildInfo variable into the plugin, and refuses an output whose Go version differs from the host’s. Always build with the binary that will load the plugin. plugin inspect opens the file and prints its manifest, config schema, and build info. A refused load names both sides:

plugin file /app/plugins/x.so was built with a different toolchain, flags, or
pluginapi sources: it was built with go1.27.1, gomodel v0.1.90, flags -trimpath;
this binary was built with go1.27.1, gomodel v0.1.91, flags (none). Rebuild it
with `gomodel plugin build` from this GoModel version

In Docker, build the plugin-builder target of Dockerfile.plugins and run it against your plugin directory:

docker build -f Dockerfile.plugins --target plugin-builder -t gomodel-plugin-builder .
docker run --rm -v "$PWD/my-plugin:/src" -v "$PWD/plugins:/out" \
  gomodel-plugin-builder -o /out/my-plugin.so /src

gomodel --version prints the Go and pluginapi versions of the binary. docs/example_plugins/README.md covers building a plugin in its own Go module.

Writing a plugin

A package main that exports func GoModelPlugin() pluginapi.Plugin is a shared-object plugin; the same type registered with ext.RegisterPlugin is a compiled-in one. This one blocks a request when the last user message contains a configured word:

package main

import (
	"context"
	"encoding/json"
	"strings"

	"github.com/enterpilot/gomodel/pluginapi"
)

type wordBlock struct {
	words   []string
	enforce pluginapi.Enforcement
}

func (p *wordBlock) Manifest() pluginapi.Manifest {
	return pluginapi.Manifest{
		Name:      "word_block",
		Version:   "1.0.0",
		Kinds:     []pluginapi.Kind{pluginapi.KindPrompt},
		Guardrail: true,
		ConfigSchema: []pluginapi.Field{
			{
				Key: "words", Label: "Blocked words", Input: pluginapi.InputList,
				Required: true, Help: "One word per line.",
			},
			{
				Key: "action", Label: "On match", Input: pluginapi.InputSelect, Default: "block",
				Help: "Block rejects, respond answers with the message, warn records the match.",
				Options: []pluginapi.Option{{Value: "block", Label: "Block"}, {Value: "respond", Label: "Respond"}, {Value: "warn", Label: "Warn"}},
			},
			{Key: "message", Label: "Message", Input: pluginapi.InputText, Default: "request blocked by policy", Help: "Error, reply, or audit note."},
			pluginapi.BlockStatusField(),
		},
	}
}

func (p *wordBlock) Init(_ context.Context, raw json.RawMessage, _ pluginapi.Host) error {
	cfg, err := pluginapi.ParseConfig("word_block", p.Manifest().ConfigSchema, raw)
	if err != nil {
		return err
	}
	for _, w := range cfg.List("words") {
		p.words = append(p.words, strings.ToLower(w))
	}
	p.enforce = pluginapi.Enforcement{
		Action:      pluginapi.Action(cfg.Choice("action", "block", "block", "respond", "warn")),
		Message:     cfg.String("message", "request blocked by policy"),
		BlockStatus: cfg.BlockStatus("block_status"),
	}
	return cfg.Err()
}

func (p *wordBlock) Close(context.Context) error { return nil }

func (p *wordBlock) OnPrompt(_ context.Context, x *pluginapi.Exchange) (pluginapi.Decision, error) {
	if x.Prompt == nil || x.Prompt.LastUser() == nil {
		return pluginapi.Allow(), nil
	}
	text := strings.ToLower(x.Prompt.LastUser().Text())
	for _, w := range p.words {
		if strings.Contains(text, w) {
			return p.enforce.Enforce("word_block", map[string]any{"word": w}), nil
		}
	}
	return pluginapi.Allow(), nil
}

func GoModelPlugin() pluginapi.Plugin { return &wordBlock{} }

func main() {}

Notes for authors:

Init receives the config after validation against ConfigSchema: defaults applied, numbers, booleans, and lists coerced, unknown keys rejected, required keys checked. Line-oriented textarea fields accept a YAML list of strings as well as a block scalar; list fields accept a list or one comma-separated line and arrive as a JSON array; bool fields accept true/false as booleans or text and arrive as a JSON boolean. It is called once per configured instance and again when the instance is edited.
Read the config with pluginapi.ParseConfig and its typed readers (String, Choice, Bool, Int, Float, OptionalFloat, List, Lines, Roles, BlockStatus) instead of a struct: each reader applies the same coercions the dashboard and config.yaml need, returns its default on a problem, and Err() reports the first one with the key named, so a decoder is a straight list of assignments and one check. RoleOptions() is the checkbox list Roles understands.
Render findings through pluginapi.Enforcement: read action, message, block_status, and optionally respond_text into it, then Enforce(code, detail) gives the block error, the respond completion, or the warning, and Reject(code, detail) gives block or respond even when the action is warn (for a finding that must not pass). BlockStatusField() is the conventional block_status field. The built-in guardrails share this, so their block, respond, and warn behave alike.
Export a constructor (func GoModelPlugin() pluginapi.Plugin) so one file can back several instances. A var GoModelPlugin pluginapi.Plugin works too but limits the file to a single instance.
Set Mutates: true when the plugin edits Prompt, Response, or the stream. Non-mutating plugins may share a step and run concurrently.
Set Guardrail: true when the plugin’s instances are guardrails: policies applied to prompts, responses, or streams, whether they block, answer, warn, or rewrite. Every built-in traffic plugin sets it (llm_judge, string_replace, header_edit, system_prompt, llm_based_altering); a routing strategy such as cheapest_healthy never touches traffic and leaves it false. The runtime ignores the flag; the dashboard marks guardrail plugins and their instances with a shield and lists them first.
The Host passed to Init offers Logger() (pre-tagged with plugin and instance), Inference().Complete(...) (a chat completion through the gateway with origin plugin, scoped to <user path>/guardrails/<instance> for budgets and audit), Metrics() (names prefixed plugin_<name>_), and HTTPClient() for calls to external services such as a classifier or a PII detector. The client is shared by every instance, uses the gateway’s proxy environment and connection pool, and has a 60 s timeout as a backstop; build requests with http.NewRequestWithContext so the instance timeout_ms bounds each call. History() returns an error in this release.
Add Summarize(config json.RawMessage) string to render a one-line summary in the guardrails list, and Normalize(config) (json.RawMessage, error) to canonicalize a config before it is stored. Both are optional.
Implement pluginapi.HealthChecker (Health(ctx) error) when an instance depends on something outside the process, such as a classifier sidecar. GoModel probes it off the request path once the instance is built and on every guardrail refresh (workflows.refresh_interval, one minute by default), with a 5 s deadline. A failing probe marks the instance degraded in GET /admin/guardrails and on the dashboard, with the error as the reason, and is logged once per transition; the error text must not contain secrets and is truncated to 256 characters. Health is informational: traffic still runs through the instance under its fail_mode.
Field inputs secret and model exist for API keys of external classifiers and for model pickers; a secret value is masked in every admin response. Use bool for a switch and list for an open-ended list of strings (entity types, an allow list) instead of encoding them in a select or a textarea.
Add Scope: pluginapi.ScopeRoute to fields that belong to a virtual model rather than to the instance (routing strategies only).

docs/example_plugins/keywordblock/main.go is a fuller, commented example with prompt and response hooks, and the built-ins under internal/plugins/builtin/ are reference implementations of every hook kind.

Testing a plugin

pluginapi/plugintest tests a plugin without GoModel:

plugintest.NewHost(replies...) is a fake Host: scripted inference replies (or a Reply function, or an Err), recorded Requests(), recorded Recorded() metrics, and a settable HTTP client for a fake sidecar started with httptest.
plugintest.Init, Text, Prompt, Completion, and Exchange build the fixtures a hook call needs, with clean change tracking and a Values bag.
plugintest.RunStream(ctx, hook, x, events) drives a StreamHook the way the host does under its StreamPolicy: chunk coalescing, the withheld lookbehind tail shown again with Overlap, pass, replace, drop, and terminate applied to the whole window, and the buffered mode assembling the completion for OnResponse. The result holds the text a client would have received per choice, the delivered events, the terminate decision, and the end decision.

From the string_replace tests, where New is that plugin’s constructor:

p := plugintest.Init(t, New, `{"rules": "secret => [x]", "stream_lookbehind": 4}`, nil)
res, err := plugintest.RunStream(ctx, p.(pluginapi.StreamHook), plugintest.Exchange(nil, nil), []*pluginapi.StreamEvent{
	plugintest.TextDelta("my se"), plugintest.TextDelta("cret is safe"),
})
// res.Text[0] == "my [x] is safe": the match split across two deltas was rewritten once.

Buffered mode assembles the completion from the text and reasoning deltas alone; set x.Response yourself before calling RunStream when the hook must see tool calls, usage, or a finish reason.

Routing strategy plugins

A plugin that implements pluginapi.RouteStrategy is a load-balancing strategy for virtual models:

type RouteStrategy interface {
	Select(ctx context.Context, req RouteRequest) (RouteChoice, error)
	OnAttemptEnd(outcome RouteOutcome)
}

RouteRequest carries the virtual model Source, the Candidates in configured order (provider, model, provider/model, weight, and prices per million tokens when known), the session id and sticky SessionTarget, Meta, and the virtual model’s strategy_config as JSON (Prompt is nil in this release). Select returns the chosen Qualified target and an optional Reason (debug logs only); OnAttemptEnd reports success, status, latency, and timeouts so the strategy can adapt. Failover chains, capacity probes, and session pinning stay GoModel’s job. Select the strategy on the virtual model:

virtual_models:
  - source: smart-router
    strategy: plugin                    # round_robin | cost | failover | adaptive | plugin
    strategy_plugin: cheapest_healthy   # plugin name; required with strategy: plugin
    strategy_config:                    # the plugin's Scope: route fields; defaults apply when omitted
      prefer: cheapest                  # cheapest | fastest
      max_error_rate: 0.2
    targets:
      - openai/gpt-4o-mini
      - anthropic/claude-haiku-4-5

strategy_config is validated against the plugin’s route-scoped fields (route_fields in GET /admin/plugins). strategy: plugin without strategy_plugin is a validation error, and PUT /admin/virtual-models rejects a name that is not a loaded plugin with a route hook.
Target weight is ignored under the plugin strategy.
Instance-scoped fields of a route plugin (an endpoint, an API key) come from a guardrail definition whose name and type both equal the plugin name; without one the plugin is initialized with {}. Instance-scoped secret values currently reach a route plugin redacted. Saving that definition rebuilds the instance; the previous one is closed once its in-flight Select calls return.
GoModel falls back to weighted round robin, logging one warning per virtual model, when the plugin is missing, is not a route plugin, failed to initialize, strategy_config is invalid, or Select errors, panics, takes longer than 250 ms, or returns a target outside the viable pool.

On the dashboard, the virtual model editor’s strategy dropdown lists one plugin:<name> entry per loaded route plugin (VIRTUAL_MODEL_STRATEGIES in the runtime config is round_robin,cost,failover[,adaptive],plugin:<name>,...); choosing it renders the plugin’s route fields under the targets table. cheapest_healthy is the built-in reference strategy (internal/plugins/builtin/routeexample). It keeps the last 50 outcomes per target and picks the cheapest (prefer: cheapest, the default) or the lowest-median-latency (prefer: fastest) target whose error rate is at or below max_error_rate (default 0.2), keeps a healthy SessionTarget, and considers every candidate when none is healthy.

Admin endpoints

All endpoints require the same admin credentials as the rest of /admin.

GET /admin/plugins

Every plugin type known to the gateway, including ones that failed to load:

[
  {
    "name": "string_replace",
    "label": "String Replace",
    "version": "1.0.0",
    "description": "Replaces, flags, or blocks text matching literal or regex rules in prompts, responses, and streams.",
    "kinds": ["prompt", "response", "stream"],
    "mutates": true,
    "guardrail": true,
    "source": "builtin",
    "fields": [{ "key": "rules", "label": "Rules", "input": "textarea", "required": true }],
    "route_fields": [],
    "health": "ok",
    "error": ""
  }
]

source is builtin, registered (compiled in through ext), or the absolute path of the .so. fields are the instance-scoped schema fields, route_fields the route-scoped ones. A .so built with gomodel plugin build also reports built_with (Go and pluginapi versions).

GET /admin/guardrails/types

One entry per plugin that implements a prompt, response, or stream hook, in the shape the guardrail editor renders:

{
  "type": "llm_judge",
  "label": "LLM Judge",
  "description": "Asks a judge model whether prompts and responses violate a policy, then blocks, answers, or flags them.",
  "defaults": { "action": "block", "max_tokens": 256, "target": "auto" },
  "fields": [{ "key": "model", "label": "Judge model", "input": "model", "required": true }],
  "phases": ["prompt", "response", "stream"],
  "source": "builtin",
  "mutates": false,
  "guardrail": true
}

phases, source, mutates, and guardrail are new (instance views from GET /admin/guardrails carry guardrail as well); fields[].input may now be secret, model, bool, or list, and fields[].scope is "" (instance) or "route".

GET, PUT, DELETE /admin/guardrails

Definitions gain fail_mode (closed | open, empty for the phase default) and timeout_ms (0 for none); views additionally carry phases (from the plugin manifest), summary, and health (ok | degraded, with health_error and health_checked_at for plugins that implement a health probe). PUT accepts both new fields. A stored secret value is returned as ********; send it back unchanged to keep it, or "" to clear it.

GET /admin/workflows/guardrails

The instances available to workflow steps, with the phases each supports:

[{ "name": "pii-redact", "type": "llm_based_altering", "phases": ["prompt", "response"], "summary": "openai/gpt-4o-mini • user • default prompt" }]

Workflow payload version 2

POST /admin/workflows accepts schema_version: 2 with steps[] carrying a phase; version 1 payloads (guardrails: [{ref, step}]) stay valid and are returned unchanged. Workflow views expose the per-phase chain hashes as chain_hashes. See Workflows.

Getting Started

Features

Pro features

MCP Proxy

Advanced

About

Overview

Phases

Chain execution

Decisions

Audit trail

Fail modes and timeouts

Instance lifecycle

Streaming

The Exchange

Configuration

Enabling the plugin system

Instances

Loading `.so` files

Dashboard

Shared-object plugins

Build and inspect

Writing a plugin

Testing a plugin

Routing strategy plugins

Admin endpoints

GET /admin/plugins

GET /admin/guardrails/types

GET, PUT, DELETE /admin/guardrails

GET /admin/workflows/guardrails

Workflow payload version 2

​Overview

​Phases

​Chain execution

​Decisions

​Audit trail

​Fail modes and timeouts

​Instance lifecycle

​Streaming

​The Exchange

​Configuration

​Enabling the plugin system

​Instances

​Loading .so files

​Dashboard

​Shared-object plugins

​Build and inspect

​Writing a plugin

​Testing a plugin

​Routing strategy plugins

​Admin endpoints

​GET /admin/plugins

​GET /admin/guardrails/types

​GET, PUT, DELETE /admin/guardrails

​GET /admin/workflows/guardrails

​Workflow payload version 2

Overview

Phases

Chain execution

Decisions

Audit trail

Fail modes and timeouts

Instance lifecycle

Streaming

The Exchange

Configuration

Enabling the plugin system

Instances

Loading `.so` files

Dashboard

Shared-object plugins

Build and inspect

Writing a plugin

Testing a plugin

Routing strategy plugins

Admin endpoints

GET /admin/plugins

GET /admin/guardrails/types

GET, PUT, DELETE /admin/guardrails

GET /admin/workflows/guardrails

Workflow payload version 2