AgentSpec & OpenTelemetry GenAI: Agent Enforcement & Tracing

An LLM agent that can call external tools, modify records, or execute code cannot be secured by system-prompt instructions alone. A prompt is a probabilistic suggestion to a language model; it is not an execution boundary. When an agent misinterprets an ambiguous instruction, encounters indirect prompt injection, or enters a runaway retry loop, teams need two capabilities operating in lockstep: deterministic runtime enforcement that intercepts unsafe actions before they execute, and standardised telemetry that records every step of the agent's control loop.

In 2025 and 2026, two technical building blocks have become central reference points for this architecture: AgentSpec for declarative runtime enforcement, and the OpenTelemetry GenAI semantic conventions (alongside OpenInference) for structured agent tracing.

Prompt instructions advise the model; runtime enforcement constrains the execution environment; semantic telemetry proves what the agent attempted, what the policy decided, and what actually ran.

Why prompt guardrails and unstructured logs fail in agentic workflows

Traditional LLM applications followed a single request–response turn: a user sent a prompt, the application retrieved context, and the model returned text. In that setup, post-hoc text filters and simple request logs were often enough to spot obvious failures.

Tool-using and multi-step agents break both assumptions:

Side effects happen mid-trajectory. By the time an agent returns its final summary to the user, it may have already executed SQL queries, sent API mutations, or invoked subagents. Checking only the final text output cannot undo an unauthorised tool call. This is why the OWASP Top 10 for LLM and Generative AI Applications highlights Excessive Agency and Prompt Injection as critical architectural risks requiring deterministic permission and action controls outside the model.
Free-text logs hide causal chains. A single user task can trigger fifteen model calls, six tool executions, and three policy retries. Without a shared span hierarchy and attribute schema, engineers cannot query which tool arguments caused a failure, how much latency or token budget a specific subagent consumed, or whether a guardrail fired as designed.

What is AgentSpec? Deterministic runtime enforcement for LLM agents

Introduced in the research paper AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents (arXiv:2503.18666), AgentSpec is a lightweight domain-specific language (DSL) and runtime enforcement framework designed to specify and enforce safety and reliability constraints over LLM agents.

Instead of asking the language model to police itself, AgentSpec places a deterministic monitor between the agent's reasoning step and the execution environment. Every AgentSpec rule is structured around three explicit components:

rule <rule_name> {
  trigger:     <agent_event_or_tool_invocation>
  predicate:   <deterministic_state_and_argument_checks>
  enforcement: <action_when_predicate_holds>
}

1. Trigger (when to inspect)

The trigger identifies the exact lifecycle event where execution must pause for inspection—most commonly right before an external tool call, code execution step, state transition, or external message transmission.

2. Predicate (what to verify deterministically)

The predicate evaluates boolean conditions over the current execution state, trajectory history, and proposed tool arguments. Because predicates run in deterministic code rather than inside an LLM prompt, they can check invariants reliably:

Has the user session verified authentication before calling a billing or account-mutation tool?
Does a proposed shell command or file path stay strictly inside the allowed workspace directory?
Has the agent exceeded its maximum retry count or budget ceiling (see resource-bounded agent contracts)?
Does the outbound payload contain unredacted personal identifiers or secrets?

3. Enforcement (how to intervene)

When a trigger fires and the predicate matches a violation, the enforcement clause executes a pre-defined intervention:

Stop / Block: Abort the tool call immediately and return a structured error to the orchestrator.
User / Human inspection: Pause the workflow and escalate the proposed action, arguments, and risk rationale to a human reviewer.
Invoke corrective action / Precondition step: Force a prerequisite verification step (such as schema validation or dry-run diffing) before permitting execution.
Self-reflection / Bounded retry: Return the specific policy violation to the agent so it can reformulate a compliant plan within a capped retry budget.

In the broader safety landscape, AgentSpec sits alongside agent behavioral contracts (ABC) and production AI guardrails: it turns implicit assumptions about "what the agent should never do" into versioned, testable code.

OpenTelemetry GenAI semantic conventions and OpenInference explained

Enforcing rules at runtime solves only half the problem. Platform and reliability engineers also need to observe, query, and audit every enforcement decision across distributed services.

The Cloud Native Computing Foundation's OpenTelemetry Semantic Conventions for Generative AI define a vendor-neutral standard for naming spans, attributes, metrics, and events in LLM and agent systems. Its dedicated specification for GenAI agent spans standardises how frameworks represent agent creation, agent invocation (invoke_agent), and tool execution (execute_tool).

In parallel, teams frequently encounter OpenInference, an open telemetry specification developed in the AI observability ecosystem that complements OpenTelemetry with conventions tailored for RAG retrievers, embeddings, rerankers, and LLM judge evaluations.

Dimension	OpenTelemetry GenAI Semantic Conventions (`gen_ai.*`)	OpenInference Conventions	How to use them together
Primary stewardship	OpenTelemetry / CNCF Semantic Conventions Working Group.	Open-source AI evaluation & observability community (Arize Phoenix and ecosystem).	Use OpenTelemetry as the core transport, collector, and trace backbone across all microservices.
Core span & attribute focus	`gen_ai.operation.name`, `gen_ai.system`, `gen_ai.request.model`, `gen_ai.usage.input_tokens`, `gen_ai.agent.id`, `gen_ai.tool.name`.	Rich semantic attributes for retrieval chunks, document scores, embedding vectors, prompt templates, and evaluator rubrics.	Standardise service-level metrics, latency, cost, and GenAI agent spans on `gen_ai.*`.
Privacy & payload handling	Separates metadata attributes from sensitive prompt/completion events so payloads can be disabled by default in production.	Supports explicit masking and redaction configuration for retrieval and evaluation payloads.	Keep raw prompt/document payloads gated behind privacy rules while always emitting structural metadata and policy outcomes.

Combining AgentSpec enforcement with OpenTelemetry spans in one control loop

The real engineering leverage appears when AgentSpec runtime enforcement and OpenTelemetry GenAI tracing are wired into the same execution wrapper. Every time an agent proposes an action, the runtime creates a child span under the parent invoke_agent trace, evaluates the relevant AgentSpec rules, and records the result as structured span attributes before any external tool is touched.

Lifecycle stage	Runtime action	AgentSpec role	OpenTelemetry / Agent Span telemetry emitted
1. Agent turn start	Orchestrator receives task and initializes state budget.	Loads active rule set version and session invariants.	Root span `invoke_agent` with `gen_ai.agent.name`, `gen_ai.request.model`, policy bundle version hash.
2. Model reasoning	LLM proposes a tool call with structured JSON arguments.	Matches `trigger` on target `tool_name` and event type.	Child span `chat` with token counts (`gen_ai.usage.*`), finish reason `tool_calls`, latency.
3. Pre-execution gate	Interceptor pauses tool dispatch and runs deterministic checks.	Evaluates `predicate` over session state, trajectory history, and tool arguments.	Event or span `policy.evaluate` recording `policy.rule_id`, `policy.predicate_result`, evaluation duration (µs).
4a. Compliant execution	Predicate passes; tool executes within sandbox/permissions.	Permits call and updates trajectory state counters.	Child span `execute_tool` with `gen_ai.tool.name`, `gen_ai.tool.call.id`, status `OK`.
4b. Policy violation	Predicate fails; tool call is blocked or escalated.	Triggers `enforcement` (`block`, `escalate_human`, or `bounded_retry`).	Span status `ERROR` or `BLOCKED`, `policy.enforcement_action`, violation code, human-review ticket ID.

This combined architecture directly supports ISO/IEC 42001 and AI auditability: when an auditor asks whether a high-risk tool can be invoked without authorisation, the team can point to both the declarative AgentSpec rule in version control and the OpenTelemetry trace archive proving 100% pre-execution interception.

Prerequisite skills for engineering teams adopting AgentSpec and GenAI telemetry

Deploying runtime enforcement and semantic tracing is not a plug-and-play SDK upgrade. As we detail in our guide to skills intelligence in IT and role redesign, engineering teams need a clear sequence of prerequisite capabilities:

Distributed tracing fundamentals (OpenTelemetry). Engineers must understand context propagation, parent-child span lifecycles, sampling strategies, and collector pipelines before adding GenAI-specific conventions.
Deterministic contract and policy design. Writing effective AgentSpec predicates requires the ability to translate ambiguous business risks into crisp, side-effect-free state invariants—a core part of Spec-Driven Development (SDD).
Production LLM observability and agent evaluation. Teams must monitor false-positive enforcement rates (rules that block valid work) alongside false negatives (unsafe trajectories that lacked a predicate), feeding real production failure modes back into regression test suites.

5-step implementation checklist

To bring deterministic enforcement and semantic telemetry into an existing agent workflow without stalling delivery:

Inventory every tool with external side effects. Classify all agent tools into read-only queries, reversible state changes, and irreversible external actions (payments, emails, database deletes, permission changes).
Instrument invoke_agent and execute_tool spans first. Adopt the OpenTelemetry GenAI semantic conventions across your agent runner so every model call and tool invocation shares a single trace ID.
Write AgentSpec-style rules for your top five irreversible tools. Start with deterministic predicates covering authentication state, parameter bounds, workspace path confinement, and per-run cost/step limits.
Run enforcement rules in shadow-audit mode for one sprint. Emit policy.evaluate telemetry spans without blocking execution for a week to verify that your predicates do not misfire on legitimate edge cases.
Turn on blocking enforcement and wire alerts to incident response. Enable hard blocks and human-approval escalations for high-risk predicates, and review blocked-action traces weekly to refine both your specs and your evaluation datasets.

AgentSpec and OpenTelemetry GenAI semantic conventions: runtime enforcement and agent tracing