Pydantic AI OpenTelemetry instrumentation: retry prompt content is not redacted when `include_content=False`

Summary

Pydantic AI's OpenTelemetry instrumentation supports InstrumentationSettings(include_content=False) to exclude message content — user prompts, model completions, tool call arguments and responses — from exported telemetry, so that agents can be monitored without sending sensitive content to the observability backend.

Retry prompts — the feedback messages Pydantic AI sends back to the model when its output fails validation or an output validator raises ModelRetry — were not covered by this setting when they were not associated with a tool call, as with structured output modes that don't use tool calls (e.g. NativeOutput or PromptedOutput) and output validators applied to text output. Their full content was recorded on model request spans even with include_content=False. Since validation feedback can quote invalid values from the model's response, message content that the setting was configured to withhold could reach the telemetry backend.

Details

Retry prompts without an associated tool call were serialized into span message attributes (such as gen_ai.input.messages and pydantic_ai.all_messages) as regular text parts, and this path did not apply the include_content check that every other message part applies. Retry prompts tied to a tool call, and all other message content, were redacted correctly.

Impact

This does not grant an attacker any new access to the agent or its data: the content is only visible to whoever can read the exported telemetry. The exposure matters when traces are exported to a destination whose audience is broader or less trusted than the agent's own data — a third-party observability vendor or a shared backend — which is the scenario include_content=False exists for. The disclosed content is limited to retry feedback: validation error details, which can quote invalid values from the model's output, or output-validator ModelRetry messages.

Who Is Affected

You are affected if you enabled instrumentation with include_content=False and your agent can produce retry prompts outside of tool calls: it uses a structured output mode that doesn't rely on tool calls (such as NativeOutput or PromptedOutput), or output validators on text output. You are not affected if you don't set include_content=False, or if all of your agent's retries come from tool calls (including the default tool-based structured output mode), which were redacted correctly.

Remediation

Upgrade to a patched version; retry prompt content now honors include_content=False like all other message content. If you rely on this setting, consider reviewing previously exported traces for retry prompt content that should not have been recorded.

Workaround for Unpatched Versions

If you cannot upgrade, scrub or drop the message attributes (gen_ai.input.messages, gen_ai.output.messages, pydantic_ai.all_messages) in your telemetry pipeline (for example with an OpenTelemetry Collector processor), or use tool-based structured output modes, whose retry feedback honors include_content=False.

Credits

Reported privately by the University of Sydney Security Research Team (Liyi Zhou, Ziyue Wang, Strick, Maurice, Chenchen Yu), and later independently discovered and reported publicly, with a fix, by @sean-kim05.