Building an AI Agent for Internal Security Standard Exception Requests at Woven, a Toyota Company

A presentation by Juan, a security engineer at Woven (Toyota), on building an AI agent that answers questions and supports decisions on exception requests for internal security standards. The session covers the manual process that preceded the agent, the agent's architecture built with Go and MCP, and the quality evaluation approach used to make its outputs reviewable by humans.

■ Problem and existing workflow
- Roughly 300 internal standards, processes, policies and guidelines that teams must account for
- The manual exception request flow, its security review step, and the friction from multiple time zones

■ Agent design and implementation
- Where the AI agent was inserted and what outputs it produces (approve, reject, request more information)
- Options weighed for context handling: full context, RAG approaches, and fine-tuning
- Context delivery via MCP tools over Markdown standards stored in GitHub, implemented in Go

■ Evaluation and quality control
- Deterministic controls: tool-call verification, output format checks, and a retry loop
- Internal metrics (coherence, specificity) plus Ragas metrics (relevance, faithfulness, precision)
- Processing results across a few hundred tickets and identified areas for further work

■ Q&A
- Retrieval cadence through CI, token usage per ticket, and handling of context size

Engineers and teams building LLM agents, or working in compliance-heavy organizations, can take away a concrete pattern for combining automated assistance with human accountability, along with a practical framework for measuring agent output quality.