Using Microsoft Foundry Agent Optimizer to Improve AI Agent Reliability

Developers who see occasional format or behavior inconsistencies in an AI agent can use Microsoft Foundry's agent optimizer to compare models, prompts, and tool-calling setups against production traces. The video walks through configuring an optimization run, selecting candidate and evaluation models, and reviewing quality and latency outcomes from the resulting evaluation report. It also frames optimization as one step in a broader improvement path that may later include fine-tuning and RFT.

■ Optimization setup in Microsoft Foundry
- Selecting an agent version and an automatic optimization approach
- Adding candidate models and an evaluation model
- Using production traces or datasets, with minimum trace requirements
- Reviewing estimated cost before submitting the run

■ Results and evaluation review
- Comparing baseline and candidate performance for quality and latency
- Inspecting evaluation runs, pass/fail rubric outcomes, and JSON details
- Identifying issues such as sensitive-information disclosure for follow-up guardrails

This is aimed at developers and teams building agents who want a practical overview of optimizing prompts, model choice, and tool use. Viewers can expect to understand the workflow for running an agent optimizer, interpreting its evaluation output, and deciding on next steps such as documentation review and further testing.