Operating Production AI Applications with Azure Cosmos DB: Observe, Evaluate, and Optimize

This episode of Code with Azure Cosmos DB explains a lifecycle for operating production AI applications with confidence, focusing on observing, evaluating, and optimizing agent behavior. It continues a series that builds a travel planner application and shows how operational data, vector search, and agent memory reside in Azure Cosmos DB.

■ Observability
- Traces capture tool calls, routing decisions, and memory retrievals
- LangSmith, Microsoft Agent Framework, and Microsoft Foundry provide visibility into agent behavior
- Open telemetry allows traces to flow into existing observability stacks

■ Evaluation
- LangSmith supports evaluation of routing decisions, tool calls, and end-to-end performance
- LLM-as-a-judge patterns, regression testing, and CI/CD evaluations help catch issues before production
- Azure Cosmos DB-specific evaluation verifies cross-session preference recall

■ Optimization
- RU optimization through indexing strategy, partitioning model, and query patterns
- Retrieval tuning for vector and hybrid search
- Memory summarization and compression
- LLM token reduction through tighter prompts, smaller context windows, and fewer model calls

Viewers building AI applications on Azure gain a framework for understanding agent behavior, catching regressions, and improving production efficiency. The episode is suitable for developers and product teams interested in observability, evaluation, and optimization practices for stateful agents. Viewers can apply the observe-evaluate-optimize approach to their own AI application lifecycle.