Azure AI Foundry adapter

Routes ABS LLM-based evaluations through Azure AI Foundry (local SDK). The adapter never re-runs your agent — it receives the trace ABS collected, maps it to Azure's input, and returns a normalized EvalResult.

Install

pip install "abslang[azure]"

Optional (cloud evaluation runs, Foundry dashboard): pip install "abslang[azure-cloud]".

Configure

export AZURE_OPENAI_ENDPOINT="https://<account>.services.ai.azure.com"
export AZURE_OPENAI_KEY="..."
export AZURE_OPENAI_DEPLOYMENT="<judge-model-deployment>"

The evaluators are AI-assisted — they call an Azure OpenAI model as the judge and incur token usage per evaluation.

Run

abslang run session.abs.yaml --agent $AGENT_URL --adapter azure

Or per rule with adapter: azure. If a rule declares adapter: azure but azure was never registered, the runner reports a clear error instead of silently falling back.

TypeScript (npm)

The npm package ships the same --adapter azure without the Python azure-ai-evaluation dependency. The adapter renders the official Azure prompt templates (MIT, pinned to azure-ai-evaluation 1.18.5) locally and calls your Azure OpenAI deployment's chat completions endpoint:

export AZURE_OPENAI_ENDPOINT=... AZURE_OPENAI_KEY=... AZURE_OPENAI_DEPLOYMENT=...
abslang run session.abs.yaml --agent $AGENT_URL --adapter azure

Quality dimensions and llm_judge are faithful to the Python SDK. The agentic evaluators reuse the official templates with a simplified conversation rendering from the ABS trace, so scores are comparable in direction, not guaranteed byte-identical.

Supported evaluators

Score normalization

Azure returns 1–5 Likert scores for many evaluators. The adapter normalizes everything to 0–1 (values above 1 are divided by 5) before threshold is applied.