Tutorial

A step-by-step guide to learning ABS — Agent Behavior Specification — from scratch. If you can write a bullet list, you can write ABS. No code required.

We'll work through a single example — a refund agent handling a damaged item — and build it up from a plain conversation to a fully automated test suite with multi-stage evaluations and parametrized datasets.

The 20-minute path

1. What ABS is and its structure — the acronym, the YAML format, the three fields, the four actors
2. Write a real conversation — model a refund bot across three turns: classification, resolution, closing
3. Step-level evaluations — add llm_judge, contains, and Relevance at three distinct stages of the same flow
4. Chain evaluations — sequence and variable_consistency: properties of the entire trace
5. Run it — terminal, browser, or VSCode: execute against your agent
6. Datasets and golden data — from one happy-path case to N parametrized scenarios via .jsonl
7. Common mistakes — pitfalls every QA hits the first time, and how to avoid them
8. Available tools — CLI, Designer, VSCode extension, mock agent, chatbot
9. Cheat sheet — all actions and evaluators on one page

Each page takes ~5 minutes. Start at the top and work through — by the end you'll have a complete automated test suite for your agent, with evaluations running at every stage of the conversation.