8. Available tools

ABS isn't just a format: you have a tool ecosystem to write, run, and debug specifications.

CLI — the Swiss Army knife

Install with npm or pip:

npm install -g abslang
# or
pip install abslang

Three commands:

abslang init                               # Scaffold a project from scratch
abslang run session.abs.yaml --agent $URL  # Run against your agent
abslang report report.json                 # Read a previous report

ABS Designer — visual editor in the browser

Zero install. Open the Designer and build Sessions by dragging blocks. Ideal for Three Amigos sessions with PO and dev. Export YAML ready to run.

VSCode extension

Install from the VSCode marketplace. Open any .abs.yaml and the visual editor opens automatically. Edit visually, save, and run against your agent with ▶ Run. All in one place.

Mock agent — test without a real agent

If you're writing the spec and your agent doesn't exist yet or isn't available, use the mock agent to verify your YAML parses and evaluates correctly:

python tools/mock_agent.py --port 8080
abslang run session.abs.yaml --agent http://localhost:8080/chat

The mock agent returns predefined responses, so you can validate the entire chain — parsing, execution, evaluations — without depending on a real agent.

Chatbot — generate specs by chatting

You already know how to write specs by hand. But ABS can do more: instead of remembering every field and action, tell the chatbot what you want to test and it generates the file for you.

It's available in two places:

On the website. Go to the ABS Designer and open the chat. Type something like:

"I want to test a refund bot: the customer asks to return a damaged item, the bot verifies with Orders API and Refunds API, and confirms the amount and timeline."

The chatbot returns a complete .abs.yaml with Behaviors, tool calls, and suggested evaluations. Review it, tweak what you need, and run it.

In the CLI. The same chat is integrated:

abs chat

It works the same way: describe the scenario in natural language, and the chatbot generates the spec.

Why does this matter? Because you can go from an idea to an executable test in two minutes. You tell the chatbot the same thing you'd tell a coworker at standup, and you get a file ready for abslang run. Then you refine it with datasets, advanced evaluations, and everything you learned in this tutorial. But 80% of the initial work is already done.

Full documentation

• Specification — normative reference
• Evaluations — all evaluators with examples
• Patterns — recipes for real-world cases (routing, escalation, multi-step verification)
• GitHub — source code, examples, issues

Next: Cheat sheet →