Why ABS exists, its principles, and its scope.
AI agents are shipped today with almost no shared vocabulary for describing what they are actually supposed to do. Requirements get written as prose ("the bot should ask for the order number if it's missing"), QA teams turn that prose into ad-hoc scripts tied to one specific framework, and product owners have no artifact they can read, review, and sign off on that isn't either a wall of text or a pile of internal code.
API design solved an equivalent problem two decades ago. OpenAPI gave HTTP APIs a shared, tool-independent contract. AsyncAPI did the same for event-driven systems. Agent behavior has no equivalent. Every framework — LangChain, Rasa, a hand-rolled orchestration layer — describes behavior in its own internal, non-portable way, and that description usually lives nowhere except the code itself.
ABS is an attempt to close that gap: a shared, human-readable, tool-independent format to describe the observable behavior of an agent, so it can be written once and read by developers, QA, product owners, and machines (evaluators, CI pipelines, visual editors) alike.
OpenAPI describes the shape of requests and responses. AsyncAPI describes the shape of events. Neither describes a sequence of behavior over time involving a conversational actor, and neither makes room for the kind of evaluation agent QA actually needs — exact match, fuzzy match, LLM-as-judge, schema validation. ABS borrows their spirit — plain text, versionable, tool-independent, both human- and machine-readable — but solves a different shape of problem: closer to a behavioral trace than to a data contract.
Gherkin (Given/When/Then) already solved "human-readable behavioral spec" for software in general, and ABS deliberately keeps a similar spirit. But agent interactions have actors and asymmetric obligations that a generic Given/When/Then doesn't model cleanly: who said or did something, what tool was invoked and with which parameters, and how a free-text or generative response should be evaluated — a fixed-string assertion is rarely enough for an LLM response. ABS's Behavior model (Actor + Action + Target + Content + Evaluations) is, in effect, a specialization of that same idea for conversational, tool-using agents.
Observable first. ABS describes what can be observed from outside the agent — messages, tool calls, UI, outcomes. It intentionally excludes prompts, chains of thought, and model internals, so a specification survives a change of model, framework, or vendor.
Human readable. A product owner and a QA engineer should both be able to read an ABS session and agree on what it means, without reading code.
Tool independent. The same behavior can be implemented with MCP, REST, function calling, plugins, or a hand-rolled backend, and the ABS document that describes it doesn't change.
Composable, not exhaustive. ABS defines a small core vocabulary plus an extension mechanism, not an exhaustive catalog of every possible agent action.
Testable. Any Behavior can optionally carry Evaluations, so the same document that describes intended behavior can drive automated verification.
No vendor lock-in. You run your agent on your infrastructure. abslang runs it, captures the trace, and sends only the relevant data to the evaluator. The evaluator never calls your agent — it receives {type, input, context, response, threshold} and returns {passed, score, reason}. Any provider can implement this adapter in an afternoon.
Evaluate outputs, not reasons. ABS tests what the agent emitted — messages, tool calls, outcomes — never why it chose them. Decisions like intent detection or tool selection are verified by declaring the expected behavior (optional + expected + when + dataset) and checking it deterministically; asking an LLM to explain a decision is telemetry, not testing. Output quality and safety — groundedness, relevance, coherence, fluency, toxicity — is exactly what evaluators are for, and is supported through evaluator types and adapters.
The industry standard today couples agent execution and evaluation in the same platform — you deploy your agent to their infrastructure, they run it, they evaluate it. Change platforms and you rewrite everything.
ABS separates these concerns:
Industry today: [agent execution + evaluation] in one platform → vendor lock-in
ABS: [agent] in your infra → [trace] → [evaluation] wherever you want
The adapter contract is deliberately minimal. Any provider — Azure AI Foundry, AWS Bedrock, Google Vertex AI, LangSmith, Galileo, a local Ollama instance — implements the same interface:
// All an adapter does:
adapter.evaluate(trace, {
type: "Groundedness",
input: "Can I return sale items?", // resolved from query: user_asks.says
context: "Returns within 14 days.", // resolved from context: kb_result.responds
response: "Yes, within 14 days.", // resolved from response: self
threshold: 0.8
}) → { passed: true, score: 0.92, reason: "..." }
Your session file never changes. Only the --adapter flag.
v0.2 — the conceptual core is closed (Session, Behavior, Actor, Action, Target, Content, Variables, Evaluations, Fragments, Tool Interaction, JSON Schema, Optional Behaviors). Implementations in TypeScript and Python, plus a React UI editor, a Next.js docs site, and a VSCode extension. See Roadmap for open design questions deferred to v0.3+.