All ABS actions and evaluators on one page.
| Category | Actions | When to use |
|---|---|---|
| Communication | says, asks, informs, greets, clarifies, confirms, rejects, suggests, shows | Bot or user speaks |
| Execution | calls, submits, retrieves, stores, updates | Bot invokes tools |
| Interaction | selects, uploads, approves | User interacts with UI |
| Delegation | hands_off | Bot transfers to a human |
| Type | Level | What it verifies |
|---|---|---|
contains | Step | Text includes substring |
exact_match | Step | Exact text match |
regex | Step | Pattern match |
f1 / bleu / rouge | Step | Graded match against a reference (ground_truth) |
schema | Step | Valid JSON structure |
tool_call | Step | Correct tool call |
llm_judge | Step | Qualitative criteria |
Groundedness | Step | Response supported by context |
Relevance | Step | Response addresses the query |
Coherence | Step | Logical flow and consistency |
Fluency | Step | Natural language quality |
sequence | Chain | Step order |
eventually | Chain | Something happens at least once |
never | Chain | Something never happens |
count | Chain | Something happens N times |
within | Chain | A happens within N steps of B |
variable_consistency | Chain | A value stays unchanged |
HateUnfairness | Step | No hate speech or bias |
Violence | Step | No violence or threats |
Sexual | Step | No explicit content |
SelfHarm | Step | No self-harm encouragement |
Where the judgment runs: deterministic checks (contains, sequence, never, …) and the reference metrics (f1, bleu, rouge) run locally with no model. llm_judge runs on the built-in judge (OpenAI/Anthropic/Gemini), Azure, AWS Bedrock, Google Vertex AI, or AI Evaluator. Safety dimensions run on the built-in judge with a curated rubric — no criteria required. Pick the engine with --adapter.