Route ABS evaluations through Google Vertex AI Gen AI Evaluation service.
Routes ABS LLM-based evaluations through the Vertex AI Gen AI Evaluation service (bring-your-own-response mode). The adapter never re-runs your agent.
Availability. Ships with the Python CLI and the npm CLI. On npm, install
@google-cloud/vertexai; the adapter renders the official pointwise prompt templates against a Gemini model.
pip install "abslang[google]"
export GOOGLE_CLOUD_PROJECT="your-project"
export GOOGLE_CLOUD_LOCATION="us-central1"
gcloud auth application-default login
The judge is Gemini internally. GCP's free trial gives $300 for 90 days.
abslang run session.abs.yaml --agent $AGENT_URL --adapter google
| ABS | Vertex metric |
|---|---|
Groundedness | groundedness |
Relevance | question_answering_quality |
Coherence | coherence |
Fluency | fluency |
HateUnfairness / Violence / Sexual / SelfHarm | safety |
llm_judge | custom pointwise metric |
custom id google.* | custom pointwise metric |
The four safety dimensions map to Vertex's single safety metric (0 = unsafe,
1 = safe). Rubric-based scores are normalized to 0–1 before threshold is applied.