Playground & evals

Chat against your live agent configuration in the Playground, and measure answer quality with evals.

View as Markdown

Before the agent talks to customers, talk to it yourself. The Playground runs the same pipeline as production — knowledge retrieval, custom actions, MCP tools, escalation rules — so what you see is what customers get.

The Playground

Open app.asks.app/ai-agent/playground and start chatting. Nothing here is visible to customers.

The chat — styled exactly like your live widget, down to multi-bubble replies, with suggested starter questions to get going and attachment upload from the composer.
The toolbar — the agent's status dot, name, and active model tier badge, plus History, a dropdown of past playground conversations you can revisit, resume, or delete, and New chat for a clean slate.
Response details — an optional rail (toggle it on from the toolbar) showing what the agent did behind the scenes for its latest reply: a thumbs up/down to rate it, every tool call with expandable arguments and results, the sources cited, and a Setup snapshot of the live configuration — guidance tier and tone, escalation on or off, knowledge — each linking to its page.

Playground with the toolbar showing the agent name and tier badge, a widget-styled chat with suggested starter questions, and the Response details rail open showing the reply rating, tool calls, and the Setup snapshot — The Playground with the Response details rail open.

Test the edges, not the happy path: ask something outside your knowledge base, type an escalation keyword, request an order lookup. Then adjust guidance or escalation rules and re-run in the same conversation.

Evals

Premium Evals put a number on answer quality instead of a gut feeling. At app.asks.app/ai-agent/evals you create datasets — collections of test questions with expected outcomes — and execute runs against them to track pass rates. Each run picks the model tier to test, so you can compare tiers on the same questions; leave a question's expected output empty and it's flagged for manual review instead of auto-judging.

Every run has a detail page: per-question verdicts with the judge's reasoning, latency and token counts, the full tool-call trace, and the option to override a verdict by hand. Re-running the same dataset after a knowledge base or guidance change tells you whether quality moved, and in which direction.

Evals page on a Premium workspace with the Datasets card listing test datasets and the runs table showing run status badges and pass rates — Datasets and runs with pass rates.

What the agent understands

The Playground and every channel share the same input handling, so you can test these directly:

Images — the agent sees image attachments and can answer about them (a screenshot of an error, a photo of a product).
Voice notes — audio attachments up to 25 MB are transcribed and treated as the customer's message.
Documents — PDFs and similar files are read as part of the conversation.
Any language — the agent detects the language of the customer's latest message and replies in it, translating your templates and instructions as needed. No configuration required.

Once the Playground behaves the way you want, enable the agent on real channels from the deploy page.

← Previous

Data collection

Overview