Create a Scenario

Set up your first scenario, add a test, and write assertions to verify your agent’s behavior.

Create a scenario

A scenario is a container for related tests. Start by creating one from the Scenarios page.

1. Go to the Scenarios page in the left sidebar.
2. Click Create scenario in the top-right corner.
3. Enter a name for your scenario (e.g., “Quick Order Check”).
4. Optionally add a description.
5. Click Create.

You land on the scenario detail page, ready to add tests.

Add a test

Each test represents a single conversation between the AI tester and your agent.

1. On the scenario detail page, click Create test.
2. Enter a test name (e.g., “Quick order lookup”).
3. Write an instruction - this tells the AI tester what to do during the conversation.

The instruction is written as a persona with context and actions. For example:

You’re Sarah. Your order number is NS-28479 and your email is sarah@example.com. Ask about your order status. Provide your details when asked. Once you get an answer, say thanks.

The AI tester follows these instructions and has a real conversation with your agent.

Write assertions

Assertions are pass/fail checks evaluated after the conversation completes. They define what “correct” means for your test.

1. In the test form, scroll to the Assertions section.
2. Click Add item to add an assertion.
3. Write the criteria - a plain-language description of what to check. For example: “The agent calls check_order with the correct order ID.”
4. Choose a severity level:

• Blocker - Critical. If this fails, the entire test is considered failed.
• Medium - Important, but not critical for the test to pass.
• Low - Nice to have.

5. Add more assertions as needed (up to 10 per test).

Severity affects the test score. See Assertions for details on how the scoring formula works.

Save and review

Click Save to create the test. It appears in the test table on the scenario detail page, showing the test name and assertion count.

You can add more tests to the scenario by clicking Create test again. Each test can cover a different aspect of your agent’s behavior - order lookup, cancellations, refund requests, and so on.

What’s next

Your scenario and test are ready. Head to Run a Test to execute it against an agent and see the results.