Guide
Test your agent and read the diagnostics of each reply
Write to it as a customer would in the Playground tab and see in Diagnostics which sources it uses, how confident it is, what it is missing and what each reply costs.
- Reviewed on
- 6 min read
In this article
Your agent’s Playground tab is a chat where you write to it as a customer would. Next to each reply, Diagnostics shows which sources it used, how confident it was, what information it was missing and how many credits it cost. It uses the real agent and real credits, but the conversation does not appear in the inbox or in analytics.
Before you start
- Role. People with the Member, Admin or Owner role can test the agent. With Read only, “Your role cannot test the agent” appears, because tests use credits.
- Credits. Each test reply costs the same as a real one and shows up under Usage, in the Playground channel. To check the knowledge without spending anything, use test a question.
- Agent status. It does not have to be active: tests work with an agent that is a Draft or is Paused.
Run a test
Go to Agents, open your agent and press the Playground tab.
Write a question as if you were a customer and press Send message or the Enter key. If your website chat has suggested questions, they appear as buttons to start with.
Read the reply and its diagnostics. While it is being written you can stop it with Stop the reply.
Carry on the conversation with more questions. The agent remembers what came before, as it does with a customer.
Press New conversation to start from scratch: for example, after a handoff, or to repeat a question without what was said before.
If your agent answers in more than one language, the Reply language selector lets you leave Automatic language, which is how it answers your customers, or force one of its languages.
What you change under Behaviour, Profile or Knowledge applies to the next reply. A new source counts once its status is Ready. When you leave the tab, the test conversation ends.
How to read the diagnostics
The Diagnostics panel explains how the last reply was built. If there are several, press See diagnostics under the one you are interested in.
| Section | What it tells you |
|---|---|
| Question | The question the reply belongs to |
| Confidence | How closely what it found in the knowledge matches the question: High, Medium or Low |
| Sources used | The knowledge sources it consulted, each with its relevance as a percentage. Press one to open it under Knowledge |
| Missing information | Only appears when the agent has no knowledge or found nothing relevant for that question |
| Cost of the reply | The credits that reply used |
| Credits remaining | The account’s balance after it |
| Language | The language it answered in |
| Handoff | Appears if the agent handed the conversation to a person |
| Lead captured | Appears if the agent collected the contact details |
The confidence levels
| Level | What it means | What to do |
|---|---|---|
| High | It found information closely related to the question. The most relevant source reaches 55% or more | Check that the reply says what you want |
| Medium | It found related information, though not an exact match. The most relevant source is between 38% and 55% | Read the reply carefully. If it is not exact, add an FAQ in the customer’s words |
| Low | It found no clear information: the reply may be generic or say it does not know | Add what is missing to the knowledge |
Confidence measures what the agent found, not whether the reply is correct, so always read it. Also, only the knowledge counts: what the agent knows from its profile, such as the opening hours or the contact details, does not appear as a source. A correct reply about your opening hours can come out with Low confidence and no sources.
With your customers, questions answered with Low confidence are collected under Analytics › Questions your agent could not answer. The ones from tests are not.
Missing information
There are two possible notices:
- “This agent has no knowledge base yet. Add sources so it can answer with facts about your business.”
- “No relevant information was found in the knowledge base for this question. Consider adding a source or FAQ that covers it.”
Below them are two buttons:
- Add as an FAQ opens Knowledge with the question already written. Fill in the answer and save it.
- Add a text opens Knowledge so that you can paste longer information.
When the source is Ready, repeat the question in a New conversation. If the agent still does not use the fact, see why your agent says it does not know.
Messages that can appear instead of the reply
| Message | What it means | What to do |
|---|---|---|
| “No credits left” | The agent cannot answer, in tests or to your customers | Press Buy credits. See what happens when the credits run out |
| “Your trial has ended” | The free trial is over. Your configuration and knowledge are intact | Press Choose a plan. See what happens when the trial ends |
| “Your subscription is not active” | The agent does not answer while the subscription is inactive | Press Review billing |
| “Verify your email” | The trial credits are activated when you verify your email | Resend the link from the notice at the top of the app |
| “Too many messages in a row” | You sent many questions in a short time. The limit only affects tests | Wait a minute and press Try again |
| “The AI service is not responding right now” | A temporary failure. No credits were charged | Press Try again in a few minutes |
| “The AI service is not available yet” | It is not about your account: iAgentify’s setup is still being finished | If it stays like this, press Contact support |
| “This account is suspended” | Agents do not answer while the account is suspended | Press Contact support |
| “We could not get the reply” | Something failed on our side. No credits were charged | Press Try again |
Inside the chat you may also see “The agent has handed this conversation to a person”. That is a handoff, as it would do with a customer: on a real channel, your team would be notified and continue from Conversations. To keep testing, press New conversation.
Check the knowledge without using credits
Under Knowledge, the Test a question section shows the passages the agent would find for a question, most relevant first, each with its relevance: High, Medium or Low. It writes no reply, so it uses no credits, and anyone on the team can use it, including the Read only role.
Go to Knowledge and choose the agent, or open your agent’s Knowledge tab.
Under Test a question, type the question and press Search.
It tells you whether the fact is in the knowledge. To see how the agent puts it, with its tone and its instructions, you need the Playground tab. How to read the result is in see what it finds for that question.
Related articles
- Why your agent says it does not know, and how to fix itYour agent only answers with what is in its knowledge. If it says it does not have that information, here is how to find the gap and close it.
- Add knowledge: your website, documents, texts and FAQsHow to add each kind of source to your agent (website, documents, FAQs, text and business info), which one to choose and what happens afterwards.
- Adjust how your agent answers: tone, languages and instructionsChange your agent’s tone, languages, instructions, handoff to a person and the details it asks for, and bring back an earlier version if you need to.
- The free trial and credits: what happens when they run outThe trial lasts 30 days and includes 300 credits, with no card. What uses credits, where to see your balance, and what happens when they run out or the trial ends.