Using Analytics to Understand and Improve Your Agent

Your agent gets better over time — but only if you use the data it generates. MerchantAI’s analytics surfaces what your visitors are actually asking, where your agent is falling short, which knowledge sources are being used most, and which conversations your human team had to handle. Together, these signals give you a clear picture of where to focus your improvement effort and how to make the agent more capable with each iteration.

​

Topic clustering

MerchantAI automatically groups visitor questions by theme, so you can see the most common areas of interest at a glance without reading individual transcripts. Topics are identified by analysing the language patterns across all conversations and clustering semantically similar questions together. You can use topic clustering to:

• Identify which product categories or policies generate the most questions
• Spot emerging trends in visitor interest before they become support volume spikes
• Prioritise which parts of your knowledge base to expand first

Topic clusters are visible in Analytics → Topics and update continuously as new conversations come in.

​

Missed-answer detection

A missed answer is any query where the agent could not return a confident response — either because the topic wasn’t covered in the knowledge base, or because the relevant content was too vague to produce a high-confidence answer. MerchantAI flags these automatically and collects them in Analytics → Missed Answers. Each flagged entry shows the original visitor query, the confidence score the agent assigned, and whether the conversation was escalated to a human. This list is your most direct input for improving the knowledge base: each entry represents a real question your visitors are asking that your agent currently cannot handle well.

​

Source gap review

Source gap review shows you which knowledge sources are being drawn on most frequently and which visitor queries are not being matched to any source at all. This is distinct from missed-answer detection: a source gap means the relevant content may not exist in your knowledge base yet, rather than being present but low-confidence.

How to read source gap data

• High-use sources — Pages or files that are being retrieved frequently. These are your most valuable knowledge assets. Keep them accurate and up to date.
• Low-use sources — Pages or files that are rarely retrieved. Consider whether they are indexed correctly, or whether the content is genuinely not relevant to visitor queries.
• Uncovered queries — Queries that did not match any source. These are the clearest indicator of content you should create or upload.

Source gap data is available in Analytics → Source Gaps.

​

Human follow-up queues

Every conversation that was handed over to a human is tracked in the follow-up queue, visible in Analytics → Handovers. The queue shows which conversations have been resolved, which are still open, and how long each one took to receive a response from your team. This data helps you monitor the volume of escalations, identify patterns in what is triggering handovers, and track your team’s response time. If handover volume is high for a specific topic, that is a strong signal to add more content to the knowledge base or adjust your routing rules for that topic.

​

Using analytics to improve your agent

Analytics is most valuable when you review it on a regular cadence and act on what you find. The following process works well for most teams: