You Sampled Eight Calls. The Floor Made Four Hundred.

I used to sit in a QA seat and listen to eight calls from a four-hundred-call day. We scored them. We coached the misses. We wrote the weekly report. It looked like we knew how the floor was running.

We didn’t. We knew how eight conversations went. The other three hundred and ninety-two used whatever the agent remembered, or whatever the person next to them said last week, or the version of the process that was in their head when they trained. Most of those calls were fine. Some weren’t. We only found the ones that landed in the sample.

Sampling is a good way to coach people. It is a terrible way to find out if your knowledge is working.

What the sample actually shows

A failed call in QA usually gets treated as a person problem. The agent didn’t follow the process. So you play the call back, you walk them through the steps, you ask them to confirm they understand. Next week’s sample is a different eight people. The original miss never shows up again, which everyone takes as a win.

Sometimes it is. Sometimes the same miss is happening twenty times a day and you just didn’t pull those calls. The process page is unclear. Two articles disagree. The exception everyone uses isn’t written down. The update went live and half the floor didn’t read it. We’ve written about that last one.

QA can’t see any of that from eight recordings. It sees one agent, one call, one score. Then it writes a coaching note and moves on.

In regulated work that gap matters more than the score. Two customers, same situation, different outcomes. The sample might catch one of them. Risk will eventually ask about the ones it didn’t.

The floor already tells you, if you look somewhere else

The knowledge base sees the whole day. Not a sample. Search analytics show everyone who searched, everyone who opened an article, everyone who typed a question and got nothing back.

Failed searches are the honest list. They are the questions the floor asked and couldn’t answer, in the words they actually used. We’ve gone through those signals in more detail. You do not need to guess what to write next. The empty results already named it.

Low-rated articles are the other tell. If people keep opening a page and marking it unhelpful, that page is in the live process whether or not it appeared in this week’s sample. Coaching the agent who followed it is treating the symptom.

None of this replaces listening to calls. A recording still tells you things a search log never will: tone, the bit they skipped, the moment they guessed. Use both. One shows you a person. The other shows you the system they were standing in.

What actually changes the other 392

Write the thing the sample keeps finding. If three agents missed the hardship steps this month, the article is the problem, not the third agent. Fix the page. Make it the only current version. Put a review date on it so it doesn’t quietly rot.

Then look at whether people can find it. If search returns two near-identical articles, you haven’t given the floor a process. You’ve given them a coin toss. Kill the extras.

Required reading still only proves they were asked. It does not prove the next call will be right. A tick is not understanding. It does stop you pretending a Teams post was a rollout.

The honest bit

I am not going to tell you to stop sampling. You shouldn’t. A contact centre that never listens to calls is flying blind in a different way. Coaching still changes behaviour in a way a notification never will.

What sampling cannot do is stand in for a knowledge system. Eight calls are not a census. If the only time you find out the process is broken is when it lands in QA, you are finding it late, and you are finding it as a people problem when it is often a page problem.

The job is to use the sample to see the person, and the knowledge base to see the floor. I built KnowledgeScout for the second half of that. The first half is still you, with headphones on, doing the work a tool has no business claiming it can do.

The KnowledgeScout Team