Skip to content

Intent Confidence Risks

Purpose

This charts shows the intents of your test set with the weakest average confidence score:

  • All test cases which resolved to an intent are collected, together with the confidence score

  • For all intents, the average confidence score is calculated

A low average confidence score is an indicator that

  • either there are single utterances with very low confidence score having high impact on the average, due to the lack of training data for an intent

  • or for a large amount of utterances the NLU engine is not able to clearly decide for an intent

Alerts

If the average confidence score falls below a threshold (depending on the NLU engine typically between 0.4 and 0.6), this is a sign that the training data has to be balanced and/or sharpened for the intents under question.

Actions

Investigate the details for the intents with low average confidence score.

  • In case there are single utterances having a high impact on the average, there is either additional training data needed for this intent, or the utterance is not specific enough and can be removed from the training data

  • Otherwise it is an indicator that the training data in general is not specific enough for the NLU engine, and it should be sharpened and enlarged by providing additional user examples for all weak intents.