Skip to content

Intent Confidence Deviation

Purpose

The confidence deviation is a measure for the bandwidth of the predicted confidence score for all the utterances of an intent. It is calculated as standard deviation of the confidence scores.

Alerts

A high deviation of confidence scores typically means a mixture of strong and weak confidence scores within the user examples of an intent. A standard deviation of 0.2 - 0.3 usually is acceptable. Everything above should be handled.

Actions

  • Move the average confidence score to a higher range by adding more specific user examples or by removing the less specific ones

  • Remove the user examples with the weakest confidence score or replace them with more specific instances