Best Practice: Humanification of Test Sets
Humanification in Botium stands for simulation of human behaviour or habits. Read on to know more.
Why Humanification ?
BotiumScript makes it is easy to verify the chatbot’s ability to follow a conversation flow. It is an important step to recognize the need for automation in this area. But in real world, you cannot expect human users to act like a computer script:
- typographic errors are introduced
- different typing speeds
- sausage finger syndrom
With Botiums Humanification layer it is possible to evaluate how your chatbot deals with typical human typing habits.
Humanification is an additional layer on your existing test cases in Botium Box. Our recommendation is to use your own, clean test cases for automated testing, and from time to time use the humanification layer to verify your chatbots ability to handle those common input situations.
- Botium Box Enterprise Plan
- Clean Test Set
Step by step
Add Humanification Layer to Test Set
- Open the detail view of your test set to apply the humanification
- Go to the Transformation Wizard an open the Humanification Transformer
- Select from the list of Humanification Algorithms you want to apply (see below for details)
- You can set a percentage of how many of the utterances will undergo the humanification.
- Click on Download Test Set and Apply Humanification to clone the test set and apply the selected humanification algorithm(s).
All the humanification algorithms will only be applied to utterances, not convo files.
Evaluate cloned test set
The test set has been cloned to a shared folder in your default folder location. It can be used as any other test set now. If you want to apply additional humanification, you will have to do the clone process again.
Here is an example for a clean utterances list:
And here how it looks like with the humanification layer:
- Case sensitivity - will convert everything to lower case
- Duplicate space - will randomly duplicate usage of space key
- Emojis - will randomly insert some emojis
- Missing punctuation mark - will randomly remove punctuation marks
Common Typographic Errors
- Double character - will randomly duplicate a character
- Flip character - will randomly flip some nearby characters
- Homophones - will randomly replace with some similar sounding words
- Mishit - will randomly replace with a key located nearby on the keyboard
- Missing character - will randomly omit some characters
- Mixing character - will randomly mix up some characters
- qwertz vs qwerty keyboard - will randomly replace mishits from different keyboard layouts
- Sausage fingers - will randomly insert additional key strokes nearby on the keyboard
- Shift characters - will randomly assume the shift key has/has not been pressed at the beginning of a word
Other Humanification Options
Botium Box provides additional humanification options to make your test cases appear human.
Delay user input with PAUSE
PAUSE is one of the integrated logic hooks, which will just wait for a defined amount of time. See here for details.
Call Me Captain A simple Test Case #me hello PAUSE 5000
When using this, your test cases will run in real time and will be significantly slower.
Simulate human writing speed
The capability SIMULATE_WRITING_SPEED simulates human typing speed: the longer the text to send to the chatbot, the more time it will take - just as if a human is typing.
Average typing speed is about 290 ms/keystroke