What are the "Scripting Settings" in Botium Box ?
In Botium Box, there are some settings hidden behind the “Scripting Settings” - what are they and how can I use them ?
Short question, long answer.
Text Matching mode and Normalize Text before Matching
The Text Matching mode is the logic to use for comparing the bot response to the utterance (see capability).
Let’s say, your bot is answering a question for an order number with this text:
Your order number is PO-15342
When using text matching mode Substring matching, this BotiumScript will succeed:
#bot Your order number is
And this one as well:
#bot order number
As long as the bot response includes the snippet from BotiumScript, it is fine.
There is also the option to switch off the case sensitivity.
Using the text matching mode wildcard, you can introduce simple wildcard matching in your assertions by using the asterisk - so this one will succeed:
#bot Your order number is *
And this one as well:
#bot Your order number is PO-*
There is also the option to switch off the case sensitivity, which is the default if nothing else is selected.
Using the text matching mode regular expressions, assertion only succeeds if the BotiumScript snippet interpreted as regular expression is matching the bot response.
For example, you can assert that the order number follows your conventions:
#bot Your order number is PO-[0-9]+
Recommending this tool to quick test regular expressions.
Using the Normalize Text before Matching option you can simplify the bot response before doing the assertions, to make it easier to compose regular expressions or substring patterns. HTML tags are removed, line breaks are removed, non printable characters are removed, … (for details, you can inspect the Botium Core code).
Your bot is now returning a more sophisticated answer:
Your order number is <a href="my-site/show/15342">PO-15342</a> > <a href="my-site/show/15342">Show PO</a> > <a href="my-site/show/15342/items">Show Items</>
The Substring matching examples from above will continue to work, but the regular expression certainly won’t. When using the Normalize Text before Matching option, this bot response is converted to this text before the matching is done:
Your order number is PO-15342 > Show PO > Show Items
No line breaks, no HTML tags, no HTML entities. The BotiumScript snippets from above will continue to work.
Expand Conversations and Convo Expansion Mode
If you are new to Botium and don’t know about convos, utterances and scripting memory, please read Botium in a nutshell first.
Enable the Expand Conversations option if your Test Set contains utterances files to be merged with the convo files. See this BotiumScript convo:
And this BotiumScript utterances:
HELLO_UTT hello hi what's up
If Expand Conversations is disabled, the literal text HELLO_UTT is sent to the bot (which is most likely not what you want).
If it is enabled, the Convo Expansion Mode comes into play to select the number of utterances to actually send to the bot - all, the first one, or randomly select some of them (see capability).
When having a simple two-step conversation, it is not a big deal to send all the utterances. But when you have more complex convo, the number of possible combinations increments very quick when letting Botium use all of them.
Expand Utterances to Conversations and Incomprehension Utterances
There are cases where you are not really interested in asserting conversations with your bot. Maybe you are just interested in sending as much text as possible to the bot (for training purposes), or you just want to make sure the bot understands all of the given texts.
When you enable the Expand Utterances to Conversations option, all utterances in the test set are converted to simple 2-step conversations (text sent to bot, bot answer received). For example, you have these utterances file:
HELLO_UTT hello hi what's up
Botium converts it into three convos:
#me hello #bot
#me hi #bot
#me what's up #bot
All three utterances are sent to the bot, and the answer is ignored.
If you want to make sure your bot actually understands what you sent, it is possible to specify the Incomprehension Utterances, the utterances your bot is returning in incomprehension cases.
For example, in addition to the HELLO_UTT above, you have an INCOMPREHENSION_UTT utterances list:
INCOMPREHENSION_UTT Sorry, I don't understand Can you please repeat ? Please say it another way
Now, when you enter INCOMPREHENSION_UTT in the Incomprehension Utterances field, Botium converts the test set into these three convos:
#me hello #bot !INCOMPREHENSION_UTT
#me hi #bot !INCOMPREHENSION_UTT
#me what's up #bot !INCOMPREHENSION_UTT
Now Botium makes sure that for every utterance from the HELLO_UTT list, the bot actually doesn’t return a “don’t understand”-like phrase.
Enable Scripting Memory and Scripting Memory Matching Mode
The Scripting Memory in Botium has several use cases and can be enabled with this option:
Extract and reuse text from a bot response
Use dynamic content in BotiumScript
Running a single convo multiple times with different parameters
The Scripting Memory Matching Mode option is for finetuning the text extraction logic - see capability.
Enable the Expand Scripting Memory option if you have scripting memory files for parameterizing your convos in the test set.