Howto develop your own Botium Connector
Connectors are the bridge between the generalized Botium Core, and a specialized chatbot.
Botium supports many connectors, even a general purpose REST based connector. But if you dont find a proper connector, you can write a new one.
If you need a quickstart, you can try Connector skeleton for sync Chatbot API section
Important
For low to medium complex HTTP/JSON APIs it is typically more easy to extend the Generic HTTP/JSON connector instead of developing your own Botium Connector from scratch - see Howto develop your own HTTP/JSON Botium Connector
Generate a Boilerplate
The Botium CLI includes an option to generate boilerplate code for you. Run this command to generate a boilerplate project in the current directory, including a simple Echo connector, a botium.json and a sample convo file:
> botium-cli init-dev connector
Lets build the Echo Bot Connector
This Echo Bot Connector does not connects to any Chatbot API, just sends the text back to user without change.
The simpliest solution is putting the connector into Botium Config.
Capabilities: {
...
CONTAINERMODE: ({ queueBotSays }) => {
return {
UserSays (msg) {
const botMsg = { messageText: msg.messageText }
queueBotSays(botMsg)
}
}
}
}
queueBotSays
is a function to send the generalized bot message to
Botium Core.
UserSays
is the only required function of the created Object. It is
called if the user (Botium Core, emulating user messages) sends a
message
botMsg
is the generalized message. It has no required fields.
Note
We wont run this example. Botium CLI, Botium Bindings, and Botium Box works with json config (botium.json) and it is not possible to put function into this json. And it is good, is better to keep code and config separated.
(You can check how we are creating connector using capabilities in a unit test of Botium Core)
Separate code and config
There are two ways to separate them. You have to choose depending on your needs.
Simplier way, storing connector in your project
This is the simplier way.
It is good
-
if you want just to play with connectors,
-
if your connector belongs to the project logical, it wont be used otherwhere
You can do it in two steps:
- Create the Connector in separate file.
(example:
./ConnectorAsFile.js
):
class ConnectorAsFile {
constructor ({ queueBotSays }) {
this.queueBotSays = queueBotSays
}
UserSays (msg) {
const botMsg = { messageText: msg.messageText }
setTimeout(() => this.queueBotSays(botMsg), 0)
}
}
module.exports = {
PluginVersion: 1,
PluginClass: ConnectorAsFile
}
- Reference it from your ./botium.json
Capabilities: {
...
"CONTAINERMODE": "ConnectorAsFile.js"
}
And we have first example which works using Botium CLI, Botium Bindings, and Botium Box. You can find it there.
Sharable, better maintainable way, storing connector in npm package
If you put your Connector into an npm package, and add it to your
project, then you can use it simply using npm package name as container
mode in botium.json
:
"CONTAINERMODE": "<your module name>"
(If you use a Botium Connector, then you are doing the same. If you set “CONTAINERMODE” to “luis” Botium Core tries to load “luis“ module first. If it is not found then “botium-connector-” prefix, and tries again.)
Sync and Async Chatbot API
Connector skeleton for sync Chatbot API
If your Chatbot API is sync then you can keep your connector simple. (Chatbot API is sync if it sends the bot response as HTTP response)
Sync Chatbot API is passive, can just react to user message.
class MyCustomConnector {
constructor ({ queueBotSays, caps }) {
this.queueBotSays = queueBotSays
this.caps = caps
}
UserSays (msg) {
const requestObject = this._msgToRequestObject(msg)
const responseObject = this._doRequestChatbotApi(requestObject)
const botMsg = this._responseObjectToMsg(responseObject)
console.log(`MyCustomConnector: ${msg.messageText} => ${botMsg.messageText}`)
setTimeout(() => this.queueBotSays(botMsg), 0)
}
_msgToRequestObject (msg) {
// TODO convert generic msg to chatbot specific requestObject
return msg.messageText
}
_doRequestChatbotApi (requestObject) {
// TODO request the Chatbot API using chatbot specific requestO0bject
// and return bot response as responseObject
return (this.caps.MYCUSTOMCONNECTOR_PREFIX || '') + requestObject
}
_responseObjectToMsg (msg) {
// TODO convert chatbot specific requestObject to generic msg
return { messageText: msg }
}
}
module.exports = {
PluginVersion: 1,
PluginClass: MyCustomConnector
}
(setTimeout(() => this.queueBotSays(botMsg), 0)
is required just if
the Chatbot API is sync. UserSays()
must be finished before connector
sends the response)
Async chatbot API
Chatbot API is async, if the user and the bot are equal. Every side can send a message everytime.
Async Chatbot can communicate with Connector via Webhook, or via WebSocket for example.
They are requiring more complex architecture. You can use Botium Connector for Facebook Messenger Bots as example.
Using Capabilities
You can use Capabilities to add some parameters to your connector. The process is simple:
- Put a new capability into your
./botium.json
Capabilities: {
...
"CONTAINERMODE": "ConnectorAsFile.js",
"CONNECTOR_AS_FILE_RESPONSE": "Hello World!"
}
(You can name a capability as you want, just dont use existing name)
- And use it from connector:
const Capabilities = {
CONNECTOR_AS_FILE_RESPONSE: 'CONNECTOR_AS_FILE_RESPONSE'
}
class ConnectorAsFile {
// 2: catching a new 'caps' parameter.
// you got all capabillities here, not just the ones belonging to your connector
constructor ({ queueBotSays, caps }) {
this.queueBotSays = queueBotSays
this.caps = caps
}
Validate () {
// 3: Checking its validity
if (!this.caps[Capabilities.CONNECTOR_AS_FILE_RESPONSE]) throw new Error('CONNECTOR_AS_FILE_RESPONSE capability required')
return Promise.resolve()
}
UserSays (msg) {
// 4: Using it
const botMsg = { messageText: this.caps[Capabilities.CONNECTOR_AS_FILE_RESPONSE] }
setTimeout(() => this.queueBotSays(botMsg), 0)
}
}
module.exports = {
PluginVersion: 1,
PluginClass: ConnectorAsFile
}
You can find it here.
Other functions of a connector
Examples are from Botium Dialogflow Connector.
Build
Before starting the test, we can do some preparations. Initializing a library to communicate with bot, or create a parameter to the next steps.
Build () {
debug('Build called') // Botium uses debug library for debug messages
this.sessionOpts = {
...
}
return Promise.resolve()
}
This function is executed asynchron, it has to return Promise
Start
It is executed before every conversation. Use it to create new session on client. Or force the server to doing it (for example with generating new userid)
Start () {
debug('Start called')
this.sessionClient = new dialogflow.SessionsClient(this.sessionOpts)
...
return Promise.all(...)
}
This function is executed asynchron, it has to return Promise
Stop
Pair of the Start. Use it to clear session-like variables, close connections created in Start.
Stop () {
debug('Stop called')
this.sessionClient = null
...
return Promise.resolve()
}
This function is executed asynchron, it has to return Promise
Clean
Pair of the Build. Use it to clear variables, close connections created in Build.
Clean () {
debug('Clean called')
this.sessionOpts = null
return Promise.resolve()
}
This function is executed asynchron, it has to return Promise
The exported fields of the connector
class MyCustomConnector {
...
}
module.exports = {
PluginVersion: 1,
PluginClass: MyConnector,
PluginDesc: {
name: 'XXX connector',
avatar: '<Base64 encoded png>',
provider: 'My company',
capabilities: [
{
name: '<connector_name>_URL',
label: 'URL',
description: 'Chatbot endpoint url',
type: 'url',
required: true
}
],
features: {
intentResolution: true,
intentConfidenceScore: true,
alternateIntents: true,
entityResolution: true,
entityConfidenceScore: true,
testCaseGeneration: true,
securityTesting: true
}
}
}
PluginVersion
The version of the Connector. Required.
PluginClass
The connector class. Required.
PluginDesc
Optional.
name
The user readable name of the connector. Optional.
avatar
Base64 encoded png. Optional.
provider
Manufacturing company / The author. Optional
helperText
The description of the connector. Optional.
capabilities
Descriptor for UI. Used to display specialized form to edit the Connector Capabilities in Botium Box.
For example if your connector requires an URL field:
capabilities: [
{
name: 'MYCONNECTOR_URL',
label: 'URL',
description: 'Chatbot endpoint url',
type: 'url',
required: true
},
Note
If you dont need any capability then set capabilities to empty array.
In the case you dont set this field at all, or to null, the general
purpose Capability Editor will be displayed for your connector.
Othewise you have to use the general purpose Advanced Mode. Optional.
capabilities.name
The name of the capability. The key in the caps object. Required.
Note
It is a convention, that the capability names are starting with
connector name. Connector name is usually the postfix of the connector
file, or directory name.
For example connector name is “myconnector” for
“botium-connector-myconnector.js” so the capability names are starting
with MY_CONNECTOR
capabilities.label
The label on the form. Required.
capabilities.description
The description of the capability. Optional.
capabilities.type
The type of the field. Possible values are:
-
string
-
url
-
inboundurl
-
int
-
boolean
-
json
-
dictionary
-
choice
-
query
-
secret
Note
query type is not supported using custom connector with Botium Box.
Required.
capabilities.required
Set it to true
if the field is required. Optional.
capabilities.advanced
Set it to true
if the field is not required, and should not be
displayed when creating a connector. Optional.
capabilities.choices
For type choice, the list of choices to present. Array of JSON Objects with key and name attributes.
{
name: 'LUIS_PREDICTION_ENDPOINT_SLOT',
label: 'LUIS Prediction Endpoint Slot',
description: '"staging" or "production"',
type: 'choice',
required: false,
choices: [
{ key: 'staging', name: 'Staging' },
{ key: 'production', name: 'Production' }
]
}
capabilities.query
For type query, a function returning choice options
{
name: 'LEX_PROJECT_NAME',
label: 'Name of the Lex Bot (project name)',
type: 'query',
required: true,
query: async (caps) => {
if (caps && caps.LEX_ACCESS_KEY_ID && caps.LEX_SECRET_ACCESS_KEY && caps.LEX_REGION) {
const client = new AWS.LexModelBuildingService({
apiVersion: '2017-04-19',
region: caps.LEX_REGION,
accessKeyId: caps.LEX_ACCESS_KEY_ID,
secretAccessKey: caps.LEX_SECRET_ACCESS_KEY
})
const response = await client.getBots({ maxResults: 50 }).promise()
if (response.bots && response.bots.length > 0) {
return response.bots.map(b => ({
key: b.name,
name: b.name,
description: b.description
}))
}
}
}
},
features
Hint about the supported features. Optional
features.intentResolution
NLP feature, Intent resolution. Default is false.
features.intentConfidenceScore
NLP feature, Intent resolution with confidence score. Default is false.
features.alternateIntents
NLP feature, Intent resolution with alternative intent list. Default is false.
features.entityResolution
NLP feature, Entity resolution. Default is false.
features.entityConfidenceScore
NLP feature, Entity resolution with confidence score. Default is false.
features.testCaseGeneration
Ability to generate test cases. Default is false.
features.testCaseExport
Ability to export test cases. Default is false.
features.securityTesting
The connector allows security testing (the Chatbot Engine works with a proxy between). Default is false.
features.audioInput
Can handle audio input. Default is false.
features.sendAttachments
Can handle file attachments. Default is false.
features.supportedFileExtensions
Array of allowed file extensions. Default is to allow all extensions.
Example:
supportedFileExtensions: ['.wav', '.pcm', '.m4a', '.flac', '.riff', '.wma', '.aac', '.ogg', '.oga', '.mp3', '.amr']
The incoming message
Incoming message is the generalized form of a user-to-bot messages. Connector has to convert it to a request to the Chatbot API.
We already know that they are coming from Botium Core to the Connector
via theUserSays (msg)
function. But how is this msg
looks like?
{
"sender": "me",
"messageText": "message from user",
"buttons": [
{
"text": "Push me!",
"payload": "Push me!"
}
],
"media": [
{
"mediaUri": "media url",
"altText": "alt text"
}
]
}
Typically it contains a message text, or one (pushed) button. For
buttons istext
andpayload
the same, choose which fits better for
you.
Notize that the message contains much more fields. They are special fields. You dont need them to write an average connector:
-
header - convo header (header.name contains the test case name)
-
conversation - all conversation steps from a convo file
-
currentStepIndex - the current step index from a convo file
-
scriptingMemory - a dictionary with current scripting memory values
The outgoing message
Outgoing message is the generalized form of a bot-to-user messages.
Connector has to create it from the response of the Chatbot API, and
send to Botium Core via queueBotSays(msg)
function.
It looks like this:
{
"sender": "bot",
"sourceData": "raw but user readabe format of the message received Chat"
"messageText": "Hello!",
"buttons": [
{
"text": "button text",
"payload": "button payload"
}
],
"media": [
{
"mediaUri": "media url",
"altText": "alt text"
}
],
"forms": [
{
"name": "userName",
"label": "Your name"
}
],
{
"cards": [
{
"text": "card text",
"subtext": "card subtext",
"content": "card content",
"image": {
"mediaUri": "card image media url",
"mimeType": "card image mime type",
"altText": "card image alt text"
},
"buttons": [
{
"text": "card button text",
"payload": "card button payload",
"imageUri": "card button image uri"
}
],
"media": [
{
"mediaUri": "card media media url",
"mimeType": "card media mime type",
"altText": "card media alt text"
}
],
"forms": [
{
"name": "userName",
"label": "Your name"
}
]
}
],
"nlp": {
"intent": {
"name": "intent name",
"confidence": 0.9999198913574219,
"incomprehension": true,
"intents": [
{
"name": "intent name",
"confidence": 0.9999198913574219
},
{
"name": "second intent name",
"confidence": 0.24462083876132967
}
]
},
"entities": [
{
"name": "genre",
"value": "jazz",
"confidence": 1
},
{
"name": "appliance",
"value": "music",
"confidence": 1
}
]
}
}
sourceData
is the raw, human readable (if possible) message get from
Chatbot API. It is just for user, he can get details about the
conversation.
buttons
, media
, nlp
, cards
fields are similar. You can extract
them if the Chatbot Provider supports them. (And you can check them with
corresponding
asserter
(cards has no asserter yet!))
mimeType
is optional, it is guessed from the extrensions of the
mediaUri
.
nlp.incomprehension
is an optional flag. Used for statistics only. An
intent will be i-dont-understand-intent if its name is ‘None’, or ‘none’
or if this flag is true.
Tips
Chatbot API response with more messages
Some Chatbot APIs are returning an array as responses. Someting like this, for example:
[
{
"messageText: "Choose these"
},
{
"buttons": ["button1", "button2"]
},
{
"messageText: "Or these"
},
{
"buttons": ["button3", "button4"]
}
]
It is up to you how you structure them. It can be converted to one message, two messages, or four messages.
If you convert it to one message, then your test conversation can be:
#bot
Choose these
Or these
BUTTONS button1, button2, button3, button4
You are loosing some information. Preferred way is to convert such array of responses into two messages.