Skip to content

Howto develop your own Botium Connector

Connectors are the bridge between the generalized Botium Core, and a specialized chatbot.

Botium supports many connectors, even a general purpose REST based connector. But if you dont find a proper connector, you can write a new one.

If you need a quickstart, you can try Connector skeleton for sync Chatbot API section

Important

For low to medium complex HTTP/JSON APIs it is typically more easy to extend the Generic HTTP/JSON connector instead of developing your own Botium Connector from scratch - see Howto develop your own HTTP/JSON Botium Connector

Generate a Boilerplate

The Botium CLI includes an option to generate boilerplate code for you. Run this command to generate a boilerplate project in the current directory, including a simple Echo connector, a botium.json and a sample convo file:

> botium-cli init-dev connector

Lets build the Echo Bot Connector

This Echo Bot Connector does not connects to any Chatbot API, just sends the text back to user without change.

The simpliest solution is putting the connector into Botium Config.

Capabilities: {
  ...
  CONTAINERMODE: ({ queueBotSays }) => {
    return {
      UserSays (msg) {
        const botMsg = { messageText: msg.messageText }
        queueBotSays(botMsg)
      }
    }
  }
}

queueBotSays is a function to send the generalized bot message to Botium Core.
UserSays is the only required function of the created Object. It is called if the user (Botium Core, emulating user messages) sends a message
botMsg is the generalized message. It has no required fields.

Note

We wont run this example. Botium CLI, Botium Bindings, and Botium Box works with json config (botium.json) and it is not possible to put function into this json. And it is good, is better to keep code and config separated.

(You can check how we are creating connector using capabilities in a unit test of Botium Core)

Separate code and config

There are two ways to separate them. You have to choose depending on your needs.

Simplier way, storing connector in your project

This is the simplier way.

It is good

  • if you want just to play with connectors,

  • if your connector belongs to the project logical, it wont be used otherwhere

You can do it in two steps:

  • Create the Connector in separate file. (example:./ConnectorAsFile.js):
class ConnectorAsFile {
  constructor ({ queueBotSays }) {
    this.queueBotSays = queueBotSays
  }

  UserSays (msg) {
    const botMsg = { messageText: msg.messageText }
    setTimeout(() => this.queueBotSays(botMsg), 0)
  }
}

module.exports = {
  PluginVersion: 1,
  PluginClass: ConnectorAsFile
}
  • Reference it from your ./botium.json
Capabilities: {
  ...
  "CONTAINERMODE": "ConnectorAsFile.js"
}

And we have first example which works using Botium CLI, Botium Bindings, and Botium Box. You can find it there.

Sharable, better maintainable way, storing connector in npm package

If you put your Connector into an npm package, and add it to your project, then you can use it simply using npm package name as container mode in botium.json:

"CONTAINERMODE": "<your module name>"

(If you use a Botium Connector, then you are doing the same. If you set “CONTAINERMODE” to “luis” Botium Core tries to load “luis“ module first. If it is not found then “botium-connector-” prefix, and tries again.)

Sync and Async Chatbot API

Connector skeleton for sync Chatbot API

If your Chatbot API is sync then you can keep your connector simple. (Chatbot API is sync if it sends the bot response as HTTP response)

Sync Chatbot API is passive, can just react to user message.

class MyCustomConnector {
  constructor ({ queueBotSays, caps }) {
    this.queueBotSays = queueBotSays
    this.caps = caps
  }

  UserSays (msg) {
    const requestObject = this._msgToRequestObject(msg)
    const responseObject = this._doRequestChatbotApi(requestObject)
    const botMsg = this._responseObjectToMsg(responseObject)
    console.log(`MyCustomConnector: ${msg.messageText} => ${botMsg.messageText}`)
    setTimeout(() => this.queueBotSays(botMsg), 0)
  }

  _msgToRequestObject (msg) {
    // TODO convert generic msg to chatbot specific requestObject
    return msg.messageText
  }

  _doRequestChatbotApi (requestObject) {
    // TODO request the Chatbot API using chatbot specific requestO0bject
    // and return bot response as responseObject
    return (this.caps.MYCUSTOMCONNECTOR_PREFIX || '') + requestObject
  }

  _responseObjectToMsg (msg) {
    // TODO convert chatbot specific requestObject to generic msg
    return { messageText: msg }
  }
}

module.exports = {
  PluginVersion: 1,
  PluginClass: MyCustomConnector
}

(setTimeout(() => this.queueBotSays(botMsg), 0) is required just if the Chatbot API is sync. UserSays() must be finished before connector sends the response)

Async chatbot API

Chatbot API is async, if the user and the bot are equal. Every side can send a message everytime.

Async Chatbot can communicate with Connector via Webhook, or via WebSocket for example.

They are requiring more complex architecture. You can use Botium Connector for Facebook Messenger Bots as example.

Using Capabilities

You can use Capabilities to add some parameters to your connector. The process is simple:

  • Put a new capability into your ./botium.json
Capabilities: {
  ...
  "CONTAINERMODE": "ConnectorAsFile.js",
  "CONNECTOR_AS_FILE_RESPONSE": "Hello World!" 
}

(You can name a capability as you want, just dont use existing name)

  • And use it from connector:
const Capabilities = {
  CONNECTOR_AS_FILE_RESPONSE: 'CONNECTOR_AS_FILE_RESPONSE'
}


class ConnectorAsFile {
  // 2: catching a new 'caps' parameter.
  // you got all capabillities here, not just the ones belonging to your connector
  constructor ({ queueBotSays, caps }) {
    this.queueBotSays = queueBotSays
    this.caps = caps
  }

  Validate () {
    // 3: Checking its validity
    if (!this.caps[Capabilities.CONNECTOR_AS_FILE_RESPONSE]) throw new Error('CONNECTOR_AS_FILE_RESPONSE capability required')
    return Promise.resolve()
  }

  UserSays (msg) {
    // 4: Using it 
    const botMsg = { messageText: this.caps[Capabilities.CONNECTOR_AS_FILE_RESPONSE] }
    setTimeout(() => this.queueBotSays(botMsg), 0)
  }
}

module.exports = {
  PluginVersion: 1,
  PluginClass: ConnectorAsFile
}

You can find it here.

Other functions of a connector

Examples are from Botium Dialogflow Connector.

Build

Before starting the test, we can do some preparations. Initializing a library to communicate with bot, or create a parameter to the next steps.

  Build () {
    debug('Build called') // Botium uses debug library for debug messages
    this.sessionOpts = {
      ...
    }
    return Promise.resolve()
  }

This function is executed asynchron, it has to return Promise

Start

It is executed before every conversation. Use it to create new session on client. Or force the server to doing it (for example with generating new userid)

  Start () {
    debug('Start called')

    this.sessionClient = new dialogflow.SessionsClient(this.sessionOpts)
    ...
    return Promise.all(...)
  }

This function is executed asynchron, it has to return Promise

Stop

Pair of the Start. Use it to clear session-like variables, close connections created in Start.

  Stop () {
    debug('Stop called')
    this.sessionClient = null
    ...
    return Promise.resolve()
  }

This function is executed asynchron, it has to return Promise

Clean

Pair of the Build. Use it to clear variables, close connections created in Build.

  Clean () {
    debug('Clean called')
    this.sessionOpts = null
    return Promise.resolve()
  }

This function is executed asynchron, it has to return Promise

The exported fields of the connector

class MyCustomConnector {
  ...
}

module.exports = {
  PluginVersion: 1,
  PluginClass: MyConnector,
  PluginDesc: {
    name: 'XXX connector',
    avatar: '<Base64 encoded png>',
    provider: 'My company',
    capabilities: [
      {
        name: '<connector_name>_URL',
        label: 'URL',
        description: 'Chatbot endpoint url',
        type: 'url',
        required: true
      }
    ],
    features: {
      intentResolution: true,
      intentConfidenceScore: true,
      alternateIntents: true,
      entityResolution: true,
      entityConfidenceScore: true,
      testCaseGeneration: true,
      securityTesting: true
     }
   }
}

PluginVersion

The version of the Connector. Required.

PluginClass

The connector class. Required.

PluginDesc

Optional.

name

The user readable name of the connector. Optional.

avatar

Base64 encoded png. Optional.

provider

Manufacturing company / The author. Optional

helperText

The description of the connector. Optional.

capabilities

Descriptor for UI. Used to display specialized form to edit the Connector Capabilities in Botium Box.

For example if your connector requires an URL field:

    capabilities: [
      {
        name: 'MYCONNECTOR_URL',
        label: 'URL',
        description: 'Chatbot endpoint url',
        type: 'url',
        required: true
      },

Note

If you dont need any capability then set capabilities to empty array.
In the case you dont set this field at all, or to null, the general purpose Capability Editor will be displayed for your connector.

Othewise you have to use the general purpose Advanced Mode. Optional.

capabilities.name

The name of the capability. The key in the caps object. Required.

Note

It is a convention, that the capability names are starting with connector name. Connector name is usually the postfix of the connector file, or directory name.
For example connector name is “myconnector” for “botium-connector-myconnector.js” so the capability names are starting with MY_CONNECTOR

capabilities.label

The label on the form. Required.

capabilities.description

The description of the capability. Optional.

capabilities.type

The type of the field. Possible values are:

  • string

  • url

  • inboundurl

  • int

  • boolean

  • json

  • dictionary

  • choice

  • query

  • secret

Note

query type is not supported using custom connector with Botium Box.

Required.

capabilities.required

Set it to true if the field is required. Optional.

capabilities.advanced

Set it to true if the field is not required, and should not be displayed when creating a connector. Optional.

capabilities.choices

For type choice, the list of choices to present. Array of JSON Objects with key and name attributes.

      {
        name: 'LUIS_PREDICTION_ENDPOINT_SLOT',
        label: 'LUIS Prediction Endpoint Slot',
        description: '"staging" or "production"',
        type: 'choice',
        required: false,
        choices: [
          { key: 'staging', name: 'Staging' },
          { key: 'production', name: 'Production' }
        ]
      }

capabilities.query

For type query, a function returning choice options

      {
        name: 'LEX_PROJECT_NAME',
        label: 'Name of the Lex Bot (project name)',
        type: 'query',
        required: true,
        query: async (caps) => {
          if (caps && caps.LEX_ACCESS_KEY_ID && caps.LEX_SECRET_ACCESS_KEY && caps.LEX_REGION) {
            const client = new AWS.LexModelBuildingService({
              apiVersion: '2017-04-19',
              region: caps.LEX_REGION,
              accessKeyId: caps.LEX_ACCESS_KEY_ID,
              secretAccessKey: caps.LEX_SECRET_ACCESS_KEY
            })
            const response = await client.getBots({ maxResults: 50 }).promise()
            if (response.bots && response.bots.length > 0) {
              return response.bots.map(b => ({
                key: b.name,
                name: b.name,
                description: b.description
              }))
            }
          }
        }
      },

features

Hint about the supported features. Optional

features.intentResolution

NLP feature, Intent resolution. Default is false.

features.intentConfidenceScore

NLP feature, Intent resolution with confidence score. Default is false.

features.alternateIntents

NLP feature, Intent resolution with alternative intent list. Default is false.

features.entityResolution

NLP feature, Entity resolution. Default is false.

features.entityConfidenceScore

NLP feature, Entity resolution with confidence score. Default is false.

features.testCaseGeneration

Ability to generate test cases. Default is false.

features.testCaseExport

Ability to export test cases. Default is false.

features.securityTesting

The connector allows security testing (the Chatbot Engine works with a proxy between). Default is false.

features.audioInput

Can handle audio input. Default is false.

features.sendAttachments

Can handle file attachments. Default is false.

features.supportedFileExtensions

Array of allowed file extensions. Default is to allow all extensions.

Example:

supportedFileExtensions: ['.wav', '.pcm', '.m4a', '.flac', '.riff', '.wma', '.aac', '.ogg', '.oga', '.mp3', '.amr']

The incoming message

Incoming message is the generalized form of a user-to-bot messages. Connector has to convert it to a request to the Chatbot API.

We already know that they are coming from Botium Core to the Connector via theUserSays (msg) function. But how is this msglooks like?

{
  "sender": "me",
  "messageText": "message from user",
  "buttons": [
    {
      "text": "Push me!",
      "payload": "Push me!"
    }
  ],
  "media": [
    {
      "mediaUri": "media url",
      "altText": "alt text"
    }
  ]
}

Typically it contains a message text, or one (pushed) button. For buttons istext andpayload the same, choose which fits better for you.

Notize that the message contains much more fields. They are special fields. You dont need them to write an average connector:

  • header - convo header (header.name contains the test case name)

  • conversation - all conversation steps from a convo file

  • currentStepIndex - the current step index from a convo file

  • scriptingMemory - a dictionary with current scripting memory values

The outgoing message

Outgoing message is the generalized form of a bot-to-user messages. Connector has to create it from the response of the Chatbot API, and send to Botium Core via queueBotSays(msg) function.

It looks like this:

{
  "sender": "bot",
  "sourceData": "raw but user readabe format of the message received  Chat"
  "messageText": "Hello!",
  "buttons": [
    {
      "text": "button text",
      "payload": "button payload"
    }
  ],
  "media": [
    {
      "mediaUri": "media url",
      "altText": "alt text"
    }
  ],
  "forms": [
    {
      "name": "userName",
      "label": "Your name"
    }
  ],
  {
  "cards": [
    {
      "text": "card text",
      "subtext": "card subtext",
      "content": "card content",
      "image": {
        "mediaUri": "card image media url",
        "mimeType": "card image mime type",
        "altText": "card image alt text"
      },
      "buttons": [
        {
          "text": "card button text",
          "payload": "card button payload",
          "imageUri": "card button image uri"
        }
      ],
      "media": [
        {
        "mediaUri": "card media media url",
        "mimeType": "card media mime type",
        "altText": "card media alt text"
        }
      ],
      "forms": [
        {
          "name": "userName",
          "label": "Your name"
        }
      ]
    }
  ],
  "nlp": {
    "intent": {
      "name": "intent name",
      "confidence": 0.9999198913574219,
      "incomprehension": true,
      "intents": [
        {
          "name": "intent name",
          "confidence": 0.9999198913574219
        },
        {
          "name": "second intent name",
          "confidence": 0.24462083876132967
        }
      ]
    },
    "entities": [
      {
        "name": "genre",
        "value": "jazz",
        "confidence": 1
      },
      {
        "name": "appliance",
        "value": "music",
        "confidence": 1
      }
    ]
  }
}

sourceData is the raw, human readable (if possible) message get from Chatbot API. It is just for user, he can get details about the conversation.

buttons, media, nlp, cards fields are similar. You can extract them if the Chatbot Provider supports them. (And you can check them with corresponding asserter (cards has no asserter yet!))

mimeTypeis optional, it is guessed from the extrensions of the mediaUri.

nlp.incomprehensionis an optional flag. Used for statistics only. An intent will be i-dont-understand-intent if its name is ‘None’, or ‘none’ or if this flag is true.

Tips

Chatbot API response with more messages

Some Chatbot APIs are returning an array as responses. Someting like this, for example:

[
  {
    "messageText: "Choose these"
  }, 
  {
    "buttons": ["button1", "button2"]
  }, 
  {
    "messageText: "Or these"
  }, 
  {
    "buttons": ["button3", "button4"]
  }

]

It is up to you how you structure them. It can be converted to one message, two messages, or four messages.

If you convert it to one message, then your test conversation can be:

#bot
Choose these
Or these
BUTTONS button1, button2, button3, button4

You are loosing some information. Preferred way is to convert such array of responses into two messages.

Using custom connector with Botium Box

See Howto deploy my own Botium Connector