Skip to content

Developing Custom Asserters

With some Javascript development know-how, you can extend Botium with your own asserter logic.

Asserter Hooks

There are currently three types of asserter hooks where you can add your custom logic:

  • before a convo begins (assertConvoBegin)

  • on a convo step to assert the bot response (assertConvoStep)

  • after convo ends (assertConvoEnd)

  • and for all of them you may provide special implementations for inverted assertion logic (assertNotConvoBegin, assertNotConvoStep, assertNotConvoEnd)

The logic is implemented as Javascript functions. Implementation of the hooks is optional, you just have to implement the hooks required by your logic. For the inverted logic, if you don’t explicitly implement the logic, a default logic is used (calling the positive logic, and expect a failure).

Note

When adding a global asserter to Botium Box (Register as global scripting component), the asserter hooks will be called on every convo and every convo step. For example, it maybe is a good a idea to use the Hyperlink Asserter in this way to have all your hyperlinks in all your test runs checked automatically.

Note

Please keep in mind, that the asserter hooks are blocking the test.

Some tips for better performance:

  • Put your time consuming initialization into constructor, or into the assertConvoBegin function.

  • Execute write processes (which have no influence on the test process) parallel.

Using Botium CLI to generate Skeleton Project

You can use the Botium CLI to generate a skeleton project with a sample custom asserter in the current directory:

> botium-cli init-dev asserter
> botium-cli run

Code Sample - Asserter Class Skeleton

Best practice is to place your custom asserter logic code in a Javascript file and export a Javascript class definition. As an example, place this in a file called CustomAsserter.js:

module.exports = class CustomAsserter {
  constructor (context, caps, globalArgs) {
    this.context = context
    this.caps = caps
    this.globalArgs = globalArgs
  }

  assertConvoBegin ({convo, args, isGlobal}) {
    return Promise.resolve()
  }

  assertConvoStep ({convo, convoStep, args, isGlobal, botMsg}) {
    return Promise.resolve()
  }

  assertConvoEnd ({convo, transcript, args, isGlobal}) {
    return Promise.resolve()
  }
}

Code Sample - Asserter Code

For short asserter logic it is possible to place the Javascript code directly in the component configuration:

{
  "botium": {
    "Capabilities": {
      ...
      "ASSERTERS": [
        {
          "ref": "MY-ASSERTER-NAME",
          "src": {
            "assertConvoStep": "if (botMsg.messageText !== 'hugo') throw 'expected hugo'"
          },
          "global": false,
          "args": {
            "my-arg-1": "something"
          }
        }
      ]
    }
  }
}

Asserter Hooks API

Class Constructor Arguments (only for class definitions)

  • context - Scripting Context with several Botium internal functions

  • caps - Botium capabilities

  • globalArgs - the component configuration (from the Component Configuration field in Botium Box, or from the args field in botium.json) as JSON structure

assertConvoBegin/assertNotConvoBegin Arguments

  • convo - current convo

  • args - the asserter args from the convo file (as JSON array - split by “|”)

  • scriptingMemory - scripting memory variables (a JSON object)

  • isGlobal - wether the asserter logic has been registered globally or not

  • container - Botium Connector container (provides access to internal connector data)

assertConvoStep/assertNotConvoStep Arguments

  • convo - see above

  • convoStep - the current convo step

  • args - see above

  • scriptingMemory - see above

  • isGlobal - see above

  • botMsg - the message from the bot

  • container - see above

assertConvoEnd/assertNotConvoEnd Arguments

  • convo - see above

  • transcript - the full conversation transcript

  • args - see above

  • scriptingMemory - see above

  • isGlobal - see above

  • container - see above

Deployment to Botium Box

There are 2 ways to deploy your custom asserter logic to Botium Box.

Option 1: NPM package

Build an NPM package, exporting a Javascript class holding your asserter logic (see the Botium Hyperlink Asserter as an example). You have to add the NPM package to your Botium Box server - unpack the NPM package to the resources directory - the package.json of your NPM package is in the resources/your-asserter-name directory - and run “npm install” there.

Note

An NPM package is nothing more than a ZIP-File. For Botium asserters hosted on Github, you can download the package as ZIP-file from Github (Clone or Download => Download as ZIP) and unpack them - make sure to rename the resulting directory to match the asserter package name (Github adds the -master suffix to the downloaded files)

Now you can use the Registered Components view to register your custom asserter, using your-asserter-name as Component Source.

Option 2: Shared Javascript file

You can just tell Botium Box to load the asserter code from a Javascript file. The Javascript file has to be available on all Botium Box servers and agents, obviously, by placing it in the resources directory.

Now you can use the Registered Components view to register your custom asserter, using your-asserter-filename.js as Component Source (the relative path in the resources directory).

Note

In booth cases you can use botium-asserter- prefix to automatically register your asserter.

Note

I booth cases asserters are loaded if plugins are enabled.

Deployment to botium.json

When using botium-cli or botium-bindings, you have to register your asserter in the botium.json file:

{
  "botium": {
    "Capabilities": {
      ...
      "ASSERTERS": [
        {
          "ref": "MY-ASSERTER-NAME",
          "src": "path-to-my-javascript-file-or-npm-package-name",
          "global": false,
          "args": {
            "my-arg-1": "something"
          }
        }
      ]
    }
  }
}

Using the Asserter

You trigger the asserter call by using the defined Component Ref Code of the asserter (in our example below: DUMMY). Everything after DUMMY are args separated by |

  • begin: Before the conversation starts

  • bot: after bot response

  • end: After the conversation finished

restaurant
#begin
DUMMY dbUrl | dbPassword | INSERT INTO dummy(name, birthday) VALUES ('Max Mustermann', 1991-03-26);

#me
hi

#bot
Hi. It looks like a nice drive today. What would you like me to do?

#me
where is the next restauran

#bot
I understand you want me to find a location. I can find restaurants, gas stations and restrooms nearby.


#me
find my a restaurant

#bot
Of course. Do you have a specific cuisine in mind?

#me
pizza

#bot
Super! I've found 5 locations for you. Which one would you like to drive to?
DUMMY arg1 | arg2

#me
1

#bot
Sure! Restaurant 1 on the list gets great reviews.

#bot
What day/time did you want to go to the restaurant?

#me
10th of january

#bot
OK

#end
DUMMY dbUrl | dbPassword | DELETE FROM dummy WHERE name='Max Mustermann';