Skip to content

2019–11–12: Botium Box v2.0 available now

We finally published Botium Box 2.0 - it is really exciting to have now finished all stories we considered as relevant for providing robust chatbot test automation a year ago.

We started a blog series about Quality Metrics for NLU/Chatbot Training Data and describe how Botium Box can be used to analyze the performance of your NLU engine - more to come in future Botium Box releases!

We added 3 new connectors (for Whatsapp and Facebook developers, and for HTTP inbound traffic). Our connectors are now capable of dealing with composite entities. With our Directline connector you can now control the activity structure in detail.

With Botium Box 2.0, we now officially support deployments to Kubernetes clusters. We added a remote file system browser mainly for Kubernetes cloud users without direct filesystem access. Also it is possible to define system environment variables on chatbot and test project level.

A new dashboard widget shows Botium Box testing activity. Quick connectivity checks are now possible with the new “Say Hello” function.

New Documentation and Tutorials:

Includes Botium Core 1.7.3

Breaking Changes

  • Custom Hooks are not used for running external programs, but for launching NPM modules or custom Javascript

Added

  • #412 Added support for YAML and JSON convo files
  • #413 Added support for multiple JSON responses in the Generic HTTP/JSON Connector, see here
  • #423 JSONPath Asserter matching for non-string values - contributed by Sushank Rana
  • #419 Additional capabilities for Generic HTTP/JSON asserter (Ping headers, start hook, stop hook) - see here
  • #412 Generic HTTP/JSON connector now supports inbound messages by launching an own HTTP/JSON endpoint - see here
  • #422 Custom hooks to manipulate ingoing and outgoing messages on Botium Core level - see here

Changed

  • #404 Renamed library opn to open - contributed by Gintautas Miselis
  • #417 Cleanup Temp Directory on Build error
  • #420 UPDATE_CUSTOM logic hook can now be used as global logic hook, see here

Botium CLI and Botium Bindings

  • BOT-336 Botium CLI can now export screenshots from Webdriver test cases
  • BOT-857 Added boilerplate project generator for Botium connector development
  • Test Results showing Actual vs Expected chatbot responses

Botium Connectors

  • We built a new Botium connector for Whatsapp Business API Webhooks
  • We built a new Botium connector for Facebook Messenger Platform Webhooks
  • We built a new Botium connector for generic JSON-based Websocket endpoints
  • We updated our Rasa connector
    • BOT-741 Support for Rasa NLU
    • Option to import Rasa NLU file and generate test cases out of it
  • We updated our Directline3 connector
    • BOT-886 support for deeply nested button click and form entry simulation
    • BOT-920 support for activity templates for special cases
  • We updated our Wit.ai connector
    • BOT-865 added support for composite entities
  • We updated our SAP Conversational AI connector
  • We updated our Generic HTTP/JSON connector
    • BOT-881 added support for inbound HTTP calls
  • We updated our Dialogflow connector
    • BOT-860 Dialogflow query parameter manipulation
    • BOT-828 added support for composite entities
    • BOT-883 added support for incomprehension intent
  • We updated our Microsoft LUIS connector
    • BOT-864 added support for composite entities

New in Conversation Testing

New in Botium Box:

  • BOT-805 Test Set statistics showing utterances lists
  • BOT-840 added support for YAML and JSON files
  • BOT-688 we added a file browser for the Botium volumes to Botium Box - especially important for cloud deployments without direct file system access
  • BOT-798 As quick connectivity chat we now have a “Say Hello” function, to, well, say “hello” to your chatbot
  • BOT-916 New dashboard widget to show Botium Box activity
  • BOT-875 Easier Botium Box agent handling by automatically removing offline agents from the list *

Infrastructure Enhancements

  • BOT-822 Register Chatbot- or Test Project specific system environment variables (used in Live Chat as well BOT-821). Handy when dealing with HTTP(S) proxies or invalid SSL certificates

  • BOT-858 added resources directory to NODE_PATH to load custom modules

  • Only available in Botium Platform (enterprise plan)