2019–11–12: Botium Box v2.0 available now
We finally published Botium Box 2.0 - it is really exciting to have now finished all stories we considered as relevant for providing robust chatbot test automation a year ago.
We started a blog series about Quality Metrics for NLU/Chatbot Training Data and describe how Botium Box can be used to analyze the performance of your NLU engine - more to come in future Botium Box releases!
We added 3 new connectors (for Whatsapp and Facebook developers, and for HTTP inbound traffic). Our connectors are now capable of dealing with composite entities. With our Directline connector you can now control the activity structure in detail.
With Botium Box 2.0, we now officially support deployments to Kubernetes clusters. We added a remote file system browser mainly for Kubernetes cloud users without direct filesystem access. Also it is possible to define system environment variables on chatbot and test project level.
A new dashboard widget shows Botium Box testing activity. Quick connectivity checks are now possible with the new “Say Hello” function.
New Documentation and Tutorials:
- Installation to Kubernetes Cluster
- Quality Metrics for NLU/Chatbot Training Data, Part 1: Confusion Matrix
- Botium in a Nutshell, Part 9: Botium Test Levels or Selecting a Botium Connector
- Howto develop your own Botium connector
- and more in our Botium Wiki
Includes Botium Core 1.7.3
Breaking Changes
- Custom Hooks are not used for running external programs, but for launching NPM modules or custom Javascript
Added
- #412 Added support for YAML and JSON convo files
- #413 Added support for multiple JSON responses in the Generic HTTP/JSON Connector, see here
- #423 JSONPath Asserter matching for non-string values - contributed by Sushank Rana
- #419 Additional capabilities for Generic HTTP/JSON asserter (Ping headers, start hook, stop hook) - see here
- #412 Generic HTTP/JSON connector now supports inbound messages by launching an own HTTP/JSON endpoint - see here
- #422 Custom hooks to manipulate ingoing and outgoing messages on Botium Core level - see here
Changed
- #404 Renamed library opn to open - contributed by Gintautas Miselis
- #417 Cleanup Temp Directory on Build error
- #420 UPDATE_CUSTOM logic hook can now be used as global logic hook, see here
Botium CLI and Botium Bindings
- BOT-336 Botium CLI can now export screenshots from Webdriver test cases
- BOT-857 Added boilerplate project generator for Botium connector development
- Test Results showing Actual vs Expected chatbot responses
Botium Connectors
- We built a new Botium connector for Whatsapp Business API Webhooks
- We built a new Botium connector for Facebook Messenger Platform Webhooks
- We built a new Botium connector for generic JSON-based Websocket endpoints
- We updated our Rasa
connector
- BOT-741 Support for Rasa NLU
- Option to import Rasa NLU file and generate test cases out of it
- We updated our Directline3
connector
- BOT-886 support for deeply nested button click and form entry simulation
- BOT-920 support for activity templates for special cases
- We updated our Wit.ai
connector
- BOT-865 added support for composite entities
- We updated our SAP Conversational AI
connector
- BOT-901 added support for NLP/NLU testing
- We updated our Generic HTTP/JSON
connector
- BOT-881 added support for inbound HTTP calls
- We updated our Dialogflow
connector
- BOT-860 Dialogflow query parameter manipulation
- BOT-828 added support for composite entities
- BOT-883 added support for incomprehension intent
- We updated our Microsoft LUIS
connector
- BOT-864 added support for composite entities
New in Conversation Testing
- Added NLP Analytics to Botium Box, see this blog article about
Confusing Matrix and Quality
Metrics
- New asserter for analyzing and verifying the tone of a chatbot, based on Watson Tone Analyzer.
- BOT-885 Humanification with UTF-8 emojis
New in Botium Box:
- BOT-805 Test Set statistics showing utterances lists
- BOT-840 added support for
YAML
and
JSON
files
- BOT-688 we added a file browser for the Botium
volumes
to Botium Box - especially important for cloud deployments without
direct file system access
- BOT-798 As quick connectivity chat we now have a “Say Hello”
function, to, well, say “hello” to your chatbot
- BOT-916 New dashboard widget to show Botium Box activity
- BOT-875 Easier Botium Box agent handling by automatically removing offline agents from the list *
Infrastructure Enhancements
- BOT-822 Register Chatbot- or Test Project specific system
environment variables (used in Live Chat as well BOT-821). Handy
when dealing with HTTP(S)
proxies
or invalid SSL
certificates
-
BOT-858 added resources directory to NODE_PATH to load custom modules
-
Only available in Botium Platform (enterprise plan)