Skip to content
Botium Wiki
Performance Testing
Initializing search
Botium Wiki
Overview
External Resources
External Resources
Botium Blog
Community Forum
Get Started
Get Started
Botium In a Nutshell
Get Started with Botium Box
Get Started with Testing Conversational AI
How-To Guides
How-To Guides
Best Practices
Best Practices
Best Practice: Humanification of Test Sets
Best Practice: Prepare for Large Test Sets
Best Practice: Setup a Process for Test Case Development
Best Practice: Use Botium Crawler for Detecting Conversation Model
How to create & build your perfect chatbot
NLU/NLP Testing
NLU/NLP Testing
Analyze and Improve Performance of IBM Watson Assistant Skill
Assessing Quality of Chatbot Training Phrases for Watson, Dialogflow and everything else
Using Paraphrasing to Increase Conversational AI Test Coverage
How to do NLU/NLP Testing with Botium Box
How to do NLU/NLP Testing with Botium Box
Step 1: Connect Botium Coach to Your Conversational AI
Step 2: Prepare Datasets for Training and Testing
Step 3: Start a Test Session on Botium Box
Step 4: Identify NLP-related Issues and Solutions
Step 5: Annotate and Augment Training Dataset
Step 6: Training your NLU engine
Step 7: Validate Improvements
CI/CD Pipeline Integration
CI/CD Pipeline Integration
Add Botium Test Suite to Azure DevOps Pipeline
Add Botium Test Suite to Jenkins Pipeline
Building a CI Pipeline for Chatbot Developers with Rasa X and Botium Box
Connect to Azure DevOps Repos
Voice App Testing
Voice App Testing
Automated Testing and Monitoring for Alexa Skills with Botium Box
Beginners Guide to Automated Voice App Testing
How to do Voice Testing in Botium Box
How to do Voice Testing in Botium Box
Install Prerequisites
Step 1: Connect Botium Box to Your Voice Bot
Step 1b: Test Voice Connection with Live Chat
Step 2: Prepare Test Set for Voice Testing
Step 3: Prepare or Record Audio Samples
Step 4: Run First Voice Test Session
Step 5: Add Voice Effects to Simulate Real-Life Scenarios
E2E Testing
E2E Testing
E2E-Testing with Botium Box and Selenium/Appium
How to Automate Testing of Your WhatsApp Chatbot
Run Selenium Grid and Connect to Botium Box
Smartphone App Testing with Botium
Testing a Chatbot End-to-End with Perfecto and Botium
Tutorial: Run Your First End-2-End-Test with Botium and Kuki
Performance Testing
Performance Testing
Performance Testing with Botium Box
Security Testing
Security Testing
Security Testing with Botium Box
Test Strategy and Test Case Authoring
Test Strategy and Test Case Authoring
Howto Import Botium Samples
Implementing Token-Based Authentication for Test Cases
Preparing Multi-Brand and Multi-Environment Tests
Preparing Multi-Language and Multi-Environment Test Sets
Using Excel Sheets to Write Test Cases
Asserting on Custom Chatbot Content
Technical Reference
Technical Reference
Setup and Configuration
Setup and Configuration
Prequisites & Requirements
Architecture & Components
Architecture & Components
System Architecture and Tech Stack
Installation & Update Instructions
Installation & Update Instructions
Botium Box Migration Steps
Installation as Azure Multi-Container App
Installation Instructions (On-Premise)
Installation to Kubernetes Cluster
Installation to Openshift
Offline Installation
Quick Installation Steps for Amazon Linux (cloud)
Quick Installation Steps for Ubuntu Server 20.04
Quick Installation Steps (On-Premise)
Uninstallation
Update Instructions (On-Premise/Docker)
Authentication & Authorization
Authentication & Authorization
Botium Box User Management
Google Authentication (On-Premise Only)
LDAP Authentication (On-Premise Only)
Roles & Permissions
Programmatic Access with API Keys
Botium Box Configuration
Operational Tasks: Backup/Restore/Monitoring
CI/CD Integration
Git Repository Integration
NLP/NLU Testing
NLP/NLU Testing
Reference: NLP Quality Metrics
Reference: NLP Quality Metrics
Confusion Matrix / Precision / Recall / F1-Score
Entity Confidence Deviation
Entity Confidence Risks
Entity Utterance Distribution
Intent Confidence Deviation
Intent Confidence Distribution
Intent Confidence Risks
Intent Mismatch Probability
Intent Utterance Distribution
Training Progress
Confidence threshold
Confidence threshold
Confidence Score Reliability
Reference: Test Set Insights
Reference: Test Set Insights
Humanification Insights
LOOCV Insights
Term Importance Insights
Test Set Scores
Utterance Similarity Insights
Botium Connectors
Botium Connectors
Botium Coach / NLP Analytics Support
Generic HTTP(S)/JSON Connector
Supported technologies
Supported technologies
Google Dialogflow ES (Essentials)
Botium Asserters
Botium Asserters
Asserting Email Inbox Content
Asserting with HTTP/JSON API Endpoints
Asserting with SQL Database Content
Dialogflow Asserters (Botium Box Only)
Watson Asserters (Botium Box Only)
Asserting hyperlinks content validity
Asserting hyperlinks content validity
Hyperlink Existance Asserter
Hyperlink Validity Asserter (Enterprise)
Botium Logic Hooks
Botium Logic Hooks
Inject Custom API Content to Botium
Background
Background
Botium Box Release Notes
Botium Box Release Notes
2021–12-20: Botium Box v2.12.0
2021–11-08: Botium Box v2.11.0
2021–09-27: Botium Box v2.10.2
2021–08-24: Botium Box v2.9.1
2021–05-07: Botium Box v2.8.8 is out
2021–04-12: Botium Box v2.8.7 is out
2021–01-28: Botium Box v2.8.4 is out
2020–12-21: Botium Box v2.8.3 is out
2020–11-27: Botium Box v2.8.2 is out
2020–11-16: Botium Box v2.8.1 is out
2020–09-13: Botium Box v2.7 is out
2020–06-24: Botium Box v2.6 ready
2020–05-10: Botium Box v2.5 ready
2020–02-22: Botium Box v2.3 ready
2020–01-22: Botium Box v2.2 ready
2019–12–06: Botium Box v2.1 published
2019–11–12: Botium Box v2.0 available now
2019–10–03: Botium Box v1.8 out now
2019–09–06: Botium Box v1.7.5 out now
2019–07–28: Botium Box v1.7.4 out now
2019–06–22: Botium Box v1.7 out now
2019–05–23: Botium Box v1.6 out now
2019–04–29: Botium Box v1.5 out now
2019–04–01: Botium Box v1.4 out now
2019–03–08: Botium Box v1.3 out now
2019–02–22: Botium Box v1.2 out now
2019–02–11: Botium Box v1.1 out now
2019–01–19: Botium Box v1.0
Botium Box Roadmap
Chatbot Testing Theory
Chatbot Testing Theory
4 DO’s and 3 DON’Ts for Chatbot Testing Strategies
How to Test a Chatbot in 3 Lessons
Security Threats and Security Testing for Chatbots
Botium Test Levels or Selecting a Botium Connector
Unveiling the Value of Chatbot Testing
NLP/NLU Quality Metrics
NLP/NLU Quality Metrics
Quality Metrics for NLU/Chatbot Training Data / Part 2: Embeddings
Plans and Pricing
Plans and Pricing
Botium Box Plans and Features
Botium Box Ultimate
Botium Core vs Botium Box - What is My Choice ?
Developer Section
Developer Section
Botium Extension Points for Developers
Customizing Test Results in Botium Box
Developing Custom Asserters
Developing Custom Logic Hooks
How to deploy my own Botium Connector
Howto develop your own Botium Connector
Howto develop your own HTTP/JSON Botium Connector
Using the Botium Box GraphQL API
Frequently Asked Questions
Frequently Asked Questions
Botium Box Usage
Botium Box Usage
How can I inspect detailed test case logs in Botium Box ?
I want to run tests in parallel - how do I do this ?
What are the "Scripting Settings" in Botium Box ?
Common Problems and Solutions
Common Problems and Solutions
Enable Diagnostics and Inspect Logs
Error "self signed certificate in certificate chain" or other SSL-related error
Git Error 128
I am behind a corporate proxy, can I use Botium Box ?
Verify Network Connectivity for Botium Agents
When "Security Checks" is enabled, all test cases are failing ?
Installation and Infrastructure
Installation and Infrastructure
Does Botium Box store the test data secure, reliable and long-term ?
Failed to parse license key: JsonWebTokenError: jwt malformed.
I am getting docker error like: X509 Certificated signed by unknown authority
I am receiving an error Are you trying to mount a directory onto a file (or vice-versa)? when starting docker-compose
I am receiving an error TLSV1_ALERT_PROTOCOL_VERSION when starting docker-compose
I am receiving errors from the mysql service (something with InnoDb) when starting docker-compose (on migration)
What are the Docker volumes for in Botium Box ?
When running large test sets, Botium Box gets slower and slower
Test Case Development
Test Case Development
Can I use Google Sheets to manage my test cases ?
Can I use JIRA to manage my test cases ?
Can I use Sharepoint to manage my test cases ?
What CI/CD Pipelines does Botium Box support ?
How Can I Simulate Buttons with my Azure Bot ?
How can I test a Bot Framework Chatbot without going over Directline ?
I don't know how to set a capability in Botium Box.
Performance Testing
Performance Testing with Botium Box