Prequisites & Requirements
High Level Component Overview
Botium Box Components
- The Botium Box User Interface is a browser-based application
- The Botium Box Agent(s) are the work-horses, which actually run the test cases. Usually, there are several Botium Agents active for parallel execution of test cases
- The Botium Box Server is the central instance connecting the Agents and the User Interface
Required 3rd-Party Components
- Recommended: NGINX as frontend webserver (upgrade to load balancer and failover)
- Prisma as data storage engine, which is itself backed by either MySQL or PostgresQL
- Redis for reliable messaging, task scheduling and session storage
- Recommended: some kind of continuous integration service such as Jenkins or Bamboo
Note
All of the 3rd-Party components (except continuous integration) are part of the distribution of Botium Box, with a basic working default configuration.
Server Hardware Requirements
It is not possible to talk about the hardware requirements without any knowledge about
- how many test projects you plan
- how many test cases you plan
- how many test runs you plan
Furthermore, there are several deployment options, from a standalone, single server deployment to a cluster or swarm of countless machines connected to the Botium Box.
For a minimum standalone, single server deployment (suitable for a PoC or Demo environment), you are usually safe with this configuration (which corresponds to an Amazon t2/t3/t3a.medium instance):
- 2 CPU (dual-core)
- 4 GB RAM
- 100GB HDD
For a medium-sized single server production deployment, we recommend at least (corresponding to an Amazon t2/t3/t3a.xlarge instance):
- 4 CPU (or quad-core)
- 8 GB RAM
- 200 GB HDD
Resource Requirements Per Component (Pod)
When installing Botium Box on a Kubernetes cluster or in a distributed Docker environment, here are the the minimum resource requirements for each component (multiplied by the number of instances). The recommended resource allocation is double the size.
- Botium Box Server
- 1 CPU core
- 1 GB RAM
- Botium Box Agent
- 0,5 CPU cores
- 1 GB RAM
- (optional) Botium Coach Worker
- not a required component, used for test set insights
- 1 CPU core
- 4 GB RAM
- Prisma
- 1 CPU core
- 2 GB RAM
- Redis
- 0,5 CPU cores
- 0,5 GB RAM
Additional Resource Requirements - Test Set Insights
When calculating advanced Test Set Insights (like cohesion/separation/similarity charts), you have to enable the additional Coach Worker component.
- Coach Worker
- 1 CPU core
- 4 GB RAM
Additional Resource Requirements - Voice Testing
When using Voice Testing you have to enable the additional Botium Speech Processing component. Depending on the speech backend you are using the hardware requirements are different - when connected to a cloud service like Google Speech, the local resource requirements are of course way lower than when using Kaldi and MaryTTS locally.
- Botium Speech Processing with cloud service like Google
- 1 CPU core
- 1 GB RAM
- Botium Speech Processing with Kaldi and MaryTTS
- 2 CPU cores
- 8 GB RAM
Additional Resource Requirements - Performance Testing
For generating conversation load on the chatbot, there is CPU power required on Botium side. Our experiments show that as rule of thumb Botium will be able to simulate ~300 parallel users per CPU core. Following are the results of our experiments.
For the simulation of ~ 500 parallel users per second:
- 2 Botium Box Agents
- 1 CPU core each
- Increase Redis resources (for Kubernetes installation)
- 1 CPU core
- 2 GB RAM
- Sample configuration
- 2 CPU cores
- 8GB RAM
- ~AWS t2/t3/t3a.large instance type
For the simulation ~1000 parallel users per second:
- 4 Botium Box Agents
- 1 CPU core each
- Sample configuration
- 4 CPU cores
- 16GB RAM
- ~AWS t2/t3/t3a.xlarge instance type
For the simulation ~2000 parallel users per second:
- 8 Botium Box Agents
- 1 CPU core each
- Sample configuration
- 8 CPU cores
- 16GB RAM
- ~AWS t3a.2xlarge instance type
Note
Parallel in this case actually is not full parallelity, but has to be seen as a total amount of requests within a given time frame (100 user requests per second for example)
Server Software Requirements - Docker
All Botium Box components are distributed as Docker containers, so you need a working Docker installation on all servers.
- Linux as server operating system
- Docker Engine Version 17 or later is required - https://docs.docker.com/install/
- Docker Compose Version 1.16 or later is required - https://docs.docker.com/compose/install/
Check Docker
Run this command to verify that your Docker installation is actually working:
docker run hello-world
Client Requirements
Any current desktop browser will do. We support Internet Explorer, Edge, Chrome, Firefox, Safari.
Currently not supported: Using Botium Box with a Smartphone.
Networking Requirements
There are several firewall rules to be opened for running the Botium Box. Shell access (usually SSH, Port 22) is required.
Github
On first installation you have to download the Botium Box repository containing the installation files (Docker-Compose, Kubernetes) from Github. Github doesn’t have a fixed list of IP addresses, so either open up the firewall for the first download, or upload the Github files manually to the server.
Docker Hub
During installation or update of Botium Box, Docker will pull the required images from Docker Hub. As there is no fixed set of IP addresses or hostnames available for whitelisting Docker Hub, it is recommended either to temporarily open the firewall or copying the required Docker images to the server manually for Offline installation.
Botium Box Server
Botium Box listens on the default ports 80 (for HTTP - disabled by default) and 443 (for HTTPS - enabled by default) if NGinx is used, and port 4000 (HTTP) otherwise. Clients accessing the Botium Box, as well as any connected CI/CD products have to get access to these ports.
Prisma (multi server deployments only)
Please enable connectivity from the Botium Box Server to the Prisma Server on port 4466.
Redis (multi server deployments only)
Please enable connectivity from the Botium Box Server and the Botium Box Agents to the Redis Server on port 6379.
Other ports
You have to make sure that Botium Box Server and Botium Box Agents connectivity to the chatbot under test is available. This depends on the used connector. If you don’t know, we are happy to consult.
When using LDAP authentification, you have to open the port to your LDAP server (usually 389).
Server Software Requirements - Kubernetes/Openshift
See Installation to Kubernetes Cluster / Installation to Openshift
Security Requirements
We recommend to generate and use an SSL certificate for serving the Botium Box User Interface. It should match the hostname of the machine the Botium Box Server is running on.
Note
By default, a self-signed certificate for server 127.0.0.1 is included, which at least will cause security warnings in our browser.