Skip to content

Security Testing with Botium Box

Introduction

Botium Box is providing you the option to test your chatbot against common security issues. OWASP Top Ten is a list of the current security threats for web applications and therefore also apply to chatbots endpoints hosted on HTTP(S) servers (which is nothing else than a conversational web application). This list is continuously updated by thousands of active community members. Based on this list the OWASP Zed Attack Proxy (ZAProxy) is developed and continuously adopted to the current threats.

Note

Security Testing in Botium Box is in no way comparable to a full security audit. It is able to highlight red flags and shows an entry point to serious security testing.

Note

Security Testing is part of the paid plans only

Security Testing in Botium Box consist of two components:

  • Continuous Security Testing: We built the ZAProxy into Botium Box, to continuously scan for security issues of your chatbot and solving them with the helpful information provided by ZAProxy.

  • Security Test Set: There is a test set available representing common hacking techniques for a chatbot exposed on a website (SQL Injection, XSS Injection).

Continuous Security Testing

With the standard installation, no additional setup is required. If you already have your own installation of ZAProxy, you can link it to Botium Box.

From a technical perspective, the ZAProxy is used as an HTTP(S) proxy and sits between Botium Box and your chatbot endpoint. This setup only makes sense when you are actually hosting your own HTTP(S) endpoint. It does not make sense to use continuous security testing for hosted 3rd party endpoints such as IBM Watson or Google Dialogflow.

  • These technologies use other HTTP(S) proxy settings than linked by Botium Box

  • The big SaaS providers already have minimum security vulnerabilities

    • and if they have one, Botium Box and ZAProxy are not the right tools to find them

Activate Continuous Security Testing

As soon as you have defined your chatbot connection (most likely with the Generic HTTP(S)/JSON connector), test sets and a test project, enable Security check for the test project. When running the test project, the ZAProxy will be active.

After each test session, the identified security issues are shown in the Security Scan tab of the test results.

Note

When running it for the first time, no scan results will be shown - see below, Troubleshooting.

Setup your own ZAProxy

  1. ZAProxy is reachable from Botium Box or a running internal ZAProxy.

    1. This is how you would start ZAProxy in a docker-compose file:

      zaproxy:
          container_name: zap
          image: owasp/zap2docker-stable:2.7.0
          command: [
            "zap.sh",
            "-daemon",
            "-host", "0.0.0.0", "-port", "8080",
            "-config", "api.disablekey=true",
            "-config", "api.addrs.addr.name=.*",
            "-config", "api.addrs.addr.regex=true"
          ]
      
  2. In Botium Box, the environment variable ZAP_ENDPOINT is set to the ZAProxy endpoint. How to set environment variables depends on your installation.

    ZAP_ENDPOINT="http://my-zap-server:8080"
    
  3. Restart Botium Box

Security Report

Risk/Confidence

Risk is describing the impact of the security issue and Confidence is measuring the accuracy of the security threat.

Alert

Alert is describing the root cause of the security issue. In this case X-Content-Type header is missing.

Solution

Is a possible solution described to solve the security issue.

Those links are a references to blog articles or wikis, which are describing the security issue in detail.

Proposed workflow

Once you reached the state to test your chatbot against security threats, simply turn on the security check in the test project. Then you are running the tests → solving the security threats from the report → run tests → solving new appeared security threats → … till the security check is not finding any security threats anymore.

Security Test Set

The OWASP Top Ten is a list of the current security threats for web applications. Some of them are very content-specific and should be covered by conversational flow tests:

Other points from the list are convered by the Continuous Security Testing with ZAProxy as shown above:

There is one point in the list, Insufficient Logging & Monitoring, which cannot be covered by tests at all, at it fully depends on the application.

Finally, there are points on the list which can be detected by writing custom test sets like a malicious hacker would try:

  • For Injection (SQL, NoSQL) an attacker would send malicious input typically interpreted by SQL or noSQL databases. If there is no data store involved, most likely you are safe.

  • For Cross-Site-Scripting (XSS) an attacker would send input interpreted by the web browser to inject malicious functionality into the website. If you chatbot doesn’t allow text input,most likely you are safe.

For these two, here are two test sets full of malicious input for XSS Injection and SQL Injection:

XXS_Injection.zip

SQL_Injection.zip

You can import those files with the Import to Local Repository function in Botium Box and let it run against your chatbot.

The XSS Injection makes most sense when running E2E tests in a web browser:

The SQL Injection should be run on the HTTP(S) endpoint of your chatbot (see Generic HTTP(S)/JSON Connector).

Troubleshooting

1. The first time I run a test with security checks enabled I don’t get any information displayed?

Explanation:

This happens just the first time after you start ZAProxy. The reason for that is that it needs time to build up the folder structure in the zap container and most of the time botium is to fast in executing tests. With the next scans zap already setup the folder structure and you get the alerts/warning immediately.