The Explorer cannot connect to the console because of TLS failures

Modified on Wed, Oct 25, 2023 at 2:19 PM

The Explorer connects to the console.runzero.com host on TCP port 443 using TLS and two static IPv4 addresses and two static IPv6 addresses listed in the documentation. This connection is used for Explorer registration, job scheduling, status messages, and submission of completed scan jobs. The host console.runzero.com is also used for automatic updates of the Explorer executable.

To verify that TLS certificate errors are the problem, you can use certigo (easier) or openssl.

Certigo: certigo connect console.runzero.com:443

OpenSSL: openssl s_client -connect console.runzero.com:443

In the case of OpenSSL, look for a line starting Verification that will tell you whether the certificate could be verified.

Ideally this test should be performed from the machine(s) hosting the runZero Explorer.

If the certificate could not be verified, there could be a router inspecting TLS connections from the host machine. This involves the router decrypting the connection and then re-encrypting using a different key and certificate. In this case the output from openssl or certigo will show a certificate signed by a CA belonging to your organization, rather than an Amazon CA.

Assuming a router is the cause, there are several options.

The best option is to have your security team put in a bypass rule from the Explorer host system to console.runzero.com, so that it doesn’t interfere with the encrypted traffic between those two specific systems.

If that’s not possible, the Explorer should accept TLS CA certificates set up in the operating system. So if the intercepting firewall’s certificate is signed by an internal CA, you can add that CA to the operating system’s trust store on the machine hosting the Explorer. Alternatively, you can set an environment variable in the Explorer configuration and provide the internal CA certificates in a file in PEM format.

A third option is to set up the Explorer to use a web proxy provided by your security team. This is handled using the HTTPS_PROXY variable in the Explorer configuration file.

Please note that certain web proxies that perform TLS inspection do not handle WebSocket communication properly. See below for information about diagnosing this. The most popular products with this problem are Sophos security appliances (previously Cyberoam).

The procedure for testing your web proxy for WebSocket compatibility is as follows:

First, prep a file containing runZero’s valid TLS CA certificate. You can do this by viewing the certificate in Chrome – click the padlock, select Connection is secure option, choose Certificate is valid option, click on Details tab, make sure you have the *.runzero.com certificate selected in the top part of the window, and click Export… Put the result in a file called (say) console.runzero.com.cacert with the default format of base64 encoded ASCII, single certificate.

Here’s one as of October 2023:

Next, make sure that curl is set up to use your organization’s web proxy. Then, use curl to probe the websocket endpoint and attempt to make a connection, telling it to only accept the Rumble CA certificate you just downloaded:

curl --cacert console.runzero.com.cacert -i -N -H "Connection: upgrade" \
 -H "Upgrade: websocket" -H "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" \
 -H "Sec-WebSocket-Version: 13" \
 https://console.runzero.com/socket/explorer/connect

You should see something like this:

HTTP/1.1 101 Switching Protocols
Date: Thu, 07 Apr 2022 21:37:20 GMT
Connection: upgrade
Upgrade: websocket
Sec-WebSocket-Accept: qGEgH3En71di5rrssAZTmtRTyFk=

If not, the proxy is preventing WebSocket access, and you will need to find a way to bypass it.