What to do when the self-hosted runZero console doesn't start

Start by checking that /opt/rumble has the right permissions:

$ ls -la /opt/rumble/
total 8
drwxr-xr-x. 10 root   root    112 Oct  8 12:09 .
drwxr-xr-x.  3 root   root     20 Oct  8 12:08 ..
drwxr-xr-x.  2 root   root   4096 Oct  8 12:08 agent
drwx------.  2 root   root      6 Oct  8 12:08 config
drwxr-xr-x.  2 root   root     32 Oct  8 12:08 console
drwxr-xr-x.  3 root   root     66 Oct  8 12:09 etc
drwxr-xr-x.  2 root   root     51 Oct  8 12:09 proc
drwxr-xr-x.  2 root   root   4096 Oct  8 12:08 scanner
drwx------.  2 rumble rumble    6 Oct  8 12:09 storage
drwxr-xr-x.  2 rumble rumble    6 Oct  8 12:09 tmp

Next, check that /opt/rumble/console/rumble-console.bin has the right permissions and SELinux context:

$ ls -laZ /opt/rumble/console/
drwxr-xr-x. root root unconfined_u:object_r:usr_t:s0   .
drwxr-xr-x. root root unconfined_u:object_r:usr_t:s0   ..
-rwxr-xr-x. root root unconfined_u:object_r:usr_t:s0   rumble-console.bin

(If you don’t have SELinux enabled, that’s OK; just omit the Z argument to ls.)

The permissions should be r-x for other, because on startup the console gets re-run by a restricted user ID.

Assuming the permissions all look OK, check journalctl -u rumble-console and see if there are any errors reported from when the console attempted to start.

If the log says the service started OK, the command ps aux | grep rumble should show two Rumble processes, plus postgres. Example output:

$ ps aux | grep rumble
root      1742  0.0  2.3 781552 86100 ?        Ssl  14:51   0:00 /opt/rumble/console/rumble-console.bin server
rumble    1747  0.2  5.0 986080 189832 ?       Sl   14:51   0:03 /console/rumble-console.bin server
postgres  1753  0.0  0.5 402416 18784 ?        Ss   14:51   0:00 postgres: rumble rumble 127.0.0.1(43092) idle
postgres  1754  0.0  0.5 402540 18700 ?        Ss   14:51   0:00 postgres: rumble rumble 127.0.0.1(43094) idle

(You will also see the grep process itself.)

The rumble-console.bin running as root is the one rumblectl starts; the one running as rumble is the actual server task, which the root task runs in a container with restricted permissions.

If you have SELinux enabled, try grep type=AVC /var/log/audit/audit.log to see if there are any SELinux denials. Assuming it’s a clean system set up for the Rumble console, there shouldn’t be any.

Also, check the rumble user ID’s SELinux context:

$ sudo -u rumble id -Z
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

The next step is to try starting the console manually, as the rumble user:

$ sudo -u rumble /bin/bash
bash-4.2$ cd /opt/rumble/console/
bash-4.2$ ./rumble-console.bin
FATA[0000] rumble requires root privileges, re-run with sudo or from a root shell

The error at the end is expected – we’re just checking if the binary can be executed as the rumble user.

If that works, you can try the same thing, but in the chroot environment. First confirm the UID and GID for the rumble restricted user:

$ grep rumble /etc/passwd
rumble:x:1001:1001::/home/rumble:/bin/false

Then execute the chroot, replacing the 1001:1001 if your system has a different UID and GID for the user:

$ sudo chroot --userspec=1001:1001 /opt/rumble /console/rumble-console.bin
FATA[0000] rumble requires root privileges, re-run with sudo or from a root shell

Again, the error is expected, we’re just checking the binary runs.

If the console runs in the chroot environment, try systemctl restart rumble-console and then journalctl -u rumble-console and see if any more errors have turned up in the log.