@jspecht we found the problem: the reverse proxy was not upgrading the web socket connection to HTTP, we needed to add the following to our nginx config for it to work:
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";