One of the things that happens when the session is activated is the server ends up doing a call to InetAddress.getLocalHost().getHostAddress() (for reasons that don’t make any sense; this call should be removed).
We’ve seen DNS lookups take a long time on Linux in the past. The behavior you’re seeing makes it seem like the lookup succeeds quickly when you’re assigned to the off-subnet test IP, but takes a long time with the correct IP.
Try adding your IP address and hostname to /etc/hosts and see if it works.