In the gateway, do you monitor performance? How much RAM do you have allocated, Perspective I would hope 4 GB at absolute minimum, more if you have multiple clients and you say you have a couple. Remember all scripts run ON the gateway. 20-30 users should require a beefy server. But if it is happening with just 2-3 users - I suspect an issue with the way the view is setup.
What are the bindings like - do you have a runScript in any of them? Do a control+F for runScript and see if you have it in any bindings, poor usage of it can cause massive performance issues if you're using it to do something complicated.
Do you have use property changes somewhere? Poor scripting on a property change can accidentally induce an infinite loop (property change script A causes property B to change who's script causes property A to change and so on) and this can crush your performance as well.
I would first look at all the scripting and bindings. Perspective is certainly more of resource hog but if it really was just tag bindings, then with a sufficiently strong gateway I wouldn't expect too many issues - I am working on a project with 13 always on perspective HMI's, all just tag bindings, and it basically never goes above 1 GB of ram usage.