The heap memory usage in a JVM is not categorized conveniently, so you need some knowledge of Ignition's architecture to dig around, but heap dumps (via jmap or jcmd) or heap monitoring (like in VisualVM) are the techniques to use.
There's a great deal of "noise" in a normal sawtooth pattern, so I usually use a multiple heap dumps over time, taken at the bottom of a sawtooth, to try to zero in on what is growing.