Ignition 7.5.12 very slow and lagging

Hello.
Our company uses Ignition 7.5.12 for one of our projects for a long time.
Within the project, we have near 90 connected devices and more than 300000 tags.
Now we have a problem. Ignition started to work very slowly and extremely lagging (can’t save project changes, designer stops working, windows opens very slow, etc.).
Can somebody help to solve it?

There could be many reasons for slow performance - we will need more info to help.

Have you checked gateway/client logs to see if there’s anything in there?

What’s your system performance stats?

Is it all from one physical server? What condition are the HDD(s) in?

I didn’t find anything in logs. Ignition gateway installed on one server where java.exe process tooks 4.5Gb of RAM and there is more than 100Gb of free disk space. Max heap memory usage is 4Gb.
Sometimes some threads are in the BLOKED status

For example:
Thread [TagSubscriptionModel-thread-1] id=53, (BLOCKED)
owns monitor: com.google.common.collect.MapMakerInternalMap$StrongEntry@1e3f1ad7
owns monitor: com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags$ClientSubscriptionProxy@30af8fd7
owns synchronizer: java.util.concurrent.ThreadPoolExecutor$Worker@44e0ef91
waiting for: org.apache.log4j.spi.RootLogger@18c3611 (owned by 349)
org.apache.log4j.Category.callAppenders(Category.java:204)
org.apache.log4j.Category.forcedLog(Category.java:391)
org.apache.log4j.Category.error(Category.java:322)
com.inductiveautomation.ignition.gateway.sqltags.providers.datasource.tagstore.DBTagStore.loadDefinition(DBTagStore.java:540)
com.inductiveautomation.ignition.gateway.sqltags.providers.datasource.tagstore.DBTagStore$1.apply(DBTagStore.java:183)
com.inductiveautomation.ignition.gateway.sqltags.providers.datasource.tagstore.DBTagStore$1.apply(DBTagStore.java:180)
com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute(ComputingConcurrentHashMap.java:355)
com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.compute(ComputingConcurrentHashMap.java:184)
com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.getOrCompute(ComputingConcurrentHashMap.java:153)
com.google.common.collect.ComputingConcurrentHashMap.getOrCompute(ComputingConcurrentHashMap.java:69)
com.google.common.collect.ComputingConcurrentHashMap$ComputingMapAdapter.get(ComputingConcurrentHashMap.java:393)
com.inductiveautomation.ignition.gateway.sqltags.providers.datasource.tagstore.DBTagStore.getDefinition(DBTagStore.java:515)
com.inductiveautomation.ignition.gateway.sqltags.providers.datasource.tagstore.DBTagStore.getProperty(DBTagStore.java:547)
com.inductiveautomation.ignition.gateway.sqltags.execution.tags.AbstractGatewayTag.getAttribute(AbstractGatewayTag.java:135)
com.inductiveautomation.ignition.gateway.sqltags.execution.tags.ExternalTag.getAttribute(ExternalTag.java:166)
com.inductiveautomation.ignition.common.sqltags.TagDefinition.getAttributesFrom(TagDefinition.java:342)
com.inductiveautomation.ignition.common.sqltags.TagDefinition.(TagDefinition.java:141)
com.inductiveautomation.ignition.common.sqltags.TagDefinition.(TagDefinition.java:119)
com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags.wrap(SQLTags.java:256)
com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags.access$500(SQLTags.java:85)
com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags$ClientSubscriptionProxy.tagChanged(SQLTags.java:964)
com.inductiveautomation.ignition.gateway.sqltags.tags.TagReference.notifyValueListener(TagReference.java:247)
com.inductiveautomation.ignition.gateway.sqltags.tags.TagReference.addTagValueChangeListener(TagReference.java:155)
com.inductiveautomation.ignition.gateway.sqltags.providers.ProviderSubscriptionManagerImpl.subscribe(ProviderSubscriptionManagerImpl.java:182)
com.inductiveautomation.ignition.gateway.sqltags.providers.AbstractStoreBasedTagProvider.manageSubscriptions(AbstractStoreBasedTagProvider.java:2041)
com.inductiveautomation.ignition.gateway.sqltags.providers.AbstractStoreBasedTagProvider.subscriptionChanged(AbstractStoreBasedTagProvider.java:2031)
com.inductiveautomation.ignition.gateway.sqltags.model.TagSubscriptionModel$ChangeNotificationJob.run(TagSubscriptionModel.java:250)
com.inductiveautomation.ignition.common.execution.impl.BasicExecutionEngine$ThrowableCatchingRunnable.run(BasicExecutionEngine.java:526)
java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
java.util.concurrent.FutureTask.run(Unknown Source)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

Daemon Thread [http-8088-86] id=20840, (BLOCKED)
waiting for: com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags$ClientSubscriptionProxy@30af8fd7 (owned by 53)
com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags$ClientSubscriptionProxy.poll(SQLTags.java:891)
com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags.poll(SQLTags.java:779)
sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
com.inductiveautomation.ignition.gateway.servlets.gateway.AbstractGatewayFunction.invoke(AbstractGatewayFunction.java:176)
com.inductiveautomation.ignition.gateway.servlets.Gateway.doPost(Gateway.java:394)
javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
com.inductiveautomation.ignition.gateway.bootstrap.MapServlet.service(MapServlet.java:85)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
java.lang.Thread.run(Unknown Source)

Thread [ExecEngine-1-thread-5] id=38, (BLOCKED)
owns monitor: com.inductiveautomation.ignition.gateway.sqltags.execution.tags.ExpressionTag@6b5aad3e
owns synchronizer: java.util.concurrent.ThreadPoolExecutor$Worker@524e8b33
waiting for: com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags$ClientSubscriptionProxy@30af8fd7 (owned by 53)
com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags$ClientSubscriptionProxy.tagChanged(SQLTags.java:947)
com.inductiveautomation.ignition.gateway.project.ProjectSettingsCache$ProjectDefaultTagManagerFacade$DefaultListenerUnWrapper.tagChanged(ProjectSettingsCache.java:228)
com.inductiveautomation.ignition.gateway.sqltags.tags.TagReference.notifyValueListeners(TagReference.java:202)
com.inductiveautomation.ignition.gateway.sqltags.tags.TagReference$1.tagChanged(TagReference.java:64)
com.inductiveautomation.ignition.gateway.sqltags.execution.tags.AbstractGatewayTag.fireChangeEvent(AbstractGatewayTag.java:157)
com.inductiveautomation.ignition.gateway.sqltags.execution.tags.AbstractExecutableTag.setCurrentValue(AbstractExecutableTag.java:220)
com.inductiveautomation.ignition.gateway.sqltags.execution.tags.ExpressionTag.internalEvaluate(ExpressionTag.java:214)
com.inductiveautomation.ignition.gateway.sqltags.execution.tags.AbstractExecutableTag.evaluate(AbstractExecutableTag.java:438)
com.inductiveautomation.ignition.gateway.sqltags.scanclasses.BasicScanClassInstance.evaluateTags(BasicScanClassInstance.java:177)
com.inductiveautomation.ignition.gateway.sqltags.scanclasses.BasicScanClassInstance.runTagEvaluation(BasicScanClassInstance.java:164)
com.inductiveautomation.ignition.gateway.sqltags.scanclasses.SimpleExecutableScanClass.runTagEvaluation(SimpleExecutableScanClass.java:749)
com.inductiveautomation.ignition.gateway.sqltags.scanclasses.SimpleExecutableScanClass.run(SimpleExecutableScanClass.java:867)
com.inductiveautomation.ignition.common.execution.impl.BasicExecutionEngine$TrackedTask.run(BasicExecutionEngine.java:573)
java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
java.util.concurrent.FutureTask.runAndReset(Unknown Source)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)