[IGN-5354] Metrics Dashboard has no metrics

Just updated to 8.1.13 and trying out the metrics dashboard, but it has no metrics! Do these need to be created? According to the docs it looks like there should be a bunch to select from…

Anything out of the ordinary in the Gateway log files?


I just looked into several of our 8.1.13 environments and they all have the metrics as expected, so there does seem to be some sort of issue in your environment. What version were you on before the upgrade? Is there any other area of your Gateway impacted?

Upgraded from 8.1.11. Ran on Installer on my EAM Controller, then second GW using EAM Upgrade, same results. No other issues with the GW so far, but still checking.

This looks like the culprit

java.lang.ArithmeticException: / by zero

at com.inductiveautomation.ignition.gateway.web.pages.status.serializers.MetricSnapshotSerializer.serialize(MetricSnapshotSerializer.java:42)

at com.inductiveautomation.ignition.gateway.web.pages.status.serializers.MetricSnapshotSerializer.serialize(MetricSnapshotSerializer.java:15)

at com.inductiveautomation.ignition.common.gson.internal.bind.TreeTypeAdapter.write(TreeTypeAdapter.java:81)

at com.inductiveautomation.ignition.common.gson.Gson.toJson(Gson.java:704)

at com.inductiveautomation.ignition.common.gson.Gson.toJsonTree(Gson.java:597)

at com.inductiveautomation.ignition.common.gson.Gson.toJsonTree(Gson.java:576)

at com.inductiveautomation.ignition.common.gson.internal.bind.TreeTypeAdapter$GsonContextImpl.serialize(TreeTypeAdapter.java:155)

at com.inductiveautomation.ignition.gateway.web.pages.status.serializers.MetricTimerSerializer.serialize(MetricTimerSerializer.java:30)

at com.inductiveautomation.ignition.gateway.web.pages.status.serializers.MetricTimerSerializer.serialize(MetricTimerSerializer.java:17)

at com.inductiveautomation.ignition.common.gson.internal.bind.TreeTypeAdapter.write(TreeTypeAdapter.java:81)

at com.inductiveautomation.ignition.common.gson.Gson.toJson(Gson.java:704)

at com.inductiveautomation.ignition.common.gson.Gson.toJsonTree(Gson.java:597)

at com.inductiveautomation.ignition.gateway.web.pages.status.routes.MetricsRoutes.lambda$getAvailableMetrics$0(MetricsRoutes.java:65)

at java.base/java.util.concurrent.ConcurrentHashMap.forEach(Unknown Source)

at java.base/java.util.Collections$UnmodifiableMap.forEach(Unknown Source)

at com.inductiveautomation.ignition.gateway.web.pages.status.routes.MetricsRoutes.getAvailableMetrics(MetricsRoutes.java:62)

at com.inductiveautomation.ignition.gateway.dataroutes.Route.service(Route.java:254)

at com.inductiveautomation.ignition.gateway.dataroutes.RouteGroupImpl.service(RouteGroupImpl.java:61)

at com.inductiveautomation.ignition.gateway.dataroutes.RouteGroupCollectionServlet.serviceInternal(RouteGroupCollectionServlet.java:59)

at com.inductiveautomation.ignition.gateway.dataroutes.AbstractRouteGroupServlet.service(AbstractRouteGroupServlet.java:38)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)

at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)

at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

at com.inductiveautomation.catapult.handlers.RemoteHostNameLookupHandler.handle(RemoteHostNameLookupHandler.java:116)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)

at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)

at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)

at org.eclipse.jetty.server.Server.handle(Server.java:516)

at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)

at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)

at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)

at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)

at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)

at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)

at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)

at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)


That’s a good bet. We’ll look into it.

I have the exact same issue. Also upgraded using EAM.

Is there any additional info I can provide for investigation?

No, thank you; we’re reasonably confident we found the cause from the previous stacktrace. There is a fix in place as part of the upcoming 8.1.14 release which is going through the RC process right now.

That’s a bummer. We are experiencing the same issue. We have 9 gateways all running 8.1.13. It is a large task to upgrade all of them, especially working around plant production. 1 of our gateways does get the list to populate, but the others do not. Weird. Is there any other work around?

I am having the same issue on 8.1.24. Updated recently from .21
Is it possible the problem has been reintroduced?

When loading up a saved dashboard, this message appears:
image

I'm not seeing any changes to the Metrics Dashboard between 8.1.21 and 8.1.24 that would have reintroduced the issue. I would suggest reaching out to support to dig into it more.

I tested with a new implementation of 8.1.24 and didn't see any issues with those metrics. Perhaps something went wrong during the upgrade. It will help if you have a copy of the install.log and wrapper.log files when reaching out to support.

2 Likes

Strange, i just updated from 8.1.23 to 8.1.39 and metrics are gone. How do I get them back?

image

Check your auth status. I typically see that behavior when my session has become disconnected.

Was the Perspective metric associated with a specific session?

Actually, this is the Metrics Dashboard on the gateway website under Status tab, Diagnostics/Metrics Dashboard. Not in a perspective project.

By "session" I meant Gateway Session. Your screenshot has a Perspective metric tile displayed, and if that metric tile was associated with a specific session, that session ID would no longer be in use after a Gateway restart.

One of the metrics was showing the total number of Perspective Sessions, I had another looking at thread times waiting. And I had another showing total number of database queries. Perhaps things have changed since .23, but when I try to add a new metric, the popup box doesn't have any to pick from. Empty.

image

image

image

I see. If you've verified you are still authenticated, you'll need to contact Support for assistance. The issue referenced in this thread was fixed as part of 8.1.14. If you are indeed experiencing a similar issue, we will need to look at what could be causing the same behavior in 8.1.39.

Ok thanks. Will do. And yes, still authenticated with the same admin user as before. I did try installing Ignition on a fresh/brand new PC and the metrics popup does show up. Somehow on this production server upgrade, it failed to populate.

If these metrics are located in some file on the server, I could check if the file is physically missing. Or if they are in the sqlite database I could search there. Do you know where this part of the gateway gets the list of metrics from?

New installation shows the following metrics to choose from.