PSA: Ignition Certificates Kepware Certificate 3 Year Expiration

Ignition
1

I thought I would give a Public Service Announcement as a warning about expiring certificates.

Expiring Certificates created a time bomb for me this past weekend in terms of their visibiltiy within Ignition.

Sunday morning 6:02 am, I got a call from an operator, then an instrument tech, I am already thinking, this is trouble, right when the shift started, they must have been waiting to contact me on this, I wonder how long this problem has been going on. I call and they said that there were black dots on top of the data values and nothing is moving. I remotely login and see the same, I am actually not familiar with black dots on data, but the data isn't moving, so I start my debug process there.

I look at Ignition, its running, I look at the PLC, its running, so it must be the Kepware and the Kepware logs tell me that it is a certificate problem, I see the connection is faulted within Ignition. The date is 5/17/2025, I am thinking, how do I have a certificate fail on a Saturday night at midnight with no warning? Fortunately the plant is running in steady state, no changes are required, the PLCs are running all the processes, but no visiblity, no control, no history data, this is going to be a mess.

But no problem, I have an Ignition Backup running, I go look at it and all expired certificates of the same date. I say again, no problem, I have an offline server for exactly this situation, I boot it up, connect it to the network, and the certificates are all expired on the same day, which makes sense, they all came from the same source.

I am not an Ignition expert, but I deal with certificates all the time, not a big deal, I search for posts about expiring certificates and I find a few posts, and finally a link to a setup set of instructions. I see that the Kepware is straight forward, just click the button, reissue and 3 more years, but it still doesn't work, so I now I go to the Ignition set of instructions.

It is a long set, but not too complicated, so I dig in, go through the steps, and it doesn't work. I try again, and again, the definition of insanity is setting in. I search for more instructions, any deviations, try again. Now I am desperate.

I open a support ticket on the Ignition site, take all the screen shots that I can and send a plea for help. And I wait, its now in the evening, the over night lone plant operator is instructed go sit in the plant, even though the visibility of plant operations is limited, and they have no control. Fortunately, they are in steady state, no changes are required and I just cross my fingers.

6am Monday morning comes, no disasters have happened, and I finally get an email from support asking for my serial number. We do the exhange, I now have the priveleged access and I get a phone call, we go to share screens. I knew it would be something simple, and in 15 minutes, we solved it, but it wasn't actually intuitive. I needed to reset the password on the connection. I should have tried it, but I had the password, I just couldn't imagine resetting it.

I hope this was somewhat entertaining. I still say, that anything that will stop your plant systems from running without warning is a time bomb and everyone with an Ignition 8.1 system has one ticking inside.

3 Likes
2

Nothing about reissuing certificates on either side would require this. Something else changed, accidentally or not, that made this part of your recovery.

3

The link that I found was one that you posted. I still say, this is a time bomb, having a system expire at midnight is something that I have never seen before.

4

Yeah I'm not arguing that certificate expiration when you don't expect can be an issue. I just don't want future readers finding this to think changing/resetting/reconfiguring the password is part of some normal course of actions for reissuing certs.

On the Ignition side all that is needed is to regenerate certificates here:

image
image1287×761 44.6 KB

and then restart the OPC UA module or Ignition Gateway.

Every OPC UA application from any vendor has certificates. There are other certs that can expire as well. SSL/TLS certs if you have HTTPS enabled on your Gateway. GAN certificates if you use the Gateway Network.

5

Is there a way to read the expiry date on that certificate via a tag or script/html/json etc? Then we can configure a Gateway script to alert just before the certificate expires. That seems to be a workaround to avoid nasty midnight surprises!!

6

It's not pretty, but:

import java.io.FileInputStream
import java.security.KeyStore 
import java.security.Security

client_pfx_path = "./data/opcua/client/security/certificates.pfx"
server_pfx_path = "./data/opcua/server/security/certificates.pfx"

def get_certificate_expiration_date(pfx_file_path, keystore_password, certificate_alias, certificate_password):
	ks = java.security.KeyStore.getInstance("PKCS12")
	fis = java.io.FileInputStream(pfx_file_path)
	
	ks.load(fis, list(keystore_password))
	fis.close()
	
	certificate = ks.getCertificate(certificate_alias)
	
	if isinstance(certificate, java.security.cert.X509Certificate):
		return certificate.getNotAfter()
	else:
		return None
		
def get_client_expiration_date():
	return get_certificate_expiration_date(client_pfx_path, "password", "client", "password")
	
def get_server_expiration_date():
	return get_certificate_expiration_date(server_pfx_path, "password", "server", "password")

Check it every day or week in a gateway scheduled script maybe? For future readers: this will work on 8.1, it will not work on 8.3.

This is only going to tell you the expiration date of Ignition's client and server certificates. If you're connected to other 3rd party servers they have their own certs, with their own expiration dates, that will expire on you some day as well.

3 Likes
7

So does this mean on the local OPC-UA connection and redundancy gateway network connections will expire and shut down comms if not renewed before they expire? Or is this mainly an issue for connecting with 3rd party OPC-UA servers?

8

This bit me in the butt today on one of our gateways.

All I needed to do was regenerate the certs in the above places, then in KepServerEX OPC Config remove and retrust the certs.

I did not have to restart the OPC Module or the gateway.

9

No, I have the OPC UA certificate validation configured such that all optional checks are suppressed (allowed, but a warning is logged), and that includes certificate expiration. The only mandatory checks are for matching application URI and PKIX cert path validation.

What this means is for the loopback connection it doesn't matter if it expires.

When you are connected to e.g. Kepware, the Kepware cert can expire and it will still work. If the Ignition cert expires Kepware will balk an deny the connection.

Not an expert here so I don't want to say. Maybe @mgross knows. I just know there are certs involved and the Gateway Network security model loosely mimics what OPC UA does.

10

For better or worse, the gateway network system doesn't throw out a certificate error if it is expired either. Your redundancy connection should continue to work even if its certificate has expired. Down the road, we will likely implement optional certificate checks using the same pattern as the OPC-UA system.

1 Like