I’m trying to get some info how people architect their Ignition systems, specifically a cloud or remote system.
Do people actually run their Ignition systems from a public IP address when remote access is needed?
Or do people have their sysyem locked down and requires user to connect via a VPN?
I remember seeing a webinar in last years ICC putting Ignition behind a reverse proxy so you wouldn’t have to use a VPN to access the Ignition system. Do people actually do that? Or is that security issue?
We created our own “Cellular Private Network”, working with Verizon and special equipment. This network is not part of the global internet but can access internet data such as NTP servers.
Even thought our equipment may be anywhere in the world I can access any device just as if it is on my bench.
The configuration of this network is a bit complicated but once it is up and running it operates flawlessly.
It operates just like a VPN but on a MUCH larger scale.
Super affordable, I would not understand why anybody looking for security and global access to their devices would not use a network such as this!
We have been using this for 6 years now without the first problem.
I have IP cameras, Plc’s, Hmi’s, local wireless mesh system (this just saves me some IP’s), VFD’s, Extended I/O, modems, routers, Etc… I can manage every piece of equipment remotely, I store all of the IP Camera Video locally and can view/review/download clips to any computer that has access rights to do so.
It's not intrinsically a security issue, but it does significantly change the threat calculus.
In a perfect world, a VPN means you've got at least two layers of security: Authorization/authentication on the VPN, and authorization/authentication on the Ignition gateway itself.
If you're on the public internet, anyone can attack your server. While Ignition is secure[1] (and we do regular third-party penetration tests) it's effectively a single point of failure. (Note: there are lots of caveats and footnotes on this paragraph, but that's the basic story).
As an aside, the reverse proxy setup is probably more secure than plain Ignition. The 'household name' reverse proxy solutions (nginx, Traefik, etc) are battle tested in millions of servers around the globe. Having them serve your actual traffic (which must be HTTPS) also allows soft caching and plenty of other fun features. The counterpoint is that there's a whole lot more people trying to hack nginx and Traefik (and VPN solutions, for that matter) than Ignition.
The tl;dr: If you don't have a compelling need for wide-open internet access...don't.
A 'middle ground' solution I've seen used is to host a frontend gateway somewhere that's publicly accessible, and use the gateway network to give it read-only access to your 'real' gateway somewhere locally. Then the two gateways are secured independently, and the only communication between the two is a secure websocket.
[1] Assuming proper configuration, and with the caveat that it's impossible to prove something is secure.
Yes, private VPN from Verizon. If you need to communicate with several hundred units across a large area this is the only way to go. Very secure, and all devices communicate just as if they was on your bench.
I think he is talking about accessing the Ignition server more than accessing the devices, which your Verizon private apn is a great option for end devices.
One backend gateway which would get data from devices which are connected via some private network. Then a front end gateway with a public IP address accessible via the internet.
The only allowable communication, not from devices, to the backend gateway is directly from the frobt end gateway only.
Yes, exactly. With the way websockets work, you can initiate the gateway network connection from the more restricted gateway; there won't be any firewall relaxation necessary.
Answers to this question may also be forced by your requirements. In our case, both the clients and the data sources needed to be able to connect from anywhere in the world and it was unreasonable to require VPN software on our clients.
We had more control of our data sources (embedded devices we manufacture), so we load all of them with certificates from our internal CA and only allow them to connect to an MQTT broker over TLS with full bi-directional certificate validation against our CA.
For the clients, we have a web proxy in front of our ignition servers which forces all HTTP-to-HTTPS, and forward HTTPS to Ignition.
The proxy, MQTT, and Ignition servers are in a private cloud together. The only ports open to the outside world are the TLS port on the MQTT server and the HTTP/HTTPS ports on the proxy server. None of the Ignition ports are open to the outside world.
Our setup is as safe as we can make it currently. I feel very confident that no one is easily getting into our MQTT broker. I feel reasonably confident that the team at IA is keeping Ignition safe from attackers who don’t know a valid username/password. The weakest spot is if a username/password login is compromised. I’m eagerly awaiting fix 1953 to land in a stable build so I can restrict more powerful actions to certain source IPs.
Thank you everyone for your feedback. I have a better idea of other Ignition architectures that I have not had the experience of working with or building.
This was a post for client connections to the gateway not the devices.
That said we are discussing on how we want to build the network for devices. These will be geographically dispersed equipment that is mobile. We will be using MQTT with TLS security on the backend server which is locked down so would it still be a security threat to have public IP addresses on devices that are accessible via the internet?
I know there are options to get on a private network with Verizon or ATT, but to save time and money to just get public IPs. Or is it not worth the security threat?
In your first response you said read only access. Is there a good (secure) way to have read and write access?
We need a way for users to write to the equipment in the field from the perspective HMIs which would be very beneficial for multiple reasons to be publicly accessible as opposed to making every user use a VPN first before they could even run a HMI.
Ultimately, if you’re giving a publicly accessible gateway write access, you’re (assuming no other security exploits) trusting the usernames and passwords on the publicly accessible gateway. You can also use security zones to narrow things somewhat, but that’s also subject to the same caveats with unknowable unknowns.
As with most security decisions, there are many factors, and security and convenience are usually at opposite ends of the spectrum.
I have found that most people looking for a secure network “assumes” that a PRIVATE VPN network from Verizon or AT&T is costly!
IT IS NOT!! - I can only speak from experience with Verizon. We have a Verizon VPN network and I really do have to say anybody on any budget can afford a Verizon VPN network.
If you don’t at least talk with a rep you will never know!!
I agree with you but my question here is not data from devices to the ignition servers, it’s client connectivity to ignition.
Our users will be scattered throughout the US. They all will need read and write access.
This works when each user is given a VPN connection to the server but we don’t want to give everyone VPN accounts. We don’t want everyone to have that type of connectivity to our infrastructure just accessibility through Ignition screens which we can control.
These VPN accounts are so affordable then why not have 2 - one for the devices and one for whatever else you want, You owe it to yourself to talk with a provider. They can create a VPN Access for any needs.
But this is just my opinion from a user who has worked with Verizon to set up 2 VPN Accounts. They can point these Tunnels to any “PORT or PORTS” and if you chose the correct router for the job, then you will also be able to redirect IP’s and Ports also.
This works when each user is given a VPN connection to the server but we don’t want to give everyone VPN accounts. We don’t want everyone to have that type of connectivity to our infrastructure just accessibility through Ignition screens which we can control.
A VPN is an additional layer of security that ensures that an adversary has to break not only your application's security (which may be comparatively weak due to its low user count), but also the encryption of the VPN (which if you use an established product has a massive user base). You can't replace the VPN with anything on the application level and get the same benefits, so you either want to give up on it completely or focus your efforts on identifying if there is a suitable VPN set up that would give you those protections without being cumbersome to use. In my opinion a public gateway is never an option for anything mission critical or could ever be connected up to our internal networks and expose us to malware.
We bring our users into a central virtual router via a VPN and then give them routes and ACLs for various sites as required. They get one pane of glass and access to as few or as many sites as they need to. None of them get access to our internal applications. You might want to consider something similar. I haven't investigated the carrier VPNs @dcamp1 is talking about, but you can achieve the same thing with an AWS VM with routing software installed on it. I recommend using an established IT company for VPN work rather than trying to roll your own (though this might not be how everyone looks at it).