I would like to install ignition in the cloud to access remote pumping stations and a few other IIOT applications. I’m having trouble figuring out a suitable VPN infrastructure that will allow access to the PLCs via gateway. The PLCs share the same IPs so the gateway will have to have NAT or something. Then I will have to try to address the PLCs IPs in Ignition, I don’t know how to get around this since the PLCs will share the same IP. I am using s7-1200 and there is no way to change ports and can’t do port translations or anything. I need full VPN access to the Local network for maintenance and accessing PLC/VSD as the sites are all over the country. If anyone has any guidance or done something similar and would like to share their ideas as I’m stuck on how to achieve this.
I assume the duplicate address of 192.168.10.4 is a typo as you cannot have two identical addresses on the same network.
You cannot setup NAT at the Ignition Gateway as NAT needs access to the end device, which it does not have at the Ignition Gateway. If you want to use NAT it has to be done on the router at each site.
I’m not sure what you mean by “can’t do port translations or anything”. Why not?
Just use port forwarding on the site routers. Assign a different port to each unit.
Then on the Ignition gateway configure the S7-1200 address with the different ports, such as:
Node 1 - Unit 1: 192.168.10.2:102
Node 1 - Unit 2: 192.168.10.2:103
Node 1 - Unit 3: 192.168.10.2:104
Node 2 - Unit 1: 192.168.10.3:102
Node 2 - Unit 2: 192.168.10.3:103
Node 3 - Unit 1: 192.168.10.4:102
Node 4 - Unit 1: 192.168.10.X:102
No matter which method you choose, it will need to be configured on the routers at each site.
Yes that is a typo, my bad.
The routers are industrial raspberry pis using compute modules I have had made for me. So basically any routing port forwarding etc is all done via IP tables. I know I will need to setup some sort of NAT on each router, the problem I also have is I have a few sites with win10 IOT panel PCs and you can;t do NAT with windows (can on windows server)
Can I create routes on the Ignition gateway? I’m new to Ignition so i’m not sure what is possible.
Could Possible have have VPN tunnel per site and different interfaces on the server per site? I guess that would not scale well for hundreds of sites.
Can you change ports on ignition siemens drivers? I don’t think they would allow you since you can’t change the port on the PLC. For a site where I have 3 PLCs will be an issue since the will all have same port but different IPs, so port forwarding won’t work on the router for those sites.
There are two ways that I can think of to solve this problem, without involving a ton of port forwarding.
I am making the assumption that you only need to access these plcs to make changes, and full time access is not needed.
-
Have the PIs act as a VPN server. Therefore, your workstation connects to the vpn of the respective pi, and you gain access to that singular network. No worry of ip subnet conflicts this way. You could automate this further, because you are using MQTT, to only enable the vpn service when you are wanting to connect, and disable it when you don;t need it.
-
Look into using an overlay network. Something like Slack Nebula or Zerotier. Have’t tried this yet, but the PIs would act as the overlay network bridge.
The VPN will be connected at all times since the PLCs will be getting controlled from Ignition and logging data. The MQTT is for sites with just sensors and no PLCs.
I thought if I have multiple VPN Servers/interfaces running on same server I could do 1:1 NAT so one server has
Server 1 <------192.168.10.1------> Gateway/NAT 192.168.5.0/24<---->PLC1 192.168.5.5
Server 2 <------192.168.11.1------> Gateway/NAT 192.168.5.0/24<---->PLC2 192.168.5.5
Then in Ignition I could just point 192.168.10.5 to access PLC1 and then 192.168.11.5 to access PLC2
I would have thought this would possibly work, but may not scale well for hundreds of sites
Why not use MQTT for connecting the PLCs to Ignition. This is where that protocol shines (among other things). If you use MQTT, then the IP addressing downstream of the pi and be anything.
If you still want to go the permanent route, then I would look into the overlay network approach.
Your approach is essentially what the overlay network would do, but would instead of going workstation->server1>pi>plc, it would allow for workstation>pi>plc, which will scale better, in my opinion. However, if I was designing this network, I would use MQTT as the primary communication protocol, NOT have any permanent VPN set up, and only bring up the VPNs are required.
I agree with you on using MQTT, I would probably do this for future sites. But these are existing sites and having access to the PLC and VSDs etc is great, I can make the site changes remotely and install and monitor online with PLC and don’t have to send customer program on MC card hoping I got it right first time. Some sites have 150 tags connected back to the scada so it becomes difficult to setup the scale with MQTT. I will have a look into these overlay networks, how do they work without server? Would my sites need static public IP (this can’t be done on cell network)
Using an on demand VPN as I stated preiously would still let you access your site as required.
I believe the overlay networks (zt,nebula) allow you to implement routing rules that would allow you to use a virtual ip and map it to the PLCs. I know this is generally possible with any VPN, but would probably get messier at scale. You could assign multiple ip addresses to the PI’s vpn, and do full destination natting to target each plc. As an example, you could assign 5 ip addresses per site, and have a direct ip address to plc address mapping. Messy, but it will work.
Really, using the PI to translate the native PLC protocol to MQTT would be preferred, and connecting to each site as required for engineering would probably be the easiest to manage at scale. Once you get MQTT and your route/vpn control onto the PI, it will be a very flexible solution.
I will have a look into away of the gateway talking to the PLC and then sending data via MQTT or another solution is using the built in MQTT client the S7-1200 PLCs have. I will do some test runs with my broker. It will need to be bi directional comms as I have to write and read from the PLC. How fast is MQTT for control type stuff?
I do something similar.
Siemens plc (1200’s and 1500’s), connected to a raspberry pi Via Ethernet which is connected to the internet via WiFi
I run logmein hamachi on the pi’s to create the VPN network. I use modified ip tables on the pi to route the Siemens port from WiFi to Ethernet. This lets me remote access the plc’s web page and Connect via Tia portal. I can post a copy of my ip tables if you want to try it. I have over 100 devices so I pay for hamachi but I believe a few devices is free.
I then run node-red on the pi to send required data via MQTT every 60 mins. I don’t use bi directional currently as I only send plc data for monitoring purposes. I want to try bi directional as I want to create a trigger to send the machine status at that time if a operator has issues with the machine. Like what doors or guards are open, encoder values, prox sensor status, PLC status etc. I don’t send this info every 60 mins as it would use un necessary bandwidth as some of the machines are offshore or connect via cell towers. I print QR code’s on the machines serial tags then using the perspective app (installed on the same fire tablet as below) I have a scan barcode page. Once the barcode is read it opens that machines page where there is a button to trigger the in-depth MQTT message, which is then displayed in a perspective view.
Some machines do not have a PLC so I have various sensors and inputs like motor contactors or overloads etc wired into the pi.
I’ve never tried having ignition access the PLC though it’s hamachi vpn ip. I might give that a try this weekend.
Edited to add… In my instance MQTT is instant, of course this might differ with slow networks. If I’m on the node red web page over VPN and manually trigger the pi to send it’s MQTT data, by the time I maximize ignition designer and refresh the tags the value is there. I use transaction groups to update the database when MQTT data is received using a date/time tag in the MQTT message to trigger the transaction group when it sees the value has changed.
I then took it a step further and installed a WiFi hotspot ($40 MikroTik Hap) on the machine that connects to the PI (secured of course) Then using node reds dashboard operators can access the dashboard via any devices that has a web browser. I supply amazon fire tablets for this reason as I pre configure the hotspot credentials so the operator doesnt know the WiFi password. The dashboards are for parameter changes, so only used when a “recipe” change or modification is needed. I switched from using Siemens hand held HMI’s which from memory cost around 3k each to a $140 tablet.
Plus if I’m on site messing with the plc or the pi I can connect to the hotspot rather then having to go though vpn or plug into the PLC. I password protect the node red designer as if the operator was to delete the /ui off the end of the node red url they would open the designer rather then dashboard.
I hope that is of some help and I can go into more detail if you want to try that approach. I use the cirrus MQTT modules but you could run other brokers. I think the Cirrus approach is easier and you can get it all configured and tested in the 2 hour resettable trial before you push the purchase button.
If I add my cloud based ignition server to the hamachi vpn. The server will then be able to access the Pi’s using putty or VNC (this is how I access the pi’s from my laptop). So In theory I should then be able to put the vpn address in ignition as the plc ip address (I access the plcs remotely by their the vpn address in portal). If that works (I’ll test it this weekend) your sites with 3 plcs will have a pi each and the industrial pi just acts as a internet modem/router.
Today I added my cloud based ignition server to my Logmein Hamachi VPN. In my workshop I set up a test Rapsberry pi and a test 1200 PLC. As above the PLC is connected to the pi Ethernet port. The PLC IP and the pi Ethernet port IP address is in the same range. The wifi IP is in a different range as that is DCHP from the site Router. My ip tables are passing all traffic requesting the PLC port on the wifi over to the Ethernet PLC IP. I added a Siemens 1200 device in ignition using the PI VPN address as the PLC Hostname and it connects and works.
So as per the original post, you will need a raspberry PI for each PLC so you get a VPN ip address for each Pi/PLC to add it to ignition OPC. The same VPN ip will let you remote access the PLC and remote access the Pi.
We use NATs pretty heavily in our factory, and yeah we do have to do some port forwarding but it’s actually not that intensive; the PLCs only need a few ports, so it was a relatively simple operation to put an independent piece of hardware (the NAT) on each machine and translate out the devices which need network access.
So here’s the setup:
PLC -[cat5]- machine level switch -[cat5]- NAT -[cat5]- the rest of the network (VPN is on this side)
And then if the HMI is simple enough and only uses a couple of known ports for communication (such as an AB panelview) then we can put it on the machine-side of the NAT. If the HMI is more complicated, we put it on the public side of the NAT. The NAT is transparent to all devices on the network, so as long as we maintain an organized record of public/private IPs and ports, it’s not an issue for us.
Don’t be afraid of port forwarding. If you are aware of all the types of communication which your machine requires in order to operate normally, then it’s easy to set up on most NATs. We use 1783-NATRs on a lot of our machines, and they’re very easy to use in conjunction with other Rockwell equipment. Also being aware of all that will help you to make your factory network more stable and secure.
I am building something alike starting about 4 years ago.
We have several waste and drinking treatment plants ( local dispatchers ) which have their own satellites/nodes such as small pumping stations, chlorination units , pressure etc monitoring units.
Each local dispatcher with satellites I call them clusters, having their own IP class.
After several tests we purchased from the GSM provider a VPN package consisting of fixed guaranteed band like 1Mbps for local dispatchers, where they have their own routers and configure NAT for any IP from LAN we need, M2M SIM’s for satellites where are our own routers. On these routers, depending on situation they act as VPN servers or we just port forward.
Decided on that package after having M2M SIM’s configured with public APN with public fixed IP and on several location we were attacked by hackers resulting ( luckily) just with some inflated bills.
With private APN we have private IP and GSM provider is taking care of security.
S7-1200/1500/300 port can be changed on Ignition Gateway ( cannot be changed on PLC)
So basically if you configure port 1102 on Ignition , port forward 1102->102 on local router should be OK. But with Siemens is never easy, sometimes work, sometimes not ( S7 channels issues etc ) so when is not working I am switching to NAT
On RA ( AB ) stuff you cannot change the port on Ignition nor on PLC/HMI. I have asked developers feature request as to be able to change port and the only answer was that is not safe to use port forwarding. So if there are more items NAT is one solution. Me having just two IP on LAN I just forward to PLC and when need to make modification to HMI just switch port temporarily and revert after job done.
So now I have a nice neat/messed big VPN with VPN within VPN , and extra VPN access from outside when need to work from non VPN sites 
If you have yours tables in order should not be an issue.
I have inherited local dispatcher and lots of them they have inside same private class, such as 192.168.1.0/24 It is not a problem, on VPN after NATting they come up with different class, but just for sake of order, if I would starting something like these new, I would put different private class on each.
Good part for me here bosses are not really interested on the matter, so I am working on my own pace. Though they were kind enough to approve a server just for me and let me toying around.
So bit of an update and for anyone else looking at doing anything similar. I have now modified the source code for WireGuard-Windows version to allow multiple tunnels and connections on the server. This works great and have tested 5 seperate tunnels with 3 external pis connect to server and 2 win10 pcs connected to the server, each device has it’s own VPN subnet, If i need to access one of the PIs I can connect to that PIs Tunnel from another PC and have full access SSH/VNC etc the real issue is accessing the local lan behind the PI, I have tried 1:1 NAT via NETMAP with out any luck.
Forward porting is not going to work or help and bit since one site may have 3 s7-1200 PLCs on the Network and since you can’t change the port on the PLC that becomes an issue for the programming software tia portal, it also has fixed port of 102, port forwarding would work fine if I was only accessing from ignition and change ports on the driver. But I require full access via engineering pc and tia portal.
I’m getting close and just stuck on IPtables now which still quite confusing for me. Have done ALOT of reading and not much is sticking.
I’m having trouble setting up a 1:1 NAT to translate VPN subnet to local lan subnet. I can ping VPN tunnel from both ends perfectly fine. I think my issues are with the NAT. I have tried these rules and I do have IP Forwarding enabled
sudo iptables -t nat -A PREROUTING -d 192.168.12.0/24 -i eth1 -j NETMAP --to 192.168.5.0/24
sudo iptables -t nat -A POSTROUTING -s 192.168.5.0/24 -o eth1 -j NETMAP --to 192.168.12.0/24
With these iptables I can still ping both ends of VPN tunnel and out to 8.8.8.8 but can no longer ping PLC at 192.168.5.11 from the pi or the server. Here is my routing table on the pi, I wouldn’t have thought I wouldn’t need to add any routes being direct 1:1 NAT
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 200 0 0 wlan0
0.0.0.0 192.168.5.1 0.0.0.0 UG 300 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 203 0 0 wwan0
192.168.1.0 0.0.0.0 255.255.255.0 U 200 0 0 wlan0
192.168.5.0 0.0.0.0 255.255.255.0 U 300 0 0 eth1
192.168.12.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
Note I can’t do port forwarding or anything along those lines because the s7-1200 PLC has fixed port at 102 and the programming software used to access the PLC can’t have the port changed either. So this only leaves to do 1:1 NAT which does not seem to work.
Here is an update on my network layout
I am not dealing with NAT myself, but what I know is that they do NAT for each device not an entire class.
They use some AR160 Huawei routers.
First time we tested I have asked the entire class .0/24 to be NATted, received answer no, to specify each IP I need from LAN and they will configure individually. Every month I am sending new requests
for new NAT rules to be added.
So try to look on this, why they did NAT individually.
And about programming, you are not programming each day all PLC’s. If cannot NAT just change on forward rule the IP you need when you need.
Like allnet stated, can you do something like this and just change the ip address for each PLC when programming is needed. This is why I have one PI per plc so I dont have this issue. On the Pi’s where you only have one PLC it should work.
iptables -t nat -A PREROUTING -i wlan0 -p tcp -m tcp --dport 102 -j DNAT --to-destination 192.168.5.11:102