I am currently looking at setting up an MQTT network through the Cirrus Link modules and am trying to determine the best method for configuring tags and tag providers to allow for consistency between dev/test/prod environments.
Using the MQTT Transmission module I am sending data in Sparkplug B format from production gateways to a broker on a central gateway. I can then view the tag data through the MQTT Engine module on the dev/test gateways under the "MQTT Engine" tag provider.
Based on the MQTT section in the 8 Deployment Best Practices page, it seems to imply that all of the views/logic that need to reference device data should use the tag that exists on the MQTT server through the engine module. I have concerns with this since our MQTT server does not exist on the production gateway, and if the network connection between the production gateway and MQTT server goes down then we are dead in the water.
So, the tags on the production server need to reference the device tags. This data then gets published to an MQTT server, and then dev/test have to access this data through the engine module. Ideally the dev/test environments have the same tag provider name and tag data as the prod environment. However, there will have to be differences between the tags that reference device data on the prod gateway and those same tags on the dev/test gateway. And it is here that I am struggling to come up with a robust way of doing this.
Does anyone with experience in this type of set up have any advice or good ideas?
You are connecting Dev gateways to live Production tags. Seems dicey to me. Dev should be utterly isolated, or you cannot actually develop views that are intended to control the process.
Next best is to do all of your development using tagpath parameters to your views, with careful control over write access (in the tag providers, not the views), then testing views that expect write access against dummy tag paths.
You are connecting Dev gateways to live Production tags. Seems dicey to me.
Is there a better way I'm missing for getting live data to the dev environment? Or is the idea of having the same tags/tag providers in all environments misguided?
Gets super tricky with MQTT data. We have a customer where we had to have 2 Brokers. 1-Production 1-NonProduction. Which requires two connections in Cirrus Link Engine - but the connections share the same namespaces. So, you cannot have the same topics then for Prod vs NonProd. So, your development would have to send / receive messages to some topic namespace that is not the same - Site/Prod/Area/Line/etc... vs Site/NonProd/Area/Line - can get expensive really quick if you are trying to make a true DEV / PROD environment with MQTT. Plus whatever else is consuming or producing data has to update scripts or anything else to follow what is production vs non-prod.
I prefer to set up actual PLCs in my lab with copies of the relevant logic and tags/addresses. Or otherwise simulate them. (For Rockwell or Modbus targets, I simulate with my own 3rd party modules. You can use trial mode for that, too.)
If you point at actual live data, you have to make sure your infrastructure cannot write back to the production systems.
Won't have the option for actual PLCs unfortunately.
I'm starting to wonder if I need a mix of MQTT and remote tag providers throughout the system as there will be dev servers for both HMI and SCADA servers. Everything SCADA can fetch data from MQTT for analysis/storage while the HMI projects can use remote tag providers. There's little control that the HMIs are actually doing so it's not a major concern.
While I don't know your network topology, if your central gateway and production gateways are on a fast network connection, I wouldn't use MQTT at all. MQTT works best for scenarios where you need to get data from point A to point B over a slower network like cellular or satellite, where connections may be spotty and slow. If your systems are on a decent VPN connection between everything, then just name everything consistently for tag providers so that Dev Gateway B has a tag provider (local or remote) named "TagProviderB", and the Production gateway uses the same name for its tag provider, then for the central gateway, use a remote tag provider named "TagProviderB" that points to that same tag provider. If you're wanting the dev gateway to access the same data but read-only, then you'll need to set up a remote tag provider in it also, but make sure it's read-only.and instead of using the local tag provider use it as a remote tag provider. While you won't be able to make changes to it (that would have to be done on the production gateways for any tag changes/adds), you'd have access to the live data at least.
That was the original plan, MQTT got lumped in because of a need to use the Azure Injector module at the central gateway to send data to an IoT hub. In order to handle network dropouts between the central gateway and production gateway (and another gateway in the middle), it was determined that setting up an MQTT network was the best way to handle the store-and-forwarding mechanism needed between tags at the production level and their corresponding tag representation at the central gateway level.
Using MQTT for data sharing isn't required, I'm just doing my due diligence in investigating whether it's a better option for data sharing between environments than remote tag providers.
When using MQTT, as you discovered, it adds an additional failure point to the mix. You can set up a cluster, and using ACLs you can prevent writes from specific gateways if you do need write access from some gateways and not others. As you also discovered, the tags now come from the "MQTT Engine" provider rather than the tag provider named in your gateways. What I do on my central gateways where I am using MQTT is I create reference tags to the MQTT Engine tags so that I can make my structure on the gateway however I want it. I've created duplicate UDTs on the central gateway but switched all my tags from OPC/memory tags to reference tags with a base tag reference so that I can point the base local tag to the base MQTT tag and everything maps over. It also lets me independently configure alarming, history, etc on it.
This may be a way to go, and while you won't be able to copy tags or export/import tags from dev to production, they would point to the same data.
Do you have a script that you run to browse the MQTT Engine provider that builds the mirrored structure in your default provider using the reference UDTs based on the edge OPC UDTs?
Do you mind diving into how you set up the base tag reference?
I have a couple projects that are using MQTT Modules and I am wanting to make sure I am following what has been working with others on this forum. Seems like most people are recommending building up a mirror structure of the MQTT Engine but I am having a hard time understanding the easiest way to build up that mirrored structure. I don't want to have to manually create the structure for the edge sites so I think scripting will be a good option.
I literally took a UDT from the edge, and made all the tags reference tags with parameters to dynamically point to the MQTT engine tags. I didn't have a script that monitors the MQTT engine provider for new tags since they were being put into a different folder structure on the tag provider, and they weren't adding units on a regular basis. What I did do though is use the folder structure of the reference tags in my bindings of the new tags to point to the Engine tags. So I had a structure in Engine of just a folder named"AssetType-AssetTag" with all the tags under it, but in my regular tag provider, they were structured as "AssetType/AssetTag" folders. So my tags looked at their parent folder to build their reference tag path. Not every site had the same tags, so I couldn't just build out tags automatically when a new node appeared.
We have an upcoming project we're hoping to get that will have a lot more units, and a majority will be the same. I haven't thought through yet what I'll change or improve, but they will be adding units on sometimes a weekly basis. I'd probably still keep all the tags separate just due to the quantity of tags and not wanting to combine everything into one giant UDT. If I have to make a global change, I can just copy/paste a tag between folders and due to the dynamic nature, when I paste the tag, it will automatically point to the proper tag folder in MQTT Engine.
Thanks for the detailed response. This is very helpful. I am in a similar situation where each site is slightly different with the quantity of tags but have the same structure. I like the idea of being able to copy and paste between folders and it will point to the correct MQTT engine path.
To elaborate a bit, on Transmission, my Group ID was the company name (not sure it was necessary, but it's what I used), and for the Edge Node ID was the "AssetType-AssetTag" format. I also used that exact same Edge Node ID as the username which allowed me to use it in my ACLs for security since Transmission uses the "spBv1.0/GroupID/NDATA/NodeID" and "spBv1.0/GroupID/DDATA/NodeID" topics. Yes, it's probably overkill, but the idea here was that if a unit's credentials were compromised, anyone logging into the MQTT broker with those credentials would be limited to publishing to only those topics. I also prevent subscribing to anything but the STATE topics (both the new and legacy topics), and each client's own topics so that it can receive commands from the central Ignition server.
