We are scaling out or ignition deployment. We have 650K tags and are on track for 2M in the next year, with about 5% historical. I am seeking advice on how best to organize the historian.
My current thought is that multiple backend gateways will handle the OPC and logging. The target database will be a single TimescaleDB instance and it will handle all of the partitioning and pruning. We will have a historical DB (same server) for each gateway.
I don't want to go third party if I can avoid it.
Some questions:
Is there any benefit to using the remote history provider instead of direct logging to the DB from the backend gateways? (any benchmarks?)
Is there a performance benefit to loading Ignition onto the DB server? (any benchmarks?)
Any help is appreciated.
PS - One other question. What is the highest observed throughput between ignition and a Logix Controller? Most are 1756-L82ES, or there abouts with dedicated comm cards.
I would think that would perform worse than direct, but there might be combinations of WAN and DB location where it matters. Sorry, I neither have nor have seen any benchmarks for that.
All downside. Ignition is a dramatically different kind of workload from a database. Ignition is very sensitive to latency, where databases are more demanding on CPU performance. When a database and Ignition are installed together, the DB will steal all the CPU time from Ignition, crushing it. Keep them separate. (If VMs, make sure the hypervisor is set to not steal Ignition's idle CPUs for the DB VM to use. Ignition needs idle CPUs.)
The PS is a question close to my heart (and business), and it drove me to create a third party module. If not using a third party driver (* cough *, mine), your Logix performance will be entirely dependent on how well your tags are organized for optimization. That is, consolidated into large, completely readable arrays and/or structures, with no use of AOI data types. When data is compact like that, you can expect to max out somewhere in the hundreds of kB per second range. Choose small data types to maximize the total tag quantity.
If you want to go faster, try my driver. If you want to go fastest, use PLC code to push high-rate UDP packets to my driver's Adapter mode instances.
We're probably fairly close to the 8.3 release. 8.3 adds a built in historical logging time series database that can handle time series data much better than what is available in 8.1.
I would recommend waiting for 8.3 for the historical logging requirement.
With 650k+ tags it's relevant to discuss what kind of devices (AB, Siemens, etc) you're pulling tags from, how many tags from each, etc. Based on your PS statement, I'll assume most of your stuff is AB. I only have one customer that was pulling 650k+ tags using the standard AB driver and they were having a LOT of problems using the default driver even with absolutely absurd system resources on their Ignition Gateway.
You don't want to pull a large number of tags from a single device with the out of the box AB driver because you'll overload layer 3 comms on the PLC which impacts a lot of things. @Pturmel built a module that optimizes communications that can be used to handle larger tag counts better.
I see Phil responded to your post. Definitely seriously consider anything he says on this topic.
If you're planning on getting data from multiple sites you should look into MQTT. You want to optimize the driver comms on each site using either Phil's module or FT Linx Gateway. Consider using an MQTT implementation to transport that data to a master server if that makes sense in your environment.
If you're dealing with a large number of visualization instances you should consider using a scale-out architecture. Ignition handles visualization very well but you also have the option of running your backend gateway processing on one server and your visualization on another so you can handle a large number of visualization instances without impacting your data collection. This is sometimes important at the scale you're talking about.
I would recommend checking out the training on the Opto22 site to get more details on that. I found their training to explain the benefits of MQTT pretty well. Whether you use their products or not; they explain it well.
All this to say, I'm just trying to give you good search terms to help you make a good decision.
35 machine/process gateways, 9 larger units have about 55K tags per gateway, 5 vision users, and 5 plcs. Smaller units have 7K tags per and 1 vision user and 1 plc
1 plant instance handles perspective for 70ish office users and dashboards. It also talks to 90ish devices. This is targeted for scale out.
285 total field devices. 95% are AB Logix with dedicated Ignition ports. The other 5% are Siemens or Modbus....just to keep life interesting.
Plant network is 10GB backbone, 1GB to the PLC, 40GB in the data center.
Our main plant instance is problematic and we are targeting a scale out. Our Historian is problematic. We originally went with one MSSQL database shared by all gateways and we are now running into significant deadlocks and bad query performance. The historian a result of bad decisions on my part. I never expected 650K tags, let alone 2M.
The plan is:
Split the plant instance / scale out
Consolidate small gateways running at the machine into one or two larger gateways in the VM farm.
Move to Timescale DB, with each gateway having its own historical database to mitigate / avoid deadlocks. Many, small partitions.
Not that I'm aware of, I'd get in touch with the sales engineering team with your specs and questions.
Yeah, I just wanted to clarify, because when Phil is suggesting not to mix workloads I'm pretty sure he means a regular Ignition Gateway, e.g. one that would also be handling e.g. Perspective Sessions or Vision Clients workloads.
I don't think deadlocks will be a concern on the new timescale db system coming in 8.3. I think a lot of your performance concerns from a time series data handling standpoint will be smaller concerns on the 8.3 historian stuff. It's worth considering a plan to transition to the new 8.3 model (have activations in support). You probably won't be able to migrate data from old to new but you could do 2 sets of screens or just hard cutover or something.
I would definitely recommend a scale out architecture with 70 clients. I've seen problems on a large tag count Ignition gateway that was only running like 30 clients. They were trending way too many tags on way too few PLCS at way too high of a collection rate but regardless, I think you're looking to have a more saleable setup and that's the direction I would recommend at that scale.