Home Rolled MQTT Module

When I started poking at my new driver, I thought it would be two, three months. Closing in on 18. (Not full time, though.)

Interesting discussion. I should note that our specific application is just to have a way to subscribe to messages.

These messages are from package scan tunnels where packages move quick fast so the value to look at a single tag value is almost zero. Actually if all scan tunnel messages go to one topic and we subscribe to that one topic, it'd be somewhere in the range of 1,000 messages per minute.

Instead, we just need a way to grab the messages and then store them to database. Only once they have been stored and then analytics can be run on them do they become useful.

I'm actually thinking about doing "buffered MQTT" so that the messages can come in as fast as they like and we only have to deal with them once per second as a comma separated string, very similar to the way we do Buffered TCP. But instead of an actual buffer, just stick them in an array or similar and then empty it out every one second.

Nick

Consider a ring buffer, where each entry (scan string?) publishes via MQTT. The tunnel just advances to the next, wrapping around, for each scan. With 100 items in the ring, each would publish 10 per minute. Very reasonable time period to handle on the subscriber end.

I was wrong, it looks like as SP/Tahu approaches 1.0 it now includes a Host Application API as well: tahu/java/lib/host/src/main/java/org/eclipse/tahu/host at master · eclipse/tahu · GitHub

This looks like it would significantly lower the barrier to building the host side of a SP setup.

2 Likes

Ready by Christmas, then? :slight_smile:

3 Likes

We are working towards making a full parallel tag tree mirroring our MQTT Engine tags. We have continuously bumped into cases where the functionality and behavior of native MQTT Engine tags are lacking or different compared to other Ignition tags. But we discovered that most (all?) of those bad behaviors didn't happen on a reference tag pointing at the MQTT Engine tag. We don't even want to rename/reorganize the tags, we just want the more consistent functionality. Our plan is to auto-create a tree of reference tags that mirror the edge node tags in name and everything.

We've been re-checking all of our views and scripts to find and fix every place that had [MQTT Engine] coded in the tag paths and make them all dynamic for when we switch, and also been creating the scripts that will notice new MQTT Engine tags and create the reference tags on the fly while moving any "legacy" alarm or history settings from the MQTT Engine tags to the new copies.

It's a bear of a job. We did not pick this path lightly.

I know, see my update post.

In reply to the original question, and with the caveat that I have not read the entire thread here yet, yes -- sort of. We created a module that allows MQTT publishing of data from one of our other modules, using the Eclipse Paho library previously mentioned. It's definitely not even close to a one-to-one replacement for the Cirrus Link modules (otherwise we'd probably have just used those).

SparkPlug isn’t totally decentralized or uncoupled. One subscriber must be acting as the “primary” application, responsible for monitoring and control of the Edge nodes (publishers). There can be other subscribers that are purely observing, or waiting in standby to take over, but something needs to be acting as primary.

Was reviewing the Sparkplug repo recently in hopes that version C would be a little more flexible than B and noticed this.

1 Like

This is exactly what we ended up doing, and also auto adding history to mqtt tags as well

We're moving the history to the reference tags instead of the MQTT Engine tags. It uses a more informative quality code for backfilled data (MQTT Engine just marks backfilled data as "Good") and also can optionally use scaling properly on history. If you backfill history on a scaled MQTT Engine tag, it is stored differently than non-backfilled data and history queries on that tag will jump back and forth between properly and improperly scaled values.

Interesting to know. Your derived tags won't have backfilled history though, so are you just making do without it, or come up with a workaround?

If you ensure that "Store Historical Events" is disabled on MQTT Engine config and that "Allow Back-fill Data" is enabled on the tag provider holding the reference tags, the reference tags WILL get backfilled history! (This latter option is a new feature in the past year-ish?)

3 Likes

Hi, we have done one already,,with client and broker,,

Perhaps you should PM the OP and make a commercial deal. Or publish the module you've created.