Bad_CommunicationError: A low level communication error occurred


Just for information or understanding for myself here.
The response we see here just before the Send Exception and the Checksum Mismatch, how is ignition deciding this as the complete response. There is no Slave ID in its response in the first byte like the previous messages with the 01 in the first byte?

I'm not really sure what's going on with that response... not sure if we're gobbling up a byte mistakenly before logging it or what. I'd have thought the driver wouldn't recognize 0x14 as a function code and it would keep waiting for more bytes or something.

This is where having a serial analyzer comes in handy - an objective view of what's sent and received without any potential bugs warping the view of what's happening.

Oh, I see... assuming that's actually the data, it's attempting to decode it as an exception response, which would probably fail more spectacularly if it weren't for the CRC mismatch that happens. If assertions were enabled this would have blown up earlier.

That's a bug in the driver, but the only result would be a different error handling the response... it's still not a valid response from the device.

I am seeing the serial tracker this morning behaving in a way where I think Ignition is not behaving as expected.

12:11:27 lining up with Ignition logs where this is showing a Checksum error, with the serial monitoring showing this as expected?

Is the data size of 49 causing an issue based on your comment above.

There has been a number of other strange occurrences in the middle of this log.
A request of 125 registers, in registers that I don't believe exists?

Another of request size 124.

Weird, with the bad response Ignition sees I can see parts of the response the tracker sees, but there's an extra byte at the beginning that seems to come from nowhere, and then everything is wrong from that point on. It seems like all of the failed responses have one extra byte at the start throwing it off.

I'll have to add some logging to the driver that just dumps raw data from the serial port right as we receive it. I'm not sure if there's a bug in the accumulation and parsing of RTU data or if we're just receiving bad data from the serial library.

You might try my alternate module. You'll have to set up a temporary full Ignition--it isn't whitelisted for Edge.

Thanks Phil,
Changing to full ignition isn't an option right now.
I will revisit should I not get a solution to this soon.

Any advice on how I can do this? should be quick to get results?
Maybe this goes more into a tech call issue?

No, nothing you can do at this point. I have to make changes to the module code. Opening a ticket with tech support would give you an official way to track this, though.

If you could give me Ignition logs with the ReadHoldingRegistersRequest logger on TRACE level + serial tracker logs that cover a period when requests go from working to not working that would probably be useful.

I meant temporarily, as a functionality test. If it works for you in trial mode, I can work with IA to get you a proper license. It just isn't automatic like other licenses, and untestable on Edge without a license.

Ah apologies for the misunderstanding.
Does your module allow for extra devices outside the 5 v8 of ignition edge allows?

Please leave this with me. The system has been powered down for a period. Will reinvestigate when it comes back online.

Hi Kevin,
Sent these files externally.

I'm still going to adding custom logging to the module, because I think it's possible the RTU parser is getting into a bad state, but there's also definitely something wrong with this device or serial connection and the scanner sees it too.

Right after the successful request/response pair at 17:28:07 the scanner also picks up weird stuff.

It sees a request at 17:28:17, and never even sees the partial response Ignition sees.

It sees a request at 17:28:27, and sees the same bogus response Ignition sees:

161672: MODBUS Request (packet size: 123, data size: 119) 2022-11-15 17:28:27.5414496 +0.0000000
Mode: RTU
Address: 000003 (Slave)
Function: 000058 ()
Unknown Function!

CRC: 11812 (WRONG)

Then it sees another request at 17:28:37 that it thinks gets a valid response but Ignition is only seeing it as partial/incorrect. Not sure what's going on here, this is where the logging I want to add will tell me if we're just not getting the data from the serial driver or if we've got into a bad parsing state somehow and can't recover.

I think I can force this Error by changing tag group properties at ignition designer level.
Every time I change the tag group this occurs up to now.

Also, this is the request made, not the response from the device? All response from the device is showing up as Modbus Response, not Modbus Request?

The piece that is still pointing me towards an ignition bug here is that If I change scan class properties, or make a change to the device connection, I always get 1 good read?

I can see only failed to respond events in the serial tracker, however ignition seems to be parsing expected response incorrectly?

The number of failed responses from the device is very low in comparison to the number of errors the logs are showing.

I had not got the same level or logging when I started this conversation, but the 3rd party software ran for quite some time, using same serial port with no dropouts.
I also had nearly 1 week of no issues via ignition as well in between occasion where this performed in a bad comm's state.

Donno, looks mislabeled by the tracker. It's immediately after a valid request (milliseconds after) and it matches what Ignition sees as a response at the same timestamp:

160926: MODBUS Request (packet size: 8, data size: 4) 2022-11-15 17:28:27.5192480 +0.0000144
Mode: RTU
Address: 000001 (Slave)
Function: 000003 (Read Holding Registers)
	� Starting Address     		= 5
	� Quantity Of Registers		= 5-33 (29)

CRC: 38338 (OK)

161672: MODBUS Request (packet size: 123, data size: 119) 2022-11-15 17:28:27.5414496 +0.0000000
Mode: RTU
Address: 000003 (Slave)
Function: 000058 ()
Unknown Function!

CRC: 11812 (WRONG)

It would be nice if the tracker wouldn't do Modbus decoding and just showed bytes sent/recvd instead. And it's still strange that the tracker isn't seeing responses when Ignition is somehow.

There is a feature to show us bytes I believe but I think it would need live viewing instead of exports because it doesn't have associated timestamps. It just is an array of bytes, colour coded of for ease of separation is the only nice thing it has.

What happens a tag when the tag group properties is changed, does it finish its execution 1 more time on old properties? This is what I thought was happening when seeing that behaviour.

if you changed the rate or something then it would get re-subscribed against the server. The driver isn't reset or anything, it would just possibly cause a delay in between the old request being canceled and the new one starting...

Can you try installing this module an then turning the "SerialIODelegate" logger to TRACE level?

Same debugging protocol as before with matched up logs and serial sniffer if possible.

See 1 timeout Error here.

We can see a double request made at 21:49:26

And a "Double Response to this at 21:49:48