Transaction group doesn't insert data to database even designer says it executed

Hi all,
Recently we had a problem with the transaction group, the condition was triggered, total execution was increased, there was no quarantined data, but I couldn't see any data at the time. And around 20 minutes later, all the data popped up.
Please find the attached pic and feel free to give your opinions!


Data that is buffered by the S&F engine is not "quarantined". It is simply buffered. Quarantine is for records that the DB actively refuses.

You should inspect your DB's status and performance. A long blocking query could have been holding up the inserts from your S&F subsystem.

1 Like

I had a look at store&forward but no error shows. I also had an overall look in connection tab inside gateway status, no error shows!

So I've recreated new table inside the sql database. Problem still happens as you can see in attached pic. There is no error in gateway status overall.

Does your database table have any unique constraints or indexes?

Edit: Ehh, never mind. Wasn't thinking clearly. Would throw an error if that were the culprit. Also looks like you're utilizing the transaction group to create the table, so that should be ok.

Yup! I was thinking about the unique cons or indexes, that's why I've changed the name of old one, and use the automatically create the table option. But it still behaves wrong!

Getting screenshots of the entire Action, Trigger, and Options panels will help with troubleshooting. Also, anything showing up in your "Events" tab (to the right of your "Status" tab)?

So I've figured out all misssing data are in store and forward. Even though no error in gateway log and the store and forward said no data was quarantined. But I noticed there was number 2/25000 records. And I did a backfill store and forward method, luckily got all data back. Question is what reason caused this?
@pturmel I've searched a comment from you said to turn on the bypass store and forward. Can I ask why?

IIRC, in that case, the user needed the data in the database immediately to display on the UI. The S&F buffer will always add some latency, and screw up a UI. For typical UI operations, you want the DB operation to fail with immediate notice to the user, not be buffered.

But let me rephrase and elaborate on an earlier statement: Buffering in the S&F system is not an error, nor is it quarantine. Your comments above expressing surprise that there were neither errors nor quarantined records suggests a fundamental misunderstanding of the S&F system. Buffering when the DB cannot keep up or is briefly unavailable is the intended purpose.

It was not "behaving wrong".

1 Like

Thanks for clarifying.
"you want the DB operation to fail with immediate notice to the user, not be buffered." What if the operation still continues during the fail, and bypass is on, all data during that time would not be stored anywhere else ?

The S&F system is intended to allow a recording process to continue operating at speed while a DB connection is broken or temporarily running slower than the process. When the DB returns to normal operation, the buffered entries will be stored in the DB.

If the DB does not return to normal, then the buffers will reach capacity and then you will get errors.

If you don't use the S&F system (bypass), then a recording process will either receive an error for a broken DB, or will fall behind on its sampling while waiting for the DB. Separately for each DB operation. Any process operation that needs 100% confidence of recording (product per-piece quality tracking, perhaps) should not use the S&F system, so that the production line can pause until the DB properly stores the required information.

1 Like