Hello,
I'm currently working with Ignition version 8.1.32.
One of my modules stores its configuration in the Internal DB.
According to the documentation, persistent records are automatically synchronized via redundancy.
However, at times, I notice on the redundancy status page that some changes remain indefinitely in the "pending" status. (Usually, they disappear quickly and are absorbed by the backup.)
There are no errors in the logs, and even when enabling TRACE logging for the SyncManager, no relevant details appear.
Interestingly, if I make any modification to the gateway configuration while this issue is occurring, the "Pending Updates" immediately disappear.
However, the previously pending changes are not applied or synchronized to the backup.
Has anyone experienced a similar behavior or have insights into what might be causing this?
Reliability is critical for this system, and I need to ensure redundancy is both stable and fully operational for our clients.
Thanks in advance for any help.
Are you able to share any code? Are you explicitly closing/flushing datasets/etc in your ORM sessions?
I'm not explicitly managing ORM sessions or manually flushing/closing anything, as the session and flush behavior seems to be abstracted away by the Ignition framework.
To my understanding, save() immediately applies changes, and Ignition internally handles session lifecycle and flushing.
Here's an example. This function loads a set of changes from a Document (json format).
@Override
protected void importPostItsSynchroImpl(Document document) throws Exception {
if (!document.has("postits") || !document.has("updates")) throw new RuntimeException("Invalid document.");
List<String> postits = document.getAsDocumentArray("postits").toJsonElement().asList().stream()
.map(JsonElement::getAsString)
.collect(Collectors.toList());
SQuery<PostItRecord> query = new SQuery<>(PostItRecord.META);
List<PostItRecord> postItRecords = gatewayContext.getPersistenceInterface().query(query);
for (PostItRecord record : postItRecords) {
if (!postits.contains(record.getResourcePath())) {
String resourcePath = record.getResourcePath();
String role = record.getRole();
record.deleteRecord();
gatewayContext.getPersistenceInterface().save(record);
notifyPostItChange(resourcePath, role);
}
}
for (DocumentElement element: document.getAsDocumentArray("updates")) {
Document doc = element.getAsDocument();
PostItRecord record = postItRecords.stream()
.filter(t -> !t.isDeleted() && t.getResourcePath().equals(doc.get("ResourcePath").getAsString()))
.findFirst()
.orElseGet(() -> gatewayContext.getPersistenceInterface().createNew(PostItRecord.META));
record.setResourcePath(doc.get("ResourcePath").getAsString());
record.setRole(doc.get("Role").getAsString());
record.setCreationDate(!doc.get("CreationDate").isDocumentNull() ? new java.sql.Date(doc.get("CreationDate").getAsLong()) : null);
record.setModificationDate(!doc.get("ModificationDate").isDocumentNull() ? new java.sql.Date(doc.get("ModificationDate").getAsLong()) : null);
record.setConfiguration(doc.get("Configuration").getAsString());
this.gatewayContext.getPersistenceInterface().save(record);
notifyPostItChange(record.getResourcePath(), record.getRole());
}
}
You could try setting the Redundancy.StateMonitoring.SyncProviders.InternalDb and Redundancy.State.DiskSyncManager loggers to TRACE, to see if you can get any insight as to what's getting hung up.
I'll be honest - I don't see anything wrong in the snippet of code you showed at a glance, but I've also never heard of this type of problem showing up exactly.
Thank you, @PGriffith for your response.
The loggers didn’t reveal anything, even with TRACE level enabled.
I tried to take inspiration from the OPC-UA module to see how the addDevice, deleteDevice, and setDeviceHostname functions were implemented.
Based on that, I decided to replace my use of:
gatewayContext.getPersistenceInterface().save(record);
with:
try (PersistenceSession session = gatewayContext.getPersistenceInterface().getSession()) {
...
session.commitAndDetachDataSet();
}
followed by calls to notifyRecordDeleted, notifyRecordAdded, and notifyRecordUpdated.
For some reason I don’t yet understand, I haven't encountered any issues since switching to PersistenceSession.
It seems that using PersistenceSession groups multiple updates into a single transaction, which is then correctly and reliably replicated to the backup gateway.
Final code :
@Override
protected void importPostItsSynchroImpl(Document document) throws Exception {
if (!document.has("postits") || !document.has("updates")) throw new RuntimeException("Invalid document.");
List<String> postits = document.getAsDocumentArray("postits").toJsonElement().asList().stream()
.map(JsonElement::getAsString)
.collect(Collectors.toList());
PersistenceInterface pInterface = gatewayContext.getPersistenceInterface();
try (PersistenceSession session = pInterface.getSession()) {
List<PostItRecord> postItRecords = session.query(new SQuery<>(PostItRecord.META));
List<KeyValue> deletedRecords = new ArrayList<>();
List<PostItRecord> updatedRecords = new ArrayList<>();
for (PostItRecord record : postItRecords) {
if (!postits.contains(record.getResourcePath())) {
deletedRecords.add(new KeyValue(record));
String resourcePath = record.getResourcePath();
String role = record.getRole();
record.deleteRecord();
notifyPostItChange(resourcePath, role);
}
}
for (DocumentElement element : document.getAsDocumentArray("updates")) {
Document doc = element.getAsDocument();
String resourcePath = doc.get("ResourcePath").getAsString();
PostItRecord record = postItRecords.stream()
.filter(t -> !t.isDeleted() && t.getResourcePath().equals(resourcePath))
.findFirst()
.orElseGet(() -> pInterface.createNew(PostItRecord.META, session.getDataSet()));
record.setResourcePath(resourcePath);
record.setRole(doc.get("Role").getAsString());
record.setCreationDate(!doc.get("CreationDate").isDocumentNull() ? new java.sql.Date(doc.get("CreationDate").getAsLong()) : null);
record.setModificationDate(!doc.get("ModificationDate").isDocumentNull() ? new java.sql.Date(doc.get("ModificationDate").getAsLong()) : null);
record.setConfiguration(doc.get("Configuration").getAsString());
updatedRecords.add(record);
notifyPostItChange(record.getResourcePath(), record.getRole());
}
session.commitAndDetachDataSet();
for (KeyValue k : deletedRecords) {
pInterface.notifyRecordDeleted(PostItRecord.META, k);
}
for(PostItRecord p: updatedRecords) {
if (p.isNewRow()) {
pInterface.notifyRecordAdded(p);
} else {
pInterface.notifyRecordUpdated(p);
}
}
}
}
1 Like
Interesting. Maybe something to do with the (potentially multiple) calls you were doing in sequence. Still shouldn't have been an issue, but honestly the SimpleORM stuff is already gone from 8.3 so we're not going to ever take the time to investigate whatever underlying bug there might be here. Glad you were able to work around it.
4 Likes
