Best Practices for Adding Fields to Persistent Records

When making a persistent record with lets say 10 fields, when that module is first installed on the gateway it will create a SQLITE table that is 10 columns. If someone else who uses the module and then says hey, it’d be nice to have these additional config fields too, can you please add them?

So then you add 1 field (for example) and the new module now has 11 fields. If you then try to install that new version of the module on the same gateway it will throw an error because the column for the new field doesn’t exist in the SQLITE table.

What seems like it needs to happen is this:

  1. check if the table with that name already exists, if not make it
  2. if the table exists, read in its values then drop the table
  3. Create the table and pass in the old values plus any for the new field

I’d like to hear what everyone’s best practices are for handling this and if possible see some code examples.

This sort of functionality is not in the SDK examples so if we can understand how best to handle this, I’d like to make a pull request to add it since it seems like a pretty important feature to have assuming that modules will change over time.



When your gateway hook’s .setup() method is called, it should be calling:


That will add new columns to your tables. I don’t think it will change any existing columns.

1 Like

What @pturmel said

1 Like

OK, I do see that but I did see an error raised when adding a new field. I will re-verify:

  private void verifySchema(GatewayContext context) {
      try {
      } catch (SQLException e) {
          log.error("Error verifying persistent record schemas for KafkaConnect records.", e);

The behavior I saw was after adding a new field, the module would install in a faulted state.


Are you sure you just added the one column? Did you assign an appropriate default value to that new column? When the module faulted, what showed up in the wrapper log?

What was added was one field to specify minimum alarm priority:

  public static final EnumField<alarmPriorities> MinimumPriority =
          new EnumField<>(META, "MinimumPriority", alarmPriorities.class, SFieldFlags.SMANDATORY).

I will try to find the error, need to go back in the log a bit