system.historian.storeDataPoints does not store float values correctly to MariaDB

I don’t know if this has been reported yet, but as of 8.3.2, when using system.historian.storeDataPoints to push float values into a MariaDB historian, the value is stored as an intvalue and not floatvalue in the sqlt_data table.

Environment:

Ignition Docker, seen in both 8.3.2 and 8.3.3-rc1. 8.3.1 looks to store the value in the correct column.

MariaDB Docker image, 10.5

My DB connection to MariaDB kept all the default settings in the gateway, as did the historian setup. I think tag changes get stored to the proper column, but I didn’t fully explore that.

My code that I ran in my script console to replicate this:

histProv = "db"
tagProv = "default"
tagPath = "maria/insertFloat"


serverName = system.tag.readBlocking(["[System]Gateway/SystemName"])[0].value
insertFormat = "histprov:{}:/sys:{}:/prov:{}:/tag:{}".format(histProv, serverName, tagProv, tagPath)
print "Insert format: " + insertFormat

system.historian.storeDataPoints([insertFormat], [2.05], [system.date.now()])

@Kurt.Larson

I’m away from my computer right now, but I’ll dig into this deeper tomorrow.

Regarding the scripting function, nothing has changed specifically with system.historian.storeDataPoints since 8.3.1, but we did fix an issue in 8.3.3 regarding values momentarily hitting the wrong data column within the SQL Historian during datatype changes. Because of that fix, the core tag storage logic has been pretty heavily tested recently.

If you have time to test before I get back, could you check a few things?

Isolate the Environment

  • Have you been able to reproduce this newly on 8.3.1? I want to make sure the success there wasn't just historical/cached.

  • Can you verify that a standard Float tag (driven by a device/simulator, not the script) is landing in the proper column?

Check the Configuration

  • What does the unretired sqlth_te entry look like for one of these tags? Specifically, what is the datatype value set to?

Logger Diagnostics

  • It would be helpful to see the point when the Historian Manager processes it vs. when it actually gets stored. Can you turn these loggers to TRACE and run the script?

    • historian.manager (This shows the point as it's processed initially on the gateway-side).

    • store.forward.sink.TagHistoryDatasourceSink (Warning: This will be very spammy. This shows when the SQL historian actually stores the values).

This should tell us if the SQL Historian actually thinks the tag is an integer, or if the bridge class is converting/dropping precision before it hits the Store & Forward system.

I'll follow up tomorrow. Thanks.

Have you been able to reproduce this newly on 8.3.1? I want to make sure the success there wasn't just historical/cached.

This issue doesn’t seem to show up in 8.3.1. I spun up a different 8.3.1 Docker container and the floatvalue is stored in the right column.

Can you verify that a standard Float tag (driven by a device/simulator, not the script) is landing in the proper column?

Yes, it’s storing in floatvalue.

What does the unretired sqlth_te entry look like for one of these tags? Specifically, what is the datatype value set to?

Datatype column has a value of 0.

Can you turn these loggers to TRACE and run the script?

I’m running the same script as I described earlier, except I used 3.99 to see if it was rounding or something else. The value that is stored is 3 in the intvalue column, the sqlth_te table doesn’t change.

Thanks for checking that. Seeing the sqlth_te datatype value as 0 effectively confirms the issue. It means the SQL Historian has registered these tags as Integers, regardless of the float values you're pushing.

I'm fairly confident I've isolated why this is happening. It is specific to using the storeDataPoints scripting function without a backing tag. When you invoke this function, it generates data points that are passed to the HistorianManager. The Manager routes these points to the SQL Historian's storage engine, which then converts them into TagGroupHistorySet containing HistoricalTagValues before sending them through the Store & Forward system.

The critical piece here is that a HistoricalTagValue must be decorated with a DataTypeClass. The SQL Historian uses this class (not the raw object value) to determine how to store the data. When you use standard tags, the system provides this type metadata automatically. However, when you use the scripting function for a path without a live tag, the storage engine lacks that context and defaults the DataTypeClass to Integer. As a result, even though your object value is 2.05 (a float), the system treats it as an Integer based on that default class, casting the value and causing the precision loss.

I'm getting a ticket created to address this behavior. Since it's the holidays, we'll get this prioritized and worked on right away once we're back in full swing next week.

In the meantime, you can force the correct type by explicitly storing the metadata for these paths before you store the data. One important note is that this metadata injection is transient. The SQL Historian does not persist these manual metadata entries to disk in the same way it does for live tags, so you will need to re-populate this metadata if the SQL Historian restarts (e.g., upon a Gateway restart).

Here is a script to register them as Floats using system.historian.storeMetadata:

# 1. Define the fully qualified historical path(s)
# Make sure to update the System Name and Provider to match your setup
path = "histprov:db:/sys:Ignition:/prov:default:/tag:maria/insertFloat"
paths = [path]

timestamps = [system.date.now()]

# 2. Define the metadata payload
# We use the "dataType" property with value "Float4" 
# See https://docs.inductiveautomation.com/docs/8.3/platform/tags/tag-data-types
metadata = { "dataType": "Float4" }

# 3. Store the metadata first
system.historian.storeMetadata(paths, timestamps, metadata)

# 4. Now the storeDataPoints call will respect the float type
values = [2.05]
system.historian.storeDataPoints(paths, values, timestamps)
1 Like

Hi Kurt,

Thanks for looking into that for me, your suggestion works perfect. I don’t know if it matters anymore, but this was something that I had been able to get away with doing in 8.3.1 and I think 8.3.0. I was using storeDataPoints to simulate historical data from earlier in the year.

Glad to hear that workaround worked.

I’ve tracked down the specific regression in 8.3.2 that caused this. You absolutely shouldn't need to manually store metadata prior to storing data points through scripting.

I’ll get this fixed and will update this thread once it’s available in a nightly build. Thanks again for bringing this to our attention.

@kurt.larson Hello,

I am experiencing an issue where a float‑type historical tag is not displayed in the Perspective 8.3.2 Power Chart when historical data is delivered from an MQTT Sparkplug B edge device. My setup uses MQTT Sparkplug B tags and a MySQL database as the Tag Historian provider.

The system includes three tags:

  • two boolean pump status tags, and

  • one float tag representing the water level (values between 0.6 and 0.8 m).

When the system is fully online, all three tags record and display historical data correctly. However, when the edge device reconnects and sends historical Sparkplug B data (is_historical = true), only the boolean tags appear in the Power Chart. The float tag’s historical values are not stored or visualized.

This behavior worked correctly in Ignition 8.1.x, but it no longer functions as expected in Ignition 8.3.2. Once this issue is resolved, I plan to migrate the historian to PostgreSQL.

Could you please investigate this issue or provide guidance on how to proceed?

Thank you for your assistance.

Please clarify, there is no such thing as Ignition 8.2. It went from 8.1 to 8.3.

Hello.

Thank you for repairing my mistake. It was working fine with ignition 8.1 but it is not working with ignition 8.3.2 (with latest update).

1 Like

It's the same underlying cause as the previous issue. I'm hoping to have it fixed for 8.3.3.

@Eden.Mar @Gregor_Hocevar This has been fixed for the upcoming 8.3.3 release.

2 Likes