system.tag.storeTagHistory() causing duplicate sqlth_te entries for same tag path with no structural change

Ignition
1

Ignition version: 8.1.43
Database: PostgreSQL
Module: Tag Historian (standard SQL historian)

Issue:

I am storing time-series data into Ignition's tag historian using system.tag.storeTagHistory() from a Gateway Timer Script. The values and timestamps originate from an external ERP/API source (not live tag scan-class updates) — the script fetches data on a schedule, processes it, and writes it into two memory tags using storeTagHistory.

Recently I noticed that sqlth_te contains two separate entries for the exact same tag path, with different id values — one retired, one active:

id:23 | RETIRED at <timestamp> | path:production/igu/igu 1/sealant_consumptions/filtered_sealant_consumption
id:21 | ACTIVE              | path:production/igu/igu 1/sealant_consumptions/filtered_sealant_consumption

The tag path is identical, and I have not:

  • Changed the tag's data type
  • Deleted and recreated the tag
  • Edited the tag's History configuration (Provider, Sample Mode, etc.)
  • Restarted the Gateway during a config change
  • Moved the tag to a different folder

The only operation happening repeatedly is system.tag.storeTagHistory() being called from a scheduled Gateway script, roughly every hour, writing batches of historical rows with explicit timestamps (not "now").

Questions:

  1. What internal conditions cause Ignition to retire an existing sqlth_te row and create a new one for the same tag path, beyond the documented triggers (datatype change, tag recreation, history config edit)?
  2. Can repeated calls to system.tag.storeTagHistory() with externally-sourced timestamps (not real-time system.date.now()) ever cause Ignition to treat the tag as "new" internally — for example if there's a long gap between calls, or if the Gateway restarts between calls?
  3. Is there a way to force storeTagHistory to resolve to a specific tagid rather than relying on path-based lookup against sqlth_te, to avoid this ambiguity?
  4. Are there known issues with storeTagHistory and memory tags specifically (vs. OPC tags) that could explain this re-registration behavior?

What I've tried:

  • Retiring the duplicate sqlth_te row manually via SQL and continuing to use the same tag path — this resolves the immediate issue but I'd like to understand the root cause to prevent recurrence.
  • Checked Gateway logs around the retirement timestamp — (no observations)

Any guidance on the internal logic Ignition uses to decide when a tag path "ages out" of its current sqlth_te entry would be greatly appreciated, since this is causing chart binding issues and requires manual cleanup each time it happens.

2

I've been running into a similar issue with historical backfills. As I understand it, this is the expected behavior of Ignition to preserve history and provide data validation. When querying using built-in functions or displaying trends in power charts, for example, Ignition will stitch together any historical data from the same tag path underneath the hood even if it is stored under different tag IDs in the underlying database. This, to me, says that they're aware of this interaction and have built their core systems around it rather than changing it.

Often, history injected via scripting becomes associated with a "virtual tag" while history collected through normal means (real time tag value -> historian -> DB) becomes associated with the realtime tag ID. Basically, virtual tags are the retired tags you see with creation/retired date matching the oldest and newest data in the scripted backfills, respectively. Referencing the documentation for system.tag.storeTagHistory():

image
image867×415 24.8 KB

As we can see, the relationships between tag paths and tag IDs in sqlth_te are cached in the Gateway memory and will not be refreshed until either the Gateway or Historian is refreshed.

Often, the creation of virtual tags coincides with a script inserting data older than the creation date of the realtime tag or when an identical tagpath has two data points at the same millisecond (big issue in Ignition). This could happen if your tags are storing their own history in addition to the externally-sourced history.

One known workaround for the first problem is to set the "created" date in sqlth_te for the realtime tag ID (with retired == NULL) to a time before the oldest history to be backfilled and then refresh the Historian by updating its description in the Gateway to update the relationship cache. Once this change takes effect, imported history typically backfills under the existing realtime tag ID rather than a new virtual tag.

For the second problem, you need to ensure that it is physically/mathematically impossible for multiple data sources to store history under the same tag path with identical Epoch timestamps (whatever that may look like for your project in particular).

Given that your backfill is running in a scheduled script with presumably "new" data, I am unsure if you're experiencing a separate behavior entirely. Hope this helps anyway!