Problem Statement:
Our current architecture involves 16,000 tags configured with “On Change” history logging. We are utilizing PostgreSQL with the TimescaleDB extension to optimize ingestion rates and minimize memory overhead.
Test Results & Observations:
During a pilot test involving 12,000 simulated tags via the TimescaleDB Historian Module, we identified a critical throughput imbalance. The observed Store Rate is approximately 5,000 datapoints/sec, whereas the Forward Rate is limited to roughly 3,000 datapoints/sec.
Impact Assessment:
This creates a negative net data flow, leading to a continuous accumulation of pending data in the Store-and-Forward (S&F) buffer. In a real-world scenario, if a network outage occurs for 24 hours, the system will be unable to clear the resulting backlog, as the recovery rate is lower than the ingestion rate. This poses a significant risk to data integrity and buffer overflow.