# How to batch/buffer fast data before working with them **URL:** \ **Category:** Ignition\ **Created:** [March 28, 2025, 2:38pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375 "2025-03-28T14:38:35Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Andre\_Hennecke](https://avatars.discourse-cdn.com/v4/letter/a/e95f7d/32.png) [@Andre\_Hennecke](https://forum.inductiveautomation.com/u/Andre_Hennecke)\ **Post date:** [March 28, 2025, 2:38pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/1 "2025-03-28T14:38:35Z")
Hi, Just want to ask if there are any best practices in working with "fast" data changes. Lets say we have an event tag which changes ~100ms. What is the best way to collect the data before sending them e.g. to a REST API or a SQL DB after a specific time. Lets say 1 sec. One way might be to write the data to a memory tag array via a gateway change event and then read/publish the array after in a 1 sec timer gateway event? Thank you for additional hints đŸ™‚ AndrĂ© ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [March 28, 2025, 2:57pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/2 "2025-03-28T14:57:37Z")
For those kinds of speeds, you are best off using a ring buffer data structure (array of structures) in your PLC, with `head` and `tail` integer tags for the current subscripts into the array. Only those two tags would be subscribed. `head` would be updated by the PLC as entries are filled. `tail` would be updated by Ignition as entries are delivered to the consumer. Use a timer at the desired slower pace to perform a bulk PLC read of the entries between `head` and `tail`, send them to the consumer, then update `tail`. ---
**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\ **Post date:** [March 28, 2025, 3:03pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/3 "2025-03-28T15:03:01Z")
I think this kind of problem (when it doesn't involve trying to poll fast changing data from a PLC) will be solved nicely by event streams in 8.3. There is a buffering stage with debounce time, buffer size, discard/overflow policy, etc... that can be configured, after which the downstream handler receives all of the events rather than one event. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [March 28, 2025, 3:12pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/4 "2025-03-28T15:12:57Z")
> [@Kevin.Herron](#): > > fast changing data from a PLC Yes, for v8.3 I'll probably use the ring buffer just for PLC =\> Ignition, then hand off to the event stream module. ---
**Author:** ![Andre\_Hennecke](https://avatars.discourse-cdn.com/v4/letter/a/e95f7d/32.png) [@Andre\_Hennecke](https://forum.inductiveautomation.com/u/Andre_Hennecke)\ **Post date:** [March 28, 2025, 4:24pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/5 "2025-03-28T16:24:45Z")
@Kevin.Herron Event Stream sounds interesting. @pturmel Agree, thats a good solution. Unfortionatly, we can't change the source of the data - at least not the legacy devices. We currently get the values via their OPC-UA server, which works good. All values are received in Ignition. But we want to then have a buffering stage in Ignition and forward all data after a specific period. But it looks like that v8.3 has a solution for this case. Thank you both! ---
**Author:** ![joe.bernicke](https://avatars.discourse-cdn.com/v4/letter/j/ea666f/32.png) [@joe.bernicke](https://forum.inductiveautomation.com/u/joe.bernicke)\ **Post date:** [April 15, 2025, 8:31pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/6 "2025-04-15T20:31:12Z")
What do you mean by ring buffer here? Is this a device or just logic within the PLC/ignition? ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [April 15, 2025, 9:00pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/7 "2025-04-15T21:00:38Z")
> [@joe.bernicke](#): > > mean by ring buffer Code and data structures from PLC through SCADA to DB that hold critical information that simply cannot be allowed to fall into the bit bucket on the floor. Usually configured to hold in the PLC some small multiple of the maximum connection downtime that can be predicted for the overall system. Data is delivered, usually many pieces in parallel, through SCADA to the high-reliability database, where each piece of data gets a distinct handshake to the PLC after its insert transaction succeeds. The PLC would be programmed to gracefully shut the process down if it hits some capacity threshold in the buffer. The data transfer bandwidth in normal conditions would have to be somewhat faster than average data production in the process, to have the ability to "catch up" after disruptions. The "ring" in ring buffer indicates a technique in the PLC that avoids moving data in the buffer, but instead adjusting subscripts for head and tail locations. ---
**Author:** ![joe.bernicke](https://avatars.discourse-cdn.com/v4/letter/j/ea666f/32.png) [@joe.bernicke](https://forum.inductiveautomation.com/u/joe.bernicke)\ **Post date:** [April 16, 2025, 10:18am UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/8 "2025-04-16T10:18:21Z")
Thank you so much for the help! Would this allow me to get sub 5-10ms sampling interval on data acquired? Do you have any resources I can look at? ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [April 16, 2025, 12:01pm UTC](https://forum.inductiveautomation.com/t/how-to-batch-buffer-fast-data-before-working-with-them/104375/9 "2025-04-16T12:01:21Z")
> [@joe.bernicke](#): > > allow me to get sub 5-10ms sampling interval Yes. I use it with my EtherNet/IP module to sample fast and deliver in batches. One customer samples at 15ms and delivers with a 100ms RPI. (With room for 10 samples per batch for the "catch up" capacity.) > [@joe.bernicke](#): > > Do you have any resources I can look at? No, sorry. Everything I've done with these techniques is under various NDAs. I suppose I should create a generic template for this.