# Storage after running script **URL:** \ **Category:** Ignition\ **Created:** [December 16, 2021, 7:03am UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184 "2021-12-16T07:03:38Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 7:03am UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/1 "2021-12-16T07:03:38Z")
Hi all, I am running a script which is opening .csv files and stores them in the database. The script runs good, all data is coming in nicely. But when I check the storage capacity of the machine I am working on, I see that it takes up to 2 GB after running one csv file. My personal storage gets decreased. I have to run about 300 csv files so this would take up to 600GB which we don’t have. Is the script storing data somewhere? I noticed when I restart my client (and thus also the gateway since they are on the same machine) the storage is back to normal. Anyone had experience with this? Thanks in advance! BR ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [December 16, 2021, 2:12pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/2 "2021-12-16T14:12:51Z")
Sounds like your script is creating temporary files, directly or indirectly. Share your script. (After pasting, highlight the code and click the “preformatted” button–this one: \.) ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 2:50pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/3 "2021-12-16T14:50:15Z")
```auto trigger = system.tag.read("[graphs]General/ImportOldData").value if trigger == 1: import csv import sys store = "Metingen" # Get the csv filepath path = system.tag.read('[graphs]ImportTagsOBViews/FileName').value #print path try: csvData = csv.reader(open(path)) except: sys.exit() try: header = csvData.next() # Create a dataset with the header and the rest of our CSV. dataset = system.dataset.toDataSet(header ,list(csvData)) except: sys.exit() tagPath = system.tag.read('[graphs]ImportTagsOBViews/MeasurementName').value if system.tag.exists(tagPath): system.tag.storeTagHistory("NewConnection","Graphs",[tagPath],[value],[192],[timestamp]) for row in range(0,dataset.getRowCount()) : input = dataset.getValueAt(row,0) lastSpaceIndex = input.rfind(" ") value = input[lastSpaceIndex+1:] timestampStr = input[:lastSpaceIndex] timestamp = system.date.parse(timestampStr, "yyyyMMddHHmmss") value = float(input[lastSpaceIndex+1:]) system.tag.storeTagHistory(store,"graphs",[tagPath],[value],[192],[timestamp]) else: sys.exit() system.tag.writeBlocking("[graphs]General/ImportOldData", 0) ``` Hope this helps! ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [December 16, 2021, 3:24pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/4 "2021-12-16T15:24:10Z")
I see a syntax error (indentation) at your first call to storeTagHistory, but I don’t otherwise see any temp space consumers. Unless storeTagHistory itself does so. I wouldn’t bother creating a dataset, by the way. There’s no reason you shouldn’t just insert into history as you read each row from the CSV. Your script will use much less RAM if you don’t pull the whole CSV into a dataset. ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 3:33pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/5 "2021-12-16T15:33:05Z")
Hmm ok. The explanation for the possible error is that I’ve deleted a few print statements. Maybe I took away an indentation. Do you have any idea where I can find these temp files? (if they are created). I can clear some space by restarting the machine, but I can not do that if it in production. BR ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [December 16, 2021, 3:38pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/6 "2021-12-16T15:38:13Z")
I would use FileLight or WinDirStat or similar tools to do a before & after comparison of your disk usage. ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 6:29pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/7 "2021-12-16T18:29:21Z")
Call me stupid, but I’ve noticed I still had some print actions on in the script. So I had a ton of prints that were stored in the wrapper log files. I did not noticed it since the vision client and the gateway are both running on the same machine. I still need to test if this is the root cause of course. ---
**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\ **Post date:** [December 16, 2021, 6:34pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/8 "2021-12-16T18:34:05Z")
Wrapper log files don’t grow unbounded (absent some kind of bug), they are capped at 10mb and 5 files by default. Sounds like either the actual history storage is taking up way more space than you think (how many rows?) or your script is a memory hog and you’re swapping large amounts of memory to disk. Or maybe the Jython/Python CSV functions use a temp file in some way? ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 6:38pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/9 "2021-12-16T18:38:54Z")
I had 10GB worth of wrapper log files. So I think something went wrong with the installation or someone adjusted this. There are 800000 lines in one file. Could this be a problem? BR ---
**Author:** ![paul-griffith](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/paul-griffith/32/37249_2.png) [@paul-griffith](https://forum.inductiveautomation.com/u/paul-griffith)\ **Post date:** [December 16, 2021, 6:41pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/10 "2021-12-16T18:41:12Z")
Seems suspicious. What does your `ignition.conf` file (in the `data/` directory) look like? There should be some fairly clear settings around log file rotation and maximum size there. ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 6:44pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/11 "2021-12-16T18:44:10Z")
I can give this information tomorrow. Then I have access to the system. But I was pretty stunned by the amount of log files that were in the folder. The installation was done by a coworker so I’ll ask him tomorrow as wel. As long as I know that there are no reasons to believe that it is the script, it is ok for now. But if someone sees anything suspicious in the script, please let me know 🙂 ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 8:18pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/12 "2021-12-16T20:18:35Z")
Ok I managed to get the file. I can also see that I have 60 wrapper log files at this moment. Which is strange because I can find following information in the conf file: wrapper.logfile.maxfiles=5 wrapper.logfile.maxsize=10m Are there any other parameters I should check? ---
**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\ **Post date:** [December 16, 2021, 8:22pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/13 "2021-12-16T20:22:15Z")
I don’t think you did anything wrong, it sounds like “absent a bug” isn’t the case for some reason. Delete them and keep going maybe? If you want to call support and open a case to share your script in case there is some particular about it that triggers this bug go ahead. ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 10:39pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/14 "2021-12-16T22:39:05Z")
I’ll open a ticket and put the script in there. I am running this script now and the memory is going up and down. So i guess this would be temp files. In the bigger picture, I would think this only goes up. BR ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [December 16, 2021, 10:50pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/15 "2021-12-16T22:50:15Z")
Also submit your complete ignition.conf file. ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 16, 2021, 10:58pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/16 "2021-12-16T22:58:56Z")
Thanks for the advice. I’ll add it 🙂 ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 19, 2021, 6:58pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/17 "2021-12-19T18:58:12Z")
So I’ve made a support ticket and looks like the store&forward takes up a lot of space. It makes sense since the memory storage is OK when the store&forward is done. Are there ideal settings? I see that the gateway here has some strange settings: Memory buffer size = 250 store settings: Max records = 25000000 Write size = 10000 Write time = 1000 The forward settings are the same as the store settings. Do theses make sense? I’ve reduced the Max records with factor 10 because it is way to much. Best regards, Any advice on this? There has to be a lot of data. ---
**Author:** ![Thomas\_Putteman](https://avatars.discourse-cdn.com/v4/letter/t/d2c977/32.png) [@Thomas\_Putteman](https://forum.inductiveautomation.com/u/Thomas_Putteman)\ **Post date:** [December 21, 2021, 8:37pm UTC](https://forum.inductiveautomation.com/t/storage-after-running-script/54184/18 "2021-12-21T20:37:17Z")
We have edited the gateway script and added a sleep function after every storeTagHistory. The script runs slower, but it is stable and we do not have any more problems with the store&forward. This seems to be OK for now. I think we just wanted to go to fast and thus we overloaded the system 🤪 Thanks for all the input, guys!