# Struggling to prevent ignition from bogging down external apis that it is pulling data from **URL:** \ **Category:** Ignition\ **Tags:** perspective, scripting\ **Created:** [April 11, 2025, 9:06pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021 "2025-04-11T21:06:44Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![D\_Mitri\_Tompkins](https://avatars.discourse-cdn.com/v4/letter/d/a3d4f5/32.png) [@D\_Mitri\_Tompkins](https://forum.inductiveautomation.com/u/D_Mitri_Tompkins)\ **Post date:** [April 11, 2025, 9:06pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/1 "2025-04-11T21:06:44Z")
What is the best way to include a "rate limit" for scripts that loop through an api to pull data into a dataset in the report module? Or even just a way to limit requests in general? Can a hard request limit be set in the gateway to limit more than x requests to specific / any external urls in a given time frame? ---
**Author:** ![Spencer\_Lommel1](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/spencer_lommel1/32/78679_2.png) [@Spencer\_Lommel1](https://forum.inductiveautomation.com/u/Spencer_Lommel1)\ **Post date:** [April 11, 2025, 9:09pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/2 "2025-04-11T21:09:35Z")
could you provide a bit more context on the script? Is it a gateway script, project library script, scripting on a component? An example of the code you already have would be helpful too! ---
**Author:** ![D\_Mitri\_Tompkins](https://avatars.discourse-cdn.com/v4/letter/d/a3d4f5/32.png) [@D\_Mitri\_Tompkins](https://forum.inductiveautomation.com/u/D_Mitri_Tompkins)\ **Post date:** [April 11, 2025, 9:15pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/3 "2025-04-11T21:15:29Z")
Certainly, hopefully this is useful: Image; ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/3/9/391dc14199c0206abd9f57c094dad2dcd8103788.png) ```python def updateData(data, sample): dt = system.date.format(system.date.now(), 'yyyy-MM-dd') url = "#####example#URL##########".format(2025) result = system.net.httpGet(url, username='ignition', password='ZYcSqz13Yv', useCaches = False) datasource = system.util.jsonDecode(result) max_analysis_results = [0] def sortOrders(data): product = data['InventoryTransaction']['Product']['Name'] pounds = data['InventoryTransaction']['Pounds'] quantity = data['InventoryTransaction']['Quantity'] base_data = [product, pounds, quantity] return base_data return None, None rawOrders = [sortOrders(dataset) for dataset in datasource] rawOrders = [item for item in rawOrders if item[0] is not None] headers = ["Product", "Pounds", "Quantity"] order = [] for base_data in rawOrders: row = base_data order.append(row) data['OrderInfo'] = system.dataset.toDataSet(headers,order) ``` ---
**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\ **Post date:** [April 11, 2025, 9:17pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/4 "2025-04-11T21:17:58Z")
There's nothing in Ignition that will do this for you. For data you need periodically refreshed consider a gateway timer or scheduled script that calls it periodically and puts the results into a document tag or something. p.s. your password showing in both your screenshot and script text. ---
**Author:** ![D\_Mitri\_Tompkins](https://avatars.discourse-cdn.com/v4/letter/d/a3d4f5/32.png) [@D\_Mitri\_Tompkins](https://forum.inductiveautomation.com/u/D_Mitri_Tompkins)\ **Post date:** [April 11, 2025, 9:18pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/5 "2025-04-11T21:18:09Z")
The shared code is one example of the suspected API throttling, it does a data request from a company built and maintained API that serves as a data handler, but I believe the nature of how it updates or runs is causing problems. IT says they are experiencing major throttling issues all coming from ignition requests, and while it is difficult for me to know whether these types of scripts are responsible or not, I am hoping for some kind of solution that can just straight out time lock the number of requests Ignition can push out in a given time frame. ---
**Author:** ![D\_Mitri\_Tompkins](https://avatars.discourse-cdn.com/v4/letter/d/a3d4f5/32.png) [@D\_Mitri\_Tompkins](https://forum.inductiveautomation.com/u/D_Mitri_Tompkins)\ **Post date:** [April 11, 2025, 9:19pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/6 "2025-04-11T21:19:20Z")
Not even something (as arduous as it would be) that could be added to each request script similar to this to prevent it from actually running/ querying the API too often? ---
**Author:** ![ryan.white](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/ryan.white/32/73193_2.png) [@ryan.white](https://forum.inductiveautomation.com/u/ryan.white)\ **Post date:** [April 11, 2025, 9:19pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/7 "2025-04-11T21:19:51Z")
This API poll is only getting called once per report generation, so unless you have multiple reports generated per second, this specific script is likely not the source of your throttling. ---
**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\ **Post date:** [April 11, 2025, 9:19pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/8 "2025-04-11T21:19:56Z")
Anything you want to invent to solve this can be added to it, but there's nothing ready to go for you. But @ryan.white is on to something here -- you should probably better understand what your project is doing and where these APIs are being called from and how often. ---
**Author:** ![ryan.white](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/ryan.white/32/73193_2.png) [@ryan.white](https://forum.inductiveautomation.com/u/ryan.white)\ **Post date:** [April 11, 2025, 9:22pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/9 "2025-04-11T21:22:19Z")
> [@D\_Mitri\_Tompkins](#): > > major throttling issues all coming from ignition requests How are they determining this? Can you get an explicit list of the API endpoints that are being throttled from your IT group? That might be able to help you narrow down what scripts are the cause. Otherwise you will need to crawl all your library, gateway, and client scripts to see which ones are touching APIs and determine if any are doing so on a tight loop/high rate. ---
**Author:** ![D\_Mitri\_Tompkins](https://avatars.discourse-cdn.com/v4/letter/d/a3d4f5/32.png) [@D\_Mitri\_Tompkins](https://forum.inductiveautomation.com/u/D_Mitri_Tompkins)\ **Post date:** [April 11, 2025, 9:23pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/10 "2025-04-11T21:23:33Z")
There are only 2 gateway events that are "scheduled" (run more than once a day) and they both only trigger hourly, so I don't see it being that either, there are project library scripts that allow "shortcutting" to access API of specific topics without having to fully key in, but those would also only run when called so... ---
**Author:** ![Spencer\_Lommel1](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/spencer_lommel1/32/78679_2.png) [@Spencer\_Lommel1](https://forum.inductiveautomation.com/u/Spencer_Lommel1)\ **Post date:** [April 11, 2025, 9:23pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/11 "2025-04-11T21:23:39Z")
if you solve some of the other issues that everyone else mentioned, it may also be useful to move some of the logic to a project library script or gateway event where timers are easier to implement. I've tested around with using `invokeLater` in a function to call itself with some delay to implement a timer. But I'm not sure if this is the best approach ---
**Author:** ![D\_Mitri\_Tompkins](https://avatars.discourse-cdn.com/v4/letter/d/a3d4f5/32.png) [@D\_Mitri\_Tompkins](https://forum.inductiveautomation.com/u/D_Mitri_Tompkins)\ **Post date:** [April 11, 2025, 9:25pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/12 "2025-04-11T21:25:07Z")
I'm not so patiently awaiting them to fulfill my request for such a list, and my apologies for being rather uninformed. I was a PLC programmer and HMI designer prior to this job, so everything I know about Python, network, scripting, etc. has been learned on the go. ---
**Author:** ![ryan.white](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/ryan.white/32/73193_2.png) [@ryan.white](https://forum.inductiveautomation.com/u/ryan.white)\ **Post date:** [April 11, 2025, 9:37pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/13 "2025-04-11T21:37:29Z")
> [@D\_Mitri\_Tompkins](#): > > company built and maintained API How much data on average are you pulling per request from this API(rough row count)? Is there a possibility that this API can't handle large amounts of data well so it has a very low threshold for rate limiting? > [@D\_Mitri\_Tompkins](#): > > project library scripts How many project instances do you have active at a time normally? These could add up if multiple people request something around the same time. (Would still have to be a lot and often) > [@D\_Mitri\_Tompkins](#): > > my apologies for being rather uninformed. Everybody starts somewhere. I'm in a similar boat, just with ~5 years more time in it than you. There's still so much I don't know about either. ---
**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\ **Post date:** [April 11, 2025, 9:45pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/14 "2025-04-11T21:45:19Z")
IT should be able to tell you exactly what endpoints you're hitting, where the requests are coming from (and if there are multiple Ignition Gateways, or just one), how many requests, etc... If they can't, well... IT should never be trusted in the first place. ---
**Author:** ![D\_Mitri\_Tompkins](https://avatars.discourse-cdn.com/v4/letter/d/a3d4f5/32.png) [@D\_Mitri\_Tompkins](https://forum.inductiveautomation.com/u/D_Mitri_Tompkins)\ **Post date:** [April 11, 2025, 9:48pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/15 "2025-04-11T21:48:13Z")
> [@ryan.white](#): > > How much data on average are you pulling per request from this API(rough row count)? Is there a possibility that this API can't handle large amounts of data well so it has a very low threshold for rate limiting? The amount of data differs wildly, some calls pull only 5 rows for 3 columns, some pull 450 rows for 10 columns, I'd say a safe average is about 30 rows for 5 columns. It definitely has a bit of a lower to my understanding it was mostly built to "push" data as opposed to ignition "pulling?" on it, > [@ryan.white](#): > > How many project instances do you have active at a time normally? These could add up if multiple people request something around the same time. (Would still have to be a lot and often) Pardon my arrogance again, but what do you mean when you say project instances? Number of scripts in the library, or number of dashboards or reports? ---
**Author:** ![ryan.white](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/ryan.white/32/73193_2.png) [@ryan.white](https://forum.inductiveautomation.com/u/ryan.white)\ **Post date:** [April 11, 2025, 10:45pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/16 "2025-04-11T22:45:49Z")
> [@D\_Mitri\_Tompkins](#): > > project instances Number of Perspective Sessions and/or Vision clients active at once with access to the api. > **[Perspective Sessions | Ignition User Manual](https://www.docs.inductiveautomation.com/docs/8.1/ignition-modules/perspective/perspective-sessions)** > > A Perspective Session represents one instance of a project running on a web browser or mobile app. The term "session" might be new to some users, where "client" or "runtime" may sound more familiar. So anytime we refer to a session, we are talking... > **[Connections - Vision Clients | Ignition User Manual](https://www.docs.inductiveautomation.com/docs/8.1/platform/gateway/status/connections/connections-vision-clients)** > > Vision Clients ---
**Author:** ![Chris\_Bingham](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@Chris\_Bingham](https://forum.inductiveautomation.com/u/Chris_Bingham)\ **Post date:** [April 13, 2025, 5:39pm UTC](https://forum.inductiveautomation.com/t/struggling-to-prevent-ignition-from-bogging-down-external-apis-that-it-is-pulling-data-from/105021/17 "2025-04-13T17:39:16Z")
> [@D\_Mitri\_Tompkins](#): > > there are project library scripts that allow "shortcutting" to access API My approach would be: 1. Audit your projects, identify everywhere that might be making an API call. Search your projects for "httpGet" (or perhaps "system.net.http"). 2. If multiple scripts (\>1) are making the same API request, move all of them to reference a single function from a common location (Project Library, under the Gateway Scripting Project, for example). 3. Add logging ([system.util.getLogger](https://www.docs.inductiveautomation.com/docs/8.1/appendix/scripting-functions/system-util/system-util-getLogger)) within your script, so that you can temporarily change the logging level and see how many times you are (or, Ignition is) calling the function & API, perhaps also any other useful information. 4. This: > [@Kevin.Herron](#): > > For data you need periodically refreshed consider a gateway timer or scheduled script that calls it periodically and puts the results into a document tag or something. 1. Keep being a squeaky wheel with IT, especially after confidence in findings above. Have them watch as you issue a single request, see if an improperly configured router (or load balancer) is creating issues.