Hi as per the title I have a python script which creates a csv from a number of perspective properties but I have run into an issue where the filesize of the csv is too big for the system.perspective.download function which gives me:
Websocket connection closed unexpectedly. code=1009, reason=Text message too large: (actual) 2,103,744 > (configured max text message size) 2,097,152, codeMeaning=Message Too Big, codeDescription=The endpoint is terminating the connection because a data frame was received that is too large.
Is there any work around for this? I've already split up the CSV as much as I can and I'm still way over the limit in the worst case scenario (in the above case I'm marginally).
We get this message in our logs pretty frequently. I haven't tried to find out what component / script causing it yet as we haven't heard any complaints knock on wood. Hopefully the linked reply resolves it for you.
Hmm. system.perspective.download doesn't send the actual file contents over the websocket. It temporarily hosts the file at a unique ID and sends the URL to fetch it down to the browser to download. It's odd it's correlated with these messages.
If you look in browser devtools when you first open the session, you can see the initial websocket message and open it to see all the traffic back and forth. I'd be interested (from the browser's side) what's actually being sent when you get these messages on the gateway.
Are you trying to also apply the csv content to a Perspective session or component property? That would cause it to be transferred in the websocket, too. (Don't do that.)
Thanks for the replies everyone. Ideally I don't want to mess with the JVM parameters as I don't want any roll on effects which might cause gateway performance to be affected.
This is very helpful information @PGriffith and @pturmel I had a bit more of a look inside. I am transferring data in and out of perspective session properties as a temporary storage area to work on the data. I have some code that communicates with gateways, pulls the data across and then joins it together and stores it in a perspective session component for manipulation and export (or sending back to the gateways immediately if desired).
Is there a better place to store this type of information? I was using the perspective session properties because it is persistent across a single session (which is relevant for this as I don't want to use a gateway tag etc as I want the data unique to a particular session so users cannot affect each others data). The data all comes in asynchronously from the gateway network as the operations take quite a while for the gateways to perform which can lead to timeouts otherwise.
(screenshot below of the download inside of the perspective session, I'm using a much smaller dataset here to try and get the operation to succeed)
If you don't actually need all of those properties on the frontend (in components) then I believe right clicking them and marking them as private prevents them from being synced over the websocket. If that doesn't work, or you want more control, you could also store a dictionary in system.util.getGlobals - use the session ID as a key, and then you can store whatever values you want of whatever size you want, subject only to your gateway's memory limitations. Be careful to only store native Python types (not your own defined classes) to avoid a memory leak.
What are my options if I do need those in the front end?
I make the data available to the user to inspect the data and make tweaks before deployment as well as making large scale changes to the gateway device and tag configuration.
setting them as private does indeed help and allow the operation to work but it breaks a bunch of things on the front end (as expected)
Another option would be to manually render your large file into an iframe via the WebDev module. That would deliver content outside Perspective's websocket.
Keep in mind that sending very large payloads through the websocket can greatly harm interactivity for other components.
Yeah I am very much not inclined to make the websocket size excessively large, the gateways I have are already precarious as it is. @pturmel I was just patiently waiting for you to let me know how WebDev module would solve all my problems!
