How to "chunk" large datasets?

Ignition
,
1

Hi all,

I have a query returning 3000 plus rows and have the return format set to JSON, so I can use one embedded view (for styling purposes), displayed with the rest of the data.

I keep getting a Message too big message, and the Designer loses connection with the GW.

In another project, I have the exact same setup but it does not lose connection. If you ask why the same setup, I am rebuilding this project into the new one.

This is the message from the GW Log:

Websocket connection closed unexpectedly. code=1009, reason=Text message too large: (actual) 2,103,240 > (configured max text message size) 2,097,152, codeMeaning=Message Too Big, codeDescription=The endpoint is terminating the connection because a data frame was received that is too large.

I have seen Phil mention chunking datasets here and there, but not sure how to do that.
I could limit the query in SQL Server, then send a new query request based on user input.
Or, I could change the max message size, but I did not see anything relating to this in the conf file.

Advice is appreciated.

EDIT: I just realized that by limiting the size from the DB would also affect using the table filter. So, if there is a way to chunk the data maybe as it loads into the View or Table?

2

Is this in a binding that you are returning this?
I'm pretty sure theres a bunch of threads where massive datasets were sent from the gateway using message handlers and sidestep this completely.

3

Iā€™m going to assume a few things based on your message. Assume you have a binding on your view called view.custom.data that is holding your large dataset. You then have an embedded view that you are passing the data into, and within that embedded view, you assumably have a table.

If the table is not virtualized, you are sending 3 copies of that data to the browser DOM.

  1. view.custom.data
  2. Embedded view prop
  3. Table.props.data

You could set view.custom.data to Access: Private, which would stop it from propagating to the DOM. You now have 2 copies of the data being sent.

You could virtualize the table and add pagination, which only sends the visible rows to the DOM. You now have 1.01 copies sent to the DOM (25 out of 3,000 rows)

Unfortunately, the only way to remove the last fully copy is to not send the data to an embedded view. You can add the table to your view, and then use the Advanced Stylesheet to build a style class to apply to your table for consistency.

1 Like