Inspecting elements in a Perspective Page

Ignition
1

We are "screen scraping" web pages to fetch information that we want to store, without having to do the math behind the web page. For example, there's a web page that displays "planned qty" for a production line ... with values "54" and then a minute later "55", etc.

We have been using:

client = system.net.httpClient()
response = client.get(url)
listOfLines = response.splitlines()

Some web pages have very straightforward source code and we can then find the element data we are looking for (i.e. "54" (which is the PlannedQty value), is found on Line 22). We can refresh the "get", look at what's on line 22, and store the value when it changes. Works great.

However, this approach is NOT working for the web pages generated by Perspective.

In MicrosoftEdge, however, I can use the Developer tools, and hover over "54" and I can see the value in the /Elements pane.

What is the recommended approach to fetch the contents of the Perspective page in such a way that I can reliably find the element I'm looking for, so I can store the element value?

2

Buy the webdev module for the Ignition server hosting the page and use it to provide an endpoint that you can send requests to to fetch the necessary data. This approach will be much less fragile than trying to scrape data from a perspective page.

3 Likes
3

You basically cannot. Perspective DOM is constructed entirely in the browser with javascript.

(Why are you scraping, anyways? That is really error-prone and hard on the source website, too. If you don't own those sites, you are probably breaking their TOS.)

3 Likes
4

Thanks for the idea ... however ... it is NOT hosted by Ignition. I'll put more info in my reply to Phil's post.

5

Thanks for the concerns and advice.

These web-pages were written by our own developers (different team) but they didn't put in an API of any kind to fetch the data. We are therefore screen-scraping because the data and math behind the displayed values is difficult for us to get access to.

We understand that it is fragile. On our next iterations, we expect to fix this properly.