SSD Writes with Perspective and Chrome

Okay, I have a weird question for you guys. We are using Ignition for the HMI for an industrial product, and have started running into an issue due to our architecture that is burning out the SSD. Now the majority of the issue stems from stuff outside ignition, mainly due to the DB setup. However, as part of our investigations, we are examining SSD writes for all subsystems, which leads me to this:

Running iotop -oPa on our linux server shows a high amount of data writing to the SSD via chrome “network.mojom.NetworkService” and running inotify shows a high frequency of writes through chrome Cookies-Journal. This cookie journal appears to write every minute.

Messing with it, I can say that its only happening when perspective is open on the server, so the question is what process is causing this?

Why have you a session open on the server at all? A good server is, IMO, A) Linux and B) Headless.

3 Likes

Probably running Edge Panel.

1 Like

Perspective Edge doesn’t have the same limitation as Vision Edge, i.e. both instances can be run remotely from the Gateway with Perspetive Edge.

Chrome is updating last access time for cookies in the sqlite DB it uses to store cookie info. Perspective is polling the Gateway for a variety of reasons at a variety of rates, and these polls will cause the last access time to change.

Nothing wrong here.

True, and I would not set it up it like this if I had my way but I do run into this setup regularly. Not enough space for 2 separate cpus, trying to save money, or they just think it has to be done this way because all other hmi’s they used had to be.

Would running perspective workstation instead of a browser change anything? Or does it just still run chrome internally and do the same disk caching?

Perspective Workstation is just a wrapper around JXBrowser / Chromium. I would expect the same.

Consider configuring Chrome to throw away all cookies when closed. That might stop/slow the disk writes.

Thank you guys for the info. It isn’t edge, but I think Kevin has the right idea on the source. I ended up setting it up so that Chrome stores its user data in RAM and that seemed to solve the issue.

2 Likes

Mind posting the steps to change that? We use a lot of small industrial PCs with SSDs running Edge as the onsite HMI and now I'm curious to do some tests to see if this is something I should look at.

1 Like

Yes!

For a bit more context, this is for a product that was dealing with lots and lots of DB writes, which were definitely the main issue, but while hunting we stumbled on this. Its up to you/the much more experienced developers here to decide if its worth it for longevity.

ALSO BIG NOTE: We haven’t really tested this in full production or under any kind of stress yet. Only using iotop and inotify and pidstat to see the differences. Please do at your own risk lol

Okay, what I did was very very simple. And being completely honest, I used chatgpt to guide me through this as I am an Ignition guy, not a chrome dev. But in short, you can set chrome to open a browser with a dedicated user directory. Link to Docs

The goal was to get rid of this top line “Chrome network.mojom.NetworkService”. As far as I could tell (and chatgpt said), this is the source of the CookieJournal, which seemed to be the source of the high amounts of writes (about 1/min, and at least 500kb I think).

This can be done via command line, which is what I did for my testing, but in theory you can also embed it into a shortcut. I am working on getting that part working. Also this was all done on Linux.

Command (with some spots for you to insert your stuff)

sudo -u YOUR_USER bash -lc '
export DISPLAY=:0
export XAUTHORITY=/home/*YOUR_USER*/.Xauthority
/opt/google/chrome/google-chrome \
  --user-data-dir=/dev/shm/*RAM_FOLDER_NAME* \
  --no-first-run \
  --profile-directory=Default \
  http://localhost:8088/data/perspective/client/*PROJECT_NAME* \
  --start-fullscreen &
'

Now this could have been our system, as we aren’t the best at these kinds of optimizing settings. So, If you want to test like I did, you can run this command to watch SSD writing. Keep in mind, this only happens if the perspective client is on the gateway computer. This command will watch and add on for as long as it runs. Link to Guide

sudo iotop -oPa

I also was looking for frequency, so I ran a few versions of this, but here was my favorite, as it logged 30 minutes to get a good scale:

sudo -u *YOUR_USER* bash -lc '
timeout 1800s inotifywait -mr \
  -e close_write,modify,create,delete,move \
  --timefmt "%F %T" \
  --format "%T %e %w%f" \
  ~/.config/google-chrome/Default \
  > /tmp/chrome_inotify_30min.log
'

Finally, I tested with pidstat as well: Link to Guide

sudo pidstat -d 60 5
1 Like

Just in case you aren’t currently aware, some advice before you deploy this permanently to the field; it is best practice to have the DB on a separate VM or separate bare metal server, not run concurrently on the same host as the Ignition Gateway.