OPC UA Faulted "User limit of inotify intances reached or too many open files"

Ignition
,
1

I see the OPC UA Module shows this error "User limit of inotify intances reached or too many open files". This is running from a custom build docker image (3rd Party Module Pre-Installed). The only way to solve this is by deleting the container and the volume. I have noticed that after the first ish trial expires counts, this error appears.

8.1.48

image
image1639×593 58 KB

2

You'll probably have to tweak the Linux ulimit setting for open files. (I am not sufficiently versed in docker, but those search terms should point the way.)

3

I believe docker run has a ulimit param?

https://stackoverflow.com/questions/24318543/how-to-set-ulimit-file-descriptor-on-docker-container-the-image-tag-is-phusion seems to suggest so

probably has what you want

2 Likes
4

I will try. Shouldn't the official docker image be tweaked properly?
@kcollins1 might have some thoughts about this one.

5

Any advice on the limit numbers to use?

6

That I'm unsure of, sorry -- my best heuristic is going to be 'look at the defaults they suggest on the docker run page, then double them til something works'

7

The default ulimit is a property of the host system, and the argument Leor referenced above is a runtime argument. You can't bake that into the Docker image.

2 Likes
8

I see, the host is also Linux based. Good news if the problem comes from the shared host kernel. Can the problem be in the number of docker containers running on the kernel? I'm running 12 by now but this number might grow up to 30 or so. How the number of containers affect the kernel in a way that the OPC UA module on all edge shows as faulted?

image
image698×550 9.3 KB

9

If you're running a ton of Ignition Gateways on a single host, you'll likely have to tweak some kernel settings. Try setting fs.inotify.max_user_instances=1024, e.g.:

# Check existing setting (I've seen defaults of 128)
sysctl fs.inotify.max_user_instances

# Change the setting to 1024 live (does not persist)
sysctl fs.inotify.max_user_instances=1024

# Run the following as `root` to persist the setting
cat << EOF > /etc/sysctl.d/98-custom.conf
fs.inotify.max_user_instances=1024
EOF

The above is what we do on some hosts where we run a lot of Ignition containers concurrently. It is also what is automatically set if you install something like LXD/Incus (it presumably changes these defaults for similar reasons).

If you're provisioning hosts with Ansible, drop this task in:

# ...
  - name: Adjust inotify max_user_instances from default to 1024
    ansible.posix.sysctl:
      name: fs.inotify.max_user_instances
      value: '1024'
      state: present
      sysctl_file: /etc/sysctl.d/98-custom.conf
# ...
2 Likes