Is it possible for gateway settings to be stored on an external database?

Ignition
1

I come from a Full Stack developer background and was thinking how for frontend applications the state and settings are retrieved from the database on the backend.

I recall that ignition 8.1 stored gateway settings in .idb files, which are just sqlite databases, and was wondering is it possible to move that sqlite database to be outside the ignition container? That way ignition gateways could be stateless and when booting they can get their settings from the database server.

This could make it easier for changes to be stored without the need for making gateway backups. It also would make deployments with ignition containers a lot more flexible because they can get the information they need from a single source of truth.

2

We moved away from storing config in SQLite in 8.3…

You could mount that idb or config directory elsewhere, but external databases as in other databases that aren’t SQLite - probably not.

3

I understand that its being moved away from, are you saying its impossible to do that now? Or is there support in the ignition sdk to implement a solution like this?

4

Hi Derek,

8.3 configuration moved to the filesystem so you can point a volume mount to the configuration folder.

When I used VCS for 8.1, each gateway had a gateway backup (.gwbk) although I had to rip out all of the projects, themes, and tags so when we restarted the gateway, it didn't overwrite any changes.

If you are curious:

5

Shouldn't need the SDK to do what you are proposing.

6

I know you can do a bind mount, but that doesn't work for the deployment I am trying to do. Bind mounts require the container is on the same machine or VM as the container. This doesn't work for cluster deployments. We are using HashiCorp nomad for our container orchestrator. The ignition gateways are built into docker images based on the files in the git repo. Git is used to track the changes for each gateway and act as a source of truth. At the same time it makes it difficult to get the changes from the container back to the VM do to the structure of the git repository. If the gateway settings were stored in a central database it would add a lot more flexibility to deploying updated images because they could get their settings from the database and could be deployed anywhere in the cluster.

7

Thanks for the response, I don't have specifics so feel free to include more information.

Bind mounts require the container is on the same machine or VM as the container

Could you not use a network backed persistent volume instead? If a node fails, you can reattach the volume to the replacement node. (I don't know how this behaves with licensing)

The ignition gateways are built into docker images based on the files in the git repo

Can you clarify what you mean by this? Are you referring to the versionable resources like scripts, views, etc? Or the entirety of the gateway configuration is built into the image?

Git is used to track the changes for each gateway and act as a source of truth. At the same time it makes it difficult to get the changes from the container back to the VM due to the structure of the git repository.

Correct me if I'm wrong, if you're trying to synchronize production runtime changes from the container back into Git, then the running gateway is the source of truth for the configuration.

GitOps
dev -> commit & push -> git -> deploy -> prod

Runtime Changes
prod -> config change -> commit & push -> git

With 8.3, deployment modes and overrides make it more feasible for Git to actually be the source of truth.

If the gateway settings were stored in a central database it would add a lot more flexibility to deploying updated images.

Even if this was possible, I would be wary of this approach because it introduces a SPOF. Your source of truth would still be split between said database and the git repo and would break on the 8.3 upgrade.

Have you considered upgrading to 8.3? It's getting quite stable with the recent bug fixes.

Hope this helps!

8

The canonical way to do this (as of 8.3) is to deliver container configuration that should be immutable via the special external configuration (which the gateway knows not to touch).

We have no interest in going back to a single canonical 'database' model for gateway configuration, and honestly you're the first person I've ever heard ask for it.