# Kubernetes PersistentVolume - Backup /data

**URL:** <https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306>\
**Category:** Ignition\
**Created:** [December 8, 2020, 6:33pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306 "2020-12-08T18:33:10Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![hodgins.e](https://avatars.discourse-cdn.com/v4/letter/h/977dab/32.png) [@hodgins.e](https://forum.inductiveautomation.com/u/hodgins.e)\
**Post date:** [December 8, 2020, 6:33pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/1 "2020-12-08T18:33:10Z")

</div>

I was trying to see how Ignition is in the Kubernetes world. I’m simply just trying to backup the path `/usr/local/bin/ignition/data`. But I can not get it to work. If I remove the mount path the container will at least run. If I leave it as is the pod runs but Ignition never starts up. Wondering if there’s a command that needs to be run or something?

Here’s my deployment:

```auto
apiVersion: apps/v1

kind: Deployment

metadata:

    name: ignition-depl

spec:

    replicas: 1

    selector:

        matchLabels:

            component: ignition

    template:

        metadata:

            labels:

                component: ignition

        spec:

            volumes:

                - name: ignition-storage

                  persistentVolumeClaim:

                    claimName: ignition-pv-claim

            containers:

                - name: ignition

                  image: inductiveautomation/ignition:nightly 

                  ports:  

                    - containerPort: 8088

                  volumeMounts:

                    - name: ignition-storage                      

                      mountPath: /usr/local/bin/ignition/data

```

And the Persistent volume claim

```auto
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
    name: ignition-pv-claim
spec:
    accessModes:
        - ReadWriteOnce
    resources:
        requests:
            storage: 2Gi

```

LOGS

```auto
Creating init.properties file
Creating gateway.xml
cp: cannot stat 'data/gateway.xml_clean': No such file or directory
Adding gateway.publicAddress.autoDetect=false to gateway.xml
failed to load external entity "data/gateway.xml"
Adding gateway.publicAddress.address=0.0.0.0 to gateway.xml
failed to load external entity "data/gateway.xml"
Adding gateway.publicAddress.httpPort=8088 to gateway.xml
failed to load external entity "data/gateway.xml"
Adding gateway.publicAddress.httpsPort=8043 to gateway.xml
failed to load external entity "data/gateway.xml"
Enabling arg pass through
Disabling PID file check
Starting Ignition gateway
Following wrapper.log

```

On container (nothing in the data folder really)

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/b/6/b643b31da76859b748b88ef8dc12449e10a9cf12.png)

---

<div class="post-metadata">

**Author:** ![kcollins](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kcollins/32/19646_2.png) [@kcollins](https://forum.inductiveautomation.com/u/kcollins)\
**Post date:** [December 8, 2020, 6:55pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/2 "2020-12-08T18:55:37Z")

</div>

Right now I think this will likely not work [on the official image at this time], for some of the reasons I describe in my unofficial image docs at [ALTERNATIVE: How to persist Gateway data](https://github.com/thirdgen88/ignition-docker/tree/master/docs#alternative-how-to-persist-gateway-data). You might have a go at subbing for `kcollins/ignition:8.1` and pointing your volume target at `/data`. Let me know if that works out for ya…

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [December 8, 2020, 7:37pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/3 "2020-12-08T19:37:42Z")

</div>

Nice badge on your avatar, @kcollins !

---

<div class="post-metadata">

**Author:** ![hodgins.e](https://avatars.discourse-cdn.com/v4/letter/h/977dab/32.png) [@hodgins.e](https://forum.inductiveautomation.com/u/hodgins.e)\
**Post date:** [December 8, 2020, 9:09pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/4 "2020-12-08T21:09:42Z")

</div>

well that’s awesome, that did work out for me. Had some issues initially but once I restarted my local environment it worked. Thanks @kcollins !

---

<div class="post-metadata">

**Author:** ![mazhar](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazhar/32/20008_2.png) [@mazhar](https://forum.inductiveautomation.com/u/mazhar)\
**Post date:** [December 9, 2020, 10:26pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/5 "2020-12-09T22:26:13Z")

</div>

@hodgins.e Do you mind sharing your k8 deployment def with the workaround. We are facing the same issue.

---

<div class="post-metadata">

**Author:** ![hodgins.e](https://avatars.discourse-cdn.com/v4/letter/h/977dab/32.png) [@hodgins.e](https://forum.inductiveautomation.com/u/hodgins.e)\
**Post date:** [December 10, 2020, 4:45pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/6 "2020-12-10T16:45:30Z")

</div>

hey @mazhar no problem. Just use my deployment that I included in the my question but change the containers config to this:

```auto
            containers:
                - name: ignition
                  image: kcollins/ignition:8.1
                  ports:  
                    - containerPort: 8088
                  volumeMounts:
                    - name: ignition-storage                      
                      mountPath: /data     

```

I am also using a nginx-ingress controller ([https://kubernetes.github.io/ingress-nginx/deploy/](https://kubernetes.github.io/ingress-nginx/deploy/))

my clusterIp service for Ignition is this:

```auto
apiVersion: v1
kind: Service
metadata:
    name: ignition-cluster-ip-service
spec:
    type: ClusterIP
    selector:
        component: ignition
    ports:
        - port: 8088
          targetPort: 8088

```

Hopefully that helps!

---

<div class="post-metadata">

**Author:** ![mazhar](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazhar/32/20008_2.png) [@mazhar](https://forum.inductiveautomation.com/u/mazhar)\
**Post date:** [December 13, 2020, 9:49pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/7 "2020-12-13T21:49:55Z")

</div>

Thanks for sharing. Official image worked for me too with this workaround.

---

<div class="post-metadata">

**Author:** ![Kevin.Collins](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.collins/32/13497_2.png) [@Kevin.Collins](https://forum.inductiveautomation.com/u/Kevin.Collins)\
**Post date:** [December 14, 2020, 3:08am UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/8 "2020-12-14T03:08:54Z")

</div>

@mazhar, do keep in mind that with the official image there is no special handling for mounts at `/data` and thusly, your gateway state will **not** be persisted.

---

<div class="post-metadata">

**Author:** ![mazhar](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazhar/32/20008_2.png) [@mazhar](https://forum.inductiveautomation.com/u/mazhar)\
**Post date:** [December 14, 2020, 12:22pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/9 "2020-12-14T12:22:47Z")

</div>

@Kevin.Collins Understood. ‘kcollins/ignition’ apparently needs initialization unlike official image, @hodgins.e can you share how you accomplished the initialization?

---

<div class="post-metadata">

**Author:** ![mazhar](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazhar/32/20008_2.png) [@mazhar](https://forum.inductiveautomation.com/u/mazhar)\
**Post date:** [December 14, 2020, 12:39pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/10 "2020-12-14T12:39:45Z")

</div>

> [@hodgins.e](#):
>
> kcollins/ignition:8.1

never mind, "kcollins/ignition:8.1" worked too, thanks for the help

---

<div class="post-metadata">

**Author:** ![kcollins](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kcollins/32/19646_2.png) [@kcollins](https://forum.inductiveautomation.com/u/kcollins)\
**Post date:** [June 1, 2022, 7:36pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/11 "2022-06-01T19:36:17Z")

</div>

Just augmenting this thread for those that find it (since I had a couple questions on this topic, recently)… In K8s, you can leverage [initContainers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) to seed the data volume prior to initial launch of the Gateway container [in the pod].

Here is an example deployment that leverages an _init container_ to seed the data volume with files from the image:

```auto
# Ignition K8s Example Deployment
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ignition-deployment
spec:
  selector:
    matchLabels:
      app: ignition-app
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: ignition-app
    spec:
      initContainers:
      - name: seed-volume
        image: inductiveautomation/ignition:8.1.17
        resources:
          limits:
            memory: "256Mi"
            cpu: "1000m"
        command:
        - sh
        - -c
        - >
          if [! -f /data/.ignition-seed-complete]; then
            touch /data/.ignition-seed-complete ;
            cp -dpR /usr/local/bin/ignition/data/* /data/ ;
          fi
        volumeMounts:
        - mountPath: /data
          name: ignition-data
      containers:
      - name: ignition
        image: inductiveautomation/ignition:8.1.17
        resources:
          limits:
            memory: "2048Mi"
            cpu: "1000m"
        args:
        - -n
        - Ignition-k8s
        - -m
        - "2048"
        env:
        - name: ACCEPT_IGNITION_EULA
          value: "Y"
        - name: GATEWAY_ADMIN_PASSWORD_FILE
          value: /run/secrets/gateway-admin/password
        - name: IGNITION_EDITION
          value: standard
        ports:
        - name: ignition-web
          containerPort: 8088
        volumeMounts:
        - mountPath: /usr/local/bin/ignition/data
          name: ignition-data
        - mountPath: /run/secrets/gateway-admin
          name: gateway-admin
          readOnly: true
        readinessProbe:
          exec:
            command:
            - health-check.sh
            - -t
            - "3"
          initialDelaySeconds: 60
          periodSeconds: 10
          failureThreshold: 10
          timeoutSeconds: 3
      volumes:
      - name: ignition-data
        persistentVolumeClaim:
          claimName: ignition-pv-claim
      - name: gateway-admin
        secret:
          secretName: gateway-admin

---
# Secret (which you wouldn't normally just have hanging out here in your deployment yaml)
apiVersion: v1
kind: Secret
metadata:
  name: gateway-admin
type: Opaque
data:
  password: UEBzc3cwcmQ=
---
# Service
apiVersion: v1
kind: Service
metadata: 
  name: ignition-lb
spec:
  selector:
    app: ignition-app
  type: LoadBalancer
  ports:
    - name: http
      port: 8088
      targetPort: 8088
---
# PersistentVolume
apiVersion: v1
kind: PersistentVolume
metadata:
  name: ignition-pv-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 10Gi
  hostPath:
    path: "${PWD}/ignition-data"
  accessModes: # ideally we want ReadWriteOncePod, but still in alpha?
    - ReadWriteOnce
---
# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ignition-pv-claim
spec:
  storageClassName: manual
  resources:
    requests:
      storage: 3Gi
  accessModes:
    - ReadWriteOnce

```

This is designed to be run from a folder on your host system with a fresh (and empty, initially) `ignition-data` subfolder. I use `envsubst` to substitute the current working directory. So you could deploy this with, for example:

```auto
$ kubectl create ns k8s-testing
$ kubectl config set-context --current --namespace=k8s-testing
$ envsubst < ignition-app.yaml | kubectl apply -f -

```

Keep in mind that this is just an example deployment that you can test locally, not a production-ready configuration.

Here’s a little video to check out with this too, in the event that moving pictures\>pictures\>words. 😀  
[k8s-testing-2.mp4](https://inductiveautomation1-my.sharepoint.com/:v:/g/personal/kcollins_inductiveautomation_com/Ea0xFu7czgZCoCuG1BVg3c0B-zWpSf_4fQonzz6_T4mWeg)

---

<div class="post-metadata">

**Author:** ![Sanat\_Pattanaik](https://avatars.discourse-cdn.com/v4/letter/s/54ee81/32.png) [@Sanat\_Pattanaik](https://forum.inductiveautomation.com/u/Sanat_Pattanaik)\
**Post date:** [November 9, 2023, 5:10pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/12 "2023-11-09T17:10:19Z")

</div>

Hello,  
I tried to follow this approach. But /data folder gets created without write permission  
 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/4/b/4b64bcb9e0de5a459a89ca45d7ce2cf09045e04f.png)

So the touch and copy are failing.

---

<div class="post-metadata">

**Author:** ![kcollins](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kcollins/32/19646_2.png) [@kcollins](https://forum.inductiveautomation.com/u/kcollins)\
**Post date:** [November 9, 2023, 5:23pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/13 "2023-11-09T17:23:30Z")

</div>

I'll mention that at ICC 2023 this year, I gave a presentation on Ignition on K8s with some useful ( **and updated!** ) examples. See those here: [GitHub - thirdgen88/icc2023-ignition-k8s: ICC 2023 Session - Deployment Patterns for Ignition on Kubernetes](https://github.com/thirdgen88/icc2023-ignition-k8s)

> [@Sanat\_Pattanaik](#):
>
> I tried to follow this approach. But /data folder gets created without write permission

What kind of backing storage are you using for your PVC's? Perhaps share the output of the command below for your cluster:

```bash
kubectl get storageclass

```

---

<div class="post-metadata">

**Author:** ![Sanat\_Pattanaik](https://avatars.discourse-cdn.com/v4/letter/s/54ee81/32.png) [@Sanat\_Pattanaik](https://forum.inductiveautomation.com/u/Sanat_Pattanaik)\
**Post date:** [November 9, 2023, 5:34pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/14 "2023-11-09T17:34:40Z")

</div>

[kubernetes.io/aws-ebs](http://kubernetes.io/aws-ebs)

---

<div class="post-metadata">

**Author:** ![Sanat\_Pattanaik](https://avatars.discourse-cdn.com/v4/letter/s/54ee81/32.png) [@Sanat\_Pattanaik](https://forum.inductiveautomation.com/u/Sanat_Pattanaik)\
**Post date:** [November 9, 2023, 5:41pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/15 "2023-11-09T17:41:32Z")

</div>

![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/a/1/a1c5178d7c2f544d901136715e89505815f1589e.png)

---

<div class="post-metadata">

**Author:** ![Sanat\_Pattanaik](https://avatars.discourse-cdn.com/v4/letter/s/54ee81/32.png) [@Sanat\_Pattanaik](https://forum.inductiveautomation.com/u/Sanat_Pattanaik)\
**Post date:** [November 9, 2023, 5:50pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/16 "2023-11-09T17:50:05Z")

</div>

Thank you for your response. After looking thru your github, I added the security context and it is working now.

---

<div class="post-metadata">

**Author:** ![kcollins](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kcollins/32/19646_2.png) [@kcollins](https://forum.inductiveautomation.com/u/kcollins)\
**Post date:** [November 9, 2023, 5:53pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/17 "2023-11-09T17:53:20Z")

</div>

Okay.. You shouldn't have any issue with EBS volumes. Feel free to share your YAML, but ultimately, I'd recommend looking at that GH repo I referenced above as a better baseline.

EDIT: wrote the above before I saw your most recent reply.. Glad you're up and running! 👍

---

<div class="post-metadata">

**Author:** ![Pratik\_Kansagara](https://avatars.discourse-cdn.com/v4/letter/p/ba8739/32.png) [@Pratik\_Kansagara](https://forum.inductiveautomation.com/u/Pratik_Kansagara)\
**Post date:** [November 30, 2023, 6:19am UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/18 "2023-11-30T06:19:49Z")

</div>

> [@kcollins](#):
>
> DIT: wrote the above before I

Do you have any example to deploy it in AKS with azure disk as a presist volume.

---

<div class="post-metadata">

**Author:** ![kcollins](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kcollins/32/19646_2.png) [@kcollins](https://forum.inductiveautomation.com/u/kcollins)\
**Post date:** [December 4, 2023, 4:08pm UTC](https://forum.inductiveautomation.com/t/kubernetes-persistentvolume-backup-data/41306/19 "2023-12-04T16:08:09Z")

</div>

I don't have any AKS examples prepared, but you're on the right track targeting [Azure Disk](https://learn.microsoft.com/en-us/azure/aks/concepts-storage#azure-disk) as the storage class for your PVC.