Categories
Mastering Development System & Network

Postgres mount volume error in k8s

I’m trying to deploy Postgres with a PersistentVolumeClaim in GKE. As per the description here (look at PGDATA), it is recommended to create a subdirectory if mounting an external volume. So, this is what I did with my PG deployment:

spec:
  containers:
    - name: postgres-dev
      image: "postgres:12.1"
      imagePullPolicy: IfNotPresent
      ports:
        - containerPort: 5432
      env:
        - name: PGDATA
          value: "/var/lib/postgresql/data/pgdata"
      volumeMounts:
        - name: postgres-persistent-storage-dev
          mountPath: "/var/lib/postgresql/data/pgdata"
  volumes:
    - name: postgres-persistent-storage-dev
      persistentVolumeClaim:
        claimName: postgres-pvc-dev

I create the /pgdata subdir in /postgresql/data, which would be the original mount point for Postgres’ data.

But when I try to deploy the Postgres pod, it stalls at Back-off error, and I get the following events:

Events:
  Type     Reason                  Age               From                                                  Message
  ----     ------                  ----              ----                                                  -------
  Normal   Scheduled               20s               default-scheduler                                     Successfully assigned default/postgres-dev-7f549754d-nm2lf to gke-cluster-pool-1-4ce1e4d6-m33c
  Normal   SuccessfulAttachVolume  15s               attachdetach-controller                               AttachVolume.Attach succeeded for volume "pvc-eea8ee4d-9d00-11ea-a854-42010a8a0057"
  Normal   Pulled                  9s (x2 over 10s)  kubelet, gke-cluster-pool-1-4ce1e4d6-m33c             Container image "postgres:12.1" already present on machine
  Normal   Created                 9s (x2 over 10s)  kubelet, gke-cluster-pool-1-4ce1e4d6-m33c             Created container postgres-dev
  Normal   Started                 9s (x2 over 10s)  kubelet, gke-cluster-pool-1-4ce1e4d6-m33c             Started container postgres-dev
  Warning  BackOff                 7s (x2 over 8s)   kubelet, gke-cluster-pool-1-4ce1e4d6-m33c             Back-off restarting failed container

These are the logs from the Postgres container:

initdb: error: directory "/var/lib/postgresql/data/pgdata" exists but is not empty
It contains a lost+found directory, perhaps due to it being a mount point.
Using a mount point directly as the data directory is not recommended.
Create a subdirectory under the mount point.

So, I create the subdir for mountpoint, as recommented for initdb, and then it complains about it. The PVC was created just right before the deployment, so there is no data in it.

I tried with completely new, un-existent dir like /var/lib/postgres/data as well, but the effect is the same.

Another interesting point is that when I set this up in minikube, everything works fine.

How do I approach this?

Leave a Reply

Your email address will not be published. Required fields are marked *