Watcher db container fails to start #451

New Issue

dboreham · 2023-07-17T17:33:47Z

dboreham commented

2023-07-17 17:33:47 +00:00

Testing the mobymask-v2 stack in a deployment, the mobymask-watcher-db container fails to start.
Looking at its log, it actually does seem to start but logs these errors which probably makes the healthcheck think it failed:

server started
CREATE DATABASE


/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/multiple-postgressql-databases.sh
/usr/local/bin/docker-entrypoint.sh: line 170: /docker-entrypoint-initdb.d/multiple-postgressql-databases.sh: Is a directory

PostgreSQL Database directory appears to contain a database; Skipping initialization

2023-07-17 13:10:40.935 UTC [1] LOG:  starting PostgreSQL 14.8 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r10) 12.2.1 20220924, 64-bit
2023-07-17 13:10:40.935 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-07-17 13:10:40.935 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2023-07-17 13:10:40.944 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-07-17 13:10:40.954 UTC [22] LOG:  database system was interrupted; last known up at 2023-07-17 13:10:39 UTC
2023-07-17 13:10:41.596 UTC [22] LOG:  database system was not properly shut down; automatic recovery in progress
2023-07-17 13:10:41.601 UTC [22] LOG:  redo starts at 0/16FE878
2023-07-17 13:10:41.601 UTC [22] LOG:  invalid record length at 0/16FE970: wanted 24, got 0
2023-07-17 13:10:41.601 UTC [22] LOG:  redo done at 0/16FE928 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
2023-07-17 13:10:41.638 UTC [1] LOG:  database system is ready to accept connections

Testing the mobymask-v2 stack in a deployment, the mobymask-watcher-db container fails to start. Looking at its log, it actually does seem to start but logs these errors which probably makes the healthcheck think it failed: ``` server started CREATE DATABASE /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/multiple-postgressql-databases.sh /usr/local/bin/docker-entrypoint.sh: line 170: /docker-entrypoint-initdb.d/multiple-postgressql-databases.sh: Is a directory PostgreSQL Database directory appears to contain a database; Skipping initialization 2023-07-17 13:10:40.935 UTC [1] LOG: starting PostgreSQL 14.8 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r10) 12.2.1 20220924, 64-bit 2023-07-17 13:10:40.935 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 2023-07-17 13:10:40.935 UTC [1] LOG: listening on IPv6 address "::", port 5432 2023-07-17 13:10:40.944 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 2023-07-17 13:10:40.954 UTC [22] LOG: database system was interrupted; last known up at 2023-07-17 13:10:39 UTC 2023-07-17 13:10:41.596 UTC [22] LOG: database system was not properly shut down; automatic recovery in progress 2023-07-17 13:10:41.601 UTC [22] LOG: redo starts at 0/16FE878 2023-07-17 13:10:41.601 UTC [22] LOG: invalid record length at 0/16FE970: wanted 24, got 0 2023-07-17 13:10:41.601 UTC [22] LOG: redo done at 0/16FE928 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s 2023-07-17 13:10:41.638 UTC [1] LOG: database system is ready to accept connections ```

dboreham commented

2023-07-17 19:00:21 +00:00

The script gets mounted with this in the compose file:

    volumes:
      - ../config/postgresql/multiple-postgressql-databases.sh:/docker-entrypoint-initdb.d/multiple-postgressql-databases.sh

and the file appears to be present:

./deployment-001/config/postgresql/multiple-postgressql-databases.sh

But actually it is not. This is a directory created by docker when it tried to mount a nonexistent file:

$ ls -l ../config/postgresql/
total 4
drwxr-xr-x 2 root root 4096 Jul 17 07:10 multiple-postgressql-databases.sh

The script gets mounted with this in the compose file: ``` volumes: - ../config/postgresql/multiple-postgressql-databases.sh:/docker-entrypoint-initdb.d/multiple-postgressql-databases.sh ``` and the file _appears_ to be present: ``` ./deployment-001/config/postgresql/multiple-postgressql-databases.sh ``` But actually it is not. This is a directory created by docker when it tried to mount a nonexistent file: ``` $ ls -l ../config/postgresql/ total 4 drwxr-xr-x 2 root root 4096 Jul 17 07:10 multiple-postgressql-databases.sh ```

dboreham commented

2023-07-17 19:25:48 +00:00

Problem is that the current code assumes that config files are in a directory with the same name as the pod.
That is not the case here.

Problem is that the current code assumes that config files are in a directory with the same name as the pod. That is not the case here.

dboreham commented

2023-07-18 18:35:17 +00:00

Fixed

Sign in to join this conversation.

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: cerc-io/stack-orchestrator#451