Good afternoon,
Having an issue where our pe-console-services fail upon start, and hoping someone can help lead me in the right direction. All other puppet services are running. We are running Puppet Enterprise 2015.2 on CentOS 7
The console-services log gives the following info:
>ERROR [p.t.internal] Error during service init!!!
>java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:postgresql://puppet-corp-02.domain.com:5432/pe-rbac?ssl=true&sslfactory=org.postgresql.ssl.jdbc4.LibPQFactory&sslmode=verify-full&sslrootcert=/etc/puppetlabs/puppet/ssl/certs/ca.pem&stringtype=unspecified&characterEncoding=UTF-8, username = pe-rbac. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
...followed by:
>Caused by: java.net.ConnectException: Connection refused
Am also seeing various SSL related errors:
mcollective.log:>ERROR -- : activemq.rb:149:in `on_ssl_connectfail' SSL session creation with stomp+ssl://mcollective@puppet-corp-02.domain.com:61613 failed: Connection refused - connect(2) for "puppet-corp-02.domain.com" port 61613
puppetdb.log:>2017-08-10 00:00:55,817 ERROR [c.j.b.h.AbstractConnectionHook] Failed to acquire connection Sleeping for 7000ms and trying again. Attempts left: 1. Exception: null>2017-08-10 00:01:02,817 ERROR [c.j.b.PoolWatchThread] Error in trying to obtain a connection. Retrying in 7000ms>org.postgresql.util.PSQLException: The server does not support SSL.
I know that this is a lot of scattered information, but I hope that it can at least start a conversation to lead to a solution. Currently no agents can talk to the puppet server. This all started happening Tuesday night after a reboot. However, we restored a snapshot of the server from July 5 and are seeing the same issue, so it's apparently been a problem for awhile, but didn't show up until the services had to be started again.
Any suggestions are welcome at this point, as we feel like we've exhausted our resources on our end with various troubleshooting.
Thanks, Mike
The console-services log gives the following info:
>ERROR [p.t.internal] Error during service init!!!
>java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:postgresql://puppet-corp-02.domain.com:5432/pe-rbac?ssl=true&sslfactory=org.postgresql.ssl.jdbc4.LibPQFactory&sslmode=verify-full&sslrootcert=/etc/puppetlabs/puppet/ssl/certs/ca.pem&stringtype=unspecified&characterEncoding=UTF-8, username = pe-rbac. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
...followed by:
>Caused by: java.net.ConnectException: Connection refused
Am also seeing various SSL related errors:
mcollective.log:>ERROR -- : activemq.rb:149:in `on_ssl_connectfail' SSL session creation with stomp+ssl://mcollective@puppet-corp-02.domain.com:61613 failed: Connection refused - connect(2) for "puppet-corp-02.domain.com" port 61613
puppetdb.log:>2017-08-10 00:00:55,817 ERROR [c.j.b.h.AbstractConnectionHook] Failed to acquire connection Sleeping for 7000ms and trying again. Attempts left: 1. Exception: null>2017-08-10 00:01:02,817 ERROR [c.j.b.PoolWatchThread] Error in trying to obtain a connection. Retrying in 7000ms>org.postgresql.util.PSQLException: The server does not support SSL.
I know that this is a lot of scattered information, but I hope that it can at least start a conversation to lead to a solution. Currently no agents can talk to the puppet server. This all started happening Tuesday night after a reboot. However, we restored a snapshot of the server from July 5 and are seeing the same issue, so it's apparently been a problem for awhile, but didn't show up until the services had to be started again.
Any suggestions are welcome at this point, as we feel like we've exhausted our resources on our end with various troubleshooting.
Thanks, Mike