Hi,
Where I work, we're using PuppetDB 2.3.8 and Puppet Servers 1.2.0, and are facing an issue that occurs during PuppetDB outages. Note that PuppetDB outages are not at all frequent for us, but it happened twice (not because of PuppetDB itself but for other reasons) and our entire fleet of servers was completely unable to run Puppet.
When PuppetDB is down and a Puppet Agent on a random server runs Puppet, the Puppet Servers hang right after outputting the following in the logs:
> 2017-02-22 16:27:04,504 INFO> [puppet-server] Puppet Caching node> for agent_hostname
Consequentially, the Puppet agents also hang waiting for the Puppet servers. After restoring PuppetDB, I'm forced to restart the Puppet Servers, after which they eventually output the following to the log:
> 2017-02-22 16:32:16,144 ERROR> [puppet-server] Puppet Failed to> submit 'replace facts' command for> agent-hostname to PuppetDB at> puppetdb-hostname: Error while waiting> for http response
Based on my interpretation of the Puppet documentation, I expected the Puppet Servers to at least hard fail due to lack of response from PuppetDB, but not block eternally. It's as if the Puppet Servers don't timeout trying to reach PuppetDB (setting server_url_timeout to a lower value than the default made no difference either). I also attempted to set `storeconfigs` to false (since we have that set to true) hoping it would make any difference as the documentation states soft fail works if set to true and if `storeconfigs` is set to false. Unfortunately, it made no difference.
What are we missing? There's certainly a misconfiguration on our end. That or we ran into a bug of some sort. I can't seem to find anybody else complaining about this. Any ideas?
Any help is appreciated.
Thank you.
Best Regards,
Pedro
↧