Automate IT and infrastructure, manage complex workflows, and mitigate risk at scale.
Try the full-featured Puppet Enterprise for free on 10 nodes.
Find and prevent compliance failures
Continuous Delivery for Puppet Enterprise
Build, test, and deploy infrastructure as code faster and easier
Compliance Enforcement Modules
Remediate to stay in compliance
Content & Modules
Pre-built scripts to automate common tasks
Get Puppet Enterprise
First 10 nodes are free!
Try it now
Request a demo
Find thousands of component modules built by the community and guidance on using them in your own infrastructure.
Visit Puppet Forge >>
Open Source Puppet
Perfect for individuals and small infrastructure
Automate tasks in orchestration workflows
See all open source projects >>
Contribute to open source projects >>
Over the past few months, we set out to drastically reduce the amount of time Code Manager takes to deploy code and sand down some rough edges to make it more stable and robust. Learn more about how you can make Puppet Code Manager work for you.
Table of Contents:
Puppet Code Manager allows users to deploy and sync modules to ensure that each version is correct. Puppet Code Manager also purges the environment to make sure that there aren't any extra things that shouldn't be there.
In order to understand what we were able to achieve, we need a quick primer on how code is deployed to a Puppet Server in the first place.
There are three parts to a Code Manager code deployment:
We’ve improved each of these three parts of the Code Manager code deployment.
We added the ability to set the number of workers that can deploy modules and changed the default from 1 to 4. This shipped in 2019.8.3 and we saw a 40% reduction in the amount of time it takes to deploy 83 modules for the first time (110 seconds down to 68). The more latency between your Puppet Server and where modules are retrieved from (Puppet Forge, Github, etc.). the more benefit you will see from this change. During testing, we didn’t see a change in performance when using more than four deploy workers; however, you can configure download_pool_size to a higher number and see for yourself if it makes things faster.
Currently, Code Manager will parse the Puppetfile and check each module’s desired version against what is actually on disk to determine what to deploy. This implementation isn’t generally useful because we do not expect users to modify module files on disk between code deploys. When making changes to modules, you’ll do so in your Git repository hosting the Puppet module then deploy the new version of the module via Code Manager. Any changes made directly on a Puppet Server will be reverted during a code deployment.
With new improvements coming in PE 2019.8.9 and 2021.4, we check the currently deployed Puppetfile and compare it to the Puppetfile we are about to deploy. If the version of a module has not changed, we no longer check the state of the module on disk and assume it has not been altered. For floating versions, like “latest” or a Git branch, we continue to compare with what’s on disk because the source content could have changed.
When testing the performance of this change, we used a very large Puppetfile with 1200 module entries in it. Without changing the Puppetfile at all, a code manager deployment would spend three minutes and 45 seconds checking all of the modules on disk before the change. Now it takes three seconds because we no longer check what’s on disk; we simply compare the current Puppetfile to the new Puppetfile and see there are no changes.
Code Manager does an extra level of purging that r10k does not do by default. We compare the environment on disk with the environment in the control repo to ensure it doesn’t have any extra content.
Previously, we included modules in this comparison, despite already making the same comparison before syncing modules. The more modules one deployed, the longer the purge took because we were creating a list of all of the files in those modules even though we did not need to look at them.
After this change, we no longer descend into the modules directory when looking for files to purge, resulting in large performance benefits. When testing just this purge functionality in isolation, we see a reduction from three minutes to three seconds with a Puppetfile containing 1200 modules. You can get this improvement in PE 2019.8.8.
Here’s a list of other Code Manager improvements that are smaller in scope, but will greatly help if you were affected by them:
As you work through the parameters of Puppet Code Manager, we have a handy resource for configuration within our docs. This can help you work through next-steps and revisit the most current Code Manager updates.
Contact Puppet Experts for Support
Technical Product Marketing Director, Puppet by Perforce