BreadcrumbHomeResourcesBlog How To Use Onceover For Repository Testing May 20, 2020 How to Use Onceover For Repository TestingEcosystems & IntegrationsHow to & Use CasesBy Henry WangIs there an easier way to do repository testing for your Puppet code? Yes! The answer is using Onceover.Table of Contents:What Is Onceover?Why Use Onceover For Repository Testing?How to Use Onceover Repository Testing For PuppetResults of Onceover Repository Testing What Is Onceover?Onceover is a free, open source tool that you can use to test your Puppet repository. Why Use Onceover For Repository Testing?Onceover makes it easy to do repository testing on your Puppet code.I am a Puppet beginner and I’m happy to make the code manager integration happen. I like to play around with my code and test my code changes on my agent nodes. However, the way I’m testing my code is a bit tedious: Save my code change, push into remote repo, run code deploy, and run Puppet on agents.Until I tried Onceover. How to Use Onceover Repository Testing For Puppet1. Prepare the Lab EnvironmentI am using a fresh installation of RHEL 7.Install Gitsudo yum install git -yInstall Ruby by RVMAs to the Ruby version we want to install, review the Ruby versions for PE releases. In our example here, we are going to install Ruby v2.5.7. We can use RVM to install wanted Ruby versions easily.##Install RVM sudo gpg2 --keyserver hkp://ipv4.pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB curl -sSL https://get.rvm.io | sudo bash -s stable sudo usermod -a -G rvm `whoami` ##Logout and Login to use RVM exit ##Install Ruby 2.5.7 rvm install ruby-2.5.7 ruby -vNow we are ready to install Onceover and start our testing!2. Install Onceover and Test the Puppet CodeClone down my control-repoYou can use this example repo for learning purposes, but you should ultimately try Onceover at your own control repo.git clone https://github.com/henrywangpuppet/onceover-lab-repo.git cd onceover-lab-repo/ git checkout onceoverlabOkay, now I need to install the onceover gem.gem install bundler bundler init echo "gem \"onceover\"" >> Gemfile bundle installNow it’s time to configure and test.Now Onceover has been installed. The README says I need to configure it, and I can do so by running bundle exec onceover init.bundle exec onceover initAs the command runs, I can see that it created a number of files for me.created spec created spec/onceover.yaml created spec/pre_conditions created spec/pre_conditions/README.md created spec/factsets created spec/factsets/README.md created Rakefile skipped Gemfile (exists)I’m curious what’s going to happen if I just try to run the spec tests. Back to the README, where I discover I can run bundle exec onceover run spec.bundle exec onceover run specStarting at the beginning of the output, I see that it’s deploying my modules from the Puppetfile.INFO -> Using Puppetfile '/root/onceover-lab-repo/Puppetfile' INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/concat INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/stdlib INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/translate INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/haproxy INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/puppet_agent INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/apt INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/facts INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/inifile INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/autosign INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/firewall INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/satellite_pe_tools INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/puppet_metrics_collector INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/grafana INFO -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/telegrafAfter that, I can see the tests begin!role::augeastest: F F F F F F F F F F F F F F F F F F F F F F F F F role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P role::example: P P P P P P P P P P P P P P P P P P P P P P P P P role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P role::loadbalancer: F F F F F F F F F F F F F F F F F F F F F F F F F role::webserver: P P P P P P P P P P P P P P P P P P P P P P P P PNow I’m seeing a couple of errors. For example:role::augeastest: failed errors: Evaluation Error: Error while evaluating a Resource Statement, Unknown resource type: 'augeas' file: site-modules/role/manifests/augeastest.pp line: 2 column: 3 factsets: AIX-6.1-powerpc, AIX-7.1-powerpc, Amazon-2018.03, CentOS-5.11-32, CentOS-5.11-64, CentOS-6.6-32, CentOS-6.6-64, CentOS-7.0-64, Debian-6.0.10-32, Debian-6.0.10-64, Debian-7.8-32, Debian-7.8-64, RHEL-6.7, RHEL-7.4, SLES-11.3-64, SLES-12.1-64, Ubuntu-12.04-32, Ubuntu-12.04-64, Ubuntu-14.04-32, Ubuntu-14.04-64, Windows_Server-2008r2-64, Windows_Server-2012r2-64, solaris-10_u9-sparc-64, solaris-11.2-sparc-64, windows-10-64After reading Puppet documentation, I realized that augeas has been removed from Puppet 6, and I have to install a module called augeas_core to use augeas resource type. Thus, I add the following line into my Puppetfile:mod 'puppetlabs-augeas_core', '1.0.5'If I run bundle exec onceover run spec again, I can see that my above error message is gone.role::augeastest: P P P P P P P P P P P P P P P P P P P P P P P P P role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P role::example: P P P P P P P P P P P P P P P P P P P P P P P P P role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P role::loadbalancer: F F F F F F F F F F F F F F F F F F F F F F F F F role::webserver: P P P P P P P P P P P P P P P P P P P P P P P P PThen, I notice there are still two errors for role::loadbalancer.First, a few messages saying The XXX system is not supported with the haproxy module. Now my question is: How can I tell Onceover to test modules with only wanted OSes? After checking with Onceover Configure, I understand I am able to configure class and node groups for tests. The Onceover configuration file is located at onceover-lab-repo/spec/onceover.yaml. I have put an example onceover-example.yaml in my control repo so we can just copy it over.cp -f onceover-example.yaml spec/onceover.yamlNow I’m going to explain the three main changes made from the onceover-example.yaml. First, I created a new class group called general_class and included all classes but role::loadbalancer:class_groups: general_class: - role::database_server - role::webserver - role::goldload_server - role::example - role::augeastestSecondly, I created a new node group called haproxy_nodes and excluded nodes that I didn’t want to test against role::loadbalancer:node_groups: windows_nodes: - Windows_Server-2012r2-64 - windows-10-64 - Windows_Server-2008r2-64 non_windows_nodes: include: 'all_nodes' exclude: 'windows_nodes' haproxy_nodes: - SLES-12.1-64 - Debian-6.0.10-32 - CentOS-6.6-64 - Ubuntu-12.04-32 - Amazon-2018.03 - Ubuntu-12.04-64 - CentOS-6.6-32 - Debian-6.0.10-64 - RHEL-7.4 - Debian-7.8-32 - SLES-11.3-64 - Debian-7.8-64 - Ubuntu-14.04-32 - CentOS-5.11-64 - CentOS-5.11-32 - RHEL-6.7 - CentOS-7.0-64 - Ubuntu-14.04-64Lastly, I needed to update my test_matrix to match the test classes and the node groups:test_matrix: - all_nodes: classes: 'general_class' tests: 'spec' - haproxy_nodes: classes: 'role::loadbalancer' tests: 'spec'If I ran bundle exec onceover run spec once more, I would see that Onceover stops testing some of the OSes that I filtered out and the above error is gone:role::augeastest: P P P P P P P P P P P P P P P P P P P P P P P P P role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P role::example: P P P P P P P P P P P P P P P P P P P P P P P P P role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P role::loadbalancer: F F F F F F F F F F F F F F F F F F role::webserver: P P P P P P P P P P P P P P P P P P P P P P P P PMy only test error left is:role::loadbalancer: failed errors: Evaluation Error: Missing title. The title expression resulted in undef file: site-modules/role/manifests/loadbalancer.pp line: 52 column: 21Onceover clearly tells me which file and which line is reporting this error. After checking that loadbalancer.pp manifest file, I realized there’s a bug in my code, which I never realized in my real Puppet production setup! To fix this bug, I need to add a conditional check to the manifest file site-modules/role/manifests/loadbalancer.pp. I have included a loadbalancer-improved.pp file in my repo. The main enhancement here is to add a if statement before managing rule2 configurations:if $rule2 != undef { haproxy::listen { $rule2 : collect_exported => false, ipaddress => $::ipaddress, ports => $ports2, } ... ...Without this, there will be a chance when $rule2 is not needed and its value will take the default value undef. Consequently, the title for this rule will be undef, which is not accepted. I’d never realized this issue in my production environment as I had always configured the second rule.Now we can copy over the file:cp -f loadbalancer-improved.pp site-modules/role/manifests/loadbalancer.ppNow, if I run bundle exec onceover run spec again…role::augeastest: P P P P P P P P P P P P P P P P P P P P P P P P P role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P role::example: P P P P P P P P P P P P P P P P P P P P P P P P P role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P role::loadbalancer: P P P P P P P P P P P P P P P P P P role::webserver: P P P P P P P P P P P P P P P P P P P P P P P P PHooray!!! All of my module tests passed and I am glad to push the current code status to my remote repository!Results of Onceover Repository TestingOnceover provides me the flexibility to test my modules with wanted OSesOnceover helps me identify my code bugs which I can’t easily find from my Puppet environmentOnceover helps me to test Puppet code with the latest versionThis is a good start; I’ve built more confidence in my Puppet code before I can push them into my production control repo. Onceover provides an easy way to test my codes, and the tests are reusable with just a simple onceover run command!Meanwhile, I’m joined by my Puppet colleagues, because Puppet has other cool tools like Puppet Development Kit (PDK) and Continuous Delivery for Puppet Enterprise. PDK provides integrated testing tools and a command line interface to help you develop, validate, and test modules. Continuous Delivery for Puppet Enterprise offers a prescriptive workflow to test and deploy Puppet code across environments! Feel excited? START MY TRIALLearn MoreClass containment in Puppet Enterprise: What it is and why it's important.Get rid of your unused modules with Dropsonde.
Henry Wang Customer Support Team Lead , Puppet by Perforce Henry is a customer support team lead for Asia-Pacific & Japan at Puppet.