May 20, 2020

How to Use Onceover to Test Your Puppet Control Repository

Ecosystems & Integrations

How to & Use Cases

Is there an easier way to do repository testing for your Puppet code? Yes! The answer is using Onceover.

What is Onceover?
How to Use Onceover for Puppet Repo Testing
The Benefits of Onceover Repository Testing

What is Onceover?

Onceover is a free, open source tool for testing code in a Puppet control repository.

Testing code on Puppet agent nodes can be a bit tedious. Normally, you'd have to...

Save code changes,
Push into remote repo,
Run code deploy, and
Run Puppet on agents.

Rather than setting up a testing framework inside a controlrepo for basic tests, Onceover streamlines the process of setting up and running rspec-puppet tests for a control repository.

You can download Onceover for Puppet from GitHub.

How to Use Onceover for Puppet Repo Testing

For as handy as it is, Onceover does require some configuration to get started.

1. Prepare the Lab Environment

In this example, I am using a fresh installation of RHEL 7.

Install Git

sudo yum install git -y

Install Ruby by RVM

As to the Ruby version we want to install, review the Ruby versions for PE releases. In our example here, we are going to install Ruby v2.5.7. We can use RVM to install wanted Ruby versions easily.

##Install RVM
sudo gpg2 --keyserver hkp://ipv4.pool.sks-keyservers.net  --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
curl -sSL https://get.rvm.io | sudo bash -s stable
sudo usermod -a -G rvm `whoami`

##Logout and Login to use RVM
exit

##Install Ruby 2.5.7
rvm install ruby-2.5.7
ruby -v

Now we are ready to install Onceover and start our testing!

2. Install Onceover and Test the Puppet Code

Clone down my control-repo

You can use this example repo for learning purposes, but you should ultimately try Onceover at your own control repo.

git clone https://github.com/henrywangpuppet/onceover-lab-repo.git
cd onceover-lab-repo/
git checkout onceoverlab

Okay, now I need to install the onceover gem.

gem install bundler
bundler init
echo "gem \"onceover\"" >> Gemfile
bundle install

Now it’s time to configure and test.

Now Onceover has been installed. The README says I need to configure it, and I can do so by running bundle exec onceover init.

bundle exec onceover init

As the command runs, I can see that it created a number of files for me.

created spec
created spec/onceover.yaml
created spec/pre_conditions
created spec/pre_conditions/README.md
created spec/factsets
created spec/factsets/README.md
created Rakefile
skipped Gemfile (exists)

I’m curious what’s going to happen if I just try to run the spec tests. Back to the README, where I discover I can run bundle exec onceover run spec.

bundle exec onceover run spec

Starting at the beginning of the output, I see that it’s deploying my modules from the Puppetfile.

INFO     -> Using Puppetfile '/root/onceover-lab-repo/Puppetfile'
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/concat
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/stdlib
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/translate
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/haproxy
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/puppet_agent
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/apt
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/facts
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/inifile
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/autosign
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/firewall
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/satellite_pe_tools
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/puppet_metrics_collector
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/grafana
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/telegraf

After that, I can see the tests begin!

role::augeastest:      F F F F F F F F F F F F F F F F F F F F F F F F F
role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::example:         P P P P P P P P P P P P P P P P P P P P P P P P P
role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::loadbalancer:    F F F F F F F F F F F F F F F F F F F F F F F F F
role::webserver:       P P P P P P P P P P P P P P P P P P P P P P P P P

Now I’m seeing a couple of errors. For example:

role::augeastest: failed
  errors:
    Evaluation Error: Error while evaluating a Resource Statement, Unknown resource type: 'augeas'
      file: site-modules/role/manifests/augeastest.pp
      line: 2
      column: 3
      factsets: AIX-6.1-powerpc, AIX-7.1-powerpc, Amazon-2018.03, CentOS-5.11-32, CentOS-5.11-64, CentOS-6.6-32, CentOS-6.6-64, CentOS-7.0-64, Debian-6.0.10-32, Debian-6.0.10-64, Debian-7.8-32, Debian-7.8-64, RHEL-6.7, RHEL-7.4, SLES-11.3-64, SLES-12.1-64, Ubuntu-12.04-32, Ubuntu-12.04-64, Ubuntu-14.04-32, Ubuntu-14.04-64, Windows_Server-2008r2-64, Windows_Server-2012r2-64, solaris-10_u9-sparc-64, solaris-11.2-sparc-64, windows-10-64

After reading Puppet documentation, I realized that augeas has been removed from Puppet 6, and I have to install a module called augeas_core to use augeas resource type. Thus, I add the following line into my Puppetfile:

mod 'puppetlabs-augeas_core', '1.0.5'

If I run bundle exec onceover run spec again, I can see that my above error message is gone.

role::augeastest:      P P P P P P P P P P P P P P P P P P P P P P P P P
role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::example:         P P P P P P P P P P P P P P P P P P P P P P P P P
role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::loadbalancer:    F F F F F F F F F F F F F F F F F F F F F F F F F
role::webserver:       P P P P P P P P P P P P P P P P P P P P P P P P P

Then, I notice there are still two errors for role::loadbalancer.

First, a few messages saying The XXX system is not supported with the haproxy module. Now my question is: How can I tell Onceover to test modules with only wanted OSes? After checking with Onceover Configure, I understand I am able to configure class and node groups for tests. The Onceover configuration file is located at onceover-lab-repo/spec/onceover.yaml. I have put an example onceover-example.yaml in my control repo so we can just copy it over.

cp -f onceover-example.yaml spec/onceover.yaml

Now I’m going to explain the three main changes made from the onceover-example.yaml. First, I created a new class group called general_class and included all classes but role::loadbalancer:

class_groups:
  general_class:
    - role::database_server
    - role::webserver
    - role::goldload_server
    - role::example
    - role::augeastest

Secondly, I created a new node group called haproxy_nodes and excluded nodes that I didn’t want to test against role::loadbalancer:

node_groups:
  windows_nodes:
    - Windows_Server-2012r2-64
    - windows-10-64
    - Windows_Server-2008r2-64
  non_windows_nodes:
    include: 'all_nodes'
    exclude: 'windows_nodes'
  haproxy_nodes:
    - SLES-12.1-64
    - Debian-6.0.10-32
    - CentOS-6.6-64
    - Ubuntu-12.04-32
    - Amazon-2018.03
    - Ubuntu-12.04-64
    - CentOS-6.6-32
    - Debian-6.0.10-64
    - RHEL-7.4
    - Debian-7.8-32
    - SLES-11.3-64
    - Debian-7.8-64
    - Ubuntu-14.04-32
    - CentOS-5.11-64
    - CentOS-5.11-32
    - RHEL-6.7
    - CentOS-7.0-64
    - Ubuntu-14.04-64

Lastly, I needed to update my test_matrix to match the test classes and the node groups:

test_matrix:
  - all_nodes:
      classes: 'general_class'
      tests: 'spec'
  - haproxy_nodes:
      classes: 'role::loadbalancer'
      tests: 'spec'

If I ran bundle exec onceover run spec once more, I would see that Onceover stops testing some of the OSes that I filtered out and the above error is gone:

role::augeastest:      P P P P P P P P P P P P P P P P P P P P P P P P P
role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::example:         P P P P P P P P P P P P P P P P P P P P P P P P P
role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::loadbalancer:    F F F F F F F F F F F F F F F F F F
role::webserver:       P P P P P P P P P P P P P P P P P P P P P P P P P

My only test error left is:

role::loadbalancer: failed
  errors:
    Evaluation Error: Missing title. The title expression resulted in undef
      file: site-modules/role/manifests/loadbalancer.pp
      line: 52
      column: 21

Onceover clearly tells me which file and which line is reporting this error. After checking that loadbalancer.pp manifest file, I realized there’s a bug in my code, which I never realized in my real Puppet production setup! To fix this bug, I need to add a conditional check to the manifest file site-modules/role/manifests/loadbalancer.pp. I have included a loadbalancer-improved.pp file in my repo. The main enhancement here is to add a if statement before managing rule2 configurations:

if $rule2 != undef
   {
     haproxy::listen { $rule2 :
      collect_exported => false,
      ipaddress        => $::ipaddress,
      ports            => $ports2,
     }
...
...

Without this, there will be a chance when $rule2 is not needed and its value will take the default value undef. Consequently, the title for this rule will be undef, which is not accepted. I’d never realized this issue in my production environment as I had always configured the second rule.

Now we can copy over the file:

cp -f loadbalancer-improved.pp site-modules/role/manifests/loadbalancer.pp

Now, if I run bundle exec onceover run spec again…

role::augeastest:      P P P P P P P P P P P P P P P P P P P P P P P P P
role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::example:         P P P P P P P P P P P P P P P P P P P P P P P P P
role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::loadbalancer:    P P P P P P P P P P P P P P P P P P
role::webserver:       P P P P P P P P P P P P P P P P P P P P P P P P P

Hooray!!! All of my module tests passed and I am glad to push the current code status to my remote repository!

The Benefits of Onceover Repository Testing

Onceover helps test Puppet code with the latest version
Onceover provides the flexibility to test modules with wanted OSes
Onceover helps identify code bugs (which aren't easy to find from a Puppet environment)

This is a good start; I’ve built more confidence in my Puppet code before I can push them into my production control repo.

Onceover provides an easy way to test my code, and the tests are reusable with just a simple onceover run command!

If you're excited at the idea of doing more with Puppet, try a free training course from Puppet Tech Ed. You can learn the basics and take more advanced training on your schedule or with the Puppet pros.

Learn More

Class containment in Puppet Enterprise: What it is and why it's important
Get rid of your unused modules with Dropsonde

Achieving Zero Trust Security with Puppet Enterprise

Achieving Zero Trust Security with Puppet Enterprise

How to Use Onceover to Test Your Puppet Control Repository

Table of Contents

What is Onceover?

How to Use Onceover for Puppet Repo Testing

1. Prepare the Lab Environment

Install Git

Install Ruby by RVM

2. Install Onceover and Test the Puppet Code

Clone down my control-repo

The Benefits of Onceover Repository Testing

Learn More

Get Started

Puppet Enterprise

Puppet Enterprise Extensions

How to Use Onceover to Test Your Puppet Control Repository

Table of Contents

What is Onceover?

How to Use Onceover for Puppet Repo Testing

1. Prepare the Lab Environment

Install Git

Install Ruby by RVM

2. Install Onceover and Test the Puppet Code

Clone down my control-repo

The Benefits of Onceover Repository Testing

Learn More