May 20, 2020

How to Use Onceover For Repository Testing

Ecosystems & Integrations

How to & Use Cases

By Henry Wang

Is there an easier way to do repository testing for your Puppet code? Yes! The answer is using Onceover.

Table of Contents:

What Is Onceover?
Why Use Onceover For Repository Testing?
How to Use Onceover Repository Testing For Puppet
Results of Onceover Repository Testing

What Is Onceover?

Onceover is a free, open source tool that you can use to test your Puppet repository.

Why Use Onceover For Repository Testing?

Onceover makes it easy to do repository testing on your Puppet code.

I am a Puppet beginner and I’m happy to make the code manager integration happen. I like to play around with my code and test my code changes on my agent nodes. However, the way I’m testing my code is a bit tedious: Save my code change, push into remote repo, run code deploy, and run Puppet on agents.

Until I tried Onceover.

How to Use Onceover Repository Testing For Puppet

1. Prepare the Lab Environment

I am using a fresh installation of RHEL 7.

Install Git

sudo yum install git -y

Install Ruby by RVM

As to the Ruby version we want to install, review the Ruby versions for PE releases. In our example here, we are going to install Ruby v2.5.7. We can use RVM to install wanted Ruby versions easily.

##Install RVM
sudo gpg2 --keyserver hkp://ipv4.pool.sks-keyservers.net  --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
curl -sSL https://get.rvm.io | sudo bash -s stable
sudo usermod -a -G rvm `whoami`

##Logout and Login to use RVM
exit

##Install Ruby 2.5.7
rvm install ruby-2.5.7
ruby -v

Now we are ready to install Onceover and start our testing!

2. Install Onceover and Test the Puppet Code

Clone down my control-repo

You can use this example repo for learning purposes, but you should ultimately try Onceover at your own control repo.

git clone https://github.com/henrywangpuppet/onceover-lab-repo.git
cd onceover-lab-repo/
git checkout onceoverlab

Okay, now I need to install the onceover gem.

gem install bundler
bundler init
echo "gem \"onceover\"" >> Gemfile
bundle install

Now it’s time to configure and test.

Now Onceover has been installed. The README says I need to configure it, and I can do so by running bundle exec onceover init.

bundle exec onceover init

As the command runs, I can see that it created a number of files for me.

created spec
created spec/onceover.yaml
created spec/pre_conditions
created spec/pre_conditions/README.md
created spec/factsets
created spec/factsets/README.md
created Rakefile
skipped Gemfile (exists)

I’m curious what’s going to happen if I just try to run the spec tests. Back to the README, where I discover I can run bundle exec onceover run spec.

bundle exec onceover run spec

Starting at the beginning of the output, I see that it’s deploying my modules from the Puppetfile.

INFO     -> Using Puppetfile '/root/onceover-lab-repo/Puppetfile'
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/concat
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/stdlib
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/translate
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/haproxy
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/puppet_agent
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/apt
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/facts
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/inifile
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/autosign
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/firewall
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/satellite_pe_tools
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/puppet_metrics_collector
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/grafana
INFO     -> Updating module /root/onceover-lab-repo/.onceover/etc/puppetlabs/code/environments/production/modules/telegraf

After that, I can see the tests begin!

role::augeastest:      F F F F F F F F F F F F F F F F F F F F F F F F F
role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::example:         P P P P P P P P P P P P P P P P P P P P P P P P P
role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::loadbalancer:    F F F F F F F F F F F F F F F F F F F F F F F F F
role::webserver:       P P P P P P P P P P P P P P P P P P P P P P P P P

Now I’m seeing a couple of errors. For example:

role::augeastest: failed
  errors:
    Evaluation Error: Error while evaluating a Resource Statement, Unknown resource type: 'augeas'
      file: site-modules/role/manifests/augeastest.pp
      line: 2
      column: 3
      factsets: AIX-6.1-powerpc, AIX-7.1-powerpc, Amazon-2018.03, CentOS-5.11-32, CentOS-5.11-64, CentOS-6.6-32, CentOS-6.6-64, CentOS-7.0-64, Debian-6.0.10-32, Debian-6.0.10-64, Debian-7.8-32, Debian-7.8-64, RHEL-6.7, RHEL-7.4, SLES-11.3-64, SLES-12.1-64, Ubuntu-12.04-32, Ubuntu-12.04-64, Ubuntu-14.04-32, Ubuntu-14.04-64, Windows_Server-2008r2-64, Windows_Server-2012r2-64, solaris-10_u9-sparc-64, solaris-11.2-sparc-64, windows-10-64

After reading Puppet documentation, I realized that augeas has been removed from Puppet 6, and I have to install a module called augeas_core to use augeas resource type. Thus, I add the following line into my Puppetfile:

mod 'puppetlabs-augeas_core', '1.0.5'

If I run bundle exec onceover run spec again, I can see that my above error message is gone.

role::augeastest:      P P P P P P P P P P P P P P P P P P P P P P P P P
role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::example:         P P P P P P P P P P P P P P P P P P P P P P P P P
role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::loadbalancer:    F F F F F F F F F F F F F F F F F F F F F F F F F
role::webserver:       P P P P P P P P P P P P P P P P P P P P P P P P P

Then, I notice there are still two errors for role::loadbalancer.

First, a few messages saying The XXX system is not supported with the haproxy module. Now my question is: How can I tell Onceover to test modules with only wanted OSes? After checking with Onceover Configure, I understand I am able to configure class and node groups for tests. The Onceover configuration file is located at onceover-lab-repo/spec/onceover.yaml. I have put an example onceover-example.yaml in my control repo so we can just copy it over.

cp -f onceover-example.yaml spec/onceover.yaml

Now I’m going to explain the three main changes made from the onceover-example.yaml. First, I created a new class group called general_class and included all classes but role::loadbalancer:

class_groups:
  general_class:
    - role::database_server
    - role::webserver
    - role::goldload_server
    - role::example
    - role::augeastest

Secondly, I created a new node group called haproxy_nodes and excluded nodes that I didn’t want to test against role::loadbalancer:

node_groups:
  windows_nodes:
    - Windows_Server-2012r2-64
    - windows-10-64
    - Windows_Server-2008r2-64
  non_windows_nodes:
    include: 'all_nodes'
    exclude: 'windows_nodes'
  haproxy_nodes:
    - SLES-12.1-64
    - Debian-6.0.10-32
    - CentOS-6.6-64
    - Ubuntu-12.04-32
    - Amazon-2018.03
    - Ubuntu-12.04-64
    - CentOS-6.6-32
    - Debian-6.0.10-64
    - RHEL-7.4
    - Debian-7.8-32
    - SLES-11.3-64
    - Debian-7.8-64
    - Ubuntu-14.04-32
    - CentOS-5.11-64
    - CentOS-5.11-32
    - RHEL-6.7
    - CentOS-7.0-64
    - Ubuntu-14.04-64

Lastly, I needed to update my test_matrix to match the test classes and the node groups:

test_matrix:
  - all_nodes:
      classes: 'general_class'
      tests: 'spec'
  - haproxy_nodes:
      classes: 'role::loadbalancer'
      tests: 'spec'

If I ran bundle exec onceover run spec once more, I would see that Onceover stops testing some of the OSes that I filtered out and the above error is gone:

role::augeastest:      P P P P P P P P P P P P P P P P P P P P P P P P P
role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::example:         P P P P P P P P P P P P P P P P P P P P P P P P P
role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::loadbalancer:    F F F F F F F F F F F F F F F F F F
role::webserver:       P P P P P P P P P P P P P P P P P P P P P P P P P

My only test error left is:

role::loadbalancer: failed
  errors:
    Evaluation Error: Missing title. The title expression resulted in undef
      file: site-modules/role/manifests/loadbalancer.pp
      line: 52
      column: 21

Onceover clearly tells me which file and which line is reporting this error. After checking that loadbalancer.pp manifest file, I realized there’s a bug in my code, which I never realized in my real Puppet production setup! To fix this bug, I need to add a conditional check to the manifest file site-modules/role/manifests/loadbalancer.pp. I have included a loadbalancer-improved.pp file in my repo. The main enhancement here is to add a if statement before managing rule2 configurations:

if $rule2 != undef
   {
     haproxy::listen { $rule2 :
      collect_exported => false,
      ipaddress        => $::ipaddress,
      ports            => $ports2,
     }
...
...

Without this, there will be a chance when $rule2 is not needed and its value will take the default value undef. Consequently, the title for this rule will be undef, which is not accepted. I’d never realized this issue in my production environment as I had always configured the second rule.

Now we can copy over the file:

cp -f loadbalancer-improved.pp site-modules/role/manifests/loadbalancer.pp

Now, if I run bundle exec onceover run spec again…

role::augeastest:      P P P P P P P P P P P P P P P P P P P P P P P P P
role::database_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::example:         P P P P P P P P P P P P P P P P P P P P P P P P P
role::goldload_server: P P P P P P P P P P P P P P P P P P P P P P P P P
role::loadbalancer:    P P P P P P P P P P P P P P P P P P
role::webserver:       P P P P P P P P P P P P P P P P P P P P P P P P P

Hooray!!! All of my module tests passed and I am glad to push the current code status to my remote repository!

Results of Onceover Repository Testing

Onceover provides me the flexibility to test my modules with wanted OSes
Onceover helps me identify my code bugs which I can’t easily find from my Puppet environment
Onceover helps me to test Puppet code with the latest version

This is a good start; I’ve built more confidence in my Puppet code before I can push them into my production control repo. Onceover provides an easy way to test my codes, and the tests are reusable with just a simple onceover run command!

Meanwhile, I’m joined by my Puppet colleagues, because Puppet has other cool tools like Puppet Development Kit (PDK) and Continuous Delivery for Puppet Enterprise. PDK provides integrated testing tools and a command line interface to help you develop, validate, and test modules. Continuous Delivery for Puppet Enterprise offers a prescriptive workflow to test and deploy Puppet code across environments! Feel excited?

START MY TRIAL

Learn More

Class containment in Puppet Enterprise: What it is and why it's important.
Get rid of your unused modules with Dropsonde.

Achieving Zero Trust Security with Puppet Enterprise

Achieving Zero Trust Security with Puppet Enterprise

How to Use Onceover For Repository Testing

What Is Onceover?

Why Use Onceover For Repository Testing?

How to Use Onceover Repository Testing For Puppet

1. Prepare the Lab Environment

Install Git

Install Ruby by RVM

2. Install Onceover and Test the Puppet Code

Clone down my control-repo

Results of Onceover Repository Testing

Learn More

Henry Wang

Puppet Enterprise

Puppet Enterprise Extensions

Get Started

How to Use Onceover For Repository Testing

What Is Onceover?

Why Use Onceover For Repository Testing?

How to Use Onceover Repository Testing For Puppet

1. Prepare the Lab Environment

Install Git

Install Ruby by RVM

2. Install Onceover and Test the Puppet Code

Clone down my control-repo

Results of Onceover Repository Testing

Learn More

Henry Wang