Contributing to PuppetDB
Third-party patches are essential for keeping puppet great. We simply can't access the huge number of platforms and myriad configurations for running puppet. We want to keep it as easy as possible to contribute changes that get things working in your environment. There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things.
Make sure you have a Jira account
Make sure you have a GitHub account
Submit a ticket for your issue, assuming one does not already exist.
Clearly describe the issue including steps to reproduce when it is a bug.
Make sure you fill in the earliest version that you know has the issue.
Fork the repository on GitHub
Create a topic branch from where you want to base your work.
This is usually the main branch.
Only target release branches if you are certain your fix must be on that branch.
To quickly create a topic branch based on main;
git checkout -b fix/main/my_contribution main. Please avoid working directly on the
Make commits of logical units.
Check for unnecessary whitespace with
git diff --checkbefore committing.
Make sure your commit messages are in the proper format.
(PUP-1234) Make the example in CONTRIBUTING imperative and concrete Without this patch applied the example commit message in the CONTRIBUTING document is not a concrete example. This is a problem because the contributor is left to imagine what the commit message should look like based on a description rather than an example. This patch fixes the problem by making the example concrete and imperative. The first line is a real life imperative statement with a ticket number from our issue tracker. The body describes the behavior without the patch, why this is a problem, and how the patch fixes the problem when applied.
Make sure you have added the necessary tests for your changes.
Run all the tests to assure nothing else was accidentally broken.
Before you do anything else, you may want to consider setting
PUPPET_SUPPRESS_INTERNAL_LEIN_REPOS=1 in your environment. We'll
eventually make that the default, but for now that setting may help
avoid delays incurred if lein tries to reach unreachable internal
The easiest way to run the tests until you need to do it often is to use the built-in sandbox harness. If you just want to check some changes against "all the normal tests", this should work (assuming you're not running a server on port 34335):
$ ext/bin/test-config --set pgport 34335 $ ext/bin/test-config --reset puppet-ref $ ext/bin/test-config --reset puppetserver-ref $ ext/bin/run-normal-tests
This will run the core, integration, and external tests, and in some
cases may be all that you need, but in many cases, you may want to be
able to run tests more selectively as detailed below. Copies of tools
pgbox may be downloaded and installed to a temporary
directory during the process, if you don't already have the expected
When using the sandbox, you need to either specify the PostgreSQL port
it should use by providing a
--pgport PORT argument to each relevant
test invocation, or you can set a default (as above) for the source
$ ./ext/bin/test-config --set pgport 34335
After you've set the default pgport, you should be able to run the core tests like this:
$ ext/bin/boxed-core-tests -- lein test
Similarly you should be able to configure and run the integration tests against the default Puppet and Puppetserver versions like this:
$ ext/bin/test-config --reset puppet-ref $ ext/bin/test-config --reset puppetserver-ref $ ext/bin/boxed-integration-tests \ -- lein test :integration
Note that you only need to configure the puppet-ref and
puppetserver-ref one time for each tree, but you can also change the refs
when you like with
$ ext/bin/test-config --set puppet-ref 5.5.x $ ext/bin/test-config --set puppetserver-ref 5.2.x
--reset for an option resets it to the tree default, and at
the moment, you'll need to do that manually whenever you're using the
default and the relevant
*-default file in ext/test-conf changes in
The sandboxes are destroyed when the commands finish, but you can arrange to inspect the environment after a failure like this:
$ ext/bin/boxed-integration-tests \ -- bash -c 'lein test || bash'
which will drop you into a shell if anything goes wrong.
To run the local rspec tests (e.g. for the PuppetDB terminus code),
you must have configured the
described above, and then from within the
puppet/ directory you can
$ bundle exec rspec spec
If you'd like to preserve the temporary test databases on failure, you can
PDB_TEST_PRESERVE_DB_ON_FAIL to true:
$ PDB_TEST_KEEP_DB_ON_FAIL=true lein test
The sandboxed tests will try to find and use the version of PostgreSQL specified by:
$ ./ext/bin/test-config --get pgver
Unless you override that with
$ ext/bin/test-config --set pgver 9.6
Given just the version, the tests will try to find a suitable PostgreSQL
installation directory (containing executables like
etc.), but you can specify one directly like this:
$ ext/bin/test-config --set pgbin /usr/lib/postgresql/9.6/bin
pgbin is set, the pgver setting will be irrelevant until/unless you
$ ext/bin/test-config --reset pgbin
If you're running the tests all the time, you might want to set up your own persistent sandbox instead so you can run tests directly against that:
$ ext/bin/pdbbox-init \ --sandbox ~/tmp/pdb-sandbox \ --pgbin /usr/lib/postgresql-9.6/bin \ --pgport 17961
Using a persistent sandbox requires that you install
pgbox somewhere in your
is a single-file Python script for creating and using PostgreSQL sandboxes. You
pgbox manually from GitLab or use the
$ ext/bin/require-pgbox 0.0.0 /tmp/pgbox-install $ mv /tmp/pgbox-install/bin/pgbox ~/.local/bin
ext/bin/pdbbox-init script uses
pgbox under the hood to create a
PostgreSQL sandbox inside of your PuppetDB sandbox in the directory
directory contains configuration files, a database cluster, and a symlink to a
PostgreSQL installation directory (defined with
You can use
pgbox directly by setting an environment variable called
with the path to the internal PostgreSQL sandbox:
$ PGBOX=~/tmp/pdb-sandbox/pg pgbox pg_ctl start -w
ext/bin/pdbbox-env script is a thin layer on top of
pgbox that wraps
the given command in a PuppetDB sandbox as well. This script is required for
running and testing PuppetDB. It can also be used instead of
interact with the PuppetDB test database:
$ export PDBBOX=~/tmp/pdb-sandbox $ ext/bin/pdbbox-env pg_ctl start -w $ ext/bin/pdbbox-env pg_ctl stop
Once the database server is running you can run the tests like this:
$ export PDBBOX=~/tmp/pdb-sandbox $ ext/bin/pdbbox-env lein test
Note that in cases where durability and realistic performance aren't
important (say for routine
lein test runs), you may see substantially
better performance if you disable PostgreSQL's fsync calls with
$ ext/bin/pdbbox-env pg_ctl start -o -F -w
Before you can run the integration tests directly, you'll need to configure the puppet and puppetserver versions you want to use. Assuming you have suitable versions of Ruby and Bundler available, you can do this:
$ ext/bin/test-config --reset puppet-ref $ ext/bin/test-config --reset puppetserver-ref
The default puppet and puppetserver versions are recorded in
ext/test-conf/. You can request specific versions of puppet or
puppetserver like this:
$ ext/bin/test-config --set puppet-ref 5.3.x $ ext/bin/test-config --set puppetserver-ref 5.3.x
Run the tools again to change the requested versions, and
distclean will completely undo the configurations.
After configuration you should be able to run the tests by specifying
$ export PDBBOX=~/tmp/pdb-sandbox $ ext/bin/pdbbox-env lein test :integration
You can also run puppetdb itself with the config file included in the sandbox:
$ export PDBBOX=~/tmp/pdb-sandbox $ ext/bin/pdbbox-env lein run services \ -c ~/tmp/pdb-sandbox/conf.d
lein run services may occasionally fail with an
Unable to resolve
symbol error. If this happens, run
lein distclean and try again.
And finally, you can of course set up and configure your own PostgreSQL server for testing, but then you'll need to create the test users:
$ createuser -DRSP pdb_test $ createuser -dRsP pdb_test_admin
and do the other things that
pdbbox-init normally handles, like
setting environment variables if the default values aren't
PDB_TEST_DB_HOST(defaults to localhost)
PDB_TEST_DB_PORT(defaults to 5432)
lein clean will clean up the relevant items related to
Clojure, but won't affect some other things, including the integration
test configuration. To clean up "everything", run
Making Trivial Changes
For changes of a trivial nature to comments and documentation, it is not always necessary to create a new ticket in Jira. In this case, it is appropriate to start the first line of a commit with '(doc)' instead of a ticket number.
(doc) Add documentation commit example to CONTRIBUTING There is no example for contributing a documentation commit to the Puppet repository. This is a problem because the contributor is left to assume how a commit of this nature may appear. The first line is a real life imperative statement with '(doc)' in place of what would have been the ticket number in a non-documentation related commit. The body describes the nature of the new documentation or comments added.
Sign the Contributor License Agreement.
Push your changes to a topic branch in your fork of the repository.
Submit a pull request to the repository in the puppetlabs organization.
Update your Jira ticket to mark that you have submitted code and are ready for it to be reviewed (Status: Ready for Merge).
Include a link to the pull request in the ticket.
After feedback has been given we expect responses within two weeks. After two weeks will may close the pull request if it isn't showing any activity.