Writing plans in Puppet language
Writing plans in the Puppet language gives you better error handling and more sophisticated control than YAML plans. Plans written in the Puppet language also allow you to apply blocks of Puppet code to remote targets.
Naming plans
Name plans according to the module name, file name, and path to ensure code readability.
./plans
directory, using these file extensions:-
Puppet plans —
.pp
-
YAML plans —
.yaml
, not.yml
Plan names are composed of two or more name segments, indicating:
-
The name of the module the plan is located in.
-
The name of the plan file, without the extension.
-
If the plan is in a subdirectory of
./plans
, the path within the module.
For example, given a module called mymodule
with a plan defined in ./mymodule/plans/myplan.pp
, the plan name is mymodule::myplan
.
A plan defined in ./mymodule/plans/service/myplan.pp
would be
mymodule::service::myplan
. Use teh plan name to refer to the
plan when you run commands.
The plan filename init
is special because the plan it defines
is referenced using the module name only. For example, in a module called mymodule
, the plan defined in init.pp
is the mymodule
plan.
Avoid giving plans the same names as constructs in the Puppet language. Although plans don't share their namespace with other language constructs, giving plans these names makes your code difficult to read.
Each plan name segment:
-
Must begin with a lowercase letter.
- Can include lowercase letters, digits, or underscores.
-
Must not be a reserved word.
-
Must not have the same name as any Puppet data types.
-
Namespace segments must match the regular expression
\A[a-z][a-z0-9_]*\Z
Defining plan permissions
RBAC for plans is distinct from RBAC for individual tasks. This distinction means that a user can be excluded from running a certain task, but still have permission to run a plan that contains that task.
The RBAC structure for plans allows you to write plans with more robust, custom control over task permissions. Instead of allowing a user free rein to run a task that can potentially damage your infrastructure, you can wrap a task in a plan and only allow them to run it under circumstances you control.
For example, if you are configuring permissions for a new user to run plan
infra::upgrade_git
, you can allow them to run the package
task but limit it to the git
package only.
plan infra::upgrade_git (
TargetSpec $targets,
Integer $version,
) {
run_task(‘package’, $targets, name => ’git’, action => ‘upgrade’, version => $version)
}
Use parameter types to fine-tune access
Parameter types provide another layer of control over user permissions. In the
upgrade_git
example above, the plan only provides access to the
git
package, but the user can choose whatever version of
git
they want. If there are known vulnerabilities in some versions of the
git
package, you can use parameter types like Enum
to restrict the version
parameter to
versions that are safe enough for deployment.
Enum
restricts the
$version
parameter to versions 1:2.17.0-1ubuntu1 and 1:2.17.1-1ubuntu0.4
only.plan infra::upgrade_git (
TargetSpec $targets,
Enum['1:2.17.0-1ubuntu1', '1:2.17.1-1ubuntu0.4'] $version,
) {
run_task(‘package’, $targets, name => ‘git’, action => ‘upgrade’, version => $version)
}
You can also use PuppetDB queries to select parameter types.
infra::upgrade_git
can run on, use a PuppetDB query to identify which targets
are selected for the git upgrade.plan infra::upgrade_git (
Enum['1:2.17.0-1ubuntu1', '1:2.17.1-1ubuntu0.4'] $version,
) {
# Use puppetdb to find the nodes from the “other” team's web cluster
$query = [from, nodes, ['=', [fact, cluster], "other_team"]]
$selected_nodes = puppetdb_query($query).map() |$target| {
$target[certname]
}
run_task(‘package’, $selected_nodes, name => ‘git’, action => ‘upgrade’, version => $version)
}
Specifying plan parameters
Specify plan parameters to do things like determine which targets to run different parts
of your plan on. You can pass a parameter as a single target name, comma-separated list of
target names, Target
data type, or array. The target names can
be either certnames or inventory node names.
The example plan below shows the target parameters $load_balancers
and $webservers
specified as data
type TargetSpec
. The plan then calls the run_task
function to specify which targets to run the tasks on. The
Target
names are collected and stored in $webserver_names
by iterating over the list of Target
objects returned by
get_targets
. Task parameters are serialized to JSON format so that
extracting the names into an array of strings ensures that the webservers
parameter is in a format that can be converted to JSON.
plan mymodule::my_plan(
TargetSpec $load_balancer,
TargetSpec $webservers,
) {
# Extract the Target name from $webservers
$webserver_names = get_targets($webservers).map |$n| { $n.name }
# process webservers
run_task('mymodule::lb_remove', $load_balancer, webservers => $webserver_names)
run_task('mymodule::update_frontend_app', $webservers, version => '1.2.3')
run_task('mymodule::lb_add', $load_balancer, webservers => $webserver_names)
}
To execute this plan from the command line, pass the parameters as parameter=value
. The Targetspec
accepts either an
array as JSON or a comma separated string of target names.
puppet plan run mymodule::myplan
load_balancer=lb.myorg.com
webservers='["kermit.myorg.com","gonzo.myorg.com"]'
curl -k -X POST -H "Content-Type: application/json" \
-H "X-Authentication:$TOKEN" \
-d '{ "environment": "$ENV", \
"plan_name": "mymodule::myplan", \
"params": {"targets": "$TARGET_NAME", \
"load_balancer": "lb.myorg.com", \
"webservers": ["kermit.myorg.com", "gonzo.myorg.com"]} }' \
"https://$PRIMARY_HOST:8143/orchestrator/v1/command/plan_run"
run_*
plan functions are serialized to JSON.
For example, in the plan below, the default value of $example_nul
is
undef
. The plan calls the test::demo_undef_bash
with the
example_nul
parameter.plan test::parameter_passing (
TargetSpec $targets,
Optional[String[1]] $example_nul = undef,
) {
return run_task('test::demo_undef_bash', $targets, example_nul => $example_nul)
}
demo_undef_bash.sh
task
is:#!/bin/bash
example_env=$PT_example_nul
echo "Environment: $PT_example_nul"
echo "Stdin:"
cat -
By default, the task expects parameters passed as a JSON string on stdin
to be accessible in prefixed environment variables.
curl -k -X POST -H "Content-Type: application/json" \
-H "X-Authentication:$TOKEN" \
-d '{ "environment": "$ENV", \
"plan_name": "test::parameter_passing", \
"params": {"targets": "$TARGET_NAME"} }' \
"https://$PRIMARY_HOST:8143/orchestrator/v1/command/plan_run"
Using Hiera data in plans
Use the lookup()
function in plans to look up Hiera data. You can look up data inside or outside of apply
blocks, or use the plan_hierarchy
key to look up data both
inside and outside apply blocks within the same plan.
Inside apply blocks, PE compiles catalogs for each target and has unlimited access to your Hiera data. You can use the same Hiera configuration, data, and lookup process as you do throughout PE.
Outside apply blocks, the plan executes a script, doesn't have a concept of a target or context, and cannot load per-target data. These limitations make some common Hiera features, like interpolating target facts, incompatible with plans in PE outside of apply blocks.
You can look up static Hiera data outside of apply blocks
by adding a plan_hierarchy
key to your Hiera configuration at the same level as the hierarchy
key. This allows you to look up data inside and outside apply blocks in the same plan,
enabling you to use your existing Hiera configuration in
plans without encountering an error if per-target interpolations exist and your plan
tries to look up data outside an apply block.
Static Hiera data is also useful for user-specific data that you want the plan to look up.
<ENV_DIR>/hiera.yaml
.version: 5
hierarchy:
- name: "Target specific data"
path: "targets/%{trusted.certname}.yaml"
- name: "Per-OS defaults"
path: "os/%{facts.os.family}.yaml"
- name: Common
path: hierarchy.yaml
plan_hierarchy:
- name: Common
path: plan_hierarchy.yaml
You can set a user-specific API key in the plan_hierarchy.yaml
data file, as well as use Hiera to look up a per-target filepath inside an apply
block by using the following pieces of data:
<ENV_DIR>/data/plan_hierarchy.yaml
:api_key: 12345
<ENV_DIR>/data/targets/myhost.com
:confpath: "C:\Program Files\Common Files\mytool.conf"
plan plan_lookup(
TargetSpec $targets
) {
$outside_apply = lookup('api_key')
run_task("make_request", $targets, 'api_key' => $outside_apply)
$in_apply = apply($targets) {
file { ${confpath}:
ensure => file,
content => "setting: false"
}
}
}
Target objects
The Target
object represents a target and its specific
connection options.
The state of a target is stored in the code for the duration of a plan, allowing you to collect facts or set variables for a target and retrieve them later. Target objects must reference a target in the PE inventory. This includes targets connected via the PCP protocol that have puppet-agent installed, or targets in the PE inventory added with either SSH or WinRM credentials or as network devices. Target objects in PE do not have control over their connection information, and the connection info cannot be changed from within a plan.
Because target objects in PE are references, and cannot control their own configuration, accessing target connection info will return empty data.
TargetSpec
The TargetSpec
type is a wrapper for defining targets that
allows you to pass a target, or multiple targets, into a plan. Use TargetSpec
for plans that accept a set of targets as a parameter to ensure clean
interaction with the CLI and other plans.
TargetSpec
accepts strings allowed by --targets
, a single target object, or an array of targets and
target patterns. To operate on an individual target, resolve the target to a list via
get_targets
.
For example, to loop over each target in a plan, accept a TargetSpec
argument, but call get_targets
on it
before looping.
plan loop(TargetSpec $targets) {
get_targets($targets).each |$target| {
run_task('my_task', $target)
}
}
Set variables and facts on targets
You can use the $target.facts()
and $target.vars()
functions to set transport configuration values, variables, and
facts from a plan. Facts come from running facter
or another
fact collection application on the target, or from a fact store like PuppetDB. Variables are computed externally or assigned
directly.
For example, set variables in a plan using $target.set_var
:
plan vars(String $host) {
$target = get_targets($host)[0]
$target.set_var('newly_provisioned', true)
$targetvars = $target.vars
run_command("echo 'Vars for ${host}: ${$targetvars}'", $host)
}
Or set variables in the inventory file using the vars
key
at the group level.
groups:
- name: my_targets
targets:
- localhost
vars:
operatingsystem: windows
config:
transport: ssh
Collect facts from targets
The facts
plan connects to targets,discovers facts, and
stores these facts on the targets.
The plan uses these methods to collect facts:
- On
ssh
targets, it runs a Bash script. - On
winrm
targets, it runs a PowerShell script. - On
pcp
or targets where the Puppet agent is present, it runs Facter.
facts
plan to collect facts and then
uses those facts to decide which task to run on the
targets.plan run_with_facts(TargetSpec $targets) {
# This collects facts on targets and update the inventory
run_plan(facts, targets => $targets)
$centos_targets = get_targets($targets).filter |$n| { $n.facts['os']['name'] == 'CentOS' }
$ubuntu_targets = get_targets($targets).filter |$n| { $n.facts['os']['name'] == 'Ubuntu' }
run_task(centos_task, $centos_targets)
run_task(ubuntu_task, $ubuntu_targets)
}
Collect facts from PuppetDB
You can use the puppetdb_fact
plan to collect facts for
targets when they are running a Puppet agent and sending
facts to PuppetDB.
For example, use the puppetdb_fact
plan to collect facts,
and then use those facts to decide which task to run on the targets.
plan run_with_facts(TargetSpec $targets) {
# This collects facts on targets and update the inventory
run_plan(puppetdb_fact, targets => $targets)
$centos_targets = get_targets($targets).filter |$n| { $n.facts['os']['name'] == 'CentOS' }
$ubuntu_targets = get_targets($targets).filter |$n| { $n.facts['os']['name'] == 'Ubuntu' }
run_task(centos_task, $centos_targets)
run_task(ubuntu_task, $ubuntu_targets)
}
Collect general data from PuppetDB
You can use the puppetdb_query
function in plans to make direct queries to
PuppetDB.
plan pdb_discover {
$result = puppetdb_query("inventory[certname] { app_role == 'web_server' }")
# extract the certnames into an array
$names = $result.map |$r| { $r["certname"] }
# wrap in url. You can skip this if the default transport is pcp
$targets = $names.map |$n| { "pcp://${n}" }
run_task('my_task', $targets)
}
Returning results from plans
Use the return
function to return results that you can
use in other plans or save for other uses.
Any plan that does not call the return
function returns undef
.
For example,
plan return_result(
$targets
) {
return run_task('mytask', $targets)
}
The result of a plan must match the PlanResult
type alias. This
includes JSON types as well as the plan language types, which have well defined JSON.
-
Undef
-
String
-
Numeric
-
Boolean
-
Target
ApplyResult
-
Result
-
ResultSet
-
Error
-
Array
with onlyPlanResult
- Hash with
String
keys andPlanResult
values
or
Variant[Data, String, Numeric, Boolean, Error, Result, ResultSet, Target, Array[Boltlib::PlanResult], Hash[String, Boltlib::PlanResult]]
Plan errors and failure
Any plan that completes execution without an error is considered successful. There are
some specific scenarios that always cause a plan failure, such as calling the fail_plan
function.
Plan failure due to absent catch_errors
option
_catch_errors
option
and they fail on any target, the plan itself fails. These functions include:upload_file
-
run_command
run_script
run_task
run_plan
If there is a plan failure due to an absent _catch_errors
option when using run_plan
, any calling plans also halt
until a run_plan
call with _catch_errors
or a catch_errors
block is
reached.
Failing a plan
If you are writing a plan and think it's failing, you can fail the plan with the fail_plan
function. This function fails the plan and prevents
calling plans from executing any further, unless run_plan
was called with _catch_errors
or in a catch_errors
block.
For example, use the fail_plan
function to pass an existing
error or create a new error with a message that includes the kind, details, or issue code.
fail_plan('The plan is failing', 'mymodules/pear-shaped', {'failednodes' => $result.error_set.names})
# or
fail_plan($errorobject)
Catching errors in plans
When you use the catch_errors
function, it executes a block of code and
returns any errors, or returns the result of the block if no errors are raised.
catch_errors
function.plan test (String[1] $role) {
$result_or_error = catch_errors(['pe/puppetdb-error']) || {
puppetdb_query("inventory[certname] { app_role == ${role} }")
}
$targets = if $result_or_error =~ Error {
# If the PuppetDB query fails
warning("Could not fetch from puppet. Using defaults instead")
# TargetSpec string
"all"
} else {
$result_or_error
}
}
If there is an error in a plan, it returns the Error
data
type, which includes:
-
msg
: The error message string. -
kind
: A string that defines the kind of error similar to an error class. -
details
: A hash with details about the error from a task or from information about the state of a plan when it fails, for example,exit_code
orstack_trace
. -
issue_code
: A unique code for the message that can be used for translation.
Use the Error
data type in a case expression to match
against different kinds of errors. To recover from certain errors and fail on others, set up
your plan to include conditionals based on errors that occur while your plan runs. For
example, you can set up a plan to retry a task when a timeout error occurs, but fail when
there is an authentication error.
mymodule/not-serious
error. Other errors cause the plan to fail.
plan mymodule::handle_errors {
$result = run_plan('mymodule::myplan', '_catch_errors' => true)
case $result {
Error['mymodule/not-serious'] : {
notice("${result.message}")
}
Error : { fail_plan($result) } }
run_plan('mymodule::plan2')
}
Puppet and Ruby functions in plans
You can package some common general logic in plans using Puppet language and Ruby functions;
however, some functions are not allowed. You can also call plan functions, such as run_task
or run_plan
, from within a
function.
These Puppet language constructs are not allowed in plans:
-
Defined types.
-
Classes.
-
Resource expressions, such as
file { title: mode => '0777' }
-
Resource default expressions, such as
File { mode => '0666' }
-
Resource overrides, such as
File['/tmp/foo'] { mode => '0444' }
-
Relationship operators:
-> <- ~> <~
-
Functions that operate on a catalog:
include
,require
,contain
,create_resources
. -
Collector expressions, such as
SomeType <| |>
,SomeType <<| |>>
-
ERB templates.
Additionally, there are some nuances of the Puppet language to keep in mind when writing plans:
-
The
--strict_variables
option is on, so if you reference a variable that is not set, you get an error. -
The
--strict=error
option is on, so minor language issues generate errors. For example{ a => 10, a => 20 }
is an error because there is a duplicate key in the hash. -
Most Puppet settings are empty and not configurable when using plans in PE.
-
Logs include "source location" (file, line) instead of resource type or name.
Handling plan function results
Each execution function, or a function you use to operate on one or more
targets, returns a ResultSet
. Each target you executed on
returns a Result
. The apply
action returns a ResultSet
containing ApplyResult
objects.
You can iterate on an instance of ResultSet
as if it were an
Array[Variant[Result, ApplyResult]]
. This means iterative
functions like each
, map
,
reduce
, or filter
work
directly on the ResultSet
returning each result.
ResultSet
may contain these functions:Function | Definition |
---|---|
names() |
Names all targets in the set as an Array . |
empty() |
Returns Boolean if the execution result set is
empty. |
count() |
Returns an Integer count of targets. |
first() |
Specifies the first Result object, useful to
unwrap single results. |
find(String $target_name) |
Specifies the Result for a specific
target. |
error_set() |
Returns a ResultSet containing only the results
of failed targets. |
ok_set() |
Returns a ResultSet containing only the
successful results. |
filter_set(block) |
Filters a ResultSet with the given block and returns a
ResultSet object (where the filter function returns an array or hash). |
targets() |
Specifies an array of all the Target objects
from every Result in the set. |
ok() |
Specifies a Boolean that is the same as error_set.empty . |
to_data() |
Returns an array of hashes representing either Result or
ApplyResults . |
Result
may contain these functions:Function | Definition |
---|---|
value() |
Specifies the hash containing the value of the Result . |
target() |
Specifies the Target object that the Result is from. |
error() |
Returns an Error object constructed from the
_error in the value. |
message() |
Specifies the _output key from the
value. |
ok() |
Returns true if the Result was successful. |
[] |
Accesses the value hash directly. |
to_data() |
Returns a hash representation of Result . |
action() |
Returns a string representation of result type (task, command, etc.). |
ApplyResult
may contain these functions. Function | Definition |
---|---|
report() |
Returns the hash containing the Puppet report from the application. |
target() |
Returns the Target object that the Result is from. |
error() |
Returnsn Error object constructed from the
_error in the value. |
ok() |
Returns true if the Result was successful. |
to_data() |
Returns a hash representation of ApplyResult . |
action() |
Returns a string representation of result type (apply). |
For example, to check if a task ran correctly on all targets, and the check fails if the task fails:
$r = run_task('sometask', ..., '_catch_errors' => true)
unless $r.ok {
fail("Running sometask failed on the targets ${r.error_set.names}")
}
You can do iteration and check if the result is an error. This example outputs feedback about the result of a task.
$r = run_task('sometask', ..., '_catch_errors' => true)
$r.each |$result| {
$target = $result.target.name
if $result.ok {
out::message("${target} returned a value: ${result.value}")
} else {
out::message("${target} errored with a message: ${result.error.message}")
}
}
to_data
on a ResultSet
and access hash values. For
example,$r = run_command('whoami')
$r.to_data.each |$result_hash| { notice($result_hash['result']['stdout']) }
filter_set
to filter a ResultSet
and apply a ResultSet
function such as
targets
to the output:
$filtered = $result.filter_set |$r| {
$r['tag'] == "you're it"
}.targets
Applying manifest blocks from a plan
You can apply manifest blocks, or chunks of Puppet
code, to remote systems during plan execution using the apply
and apply_prep
functions.
You can create manifest blocks that use existing content from the Forge, or use a plan to mix procedural orchestration and action with declarative resource configuration from a block. Most features of the Puppet language are available in a manifest block.
If your plan includes a manifest block, use the apply_prep
function in your plan before your manifest block. The
apply_prep
function syncs and caches plugins and
gathers facts by running Facter, making the facts available to the
manifest block.
apply_prep($target)
apply($target) { notify { foo: } }
apply
and apply_prep
only on targets connected via PCP.
apply
options
apply
function supports these options:Option | Default value | Description |
---|---|---|
_catch_errors |
true |
Returns a ResultSet, including failed results, rather than failing the plan. Boolean. |
_description |
none | Adds a description to the apply block. String. |
_noop |
true |
Applies the manifest block in no-operation mode, returning a report of changes it would make but does not take action. Boolean. |
# Preview installing docker as root on $targets.
apply($targets, _catch_errors => true, _noop => true) {
include 'docker'
}
How manifest blocks are applied
When you apply a manifest code from a plan, the manifest code and any facts generated for each target are sent to Puppet Server for compilation. During code compilation, variables are generated in the following order:
- Facts gathered from the targets set in your inventory.
- Local variables from the plan.
- Variables set in your inventory.
After a successful compilation, PE copies custom
module content from the module path and applies the catalog to each target. After
the catalog is executed on each target, apply
generates and returns a report about each target.
Return value
The apply
function returns a ResultSet object that contains an ApplyResult object for each target.
$results = apply($targets) { ... }
$results.each |$result| {
out::message($result.report)
}
Using Hiera data in a manifest block
plan do_thing() {
apply('node1.example.com') {
notice("Some data in Hiera: ${lookup('mydata')}")
}
}
Plan logging
You can view plan run information in log files or printed to a terminal session using
the out::message
function or built-in Puppet logging functions.
Outputting to the CLI or console
Use out::message
to display output from plans. This
function always prints message strings to STDOUT
regardless
of the log level and doesn't log them to the log file. When using out::message
in a plan, the messages are visible on the Plan
details page in the console.
Puppet log functions
In addition to out::message
, you can use Puppet logging functions. Puppet
logs messages to /var/log/puppetlabs/orchestration-services/orchestration-services.log
When using Puppet logging, each command's usual logging
level is downgraded by one level except for warn
and error
.
```
warning('logging text') - logs at warn level
err('logging text') - logs at error level
notice('logging text') - logs at info level
info('logging text') - logs at debug level
debug('logging text') - logs at trace level
```
The log level for orchestration-services.log
is configured
with normal levels. for more information about log levels for Bolt, see Puppet log functions in Bolt.
Default action logging
PE logs plan actions through the upload_file
, run_command
, run_script
, or run_task
functions. By default,
it logs an info level message when an action starts and another when it completes. You can
pass a description to the function to replace the generic log message.
run_task(my_task, $targets, "Better description", param1 => "val")
If your plan contains many small actions, you might want to suppress these messages and use
explicit calls to the Puppet log functions instead. To do
this, wrap actions in a without_default_logging
block, which
logs action messages at info
level instead of notice
.
For example, you can loop over a series of targets without logging each action.
plan deploy( TargetSpec $targets) {
without_default_logging() || {
get_targets($targets).each |$target| {
run_task(deploy, $target)
}
}
}
To avoid complications with parser ambiguity, always call without_default_logging
with ()
and empty block
args ||
.
Correct example
without_default_logging() || { run_command('echo hi', $targets) }
Incorrect example
without_default_logging { run_command('echo hi', $targets) }
Example plans
Check out some example plans for inspiration when writing your own.
Resource | Description | Level |
---|---|---|
facts module | Contains tasks and plans to discover facts about target systems. | Getting started |
facts plan | Gathers facts using the facts task and sets the facts in inventory. | Getting started |
facts::info plan | Uses the facts task to discover facts and map relevant fact values to targets. | Getting started |
reboot module | Contains tasks and plans for managing system reboots. | Intermediate |
reboot plan | Restarts a target system and waits for it to become available again. | Intermediate |
Introducing Masterless Puppet with Bolt | Blog post explaining how plans can be used to deploy a load-balanced web server. | Advanced |
profiles::nginx_install plan | Shows an example plan for deploying Nginx. | Advanced |
- Getting started resources show simple use cases such as running a task and manipulating the results.
- Intermediate resources show more advanced features in the plan language.
- Advanced resources show more complex use cases such as applying puppet code blocks and using external modules.