Writing plans in Puppet language

Writing plans in the Puppet language gives you better error handling and more sophisticated control than YAML plans. Plans written in the Puppet language also allow you to apply blocks of Puppet code to remote targets.

Naming plans

Name plans according to the module name, file name, and path to ensure code readability.

Place plan files in your module's ./plans directory, using these file extensions:
  • Puppet plans — .pp

  • YAML plans — .yaml, not .yml

Plan names are composed of two or more name segments, indicating:

  • The name of the module the plan is located in.

  • The name of the plan file, without the extension.

  • If the plan is in a subdirectory of ./plans, the path within the module.

For example, given a module called mymodule with a plan defined in ./mymodule/plans/myplan.pp, the plan name is mymodule::myplan.

A plan defined in ./mymodule/plans/service/myplan.pp would be mymodule::service::myplan. Use teh plan name to refer to the plan when you run commands.

The plan filename init is special because the plan it defines is referenced using the module name only. For example, in a module called mymodule, the plan defined in init.pp is the mymodule plan.

Avoid giving plans the same names as constructs in the Puppet language. Although plans don't share their namespace with other language constructs, giving plans these names makes your code difficult to read.

Each plan name segment:

  • Must begin with a lowercase letter.

  • Can include lowercase letters, digits, or underscores.
  • Must not be a reserved word.

  • Must not have the same name as any Puppet data types.

  • Namespace segments must match the regular expression \A[a-z][a-z0-9_]*\Z

Defining plan permissions

RBAC for plans is distinct from RBAC for individual tasks. This distinction means that a user can be excluded from running a certain task, but still have permission to run a plan that contains that task.

The RBAC structure for plans allows you to write plans with more robust, custom control over task permissions. Instead of allowing a user free rein to run a task that can potentially damage your infrastructure, you can wrap a task in a plan and only allow them to run it under circumstances you control.

For example, if you are configuring permissions for a new user to run plan infra::upgrade_git, you can allow them to run the package task but limit it to the git package only.

plan infra::upgrade_git (
  TargetSpec $targets,
  Integer $version,
) {
  run_task(‘package’, $targets, name => ’git’, action => ‘upgrade’, version => $version)
}

Use parameter types to fine-tune access

Parameter types provide another layer of control over user permissions. In the upgrade_git example above, the plan only provides access to the git package, but the user can choose whatever version of git they want. If there are known vulnerabilities in some versions of the git package, you can use parameter types like Enum to restrict the version parameter to versions that are safe enough for deployment.

For example, the Enum restricts the $version parameter to versions 1:2.17.0-1ubuntu1 and 1:2.17.1-1ubuntu0.4 only.
plan infra::upgrade_git (
  TargetSpec $targets,
  Enum['1:2.17.0-1ubuntu1', '1:2.17.1-1ubuntu0.4'] $version,
) {
  run_task(‘package’, $targets, name => ‘git’, action => ‘upgrade’, version => $version)
}

You can also use PuppetDB queries to select parameter types.

For example, if you need to restrict the targets that infra::upgrade_git can run on, use a PuppetDB query to identify which targets are selected for the git upgrade.
plan infra::upgrade_git (
   Enum['1:2.17.0-1ubuntu1', '1:2.17.1-1ubuntu0.4'] $version,
) {
  # Use puppetdb to find the nodes from the “other” team's web cluster
  $query = [from, nodes, ['=', [fact, cluster], "other_team"]]
  $selected_nodes = puppetdb_query($query).map() |$target| {
    $target[certname]
  }
  run_task(‘package’, $selected_nodes, name => ‘git’, action => ‘upgrade’, version => $version)
}

Specifying plan parameters

Specify plan parameters to do things like determine which targets to run different parts of your plan on. You can pass a parameter as a single target name, comma-separated list of target names, Target data type, or array. The target names can be either certnames or inventory node names.

The example plan below shows the target parameters $load_balancers and $webservers specified as data type TargetSpec. The plan then calls the run_task function to specify which targets to run the tasks on. The Target names are collected and stored in $webserver_names by iterating over the list of Target objects returned by get_targets. Task parameters are serialized to JSON format so that extracting the names into an array of strings ensures that the webservers parameter is in a format that can be converted to JSON.

plan mymodule::my_plan(
  TargetSpec $load_balancer,
  TargetSpec  $webservers,
) {

  # Extract the Target name from $webservers
  $webserver_names = get_targets($webservers).map |$n| { $n.name }
  
  # process webservers
  run_task('mymodule::lb_remove', $load_balancer, webservers => $webserver_names)
  run_task('mymodule::update_frontend_app', $webservers, version => '1.2.3')
  run_task('mymodule::lb_add', $load_balancer, webservers => $webserver_names)
 }

To execute this plan from the command line, pass the parameters as parameter=value. The Targetspec accepts either an array as JSON or a comma separated string of target names.

puppet plan run mymodule::myplan 
load_balancer=lb.myorg.com 
webservers='["kermit.myorg.com","gonzo.myorg.com"]'
Alternatively, here is an example of the same plan, run on the same targets, using the Orchestrator API POST /command/plan_run endpoint:
curl -k -X POST -H "Content-Type: application/json" \
-H "X-Authentication:$TOKEN" \
-d '{ "environment": "$ENV", "plan_name": "mymodule::myplan", \
"params": {"targets": "$TARGET_NAME", "load_balancer": "lb.myorg.com", \
"webservers": ["kermit.myorg.com", "gonzo.myorg.com"]} }' \
"https://$PRIMARY_HOST:8143/orchestrator/v1/command/plan_run"
Parameters that are passed to the run_* plan functions are serialized to JSON. For example, in the plan below, the default value of $example_nul is undef. The plan calls the test::demo_undef_bash with the example_nul parameter.
plan test::parameter_passing (
  TargetSpec $targets,
  Optional[String[1]] $example_nul = undef,
) {
  return run_task('test::demo_undef_bash', $targets, example_nul => $example_nul)
     }
The implementation of the demo_undef_bash.sh task is:
#!/bin/bash
example_env=$PT_example_nul
echo "Environment: $PT_example_nul"
echo "Stdin:" 
     cat -

By default, the task expects parameters passed as a JSON string on stdin to be accessible in prefixed environment variables.

Additionally, you can use the Orchestrator API POST /command/plan_run endpoint with token authentication, such as:
curl -k -X POST -H "Content-Type: application/json" -H 
"X-Authentication:$TOKEN" -d '{ "environment": "$ENV", 
"plan_name": "test::parameter_passing", "params": {"targets": 
"$TARGET_NAME"} }'
"https://$PRIMARY_HOST:8143/orchestrator/v1/command/plan_run"

Using Hiera data in plans

Use the lookup() function in plans to look up Hiera data. You can look up data inside or outside of apply blocks, or use the plan_hierarchy key to look up data both inside and outside apply blocks within the same plan.

Inside apply blocks, PE compiles catalogs for each target and has unlimited access to your Hiera data. You can use the same Hiera configuration, data, and lookup process as you do throughout PE.

Outside apply blocks, the plan executes a script, doesn't have a concept of a target or context, and cannot load per-target data. These limitations make some common Hiera features, like interpolating target facts, incompatible with plans in PE outside of apply blocks.

You can look up static Hiera data outside of apply blocks by adding a plan_hierarchy key to your Hiera configuration at the same level as the hierarchy key. This allows you to look up data inside and outside apply blocks in the same plan, enabling you to use your existing Hiera configuration in plans without encountering an error if per-target interpolations exist and your plan tries to look up data outside an apply block.

Static Hiera data is also useful for user-specific data that you want the plan to look up.

For example, consider the Hiera configuration below at <ENV_DIR>/hiera.yaml.
version: 5
hierarchy:
  - name: "Target specific data"
    path: "targets/%{trusted.certname}.yaml"
  - name: "Per-OS defaults"
    path: "os/%{facts.os.family}.yaml"
  - name: Common
    path: hierarchy.yaml
 
plan_hierarchy:
  - name: Common
    path: plan_hierarchy.yaml

You can set a user-specific API key in the plan_hierarchy.yaml data file, as well as use Hiera to look up a per-target filepath inside an apply block by using the following pieces of data:

Use the following data located at <ENV_DIR>/data/plan_hierarchy.yaml:
api_key: 12345
Use this data located at <ENV_DIR>/data/targets/myhost.com:
confpath: "C:\Program Files\Common Files\mytool.conf"
As a result, the plan looks up the API key in the first lookup() call, and the target-specific data inside the apply block:
plan plan_lookup(
  TargetSpec $targets
) {
  $outside_apply = lookup('api_key')
  run_task("make_request", $targets, 'api_key' => $outside_apply)
  $in_apply = apply($targets) {
    file { ${confpath}:
      ensure  => file,
      content => "setting: false"
    }
  }
}

Target objects

The Target object represents a target and its specific connection options.

The state of a target is stored in the code for the duration of a plan, allowing you to collect facts or set variables for a target and retrieve them later. Target objects must reference a target in the PE inventory. This includes targets connected via the PCP protocol that have puppet-agent installed, or targets in the PE inventory added with either SSH or WinRM credentials or as network devices. Target objects in PE do not have control over their connection information, and the connection info cannot be changed from within a plan.

Because target objects in PE are references, and cannot control their own configuration, accessing target connection info will return empty data.

TargetSpec

The TargetSpec type is a wrapper for defining targets that allows you to pass a target, or multiple targets, into a plan. Use TargetSpec for plans that accept a set of targets as a parameter to ensure clean interaction with the CLI and other plans.

TargetSpec accepts strings allowed by --targets, a single target object, or an array of targets and target patterns. To operate on an individual target, resolve the target to a list via get_targets.

For example, to loop over each target in a plan, accept a TargetSpec argument, but call get_targets on it before looping.

plan loop(TargetSpec $targets) {
  get_targets($targets).each |$target| {
    run_task('my_task', $target)
  }
}

Set variables and facts on targets

You can use the $target.facts() and $target.vars() functions to set transport configuration values, variables, and facts from a plan. Facts come from running facter or another fact collection application on the target, or from a fact store like PuppetDB. Variables are computed externally or assigned directly.

For example, set variables in a plan using $target.set_var:

plan vars(String $host) {
	$target = get_targets($host)[0]
	$target.set_var('newly_provisioned', true)
	$targetvars = $target.vars
	run_command("echo 'Vars for ${host}: ${$targetvars}'", $host)
}

Or set variables in the inventory file using the vars key at the group level.

groups:
  - name: my_targets
    targets:
      - localhost
    vars:
      operatingsystem: windows
    config:
      transport: ssh

Collect facts from targets

The facts plan connects to targets,discovers facts, and stores these facts on the targets.

The plan uses these methods to collect facts:

  • On ssh targets, it runs a Bash script.
  • On winrm targets, it runs a PowerShell script.
  • On pcp or targets where the Puppet agent is present, it runs Facter.
For example, use the facts plan to collect facts and then uses those facts to decide which task to run on the targets.
plan run_with_facts(TargetSpec $targets) {
  # This collects facts on targets and update the inventory
  run_plan(facts, targets => $targets)

  $centos_targets = get_targets($targets).filter |$n| { $n.facts['os']['name'] == 'CentOS' }
  $ubuntu_targets = get_targets($targets).filter |$n| { $n.facts['os']['name'] == 'Ubuntu' }
  run_task(centos_task, $centos_targets)
  run_task(ubuntu_task, $ubuntu_targets)
}

Collect facts from PuppetDB

You can use the puppetdb_fact plan to collect facts for targets when they are running a Puppet agent and sending facts to PuppetDB.

For example, use the puppetdb_fact plan to collect facts, and then use those facts to decide which task to run on the targets.

plan run_with_facts(TargetSpec $targets) {
  # This collects facts on targets and update the inventory
  run_plan(puppetdb_fact, targets => $targets)

  $centos_targets = get_targets($targets).filter |$n| { $n.facts['os']['name'] == 'CentOS' }
  $ubuntu_targets = get_targets($targets).filter |$n| { $n.facts['os']['name'] == 'Ubuntu' }
  run_task(centos_task, $centos_targets)
  run_task(ubuntu_task, $ubuntu_targets)
}

Collect general data from PuppetDB

You can use the puppetdb_query function in plans to make direct queries to PuppetDB.

For example, you can discover targets from PuppetDB and then run tasks on them. You must configure the PuppetDB client before running it. See the PQL tutorial to learn how to structure pql queries and see the PQL reference guide for query examples.
plan pdb_discover {
  $result = puppetdb_query("inventory[certname] { app_role == 'web_server' }")
  # extract the certnames into an array
  $names = $result.map |$r| { $r["certname"] }
  # wrap in url. You can skip this if the default transport is pcp
  $targets = $names.map |$n| { "pcp://${n}" }
  run_task('my_task', $targets)
}

Returning results from plans

Use the return function to return results that you can use in other plans or save for other uses.

Any plan that does not call the return function returns undef.

For example,

plan return_result(
  $targets
) {
  return run_task('mytask', $targets)
}

The result of a plan must match the PlanResult type alias. This includes JSON types as well as the plan language types, which have well defined JSON.

  • Undef
  • String
  • Numeric
  • Boolean
  • Target
  • ApplyResult
  • Result
  • ResultSet
  • Error
  • Array with only PlanResult
  • Hash with String keys and PlanResult values

or

Variant[Data, String, Numeric, Boolean, Error, Result, ResultSet, Target, Array[Boltlib::PlanResult], Hash[String, Boltlib::PlanResult]]

Plan errors and failure

Any plan that completes execution without an error is considered successful. There are some specific scenarios that always cause a plan failure, such as calling the fail_plan function.

Plan failure due to absent catch_errors option

If you call some functions without the _catch_errors option and they fail on any target, the plan itself fails. These functions include:
  • upload_file
  • run_command
  • run_script
  • run_task
  • run_plan

If there is a plan failure due to an absent _catch_errors option when using run_plan, any calling plans also halt until a run_plan call with _catch_errors or a catch_errors block is reached.

Failing a plan

If you are writing a plan and think it's failing, you can fail the plan with the fail_plan function. This function fails the plan and prevents calling plans from executing any further, unless run_plan was called with _catch_errors or in a catch_errors block.

For example, use the fail_plan function to pass an existing error or create a new error with a message that includes the kind, details, or issue code.

fail_plan('The plan is failing', 'mymodules/pear-shaped', {'failednodes' => $result.error_set.names})
# or
fail_plan($errorobject)

Catching errors in plans

When you use the catch_errors function, it executes a block of code and returns any errors, or returns the result of the block if no errors are raised.

Here is an example of the catch_errors function.
plan test (String[1] $role) {
  $result_or_error = catch_errors(['pe/puppetdb-error']) || {
    puppetdb_query("inventory[certname] { app_role == ${role} }")
  }
  $targets = if $result_or_error =~ Error {
    # If the PuppetDB query fails
    warning("Could not fetch from puppet. Using defaults instead")
    # TargetSpec string
    "all"
  } else {
    $result_or_error
  }
}

If there is an error in a plan, it returns the Error data type, which includes:

  • msg: The error message string.

  • kind: A string that defines the kind of error similar to an error class.

  • details: A hash with details about the error from a task or from information about the state of a plan when it fails, for example, exit_code or stack_trace.

  • issue_code: A unique code for the message that can be used for translation.

Use the Error data type in a case expression to match against different kinds of errors. To recover from certain errors and fail on others, set up your plan to include conditionals based on errors that occur while your plan runs. For example, you can set up a plan to retry a task when a timeout error occurs, but fail when there is an authentication error.

The first plan below continues whether it succeeds or fails with a mymodule/not-serious error. Other errors cause the plan to fail.
plan mymodule::handle_errors {
  $result = run_plan('mymodule::myplan', '_catch_errors' => true)
  case $result {
    Error['mymodule/not-serious'] : {
      notice("${result.message}")
    }
    Error : { fail_plan($result) } }
  run_plan('mymodule::plan2')
}

Puppet and Ruby functions in plans

You can package some common general logic in plans using Puppet language and Ruby functions; however, some functions are not allowed. You can also call plan functions, such as run_task or run_plan, from within a function.

These Puppet language constructs are not allowed in plans:

  • Defined types.

  • Classes.

  • Resource expressions, such as file { title: mode => '0777' }

  • Resource default expressions, such as File { mode => '0666' }

  • Resource overrides, such as File['/tmp/foo'] { mode => '0444' }

  • Relationship operators: -> <- ~> <~

  • Functions that operate on a catalog: include, require, contain, create_resources.

  • Collector expressions, such as SomeType <| |>, SomeType <<| |>>

  • ERB templates.

Additionally, there are some nuances of the Puppet language to keep in mind when writing plans:

  • The --strict_variables option is on, so if you reference a variable that is not set, you get an error.

  • The --strict=error option is on, so minor language issues generate errors. For example { a => 10, a => 20 } is an error because there is a duplicate key in the hash.

  • Most Puppet settings are empty and not configurable when using plans in PE.

  • Logs include "source location" (file, line) instead of resource type or name.

Handling plan function results

Each execution function, or a function you use to operate on one or more targets, returns a ResultSet. Each target you executed on returns a Result. The apply action returns a ResultSet containing ApplyResult objects.

You can iterate on an instance of ResultSet as if it were an Array[Variant[Result, ApplyResult]]. This means iterative functions like each, map, reduce, or filter work directly on the ResultSet returning each result.

A ResultSet may contain these functions:
Function Definition
names() Names all targets in the set as an Array.
empty() Returns Boolean if the execution result set is empty.
count() Returns an Integer count of targets.
first() Specifies the first Result object, useful to unwrap single results.
find(String $target_name) Specifies the Result for a specific target.
error_set() Returns a ResultSet containing only the results of failed targets.
ok_set() Returns a ResultSet containing only the successful results.
filter_set(block) Filters a ResultSet with the given block and returns a ResultSet object (where the filter function returns an array or hash).
targets() Specifies an array of all the Target objects from every Result in the set.
ok() Specifies a Boolean that is the same as error_set.empty.
to_data() Returns an array of hashes representing either Result or ApplyResults.
A Result may contain these functions:
Function Definition
value() Specifies the hash containing the value of the Result.
target() Specifies the Target object that the Result is from.
error() Returns an Error object constructed from the _error in the value.
message() Specifies the _output key from the value.
ok() Returns true if the Result was successful.
[] Accesses the value hash directly.
to_data() Returns a hash representation of Result.
action() Returns a string representation of result type (task, command, etc.).
An ApplyResult may contain these functions.
Function Definition
report() Returns the hash containing the Puppet report from the application.
target() Returns the Target object that the Result is from.
error() Returnsn Error object constructed from the _error in the value.
ok() Returns true if the Result was successful.
to_data() Returns a hash representation of ApplyResult.
action() Returns a string representation of result type (apply).

For example, to check if a task ran correctly on all targets, and the check fails if the task fails:

$r = run_task('sometask', ..., '_catch_errors' => true)
unless $r.ok {
  fail("Running sometask failed on the targets ${r.error_set.names}")
}

You can do iteration and check if the result is an error. This example outputs feedback about the result of a task.

$r = run_task('sometask', ..., '_catch_errors' => true)
$r.each |$result| {
  $target = $result.target.name
  if $result.ok {
    out::message("${target} returned a value: ${result.value}")
  } else {
    out::message("${target} errored with a message: ${result.error.message}")
  }
}
Similarly, you can iterate over the array of hashes returned by calling to_data on a ResultSet and access hash values. For example,
$r = run_command('whoami')
$r.to_data.each |$result_hash| { notice($result_hash['result']['stdout']) } 
You can also use filter_set to filter a ResultSet and apply a ResultSet function such as targets to the output:
$filtered = $result.filter_set |$r| {
  $r['tag'] == "you're it"
}.targets

Applying manifest blocks from a plan

You can apply manifest blocks, or chunks of Puppet code, to remote systems during plan execution using the apply and apply_prep functions.

You can create manifest blocks that use existing content from the Forge, or use a plan to mix procedural orchestration and action with declarative resource configuration from a block. Most features of the Puppet language are available in a manifest block.

If your plan includes a manifest block, use the apply_prep function in your plan before your manifest block. The apply_prep function syncs and caches plugins and gathers facts by running Facter, making the facts available to the manifest block.

For example:
apply_prep($target)
apply($target) { notify { foo: } }
Note: You can use apply and apply_prep only on targets connected via PCP.

apply options

The apply function supports these options:
Option Default value Description
_catch_errors true Returns a ResultSet, including failed results, rather than failing the plan. Boolean.
_description none Adds a description to the apply block. String.
_noop true Applies the manifest block in no-operation mode, returning a report of changes it would make but does not take action. Boolean.
For example,
# Preview installing docker as root on $targets.
apply($targets, _catch_errors => true, _noop => true) {
    include 'docker'
}

How manifest blocks are applied

When you apply a manifest code from a plan, the manifest code and any facts generated for each target are sent to Puppet Server for compilation. During code compilation, variables are generated in the following order:

  1. Facts gathered from the targets set in your inventory.
  2. Local variables from the plan.
  3. Variables set in your inventory.

After a successful compilation, PE copies custom module content from the module path and applies the catalog to each target. After the catalog is executed on each target, apply generates and returns a report about each target.

Return value

The apply function returns a ResultSet object that contains an ApplyResult object for each target.

For example:
$results = apply($targets) { ... }
$results.each |$result| {
   out::message($result.report)
}

Using Hiera data in a manifest block

Hiera is a key-value configuration data look up system, used for separating data from Puppet code. Use Hiera data to implicitly override default class parameters. You can also explicitly look up data from Hiera via the lookup function.
Note: Plans in PE currently only support Hiera version 5.
For example:
plan do_thing() {
  apply('node1.example.com') {
    notice("Some data in Hiera: ${lookup('mydata')}")
  }
}

Plan logging

You can view plan run information in log files or printed to a terminal session using the out::message function or built-in Puppet logging functions.

Outputting to the CLI or console

Use out::message to display output from plans. This function always prints message strings to STDOUT regardless of the log level and doesn't log them to the log file. When using out::message in a plan, the messages are visible on the Plan details page in the console.

Puppet log functions

In addition to out::message, you can use Puppet logging functions. Puppet logs messages to /var/log/puppetlabs/orchestration-services/orchestration-services.log

When using Puppet logging, each command's usual logging level is downgraded by one level except for warn and error.

For example, here are the Puppet logging commands with their actual level when used in plans.
```
  warning('logging text') - logs at warn level
  err('logging text') - logs at error level

  notice('logging text') - logs at info level
  info('logging text') - logs at debug level
  debug('logging text') - logs at trace level
```

The log level for orchestration-services.log is configured with normal levels. for more information about log levels for Bolt, see Puppet log functions in Bolt.

Default action logging

PE logs plan actions through the upload_file, run_command, run_script, or run_task functions. By default, it logs an info level message when an action starts and another when it completes. You can pass a description to the function to replace the generic log message.

run_task(my_task, $targets, "Better description", param1 => "val")

If your plan contains many small actions, you might want to suppress these messages and use explicit calls to the Puppet log functions instead. To do this, wrap actions in a without_default_logging block, which logs action messages at info level instead of notice.

For example, you can loop over a series of targets without logging each action.

plan deploy( TargetSpec $targets) {
  without_default_logging() || {
    get_targets($targets).each |$target| {
      run_task(deploy, $target)
    }
  }
}

To avoid complications with parser ambiguity, always call without_default_loggingwith () and empty block args ||.

Correct example

without_default_logging() || { run_command('echo hi', $targets) }

Incorrect example

without_default_logging { run_command('echo hi', $targets) }

Example plans

Check out some example plans for inspiration when writing your own.

Resource Description Level
facts module Contains tasks and plans to discover facts about target systems. Getting started
facts plan Gathers facts using the facts task and sets the facts in inventory. Getting started
facts::info plan Uses the facts task to discover facts and map relevant fact values to targets. Getting started
reboot module Contains tasks and plans for managing system reboots. Intermediate
reboot plan Restarts a target system and waits for it to become available again. Intermediate
Introducing Masterless Puppet with Bolt Blog post explaining how plans can be used to deploy a load-balanced web server. Advanced
profiles::nginx_install plan Shows an example plan for deploying Nginx. Advanced
  • Getting started resources show simple use cases such as running a task and manipulating the results.
  • Intermediate resources show more advanced features in the plan language.
  • Advanced resources show more complex use cases such as applying puppet code blocks and using external modules.