Upgrading Puppet Enterprise

Upgrade your PE installation as new versions become available.

Upgrade paths

These are the valid upgrade paths for PE.


If you're on version...	Upgrade to...	Notes
2021.7.2 (LTS)	You're up to date!	If you're on the most recent LTS release, you can stay here or upgrade to the STS.
2019.8.z or any 2021.y	2021.7.z (LTS)	Before upgrading to 2021.7.z, you might want to read about What's new since PE 2019.8.
2019.y	2019.8.12
2018.1.2 or later 2018.1.3 or later with disaster recovery	2019.8.z	You must have version 2018.1.2 or later in order to complete prerequisites for upgrade to 2019.8.z. With disaster recovery enabled, you must have version 2018.1.3 in order to upgrade to 2019.8.z. Alternatively, you can forget and then recreate your replica after upgrade.
2018.1.0 or 2018.1.1	2018.1.z

Upgrade cautions

These are the major changes to PE since the previous long-term support release, 2019.8. Review these recommendations and plan accordingly before upgrading to this version.

FIPS-enabled PE 2021.7 can't use the default system cert store

PE 2021.7 FIPS builds can't use the default system cert store, which is used automatically with some reporting services. This setting is configured by the report_include_system_store Puppet parameter that ships with PE.

Removing the puppet-cacerts file (located at /opt/puppetlabs/puppet/ssl/puppet-cacerts) can allow a report processor that eagerly loads the system store to continue with a warning that the file is missing.

If HTTP clients require external certs, we recommend using a custom cert store containing only the necessary certs. You can create this cert store by concatenating existing pem files and configuring the ssl_trust_store Puppet parameter to point to the new cert store.

PostgreSQL 14 upgrade in PE 2021.6

PE 2021.6.0 upgrades pe-postgresql from version 11 to version 14. This upgrade involves a datastore migration that requires extra disk space (110% of the current PostgreSQL 11 datastore) and extra time to upgrade (roughly two to four minutes of additional time per 10GB of datastore size). The installer issues a warning and cancels the upgrade if there is insufficient space.

To check your PostgreSQL installation size and the size and number of bytes available for the partition, run facter -p pe_postgresql_info on the node that runs the pe-postgresql service (this is either your primary server or a separate node that manages your PostgreSQL database).

To speed the migration and optimize queries, clean up the PE-PostgreSQL database prior to upgrading by applying the pe_databases module to nodes running the pe-postgresql service. This module is installed with PE versions 2021.3 and later, but you must enable it by setting the puppet_enterprise::enable_database_maintenance parameter to true. For best results, apply the module at least one week prior to upgrade to allow the module's maintenance schedule enough time to clean all databases.

Optionally, after upgrading, you can remove packages and directories from older PostgreSQL versions by running puppet infrastructure run remove_old_postgresql_versions. If applicable, the installer prompts you to complete the cleanup.

CAUTION: Don't use CONCURRENTLY with PostgreSQL 14.0 through 14.3. If you do, make sure to REINDEX without using CONCURRENTLY.

Restriction: If you use a PostgreSQL instance not managed by PE, you must separately upgrade the instance to PostgreSQL 14. For more information, see Upgrade an unmanaged PostgreSQL installation.

Platforms removed in 2021.0

Several agent platforms that were previously deprecated have been removed in PE 2021.0.

Before upgrading to this version, remove the pe_repo::platform class for these operating systems from the PE Master node group in the console, and from your code and Hiera.

AIX 6.1
Enterprise Linux 4
Enterprise Linux 6, 7 s390x
Fedora 26, 27, 28, 29
Mac OS X 10.9, 10.12, 10.13
SUSE Linux Enterprise Server 11, 12 s390x

Puppet upgrade in 2021.0

PE 2021.0 includes a new major version of Puppet, with several changes that might impact your upgrade. For details, see Upgrading from Puppet 6 to Puppet 7.

Test modules before upgrade

To ensure that your modules work with the newest version of PE, update and test them with Puppet Development Kit (PDK) before upgrading.

Before you begin

If you are already using PDK, your modules should pass validation and unit tests with your currently installed version of PDK.

Update PDK with each new release to ensure compatability with new versions of PE.

Download and install PDK. If you already have PDK installed, this updates PDK to its latest version. For detailed instructions and download links, see the installing instructions.
If you have not previously used PDK with your modules, convert them to a PDK compatible format. This makes changes to your module to enable validation and unit testing with PDK. For important usage details, see the converting modules documentation.
For example, from within the module directory, run:
```
pdk convert
```
If your modules are already compatible with PDK, update them to the latest module template. If you converted modules in step 2, you do not need to update the template. To learn more about updating, see the updating module templates documentation.
For example, from within the module directory, run:
```
pdk update
```
Validate and run unit tests for each module, specifying the version of PE you are upgrading to. When specifying a PE version, be sure to specify at least the year and the release number, such as 2018.1. For information about module validations and testing, see the validating and testing modules documentation.
For example, from within the module directory, run:
```
pdk validate
pdk test unit
```
The pdk test unit command verifies that testing dependencies and directories are present and runs the unit tests that you write. It does not create unit tests for your module.
If your module fails validation or unit tests, make any necessary changes to your code.

Results

After you've verified that your modules work with the new PE version, you can continue with your upgrade.

Upgrade PE

Upgrade PE infrastructure components to get the latest features and fixes. Follow the upgrade instructions for your installation type to ensure you upgrade components in the correct order. Coordinate upgrades to ensure all infrastructure nodes are upgraded in a timely manner, because agent runs and replication fail on infrastructure nodes running a different agent version than the primary server.

Before you begin

Review the upgrade cautions for major changes to architecture and infrastructure components which might affect your upgrade.

Configure non-production environment for infrastructure nodes

If your infrastructure nodes are in an environment other than production, you must manually configure PE to use your chosen environment before you upgrade.

Important: You only need to follow these steps if your infrastructure nodes are in an environment that is not production.

In pe.conf, set both of these parameters to the environment your infrastructure nodes are in:
- pe_install::install::classification::pe_node_group_environment
- puppet_enterprise::master::recover_configuration::pe_environment
Run puppet-enterprise-installer -- --pe-environment <env>

Upgrade a standard installation

To upgrade a standard installation, run the PE installer on your primary server, and then upgrade any additional components.

Before you begin

Back up your PE installation.

If you're upgrading a replica, ensure you have a valid admin RBAC token.

In Hiera, pe.conf, or the console (in the PE Master node group), remove any agent_version parameters you set in the pe_repo class that matches your infrastructure nodes. This ensures the upgrade isn't blocked by attempting to download non-default agent versions for your infrastructure OS and architecture.

Download the tarball for your operating system and architecture. Optionally, you can Verify the installation package.
Run tar -xf <TARBALL> to unpack the installation tarball.
You need about 1 GB of space to untar the installer.
From the installer directory on your primary server, run sudo ./puppet-enterprise-installer to start the installer, and then follow the CLI instructions to complete your server upgrade.
To specify a different pe.conf file than the existing file, use the -c flag, such as:
```
sudo ./puppet-enterprise-installer -c <FULL_PATH_TO_pe.conf>
```
This flag tells the installer to backup the previous pe.conf file to /etc/puppetlabs/enterprise/conf.d/<TIMESTAMP>.conf and create a new pe.conf file at /etc/puppetlabs/enterprise/conf.d/pe.conf.
Upgrade these additional PE infrastructure components:
- Agents
- PE client tools: On unmanaged nodes, you must re-install the client tools version that matches the PE version you upgraded to. On managed nodes and infrastructure nodes, client tools are automatically updated when you upgrade PE.
In disaster recovery installations, upgrade your replica.
The replica is temporarily unavailable to serve as backup during this step, so time upgrading your replica to minimize risk.
1. On your primary server logged in as root, run:
```
sudo puppet infrastructure upgrade replica <REPLICA_FQDN>
```
  If you want to specify an authentication token other than the default, run:
```
sudo puppet infrastructure upgrade replica <REPLICA_FQDN> --token-file <PATH_TO_TOKEN>
```
2. After the replica upgrade successfully completes, verify that primary and replica services are operational. On your primary server, run:
```
sudo /opt/puppetlabs/bin/puppet-infra status
```
3. If your replica reports errors, reinitialize the replica. On your replica, run:
```
sudo /opt/puppetlabs/bin/puppet-infra reinitialize replica -y
```
Optional: Remove old PE packages from all infrastructure nodes. On your primary server, run: puppet infrastructure run remove_old_pe_packages
All packages older than the current version are removed by default. To remove specific versions, append pe_version=<VERSION_NUMBER> to the command.

Upgrade a large installation

To upgrade a large installation, run the PE installer on your primary server, and then upgrade compilers and any additional components.

Before you begin

Back up your PE installation.

Ensure you have a valid admin RBAC token in order to upgrade compilers or a replica.

Download the tarball for your operating system and architecture. Optionally, you can Verify the installation package.
Run tar -xf <TARBALL> to unpack the installation tarball.
You need about 1 GB of space to untar the installer.
From the installer directory on your primary server, run sudo ./puppet-enterprise-installer to start the installer, and then follow the CLI instructions to complete your server upgrade.
To specify a different pe.conf file than the existing file, use the -c flag, such as:
```
sudo ./puppet-enterprise-installer -c <FULL_PATH_TO_pe.conf>
```
This flag tells the installer to backup the previous pe.conf file to /etc/puppetlabs/enterprise/conf.d/<TIMESTAMP>.conf and create a new pe.conf file at /etc/puppetlabs/enterprise/conf.d/pe.conf.
To upgrade compilers, log in to your primary server as root and run one of these commands:
- To upgrade specific compilers, run:
```
sudo puppet infrastructure upgrade compiler <COMPILER_FQDN-1>,<COMPILER_FQDN-2>
```
- To upgrade all compilers simultaneously, run:
```
sudo puppet infrastructure upgrade compiler --all
```
- To specify an authentication token location other than the default location, include --token-file <PATH_TO_TOKEN> in the command, such as:
```
sudo puppet infrastructure upgrade compiler <COMPILER_FQDN> --token-file <PATH_TO_TOKEN>
```
Upgrade these additional PE infrastructure components:
- Agents
- PE client tools: On unmanaged nodes, you must re-install the client tools version that matches the PE version you upgraded to. On managed nodes and infrastructure nodes, client tools are automatically updated when you upgrade PE.
In disaster recovery installations, upgrade your replica.
The replica is temporarily unavailable to serve as backup during this step, so time upgrading your replica to minimize risk.
1. On your primary server logged in as root, run:
```
sudo puppet infrastructure upgrade replica <REPLICA_FQDN>
```
  If you want to specify an authentication token other than the default, run:
```
sudo puppet infrastructure upgrade replica <REPLICA_FQDN> --token-file <PATH_TO_TOKEN>
```
2. After the replica upgrade successfully completes, verify that primary and replica services are operational. On your primary server, run:
```
sudo /opt/puppetlabs/bin/puppet-infra status
```
3. If your replica reports errors, reinitialize the replica. On your replica, run:
```
sudo /opt/puppetlabs/bin/puppet-infra reinitialize replica -y
```
Optional: Remove old PE packages from all infrastructure nodes. On your primary server, run: puppet infrastructure run remove_old_pe_packages
All packages older than the current version are removed by default. To remove specific versions, append pe_version=<VERSION_NUMBER> to the command.

What to do next

Optionally convert legacy compilers to the new style compiler running the PuppetDB service.

Upgrade an extra-large installation

You can use the PEADM module to upgrade extra-large installations. Contact your technical account manager for additional support.

Upgrade a standalone PE-PostgreSQL installation

To upgrade a large installation with standalone PE-PostgreSQL, run the PE installer first on your PE-PostgreSQL node, then on your primary server, and then upgrade any additional components.

Before you begin

Back up your PE installation.

Ensure you have a valid admin RBAC token in order to upgrade compilers.

Download the tarball for your operating system and architecture. Optionally, you can Verify the installation package.
Run tar -xf <TARBALL> to unpack the installation tarball.
You need about 1 GB of space to untar the installer.
Upgrade your PostgreSQL node.
1. Ensure that the user_data.conf file on your PostgreSQL node is up to date by running puppet infrastructure recover_configuration on your primary server, and then copying /etc/puppetlabs/enterprise/conf.d to the PostgreSQL node.
2. Copy the installation tarball to the PostgreSQL node, and from the installer directory, run the installer: sudo ./puppet-enterprise-installer
From the installer directory on your primary server, run sudo ./puppet-enterprise-installer to start the installer, and then follow the CLI instructions to complete your server upgrade.
To specify a different pe.conf file than the existing file, use the -c flag, such as:
```
sudo ./puppet-enterprise-installer -c <FULL_PATH_TO_pe.conf>
```
This flag tells the installer to backup the previous pe.conf file to /etc/puppetlabs/enterprise/conf.d/<TIMESTAMP>.conf and create a new pe.conf file at /etc/puppetlabs/enterprise/conf.d/pe.conf.
To upgrade compilers, log in to your primary server as root and run one of these commands:
- To upgrade specific compilers, run:
```
sudo puppet infrastructure upgrade compiler <COMPILER_FQDN-1>,<COMPILER_FQDN-2>
```
- To upgrade all compilers simultaneously, run:
```
sudo puppet infrastructure upgrade compiler --all
```
- To specify an authentication token location other than the default location, include --token-file <PATH_TO_TOKEN> in the command, such as:
```
sudo puppet infrastructure upgrade compiler <COMPILER_FQDN> --token-file <PATH_TO_TOKEN>
```
Upgrade these additional PE infrastructure components:
- Agents
- PE client tools: On unmanaged nodes, you must re-install the client tools version that matches the PE version you upgraded to. On managed nodes and infrastructure nodes, client tools are automatically updated when you upgrade PE.
Optional: Remove old PE packages from all infrastructure nodes. On your primary server, run: puppet infrastructure run remove_old_pe_packages
All packages older than the current version are removed by default. To remove specific versions, append pe_version=<VERSION_NUMBER> to the command.

What to do next

Optionally convert legacy compilers to the new style compiler running the PuppetDB service.

Upgrade an unmanaged PostgreSQL installation

To upgrade a Puppet Enterprise (PE) installation that relies on an unmanaged PostgreSQL database, you must upgrade PostgreSQL to version 14 and ensure that your PostgreSQL database roles, privileges, extensions, and configuration are up to date with the most recent version of PE. Then, proceed with the PE upgrade.

Before you begin

Back up your PE installation, and, if desired, back up your PostgreSQL databases.

Ensure that you have a valid role-based access control (RBAC) admin token to upgrade compilers (for example, Generate a token using puppet-access).

If you are upgrading from a 2019.8 version of PE, ensure that you have upgraded to the latest 2019.8.z version. As part of the upgrade to 2019.8.z, you must have followed the unmanaged PostgreSQL upgrade instructions for that 2019.8 version of PE before you begin the upgrade to 2021.

If you are upgrading from a 2021 version of PE that is earlier than 2021.2, add the PuppetDB read_user. To add the read_user, run the commands that correspond to your authentication method to generate the role and authentication, and then run the grant commands:

Password authentication: Run the following command and ensure that you specify a password:
```
CREATE ROLE "pe-puppetdb-read"  
LOGIN NOCREATEROLE NOCREATEDB NOSUPERUSER  CONNECTION LIMIT -1 PASSWORD '<PUPPETDB_USER_PASSWORD>';
```

Certificate authentication: Create the database user by adding the following lines to the pg_hba.conf file:

hostssl pe-puppetdb pe-puppetdb-read 0.0.0.0/0 
cert map=pe-puppetdb-pe-puppetdb-read-map clientcert=verify-full 
hostssl pe-puppetdb pe-puppetdb-read ::/0 cert map=pe-puppetdb-pe-puppetdb-read-map clientcert=verify-full

Map the database user role by adding the following line to the pg_ident.conf file:

pe-puppetdb-pe-puppetdb-read-map <PRIMARY_CERTNAME> pe-puppetdb-read

Create the role by running the following command. Use the psql tool:

CREATE ROLE "pe-puppetdb-read"  LOGIN NOCREATEROLE NOCREATEDB NOSUPERUSER  CONNECTION LIMIT –1;

After you complete either of the previous steps, finish granting permissions to the new role by running the following commands:

GRANT CONNECT ON DATABASE "pe-puppetdb" TO "pe-puppetdb-read"; 
ALTER DEFAULT PRIVILEGES FOR USER "pe-puppetdb" IN SCHEMA "public" GRANT SELECT ON TABLES TO "pe-puppetdb-read"; 
ALTER DEFAULT PRIVILEGES FOR USER "pe-puppetdb" IN SCHEMA "public" GRANT EXECUTE ON FUNCTIONS TO "pe-puppetdb-read"; 
ALTER DEFAULT PRIVILEGES FOR USER "pe-puppetdb" IN SCHEMA "public" GRANT USAGE ON SEQUENCES TO "pe-puppetdb-read"; 
GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO "pe-puppetdb-read"; 
GRANT USAGE ON ALL SEQUENCES IN SCHEMA "public" TO "pe-puppetdb-read";

If you are upgrading from a 2021 version of PE that is earlier than 2021.4, add the pgcrypto extension to the Puppet orchestrator database. Run the following commands by using the psql tool:
```
\c "pe-orchestrator" 
 CREATE EXTENSION IF NOT EXISTS "pgcrypto"; 
```
If you are upgrading from any PE version earlier than 2021.6, upgrade PostgreSQL to version 14:
1. On your PostgreSQL node, install these packages:
  - postgresql14
  - postgresql14-libs
  - postgresql14-contrib
  - postgresql14-server
2. Stop the PostgreSQL 11 service:
```
systemctl stop postgresql-11.service
```
3. As the PostgreSQL user, create a database:
```
su - postgres /usr/pgsql-14/bin/pg_ctl -D /var/lib/pgsql/14/data/ initdb
```
4. If you use certificate authentication, note the location of your certificate, private key, and certificate authority files. You might have to move these files to the new installation directory, and you might have to update the postgresql.conf file.
5. Copy your postgresql.conf file from the /var/lib/pgsql/11/data/ directory to the /var/lib/pgsql/14/data/ directory and update the data directory setting. If you moved the certificate, private key, and certificate authority files, update the ssl_cert_file, ssl_key_file, and ssl_ca_file settings.
6. Migrate databases to PostgreSQL 14:
```
/usr/pgsql-14/bin/pg_upgrade -b /usr/pgsql-11/bin -B /usr/pgsql-14/bin/ -d /var/lib/pgsql/11/data -D /var/lib/pgsql/14/data/
```
  Tip: This command uses pg_upgrade to upgrade and migrate your unmanaged PostgreSQL instance from version 11 to 14. If you don't want to use this command, you can back up your PostgreSQL 11 databases, manually wipe the PostgreSQL 11 installation, ensure that PostgreSQL 14 is installed, and then restore the databases to the PostgreSQL 14 installation. Or, you can back up the PostgreSQL 11 databases, set up a new node with PostgreSQL 14, restore databases to the new node, and then reconfigure PE to point to the new database_host node.
7. Copy your pg_hba.conf and pg_ident.conf files from the /var/lib/pgsql/11/data/ directory to the /var/lib/pgsql/14/data/ directory.
8. In the pg_hba.conf file, change all instances of clientcert=1 to clientcert=verify-full.
Start the PostgreSQL 14 service:
```
systemctl start postgresql-14
```
Optional: As the PostgreSQL user, clean up the database:
```
su - postgres /usr/pgsql-14/bin/vacuumdb --analyze --all
```
If you are upgrading from a version of PE earlier than 2021.7.2, ensure that the pe-puppetdb-read role is granted connect by the pe-puppetdb-migrator role.
This step is necessary so that the pe-puppetdb-migrator can revoke query connections during migrations. Additionally, the chain of pe-puppetdb-read role granted to pe-puppetdb (and pe-puppetdb granted to pe-puppetdb-migrator, already present in the database schema) makes it possible for the pe-puppetdb and pe-puppetdb-migrator roles to kill queries during migration and garbage collection.

In the PostgreSQL cluster, run the following commands:
```
REVOKE CONNECT ON DATABASE "pe-puppetdb" FROM "pe-puppetdb-read"; 
SET ROLE "pe-puppetdb-migrator"; 
GRANT CONNECT ON DATABASE "pe-puppetdb" TO "pe-puppetdb-read"; 
SET ROLE "postgres"; 
GRANT "pe-puppetdb-read" TO "pe-puppetdb"
```
Download the tarball for your operating system and architecture. Optionally, you can Verify the installation package.
Run tar -xf <TARBALL> to unpack the installation tarball.
You need about 1 GB of space to untar the installer.
From the installer directory on your primary server, run sudo ./puppet-enterprise-installer -s to start the installer, and then follow the CLI instructions to complete the server upgrade.
- The -s flag tells the installer to skip database checks.
- To specify a different pe.conf file than the existing file, use the -c flag as shown here:
```
sudo ./puppet-enterprise-installer -c <FULL_PATH_TO_pe.conf>
```
  This flag tells the installer to back up the previous pe.conf file to /etc/puppetlabs/enterprise/conf.d/<TIMESTAMP>.conf and create a new pe.conf file at /etc/puppetlabs/enterprise/conf.d/pe.conf.
To upgrade compilers, log in to your primary server as root and run one of these commands:
- To upgrade specific compilers, run:
```
sudo puppet infrastructure upgrade compiler <COMPILER_FQDN-1>,<COMPILER_FQDN-2>
```
- To upgrade all compilers simultaneously, run:
```
sudo puppet infrastructure upgrade compiler --all
```
- To specify an authentication token location other than the default location, include --token-file <PATH_TO_TOKEN> in the command, such as:
```
sudo puppet infrastructure upgrade compiler <COMPILER_FQDN> --token-file <PATH_TO_TOKEN>
```
Upgrade these additional PE infrastructure components:
- Agents
- PE client tools: On unmanaged nodes, you must re-install the client tools version that matches the PE version you upgraded to. On managed nodes and infrastructure nodes, client tools are automatically updated when you upgrade PE.
Optional: Remove old PE packages from all infrastructure nodes. On your primary server, run: puppet infrastructure run remove_old_pe_packages
All packages older than the current version are removed by default. To remove specific versions, append pe_version=<VERSION_NUMBER> to the command.

What to do next

Optionally, convert existing compilers to the new style compiler running the PuppetDB service.

CAUTION: Don't use CONCURRENTLY with PostgreSQL 14.0 through 14.3. If you do, ensure that you REINDEX without using CONCURRENTLY.

Migrate PE

As an alternative to upgrading, you can migrate your PE installation. Migrating results in little or no downtime, but it requires additional system resources because you must configure a new primary server.

Restriction: Currently, we only recommend migration for standard installations without disaster recovery.

For standard installations with disaster recovery, follow the steps to Upgrade a standard installation.

For large installations with or without disaster recovery, follow the steps to Upgrade a large installation.

For all extra-large installations or any size installation where either upgrading or migrating is not possible, contact your technical account manager.

Migrate a standard installation

Migrate a standard installation by standing up a new primary server, restoring it with your existing installation, upgrading it, and then pointing agents at the new primary.

Before you begin

Review the upgrade cautions for major changes to architecture and infrastructure components which might affect your upgrade.

Install your current PE version on a new primary server node.
Back up your existing installation.
Restore your installation on the new primary server using the backup you created.
Upgrade your new primary server to the latest PE version.
Update nodes' puppet.conf server setting to the hostname for your new primary server:
```
sudo puppet config set server <PRIMARY_HOSTNAME>
```

Was this page helpful?

We’re sorry to hear that!
Please tell us why so we can help. Enter your feedback and email. This form is sent to the Puppet docs team. We ask for your email as we might contact you regarding your feedback. If you need help with the product itself, visit Puppet Support or ask in Puppet Community on Slack. Feedback:

Email Address:

To learn about how Puppet uses your personal information, visit our privacy policy.

If you leave us your email, we may contact you regarding your feedback. For more information on how Puppet uses your personal information, see our privacy policy.

See an issue? Please file a JIRA ticket in our [DOCUMENTATION] project.