Upgrade PE using the installer tarball

Upgrade PE infrastructure components to get the latest features and fixes. Follow the upgrade instructions for your installation type to ensure you upgrade components in the correct order. Coordinate upgrades to ensure all infrastructure nodes are upgraded in a timely manner, because agent runs and replication fail on infrastructure nodes running a different agent version than the primary server.

Before you begin

Review the upgrade cautions for major changes to architecture and infrastructure components which might affect your upgrade.

Configure non-production environment for infrastructure nodes

If your infrastructure nodes are in an environment other than production, you must manually configure PE to use your chosen environment before you upgrade.

Important: You only need to follow these steps if your infrastructure nodes are in an environment that is not production.
In pe.conf, set both of these parameters to the environment your infrastructure nodes are in:
  • pe_install::install::classification::pe_node_group_environment
  • puppet_enterprise::master::recover_configuration::pe_environment

Upgrade a standard installation

To upgrade a standard installation, run the PE installer on your primary server, and then upgrade any additional components.

Before you begin

Back up your PE installation.

If you're upgrading a replica, ensure you have a valid admin RBAC token.

In Hiera, pe.conf, or the console (in the PE Master node group), remove any agent_version parameters you set in the pe_repo class that matches your infrastructure nodes. This ensures the upgrade isn't blocked by attempting to download non-default agent versions for your infrastructure OS and architecture.

  1. Download the tarball for your operating system and architecture. Optionally, you can Verify the installation package.
  2. Run tar -xf <TARBALL> to unpack the installation tarball.
    You need about 1 GB of space to untar the installer.
  3. From the installer directory on your primary server, run sudo ./puppet-enterprise-installer to start the installer, and then follow the CLI instructions to complete your server upgrade.
    To specify a different pe.conf file than the existing file, use the -c flag, such as:
    sudo ./puppet-enterprise-installer -c <FULL_PATH_TO_pe.conf>
    This flag tells the installer to backup the previous pe.conf file to /etc/puppetlabs/enterprise/conf.d/<TIMESTAMP>.conf and create a new pe.conf file at /etc/puppetlabs/enterprise/conf.d/pe.conf.
  4. Upgrade these additional PE infrastructure components:
    • Agents
    • PE client tools: On unmanaged nodes, you must re-install the client tools version that matches the PE version you upgraded to. On managed nodes and infrastructure nodes, client tools are automatically updated when you upgrade PE.
  5. In disaster recovery installations, upgrade your replica.
    The replica is temporarily unavailable to serve as backup during this step, so time upgrading your replica to minimize risk.
    1. On your primary server logged in as root, run:
      sudo puppet infrastructure upgrade replica <REPLICA_FQDN>
      If you want to specify an authentication token other than the default, run:
      sudo puppet infrastructure upgrade replica <REPLICA_FQDN> --token-file <PATH_TO_TOKEN>
    2. After the replica upgrade successfully completes, verify that primary and replica services are operational. On your primary server, run:
      sudo /opt/puppetlabs/bin/puppet-infra status
    3. If your replica reports errors, reinitialize the replica. On your replica, run:
      sudo /opt/puppetlabs/bin/puppet-infra reinitialize replica -y
  6. Optional: Remove old PE packages from all infrastructure nodes. On your primary server, run: puppet infrastructure run remove_old_pe_packages
    All packages older than the current version are removed by default. To remove specific versions, append pe_version=<VERSION_NUMBER> to the command.

Upgrade a large installation

To upgrade a large installation, run the PE installer on your primary server, and then upgrade compilers and any additional components.

Before you begin

Back up your PE installation.

Ensure you have a valid admin RBAC token in order to upgrade compilers or a replica.

In Hiera, pe.conf, or the console (in the PE Master node group), remove any agent_version parameters you set in the pe_repo class that matches your infrastructure nodes. This ensures the upgrade isn't blocked by attempting to download non-default agent versions for your infrastructure OS and architecture.

  1. Download the tarball for your operating system and architecture. Optionally, you can Verify the installation package.
  2. Run tar -xf <TARBALL> to unpack the installation tarball.
    You need about 1 GB of space to untar the installer.
  3. From the installer directory on your primary server, run sudo ./puppet-enterprise-installer to start the installer, and then follow the CLI instructions to complete your server upgrade.
    To specify a different pe.conf file than the existing file, use the -c flag, such as:
    sudo ./puppet-enterprise-installer -c <FULL_PATH_TO_pe.conf>
    This flag tells the installer to backup the previous pe.conf file to /etc/puppetlabs/enterprise/conf.d/<TIMESTAMP>.conf and create a new pe.conf file at /etc/puppetlabs/enterprise/conf.d/pe.conf.
  4. To upgrade compilers, log in to your primary server as root and run one of these commands:
    • To upgrade specific compilers, run:
      sudo puppet infrastructure upgrade compiler <COMPILER_FQDN-1>,<COMPILER_FQDN-2>
    • To upgrade all compilers simultaneously, run:
      sudo puppet infrastructure upgrade compiler --all
    • To specify an authentication token location other than the default location, include --token-file <PATH_TO_TOKEN> in the command, such as:
      sudo puppet infrastructure upgrade compiler <COMPILER_FQDN> --token-file <PATH_TO_TOKEN>
  5. Upgrade these additional PE infrastructure components:
    • Agents
    • PE client tools: On unmanaged nodes, you must re-install the client tools version that matches the PE version you upgraded to. On managed nodes and infrastructure nodes, client tools are automatically updated when you upgrade PE.
  6. In disaster recovery installations, upgrade your replica.
    The replica is temporarily unavailable to serve as backup during this step, so time upgrading your replica to minimize risk.
    1. On your primary server logged in as root, run:
      sudo puppet infrastructure upgrade replica <REPLICA_FQDN>
      If you want to specify an authentication token other than the default, run:
      sudo puppet infrastructure upgrade replica <REPLICA_FQDN> --token-file <PATH_TO_TOKEN>
    2. After the replica upgrade successfully completes, verify that primary and replica services are operational. On your primary server, run:
      sudo /opt/puppetlabs/bin/puppet-infra status
    3. If your replica reports errors, reinitialize the replica. On your replica, run:
      sudo /opt/puppetlabs/bin/puppet-infra reinitialize replica -y
  7. Optional: Remove old PE packages from all infrastructure nodes. On your primary server, run: puppet infrastructure run remove_old_pe_packages
    All packages older than the current version are removed by default. To remove specific versions, append pe_version=<VERSION_NUMBER> to the command.
What to do next

Optionally convert legacy compilers to the new style compiler running the PuppetDB service.

Upgrade an extra-large installation

You can use the PEADM module to upgrade extra-large installations. Contact your technical account manager for additional support.

Upgrade a standalone PE-PostgreSQL installation

To upgrade a large installation with standalone PE-PostgreSQL, run the PE installer first on your PE-PostgreSQL node, then on your primary server, and then upgrade any additional components.

Before you begin

Back up your PE installation.

Ensure you have a valid admin RBAC token in order to upgrade compilers.

In Hiera, pe.conf, or the console (in the PE Master node group), remove any agent_version parameters you set in the pe_repo class that matches your infrastructure nodes. This ensures the upgrade isn't blocked by attempting to download non-default agent versions for your infrastructure OS and architecture.

  1. Download the tarball for your operating system and architecture. Optionally, you can Verify the installation package.
  2. Run tar -xf <TARBALL> to unpack the installation tarball.
    You need about 1 GB of space to untar the installer.
  3. Upgrade your PostgreSQL node.
    1. Ensure that the user_data.conf file on your PostgreSQL node is up to date by running puppet infrastructure recover_configuration on your primary server, and then copying /etc/puppetlabs/enterprise/conf.d to the PostgreSQL node.
    2. Copy the installation tarball to the PostgreSQL node, and from the installer directory, run the installer: sudo ./puppet-enterprise-installer
  4. From the installer directory on your primary server, run sudo ./puppet-enterprise-installer to start the installer, and then follow the CLI instructions to complete your server upgrade.
    To specify a different pe.conf file than the existing file, use the -c flag, such as:
    sudo ./puppet-enterprise-installer -c <FULL_PATH_TO_pe.conf>
    This flag tells the installer to backup the previous pe.conf file to /etc/puppetlabs/enterprise/conf.d/<TIMESTAMP>.conf and create a new pe.conf file at /etc/puppetlabs/enterprise/conf.d/pe.conf.
  5. To upgrade compilers, log in to your primary server as root and run one of these commands:
    • To upgrade specific compilers, run:
      sudo puppet infrastructure upgrade compiler <COMPILER_FQDN-1>,<COMPILER_FQDN-2>
    • To upgrade all compilers simultaneously, run:
      sudo puppet infrastructure upgrade compiler --all
    • To specify an authentication token location other than the default location, include --token-file <PATH_TO_TOKEN> in the command, such as:
      sudo puppet infrastructure upgrade compiler <COMPILER_FQDN> --token-file <PATH_TO_TOKEN>
  6. Upgrade these additional PE infrastructure components:
    • Agents
    • PE client tools: On unmanaged nodes, you must re-install the client tools version that matches the PE version you upgraded to. On managed nodes and infrastructure nodes, client tools are automatically updated when you upgrade PE.
  7. Optional: Remove old PE packages from all infrastructure nodes. On your primary server, run: puppet infrastructure run remove_old_pe_packages
    All packages older than the current version are removed by default. To remove specific versions, append pe_version=<VERSION_NUMBER> to the command.
What to do next

Optionally convert legacy compilers to the new style compiler running the PuppetDB service.

Upgrade an unmanaged PostgreSQL installation

To upgrade a Puppet Enterprise (PE) installation that relies on an unmanaged PostgreSQL database, you must upgrade PostgreSQL to version 14 and ensure that your PostgreSQL database roles, privileges, extensions, and configuration are up to date with the most recent version of PE. Then, proceed with the PE upgrade.

Before you begin
Back up your PE installation, and, if desired, back up your PostgreSQL databases.

Ensure that you have a valid role-based access control (RBAC) admin token to upgrade compilers (for example, Generate a token using puppet-access).

In Hiera, pe.conf, or the console (in the PE Master node group), remove any agent_version parameters you set in the pe_repo class that matches your infrastructure nodes. This ensures the upgrade isn't blocked by attempting to download non-default agent versions for your infrastructure OS and architecture.

  1. If you are upgrading from a 2019.8 version of PE, ensure that you have upgraded to the latest 2019.8.z version. As part of the upgrade to 2019.8.z, you must have followed the unmanaged PostgreSQL upgrade instructions for that 2019.8 version of PE before you begin the upgrade to 2021.
  2. If you are upgrading from a 2021 version of PE that is earlier than 2021.2, add the PuppetDB read_user. To add the read_user, run the commands that correspond to your authentication method to generate the role and authentication, and then run the grant commands: 
    • Password authentication: Run the following command and ensure that you specify a password:
      CREATE ROLE "pe-puppetdb-read"  
      LOGIN NOCREATEROLE NOCREATEDB NOSUPERUSER  CONNECTION LIMIT -1 PASSWORD '<PUPPETDB_USER_PASSWORD>';
    • Certificate authentication: Create the database user by adding the following lines to the pg_hba.conf file:
      hostssl pe-puppetdb pe-puppetdb-read 0.0.0.0/0 
      cert map=pe-puppetdb-pe-puppetdb-read-map clientcert=verify-full 
      hostssl pe-puppetdb pe-puppetdb-read ::/0 cert map=pe-puppetdb-pe-puppetdb-read-map clientcert=verify-full
      Map the database user role by adding the following line to the pg_ident.conf file:
      pe-puppetdb-pe-puppetdb-read-map <PRIMARY_CERTNAME> pe-puppetdb-read
      Create the role by running the following command. Use the psql tool:
      CREATE ROLE "pe-puppetdb-read"  LOGIN NOCREATEROLE NOCREATEDB NOSUPERUSER  CONNECTION LIMIT –1;
    After you complete either of the previous steps, finish granting permissions to the new role by running the following commands:
    GRANT CONNECT ON DATABASE "pe-puppetdb" TO "pe-puppetdb-read"; 
    ALTER DEFAULT PRIVILEGES FOR USER "pe-puppetdb" IN SCHEMA "public" GRANT SELECT ON TABLES TO "pe-puppetdb-read"; 
    ALTER DEFAULT PRIVILEGES FOR USER "pe-puppetdb" IN SCHEMA "public" GRANT EXECUTE ON FUNCTIONS TO "pe-puppetdb-read"; 
    ALTER DEFAULT PRIVILEGES FOR USER "pe-puppetdb" IN SCHEMA "public" GRANT USAGE ON SEQUENCES TO "pe-puppetdb-read"; 
    GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO "pe-puppetdb-read"; 
    GRANT USAGE ON ALL SEQUENCES IN SCHEMA "public" TO "pe-puppetdb-read";
  3. If you are upgrading from a 2021 version of PE that is earlier than 2021.4, add the pgcrypto extension to the Puppet orchestrator database. Run the following commands by using the psql tool:
    \c "pe-orchestrator" 
     CREATE EXTENSION IF NOT EXISTS "pgcrypto"; 
  4. If you are upgrading from any PE version earlier than 2021.6, upgrade PostgreSQL to version 14:
    1. On your PostgreSQL node, install these packages:
      • postgresql14
      • postgresql14-libs
      • postgresql14-contrib
      • postgresql14-server
    2. Stop the PostgreSQL 11 service:
      systemctl stop postgresql-11.service
    3. As the PostgreSQL user, create a database:
      su - postgres /usr/pgsql-14/bin/pg_ctl -D /var/lib/pgsql/14/data/ initdb
    4. If you use certificate authentication, note the location of your certificate, private key, and certificate authority files. You might have to move these files to the new installation directory, and you might have to update the postgresql.conf file.
    5. Copy your postgresql.conf file from the /var/lib/pgsql/11/data/ directory to the /var/lib/pgsql/14/data/ directory and update the data directory setting. If you moved the certificate, private key, and certificate authority files, update the ssl_cert_file, ssl_key_file, and ssl_ca_file settings.
    6. Migrate databases to PostgreSQL 14:
      /usr/pgsql-14/bin/pg_upgrade -b /usr/pgsql-11/bin -B /usr/pgsql-14/bin/ -d /var/lib/pgsql/11/data -D /var/lib/pgsql/14/data/
      Tip: This command uses pg_upgrade to upgrade and migrate your unmanaged PostgreSQL instance from version 11 to 14. If you don't want to use this command, you can back up your PostgreSQL 11 databases, manually wipe the PostgreSQL 11 installation, ensure that PostgreSQL 14 is installed, and then restore the databases to the PostgreSQL 14 installation. Or, you can back up the PostgreSQL 11 databases, set up a new node with PostgreSQL 14, restore databases to the new node, and then reconfigure PE to point to the new database_host node.
    7. Copy your pg_hba.conf and pg_ident.conf files from the /var/lib/pgsql/11/data/ directory to the /var/lib/pgsql/14/data/ directory.
    8. In the pg_hba.conf file, change all instances of clientcert=1 to clientcert=verify-full.
  5. Start the PostgreSQL 14 service:
    systemctl start postgresql-14
  6. Optional: As the PostgreSQL user, clean up the database:
    su - postgres /usr/pgsql-14/bin/vacuumdb --analyze --all
  7. If you are upgrading from a version of PE earlier than 2021.7.2, ensure that the pe-puppetdb-read role is granted connect by the pe-puppetdb-migrator role.
    This step is necessary so that the pe-puppetdb-migrator can revoke query connections during migrations. Additionally, the chain of pe-puppetdb-read role granted to pe-puppetdb (and pe-puppetdb granted to pe-puppetdb-migrator, already present in the database schema) makes it possible for the pe-puppetdb and pe-puppetdb-migrator roles to kill queries during migration and garbage collection.
    In the PostgreSQL cluster, run the following commands:
    REVOKE CONNECT ON DATABASE "pe-puppetdb" FROM "pe-puppetdb-read"; 
    SET ROLE "pe-puppetdb-migrator"; 
    GRANT CONNECT ON DATABASE "pe-puppetdb" TO "pe-puppetdb-read"; 
    SET ROLE "postgres"; 
    GRANT "pe-puppetdb-read" TO "pe-puppetdb"
  8. Download the tarball for your operating system and architecture. Optionally, you can Verify the installation package.
  9. Run tar -xf <TARBALL> to unpack the installation tarball.
    You need about 1 GB of space to untar the installer.
  10. From the installer directory on your primary server, run sudo ./puppet-enterprise-installer -s to start the installer, and then follow the CLI instructions to complete the server upgrade.
    • The -s flag tells the installer to skip database checks.
    • To specify a different pe.conf file than the existing file, use the -c flag as shown here:
      sudo ./puppet-enterprise-installer -c <FULL_PATH_TO_pe.conf>

      This flag tells the installer to back up the previous pe.conf file to /etc/puppetlabs/enterprise/conf.d/<TIMESTAMP>.conf and create a new pe.conf file at /etc/puppetlabs/enterprise/conf.d/pe.conf.

  11. To upgrade compilers, log in to your primary server as root and run one of these commands:
    • To upgrade specific compilers, run:
      sudo puppet infrastructure upgrade compiler <COMPILER_FQDN-1>,<COMPILER_FQDN-2>
    • To upgrade all compilers simultaneously, run:
      sudo puppet infrastructure upgrade compiler --all
    • To specify an authentication token location other than the default location, include --token-file <PATH_TO_TOKEN> in the command, such as:
      sudo puppet infrastructure upgrade compiler <COMPILER_FQDN> --token-file <PATH_TO_TOKEN>
  12. Upgrade these additional PE infrastructure components:
    • Agents
    • PE client tools: On unmanaged nodes, you must re-install the client tools version that matches the PE version you upgraded to. On managed nodes and infrastructure nodes, client tools are automatically updated when you upgrade PE.
  13. Optional: Remove old PE packages from all infrastructure nodes. On your primary server, run: puppet infrastructure run remove_old_pe_packages
    All packages older than the current version are removed by default. To remove specific versions, append pe_version=<VERSION_NUMBER> to the command.
What to do next

Optionally, convert existing compilers to the new style compiler running the PuppetDB service.

CAUTION: Don't use CONCURRENTLY with PostgreSQL 14.0 through 14.3. If you do, ensure that you REINDEX without using CONCURRENTLY.