PE release notes

These are the enhancements and resolved issues in this version of Puppet Enterprise (PE).

For security and vulnerability announcements, see Security: Puppet's Vulnerability Submission Process.

PE 2023.2

Released June 2023

Important: Puppet Enterprise (PE) 2023 is our leading-edge PE release stream (also referred to as STS). For important information about upgrading to 2023, see Upgrading Puppet Enterprise.

If you're on the LTS stream (2021.7), you'll find release notes and other information for that series in the 2021.7 documentation.

Customers on 2019.8.z are encouraged to upgrade to either 2021.7 or 2023.

Resolved issues

Security fix
Addressed CVE-2023-2530

PE 2023.1

Released May 2023

Important: Puppet Enterprise (PE) 2023 is our leading-edge PE release stream (also referred to as STS). For important information about upgrading to 2023, see Upgrading Puppet Enterprise.

If you're on the LTS stream (2021.7), you'll find release notes and other information for that series in the 2021.7 documentation.

Customers on 2019.8.z are encouraged to upgrade to either 2021.7 or 2023.

Enhancements

Improved performance when querying PuppetDB
This enhancement helps to improve performance for PuppetDB queries that contain large arrays, for example, if many nodes are enumerated or many terms are joined by a single "and" or "or" element.
Improved performance for the each, map, and filter functions in the Puppet language
Previously, the Puppet language built-in functions each, map, and filter showed poor performance and consumed unnecessary resources when run on JRuby software. The issue was resolved to enhance performance.
Puppet Server provides more reliable warnings when it cannot check for an update
By default, Puppet Server periodically checks whether a new version of Puppet Server is available. Previously, if Puppet Server could not connect to the update server, users were not provided with adequate information about the error. Starting with Puppet Server 7.10.1, a warning about the error is available in the log file.

Deprecations and removals

Deprecated PSON
In previous releases, Pure JavaScript Open Notation (PSON) was used in Puppet to serialize data for transmission.

PSON is deprecated in Puppet 7 and will be removed in Puppet 8.

Resolved issues

Tasks page is available following a software update
After upgrading PE from 2019.8 to 2021.7.1, the Tasks overview page in the PE console sometimes failed to load because of a timeout error. The issue is fixed in PE 2021.7.3 and 2023.1.
Scheduled task jobs run successfully without a defined timeout
In PE 2023.0, task jobs failed to start if they were scheduled without an explicitly defined timeout. In PE 2023.1, the issue is resolved to help ensure that task jobs start as scheduled even without an explicitly specified timeout option. If a timeout option is not explicitly defined, the default timeout for tasks is applied.
Timeout and concurrency values for scheduled tasks can be viewed and edited in the console
In PE 2023.0, the timeout and concurrency values for a scheduled task could not be viewed or edited in the PE console. This issue is fixed in PE 2023.1:
  • When you view a scheduled task in the console, any specified timeout and concurrency values are displayed in the new Timeout and Concurrency fields.
  • When you edit a scheduled task in the console, you can update the values in the new Timeout and Concurrency fields.
  • Any timeout or concurrency values that you specify for scheduled tasks will be applied.
When tasks are rerun in the console, timeout and concurrency attributes are preserved
In PE 2023.0, tasks that were rerun in the PE console did not properly preserve the concurrency and timeout attributes of the task job. This issue is fixed in PE 2023.1.
Access rights for remote users can be revoked and reinstated from the console
In PE 2023.0, a defect was introduced that prevented the revocation or restoration of some remote users by using the PE console. This issue is resolved in PE 2023.1.
Performance issue with Puppet agent runtimes is resolved
After an upgrade from PE 2019.8.12 to PE 2021.7.1, some users saw a significant increase in Puppet agent runtimes. The increase was caused by Facter 4, which was not using cached information to resolve facts. As a result, facts were resolved multiple times. The issue is now resolved to normalize the performance of the Puppet agent.
Enabling the lockless code deploy feature no longer causes performance issues in PuppetDB catalog compilation
When the versioned_deploys setting is enabled, Puppet previously reported the full directory path to the environment after resolving symbolic links as the source for resources in a catalog. Puppet now reports the path to the resource before resolving symbolic links in the environment path to help prevent instability of the PuppetDB instance.
Certificates and keys can be backed up and restored by specifying the certs scope
Previously, if you ran the puppet-backup create command and specified a scope of certs, the command failed to back up the certificate authority root key and certificates. This issue occurred because Puppet 7 introduced a new default path for the certificate authority (CA) directory (/etc/puppetlabs/puppetserver/ca), but the puppet-backup create command failed to locate the new directory. Similarly, if you ran the puppet-backup restore command with a scope of certs, the restore operation failed. The CA directory issue is resolved so that backup and restore operations can run successfully.
Timeouts can be specified for SAML authentication
Previously, when users configured the PE console to specify session-timeout and session-maximum-lifetime values, the settings were applied to Lightweight Directory Access Protocol (LDAP) tokens and local login tokens. However, the specified settings were not applied to Security Assertion Markup Language (SAML) tokens, which are used for authentication with SAML identity providers. The issue is corrected to ensure that the specified settings also apply to SAML session lifetimes.
Updates implemented to help users enter valid URLs
In previous versions of PE, the role-based access control (RBAC) service permitted the entry of invalid URLs when users specified the Organizational URL setting. Login attempts would then fail with the following error message:
'Invalid settings: organization_not_enough_data'

In PE 2021.7.3 and 2023.1, the RBAC service is updated to enforce valid URLs when users create or update a connection to a SAML identity provider, and the PE console displays a warning if the user enters an invalid URL for the Organizational URL setting.

User-defined temporary directory is honored during PE restore operations
After you back up your PE infrastructure, you can use the puppet-backup restore command to restore the backup. Previously, if you set the —tmpdir flag or the TMPDIR environment variable to specify a temporary directory for restore operations, the directory was not honored, and the default /tmp directory was used in some cases. In addition, some files were not cleaned up after the restore operation. This issue is corrected to ensure that the user-specified directory is used, and all temporary files are removed after the restore operation.
Issue that caused an unexpected increase in CPU usage is resolved
In PE 2021.7.1, 2021.7.2, and 2023.0, an issue with Puppet Server caused an unexpected increase in central processing unit (CPU) usage in some environments. CPU usage continued to grow and some operations took longer than expected until the Puppet Server service was restarted. This issue is resolved in PE 2023.1 and 2021.7.3.
Security fixes
Addressed CVE-2023-1894 and CVE-2023-26048.

PE 2023.0

Released January 2023

Important: PE 2023 is our new leading-edge PE release stream (also referred to as STS). For important information about upgrading to 2023, see Upgrading Puppet Enterprise.

If you're on the LTS stream (2021.7), you'll find release notes and other information for that series in the 2021.7 documentation.

Customers on 2019.8.z, which is EOL, are encouraged to upgrade to either 2021.7 or 2023.

New features

Authenticate users in multiple LDAP domains
You can now connect multiple Lightweight Directory Access Protocol (LDAP) domains to PE. This new feature brings many changes to the role-based access control (RBAC) API and LDAP-related pages in the PE console.
In the PE console, view and manage all of your LDAP external directory service connections on the LDAP tab of the Access control page.
The Test connection button is removed. When you Connect to external directory services, the Connect button now automatically tests the connection before saving the configuration.
Use the Certificate chain field (or cert_chain API key) to define unique certificate chains across servers.
The following new endpoints replace deprecated or removed endpoints. For a list of deprecated and removed endpoints, refer to the Deprecations and removals section of these release notes.
Responses from these endpoints now include the identity_provider_id:
Default timeout limits for tasks and plans
Timeout limits forcibly stop tasks and plans that run too long. This feature is useful for stopping tasks and plans that are stuck without requiring you to manually monitor task or plan progress.
CAUTION: The feature for forcibly stopping tasks and plans can result in incomplete Puppet runs, partial configuration changes, and other issues. When setting timeout limits, consider the task or plan scope, typical runtime, and your infrastructure's capacity (such as concurrency limits).
The default timeout limits are 40 minutes for tasks (per node) and 60 minutes for plans (for the entire plan run). You can change the global default limits by modifying the default_task_node_timeout and default_plan_timeout settings in your Orchestrator and pe-orchestration-services parameters.
Alternatively, you can set timeout limits for an individual task or plan when Running tasks from the console, Running plans from the console, or running tasks and plans with the Orchestrator API.
You can use the timeout option with the following Orchestrator API endpoints:
Unique status for queued jobs
To better differentiate queued-but-unstarted jobs from jobs that are running, a new pending state was introduced for queued jobs.
The pending state is visible in the console and in responses from GET /plan_jobs and GET /plan_jobs/<job-id>.
View and edit scheduled tasks in the console
You can now view and edit scheduled task details in the console.

Enhancements

Java 17 upgrade
This version upgrades Java from version 11 to 17 and changes the default garbage collector from Parallel to G1.
Thoroughly test PE 2023.0 in a non-production environment before upgrading if you customized PE Java services or you use plug-ins that include Java code.
Stop in-progress plans in the console
When Running plans in PE, you can click Stop plan on the plan's run details page to stop the plan. In this way, you can prevent new tasks from starting and allow in-progress tasks to finish. To forcibly stop in-progress tasks from a stopped plan, follow the instructions in Stop a task in progress.
Forcibly stop in-progress tasks in the console
To Stop a task in progress, you can now both stop and forcibly stop in-progress tasks from the console. Previously, you had to use the Orchestrator API to forcibly stop tasks.
CAUTION: A forcible stop is the last resort when a task is stuck. This type of stop can result in incomplete Puppet runs, partial configuration changes, and other issues.
Provisioning replicas requires matching agent versions
When provisioning a replica, the target node's agent version must match the primary server's agent version. If the versions don't match, the puppet infra provision replica command fails before initializing the provisioning process. Previously, the agent version wasn't checked, and mismatched agent versions caused provisioning to fail partway through.
Increased task_concurrency limit
The default value of the task_concurrency orchestrator parameter was increased from 250 to 1000.
recover_configuration command recreates nodes files
Previously, the puppet infrastructure recover_configuration command merged new values into the nodes files (at /etc/puppetlabs/enterprise/conf.d/nodes) instead of overwriting the files. This process caused problems if you deleted a value relevant to one or more nodes, because the deleted value would remain in these files and continue to be applied.
Now, the recover_configuration command fully rewrites the nodes files on each invocation. This process matches how the command handles changes to the user_data.conf file.
Notification when session expires due to inactivity
PE redirects users to the login page when a session expires due to inactivity. When this happens, the login page now includes a message that indicates why the user was logged out.
Improved performance when regenerating agent certificates for multiple agents
The puppet infrastructure run regenerate_agent_certificate action is now faster when you Regenerate agent certificates for multiple agents. You can also now use the agent_pdb_query parameter to use a PDB query to generate a list of agents for which you want to regenerate certificates.
This action now uses the Puppet Server CA API endpoints directly, rather than relying on the puppetserver ca CLI, as it did previously. This process is faster, but, if you encounter problems, you can revert to the previous behavior by including use_puppetserver_cli=true in the command.
Specify Code Manager worker cache cleanup interval
The deploy_pool_cleanup_interval specified how often workers pause to clean their on-disk caches. Learn more about this setting in Code Manager parameters.
This release includes enhancements to cipher compatibility. For a complete list, go to Compatible ciphers.
CHACHA20 ciphers, compatible with non-FIPS PE installs
TLS_CHACHA20_POLY1305_SHA256 (TLSv1.3)
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (TLSv1.2)
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (TLSv1.2)
TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (TLSv1.2)
AES versions of two GCM ciphers, compatible with FIPS and non-FIPS installs
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (TLSv1.2)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (TLSv1.2)
Removed restrictions
TLS_CHACHA20_POLY1305_SHA256 is no longer limited to Bolt server, ACE server, and NGINX.
ECDHE-ECDSA-CHACHA20-POLY1305 is no longer limited to NGINX.
ECDHE-RSA-CHACHA20-POLY1305 is no longer limited to NGINX.

Platform support

With this release, several previously deprecated platforms were removed. Before upgrading, review the important information provided in Platforms removed in 2023.0.
Removed primary server platforms
CentOS 8
Removed agent platforms
CentOS 8
Debian 9
Fedora 32
Fedora 34
Ubuntu 16.04
Removed patch management platforms
Debian 9
Fedora 34

Deprecations and removals

Deprecated RBAC API endpoints
POST /v1/groups and POST /v2/groups are replaced by POST /command/groups/create.
PUT /v1/ds is replaced by POST /command/ldap/create, POST /command/ldap/update, and POST /command/ldap/delete.
GET /v2/ds is replaced by GET /ldap.
GET /ds/test and PUT /ds/test are replaced by POST /command/ldap/test.
Removed RBAC API endpoints
Removed the previously deprecated GET /v1/ds/, which is replaced by GET /ldap.
Removed platforms
For information about platforms removed in this release, see the Platform Support section.

Resolved issues

Code Manager respects full_deploy setting in Hiera
The full_deploy parameter is now correctly applied when you Customize Code Manager configuration in Hiera.
Previously, full_deploy was disregarded when included in your Code Manager configuration in Hiera. As a work-around, you could create a separate .conf file to manually manage this parameter.
Important: If you created a .conf file for the full_deploy parameter, you must remove this file and reconfigure the parameter in Hiera (as described in Configuring module deployment scope).
Certain plans correctly restore puppet service to pre-plan state
Due to a bug introduced in PE 2021.6, some plans that must stop the puppet service while the plans run were not restoring the puppet service to its pre-plan state after the plan finished running.
The four affected plans, and their associated puppet infra commands, are as follows:
  • The secondary_cert_regen plan, which is triggered by puppet infra run regenerate_compiler_certificate and puppet infra run regenerate_replica_certificate
  • The convert_legacy_compiler plan, which is triggered by puppet infra run convert_legacy_compiler
  • The reprovision_replica plan, which is triggered specifically by puppet infra upgrade replica --only-recreate-databases
  • The enable_ha_failover plan, which is triggered by puppet infra run enable_ha_failover
Important: If you were running PE 2021.6, 2021.7.0, or 2021.7.1 before upgrading to 2023.0, and you ran any of these four plans while running 2021.6, 2021.7.0, or 2021.7.1, check the state of the puppet service on your infrastructure nodes.
PuppetDB database user can purge reports
An issue was fixed to ensure that the PuppetDB database user can purge reports.
Corrected fact list handling in some PE console UI components
Some UI components in the PE console use fact lists. A recent change caused these component to use the entire list of fact names, which caused performance problems in environments with many facts. The handling of fact lists was corrected to fix this issue and improve performance.
Orchestrator code directories excluded from puppet-backup create --scope=config
When Customizing backup and restore scope, the orchestrator code directories (specifically /opt/puppetlabs/server/data/orchestration-services/data-dir and /opt/puppetlabs/server/data/orchestration-services/code) are excluded when you specify the config scope.
These directories are included in the code scope.
Plan action jobs have user data
Previously, jobs started as a result of plan action function didn't have an associated user stored in the database, which caused problems with some orchestrator commands. Now, user data is stored for these jobs.
Garbage collection log fixes
The introduction of Java 11 resulted in two issues relating to garbage collection logs. The issues are now fixed:
Dates and times are now included in garbage collection logs.
The maximum volume of retained garbage collection logs is 256 MB.
Security fixes
Addressed CVE-2022-41946 and CVE-2022-41404.