PE release notes

These are the new features, enhancements, resolved issues, and deprecations in this version of PE.

Security and vulnerability announcements are posted at https://puppet.com/docs/security-vulnerability-announcements.

PE 2021.7.2

Released January 2023

Important: PE 2021.7 is the current PE LTS series, and PE 2023.0 is the new STS PE series.

For information about upgrading from 2019.8.z to 2021.7 (and earlier 2021.y series release notes) go to What's new since PE 2019.8 and Upgrading Puppet Enterprise.

For information about upgrading to 2023.0, go to Upgrading Puppet Enterprise in the 2023.0 documentation.

Enhancements

recover_configuration command recreates nodes files
Previously, the puppet infrastructure recover_configuration command merged new values into the nodes files (at /etc/puppetlabs/enterprise/conf.d/nodes) instead of overwriting the files. This process caused problems if you deleted a value relevant to one or more nodes, because the deleted value would remain in these files and continue to be applied.
Now, the recover_configuration command fully rewrites the nodes files on each invocation. This process matches how the command handles changes to the user_data.conf file.
Improved performance when regenerating agent certificates for multiple agents
The puppet infrastructure run regenerate_agent_certificate action is now faster when you Regenerate agent certificates for multiple agents. You can also now use the agent_pdb_query parameter to run a PDB query to generate a list of agents for which you want to regenerate certificates.
This action now uses the Puppet Server CA API endpoints directly, rather than relying on the puppetserver ca CLI, as it did previously. This process is faster, but, if you encounter problems, you can revert to the previous behavior by including use_puppetserver_cli=true in the command.
Specify Code Manager worker cache cleanup interval
The deploy-pool-cleanup-interval parameter specifies how often workers pause to clean their on-disk caches. Learn more about this setting in Code Manager parameters.

Platform support

This version adds support for the following platforms:
Agent platforms
Solaris 10 (SPARC, i386)
Client tools platforms
Solaris 10 (SPARC, i386)

Resolved issues

Code Manager respects full_deploy setting in Hiera
The full_deploy parameter is now correctly applied when you Customize Code Manager configuration in Hiera.
Previously, the full_deploy parameter value was disregarded when included in a Code Manager configuration in Hiera. As a work-around, you could create a separate .conf file to manually manage this parameter.
Important: If you created a .conf file for the full_deploy parameter, you must remove this file and reconfigure the parameter in Hiera (as described in Configuring module deployment scope).
Certain plans correctly restore puppet service to pre-plan state
Due to a bug introduced in PE 2021.6, some plans that must stop the puppet service while the plans run were not restoring the puppet service to its pre-plan state after the plan finished running.
The four affected plans, and their associated puppet infra commands, are as follows:
  • The secondary_cert_regen plan, which is triggered by puppet infra run regenerate_compiler_certificate and puppet infra run regenerate_replica_certificate
  • The convert_legacy_compiler plan, which is triggered by puppet infra run convert_legacy_compiler
  • The reprovision_replica plan, which is specifically triggered by puppet infra upgrade replica --only-recreate-databases
  • The enable_ha_failover plan, which is triggered by puppet infra run enable_ha_failover
Important: If you ran any of these four plans since upgrading to PE 2021.6, check the state of the puppet service on your infrastructure nodes.
PuppetDB database user can purge reports
An issue was fixed to help ensure that the PuppetDB database user can purge reports.
Corrected fact list handling in some PE console UI components
Some user interface (UI) components in the PE console use fact lists. A recent change caused these components to use the entire list of fact names. This process caused performance problems in environments with many facts. The handling of fact lists was updated to fix this issue and improve performance.
Orchestrator code directories excluded from puppet-backup create --scope=config
When Customizing backup and restore scope, the orchestrator code directories (specifically opt/puppetlabs/server/data/orchestration-services/data-dir and opt/puppetlabs/server/data/orchestration-services/code) are excluded when you specify the config scope.
These directories are included in the code scope.
Garbage collection log fixes
The introduction of Java 11 resulted in two issues pertaining to garbage collection logs. The issues are now fixed:
Dates and times are included in garbage collection logs.
The maximum volume of retained garbage collection logs is 256 MB.
Security fixes
Addressed CVE-2022-41946 and CVE-2022-41404.

PE 2021.7.1

Released October 2022

Important: PE 2021.7 is our new PE LTS series. If you're preparing to upgrade, or looking for earlier 2021.y release notes, go to What's new since PE 2019.8.

For those awaiting the new STS, we're still getting things ready for the first release in that series. We thank you for your patience.

New features

Stop in-progress plans
Use POST /command/stop_plan to stop an orchestrator plan job that is currently running.

Platform support

Deprecated and removed platforms are listed under Deprecations and removals.

This version adds support for these platforms:
Agent platforms
Fedora 36
Patch management platforms
Fedora 36

Deprecations and removals

Deprecated agent platforms
Debian 9
Fedora 32, 34
Deprecated patch management platforms
Debian 9
Fedora 34

Resolved issues

Deactivated scheduled jobs could still run.
If you deactivate a recurring scheduled job, the inactive job no longer continues to run after restarting pe-orchestration-services.
Deactivated jobs aren't visible in the console or in responses from the Scheduled jobs endpoints, but, prior to this fix, some deactivated jobs could run again at their next scheduled interval as if they had not been deactivated. This issue only impacted deactivated scheduled jobs that had run within the job_prune_threshold limit after restarting pe-orchestration-services.
Orchestrator didn't properly periodically prune jobs
Fixed a calculation error introduced in PE 2021.5 that caused job records to be stored beyond the job_prune_threshold limit.
regenerate_agent_certificate couldn't verify node type if client tools were installed through a package resource
When you run the puppet infra run regenerate_agent_certificate command, the plan can now verify that a node isn't an infrastructure node if the pe-client-tools package was installed on the node through a package resource.
RBAC API command/config/remove-disclaimer endpoint erroneously required Content-Type header
The POST /command/config/remove-disclaimer endpoint no longer requires a Content-Type header, because requests to this endpoint have no body content.
Internal task jobs shared primary task thread pool
Internal task jobs (such as tasks that force stop other tasks) no longer run on the same thread pool as your user-initiated tasks. This allows internal tasks to queue separately from other tasks. For example, requests to POST /command/stop don't get stuck waiting if the regular task queue is full.
Improved PuppetDB disaster recovery sync performance
The PuppetDB disaster recovery sync process transferred more reports than necessary when syncing reports, which sometimes caused timeouts.
Empty task metadata files prevented you from running tasks in the console
Loading empty task metadata files no longer cause errors.
Some puppet infrastructure commands failed when restarting the puppet service
Previously, several puppet infrastructure commands failed when restarting the puppet service at the end of the action. While the service had successfully restarted, the effected actions couldn't properly detect the restart, which caused them to fail. This has been fixed.

PE 2021.7.0

Released August 2022

Important: PE 2021.7 is our new PE LTS. Expect to see changes to the documentation ahead of our next STS release. We apologize for any inconvenience.

If you're preparing to upgrade or looking for earlier 2021.y release notes, go to What's new since PE 2019.8.

New features

Force stop in-progress Puppet runs
By default, POST /command/stop prevents new runs from starting, but allows in-progress runs to finish. Now you can use the force option to block new runs and stop in-progress runs. This is useful, for example, if you need to stop a task that is hanging.
pe_status_check module bundled with PE
The pe_status_check module helps keep your PE installation in an ideal state. Read About the pe_status_check module to learn how the module works and how to get the module's reports.
Important: If you have previously specified a version of this module, from the Forge or other sources, in your code, we recommend removing this version before upgrading to allow the version bundled with PE to be asserted.
New Orchestrator scheduling API
This release includes a new scheduling API for the orchestrator, which introduces several new scheduled_jobs endpoints and deprecates the previous scheduling API's endpoints (for a list of deprecated endpoints, see Deprecations and removals for this release, below).
Existing scheduled jobs are automatically migrated to the new scheduling system, and the PE console now uses the new API (but there is no change to the UI).
With the new API, you can edit scheduled jobs; however, this functionality is currently only available through the API (not yet available in the PE console). To learn more about the new endpoints, go to Scheduled jobs endpoints.
Tools that rely on the deprecated endpoints must be upgraded to use the new endpoints.
Use the RBAC API to set the disclaimer text on the console login page
You can use the RBAC API v1 Disclaimer endpoints to configure the disclaimer text that appears on the PE console login page.
Automatically sync LDAP user details and group membership
Prior to this release, user details and group membership for LDAP-based users only refreshed when users logged in. Now, LDAP group bindings, user names, and descriptions update automatically every 30 minutes (by default) for every LDAP user in the system. If a user is no longer present in LDAP or has no group bindings, all user-group associations are removed from the user and all of the user's known tokens are revoked.
You can disable automatic refresh or change the refresh time by changing the puppet_enterprise::profile::console::ldap_sync_period parameter. Learn more about this parameter in Configure RBAC and token-based authentication settings.
Stop LDAP users from logging in if they have no group membership
You can use the exclude-groupless-ldap-users setting to prevent LDAP users with no group memberships from logging in. This setting is off by default. To learn how to enable this setting, go toRequire LDAP group membership to log in.
Metrics API v2 documentation
The Metrics API v2 uses the Jolokia library to query Orchestrator service metrics. This version of the API has been available for some time, but it was only described in the open source Puppet documentation.
Disaster recovery support for FIPS platforms
Disaster recovery is now supported for FIPS 140-2 compliant Red Hat Enterprise Linux (RHEL) 7 and 8.

Enhancements

Orchestrator API endpoints return "total": 0 if there are no jobs
Orchestrator API v1 endpoints that return pagination containing the total number of jobs (such as GET /jobs, GET /scheduled_jobs (deprecated), and GET /plan_jobs) now return "total": 0, instead of "total": null, when there are no jobs.
Activity service API /v2/events endpoint returns more information for orchestrator events
Responses from GET /v2/events containing information about orchestrator events (Puppet agent runs and task runs) now report additional information about the job start time, end time, duration, and status.
Upgraded JRuby
We are now shipping JRuby 9.3.4.0.
Addressed CVEs
We updated the PostgreSQL driver in some PE component to address CVE-2022-31197. The application was not vulnerable to exploit prior to this update.
We also made changes to address CVE-2022-1292 and CVE-2022-2068.

Platform support

Ubuntu 16.04 is no longer a supported agent platform.

This version adds support for these platforms:
Agent
macOS 12 M1
Ubuntu (General Availability kernels) 22.04 x86_64
Microsoft Windows 11 x64
Client tools
Ubuntu (General Availability kernels) 22.04 x86_64
macOS 12 M1, M2
Patch management
Ubuntu (General Availability kernels) 22.04 x86_64
Microsoft Windows 11 x64

Deprecations and removals

Ubuntu 16.04 is no longer a supported agent platform.

The following endpoints are deprecated due to the release of several new Scheduled jobs endpoints for the Orchestrator API. Tools that rely on deprecated endpoints must be upgraded to use the new endpoints. Existing scheduled jobs are automatically migrated to the new scheduling system that uses the new endpoints.
GET /scheduled_jobs (deprecated)
Replaced by GET /scheduled_jobs/environment_jobs and GET /scheduled_jobs/environment_jobs/<job-id>
DELETE /scheduled_jobs/<job-id> (deprecated)
Replaced by PUT /scheduled_jobs/environment_jobs/<job-id>
POST /command/schedule_deploy (deprecated)
Replaced by POST /scheduled_jobs/environment_jobs
POST /command/schedule_plan (deprecated)
Replaced by POST /scheduled_jobs/environment_jobs
POST /command/schedule_task (deprecated)
Replaced by POST /scheduled_jobs/environment_jobs

Resolved issues

full-deploy didn't override --incremental
Code Manager's full-deploy option, used for Configuring module deployment scope, now correctly overrides the default --incremental deploy behavior.
Code Manager couldn't fetch code on FIPS platforms
On FIPS platforms running PE versions 2021.5 or 2021.6, Code Manager and r10k couldn't fetch code from your code repo due to libssh attempting to use algorithms that are not allowed on FIPS. In PE 2021.7, the disallowed algorithms are disabled in libssh, allowing Code Manager and r10k to successfully fetch code.
An unreachable replica consumed all of the primary server's disk space
Previously, if a provisioned replica became unreachable, the associated primary server could quickly run out of disk space, causing a complete interruption to PE services. In larger installations, an outage could occur in under an hour. Excessive disk usage was caused by the PE-PostgreSQL service on the primary server retaining change logs that the replica hadn't acknowledged.
To resolve this, we limited available disk space for the pg_wal directory. To learn more and tune this setting in your installation, refer to PostgreSQL WAL disk space.
Orchestrator ignored _noop when passed to run_task() through a plan
When a plan passed the _noop flag to the run_task() function, the PE Orchestrator now correctly acknowledges the _noop flag.
Some RBAC endpoints returned an incorrect Content-Type
Responses for the following endpoints now return the correct Content-Type: POST /users/<uuid>/password/reset, POST /auth/reset, and PUT /users/current/password.
LDAP with anonymous binding sometimes prevented Console Services from starting or restarting
Previously, if you use anonymous binding, or another configuration with a zero-length password, Console Services sometimes couldn't start or restart. This could cause upgrade failures when upgrading to PE version 2021.4 through 2021.6 from a version earlier than 2021.4. This is resolved.
Orchestrator doesn't restart unexpectedly during the convert_legacy_compiler plan
Previously, when running the enterprise_tasks::convert_legacy_compiler plan, the hosts in the pcp-brokers array could change order. This caused the pe-orchestration-services service to restart (as a result of detecting a presumed configuration change) and, ultimately, caused the plan to fail.
Some SSO configuration fields weren't marked as required
The Organization and Contacts fields on the SSO Configuration page are now correctly marked as required.
Orchestrator couldn't run tasks within modules named tasks or scripts
You can now successfully run tasks that are within modules named tasks or scripts.
Incorrect run-time for splayed agent runs
In previous PE versions, when agent runs were splayed, the run-time reported in the PE console was incorrect.
Sensitive parameters sometimes exposed in cleartext in job results
Sensitive plan parameters from Bolt plans that execute actions over PCP transport are no longer stored in the orchestrator database and, therefore, are properly masked in the job results.