Migrate 4.x data to 5.x
The Continuous Delivery for PE 5.x series uses Bolt for installation and management rather than Puppet Application Manager, so you must migrate your 4.x series data to the new installation.
What is migrated?
- The database, which contains all your Continuous Delivery for PE 4.x installation's information, users, integrations, history, and artifacts (except job commands and job logs).
- The object store, which contains all of your 4.x installation's job commands and job logs.
- Any of these configuration settings, if in use on
CD4PE_REPO_CACHE_RETRIEVAL_TIMEOUT_MINUTES CD4PE_LDAP_GROUP_SEARCH_SIZE_LIMIT CD4PE_ENABLE_REPO_CACHING CD4PE_HTTP_CONNECTION_TIMEOUT_SEC CD4PE_HTTP_READ_TIMEOUT_SEC CD4PE_HTTP_WRITE_TIMEOUT_SEC CD4PE_HTTP_REQUEST_TIMEOUT_SEC CD4PE_INCLUDE_GIT_HISTORY_FOR_CD4PE_JOBS CD4PE_JOB_GLOBAL_TIMEOUT_MINUTES CD4PE_JOB_HTTP_READ_TIMEOUT_MINUTES
- The 4.x root user, because you create a new root user when installing 5.x.
JVM_ARGSconfiguration setting, if in use on 4.x.
- These backup tables created for you by Continuous Delivery for PE
during the 4.x series:
app-pipelines-3-0-backup.pfi, which was created during the upgrade from 3.x to 4.x
pe-ia-resource-results-cve-2020-7944-backup.pfi, which were created as part of the CVE 2020-7944 remediation in version 3.4.0
A super user or the root user must perform this process.
Migrations from installations with external databases are not supported.
Run the migration task
The migration task uses the
cd4peadm::install plan to
migrate data from existing 4.x PAM-based installation to a new 5.x
Unlike version 4.x, Continuous Delivery for Puppet Enterprise (PE) 5.x uses Bolt for installation, configuration, and administration instead of PAM.
Before migrating, please make sure the system you plan to install Bolt on has internet access as well as SSH access to both the system on which you intend to install Continuous Delivery for PE 5.x as well as the existing 4.x installation from which you are migrating. In addition, because Bolt is installed on one or more systems and used to manage one centralized Continuous Delivery for PE installation, it may be a good idea to maintain the Bolt project in its own VCS repo.
- Install Bolt version 3.27.2 or later on a jumphost. This can be the intended Continuous Delivery for PE 5.x host, or any other system.
Create the Continuous Delivery for PE
Bolt project and switch to that directory.
mkdir cd4pe-bolt-project cd cd4pe-bolt-project bolt project init cd4pe_bolt_project
Edit the bolt-project.yaml file and change the
modules: - name: puppetlabs/cd4peadm version_requirement: 5.0.0
- If you have not done so, update the module-install section to enable access to premium modules on the Puppet Forge.
Install the Bolt modules using the command:
bolt module install.
Create an inventory.yaml
file with two targets: One for the host on which Continuous
Delivery for PE 5.x is to be installed and another with
kubectlaccess to your Continuous Delivery for PE 4.x install. For example:
--- targets: - name: cd4pe-5-node uri: <IP address or DNS name> config: transport: ssh ssh: host-key-check: false user: root - name: cd4pe-4-node uri: <IP address or DNS name> config: transport: ssh ssh: host-key-check: false user: root
Create hiera-eyaml keys to encrypt data using the command:
bolt secret createkeys.
Ensure that you do not check-in/commit the private key the command generates into VCS. These keys are needed by anyone running the
cd4peadmBolt plans in the future.
Create a hiera.yaml file with the contents:
--- version: 5 defaults: datadir: data data_hash: yaml_data hierarchy: - name: 'common' lookup_key: eyaml_lookup_key options: pkcs7_private_key: keys/private_key.pkcs7.pem pkcs7_public_key: keys/public_key.pkcs7.pem paths: - 'common.yaml'
Run the install plan using:
bolt plan run cd4peadm::install.
This plan gives you option to either do a new install or to migrate data from a 4.x instance.
Select the migrate option and follow the prompts. The plan extracts your 4.x
kubectl, installs Continuous Delivery for PE 5.x with those settings, and migrates data from the 4.x database to the new 5.x database.Note: Migrating the database data creates a database dump in
/tmpon the host that is running
kubectl. Please ensure that there is sufficient space for that data on the host. You can check the size of your database using the command:
kubectl exec -it postgres-0 -- du -skh /var/lib/postgresql/data/pgdata
Test the new 5.x installation
You must thoroughly test the 5.x installation to make sure the data was properly migrated and all functionality is working as expected.
Test the data migration to make sure your 4.x installation's data is present in
your 5.x installation. For each workspace in your 5.x installation:
- Review event logs for all control repos and modules.
- Check that all jobs and hardware connections are present.
- Check that all users and user groups are present.
- If any data is missing or looks incorrect, rerun the migration task.
- Test manual pipeline runs to make sure your 5.x installation is functioning properly. For each workspace in your 5.x installation, trigger a manual pipeline run for all control repos and modules. To do this, select Trigger pipeline from the Manual actions menu above each pipeline.
As the final step in the 4.x to 5.x data migration process, update the webhooks that connect Continuous Delivery for Puppet Enterprise (PE) to your source control system.
- In the Continuous Delivery for PE 4.x root console, navigate to and copy the backend service endpoint.
- In the Continuous Delivery for PE 5.x root console, click , and enter the backend service endpoint you copied from the 4.x root console.
- Click Update webhooks.
Post-migration guidance and troubleshooting
After migrating Continuous Delivery for Puppet Enterprise (PE) data from 4.x to 5.x, thouroughly testing your 5.x installation, and updating webhooks, you're ready to begin using 5.x with your team. However, do not immediately decommission your 4.x installation. You may need to roll back to 4.x if issues arise.
Maintaining your dormant 4.x installation for an extended test period allows you to roll back to 4.x and prevent production environment disruptions if an issue arises with your 5.x migration.
Rolling back to 4.x
- In your Continuous Delivery for PE 4.x installation's web UI, navigate to a control repo's pipeline and click Manage pipelines.
- Select Manage in the web UI and locate the Automation webhook section where the webhook (including token) for your control repo is shown.
- Copy the webhook and navigate to the corresponding repository in your source control system. Add the 4.x webhook to the repository and remove the 5.x webhook.
- Repeat this process for each control repo and module repo in your 4.x installation.
- Generate a new access token for each PE instance connected to the 4.x installation. In the Continuous Delivery for PE web UI, navigate to and click .
- Select Basic authorization or API
token and enter the required information:
- For Basic authorization, enter the username and password for your "Continuous Delivery" user. Continuous Delivery for PE uses this information to generate an API token for you. The username and password are not saved.
- For API token, generate a PE access token for your "Continuous
Delivery" user using
puppet-accessor the RBAC API v1 service, and paste it into the API token field. For information about generating PE access tokens and the RBAC API v1 service, see Token-based authentication in the PE documentation.Note: To avoid unintended service interruptions, create access tokens with a multi-year lifespans.
Once the 4.x webhooks and new PE access tokens are in place, you can resume using the 4.x installation.
When you're ready to return to 5.x, repeat the migration process, starting with Run the migration task.