Migrate 4.x data to 5.x

The Continuous Delivery for PE 5.x series uses Bolt for installation and management rather than Puppet Application Manager, so you must migrate your 4.x series data to the new installation.

What is migrated?

The migration task copies and migrates to 5.x:

The database, which contains all your Continuous Delivery for PE 4.x installation's information, users, integrations, history, and artifacts (except job commands and job logs).
The object store, which contains all of your 4.x installation's job commands and job logs.

Any of these configuration settings, if in use on 4.x:

CD4PE_REPO_CACHE_RETRIEVAL_TIMEOUT_MINUTES
CD4PE_LDAP_GROUP_SEARCH_SIZE_LIMIT
CD4PE_ENABLE_REPO_CACHING
CD4PE_HTTP_CONNECTION_TIMEOUT_SEC
CD4PE_HTTP_READ_TIMEOUT_SEC
CD4PE_HTTP_WRITE_TIMEOUT_SEC
CD4PE_HTTP_REQUEST_TIMEOUT_SEC
CD4PE_INCLUDE_GIT_HISTORY_FOR_CD4PE_JOBS
CD4PE_JOB_GLOBAL_TIMEOUT_MINUTES
CD4PE_JOB_HTTP_READ_TIMEOUT_MINUTES

The migration task does not copy or move to 5.x:

The 4.x root user, because you create a new root user when installing 5.x.
The JVM_ARGS configuration setting, if in use on 4.x.
These backup tables created for you by Continuous Delivery for PE during the 4.x series:
- app-pipelines-3-0-backup.pfi, which was created during the upgrade from 3.x to 4.x
- pe-ia-node-results-cve-2020-7944-backup.pfi and pe-ia-resource-results-cve-2020-7944-backup.pfi, which were created as part of the CVE 2020-7944 remediation in version 3.4.0

Migration cautions

A super user or the root user must perform this process.

Migrations from installations with external databases are not supported.

Important: Once you initiate the migration task, any new data created in the 4.x installation may not be migrated to the 5.x installation. To avoid losing data, restrict access to Continuous Delivery for PE during migration.

Run the migration task

The migration task uses the cd4peadm::install plan to migrate data from existing 4.x PAM-based installation to a new 5.x install.

Before you begin

Unlike version 4.x, Continuous Delivery for Puppet Enterprise (PE) 5.x uses Bolt for installation, configuration, and administration instead of PAM.

Before migrating, please make sure the system you plan to install Bolt on has internet access as well as SSH access to both the system on which you intend to install Continuous Delivery for PE 5.x as well as the existing 4.x installation from which you are migrating. In addition, because Bolt is installed on one or more systems and used to manage one centralized Continuous Delivery for PE installation, it may be a good idea to maintain the Bolt project in its own VCS repo.

Once you've completed the above preparations, you're ready to run the migration task:

Install Bolt version 3.27.2 or later on a jumphost. This can be the intended Continuous Delivery for PE 5.x host, or any other system.

Create the Continuous Delivery for PE Bolt project and switch to that directory.

mkdir cd4pe-bolt-project 
cd cd4pe-bolt-project
bolt project init cd4pe_bolt_project

Edit the bolt-project.yaml file and change the modules section to:

modules:
  - name: puppetlabs/cd4peadm
    version_requirement: 5.0.0

If you have not done so, update the module-install section to enable access to premium modules on the Puppet Forge.
Install the Bolt modules using the command: bolt module install.

Create an inventory.yaml file with two targets: One for the host on which Continuous Delivery for PE 5.x is to be installed and another with kubectl access to your Continuous Delivery for PE 4.x install. For example:

---
targets:
- name: cd4pe-5-node
  uri: <IP address or DNS name>
  config:
    transport: ssh
    ssh:
      host-key-check: false
      user: root
- name: cd4pe-4-node
  uri: <IP address or DNS name>
  config:
    transport: ssh
    ssh:
      host-key-check: false
      user: root

Create hiera-eyaml keys to encrypt data using the command: bolt secret createkeys.

Ensure that you do not check-in/commit the private key the command generates into VCS. These keys are needed by anyone running the cd4peadm Bolt plans in the future.

Create a hiera.yaml file with the contents:

---
version: 5

defaults:
  datadir: data         
  data_hash: yaml_data

hierarchy:
  - name: 'common'
    lookup_key: eyaml_lookup_key
    options:
      pkcs7_private_key: keys/private_key.pkcs7.pem
      pkcs7_public_key:  keys/public_key.pkcs7.pem
    paths:
      - 'common.yaml'

Run the install plan using: bolt plan run cd4peadm::install.

This plan gives you option to either do a new install or to migrate data from a 4.x instance.
Select the migrate option and follow the prompts. The plan extracts your 4.x settings via kubectl, installs Continuous Delivery for PE 5.x with those settings, and migrates data from the 4.x database to the new 5.x database.
Note: Migrating the database data creates a database dump in /tmp on the host that is running kubectl. Please ensure that there is sufficient space for that data on the host. You can check the size of your database using the command:
```
kubectl exec -it postgres-0 -- du -skh /var/lib/postgresql/data/pgdata
```

Results

You can now log into the application at the resolvable hostname you provided during the installation, using the same root username as your 4.x install and the new root password you supplied.

Test the new 5.x installation

You must thoroughly test the 5.x installation to make sure the data was properly migrated and all functionality is working as expected.

Test the data migration to make sure your 4.x installation's data is present in your 5.x installation. For each workspace in your 5.x installation:
1. Review event logs for all control repos and modules.
2. Check that all jobs and hardware connections are present.
3. Check that all users and user groups are present.
If any data is missing or looks incorrect, rerun the migration task.
Test manual pipeline runs to make sure your 5.x installation is functioning properly. For each workspace in your 5.x installation, trigger a manual pipeline run for all control repos and modules. To do this, select Trigger pipeline from the Manual actions menu above each pipeline.

What to do next

Once you've confirmed your 4.x data is present in your 5.x installation and all manual pipeline runs succeed, Update webhooks.

Update webhooks

As the final step in the 4.x to 5.x data migration process, update the webhooks that connect Continuous Delivery for Puppet Enterprise (PE) to your source control system.

Follow these steps if you're updating webhooks after migrating data from 4.x to 5.x. There are separate instructions to Update webhooks in other scenarios.

In the Continuous Delivery for PE 4.x root console, navigate to Settings > Endpoints and copy the backend service endpoint.
In the Continuous Delivery for PE 5.x root console, click Settings > Webhooks, and enter the backend service endpoint you copied from the 4.x root console.
Click Update webhooks.

Results

Continuous Delivery for PE updates all webhooks to point to the current installation.

CAUTION: After migrating data from 4.x to 5.x, once you update webhooks to point to your 5.x installation, do not create new data in the 4.x installation, and do not run the migration task again.

Post-migration guidance and troubleshooting

After migrating Continuous Delivery for Puppet Enterprise (PE) data from 4.x to 5.x, thouroughly testing your 5.x installation, and updating webhooks, you're ready to begin using 5.x with your team. However, do not immediately decommission your 4.x installation. You may need to roll back to 4.x if issues arise.

Important: Keep your 4.x installation intact until you are fully confident that all features of the new 5.x installation are working as expected. As a precaution, we strongly advise maintaining your inactive 4.x installation for the first two months of using 5.x.

Maintaining your dormant 4.x installation for an extended test period allows you to roll back to 4.x and prevent production environment disruptions if an issue arises with your 5.x migration.

Rolling back to 4.x

If necessary, you can temporarily roll back to your 4.x installation by switching the active webhooks from the 5.x installation to the 4.x installation and generating new access tokens for all PE instances connected with the 4.x installation.

In your Continuous Delivery for PE 4.x installation's web UI, navigate to a control repo's pipeline and click Manage pipelines.
Select Manage in the web UI and locate the Automation webhook section where the webhook (including token) for your control repo is shown.
Copy the webhook and navigate to the corresponding repository in your source control system. Add the 4.x webhook to the repository and remove the 5.x webhook.
Repeat this process for each control repo and module repo in your 4.x installation.
Generate a new access token for each PE instance connected to the 4.x installation. In the Continuous Delivery for PE web UI, navigate to Settings > Puppet Enterprise and click More actions > Edit integration.
Select Basic authorization or API token and enter the required information:
- For Basic authorization, enter the username and password for your "Continuous Delivery" user. Continuous Delivery for PE uses this information to generate an API token for you. The username and password are not saved.
- For API token, generate a PE access token for your "Continuous Delivery" user using puppet-access or the RBAC API v1 service, and paste it into the API token field. For information about generating PE access tokens and the RBAC API v1 service, see Token-based authentication in the PE documentation.
  Note: To avoid unintended service interruptions, create access tokens with a multi-year lifespans.

Once the 4.x webhooks and new PE access tokens are in place, you can resume using the 4.x installation.

When you're ready to return to 5.x, repeat the migration process, starting with Run the migration task.