Limitations of impact analysis

Some code changes may impact your infrastructure beyond what an impact analysis report displays. The technical limitations of impact analysis reports are outlined here.

We've designed impact analysis to give you helpful information about the potential results of code changes to your infrastructure. However, impact analysis is not designed to be an exhaustive report — do not use it as a substitute for more exhaustive testing. For instance, there are certain circumstances in which we can't reliably calculate the full impact of a code change. As an impact analysis user, it's critical that you understand that these limitations exist, and that you remain alert to the possibility that code changes might have consequences for your infrastructure beyond what impact analysis shows.

The following is a non-exhaustive list of some of the limitations of impact analysis. This list might be updated from time to time, but is not complete.

  • Changes to an environment's environment.conf file. The downstream impact of changes to an environment.conf file cannot be analyzed.

  • Brand-new code. New Puppet classes, facts, or functions that have never before been applied to any of your nodes cannot be analyzed.

  • Changes to functions. The impact of changes to a function cannot be analyzed.

  • Changes to facts. The impact of changes to a fact cannot be analyzed.

  • Changes to class names used in classification. Changes to the names of classes that have been previously classified in the classifier cannot be analyzed. A classification update in the classifier is required before Puppet can locate the newly renamed class.

  • Imported resources. While impact analysis can provide information about nodes that export resources, the impact to nodes that receive those exported resources cannot be analyzed.

  • Sensitive data types. Any changes to data types marked as sensitive will not be analyzed.

  • Alias metaparameter. Any resources using the alias metaparameter cannot be analyzed.

  • Changes to Embedded Puppet templates. The impact of changes made to Embedded Puppet (.epp) or Embedded Ruby (.erb) template files alone cannot be analyzed.
Remember: Reports generated by impact analysis are provided "AS-IS."