Scaling Configuration Management Through Inventories at Fannie Mae

Alex Zbarcea, DevOps Engineer III, walks through the patterns and lessons he and his team at Fannie Mae learned from managing a +10K node Puppet infrastructure in a highly regulated industry. In this talk, he discusses how principles of traditional supply chain (in manufacturing), helped scale when dealing with heterogeneous infrastructure (physical/VMs and cloud), multiple operating systems, package managers, services, and multiple teams with Puppet.

Challenge

Fannie Mae, a financial service provider, needed to scale configuration management for a 10K+ node infrastructure and be able to continuously monitor the health of services.

Solution

Fannie Mae used Puppet to scale configuration management for their 10K+ node heterogeneous infrastructure while working in a highly regulated industry.

Results

• One Puppet environment used for configuration management for entire infrastructure
• 11,000+ nodes managed by Puppet
• 50 services (including 20 core) managed by Puppet
• Transparency into health of services
• Multiple teams use infrastructure-as-code to collaborate through Puppet

Puppet enforces you to have accountability for each individual artifact that you are going to manage in production, nothing is being missed, nothing is being not tracked. This is why we think Puppet brings a lot of health into configuration management, because you will not have silos that are not transparent to other teams about what is the lifecycle of each individual artifact for services that are dependent for other services.

Puppet is great at answering these questions: Are the services configured as they should be? Did anything change? Is there a conflict between configuration processes? Puppet brings consistency, and thus predictability into the environment. So when you have to upgrade a service it’s much easier.

Alex Zbarcea, DevOps Engineer III, Fannie Mae