The World’s Largest Stock Exchange Trusts Puppet to Provision 20,000+ Servers

The New York Stock Exchange (NYSE) is the world’s largest stock exchange by market capitalization. In 2015, it raised $137 billion in capital, including $6 billion from tech IPOs, making NYSE the U.S. leader in tech IPOs. NYSE is part of Intercontinental Exchange (ICE), which owns 23 exchanges and marketplaces, as well as five central clearinghouses.

Benefits of Using Puppet:

• 75% of ICE’s 20,000 servers are managed by Puppet Enterprise (as of March 2016).
• Increase from 300 servers per admin to 700 servers per admin.
• Provisioning dev environments reduced from 1 or 2 days to just 21 minutes.
• No workarounds needed: Puppet Enterprise “just works.” 

Challenge: Securely Scale — And Make It Cost-Effective  

• Needed to scale up workloads and infrastructure efficiently and cost effectively.
• Provisioning was too slow and held up developers’ work.
• ICE ran infrastructure on a mix of platforms.
• ICE was mostly satisfied with Open Source Puppet — but relied on some workarounds.

NYSE was an early adopter of Puppet Enterprise. ICE, which relied on Open Source Puppet for managing its thousands of servers, acquired NYSE in 2013. It soon became apparent that PE would help the newly combined engineering teams merge more quickly and effectively.

That realization led to migrating other exchanges, marketplaces and clearinghouses within ICE from Open Source Puppet to Puppet Enterprise. Two years after deciding to go with Puppet Enterprise, 75 percent of ICE’s 20,000-server estate is managed with Puppet.

Results: One Platform with Regulatory Compliance and Security Assurance

“The more we use Puppet Enterprise, the more we find we can do with it."

Pope Davis, Senior Director of Systems Engineering, ICE

“Puppet Enterprise comes with many modules to help us manage our Puppet setup more easily. The open source version does not come with these modules, and those we did use, we needed to modify. We don’t need to modify the modules that ship with Puppet Enterprise, especially with PE 2015, and that saves us a lot of time.”

Puppet Enterprise’s continuing development has brought new capabilities — for example, improved performance and scalability — to ICE and NYSE. “Every new upgrade of Puppet proves to be more innovative, and brings exciting features that we want,” Pope said.

As ICE grows its different business lines, it keeps adding more computing resources and more complexity. But ICE also works to keep costs in check.

“We’ve been able to throw in additional workloads, servers and environments without growing staff at the same pace,” said Pope. “Our server-to-admin ratio has never gone down.”

One Admin, 700 Servers  

Several years ago, the organization was glad that one admin could run 300 servers. Today, some teams are able run 700 or 800 servers per engineer in the ICE trading system, because they’re puppetized. “We’re proud of this,” said Pope.

Another efficiency is being able to quickly replace malfunctioning servers, whether physical or virtual. “We don’t troubleshoot systems anymore,” Pope said. “If a server is flaky or slow, we just rebuild it. Whatever machine gets pulled out and replaced at night, we’re confident it will be identical to the others in the farm, and will perform as expected, because there’s no manual touch.”

ICE’s infrastructure runs on a mix of different platforms, including Red Hat Enterprise Linux, Oracle Enterprise Linux, Solaris x86, and AIX. All of these are managed with Puppet. The next potential steps will be to expand Puppet Enterprise to all Windows machines and to ICE’s VMware infrastructure.

Meeting Regulatory Compliance and Security Assurance

NYSE and the rest of the ICE exchanges are subject to many regulatory requirements, and the organization is able to meet many of these with Puppet.  

“It’s a great tool that demonstrates our commitment to automation and consistency to auditors,” Pope said.

“They [auditors] often ask how we can ensure a server meets its goals, and we say, ‘We run Puppet at build time, and on a consistent basis.’ They are excited that we’re using a tool that enforces consistency.”

ICE also relies on Puppet to push out security changes quickly, in a consistent and orderly fashion.

From 48 Hours to 20 Minute Server Builds

One reason organizations turn to DevOps is to make it easier and faster for developers to get the environments they need. Puppet plays a big role in helping ICE’s IT team do that.

“We are providing a lot of DevOps functionality,” Pope said. “We’ve gone from taking 24 to 48 hours to set up a development environment to where a typical server build now takes 20 minutes. There’s a setup for each type of server — A, B or C, whichever flavor the development team needs, they know we will be able to deliver. Every 15 minutes they get back in their day, they can roll into more development work.”
 

Put Puppet to Work on Your Servers

See for yourself what Puppet can do for you. Try it today or request a demo with the Puppet team:

TRY PUPPET ENTERPRISE    DEMO PUPPET