NYSE and ICE Case Study

 

 

Industry

Financial services

Background

The New York Stock Exchange (NYSE) is the world’s largest stock exchange by market capitalization. In 2015, it raised $137 billion in capital, including $6 billion from tech IPOs, making NYSE the U.S. leader in tech IPOs. NYSE is part of Intercontinental Exchange (ICE), which owns 23 exchanges and marketplaces, as well as five central clearinghouses.

Challenge

  • Needed to scale up workloads and infrastructure efficiently and cost-effectively.
  • Provisioning was too slow and held up developers’ work.
  • ICE was mostly satisfied with open source Puppet, but relied on some workarounds.

Solution

  • Expand Puppet Enterprise (PE) use from NYSE to other systems within ICE.

Results

  • 75% of ICE’s 20,000 servers are managed by Puppet Enterprise (as of March 2016).
  • Increase from 300 servers per admin to 700 servers per admin.
  • Provisioning dev environments reduced from 1 or 2 days to just 21 minutes.
  • No workarounds needed: Puppet Enterprise “just works.”

From Open Source Puppet to Puppet Enterprise

NYSE was an early adopter of Puppet Enterprise. ICE, which relied on open source Puppet for managing its thousands of servers, acquired NYSE in 2013. It soon became apparent that PE would help the newly combined engineering teams merge more quickly and effectively.

That realization lead to migrating other exchanges, marketplaces and clearinghouses within ICE from open source Puppet to Puppet Enterprise. Two years after deciding to go with Puppet Enterprise, 75 percent of ICE’s 20,000-server estate is managed with Puppet.

“The more we use Puppet Enterprise, the more we find we can do with it,” said Pope Davis, senior director of systems engineering at ICE. “Puppet Enterprise comes with many modules to help us manage our Puppet setup more easily. The open source version does not come with these modules, and those we did use, we needed to modify. We don’t need to modify the modules that ship with Puppet Enterprise, especially with PE 2015, and that saves us a lot of time.”

Puppet Enterprise’s continuing development has brought new capabilities — for example, improved performance and scalability — to ICE and NYSE. “Every new upgrade of Puppet proves to be more innovative, and brings exciting features that we want,” Pope said.

Growing Infrastructure Faster than Costs

As ICE grows its different business lines, it keeps adding more computing resources and more complexity. But ICE also works to keep costs in check.

“We’ve been able to throw in additional workloads, servers and environments without growing staff at the same pace,” said Pope. “Our server-to-admin ratio has never gone down.”

Several years ago, the organization was glad that one admin could run 300 servers. Today, some teams are able run 700 or 800 servers per engineer in the ICE trading system, because they’re puppetized. “We’re proud of this,” Pope said.

Another efficiency is being able to quickly replace malfunctioning servers, whether physical or virtual. “We don’t troubleshoot systems anymore,” Pope said. “If a server is flaky or slow, we just rebuild it. Whatever machine gets pulled out and replaced at night, we’re confident it will be identical to the others in the farm, and will perform as expected, because there’s no manual touch.”

One platform to manage a heterogeneous infrastructure

ICE’s infrastructure runs on a mix of different platforms, including Red Hat Enterprise Linux, Oracle Enterprise Linux, Solaris x86, and AIX. All of these are managed with Puppet. The next potential steps will be to expand Puppet Enterprise to all Windows machines and to ICE’s VMware infrastructure.

Regulatory compliance and security assurance

NYSE and the rest of the ICE exchanges are subject to many regulatory requirements, and the organization is able to meet many of these with Puppet. “It’s a great tool that demonstrates our commitment to automation and consistency to auditors,” Pope said.

“They [auditors] often ask how we can ensure a server meets its goals, and we say, ‘We run Puppet at build time, and on a consistent basis.’ They are excited that we’re using a tool that enforces consistency.”

ICE also relies on Puppet to push out security changes quickly, in a consistent and orderly fashion.

DevOps

One reason organizations turn to DevOps is to make it easier and faster for developers to get the environments they need. Puppet plays a big role in helping ICE’s IT team do that.

“We are providing a lot of DevOps functionality,” Pope said. “We’ve gone from taking 24 to 48 hours to set up a development environment to where a typical server build now takes 20 minutes. There’s a setup for each type of server — A, B or C, whichever flavor the development team needs, they know we will be able to deliver. Every 15 minutes they get back in their day, they can roll into more development work.”

 

See for yourself what Puppet Enterprise can do for you.

TRY PUPPET ENTERPRISE

 

 

Pope Davis is the senior director of systems engineering at ICE, and leads the company’s global systems engineering teams. He has more than 12 years of experience in IT, including eight at ICE, where he’s passionate about improving efficiency and reducing mistakes through automation and standardization.