Integrating AWS CloudFormation and Chef

The cloud ecosystem was a-thundering this Friday with Amazon Web Services’ announcement on CloudFormation. Much of it positive, and a whole…

The cloud ecosystem was a-thundering this Friday with Amazon Web Services’ announcement on CloudFormation. Much of it positive, and a whole lot said about AWS whacking at the partner ecosystem. The twitter pundits who probably haven’t used CloudFormation, Chef or Puppet were quick to claim that CloudFormation is a replacement. At initial glance I thought so as well, but then I looked under the covers and quickly realized that to the contrary, CloudFormation will actually simplify and make our recipes more portable.

How? Because we have cookbooks and a fair bit of code to set up and manage AWS resources. By abstracting these resources up to the CloudFormation level the recipes should be easier to structure across multiple cloud platforms. I am hoping the other cloud providers will follow the CloudFormation lead and create similar frameworks.

I think AWS will continue to innovate and encroach on the tools and services created in its ecosystem, but CloudFormation isn’t one that invalidates configuration management tools. The best response to the twitter-trigger happy “experts” out there came from John Vincent.

But on the practical level how do we integrate the two frameworks? It actually turned out to be pretty straightforward. I had done something similar when we were moving from RightScale to Chef. Essentially, we need to pass the bootstrap as part of the launch configuration. By design our Centos JeOS images execute a script that’s passed in via the UserData EC2 metadata. Here is our rc.local:

To make things simple on myself, I changed the simple ec2 instance template provided by amazon into an erb template and wrote a little ruby snippet to generate the CloudFormation template using this template and our existing bootstrap erb templates.

The Ruby Script

The CloudFormation ERB Template

The Bootstrap ERB Template

Now to run it, the ruby code generates a CF template that can be uploaded to CloudFormation:

And viola!

Now on to moving aws resources out of the recipes into the CloudFormation template

Originally published by Rohit Amarnath at full360.com on February 27, 2011.