How ONE/NW Got AWS Up and running

From: Jon Baldivieso [mailto:jonb@onenw.org]
Sent: Wednesday, November 12, 2008 10:42 AM
To: Patrick Shaw
Cc: davidglick@onenw.org; Jesse Snyder; Patrick Tewson; David Averill
Subject: Re: Moving a site to Amazon Web Services and EC2

I believe David's already shared some of the upfront costs to ramping up with AWS, but fwiw, here is an outline (w/ funky formatting) of how we've structured this setup:

▼            Goals

          ▼  have a reusable generic Zope-hosting disk image that didn't have *any* server-specific data on it.

                •              including cron jobs, apache config, etc

          •    use a mounted EBS volume for persistent data

          •    make server restarts as easy as possible

          •    support multiple Zopes within their own production buildouts

▼            What we did

          ▼  created a 64-bit image (supports L and XL EC2 instances) with:

                ▼            basic dependencies

                      •        apache

                      •        python/PIL

                      •        svn

                      •        postfix

                •              a user account "zope"

                ▼            very basic apache config that

                      •        include ModProxy module and friends

                      •        told apache to also load all .cfg files from a specific location, which was going to be on our persistent volume

          ▼  set up an EBS volume to host all data and configuration

                •              /usr/local/zope (home dir for Zope user)

                ▼            we used extended buildout recipes to capture misc other configuration, which was all under version control

                      •        bc we like to have a history of how our configuration evolves so we can see how it's changed and roll back if needed

                      •        we can emulate the entire production environment locally, if needed

                •              we used conventions to locate each buildout's apache config and cron jobs within parts of the buildout

                ▼            we set up a shell script that we call after an EC2 system reboot and after we've remounted the EBS volume

                      •        searches convention-based paths for new cron entries to compile a new crontab

                      •        starts Apache (bc we're including configuration from the EBS, this can't happen in a more normal rc.d way)

                      ▼      searches each buildout dir for a ec2-init.sh script and calls it

                            •  create a dir on the non-persistent AMI to host the buildout's log dir, bc you pay for disk IO on EBS volumes. (we're not sure about either the performance implications nor the cost-savings for this.)

                            •  start up the relevant services for the buildout (we use the supervisor recipe, so that really just means starting up that buildout's supervisor daemon)

          ▼  we have a suite of shell scripts that do normal maintenance that isn't specific to this architecture

                •              apache log parsing (we log to the non-persistent part of the AMI but parse the logs using AWStats and store to the persistent area)

                •              zope data backup (repozo) and purge of old backups

                •              ZODB packing

                •              log rotation

                •              the main AWS-specific automated task is snapshotting the EBS volume to S3