Excellent video specifically for PostgreSQL on AWS, however the principles are pretty universal information for running anything on AWS -
yow, the latest OpenStack release is looking rather slick..
I've been working pretty extensively with Xen and Puppet in my new job, really loving it! I've been creating a whole load of Xen hosts, most of which are cloned from an initial image I built using Xen-tools. I've just finished a script which is over on my github page, which basically automates what was previously a manual process.
Basically, it copies your existing disk.img and swap.img, generates a new xen.cfg file based on some interactive input (desired hostname, IP, memory and number of vCPUs) plus a random Xen mac address, then mounts the disk.img file and changes some appropriate system files - /etc/hostname, hosts, and network/interfaces.
All quite simple and straight forward, but quite nice to have automated.
Here's the README:
A script for automating Xen VM deployment.
It requires that you have a base disk.img and swap.img already created.
I created mine with:
xen-create-image -pygrub -size=50Gb -swap=9Gb -vcpus=2 -memory 6Gb -dist=squeeze -dhcp -passwd -dir=/var/virt-machines -hostname=xen-squeeze-base
Fill in some of the variables at the top of GenXen.pl before running, then simply:
The interactive part will ask for hostname, memory size, vCPUs, IP address, then generate a unique Xen mac address, and write these all to a xen config file which will be saved in /etc/xen/
It'll copy your disk.img and swap.img to destination dir, mount the disk.img and create appropriate files for:
After that you should be good to launch with:
xm create -c /etc/xen/whatever-your-hostname-is.cfg
Vagrant is a framework for building and deploying Virtual Machine environments, using Oracle VirtualBox for the actual VMs and utilizing Chef for configuration management.
Watching through this intro video:
i was quite intrigued as it is very similar to what i was looking to achieve earlier when i was experimenting with installing Xen and configuring with Puppet.
So here's what I experienced during the setup of Vagrant on my Macbook - I decided to start with a simple Chef install to familiarise myself with Chef itself and it's own requirements CouchDB, RabbitMQ and Solr, mostly by following these instructions -
sudo gem install chef
sudo gem install ohai
Chef uses couchDB as it's datastore, so we need to install it using the instructions here
brew install couchdb
The instructions I list above also contains steps to install a couchDB user and set it up as a daemon. They didn't work for me, and after 30mins of troubleshooting, i gave up and went with the simpler option of running it under my own user - in production this will be running on a Linux server rather than my Macbook, so it seemed fair enough -
cp /usr/local/Cellar/couchdb/1.1.0/Library/LaunchDaemons/org.apache.couchdb.plist ~/Library/LaunchAgents/
launchctl load -w ~/Library/LaunchAgents/org.apache.couchdb.plist
Check its running okay by going to
which should provide something akin to :
- INSTALL RABBITMQ -
brew install rabbitmq
sudo rabbitmqctl add_vhost /chef
sudo rabbitmqctl add_user chef testing
sudo rabbitmqctl set_permissions -p /chef chef “.*” “.*” “.*”
Ok, Gettin' back to my mission, break out the whipped cream and the cherries, then I go through all the fly positions - oh, wrong mission!
brew install gecode
brew install solr
sudo gem install chef-server chef-server-api chef-server chef-solr
sudo gem install chef-server-webui
Setup a conf file -
sudo mkdir /etc/chef
sudo vi /etc/chef/server.rb - paste in the example from:
http://wiki.opscode.com/display/chef/Manual+Chef+Server+Configuration - making the appropriate changes for your FQDN
At this point, the above instructions ask you to start the indexer however the instructions haven't been updated to reflect changes to Chef version 0.10.2 in which chef-solr-indexer has been replaced with chef-expander
So, instead of running:
you instead need to run:
sudo chef-expander -n1 -d
Next i tried
which ran into
“`configure_chef': uninitialized constant Chef::Application::SocketError (NameError)”
i had to create an /etc/chef/solr.rb file and simply add this to the file:
startup now worked -
if you want to daemonize it, use:
sudo chef-solr -d
Next start Chef Server with:
sudo chef-server -N -e production -d
sudo chef-server-webui -p 4040 -e production
Now you should be up and running - you need to configure the command client ‘Knife' follwing the instructions here - under the section ‘Configure the Command Line Client‘
mkdir -p ~/.chef
sudo cp /etc/chef/validation.pem /etc/chef/webui.pem ~/.chef
sudo chown -R $USER ~/.chef
knife configure -i
(follow the instructions at the link - you only need to change the location of the two pem files you copied above)
Ok, so hopefully you're at the same place as me with this all working at least as far as being able to log into CouchDB, and verifying that Chef/Knife are both working.
- VAGRANT SETUP -
Now, onward with the original task of Vagrant setup…
Have a read over the getting started guide:
Install VirtualBox - download from http://www.virtualbox.org/wiki/Downloads
Run the installer, which should all work quite easily. Next..
gem install vagrant
this creates the base Vagrantfile, which the documentation compares to a Makefile, basically a reference file for the project to work with.
Setup our first VM -
vagrant box add lucid32 http://files.vagrantup.com/lucid32.box
This is downloaded and saved in ~/.vagrant.d/boxes/
edit the Vagrantfile which was created and change the “box” entry to be “lucid32″, the name of the file we just saved.
Bring it online with:
then ssh into with
Ace, that worked quite easily. After a little digging around, I logged out and tore the machine down again with
- TYING IT ALL TOGETHER -
Now we need to connect our Vagrant install with our Chef server
First, clone the Chef repository with:
git clone git://github.com/opscode/chef-repo.git
add this dir to your ~/.chef/knife.rb file
Download the Vagrant cookbook they use in their examples -
tar xzvf cookbooks.tar.gz
mv cookbooks/* chef-repo/cookbooks/
Add it to our Chef server using Knife:
knife cookbook upload -a
(knife uses the cookbook_path we setup above)
If you browse to your localhost at
you should see the three new cookbooks which have been added.
Now to edit Vagrantfile and add your Chef details:
Vagrant::Config.run do |config|
config.vm.box = "lucid32"
config.vm.provision :chef_client do |chef|
chef.chef_server_url = "http://SBD-IODA.local:4000"
chef.validation_key_path = "/Users/thorsten/.chef/validation.pem"
I tried to load this up with
“[default] [Fri, 05 Aug 2011 09:27:07 -0700] INFO: *** Chef 0.10.2 ***
[default] [Fri, 05 Aug 2011 09:27:07 -0700] INFO: Client key /etc/chef/client.pem is not present - registering
[default] [Fri, 05 Aug 2011 09:27:28 -0700] FATAL: Stacktrace dumped to /srv/chef/file_store/chef-stacktrace.out
[default] [Fri, 05 Aug 2011 09:27:28 -0700] FATAL: SocketError: Error connecting to http://SBD-IODA.local:4000/clients - getaddrinfo: Name or service not known”
I figured this was a networking issue, and yeah, within the VM it has no idea of my Macbook's local hostname, which i fixed by editing its /etc/hosts file and manually adding it.
Upon issuing a
vagrant reload, boom! you can see the Vagrant host following the recipes and loading up a bunch of things including apache2
However at this point, you can still only access it's webserver from within the VM, so in order to access it from our own desktop browser, we can add the following line to the Vagrantfile:
config.vm.forward_port(“web”, 80, 8080)
After another reload, you should now be able to connect to localhost:8080 and access your new VM's apache host.
In order to use this setup in any sort of dev environment will still need a good deal more work, but for the moment, this should be enough to get you up and running and able to explore both Vagrant and Chef.