Nepal on Rails

Build your own Vagrant box ready to use with chef-solo using Veewee

This is my first screencast and its a bit glitchy. The voice is not good because I’d to use the built-in microphone of my MacBook pro. I’ve to get a good one. So bare with me and the audio.

Things to shop

• Ruby versions: 1.9.3-p0, 1.9.2-p290, 1.8.7-p352
• VirtualBox 4.1.6
• VirtualBox 4.1.6 Extension pack
• Ubuntu 11.10 32 bit server image file

Assumptions

• You know what is Virtual Box and Vagrant
• You are familiar with Git and Github
• You use Bundler and RVM

Building a brand new box using Ubuntu 11.10 32-bit iso image

Visit https://github.com/jedi4ever/veewee and Fork the repo under your account

Git clone git clone git://github.com/millisami/veewee cd veewee

Bundle install (if you don’t have Bundler installed, do gem install bundler --pre)

millisami at sachin in ~/gitcodes/veewee on master
± bundle install                                                                                                                                                                                                 
Fetching gem metadata from http://rubygems.org/.......
Using Platform (0.4.0) 
Using archive-tar-minitar (0.5.2) 
...
Using bundler (1.1.rc) 
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
millisami at sachin in ~/gitcodes/veewee on master

Gems required

gem install vagrant
gem install veewee

Define a basebox template

First, lets see what are the templates available.

millisami at sachin in ~/gitcodes/veewee on master
± bundle exec vagrant basebox templates                                                                                                                                                             
The following templates are available:
...
vagrant basebox define '<boxname>' 'ubuntu-10.10-server-i386-netboot'
vagrant basebox define '<boxname>' 'ubuntu-11.04-server-amd64'
vagrant basebox define '<boxname>' 'ubuntu-11.04-server-i386'
...

So, the ubuntu 11.10 has not been defined yet. Lets build one based off the ubuntu-11.04-server-i386

millisami at sachin in ~/gitcodes/veewee on master
± bundle exec vagrant basebox define 'milli-ubuntu-11-10-32-bit' 'ubuntu-11.04-server-i386'                                                                                                         
The basebox 'milli-ubuntu-11-10-32-bit' has been successfully created from the template ''ubuntu-11.04-server-i386'
You can now edit the definition files stored in definitions/milli-ubuntu-11-10-32-bit
or build the box with:
vagrant basebox build 'milli-ubuntu-11-10-32-bit'

Lets see the structure of the generated folder

millisami at sachin in ~/gitcodes/veewee on master
± tree definitions                                                                                                                                                                                  
definitions
└── milli-ubuntu-11-10-32-bit
    ├── definition.rb
    ├── postinstall.sh
    └── preseed.cfg

1 directory, 3 files

Open up the folder in your editor

Open up the file definition.rb and change the version numbers from 11.04 to 11.10

# definition.rb
Veewee::Session.declare({
  :cpu_count => '1', :memory_size=> '384', 
  :disk_size => '10140', :disk_format => 'VDI', :hostiocache => 'off',
  :os_type_id => 'Ubuntu',
  :iso_file => "ubuntu-11.10-server-i386.iso",
  :iso_src => "http://releases.ubuntu.com/11.10/ubuntu-11.10-server-i386.iso",
  :iso_md5 => "881d188cb1ca5fb18e3d9132275dceda",
  :iso_download_timeout => "1000",
  :boot_wait => "10", :boot_cmd_sequence => [
    '<Esc><Esc><Enter>',
    '/install/vmlinuz noapic preseed/url=http://%IP%:%PORT%/preseed.cfg ',
    'debian-installer=en_US auto locale=en_US kbd-chooser/method=us ',
    'hostname=%NAME% ',
    'fb=false debconf/frontend=noninteractive ',
    'keyboard-configuration/layout=USA keyboard-configuration/variant=USA console-setup/ask_detect=false ',
    'initrd=/install/initrd.gz -- <Enter>'
  ],
  :kickstart_port => "7122", :kickstart_timeout => "10000", :kickstart_file => "preseed.cfg",
  :ssh_login_timeout => "10000", :ssh_user => "vagrant", :ssh_password => "vagrant", :ssh_key => "",
  :ssh_host_port => "7222", :ssh_guest_port => "22",
  :sudo_cmd => "echo '%p'|sudo -S sh '%f'",
  :shutdown_cmd => "shutdown -P now",
  :postinstall_files => [ "postinstall.sh"], :postinstall_timeout => "10000"
})

Open up the file postinstall.sh and change libreadline5-dev to libreadline5 because the Ubuntu Oneiric doesn’t have libreadline5-dev package.

millisami at sachin in ~/gitcodes/veewee on master
± be vagrant basebox build 'milli-ubuntu-11-10-32-bit'

Setup GitLab (Github clone) using Vagrant and Chef

Update: Nov 28, 2011: The box that is downloaded via vagrant might not work properly. Here is the box that I’d custom built to have chef pre-installed. You can download it here http://api2.ge.tt/0/8gA4QSA/0/blob/download

This post is Part 1 of the series Rails app with distributed deployment playground

Here we’ll be installing GitlabHQ.

I won’t be showing on how to install Vagrant with VirtualBox here. If you’re not familiar with Vagrant and how to set it up, then first watch the Railscasts 292 and install the Vagrant with VirtualBox.

Then you can start with the following commands in the terminal. First of all, we have to have our own chef-repo to contain the cookbooks.

You can clone my repo or you can use your own or others. Here I’ll be using mine chef-repo

cd
git clone git://github.com/millisami/chef-repo.git

Now, lets start with the gitlabhq setup.

cd
mkdir vagrants
cd vagrants
git clone git://github.com/gitlabhq/gitlabhq.git
cd gitlabhq
vagrant init

Modify the Vagrantfile as below

Vagrant::Config.run do |config|
  config.vm.box = "lucid32"
  config.vm.network "33.33.33.20"

  config.vm.customize do |vm|
    vm.name = "GitlabHQ"
    vm.memory_size = 384
  end

  config.vm.provision :chef_solo do |chef|
    chef.log_level = :debug
    chef.cookbooks_path = ["~/chef-repo/cookbooks"]
    chef.add_recipe "base"
    chef.add_recipe "rvm::install"
    chef.add_recipe "rvm::ruby_192"

    chef.json = { 
      :base => {
        :system_packages => ["tree", "htop", "vim-nox"]
        }
      }
    }
  end
end

Then from the terminal, run the following commands.

vagrant up
vagrant ssh

Once you login, install these dependencies.

sudo apt-get install -y git-core sqlite3 libsqlite3-dev gitosis libcurl4-openssl-dev python-setuptools

sudo visudo

Add the following line at the VERY BOTTOM of the file, if not at the end, it can be nullified by later entries

vagrant ALL=(ALL) NOPASSWD: ALL

Ctrl-X to leave, Y to save your changes. Logout logout and ssh vagrant ssh again to reflect the effect.

Lets install the gem dependencies.

cd /vagrant
rvmsudo gem install bundler --pre
bundle

# Seeding the database
RAILS_ENV=production bundle exec rake db:setup
RAILS_ENV=production bundle exec rake db:seed_fu

Write the admin account and its password which we’ve to use later while accessing gitlab.

...
Administrator account created:

login.........admin@local.host
password......5iveL!fe

Then, we have to setup the following:

sudo easy_install pygments
sudo echo 'export RAILS_ENV=production' >> ~/.bash_profile
sudo adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git
ssh-keygen -t rsa
sudo -H -u git gitosis-init < ~/.ssh/id_rsa.pub
sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update

echo "gem: --no-rdoc --no-ri" > ~/.gemrc
rvmsudo gem install passenger
rvmsudo passenger-install-nginx-module

Creating an Upstart file for nginx

sudo vim /etc/init/nginx.conf

description "Nginx HTTP Server"

start on filesystem
stop on runlevel [!2345]

respawn

exec /opt/nginx/sbin/nginx -g "daemon off;"

Restart the VM

sudo shutdown -r now

Verify nginx urnning

sudo initctl list | grep nginx

You should see:

nginx start/running, process 678

…Or use a browser and visit http://33.33.33.20

Open nginx.conf to configure nginx

sudo vim /opt/nginx/conf/nginx.conf

Change the user to ‘vagrant’

Then inside the http {....}, add/replace a server block looking similar to this:

...
http {
  ...

    server {
    listen 80;
    server_name localhost;
    root /vagrant/public;
    passenger_enabled on;
    }
    ...
}

Restart nginx to pick up the changes

sudo service nginx stop; sudo service nginx start

or just

sudo start|stop nginx

Setting up the first repository

1. Access http://33.33.33.20 on the HOST machine’s browser
2. Login in using admin@local.host and 5iveL!fe
3. Add your HOST key at http://33.33.33.20/keys
4. Create the first project, say (Myproject)
5. Add a remote. git remote add vagrant-chef git@33.33.33.20:Myproject.git
6. Push with tracking branch. git push -u vagrant-chef master

Dealing with the Git and SSH keys gotchas

1. Added user vagrant to group git since /home/git/repositories directory is owned by git

2. Did one clone in ~/tmp directory git clone git@localhost:gitosis-admin.git gitosis-copy to add the RSA key to the list of known hosts. So, that when later via browser, the webserver will try to create a new repo. Otherwise you might end up with the infamous 500 error page.

Thats it for this part. Hang-on for the part 2 of the series Rails app with distributed deployment playground

Rails app with distributed deployment playground

While setting up Etxpress on the production box with system wide installed RVM and Resque for background processing, monitored via Upstart using Foreman export command, I had to run the following command to export the updated Procfile.production file with capistrano and custom cap deploy:foreman task.

rvmsudo foreman export upstart /etc/init -a etxpress -u deploy \ 
        -f ./Procfile.production -c worker=1 scheduler=1 redis=1 

Since the command rvmsudo is for sudoers, it asks for sudo password which capistrano cannot handle. Similarly, there are some other commands that need the sudo access as well. So, tweaking the settings directly in production and playing with cap deploy:foreman and other sudo related tasks, I’d to manually login and do service restarts.

This is not a best practice to test or play with deployment settings directly on the production box. So, to simulate the same environment and process, here the Vagrant and Chef come in.

The idea is to host a Gitlab as a Github repo in one VM, launch another VM to simulate the production box and setup the multistage capistrano recipe to test and experiment.

It will be a long post. So, I’ll be blogging in the following series:

1. Setup Gitlab as a Github repo hosting
2. Setup a production box in another VM
3. Setup capistrano multistage to test the deployment

To automate the process, we’ll be using vagrant to launch VM(s) and chef-solo for provisioning. I might do another post using chef-server to make it more automated.

Without further ado, here is the first part of the series: Setup Gitlab as a Github repo hosting using vagrant and chef-solo.