Deploying A Ruby On Rails Application 4.1 with Capistrano 3

Introduction

This walkthrough is based on tutorial Deploying A Ruby On Rails Application on the RackSpace Cloud.
There are some differences concerning versions of software packages used. Some
changes reflect my personal preferences. The principal changes is that we use
the latest (as at the time of writing) version of Capistrano.

Our software stack will be as follows.

Ubuntu version 13.04 LTS Raring Ringtail
Passenger with Nginx
Ruby version 2.1.2
Rails version 4.1.0
MySQL

Getting Started

Our App

Before we get to setting up a server and deployment, we’ll need an app. For this purpose, we’re going to use an empty Rails application.

First, on your development machine, create a Rails app named doozy (we’ll use
MySQL for dadatabase):

rails new doozy -d mysql

Then we’ll update our .gitignore file to include

/config/database.yml
/config/secrets.yml
/config/secrets*.rb

since we don’t want to check any passwords and sensitive information into Git.
Note that database.yml and secrets.yml should already be present now. We will
create and use secrets_deploy.rb as the helper for Capistrano script later in
this walkthrough.

Next, we’ll check our code into Git:

git init
git add .
git commit -am "initial commit"

For this tutorial, we will create a clone of our repo on deployment server.
Another common configuration is to deploy from a GitHub repository. The process is very similar - check out the documentation on Github for information on how to get started if that is your need.

Finally, lets create the development database:

bundle exec rake db:create

And start it up!

bundle exec rails server

You should see the familiar empty Rails app at http://localhost:3000

Rails Index

This is all we are going to do before we get to deployment. We recommend deploying your app to a production (or staging) server as early in your development schedule as possible. Much of the time, that means right away.

Setting and Securing Your Server

Server and development machine

For deployment low-traffic Rails app a pretty standard server configuration with 512MB and 20GB disk space is more than enough. In this tutorial we’ll be using
32-bit Raring Ringtail 14.04 LTS ubuntu distro on the server and similar flavor of
ubuntu on development machine which we would refer to as development laptop.

Securing the Server

From here on out, we will show a prompt to differentiate which terminal you are using. For your development laptop, we’ll use a local> prompt. For the production server, we will use the server> prompt.

Generating Your SSH Keys

Except for our initial login to set things up, we won’t be using passwords to connect to our sever. It is more secure and convenient to use SSH keys. But it requires a little setup. On your local machine:

local>cd
local>ssh-keygen -t rsa -C "you@example.com"

#Generating public/private rsa key pair.
#Enter file in which to save the key (/Users/you/.ssh/id_rsa):

local>[hit enter]

#Enter passphrase (empty for no passphrase):

local>[enter a passphrase]

#Enter same passphrase again:

local>[enter passphrase again]

At this point some files should be saved at ~/.ssh, including the file id_rsa.pub, which is your public key.

Logging into the Server

Your hosting provider may have supplied you with root password during the server creation, or with an user account (say admin) which has sudoer priveleges. In
latter case just skip the next section and jump to With a sudoer account stage.

With root password

When your server is ready, log into it as root. On your laptop enter the command:

local>ssh root@xxx.xxx.xxx.xxx

where xxx.xxx.xxx.xxx is the ipV4 address of your server.
You may get a warning stating that “The authenticity of the host…can’t be established”. Type ‘yes’ and continue - you’ll be prompted for a password. This is the root pass you got from your hosting provider during server instance creation.

Using root account for logging in is a bad practice. Let’s create another account which has sudoer rights. The main use of this account would be system maintenance, so we’ll call it admin.

server>adduser admin

You will be asked for some information. You can leave it blank (except for the password). This command will also create a home directory at /home/admin.

Write down the password of admin user and keep it in a safe place.

Next, we’ll grant sudo privileges to the admin user. Create a new file /etc/sudoers.d/admin using your favorite editor and type the line into it:

admin ALL=(ALL) ALL

Then change the permissions of the file:

server>chmod 440 /etc/sudoers.d/admin

Now that you have a sudoer user called admin and you can logout as the root
and re-login as that user instead. Soon we will disable remote root logons to
the server altogether. But before this, you should restart the ssh service on
the server.

server>service ssh restart
server>exit
local>

With a sudoer account

local>ssh admin@xxx.xxx.xxx.xxx

where xxx.xxx.xxx.xxx is the ipV4 address of your server.
You may get a warning stating that “The authenticity of the host…can’t be established”. Type ‘yes’ and continue - you’ll be prompted for a password. Type
in the password of admin user. You shall be logged to the server as a regular
user. Type

server>sudo -s

and enter admin user password one more time when prompted. You now changed
your identity on the server to that of superuser a.k.a. as root and may proceed
to system updating/installation and maintenance.

Upgrading Packages

Now that you’re logged into the server, we’ll get on with some basic setup.

First, we’ll update Ubuntu’s packages. First run:

server>apt-get update

This command will update the server’s information on the available packages to reflect the latest offerings. It takes a moment, and will spit out quite a bit of output. Next run

server>apt-get upgrade

which will upgrade the packages already on your server. This may take a few minutes, so sit back and wait for it to finish.

Fail2Ban

Fail2Ban is a program that logs and attempts to block suspicious logins. It’s trivial to install:

server>apt-get install fail2ban

For more information on Fail2Ban see the documentation.

Creating a deploy User and Setting Up SSH Keys

For slightly higher security we would create the user for deployment and running our application. It will have no sudo permissions, so if attacker somehow cracks password for this user, he/she would not get root access.

server>adduser deploy

You will be asked for some information. You can leave it blank (except for the password). This command will also create a home directory at /home/deploy.

Next we’ll install your ssh keys so we don’t have to fiddle about with passwords.
On the development laptop, execute following commands (again, with your server’s IP address in place of xxx.xxx.xxx.xxx):

local>ssh-copy-id -i ~/.ssh/id_rsa.pub admin@xxx.xxx.xxx.xxx
local>ssh-copy-id -i ~/.ssh/id_rsa.pub deploy@xxx.xxx.xxx.xxx

and enter corresponding passwords when prompted.

Now check, that you are able to log in as deploy and as admin. On the development laptop, add you identity to ssh-agent with:

local>ssh-add ~/.ssh/id_rsa

Type the passphrase when prompted. This step is not necessary, but it helps to avoid entering your passphrase several times.

Try that you are able to login as deploy

local>ssh deploy@xxx.xxx.xxx.xxx
server>exit
local>

and as admin

local>ssh admin@xxx.xxx.xxx.xxx

using your private/public key pair. If you login successfuly without entering passwords, the keys has been installed properly.

Finally, we want to prevent password logins and root logins. Back in the shell where we are logged in as root, open up the file at /etc/ssh/sshd_config.

server>nano /etc/ssh/sshd_config

Find the line that says PermitRootLogin yes and change it to ‘no’:

PermitRootLogin no

and directly below that line add another that prevents password authentication:

PasswordAuthentication no

Then save, quit and restart ssh:

server>service ssh restart

Additional Steps

Setting uncomplicated firewall takes a couple of minutes (commands are issued from root account on the server):

server>ufw enable
server>ufw allow 22
server>ufw allow 80
server>ufw allow 443
server>apt-get install iptables-persistent
server>invoke-rc.d iptables-persistent save

For more information on ufw, see the Ubuntu documentation.

You can also install Logwatch, which is a program that will email you logs which can aid in figuring out what happened in the case where there is an unauthorized login attempt. See the Ubuntu community docs for more information.

Installing The Web Stack

Now that our sever is secured and our user accounts are set up, we can begin installing the software that we need to run our Ruby on Rails application. We will install Ruby and some required gems, Rails, Passenger with Nginx, and MySQL. That is all we need to get up and running.

Once all this is done, we will get back to our development machine and set up our Capistrano deployment recipe.

Installing the Tools

Before we can install the good stuff, we need some Ubuntu packages. Quite a few, actually. Go ahead and install them. Log in as admin and run this command:

server>sudo apt-get install curl git-core build-essential zlib1g-dev libssl-dev libreadline-gplv2-dev libyaml-dev libcurl4-openssl-dev

This will take a minute or two.

Installing Ruby and Rails

We are going to forgo Ubuntu’s packaged version of Ruby and install it from source. We’ll be using Ruby 2.1.2. Some developers prefer to run a ruby version manager like rvm in production, but it is my preference that such tools are best for matching the development ruby version to whatever is on the server, not as an installation convenience. It’s just simpler to install ruby directly.

Create a directory called source in the admin home directory. This is where we’ll download our source code for things like Ruby. I’m sure there is a preferred place for doing this, but for our purposes it really doesn’t matter and this is as good a place as any.

server>cd
server>mkdir source
server>cd source

Now, we’ll download and install Ruby:

server>wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz
server>tar -zxf ruby-2.1.2.tar.gz
server>cd ruby-2.1.2
server>./configure
server>make
server>sudo make install

Installing Ruby can take a while.

Next we’ll install Rubygems. Again, we’ll do this from source, as the Ubuntu package has proved unreliable in the past.

server>cd ~/source
server>wget http://production.cf.rubygems.org/rubygems/rubygems-2.2.2.tgz
server>tar -zxf rubygems-2.2.2.tgz
server>cd rubygems-2.2.2
server>sudo ruby setup.rb

Installing Passenger and Nginx

Passenger and Nginx are installed together, providing the web server for our Rails app. Fortunately, the team at Phusion has put a lot of effort into the installation process, and it’s about as friendly as it can get.

First we’ll install the passenger gem:

server>sudo gem install passenger

and then install Passenger/Nginx:

server>sudo passenger-install-nginx-module

Follow the on screen instructions, and choose option 1 (“Yes, download, compile, and install Nginx for me (recommended).”) when prompted. For all other prompts, just choose the default.

One thing that does not get installed is the init script for Nginx. This is the script that allows us to start and stop the Nginx server. Thankfully, Nginx provides one we can use. Create a file at /etc/init.d/nginx and copy the contents of the script found at http://wiki.nginx.org/Nginx-init-ubuntu. Then we’ll update the permissions:

server>sudo chmod u=rwx /etc/init.d/nginx
server>sudo chmod go=rx /etc/init.d/nginx

There are a couple of edits we need to make to the init script. Look for the line near the top of the file that says:

DAEMON=/usr/local/sbin/nginx

We need to change that, because our Nginx configuration file was installed at /opt/nginx, not /usr/local. Go ahead and change that line as follows. You will need to use sudo to edit it:

DAEMON=/opt/nginx/sbin/nginx

Similarly, change

NGINX_CONF_FILE="/usr/local/nginx/conf/nginx.conf"

to:

NGINX_CONF_FILE="/opt/nginx/conf/nginx.conf"

and also change

PIDSPATH=/var/run

to:

PIDSPATH=/opt/nginx/logs

Now we can start and stop, and restart Nginx with the following commands:

server>sudo /etc/init.d/nginx start
server>sudo /etc/init.d/nginx stop
server>sudo /etc/init.d/nginx restart

We would also like Nginx to start automatically when you reboot the server. There is a handy utility that allows us to set this up easily. Install sysv-rc-conf:

server>sudo apt-get install sysv-rc-conf

and run it:

server>sudo sysv-rc-conf

In the GUI (if it can be called such), find the nginx line and check the boxes for columns 2, 3, 4, and 5. You can even use your mouse if so inclined. This will direct Nginx to start itself up when the server comes on line.

We can now quit out of sysv-rc-conf by typing q.

Installing MySQL

We will use the Ubuntu package for MySQL, since it works just fine. Install it:

server>sudo apt-get install mysql-server

You will be prompted to enter a password for the root MySQL user. Go ahead and do so. We will also need the following package to use the mysql2 gem, so let’s install it now.

server>sudo apt-get install libmysqlclient-dev

Now we will create a deploy user for MySQL and set its password. Log into MySQL as root:

server>mysql -u root -p

and enter your password when prompted. At the mysql> prompt, run the following command:

GRANT ALL PRIVILEGES ON *.* TO 'deploy'@'localhost' IDENTIFIED BY 'secret' WITH GRANT OPTION;

This will create a MySQL user named deploy with the password ‘secret’. Quit mysql by typing quit at the mysql> prompt.

We’re now finished with our basic server configuration. Everything remaining will be specific to our application deployment.

Setting remote git repository on server

Now we will prepare remote repo for our deployment needs. Capistrano 3, it seems, is not able to
deploy to remote from local repository. You must have a repository available to remote host.
You may use github, if you wish, but sometimes it’s convenient to create the repository on
the host you are deploying.

local>ssh deploy@xxx.xxx.xxx.xxx

and then perform following commands

server>mkdir ~/repos
server>cd ~/repos
server>GIT_DIR=doozy.git git init
server>cd doozy.git/
server>git --bare update-server-info
server>cp hooks/post-update.sample hooks/post-update

On your development laptop add the repo you’ve just created as the remote for your local development repository. From your app’s root directory issue

local>git remote add origin deploy@xxx.xxx.xxx.xxx:~/repos/doozy.git

After that you would be able to synchronize local git repo with the remote ons:

local>git push origin master

Now that we have a repository accessible from our server, we may proceed to deployment via Capistrano 3

Capistrano - Making Deployment Easy

Deploying a Rails app is conceptually simple, you just copy the files in your app on to the server and tell the web server where to find them. In practice, things get a little more complicated than that and can quickly get out of hand if you do things manually. Thankfully, we have Capistrano to manage the process.

Capistrano is a Ruby gem similar to Rake. You use Capistrano to run tasks on a remote machine via ssh. To use it, you install it on your local machine and write ruby scripts called “recipes” that run a series of tasks - some built into Capistrano and some custom built for your application. Capistrano is very flexible. We’re going to use it in as simple a way as possible - to copy the latest commit from our local git repository onto our server and restart the web server.

Let’s get started.

Capistrano is Local

Capistrano runs on your development laptop. You don’t need it on the server. Because passwords are involved, we are going to keep our deploy script out of version control. It is possible to extract the passwords into their own file and check the rest of the script into source control. But it’s a more advanced topic that we’ll skip for now.

The first thing we need to do is install the Capistrano gem. Since we’ll only need it in development, we’ll put it in the development group of our Rails app’s Gemfile.

group :development do
  gem 'capistrano',  '~> 3.1'
  gem 'capistrano-rails', '~> 1.1'
end

Don’t forget to run bundle install.

Now that Capistrano is installed, we’ll need to “Capify” our app. In the root directory of your app run:

local>bundle exec cap install

Which generates the following files and directory structure:

+-- Capfile
+-- config
Š   +-- deploy
Š   Š   +-- production.rb
Š   Š   L-- staging.rb
Š   L-- deploy.rb
L-- lib
    L-- capistrano
            L-- tasks

Although most of our work will be in the deploy script, let’s make sure that
we our Capfile plays well with Rails setup. As the bare minimum we need three
require lines and one custom tasks import line.

require 'capistrano/setup'
require 'capistrano/deploy'
require 'capistrano/rails'
#require "whenever/capistrano" 

# Loads custom tasks from `lib/capistrano/tasks' if you have any defined.
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }

Note that I left whenever/capistrano commented out; it may come handy in more
advanced scenarios, but is not used in our setup.

Before we continue, check that your .gitignore contains the following line,
so we have sensitive data out of source control

/config/secrets*.rb

The default deploy script (config/deploy.rb) has some very helpful comments. Start by editing it to look like this:

set :application, "doozy"
set :repo_url, "/home/deploy/repos/#{fetch(:application)}.git"

set :deploy_to, "/var/www/#{fetch(:application)}" #path to your app on the production server 

set :scm, :git

load File.expand_path('../secrets_deploy.rb', __FILE__)

set :default_env, { database_url:  "mysql2://deploy:#{fetch(:dbpassword)}@localhost/#{fetch(:application)}_#{fetch(:stage)}" }

set :linked_files, %w{config/database.yml config/secrets.yml}

set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system}

namespace :deploy do

  desc 'Restart application'
  task :restart do
    on roles(:app), in: :sequence, wait: 5 do
       execute :touch, release_path.join('tmp/restart.txt')
    end
  end

  after :publishing, :restart
end

Before we continue, lets take a look at what we have. The first line sets the application variable to the string 'doozy'. (Use the set method to do this in Capistrano).

Similarly, we set the repo_url variable to the absolute path of application source reporository on server (you may use url of your repo on github or another repository available to the server). We also set the deploy_to variable to the app’s path on the production server. We’ll put it in the /var/www directory. It doesn’t exist yet, so lets log into the server as deploy and create it. We also have to make sure it is owned by the deploy user.

You must switch to your admin session and execute following commands

server>sudo mkdir /var/www
server>sudo chown deploy:deploy /var/www

Back in our deploy script, we set the scm variable to 'git', since that’s what we are using.

Next we load config/secrets_deploy.rb file (residing in the same directory as deploy.rb) with password information.

load File.expand_path('../secrets_deploy.rb', __FILE__)

That file does not exist yet, let’s create it with following contents:

set :dbpassword, 'secret'

We set custom Capistrano variable named dbpassword with the value of mysql deploy user password, which we have assigned earlier in this tutorial. It
should be harder to guess than just ‘secret’, of course. This Capistrano variable is used to construct database url on the next line of deploy script.

Two variables linked_files and linked_dirs set in the deploy.rb script tell
Capistrano what should be symlinked from shared directory and not checked out from
git repo. Those are files we do not want to be in repository because of sensitive data (passwords, keys and such) they contain. Or directories that contain files generated by application itself (like logs, temporary files, pid files and the like). We will return to this topic later, when discussing file structure of
Capistrano deployments.

The final block defines the restart task in the deploy namespace. Restarting Passenger is as simple as touch-ing a file in the /tmp directory called restart.txt. We also define a callback which says that after publishing task
in the deploy namespace Capistrano should run the restart task in deploy namespace.

You should also define environment (in Ruby jargon) or stage (in Capistrano parlance) related information in config/deploy/*.rb files. Because we will be
deploying the production environment, it will suffice to have config/deploy/production.rb file with the following contents:

server 'www.example.com', user: 'deploy', roles: %w{web app db}
set :ssh_options, user: 'deploy', forward_agent: true
set :branch, "master"

Capistrano’s File Structure

Now that we have the beginning of a deploy script set up, lets take a break and talk about how Capistrano actually works under the hood. Capistrano doesn’t just copy the app’s files into the deploy_to directory. It’s smarter than that.

Instead, a Capistrano deployment will have a file structure like this:

{deploy_to}/releases
{deploy_to}/releases/20140718013421
...several more like the one above...
{deploy_to}/shared
{deploy_to}/shared/log
{deploy_to}/shared/pids
{deploy_to}/shared/system

Capistrano copies the files to a directory (named with a timestamp) in the releases directory. Each new release adds a new timestamped version in a separate directory. Once the files are in place, Capistrano symlinks the newest release to {deploy_to}/current.

{deploy_to}/current => {deploy_to}/releases/20130318013421

So you’ll find your rails app’s app directory (for example) at {deploy_to}/current/app.

Keeping old timestamped versions like this is a good idea if you ever need to roll back to an older version. Capistrano provides a built in task just for this - deploy:rollback.

But after a while, you can accumulate quite a few old releases, which we don’t need. Capistrano has a special built-in task deploy:cleanup. It deletes old versions on the server, leaving only the most recent ones. Variable keep_releases controls how many of them are being kept during updates. Default value is 5, which should be plenty.

So what’s up with the shared directory? Those are for files that are shared between releases - the log file is an example. You wouldn’t want to overwrite it every time you deployed new code. Those files get symlinked into /current as well.

Adding Config Files

Remember how we opted to leave database.yml and secrets.yml out of our git repository? Their contents change rarely, so we will not write a special task in Capistrano to deply them, but rather create those files with a text editor in
/var/www/doozy/shared/config directory

This is database.yml file:

default: &default
  encoding: utf8
  reconnect: false
  pool: 5


production:
  <<: *default
  url: <%= ENV['DATABASE_URL'] %>

And this is secrets.yml:

production:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

Make sure both files have permissions to be read (at least) by deploy user.

There is no sensitive data in those files, but they differ from the versions I have
on development machine, which have passwords/keys for development environment.

I could be a bit paranoid, but I prefer to keep even development-time secrets off the record.

Finally we are ready to deploy!

Setting up for the First Deployment

Now that our deployment script is ready, we are going to deploy the code.

local>bundle exec cap production deploy

Pay attention to the voluminous Capistrano output - it gives you a good feel for what it is doing behind the scenes and where to look if you have trouble.

Now log into the server as deploy again and create a database. It is possible to do this within Capistrano, but we only need to do it once, so it’s easier to just log in to the server and use Rake.

server>cd /var/www/doozy/current
server>RAILS_ENV=production DATABASE_URL= "mysql2://deploy:secret@localhost/doozy_production"
 bundle exec rake db:create db:migrate

Final Nginx Config

Our code is now ready to run. We just need to configure Nginx to find it. Create a directory at /opt/nginx/sites. This is where we’ll store our app specific Nginx configuration files.

server>sudo mkdir /opt/nginx/sites

Now create a file (you’ll need sudo) in that directory called doozy with the following content, substituting your domain (make sure your DNS is configured properly to use the ‘www’ subdomain:

server {
  listen 80;
  server_name www.example.com;
  root /var/www/doozy/current/public; # note that /current/public is required here.
  passenger_enabled on;
  include /opt/nginx/secrets/doozy.conf;
}

This tells Nginx to listen on port 80 for traffic coming to www.example.com and to use the app at /var/www/doozy with Passenger. Note that you must use the /public directory as root, not the app’s root directory. Also note that the /current directory created by Capistrano must also be specified.

The sensitive data for our application we keep in an additional /opt/nginx/secrets/doozy.conf file. First, we create directory
with

server>sudo mkdir /opt/nginx/secrets
server>sudo chmod 700 /opt/nginx/secrets

Then we create doozy.conf in that dir with contents like following

passenger_set_cgi_param DATABASE_URL "mysql2://deploy:secret@localhost/doozy_production";
passenger_set_cgi_param SECRET_KEY_BASE 05a0fd20a8c410f5b469096914b973fa141a7716b9c153938041ffa611ed80b613bb44633d50388a8de31f9374d99054f8b22deb55df576048f3418e65eccbd2;

SECRET_KEY_BASE environment variable value above was generated by rake task named secret.

To ensure that only root can read the secret file:

server>sudo chmod 600 /opt/nginx/secrets/doozy.conf

Now we need to include site configuration file in the main Nginx config file. You’ll find it at /opt/nginx/conf/nginx.conf. Open it (with sudo) and at the very bottom, right before the final closing curly brace add this line:

include /opt/nginx/sites/*;

This just tells Nginx to include the contents of every file in /opt/nginx/sites. Each time you add a new app, you just add a new config file to /opt/nginx/sites.

Before we’re finished editing that file, add one more line near the top. We need to tell Passenger how many application processes to use. The default is 6, which is a bit high for our small server. We’ll use 2 instead. This is important. If you run too many processes, your server will slow to a crawl when you run out of memory.

...
events {
    worker_connections  1024;
}


http {
    passenger_root /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.19;
    passenger_ruby /usr/local/bin/ruby;

    passenger_max_pool_size 2;

    include       mime.types;
    default_type  application/octet-stream;
...

When you’re done, restart Nginx:

server>sudo /etc/init.d/nginx restart

You’re Done

From now on, new deployments are a snap. After you update your application code, check it into Git,

local>git add .
local>git commit -m "description of changes"

and push to the server,

local>git push origin master

just run

local>bundle exec cap production deploy

and Capistrano will automatically deploy your code, symlink the required files, clean up the old releases and restart Nginx.

urby by ub

Monday, July 14, 2014