Deploying a Grails app on EC2 from scratch.

Hi All

Ok so i have found some time write up how to deploy a Grails app onto EC2. This is a step by step guide to setting up Apache, Tomcat, Mysql, Java on an ubuntu Ec2 Box

For those who do not know. EC2 is Amazon's (that's right the one famous for books) hosting service of its webservice suite. It allows you to run virtual server images on it amazingly scaleable infrastructure. Plus you only pay for what you use. If your server is up for 4 hours you pay for 4 hours. $0.10 per hour (about). You can build your own server as i will show you in the steps below and manange it on their infrastructure.


These are the steps i have taken and they work for me. I am sure there are hundreds of ways to skin a cat (so the cat skinners say) but this is my way. Take it or leave it :)

Pre-Requisites

1) Get Firefox i am sure you have it already if you don't get it you will need it. As of writing this i am using 3.0.11 download here

2) Install the EC2 Firefox plugin (Elasticfox) download here

3) Install the S3 firefox plugin download here


4) Open an amazon account. This is a five min job requiring a valid credit card. Signup here
Once you have an account you then need to open a EC2 account and an S3 account.


5) Then next step is for you get familiar with the tools. You can do this by following the getting started guides in this ebook.

Gettings started


IMPORTANT !!!

You need to read the getting started guide and get familiar with things before you read the rest of this blog post. Otherwise it will not make sense. I am assuming from here onwards you have read an tried out the service.


Setting up a server to run a Grails app.

By now you should have terminated any test instances you had running on EC2 and be ready start a fresh.


1) Start with a baseline Linux image

Ok i have decided to host my application on Linux basically because there was no descion to be made. I would only ever host a serious website on Linux and never windows.

The base linux version i used is Ubuntu 8.10 (intrepid) i am also using a 32bit and not 64 bit. You can find the AMI codes here

https://help.ubuntu.com/community/EC2StartersGuide

I have chosen to go with AMI ami-5059be39 because it is based in the US and is 32Bit. If you want to run your servers in Europe then go with this ami-80c0e8f4

Ok now you should have a running instance of the ubuntu basic server. Connect to the server using an SSH terminal

Create a directory called downloads in side your home directory /home/ubuntu and cd into it.

2) Next you want to install Java

You should be in the downloads directory you created above

Download java 1.6. Right that now happens to be JDK 6 update 14. To do this i goto the java download website .

Choose JDK 6 update 14
Choose linux
Then copy the download link for the NON rpm version the second link

Now on your EC2 machine run the following command

wget <THE JDK DOWNLOAD URL>

This will now download the JDK install file onto the box

NOTE you might want to rename the file:

mv <NAME OF THE FILE>  jdk-6u14-linux-i586.bin

Next copy the file to /usr/local

sudo cp jdk-6u14-linux-i586.bin /usr/local

Change the permissions

sudo chmod +x jdk-6u14-linux-i586.bin

Then run the install file

sudo ./jdk-6u14-linux-i586.bin

This should unpack the java file to this directory /usr/local/jdk1.6.0_14

Next create a symbolic link to the directory called java

sudo ln -s /usr/local/jdk1.6.0_14 /usr/local/java

That is java installed.

3) Next install MYSQL

Download mysql via the same way you downloaded java above using wget you get the download link here

Choose the Linux (x86) download

Copy the mysql tar file into /usr/local

sudo cp mysql-5.1.35-linux-i686-glibc23.tar.gz /usr/local

Delete any existing mysql directories. (this maybe optional)

sudo rm -rf /etc/mysql

Next follow these commands to install it. IMPORTANT: Don't follow any of the step that are listed by the mysql_install_db script they will be covered later.

sudo groupadd mysql
sudo useradd -g mysql mysql
sudo cp mysql-5.1.35-linux-i686-glibc23.tar.gz /usr/local
sudo cd /usr/local
sudo gunzip mysql-5.1.35-linux-i686-glibc23.tar.gz
sudo tar -xvf mysql-5.1.35-linux-i686-glibc23.tar
sudo mv /usr/local/mysql-5.1.35-linux-i686-glibc23 /usr/local/mysql 
sudo cd mysql 
sudo mkdir logs
sudo chown -R mysql . 
sudo chgrp -R mysql . 
sudo scripts/mysql_install_db --user=mysql 
sudo chown -R root . 
sudo chown -R mysql data 
sudo bin/mysqld_safe --user=mysql &  

Right now you should have running mysql instance

Copy the standard startup script into the linux startup area

sudo cp /usr/local/mysql/support_files/mysql.server /etc/init.d/mysql

Setup mysql to run at all levels

sudo update-rc.d mysql defaults

Set a root password

sudo /usr/local/mysql/bin/mysqladmin -u root password 'new-password'
sudo /usr/local/mysql/bin/mysqladmin -u root -h 'localhost' password 'new-password'
 

Ok now you have a running mysql instance

4) Next you want to move the Mysql installation on to an amazon EBS.

You should already know about an EBS from reading the EC2 getting start guide above.

Create a new EBS using Elastic fox and attach it to your EC2 instance. You will be asked to enter a device when you are attaching the EBS to your running EC2 instance. Enter the following:

/dev/sdh
  

Once you have done that you want to mount and format that diskspace. I am simply following the instructions found here but have modified it bit as my paths are different

Install XFS. This is a secure file system.

sudo apt-get install -y xfsprogs
   

Setup the mount and fomat the disk

sudo modprobe xfs
sudo mkfs.xfs /dev/sdh

echo "/dev/sdh /vol xfs noatime 0 0" | sudo tee -a /etc/fstab
sudo mkdir /vol
sudo mount /vol
    

You should now have a new filesystem under /vol

Stop mysql if running

sudo /etc/init.d/mysql stop
    

Move all the mysql files

sudo mkdir -p /vol/usr/local/mysql

sudo mv /usr/local/mysql /vol/usr/local
     

Remount the mysql directory to the volume

sudo mkdir /usr/local/mysql

echo "/vol/usr/local/mysql /usr/local/mysql     none bind" | sudo tee -a /etc/fstab
sudo mount /usr/local/mysql

      

Restart mysql

Stop mysql if running

sudo /etc/init.d/mysql start
     

You now have mysql running from an EBS


Setup mysql database user and backup script

/usr/local/mysql/bin/mysql -p -u root 

create database

CREATE DATABASE demodb CHARACTER SET utf8 COLLATE utf8_general_ci; 

create app user and backup user and give access:

CREATE USER 'demouser'@'localhost' IDENTIFIED BY 'demouser123';
GRANT ALL PRIVILEGES ON demodb.* TO 'demouser'@'localhost' WITH GRANT OPTION;

CREATE USER 'backup'@'localhost' IDENTIFIED BY 'backup123';
GRANT LOCK TABLES, SELECT ON demodb.* TO 'backup'@'localhost';

Setup db backup:

Download the mysql backup script:

http://sourceforge.net/projects/automysqlbackup/

For putting files in s3 use this

http://s3tools.org/s3cmd

apt-get install s3cmd

6) Install Tomcat

This is easy to install
Download tomcat 6 into your /home/ubuntu/downloads using wget
Copy the jar file it to /usr/local

Use the java jar command to unpack tomcat zip

sudo /usr/local/java/bin/jar -xvf /usr/local/tomcatXXX.zip

Create a symbolic link to Tomcat

sudo ln -s /usr/local/tomcatXXXX /usr/local/tomcat

Next create tomcat user to run tomcat

sudo groupadd tomcat
sudo useradd -m -s /bin/bash -g tomcat tomcat

Change to tomcat user and edit the .profile to add java in the PATH

sudo su - tomcat
vi /home/tomcat/.profile

Edit the file using vi to contain

export JAVA_HOME=/usr/local/java
PATH=$JAVA_HOME/bin:$PATH

configure the server.xml to use an AJP connector to work with apache.

save your old /usr/local/tomcat/conf/server.xml to server.xml.orig

cp /usr/local/tomcat/conf/server.xml /usr/local/tomcat/conf/server.xml.orig

edit a new server.xml

vi /usr/local/tomcat/conf/server.xml 

copy this config in there:

<?xml version='1.0' encoding='utf-8'?>
<Server port="8005" shutdown="SHUTDOWN">

  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JasperListener" />
  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />

  <Service name="Catalina">

        <!--
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
        -->

        <Connector      port="8009"
                                protocol="AJP/1.3"
                                redirectPort="8443"
                                connectionTimeout="300000"
                                maxThreads="200"
                                backlog="50"
                                enableLookups="true"
                                emptySessionPath="true"

        />

        <Engine name="Catalina" defaultHost="localhost">

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true"
            xmlValidation="false" xmlNamespaceAware="false">

      </Host>
    </Engine>
  </Service>
</Server>

That is tomcat setup. You can run it if you like to see it working.

Now just run tomcat

sudo su - tomcat
/usr/local/tomcat/bin/startup.sh
      

6) Install Apache

This is easy

sudo apt-get install apache2

Enable AJP mod

sudo a2enmod proxy_ajp

Create a new site to configure AJP with tomcat. You can change the to whatever you want.

sudo vi /etc/apache2/sites-available/webapp

add the following config to link apache to tomcat

<ifmodule>
ProxyRequests On
ProxyVia On

<location>
Order allow,deny
Allow from all
ProxyPass ajp://localhost:8009/
ProxyPassReverse ajp://localhost:8009/
</location>

</ifmodule>

Enable your site

sudo a2ensite webapp
  

disable the default apache config

sudo a2dissite default
  

start or restart apache

sudo /etc/init.d/apache2 restart
   

That is basic apache installed

To secure apache with basic authenication:

sudo mkdir /etc/apache2/passwd
cd /etc/apache2/passwd
sudo htpasswd -c site-access 

add these lines to the webapp config to secure:

<Location />
Order allow,deny
Allow from all

AuthType Basic
AuthName "Restricted area"
AuthUserFile /etc/apache2/passwd/site-access
Require valid-user

ProxyPass ajp://localhost:8009/
ProxyPassReverse ajp://localhost:8009/
</Location>

To enable SSL on the apache instance:

Enable ssl mod

sudo a2enmod ssl

generate your .key and .crt files. The company who you bought your SSL certs from should tell you how to do this.

make a new director to put the .key and .crt file in:

sudo mkdir /etc/apache2/ssl

copy the .key and .crt files into that directory. eg:

/etc/apache2/ssl/www.mydomain.com.crt
/etc/apache2/ssl/www.mydomain.com.key

Create a new site to configure ssl.

sudo vi /etc/apache2/sites-available/ssl

copy this config

<VirtualHost *:443>
DocumentRoot /var/www/

<IfModule mod_proxy_ajp.c>

ProxyRequests On
ProxyVia On

<Location />
Order allow,deny
Allow from all
#AuthType Basic
#AuthName "Restricted area"
#AuthUserFile /etc/apache2/passwd/site-access
#Require valid-user
ProxyPass ajp://localhost:8009/
ProxyPassReverse ajp://localhost:8009/
</Location>


</IfModule>

<IfModule mod_ssl.c>

SSLEngine On
SSLCertificateFile /etc/apache2/ssl/www.mydomain.com.crt
SSLCertificateKeyFile /etc/apache2/ssl/www.mydomain.com.key

</IfModule>
</VirtualHost>

Just change the paths at the bottom to point to your .crt and .key files.

Ok few that was long ish for me to write...

By now you should have Tomcat running with Apache and Mysql.



UPDATE:

Extra steps i take for Grails:

Going live with grails

Hope this helps someone....