Difference between revisions of "Creating an AMI on EC2"

From Genome Analysis Wiki
Jump to navigationJump to search
Line 7: Line 7:
 
These notes assume you have already created an EC2 account and have the certificates and keys set up properly.
 
These notes assume you have already created an EC2 account and have the certificates and keys set up properly.
  
<b>Launch an instance</b>
+
'''Launch an instance'''
  
 
<code>
 
<code>
   Login to https://console.aws.amazon.com/ec2
+
   Login to https://console.aws.amazon.com/ec2       # EC2 Management Console
 
</code>
 
</code>
  
Line 33: Line 33:
  
  
<b>Install the pipeline software</b>
+
'''Install the pipeline software'''
  
In a terminal on your workstation, copy the  
+
Follow the instructions to install a Debian package  [[Pipeline Debian Package|debian package]]
 +
Run the tests to be sure everything is OK.
  
 +
'''Create the AMI'''
  
 +
In your browser at the EC2 Management Console do the following:
  
ssh -i ~/.ssh/AWS/CSG.pem ubuntu@ec2-23-20-32-7.compute-1.amazonaws.com mkdir debs
+
<code>
scp dev/biopipe/*.deb dev/umake/*.deb root8g:debs
+
   Create Image
 
+
    Image Name  csg-biopipe_instance
 
+
    Image Description:  Image for CSG Biopipe instance
ssh -i ~/.ssh/AWS/CSG.pem ubuntu@ec2-23-20-32-7.compute-1.amazonaws.com
+
    Volume Size:  30GB
 
+
    Take defaults otherwise
sudo dpkg -i debs/biopipe-align_0.6_amd64.deb
+
</code>
Selecting previously unselected package biopipe-align.
 
(Reading database ... 25372 files and directories currently installed.)
 
Unpacking biopipe-align (from .../biopipe-align_0.6_amd64.deb) ...
 
Setting up biopipe-align (0.6) ...
 
Created symlink '/usr/local/bin/gen_biopipeline.pl' to make running the aligner easier.
 
#============================================================
 
#   'java' is not installed, do 'sudo apt-get install java-common default-jre'
 
#============================================================
 
#============================================================
 
#  'make' is not installed, do 'sudo apt-get install make'
 
#============================================================
 
#============================================================
 
#  'libssl0.9.8' is not installed, do 'sudo apt-get install libssl0.9.8'
 
#============================================================
 
#============================================================
 
#  '/usr/local/biopipe/test/align' does not exist so you cannot test this install
 
#  Install it by doing 'sudo dpkg -i biopipe-test_*.deb'
 
#============================================================
 
etc
 
 
 
 
 
If there were messages about missing packages, install them now
 
  sudo apt-get install make libssl0.9.8 java-common default-jre
 
 
 
 
 
sudo dpkg -i debs/biopipe-testalign_0.6_amd64.deb
 
 
 
/usr/local/biopipe/bin/gen_biopipeline.pl --test ~/testalign
 
 
 
 
 
Create Image
 
  Image Name  csg-biopipe_instance
 
  Image Description:  Image for CSG Biopipe instance
 
  Volume Size:  30GB
 
  Take defaults otherwise
 
 
 
See notice about
 
  View pending image ami-d27ec7bb
 
 
 
 
 
 
 
 
 
AMI: Ubuntu Cloud Guest AMI ID ami-3d4ff254 (x86_64)
 
Name: Ubuntu Server 12.04.1 LTS
 
Description: Ubuntu Server 12.04.1 LTS with support available from Canonical (http://www.ubuntu.com/cloud/services). Edit AMI
 
Number of Instances: 1
 
Availability Zone: No Preference
 
Instance Type: Micro (t1.micro)
 
Instance Class: On Demand Edit Instance Details
 
EBS-Optimized: No
 
Monitoring: Disabled Termination Protection: Disabled
 
Tenancy: Default
 
Kernel ID: Use Default Shutdown Behavior: Stop
 
RAM Disk ID: Use Default
 
Network Interfaces:
 
Secondary IP Addresses:
 
User Data:
 
IAM Role: Edit Advanced Details
 
Key Pair Name: CSG Edit Key Pair
 
Security Group(s): sg-a098e9c8 Edit Firewall
 
  
 +
This will take several minutes to complete.
 +
In the EC2 Dashboard, you can monitor the progress.
 +
When it is done, you'll see a new AMI under the list of AMIs.
  
 +
Your new AMI should look pretty much like this:
  
 +
<code>
 +
  AMI: Ubuntu Cloud Guest AMI ID ami-3d4ff254 (x86_64)
 +
  Name: Ubuntu Server 12.04.1 LTS
 +
  Description: Ubuntu Server 12.04.1 LTS with support available from Canonical (http://www.ubuntu.com/cloud/services).
 +
  Number of Instances: 1
 +
  Availability Zone: No Preference
 +
  Instance Type: Micro (t1.micro)
 +
  Instance Class: On Demand Edit Instance Details
 +
  EBS-Optimized: No
 +
  Monitoring: Disabled Termination Protection: Disabled
 +
  Tenancy: Default
 +
  Kernel ID: Use Default Shutdown Behavior: Stop
 +
  RAM Disk ID: Use Default
 +
  Network Interfaces:
 +
  Secondary IP Addresses:
 +
  User Data:
 +
  IAM Role: Edit Advanced Details
 +
  Key Pair Name: CSG Edit Key Pair
 +
  Security Group(s): sg-a098e9c8 Edit Firewall
 +
</code>
  
 +
'''Test the new AMI'''
  
 +
Launch a new AMI instance and check that files are in the correct places.
 +
In the EC2 Management Console do:
  
 +
<code>
 
Launch an AMI
 
Launch an AMI
 
EC2 DashBoard -> AMIs -> Select CSG instance -> Launch Instance
 
EC2 DashBoard -> AMIs -> Select CSG instance -> Launch Instance

Revision as of 09:47, 23 October 2012

Notes About Creating a New EC2 AMI

Back to parent [1]


The following are notes taken when creating the Amazon Machine Instance used for the CSG pipeline process. These notes assume you have already created an EC2 account and have the certificates and keys set up properly.

Launch an instance

 Login to https://console.aws.amazon.com/ec2       # EC2 Management Console

Pay attention to the region you are using, at least for now it seems any StarCluster activity must be in us-east-1. Launch a new instance which we will use to set up the software and ultimately save it as an AMI.

 EC2 DashBoard -> Launch Instance
 Class Wizard
 Ubuntu Server 12.04.1 LTS   64 bit
 Instance type -> Micro,  EC2, no preference        # Memory size does not matter
 Advanced Instance Options  (take defaults)
 Storage Device Configuration -> Edit
 Change volume to 30G -> Save -> Continue           # Storage size does not matter
 Key Name = CSG 8G biopipe; ubuntu 12.04
 Create Key/Pair if you need to, Name the PEM and save the pem file for access by ssh
 Choose a Security Group  (take default)
 Launch
   No need to Create Status Check Alarms
   No need to Create EBS Volumes


Install the pipeline software

Follow the instructions to install a Debian package debian package Run the tests to be sure everything is OK.

Create the AMI

In your browser at the EC2 Management Console do the following:

 Create Image
   Image Name   csg-biopipe_instance
   Image Description:  Image for CSG Biopipe instance
   Volume Size:  30GB
   Take defaults otherwise

This will take several minutes to complete. In the EC2 Dashboard, you can monitor the progress. When it is done, you'll see a new AMI under the list of AMIs.

Your new AMI should look pretty much like this:

 AMI: 	Ubuntu Cloud Guest AMI ID ami-3d4ff254 (x86_64)
 Name: 	Ubuntu Server 12.04.1 LTS 	 
 Description: 	Ubuntu Server 12.04.1 LTS with support available from Canonical (http://www.ubuntu.com/cloud/services).
 Number of Instances: 	1 	 
 Availability Zone: 	No Preference 	 
 Instance Type: 	Micro (t1.micro) 	 
 Instance Class: 	On Demand 	Edit Instance Details
 EBS-Optimized: 	No
 Monitoring: 	Disabled 	Termination Protection: 	Disabled 	 
 Tenancy: 	Default 	 
 Kernel ID: 	Use Default 	Shutdown Behavior: 	Stop 	 
 RAM Disk ID: 	Use Default 	 
 Network Interfaces: 		 
 Secondary IP Addresses: 		 
 User Data: 		 
 IAM Role: 		Edit Advanced Details
 Key Pair Name: 	CSG 	Edit Key Pair
 Security Group(s): 	sg-a098e9c8 	Edit Firewall

Test the new AMI

Launch a new AMI instance and check that files are in the correct places. In the EC2 Management Console do:

Launch an AMI EC2 DashBoard -> AMIs -> Select CSG instance -> Launch Instance Launch Instances (take defaults) Advanced Instance Options (take defaults) Storage Device Configuration -> Edit Change volume to 30G or whatever -> Continue Instance Details

 Key Name = test of instance

Create Key/Pair if you need to, Name the PEM (e.g. CSG) and save the CSG.pem file

Choose a Security Group (take dfeault)

 sg-a098e9c8 - quick-start-1

Review -> Launch