Amazon Snapshot
You may run the pipeline software on a single instance we have created for you in AWS. You may also, of course, create your own instance and run it there.
Your first task is get an AWS account and keys so that you can use the AWS EC2 Console Dashboard (see https://console.aws.amazon.com/ec2/). From here you can launch instances prepared by others or create your own. We cannot assist in this step - Amazon has plenty of documentation. Once you are at the AWS EC2 Console Dashboard, you're ready to run the pipeline.
Launch Your First Instance
You'll need to know some details when launching an instance:
- What Instance to launch. You have several choices
- ami-be59d78e which is an instance we have prepared based on Ubuntu Server 12.04.1 LTS. It has all of our software installed.
- Some other instance. The instance must run 64 bit software and is either Ubuntu of any version or Redhat/CentOS 6.3. You will also need to install the Pipeline software which will require about 15 minutes.
- Instance size (memory and number of processors). The pipeline software will require at least 8GB of memory (type m1.large) and can use as many processors as is available.
- Storage is perhaps the most difficult detail as it is controlled completely by the size of your data. As a generate rule you will need three times the space required for your sequence data. For instance in the 1000 Genomes data, the data for one individual takes about 45G. If you have 1000 Genome data for nine individuals, you'll need about 1500GB of space
(9x450x3 plus a little extra space).
Testing the Installation
We recommend that at least the first time, you install the test packages so you can conveniently test the installation and make sure everything runs smoothly. The tests run within a few minutes and are self checking, so unless you see obvious errors, you can be reasonably sure everything is set up properly. You only need to do this once, unless you have made signifcant changes to your Unix system.
sudo dpkg -i debs/biopipe-test*_amd64.deb Unpacking biopipe-testalign (from .../biopipe-testalign_M.n_amd64.deb) ... Selecting previously deselected package biopipe-testumake. Unpacking biopipe-testumake (from .../biopipe-testumake_M.n_amd64.deb) ... Setting up biopipe-testalign (M.n) ... To test the pipeline, run: /usr/local/biopipe/bin/gen_biopipeline.pl --test ~/testalign This will remove the contents of ~/testalign and then run the aligner test case. The output is verified so you know if anything failed or not. Setting up biopipe-testumake (M.n) ... To test umake, run: /usr/local/biopipe/bin/umake.pl --test ~/testumake This will remove the contents of ~/testumake and then run the umake test case. The output is verified so you know if anything failed or not.
Login as a normal user (not as root) and do:
# Test the aligner (fast, about 3 minutes) /usr/local/biopipe/bin/gen_biopipeline.pl --test ~/testalign rm -rf ~/testalign # If no error # Test umake (longer, about 15 minutes) /usr/local/biopipe/bin/umake.pl --test ~/testumake rm -rf ~/testumake # If no error
