| Line 32: |
Line 32: |
| | This should only take 15 minutes. | | This should only take 15 minutes. |
| | | | |
| − | Setting up your storage is perhaps the most difficult step as it is controlled completely by the size of your data.
| + | The last step is to organize your storage so you have enough space for the input sequence data |
| − | As a generate rule you will need three times the space required for your sequence data.
| + | and the output of the aligner and umake steps. |
| − | For instance in the 1000 Genomes data, the data for one individual takes about 45G.
| + | This is described in more detail in [[Amazon Storage|Amazon Storage]]. |
| − | If you have 1000 Genome data for nine individuals, you'll need about 1500GB of space (9x450x3 plus a little extra space).
| + | If you are not using AWS, the process will be similar to that described above, |
| − | | + | but the details will vary based on your environment. |
| − | Making your data available for the Pipeline can be accomplished in many ways.
| |
| − | Here is a simple straightforward organization you might want to use.
| |
| − | | |
| − | * Using the AWS EC2 Console Dashboard create one EBS volume (ELASTIC BLOCK STORE -> Volumes) for the sequence data (e.g. 500GB).
| |
| − | * Using the Dashboard create another EBS volume for the output of the aligner step (e.g. another 500GB).
| |
| − | * Using the Dashboard create another EBS volume for the output of the umake step (e.g. another 500GB).
| |
| − | | |
| − | Configure these EBS volumes so they use separate devices devices g, g and h (e.g. /dev/sdf (probably /dev/xvdf),
| |
| − | /dev/sdg (probably /dev/xvdg) and /dev/sdh (probably /dev/xvdh)).
| |
| − | | |
| − | Launch your instance and login as explained in the AWS documentation.
| |
| − | | |
| − | | |
| − | | |
| − | | |
| − | | |
| − | Testing the Installation
| |
| − | | |
| − | We recommend that at least the first time, you install the test packages so you can conveniently test the installation and make sure everything runs smoothly. The tests run within a few minutes and are self checking, so unless you see obvious errors, you can be reasonably sure everything is set up properly. You only need to do this once, unless you have made signifcant changes to your Unix system.
| |
| − | | |
| − | sudo dpkg -i debs/biopipe-test*_amd64.deb
| |
| − | Unpacking biopipe-testalign (from .../biopipe-testalign_M.n_amd64.deb) ...
| |
| − | Selecting previously deselected package biopipe-testumake.
| |
| − | Unpacking biopipe-testumake (from .../biopipe-testumake_M.n_amd64.deb) ...
| |
| − | Setting up biopipe-testalign (M.n) ...
| |
| − | To test the pipeline, run:
| |
| − |
| |
| − | /usr/local/biopipe/bin/gen_biopipeline.pl --test ~/testalign
| |
| − |
| |
| − | This will remove the contents of ~/testalign and then run
| |
| − | the aligner test case. The output is verified so you know if
| |
| − | anything failed or not.
| |
| − |
| |
| − | Setting up biopipe-testumake (M.n) ...
| |
| − | To test umake, run:
| |
| − |
| |
| − | /usr/local/biopipe/bin/umake.pl --test ~/testumake
| |
| − |
| |
| − | This will remove the contents of ~/testumake and then run
| |
| − | the umake test case. The output is verified so you know if
| |
| − | anything failed or not.
| |
| − | | |
| − | Login as a normal user (not as root) and do:
| |
| − | | |
| − | # Test the aligner (fast, about 3 minutes)
| |
| − | /usr/local/biopipe/bin/gen_biopipeline.pl --test ~/testalign
| |
| − | rm -rf ~/testalign # If no error
| |
| − |
| |
| − | # Test umake (longer, about 15 minutes)
| |
| − | /usr/local/biopipe/bin/umake.pl --test ~/testumake
| |
| − | rm -rf ~/testumake # If no error
| |
