3,045
edits
Changes
From Genome Analysis Wiki
Jump to navigationJump to searchno edit summary
of the many variations you might choose, but should provide you a working example.
of the many variations you might choose, but should provide you a working example.
== Getting Started With StarCluster ==
== Getting Started With StarCluster ==
StarCluster provides lots of documentation.
StarCluster provides lots of documentation (http://star.mit.edu/cluster/) which will provide more information on it than we have here.
To install and setup StarCluster for the first time, you can follow the QuickStart instructions: http://star.mit.edu/cluster/docs/latest/quickstart.html
To install and setup StarCluster for the first time, you can follow the QuickStart instructions: http://star.mit.edu/cluster/docs/latest/quickstart.html
*** If you need help setting up your AWS credentials, see: [[AWS Credentials]]
*** If you need help setting up your AWS credentials, see: [[AWS Credentials]]
You can skip the cluster start section if you want.
You can skip actually starting the cluster in the QuickStart instructions if you want.
''' Troubleshooting: ''' When I tried this, the <code>starcluster start mycluster</code> step failed similar to:
''' Troubleshooting: ''' When I tried this, the <code>starcluster start mycluster</code> step failed similar to:
''' Don't forget to terminate your cluster:'''
''' Don't forget to terminate your cluster:'''
starcluster terminate mycluster
starcluster terminate mycluster
== StarCluster and GotCloud ==
== StarCluster and GotCloud ==
=== StarCluster Config Settings ===
By default, StarCluster expects a configuration file in ~/.starcluster/config.
By default, StarCluster expects a configuration file in ~/.starcluster/config.
* StarCluster will create a model file for you
* StarCluster will create a model file for you
* You should have set these in [[#Getting Started With StarCluster|Getting Started With StarCluster]] above (quickstart guide and AWS Credentials) .
* You should have set these in [[#Getting Started With StarCluster|Getting Started With StarCluster]] above (quickstart guide and AWS Credentials) .
* GotCloud Cluster Definition
** You may want to create a new cluster section for running GotCloud (or you can use smallcluster) in your configuration file: <code>~/.starcluster/config</code>
* You may want to create a new cluster section for running GotCloud (or you can use smallcluster) in your configuration file: <code>~/.starcluster/config</code>
** You can call it anything you want, for example, <pre>gccluster</pre>
** You can call it anything you want, for example, <pre>gccluster</pre>
** Example:
** Example:
** The <code>CLUSTER_SIZE</code> * CPUs in <code>NODE_INSTANCE_TYPE</code> = the number of jobs you can run concurrently in GotCloud
** The <code>CLUSTER_SIZE</code> * CPUs in <code>NODE_INSTANCE_TYPE</code> = the number of jobs you can run concurrently in GotCloud
==== Running GotCloud on StarCluster ====
* Define Data Volumes
* When Running GotCloud:
** By default, the GotCloud AMI contains about 5G of extra space that you can use
*** /home/ubuntu/ directory is visible from all machines
**** Use /home/ubuntu/ for the output directory if it is <5G
**** This directory will be deleted when you terminate the AMI
** Create your Own Volumes and attach them to the GotCloud cluster
*** '''Instructions TBD'''
=== Starting the Cluster ===
# Start the cluster:
#* <pre>starcluster start -c gccluster mycluster</pre>
#** Alternatively, if you can the default template at the start of the configuration file in the <code>[global]</code> section to gccluster: <code>DEFAULT_TEMPLATE=gccluster</code>, you can run:
#*** <pre>starcluster start mycluster</pre>
#* It will take a few minutes for the cluster to start
=== Copying Data to/from the Cluster ===
Copy data onto the cluster (command run from your local machine)
starcluster put /path/to/local/file/or/dir /remote/path/
Pull the data from the cluster onto your local machine (command run from your local machine)
starcluster get /path/to/remote/file/or/dir /local/path/
'''Reminder, if you write your output to /home/ubuntu/, it will be deleted when you terminate the cluster'''
=== Running GotCloud on StarCluster ===
* If you have not already, logon to the cluster as ubuntu:
** <pre>starcluster sshmaster -u ubuntu mycluster</pre>
*** Type <code>yes</code> if the terminal asks if you want to continue connecting
* When running GotCloud:
** Set the cluster/batch type in either configuration or on the command line:
** Set the cluster/batch type in either configuration or on the command line:
*** In Configuration:
*** In Configuration:
**: <pre>--numjobs #</pre>
**: <pre>--numjobs #</pre>
*** Replace number with the number of concurrent jobs you want to run (probably <code>CLUSTER_SIZE</code> * CPUs in <code>NODE_INSTANCE_TYPE</code>)
*** Replace number with the number of concurrent jobs you want to run (probably <code>CLUSTER_SIZE</code> * CPUs in <code>NODE_INSTANCE_TYPE</code>)
** Otherwise, run GotCloud as you normally would.
=== Monitoring Cluster Usage ===
* Monitor jobs in the queue
** <pre>qstat</pre>
** This will show you how the currently running jobs and how they are spread across the nodes in your cluster
**:[[File:Qstat.png|800px]]
*** state descriptions:
**** <code>qw</code> : queued and waiting (not yet assigned to a node)
**** <code>r</code> : running
* View Sun Grid Engine Load
** <pre>qhost</pre>
**:[[File:Qhost.png|600px]]
*** ARCH : architecture
*** NCPU : number of CPUs
*** LOAD : current load
*** MEMTOT : total memory
*** MEMUSE : memory in use
*** SWAPTO : swap space
*** SWAPUS : swap space in use
* View the average load per node using:
** <pre>qstat -f</pre>
**:[[File:Qstatf.png|650px]]
*** <code>load_avg</code> field contains the load average for each node
=== Run GotCloud Demo Using StarCluster ===
=== Terminate the Cluster ===
# Reminder, check if you need to copy any data off of the cluster that will be deleted upon termination
#* [[#Copying Data to/from the Cluster|Copying Data to/from the Cluster]]
# Terminate the cluster
#* <pre>starcluster terminate mycluster</pre>
== Run GotCloud Demo Using StarCluster ==
#Create a new cluster section in your configuration file: <code>~/.starcluster/config</code>
#Create a new cluster section in your configuration file: <code>~/.starcluster/config</code>
{{GotCloud: Amazon Demo Setup|hdr=====}}
{{GotCloud: Amazon Demo Setup|hdr=====}}
==== Run GotCloud SnpCall ====
==== Run GotCloud SnpCall Demo ====
# Run GotCloud snpcall
# Run GotCloud snpcall
#* <pre>gotcloud snpcall --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
#* <pre>gotcloud snpcall --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
#** The ubuntu user is setup to have the gotcloud program and tools in its path, so you can just type the program name and it will be found
#** The ubuntu user is setup to have the gotcloud program and tools in its path, so you can just type the program name and it will be found
#** There is enough space in /home/ubuntu to put the Demo output
#*** /home/ubuntu is visible from all nodes in the cluster
#** This will take a few minutes to run.
#** This will take a few minutes to run.
#** GotCloud first generates a makefile, and then runs the makefile
#** GotCloud first generates a makefile, and then runs the makefile
#** After a while GotCloud snpcall will print some messages to the screen. This is expected and ok.
#** After a while GotCloud snpcall will print some messages to the screen. This is expected and ok.
# See [[#Monitor Cluster Usage|Monitor Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs
# See [[#Monitoring Cluster Usage|Monitoring Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs
# When complete, GotCloud snpcall will indicate success/failure
# When complete, GotCloud snpcall will indicate success/failure
# Run GotCloud indell
#* Look at the snpcall results, see: [[#GotCloud:_Amazon_Demo#Examining_SnpCall_Output|GotCloud: Amazon Demo -> Examining SnpCall Output]]
#* <pre>gotcloud snpcall --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
==== Run GotCloud Indel Demo ====
# Run GotCloud indel
#* <pre>gotcloud indel --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
==== Terminate the Demo Cluster ====
# Terminate the cluster
# Terminate the cluster
#* Since this is just a demo, we don't have to worry about the data getting deleted upon termination
#* <pre>starcluster terminate mycluster</pre>
#* <pre>starcluster terminate mycluster</pre>
== Old Instructions==
== Old Instructions==
