Changes

From Genome Analysis Wiki
Jump to navigationJump to search

StarCluster (view source)

Revision as of 10:49, 7 November 2014

225 bytes added ,  10:49, 7 November 2014
→‎Running GotCloud on StarCluster
Line 4: Line 4:     
Back to [[GotCloud: Amazon]]
 
Back to [[GotCloud: Amazon]]
−
  −
<!--        BANNER ACROSS TOP OF PAGE        -->
  −
{| style="width:100%; background:#ffb6c1; margin-top:1.2em; border:1px solid #ccc;" |
  −
| style="width:100%; text-align:center; white-space:nowrap; color:#000;" |
  −
<div style="font-size:162%; border:none; margin:0; padding:.1em; color:#000;">This page is in the process of being updated...(10/29/14)</div>
  −
|}
  −
      
If you have access to your own cluster, your task will be much simpler.
 
If you have access to your own cluster, your task will be much simpler.
Line 77: Line 70:  
*** Set <code>MASTER_IMAGE_ID</code> to the latest GotCloud AMI, see: [[GotCloud: AMIs]]
 
*** Set <code>MASTER_IMAGE_ID</code> to the latest GotCloud AMI, see: [[GotCloud: AMIs]]
 
**** Contains GotCloud, the reference, and the demo files in the /home/ubuntu/ directory that will be visible on all nodes in the cluster
 
**** Contains GotCloud, the reference, and the demo files in the /home/ubuntu/ directory that will be visible on all nodes in the cluster
−
**** Has a 30G volume, but with only 6G available
+
**** Has a 30G volume, but only 6G available
 
*** Set <code>NODE_IMAGE_ID</code> to a StarCluster <code>ubuntu x86_64</code> AMI
 
*** Set <code>NODE_IMAGE_ID</code> to a StarCluster <code>ubuntu x86_64</code> AMI
−
**** Since each node does not need its own 30G volume containing GotCloud, the reference, and the demo files, we use a separate image for the nodes. The nodes can just access the master's copy of GotCloud, etc (thus each node does not require its own 30G volume)
+
**** Since each node does not need its own 30G volume containing GotCloud, the reference, and the demo files, we use a separate image for the nodes.
 +
*** The nodes can just access the master's copy of GotCloud, the reference, and the Amazon demo
 
*** Set <code>NODE_INSTANCE_TYPE</code> to the type of instances you want to start in your cluster
 
*** Set <code>NODE_INSTANCE_TYPE</code> to the type of instances you want to start in your cluster
 
**** See http://aws.amazon.com/ec2/pricing/ for instance descriptions and prices
 
**** See http://aws.amazon.com/ec2/pricing/ for instance descriptions and prices
Line 126: Line 120:  
*** Replace number with the number of concurrent jobs you want to run (probably <code>CLUSTER_SIZE</code> * CPUs in <code>NODE_INSTANCE_TYPE</code>)
 
*** Replace number with the number of concurrent jobs you want to run (probably <code>CLUSTER_SIZE</code> * CPUs in <code>NODE_INSTANCE_TYPE</code>)
 
** Otherwise, run GotCloud as you normally would.
 
** Otherwise, run GotCloud as you normally would.
 +
 +
 +
To login to a specific non-master node, do:
 +
starcluster sshnode -u ubuntu mycluster node001
    
=== Monitoring Cluster Usage ===
 
=== Monitoring Cluster Usage ===
Line 185: Line 183:  
# See [[#Monitoring Cluster Usage|Monitoring Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs
 
# See [[#Monitoring Cluster Usage|Monitoring Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs
 
# When complete, GotCloud snpcall will indicate success/failure
 
# When complete, GotCloud snpcall will indicate success/failure
−
#* Look at the snpcall results, see: [[#GotCloud:_Amazon_Demo#Examining_SnpCall_Output|GotCloud: Amazon Demo -> Examining SnpCall Output]]
+
#* Look at the snpcall results, see: [[GotCloud:_Amazon_Demo#Examining_SnpCall_Output|GotCloud: Amazon Demo -> Examining SnpCall Output]]
    
==== Run GotCloud Indel Demo ====
 
==== Run GotCloud Indel Demo ====
 
# Run GotCloud indel
 
# Run GotCloud indel
 
#* <pre>gotcloud indel --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
 
#* <pre>gotcloud indel --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
 +
#** The ubuntu user is setup to have the gotcloud program and tools in its path, so you can just type the program name and it will be found
 +
#** There is enough space in /home/ubuntu to put the Demo output
 +
#*** /home/ubuntu is visible from all nodes in the cluster
 +
#** This will take a few minutes to run.
 +
# See [[#Monitoring Cluster Usage|Monitoring Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs
 +
# When complete, GotCloud indel will indicate success/failure
 +
#* Look at the indel results, see: [[GotCloud:_Amazon_Demo#Examining_Indel_Output|GotCloud: Amazon Demo -> Examining Indel Output]]
    
==== Terminate the Demo Cluster ====
 
==== Terminate the Demo Cluster ====
 +
# Exit out of your master node
 +
#* <pre>exit</pre>
 
# Terminate the cluster
 
# Terminate the cluster
 
#* Since this is just a demo, we don't have to worry about the data getting deleted upon termination
 
#* Since this is just a demo, we don't have to worry about the data getting deleted upon termination
 
#* <pre>starcluster terminate mycluster</pre>
 
#* <pre>starcluster terminate mycluster</pre>
−
 
+
#** Answer <code>y</code> to the questions <code>Terminate EBS cluster mycluster (y/n)? </code>
    
== Old Instructions==
 
== Old Instructions==
Line 291: Line 298:  
If your data is visible on each node, you're ready to run the software as described
 
If your data is visible on each node, you're ready to run the software as described
 
in [[GotCloud]].
 
in [[GotCloud]].
−
  −
== Running GotCloud on StarCluster ==
  −
To tell GotCloud to run data on the StarCluster you have setup, specify the following on your gotcloud command-line:
  −
-batchtype sgei
  −
  −
Alternatively, you can set the following in your configuration file:
  −
BATCH_TYPE = sgei
 
Retrieved from "http://genome.sph.umich.edu/wiki/Special:MobileDiff/11922...11934"

Navigation menu