3,045
edits
Changes
From Genome Analysis Wiki
Jump to navigationJump to search→Run GotCloud Demo Using StarCluster
#*: <pre>[cluster gccluster] KEYNAME = mykey CLUSTER_SIZE = 4 CLUSTER_USER = sgeadmin CLUSTER_SHELL = bash MASTER_IMAGE_ID = ami-6ae65e02 NODE_IMAGE_ID = ami-3393a45a NODE_INSTANCE_TYPE = m3.large</pre>
#*: <pre>[cluster gccluster] KEYNAME = mykey CLUSTER_SIZE = 4 CLUSTER_USER = sgeadmin CLUSTER_SHELL = bash MASTER_IMAGE_ID = ami-6ae65e02 NODE_IMAGE_ID = ami-3393a45a NODE_INSTANCE_TYPE = m3.large</pre>
# Start the cluster:
# Start the cluster:
# <pre>starcluster start -c gccluster mycluster</pre>
#* <pre>starcluster start -c gccluster mycluster</pre>
#* Alternatively, you can change the default template at the start of the configuration file in the <code>[global]</code> section to gccluster: <code>DEFAULT_TEMPLATE=gccluster</code>
#** Alternatively, if you can the default template at the start of the configuration file in the <code>[global]</code> section to gccluster: <code>DEFAULT_TEMPLATE=gccluster</code>, you can run:
#*** <pre>starcluster start mycluster</pre>
#* It will take a few minutes for the cluster to start
# Logon to the cluster as ubuntu:
# Logon to the cluster as ubuntu:
#* <pre>starcluster sshmaster -u ubuntu mycluster</pre>
#* <pre>starcluster sshmaster -u ubuntu mycluster</pre>
#** Type <code>yes</code> if the terminal asks if you want to continue connecting
{{GotCloud: Amazon Demo Setup|hdr=====}}
==== Run GotCloud SnpCall ====
# Run GotCloud snpcall
# Run GotCloud snpcall
#* <pre>gotcloud snpcall --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
#* <pre>gotcloud snpcall --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
#** The ubuntu user is setup to have the gotcloud program and tools in its path, so you can just type the program name and it will be found
#** This will take a few minutes to run.
#** GotCloud first generates a makefile, and then runs the makefile
#** After a while GotCloud snpcall will print some messages to the screen. This is expected and ok.
# See [[#Monitor Cluster Usage|Monitor Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs
# When complete, GotCloud snpcall will indicate success/failure
# Run GotCloud indell
# Run GotCloud indell
#* <pre>gotcloud snpcall --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
#* <pre>gotcloud snpcall --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
== Monitor Cluster Usage ==
* Monitor jobs in the queue
** <pre>qstat</pre>
** This will show you how the currently running jobs and how they are spread across the nodes in your cluster
**:[[File:Qstat.png|800px]]
*** state descriptions:
**** <code>qw</code> : queued and waiting (not yet assigned to a node)
**** <code>r</code> : running
* View Sun Grid Engine Load
** <pre>qhost</pre>
**:[[File:Qhost.png|600px]]
*** ARCH : architecture
*** NCPU : number of CPUs
*** LOAD : current load
*** MEMTOT : total memory
*** MEMUSE : memory in use
*** SWAPTO : swap space
*** SWAPUS : swap space in use
* View the average load per node using:
** <pre>qstat -f</pre>
**:[[File:Qstatf.png|650px]]
*** <code>load_avg</code> field contains the load average for each node
== Old Instructions==
== Old Instructions==
