StarCluster - Revision history

Mktrost: /* Running GotCloud on StarCluster */

2014-11-07T14:49:24Z

Running GotCloud on StarCluster

← Older revision		Revision as of 10:49, 7 November 2014
Line 120:		Line 120:
	*** Replace number with the number of concurrent jobs you want to run (probably <code>CLUSTER_SIZE</code> * CPUs in <code>NODE_INSTANCE_TYPE</code>)		*** Replace number with the number of concurrent jobs you want to run (probably <code>CLUSTER_SIZE</code> * CPUs in <code>NODE_INSTANCE_TYPE</code>)
	** Otherwise, run GotCloud as you normally would.		** Otherwise, run GotCloud as you normally would.


			To login to a specific non-master node, do:
			starcluster sshnode -u ubuntu mycluster node001

	=== Monitoring Cluster Usage ===		=== Monitoring Cluster Usage ===

Mktrost: /* Terminate the Demo Cluster */

2014-10-31T16:57:35Z

Terminate the Demo Cluster

← Older revision		Revision as of 12:57, 31 October 2014
Line 193:		Line 193:

	==== Terminate the Demo Cluster ====		==== Terminate the Demo Cluster ====
			# Exit out of your master node
			#* <pre>exit</pre>
	# Terminate the cluster		# Terminate the cluster
	#* Since this is just a demo, we don't have to worry about the data getting deleted upon termination		#* Since this is just a demo, we don't have to worry about the data getting deleted upon termination
	#* <pre>starcluster terminate mycluster</pre>		#* <pre>starcluster terminate mycluster</pre>
			#** Answer <code>y</code> to the questions <code>Terminate EBS cluster mycluster (y/n)? </code>

	== Old Instructions==		== Old Instructions==

Mktrost at 16:53, 31 October 2014

2014-10-31T16:53:42Z

← Older revision		Revision as of 12:53, 31 October 2014
Line 4:		Line 4:

	Back to [[GotCloud: Amazon]]		Back to [[GotCloud: Amazon]]

	~~<!-- BANNER ACROSS TOP OF PAGE -->~~
	~~{\| style="width:100%; background:#ffb6c1; margin-top:1.2em; border:1px solid #ccc;" \|~~
	~~\| style="width:100%; text-align:center; white-space:nowrap; color:#000;" \|~~
	~~<div style="font-size:162%; border:none; margin:0; padding:.1em; color:#000;">This page is in the process of being updated...(10/29/14)</div>~~
	\|}


	If you have access to your own cluster, your task will be much simpler.		If you have access to your own cluster, your task will be much simpler.

Mktrost: /* Running GotCloud on StarCluster */

2014-10-31T16:53:27Z

Running GotCloud on StarCluster

← Older revision		Revision as of 12:53, 31 October 2014
Line 298:		Line 298:
	If your data is visible on each node, you're ready to run the software as described		If your data is visible on each node, you're ready to run the software as described
	in [[GotCloud]].		in [[GotCloud]].

	~~== Running GotCloud on StarCluster ==~~
	~~To tell GotCloud to run data on the StarCluster you have setup, specify the following on your gotcloud command-line:~~
	~~-batchtype sgei~~

	~~Alternatively, you can set the following in your configuration file:~~
	~~BATCH_TYPE = sgei~~

Mktrost: /* Run GotCloud Indel Demo */

2014-10-31T16:46:08Z

Run GotCloud Indel Demo

← Older revision		Revision as of 12:46, 31 October 2014
Line 195:		Line 195:
	#*** /home/ubuntu is visible from all nodes in the cluster		#*** /home/ubuntu is visible from all nodes in the cluster
	#** This will take a few minutes to run.		#** This will take a few minutes to run.
			# See [[#Monitoring Cluster Usage\|Monitoring Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs
			# When complete, GotCloud indel will indicate success/failure
			#* Look at the indel results, see: [[GotCloud:_Amazon_Demo#Examining_Indel_Output\|GotCloud: Amazon Demo -> Examining Indel Output]]

	==== Terminate the Demo Cluster ====		==== Terminate the Demo Cluster ====

Mktrost: /* Run GotCloud Demo Using StarCluster */

2014-10-31T16:45:26Z

Run GotCloud Demo Using StarCluster

← Older revision		Revision as of 12:45, 31 October 2014
Line 186:		Line 186:
	# See [[#Monitoring Cluster Usage\|Monitoring Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs		# See [[#Monitoring Cluster Usage\|Monitoring Cluster Usage]] if you are interested in monitoring the cluster usage as GotCloud runs
	# When complete, GotCloud snpcall will indicate success/failure		# When complete, GotCloud snpcall will indicate success/failure
	#* Look at the snpcall results, see: [[#GotCloud:_Amazon_Demo#Examining_SnpCall_Output\|GotCloud: Amazon Demo -> Examining SnpCall Output]]		#* Look at the snpcall results, see: [[GotCloud:_Amazon_Demo#Examining_SnpCall_Output\|GotCloud: Amazon Demo -> Examining SnpCall Output]]

	==== Run GotCloud Indel Demo ====		==== Run GotCloud Indel Demo ====
	# Run GotCloud indel		# Run GotCloud indel
	#* <pre>gotcloud indel --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>		#* <pre>gotcloud indel --conf example/test.conf --outdir output --numjobs 8 --batchtype sgei</pre>
			#** The ubuntu user is setup to have the gotcloud program and tools in its path, so you can just type the program name and it will be found
			#** There is enough space in /home/ubuntu to put the Demo output
			#*** /home/ubuntu is visible from all nodes in the cluster
			#** This will take a few minutes to run.

	==== Terminate the Demo Cluster ====		==== Terminate the Demo Cluster ====
Line 196:		Line 200:
	#* Since this is just a demo, we don't have to worry about the data getting deleted upon termination		#* Since this is just a demo, we don't have to worry about the data getting deleted upon termination
	#* <pre>starcluster terminate mycluster</pre>		#* <pre>starcluster terminate mycluster</pre>


	== Old Instructions==		== Old Instructions==

Mktrost: /* StarCluster Config Settings */

2014-10-31T16:14:36Z

StarCluster Config Settings

← Older revision		Revision as of 12:14, 31 October 2014
Line 77:		Line 77:
	*** Set <code>MASTER_IMAGE_ID</code> to the latest GotCloud AMI, see: [[GotCloud: AMIs]]		*** Set <code>MASTER_IMAGE_ID</code> to the latest GotCloud AMI, see: [[GotCloud: AMIs]]
	**** Contains GotCloud, the reference, and the demo files in the /home/ubuntu/ directory that will be visible on all nodes in the cluster		**** Contains GotCloud, the reference, and the demo files in the /home/ubuntu/ directory that will be visible on all nodes in the cluster
	**** Has a 30G volume, but ~~with~~ only 6G available		**** Has a 30G volume, but only 6G available
	*** Set <code>NODE_IMAGE_ID</code> to a StarCluster <code>ubuntu x86_64</code> AMI		*** Set <code>NODE_IMAGE_ID</code> to a StarCluster <code>ubuntu x86_64</code> AMI
	**** Since each node does not need its own 30G volume containing GotCloud, the reference, and the demo files, we use a separate image for the nodes. The nodes can just access the master's copy of GotCloud, ~~etc (thus each node does not require its own 30G volume)~~		**** Since each node does not need its own 30G volume containing GotCloud, the reference, and the demo files, we use a separate image for the nodes.
			*** The nodes can just access the master's copy of GotCloud, the reference, and the Amazon demo
	*** Set <code>NODE_INSTANCE_TYPE</code> to the type of instances you want to start in your cluster		*** Set <code>NODE_INSTANCE_TYPE</code> to the type of instances you want to start in your cluster
	**** See http://aws.amazon.com/ec2/pricing/ for instance descriptions and prices		**** See http://aws.amazon.com/ec2/pricing/ for instance descriptions and prices

Mktrost: /* StarCluster Config Settings */

2014-10-31T16:13:25Z

StarCluster Config Settings

← Older revision		Revision as of 12:13, 31 October 2014
Line 76:		Line 76:
	*** Set <code>CLUSTER_SHELL</code> to define the shell you want to use, like <code>bash</code>		*** Set <code>CLUSTER_SHELL</code> to define the shell you want to use, like <code>bash</code>
	*** Set <code>MASTER_IMAGE_ID</code> to the latest GotCloud AMI, see: [[GotCloud: AMIs]]		*** Set <code>MASTER_IMAGE_ID</code> to the latest GotCloud AMI, see: [[GotCloud: AMIs]]
			**** Contains GotCloud, the reference, and the demo files in the /home/ubuntu/ directory that will be visible on all nodes in the cluster
			**** Has a 30G volume, but with only 6G available
	*** Set <code>NODE_IMAGE_ID</code> to a StarCluster <code>ubuntu x86_64</code> AMI		*** Set <code>NODE_IMAGE_ID</code> to a StarCluster <code>ubuntu x86_64</code> AMI
			**** Since each node does not need its own 30G volume containing GotCloud, the reference, and the demo files, we use a separate image for the nodes. The nodes can just access the master's copy of GotCloud, etc (thus each node does not require its own 30G volume)
	*** Set <code>NODE_INSTANCE_TYPE</code> to the type of instances you want to start in your cluster		*** Set <code>NODE_INSTANCE_TYPE</code> to the type of instances you want to start in your cluster
	**** See http://aws.amazon.com/ec2/pricing/ for instance descriptions and prices		**** See http://aws.amazon.com/ec2/pricing/ for instance descriptions and prices
Line 89:		Line 92:
	** Create your Own Volumes and attach them to the GotCloud cluster		** Create your Own Volumes and attach them to the GotCloud cluster
	*** '''Instructions TBD'''		*** '''Instructions TBD'''


	=== Starting the Cluster ===		=== Starting the Cluster ===

Mktrost at 16:08, 31 October 2014

2014-10-31T16:08:19Z

← Older revision		Revision as of 12:08, 31 October 2014
Line 1:		Line 1:
			__TOC__

	Back to the beginning: [[GotCloud]]		Back to the beginning: [[GotCloud]]

Mktrost at 16:07, 31 October 2014

2014-10-31T16:07:57Z

Show changes