GotCloud - Revision history

Hmkang: /* Getting Help with GotCloud */

2021-09-11T21:23:16Z

Getting Help with GotCloud

← Older revision		Revision as of 17:23, 11 September 2021
Line 33:		Line 33:
	Currently the "join" button appears to be missing. Click "NEW TOPIC", then select "Join this group". You can then cancel the message post (or post a message).		Currently the "join" button appears to be missing. Click "NEW TOPIC", then select "Join this group". You can then cancel the message post (or post a message).

	See [[GotCloud: FAQs]] if you have any questions. If your questions are not answered there, ~~join~~ the ~~google group and ask, or email Mary Kate Wing (mktrost@umich.edu)~~.		See [[GotCloud: FAQs]] if you have any questions. If your questions are not answered there, ask questions in the [https://github.com/statgen/gotcloud GotCloud GitHub repository]


	=== Sequence Analysis Background Information ===		=== Sequence Analysis Background Information ===

Pjvh: /* GotCloud Setup */

2015-12-15T22:09:32Z

GotCloud Setup

← Older revision		Revision as of 18:09, 15 December 2015
Line 65:		Line 65:
	** You can run on an EC2 cluster instance created by StarCluster.		** You can run on an EC2 cluster instance created by StarCluster.

	GotCloud has been developed and tested on Linux Ubuntu 12.10 and 12.04.2 LTS. While it should work on other Linux systems, they have not yet been tested.		GotCloud has been developed and tested on Linux Ubuntu 12.10 and 12.04.2 LTS and Red Hat 6.6. While it should work on other Linux systems, they have not yet been tested.

	=== GotCloud on Amazon ===		=== GotCloud on Amazon ===

Pjvh: /* Running GotCloud Software */ Remove broken links. Update pipeline status.

2015-09-01T21:43:17Z

Running GotCloud Software: Remove broken links. Update pipeline status.

← Older revision		Revision as of 17:43, 1 September 2015
Line 121:		Line 121:
	** [[GotCloud: Alignment Sub-Pipelines\|Alignment Sub-Pipelines]] - for if you do not want to run the entire Alignment Pipeline		** [[GotCloud: Alignment Sub-Pipelines\|Alignment Sub-Pipelines]] - for if you do not want to run the entire Alignment Pipeline
	* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]		* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]
	* ~~[[GotCloud:~~ Indel Calling Pipeline~~\|Indel Calling Pipeline]]~~		* Indel Calling Pipeline
	* [[GotCloud: GenomeSTRiP Pipeline\|GenomeSTRiP Pipeline]] (Structural Variation) ~~- ''Coming Soon''~~		* [[GotCloud: GenomeSTRiP Pipeline\|GenomeSTRiP Pipeline]] (Structural Variation)
	* ~~[[GotCloud:~~ MEI Calling Pipeline~~\|MEI Calling Pipeline]]~~ - ''~~Coming Soon~~''		* MEI Calling Pipeline - ''Ask if you're interested''

	You can also create your own pipelines. Instructions are here:		You can also create your own pipelines. Instructions are here:

Mktrost at 16:52, 26 May 2015

2015-05-26T16:52:34Z

← Older revision		Revision as of 12:52, 26 May 2015
Line 25:		Line 25:

	[[File:Gotcloud.puzzles.v2.png\|500px]]		[[File:Gotcloud.puzzles.v2.png\|500px]]


	=== Getting Help with GotCloud ===		=== Getting Help with GotCloud ===
Line 43:		Line 44:
	The fastq files are processed using the [[GotCloud: Alignment Pipeline\|alignment pipeline]] which finds the most likely genomic location for each read and stores that information in a [[BAM\|BAM (Binary Sequence Alignment/Map format) file]]. In addition to the sequence and base quality information contained in FASTQ files, a BAM file also contains the genomic location and some additional information about the mapping. As part of the [[GotCloud: Alignment Pipeline\|alignment pipeline]], the base qualities are adjusted to more accurately reflect the likelihood that the base is correct.		The fastq files are processed using the [[GotCloud: Alignment Pipeline\|alignment pipeline]] which finds the most likely genomic location for each read and stores that information in a [[BAM\|BAM (Binary Sequence Alignment/Map format) file]]. In addition to the sequence and base quality information contained in FASTQ files, a BAM file also contains the genomic location and some additional information about the mapping. As part of the [[GotCloud: Alignment Pipeline\|alignment pipeline]], the base qualities are adjusted to more accurately reflect the likelihood that the base is correct.

	The [[GotCloud: Alignment Pipeline\|alignment pipeline]] can be skipped if you already have Deduped and Recalibrated BAM files.		The [[GotCloud: Alignment Pipeline\|alignment pipeline]] can be skipped if you already have Deduped and Recalibrated BAM files. If you have BAMs, but they needed to be deduped and recalibrated, you can use our [[GotCloud:_Alignment_Sub-Pipelines#recabQC_2\|recabQC pipeline]].

	The [[GotCloud: Variant Calling Pipeline\|variant calling pipeline]] processes the deduped and recalibrated BAM files produced by the alignment pipeline or that you provide it, generating an initial list of polymorphic sites and genotypes stored in a [http://www.1000genomes.org/wiki/Analysis/Variant%20Call%20Format/vcf-variant-call-format-version-41 VCF (Variant Call Format) file]. The [[GotCloud: Variant Calling Pipeline\|variant calling pipeline]] then filters the variants using both hard filters and a [[SVM Filtering\|Support Vector Machine (SVM)]]. It then uses haplotype information to refine these genotypes in an updated VCF file.		The [[GotCloud: Variant Calling Pipeline\|variant calling pipeline]] processes the deduped and recalibrated BAM files produced by the alignment pipeline or that you provide it, generating an initial list of polymorphic sites and genotypes stored in a [http://www.1000genomes.org/wiki/Analysis/Variant%20Call%20Format/vcf-variant-call-format-version-41 VCF (Variant Call Format) file]. The [[GotCloud: Variant Calling Pipeline\|variant calling pipeline]] then filters the variants using both hard filters and a [[SVM Filtering\|Support Vector Machine (SVM)]]. It then uses haplotype information to refine these genotypes in an updated VCF file.
Line 50:		Line 51:

	[[File:GotCloudDiagram.jpg\|500px]]		[[File:GotCloudDiagram.jpg\|500px]]


			== Publication ==
			If you use GotCloud, please cite our publication:
			[http://genome.cshlp.org/content/early/2015/04/14/gr.176552.114.abstract Jun, Goo, et al. "An efficient and scalable analysis framework for variant extraction and refinement from population scale DNA sequence data." Genome research (2015): gr-176552.]

	== GotCloud Setup ==		== GotCloud Setup ==

Kleckner: /* Running GotCloud Software */

2015-03-11T22:48:53Z

Running GotCloud Software

← Older revision		Revision as of 18:48, 11 March 2015
Line 113:		Line 113:

	* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]		* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]
	** [[Alignment Sub-Pipelines]] - for if you do not want to run the entire Alignment Pipeline		** [[GotCloud: Alignment Sub-Pipelines\|Alignment Sub-Pipelines]] - for if you do not want to run the entire Alignment Pipeline
	* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]		* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]
	* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]		* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]

Kleckner: /* Running GotCloud Software */

2015-03-11T22:47:24Z

Running GotCloud Software

← Older revision		Revision as of 18:47, 11 March 2015
Line 113:		Line 113:

	* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]		* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]
	** [[~~Creating an AMI on EC2~~]]		** [[Alignment Sub-Pipelines]] - for if you do not want to run the entire Alignment Pipeline
	* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]		* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]
	* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]		* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]

Kleckner: /* Running GotCloud Software */

2015-03-11T22:45:47Z

Running GotCloud Software

← Older revision		Revision as of 18:45, 11 March 2015
Line 113:		Line 113:

	* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]		* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]
			** [[Creating an AMI on EC2]]
	* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]		* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]
	* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]		* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]

Mktrost: /* Running GotCloud Software */

2015-03-09T17:04:13Z

Running GotCloud Software

← Older revision		Revision as of 13:04, 9 March 2015
Line 117:		Line 117:
	* [[GotCloud: GenomeSTRiP Pipeline\|GenomeSTRiP Pipeline]] (Structural Variation) - ''Coming Soon''		* [[GotCloud: GenomeSTRiP Pipeline\|GenomeSTRiP Pipeline]] (Structural Variation) - ''Coming Soon''
	* [[GotCloud: MEI Calling Pipeline\|MEI Calling Pipeline]] - ''Coming Soon''		* [[GotCloud: MEI Calling Pipeline\|MEI Calling Pipeline]] - ''Coming Soon''

			You can also create your own pipelines. Instructions are here:
			* [[GotCloud: Creating a New Pipeline]]

	=== GotCloud Demos ===		=== GotCloud Demos ===

Mktrost: /* UMich Development/Release How-To Notes */

2015-02-25T22:03:24Z

UMich Development/Release How-To Notes

← Older revision		Revision as of 18:03, 25 February 2015
Line 131:		Line 131:

	== UMich Development/Release How-To Notes ==		== UMich Development/Release How-To Notes ==
	* [[~~Creating Packages~~]]		* [[Releasing GotCloud]]
	* Amazon EC2		* Amazon EC2
	** [[Creating an AMI on EC2]]		** [[Creating an AMI on EC2]]

Mktrost: /* UMich Development/Release How-To Notes */

2015-02-23T18:13:48Z

UMich Development/Release How-To Notes

← Older revision		Revision as of 14:13, 23 February 2015
Line 138:		Line 138:
	** Notes on sequence data preparation in [[Amazon Storage\|Amazon Storage]].		** Notes on sequence data preparation in [[Amazon Storage\|Amazon Storage]].

	* [[Upgrade Git Subtree]]		* [[Git_FAQs#Subtrees\|Upgrade Git Subtree]]

← Older revision		Revision as of 18:48, 11 March 2015
Line 113:		Line 113:

	* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]		* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]
	** [[Alignment Sub-Pipelines]] - for if you do not want to run the entire Alignment Pipeline		** [[GotCloud: Alignment Sub-Pipelines\|Alignment Sub-Pipelines]] - for if you do not want to run the entire Alignment Pipeline
	* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]		* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]
	* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]		* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]

← Older revision		Revision as of 18:45, 11 March 2015
Line 113:		Line 113:

	* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]		* [[GotCloud: Alignment Pipeline\|Alignment Pipeline]]
			** [[Creating an AMI on EC2]]
	* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]		* [[GotCloud: Variant Calling Pipeline\|Variant Calling Pipeline]]
	* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]		* [[GotCloud: Indel Calling Pipeline\|Indel Calling Pipeline]]