GotCloud: Alignment Pipeline - Revision history

Mktrost: /* Optional Configurable Settings */

2016-01-28T22:39:14Z

Optional Configurable Settings

← Older revision		Revision as of 18:39, 28 January 2016
Line 133:		Line 133:

	* BWA_THREADS = -t N		* BWA_THREADS = -t N
	** Fill in the N with the number of threads you want BWA to run with, default is 1		** Fill in the N with the number of threads you want BWA to run with, default is 1
			* BWA_QUAL = -q N
			** Fill in the N with the trim quality you want BWA aln to run with, default is 15. This parameter is only applied to bwa aln. It is not used for BWA_MEM.
			* BWA_MEM_OPTS =
			** Specify any additional bwa mem options using this parameter.
	* SORT_MAX_MEM = 2000000000		* SORT_MAX_MEM = 2000000000
	** Maximum amount of memory used by samtools sort after running bwa		** Maximum amount of memory used by samtools sort after running bwa

Mktrost: /* Recommended Settings */

2015-02-26T18:11:39Z

Recommended Settings

← Older revision		Revision as of 14:11, 26 February 2015
Line 105:		Line 105:
	==== Recommended Settings ====		==== Recommended Settings ====

	~~By default~~, the alignment pipeline uses bwa ~~aln~~. ~~We recommend you use~~ bwa ~~mem instead~~. ~~To use that, set~~ in your configuration file:		As of GotCloud version 1.16, the alignment pipeline uses <code>bwa mem</code> by default. Prior to version 1.16, the default aligner was <code>bwa aln</code>.

			You can override the defaults by setting in your configuration file:
			* to use <code>bwa mem</code> (you do not need to set this in version 1.16 and later since it is the default)
	MAP_TYPE = BWA_MEM		MAP_TYPE = BWA_MEM
			* to use <code>bwa aln</code> (you do not need to set this prior to version 1.16 since it is the default)
			MAP_TYPE = BWA

	==== Additional Required Settings ====		==== Additional Required Settings ====

Mktrost: /* Configuration File */

2015-01-07T15:39:46Z

Configuration File

← Older revision		Revision as of 11:39, 7 January 2015
Line 102:		Line 102:
	=== Configuration File ===		=== Configuration File ===
	{{:GotCloud: Configuration}}		{{:GotCloud: Configuration}}

			==== Recommended Settings ====

			By default, the alignment pipeline uses bwa aln. We recommend you use bwa mem instead. To use that, set in your configuration file:
			MAP_TYPE = BWA_MEM


	==== Additional Required Settings ====		==== Additional Required Settings ====

Mktrost: /* Configuration File */

2014-10-23T22:02:15Z

Configuration File

← Older revision		Revision as of 18:02, 23 October 2014
Line 101:		Line 101:

	=== Configuration File ===		=== Configuration File ===
	Configuration file contains the run-time options including the software binaries and command line arguments. A default configuration file is automatically loaded. Users may specify their own configuration file specifying just the values different than the defaults. The configuration file is not required if there are no values to override.		{{:GotCloud: Configuration}}

	~~Comments begin with a <code>#</code>~~		==== Additional Required Settings ====

	~~Format: KEY = value~~

	~~Where KEY is the item being set and value is its new value~~

	~~See [[#Command-Line Options\|Command-Line Options]] for values that can be set either via command line or via configuration.~~

	~~Note: Command-line options take priority over configuration file settings~~

	==== Required Settings ====
	~~See [[#Reference Files\|Reference Files]] for the required reference file settings.~~

	See [[#FASTQ List File\|FASTQ List File]] for how to set the index file either via command line options or via configuration.		See [[#FASTQ List File\|FASTQ List File]] for how to set the index file either via command line options or via configuration.
Line 136:		Line 125:
	** Fill in the N with the number of threads you want BWA to run with, default is 1		** Fill in the N with the number of threads you want BWA to run with, default is 1
	* SORT_MAX_MEM = 2000000000		* SORT_MAX_MEM = 2000000000
	** Maximum amount of memory used by samtools sort after running bwa		** Maximum amount of memory used by samtools sort after running bwa
	*BATCH_TYPE = type
	** Tells GotCloud to use "type" for sending jobs to the cluster
	** "type" can be mosix, slurm, slurmi, pbs, sge, sgei
	*BATCH_OPTS = options
	** replace "options" with the options you want to pass to your cluster type
	** For example: BATCH_OPTS = -j36,37,38,39,40,41,45,46,47,48,49
	*** Specifies which client nodes mosix should send jobs to.

	== Running the Alignment Pipeline ==		== Running the Alignment Pipeline ==

Mktrost: /* Reference Files */

2014-10-23T18:04:50Z

Reference Files

← Older revision		Revision as of 14:04, 23 October 2014
Line 89:		Line 89:

	=== Reference Files ===		=== Reference Files ===
			See [[GotCloud: Genetic Reference and Resource Files]] for detailed information about the multiple required reference files for the alignment pipeline, including:
			* How to obtain default references
			* Configuration keys & default values
			* How to generate your own references
			* How to point GotCloud to your reference files

	~~The following Reference Files are required:~~		Required Reference File Types:
	* Reference File ~~fasta files~~		* [[GotCloud: Genetic Reference and Resource Files#Reference fasta Files\|Reference fasta Files]]
	** Files required: ~~.fa, -bs.umfa, .GCContent, .amb, .ann, .bwt, .pac, .sa~~		* [[GotCloud: Genetic Reference and Resource Files#DBSNP VCF Files\|DBSNP VCF Files]]
	*** If you don't have the -bs.umfa file, the software will try to create it in the same directory as the reference fasta.		* [[GotCloud: Genetic Reference and Resource Files#HapMap3 VCF Files\|HapMap3 VCF Files]]
	*** .GCContent can be generated using qplot, see: [[~~QPLOT~~#~~Input_files~~\| ~~QPLOT: Input~~ Files~~: --gccontent~~]] ~~and name the resulting file as <code>.fa.GCcontent</code>~~
	*** Use <code>bin/bwa index ref.fa</code> if you need to generate the bwa reference files (.amb, .ann, .bwt, .pac, .sa)
	** Configuration Name: ~~REF - specify the ref.fa/ref.fa.gz name~~
	* DBSNP ~~File - used for recalibration & qplot~~
	** VCF ~~file containing dbsnp variants~~
	** Configuration Name: DBSNP_VCF
	* HapMap3 VCF ~~- used for VerifyBamID~~
	** VCF file containing HM3 variants
	** Configuration Name: HM3_VCF

	~~For more information on obtaining/setting/generating the GotCloud reference files, see:~~ [[GotCloud: Genetic Reference and Resource Files]]

	=== Configuration File ===		=== Configuration File ===

Mktrost: /* Reference Files */

2014-10-23T14:35:46Z

Reference Files

← Older revision		Revision as of 10:35, 23 October 2014
Line 97:		Line 97:
	*** Use <code>bin/bwa index ref.fa</code> if you need to generate the bwa reference files (.amb, .ann, .bwt, .pac, .sa)		*** Use <code>bin/bwa index ref.fa</code> if you need to generate the bwa reference files (.amb, .ann, .bwt, .pac, .sa)
	** Configuration Name: REF - specify the ref.fa/ref.fa.gz name		** Configuration Name: REF - specify the ref.fa/ref.fa.gz name
	* DBSNP File		* DBSNP File - used for recalibration & qplot
	** VCF file containing dbsnp variants		** VCF file containing dbsnp variants
	** Configuration Name: DBSNP_VCF		** Configuration Name: DBSNP_VCF
	* HapMap3 VCF		* HapMap3 VCF - used for VerifyBamID
	** VCF file containing HM3 variants		** VCF file containing HM3 variants
	** Configuration Name: HM3_VCF		** Configuration Name: HM3_VCF

Mktrost: /* Alignment Pipeline Output */

2014-10-23T14:04:36Z

Alignment Pipeline Output

← Older revision		Revision as of 10:04, 23 October 2014
Line 225:		Line 225:
	**** Check the 'FREEMIX' column for genotype-free estimate of contamination 0-1 scale, the lower, the better		**** Check the 'FREEMIX' column for genotype-free estimate of contamination 0-1 scale, the lower, the better
	**** If [FREEMIX] >= 0.03 and [FREELK1]-[FREELK0] is large, possible contamination		**** If [FREEMIX] >= 0.03 and [FREELK1]-[FREELK0] is large, possible contamination
	** Qplot Output -see: [[QPLOT#Diagnose_sequencing_quality\|QPLOT: Diagnose sequencing quality]] for more info on how to use QPLOT results		** Qplot Output - see: [[QPLOT#Diagnose_sequencing_quality\|QPLOT: Diagnose sequencing quality]] for more info on how to use QPLOT results
	*** ''*.qplot.done'' - temp file indicating this step completed successfully		*** ''*.qplot.done'' - temp file indicating this step completed successfully
	*** '''*.qplot.R''' - Rscript that can be used to generate the pdf graphs		*** '''*.qplot.R''' - Rscript that can be used to generate the pdf graphs

Mktrost: /* Alignment Pipeline Output */

2014-10-22T20:50:05Z

Alignment Pipeline Output

← Older revision		Revision as of 16:50, 22 October 2014
Line 207:		Line 207:
	=== Alignment Pipeline Output ===		=== Alignment Pipeline Output ===
	Upon successful completion of the alignment pipeline, you should see the following files/ subdirectories under the user specified output directory:		Upon successful completion of the alignment pipeline, you should see the following files/ subdirectories under the user specified output directory:
	* '''bam.list''' - file containing sample -> BAM mapping that can be used in other GotCloud pipelines		* '''bam.list''' - file containing sample->BAM mapping that can be used in other GotCloud pipelines
	* '''bams/''' - contains the final BAM and bai (BAM index) files		* '''bams/''' - contains the final BAM and bai (BAM index) files
	** '''*.recal.bam'''		** '''*.recal.bam'''
Line 217:		Line 217:
	* Makefiles/ - contains the Makefiles and logs used by GotCloud to run the alignment pipeline		* Makefiles/ - contains the Makefiles and logs used by GotCloud to run the alignment pipeline
	* '''QCFiles/''' - contains quality control results if quality control is not disabled		* '''QCFiles/''' - contains quality control results if quality control is not disabled
	**		** VerifyBamID Output - see [[VerifyBamID#A_guideline_to_interpret_output_files\|VerifyBamID: A guideline to interpret output files]] for more information
			*** *.genoCheck.depthRG - depth distribution of the sequence reads per read group
			*** *.genoCheck.depthSM - depth distribution of the sequence reads per sample
			*** ''*.genoCheck.done'' - temp file indicating this step completed successfully
			*** *.genoCheck.selfRG - per-readGroup statistics describing how well each lane matches to the annotated sample
			*** '''*.genoCheck.selfSM''' - main output file containing the contamination estimate; per-sample statistics describing how well the sample matches to the annotated sample
			**** Check the 'FREEMIX' column for genotype-free estimate of contamination 0-1 scale, the lower, the better
			**** If [FREEMIX] >= 0.03 and [FREELK1]-[FREELK0] is large, possible contamination
			** Qplot Output -see: [[QPLOT#Diagnose_sequencing_quality\|QPLOT: Diagnose sequencing quality]] for more info on how to use QPLOT results
			*** ''*.qplot.done'' - temp file indicating this step completed successfully
			*** '''*.qplot.R''' - Rscript that can be used to generate the pdf graphs
			*** '''*.qplot.stats''' - sample statistics
	* tmp/ - contains intermediate files (most are deleted unless --keepTmp is specified)		* tmp/ - contains intermediate files (most are deleted unless --keepTmp is specified)
	* *.OK - one OK file per sample; indicates the Sample successfully completed alignment		* *.OK - one OK file per sample; indicates the Sample successfully completed alignment

Mktrost: /* Running the Alignment Pipeline */

2014-10-22T20:40:53Z

Running the Alignment Pipeline

@@ Line 201: / Line 201: @@
 On success, you will see:
   Processing finished in nn secs with no errors reported
-You should see a <code>.OK</code> for each Sample in the index file.
+If processing fails part way through, you can pick up where you left off by rerunning gotcloud or the make command.
 If you do not see these <code>.OK</code> files, then your Alignment Pipeline failed.
-On success, the bams/ directory contains the final BAMs and bais.
+'''On success, the bams/ directory contains the final BAMs and bais.'''

Mktrost: /* Reference Files */

2014-10-19T22:37:01Z

Reference Files

← Older revision		Revision as of 18:37, 19 October 2014
Line 103:		Line 103:
	** VCF file containing HM3 variants		** VCF file containing HM3 variants
	** Configuration Name: HM3_VCF		** Configuration Name: HM3_VCF

			For more information on obtaining/setting/generating the GotCloud reference files, see: [[GotCloud: Genetic Reference and Resource Files]]

	=== Configuration File ===		=== Configuration File ===