Sequencing Workshop Analysis of Indels - Revision history

Atks: /* Normalization */

2014-06-16T20:47:12Z

Normalization

← Older revision		Revision as of 16:47, 16 June 2014
Line 321:		Line 321:
	To normalize and remove duplicate variants:		To normalize and remove duplicate variants:

	vt normalize mills.genotypes.bcf -r ~~~/ref/vt/grch37/~~hs37d5.fa \| vt mergedups - -o mills.normalized.genotypes.bcf		vt normalize mills.genotypes.bcf -r hs37d5.fa \| vt mergedups - -o mills.normalized.genotypes.bcf

	and you will observe that 3994 variants had to be left aligned and 1092 variants were removed.		and you will observe that 3994 variants had to be left aligned and 1092 variants were removed.

Atks: /* Comparison with other data sets */

2014-06-16T20:46:43Z

Comparison with other data sets

← Older revision		Revision as of 16:46, 16 June 2014
Line 159:		Line 159:
	It is usually useful to examine the call sets against known data sets for the passed variants.		It is usually useful to examine the call sets against known data sets for the passed variants.

	vt profile_indels -g ~~/net/fantasia/home/atks/ref/vt/grch37/~~indel.reference.txt -r ~~/net/fantasia/home/atks/ref/vt/grch37/~~hs37d5.fa ~~run/final/~~all.genotypes.bcf -i 22:36000000-37000000 -f "PASS"		vt profile_indels -g indel.reference.txt -r hs37d5.fa all.genotypes.bcf -i 22:36000000-37000000 -f "PASS"

	data set		data set
Line 198:		Line 198:
	We perform the same analysis for the failed variants again, the relatively low overlap with known data sets imply a reasonable tradeoff in sensitivity and specificity.		We perform the same analysis for the failed variants again, the relatively low overlap with known data sets imply a reasonable tradeoff in sensitivity and specificity.

	vt profile_indels -g ~~/net/fantasia/home/atks/ref/vt/grch37/~~indel.reference.txt -r ~~/net/fantasia/home/atks/ref/vt/grch37/~~hs37d5.fa ~~run/final/~~all.genotypes.bcf -i 22:36000000-37000000 -f "~PASS"		vt profile_indels -g indel.reference.txt -r hs37d5.fa all.genotypes.bcf -i 22:36000000-37000000 -f "~PASS"

	data set		data set

Atks: /* Summary */

2014-06-16T20:34:58Z

Summary

← Older revision		Revision as of 16:34, 16 June 2014
Line 124:		Line 124:
	no. of observed variants : 720		no. of observed variants : 720

	The variants have filter labels ~~TPASS~~ meaning a temporary pass and overlap, meaning that the variants are overlapping with another variant, implying multiallelicity.		The variants have filter labels PASS meaning a temporary pass and overlap, meaning that the variants are overlapping with another variant, implying multiallelicity.
	We can count the number of variants with the following commands.		We can count the number of variants with the following commands.

	vt peek all.genotypes.bcf -f "FILTER.~~TPASS~~"		vt peek all.genotypes.bcf -f "FILTER.PASS"

	stats: no. of samples : 62		stats: no. of samples : 62
Line 144:		Line 144:

	#passed singletons only		#passed singletons only
	vt peek all.genotypes.bcf -f "FILTER.~~TPASS~~&&INFO.AC==1"		vt peek all.genotypes.bcf -f "FILTER.PASS&&INFO.AC==1"

	#passed indels of length 1 only		#passed indels of length 1 only
	vt peek all.genotypes.bcf -f "FILTER.~~TPASS~~&&LEN==1"		vt peek all.genotypes.bcf -f "FILTER.PASS&&LEN==1"

	#passed indels of length >4		#passed indels of length >4
	vt peek all.genotypes.bcf -f "FILTER.~~TPASS~~&&LEN>1"		vt peek all.genotypes.bcf -f "FILTER.PASS&&LEN>1"

	#passed singletons of length 4 or insertions of length 3		#passed singletons of length 4 or insertions of length 3
	vt peek all.genotypes.bcf -f "FILTER.~~TPASS~~&&(LEN==4\|\|DLEN==3)"		vt peek all.genotypes.bcf -f "FILTER.PASS&&(LEN==4\|\|DLEN==3)"

	== Comparison with other data sets ==		== Comparison with other data sets ==

Atks: /* Summary */

2014-06-16T20:20:28Z

Summary

← Older revision		Revision as of 16:20, 16 June 2014
Line 140:		Line 140:
	no. of chromosomes : 1 <br>		no. of chromosomes : 1 <br>
	no. Indels : 136		no. Indels : 136
	2 alleles (ins/del) : 136 (1.89) [89/47] #notice the difference insertion deletion ratios ~~differences~~		2 alleles (ins/del) : 136 (1.89) [89/47] #notice the difference in insertion deletion ratios
	>=3 alleles (ins/del) : 0 (-nan) [0/0]		>=3 alleles (ins/del) : 0 (-nan) [0/0]

Atks: /* Header */

2014-06-16T20:19:47Z

Header

← Older revision		Revision as of 16:19, 16 June 2014
Line 39:		Line 39:
	##INFO=<ID=FIC,Number=1,Type=Float,Description="Genotype likelihood based Inbreeding Coefficient">		##INFO=<ID=FIC,Number=1,Type=Float,Description="Genotype likelihood based Inbreeding Coefficient">
	##INFO=<ID=AB,Number=1,Type=Float,Description="Genotype likelihood based Allele Balance">		##INFO=<ID=AB,Number=1,Type=Float,Description="Genotype likelihood based Allele Balance">
	##FILTER=<ID=~~TPASS~~,Description="Temporary pass">		##FILTER=<ID=PASS,Description="Temporary pass">
	##FILTER=<ID=overlap,Description="Overlapping variant">		##FILTER=<ID=overlap,Description="Overlapping variant">

Atks: /* Comparison with other data sets */

2014-06-16T20:13:04Z

Comparison with other data sets

← Older revision		Revision as of 16:13, 16 June 2014
Line 162:		Line 162:

	data set		data set
	No Indels : 613 [0.72]		No Indels : 613 [0.72] #613 passed variants with an insertion deletion ratio of 0.72
	FS/NFS : 0.50 (2/2) #frame shift / non frameshift indels proportion, the bracket gives the counts of the frame shift and non frameshift indels		FS/NFS : 0.50 (2/2) #frame shift / non frameshift indels proportion, the bracket gives the counts of the frame shift and non frameshift indels
	Low complexity : 0.46 (283/613) #fraction of indels in low complexity region, the bracket gives the counts of the indels <br>		Low complexity : 0.46 (283/613) #fraction of indels in low complexity region, the bracket gives the counts of the indels <br>
	1000G #1000 Genomes Phase 1 data set		1000G #1000 Genomes Phase 1 data set
	A-B 371 [0.76] #variants found in ~~data~~ set only		A-B 371 [0.76] #variants found in call set only, square brackets contain insertion deletion ratio
	A&B 242 [0.66]#variants found in both data sets		A&B 242 [0.66] #variants found in both data sets
	B-A 276 [0.89] #variants found in 1000G phase 1 data set only		B-A 276 [0.89] #variants found in 1000G phase 1 data set only
	Precision 39.5% #39.6% of the call set are previously known, so 60.5% are novel variants.		Precision 39.5% #39.5% of the call set are previously known, so 60.5% are novel variants.
	Sensitivity 46.7% #sensitivity of variant calling, 46,7% of known variants from 1000 Genomes were rediscovered <br>		Sensitivity 46.7% #sensitivity of variant calling, 46,7% of known variants from 1000 Genomes were rediscovered <br>
	mills #Mills et al. ~~data~~ set		mills #The gold standard Mills et al. indel set
	A-B 542 [0.68]		A-B 542 [0.68]
	A&B 71 [1.03]		A&B 71 [1.03]
Line 177:		Line 177:
	Precision 11.6%		Precision 11.6%
	Sensitivity 69.6% <br>		Sensitivity 69.6% <br>
	dbsnp # Indels from dbSNP		dbsnp #Indels from dbSNP
	A-B 405 [0.68]		A-B 405 [0.68]
	A&B 208 [0.79]		A&B 208 [0.79]
Line 192:		Line 192:
	Overlap analysis: overlap analysis with other data sets is an indicator of sensitivity.		Overlap analysis: overlap analysis with other data sets is an indicator of sensitivity.

			* 1000G: contains Indels from 1000 Genomes, represent a wide spectrum of variants from many different populations. Variants here have an allele frequency above 0.005.
			* Mills: contains doublehit common indels from the Mills. et al paper and is a relatively good measure of sensitivity for common variants. Because not all Indels in this set is expected to be present in your sample, this actually gives you an underestimate of sensitivity.
	* dbsnp: contains Indels submitted from everywhere, I am not sure what does this represent exactly. But assuming most are real, then precision is a useful estimated quantity from this reference data set.		* dbsnp: contains Indels submitted from everywhere, I am not sure what does this represent exactly. But assuming most are real, then precision is a useful estimated quantity from this reference data set.
	* dbsnp: contains Indels submitted from everywhere, I am not sure what does this represent exactly. But assuming most are real, then precision is a useful estimated quantity from this reference data set.
	* Mills: contains doublehit common indels from the Mills. et al paper and is a relatively good measure of sensitivity for common variants. Because not all Indels in this set is expected to be present in your sample, this actually gives you an underestimate of sensitivity.

	We perform the same analysis for the failed variants again:		We perform the same analysis for the failed variants again, the relatively low overlap with known data sets imply a reasonable tradeoff in sensitivity and specificity.

	vt profile_indels -g /net/fantasia/home/atks/ref/vt/grch37/indel.reference.txt -r /net/fantasia/home/atks/ref/vt/grch37/hs37d5.fa run/final/all.genotypes.bcf -i 22:36000000-37000000 -f "~PASS"		vt profile_indels -g /net/fantasia/home/atks/ref/vt/grch37/indel.reference.txt -r /net/fantasia/home/atks/ref/vt/grch37/hs37d5.fa run/final/all.genotypes.bcf -i 22:36000000-37000000 -f "~PASS"

Atks: /* Comparison with other data sets */

2014-06-16T20:06:29Z

Comparison with other data sets

← Older revision		Revision as of 16:06, 16 June 2014
Line 166:		Line 166:
	Low complexity : 0.46 (283/613) #fraction of indels in low complexity region, the bracket gives the counts of the indels <br>		Low complexity : 0.46 (283/613) #fraction of indels in low complexity region, the bracket gives the counts of the indels <br>
	1000G #1000 Genomes Phase 1 data set		1000G #1000 Genomes Phase 1 data set
	A-B 371 [0.76] #variants found in data set only		A-B 371 [0.76] #variants found in data set only
	A&B 242 [0.66] #variants found in both data sets		A&B 242 [0.66]#variants found in both data sets
	B-A 276 [0.89] #variants found in 1000G phase 1 data set only		B-A 276 [0.89] #variants found in 1000G phase 1 data set only
	Precision 39.5% #39.6% of the call set are previously known, so 60.5% are novel variants.		Precision 39.5% #39.6% of the call set are previously known, so 60.5% are novel variants.

Atks: /* Records */

2014-06-16T20:05:51Z

Records

← Older revision		Revision as of 16:05, 16 June 2014
Line 61:		Line 61:

	22 : chromosome		22 : chromosome
	~~36990877~~ : genome position		36990878 : genome position
	. : this is the ID field that is left blank.		. : this is the ID field that is left blank.
	GGT : the reference sequence that is replaced by the alternative sequence below.		GGT : the reference sequence that is replaced by the alternative sequence below.

Atks: /* Records */

2014-06-16T20:05:29Z

Records

← Older revision		Revision as of 16:05, 16 June 2014
Line 63:		Line 63:
	36990877 : genome position		36990877 : genome position
	. : this is the ID field that is left blank.		. : this is the ID field that is left blank.
	GGT : the reference sequence that is replaced by the alternative sequence below.		GGT : the reference sequence that is replaced by the alternative sequence below.
	G : so this is basically a deletion of GT		G : so this is basically a deletion of GT
	455 : QUAL field denoting validity of this variant, higher the better.		455 : QUAL field denoting validity of this variant, higher the better.
	PASS : a passed variant.		PASS : a passed variant.
	INFO : fields containing information about the variant.		INFO : fields containing information about the variant.
	FORMAT : format field labels for the genotype columns.		FORMAT : format field labels for the genotype columns.
	0/0:0,9,108:9:3,0,6:10 : genotype information.		0/0:0,9,108:9:3,0,6:10 : genotype information.

Atks: /* Comparison with other data sets */

2014-06-16T19:52:03Z

Comparison with other data sets

← Older revision		Revision as of 15:52, 16 June 2014
Line 157:		Line 157:
	== Comparison with other data sets ==		== Comparison with other data sets ==

	It is usually useful to examine the call sets against known data sets.		It is usually useful to examine the call sets against known data sets for the passed variants.

	vt profile_indels -g /net/fantasia/home/atks/ref/vt/grch37/indel.reference.txt -r /net/fantasia/home/atks/ref/vt/grch37/hs37d5.fa run/final/all.genotypes.bcf -i 22:36000000-37000000 -f "PASS"		vt profile_indels -g /net/fantasia/home/atks/ref/vt/grch37/indel.reference.txt -r /net/fantasia/home/atks/ref/vt/grch37/hs37d5.fa run/final/all.genotypes.bcf -i 22:36000000-37000000 -f "PASS"
Line 163:		Line 163:
	data set		data set
	No Indels : 613 [0.72]		No Indels : 613 [0.72]
	FS/NFS : 0.50 (2/2)		FS/NFS : 0.50 (2/2) #frame shift / non frameshift indels proportion, the bracket gives the counts of the frame shift and non frameshift indels
	Low complexity : 0.46 (283/613) <br>		Low complexity : 0.46 (283/613) #fraction of indels in low complexity region, the bracket gives the counts of the indels <br>
	1000G		1000G #1000 Genomes Phase 1 data set
	A-B 371 [0.76]		A-B 371 [0.76] #variants found in data set only
	A&B 242 [0.66]		A&B 242 [0.66] #variants found in both data sets
	B-A 276 [0.89]		B-A 276 [0.89] #variants found in 1000G phase 1 data set only
	Precision 39.5%		Precision 39.5% #39.6% of the call set are previously known, so 60.5% are novel variants.
	Sensitivity 46.7% <br>		Sensitivity 46.7% #sensitivity of variant calling, 46,7% of known variants from 1000 Genomes were rediscovered <br>
	mills		mills #Mills et al. data set
	A-B 542 [0.68]		A-B 542 [0.68]
	A&B 71 [1.03]		A&B 71 [1.03]
	B-A 31 [1.07]		B-A 31 [1.07]
	Precision 11.6%		Precision 11.6%
	Sensitivity 69.6% <br>		Sensitivity 69.6% <br>
	dbsnp		dbsnp # Indels from dbSNP
	A-B 405 [0.68]		A-B 405 [0.68]
	A&B 208 [0.79]		A&B 208 [0.79]
Line 195:		Line 195:
	* dbsnp: contains Indels submitted from everywhere, I am not sure what does this represent exactly. But assuming most are real, then precision is a useful estimated quantity from this reference data set.		* dbsnp: contains Indels submitted from everywhere, I am not sure what does this represent exactly. But assuming most are real, then precision is a useful estimated quantity from this reference data set.
	* Mills: contains doublehit common indels from the Mills. et al paper and is a relatively good measure of sensitivity for common variants. Because not all Indels in this set is expected to be present in your sample, this actually gives you an underestimate of sensitivity.		* Mills: contains doublehit common indels from the Mills. et al paper and is a relatively good measure of sensitivity for common variants. Because not all Indels in this set is expected to be present in your sample, this actually gives you an underestimate of sensitivity.

			We perform the same analysis for the failed variants again:

	vt profile_indels -g /net/fantasia/home/atks/ref/vt/grch37/indel.reference.txt -r /net/fantasia/home/atks/ref/vt/grch37/hs37d5.fa run/final/all.genotypes.bcf -i 22:36000000-37000000 -f "~PASS"		vt profile_indels -g /net/fantasia/home/atks/ref/vt/grch37/indel.reference.txt -r /net/fantasia/home/atks/ref/vt/grch37/hs37d5.fa run/final/all.genotypes.bcf -i 22:36000000-37000000 -f "~PASS"
Line 220:		Line 222:
	Precision 4.7%		Precision 4.7%
	Sensitivity 0.7%		Sensitivity 0.7%


	This analysis supports filters too.		This analysis supports filters too.