Karma-colorspace - Revision history

Zhanxw at 03:54, 28 January 2010

2010-01-28T03:54:56Z

← Older revision		Revision as of 23:54, 27 January 2010
Line 82:		Line 82:
	== Auxiliary tools ==		== Auxiliary tools ==

	The ABI SOLiD platform generates separate FASTA and quality files named  XXX.csfasta  and  ~~XXX\_QV~~.qual.  We provide a script  ''solid2csfastq.py''  which converts these into a single color space FASTQ file named  XXX.csfastq.  We believe that a single color space FASTQ file simplifies post processing.<br>		The ABI SOLiD platform generates separate FASTA and quality files named  XXX.csfasta  and  XXX_QV.qual.  We provide a script  ''solid2csfastq.py''  which converts these into a single color space FASTQ file named  XXX.csfastq.  We believe that a single color space FASTQ file simplifies post processing.<br>

	== Choose an appropriate size for word index ==		== Choose an appropriate size for word index ==

Tblackw at 20:19, 2 December 2009

2009-12-02T20:19:38Z

Goncalo: /* Additional Information */

2009-11-20T18:21:40Z

Additional Information

← Older revision		Revision as of 14:21, 20 November 2009
Line 65:		Line 65:
	--pairedReads pair.1.csfastq pair.2.csfastq pair.3.csfastq pair.4.csfastq		--pairedReads pair.1.csfastq pair.2.csfastq pair.3.csfastq pair.4.csfastq

	= ~~<br>~~ Additional Information~~<br>~~ =		= Additional Information =

	== Input file requirement ==		== Input file requirement ==

Goncalo at 18:20, 20 November 2009

2009-11-20T18:20:52Z

← Older revision		Revision as of 14:20, 20 November 2009
Line 5:		Line 5:
	We summarize the input data requirements as following:		We summarize the input data requirements as following:

	*~~ ~~ A binary conversion of the genome reference sequence as nucleotides (see [[#Build_Binary_Reference_Genome_and_Word_Index\|Build Binary Reference Genome and Word Index]]})		* A binary conversion of the genome reference sequence as nucleotides (see [[#Build_Binary_Reference_Genome_and_Word_Index\|Build Binary Reference Genome and Word Index]]})
	*~~ ~~ A binary conversion of the genome reference sequence as colors plus word indices in color space (see [[#Build_Binary_Reference_Genome_and_Word_Index\|Build Binary Reference Genome and Word Index]])
	*~~ ~~ Color space reads in color space FASTQ format (see [[#Input_file_requirement\|Input file requirement]] for a description)		* A binary conversion of the genome reference sequence as colors plus word indices in color space (see [[#Build_Binary_Reference_Genome_and_Word_Index\|Build Binary Reference Genome and Word Index]])
	*~~ ~~ Color space reads longer than a minimum length requirement. (see [[#Minimum_read_length_requirement\|Minimum read length requirement]])
	*~~ ~~ Specify color space parameter when starting KARMA (see [[#Map_Color_Space_Reads\|Map Color Space Reads]])~~<br>~~		* Color space reads in color space FASTQ format (see [[#Input_file_requirement\|Input file requirement]] for a description)

			* Color space reads longer than a minimum length requirement. (see [[#Minimum_read_length_requirement\|Minimum read length requirement]])

			* Specify color space parameter when starting KARMA (see [[#Map_Color_Space_Reads\|Map Color Space Reads]])

	Please note the hardware requirements for KARMA are:		Please note the hardware requirements for KARMA are:

Goncalo: /* Build Binary Reference Genome and Word Index */

2009-11-20T18:19:31Z

Build Binary Reference Genome and Word Index

← Older revision		Revision as of 14:19, 20 November 2009
Line 18:		Line 18:
	<br> We show a complete example demonstrating the whole procedure from building the word index to mapping color space reads in [[#A_Complete_Example\|A Complete Example]].		<br> We show a complete example demonstrating the whole procedure from building the word index to mapping color space reads in [[#A_Complete_Example\|A Complete Example]].

	= Build Binary Reference Genome and Word Index~~<br>~~ =		= Build Binary Reference Genome and Word Index =

	First, build a binary version of the genome reference sequence as nucleotides (option: --createReference).~~ ~~ Suppose that ~~ ~~ NCBI36.fa ~~ ~~ is a FASTA file which contains nucleotide sequences for all chromosomes.~~<br>~~		First, build a binary version of the genome reference sequence as nucleotides (option: --createReference). Suppose that NCBI36.fa is a FASTA file which contains nucleotide sequences for all chromosomes.
	The command to invoke is:~~<br>~~
			The command to invoke is:

	karma --createReference --reference NCBI36.fa		karma --createReference --reference NCBI36.fa

Tblackw: /* Map Color Space Reads */

2009-11-20T02:09:00Z

Map Color Space Reads

← Older revision		Revision as of 22:09, 19 November 2009
Line 48:		Line 48:

	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \
	single.1.csfastq single.2.csfastq single.3.csfastq		single.1.csfastq single.2.csfastq single.3.csfastq

	For paired end color space reads, use the option "--pairedReads".  Suppose the paired end reads are stored in two files,  pair.1.csfastq  and  pair.2.csfastq.  The command to invoke is:<br>		For paired end color space reads, use the option "--pairedReads".  Suppose the paired end reads are stored in two files,  pair.1.csfastq  and  pair.2.csfastq.  The command to invoke is:<br>

	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \
	--pairedReads pair.1.csfastq pair.2.csfastq		--pairedReads pair.1.csfastq pair.2.csfastq

	The mapping results will be stored in a .sam  file named  "pair.1.sam", which contains reads from both files.  If multiple paired end read files are specified on the command line, KARMA will pair the 1st and 2nd files, 3rd and 4th files and etc.<br>		The mapping results will be stored in a .sam  file named  "pair.1.sam", which contains reads from both files.  If multiple paired end read files are specified on the command line, KARMA will pair the 1st and 2nd files, 3rd and 4th files and etc.<br>

	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \
	--pairedReads pair.1.csfastq pair.2.csfastq pair.3.csfastq pair.4.csfastq		--pairedReads pair.1.csfastq pair.2.csfastq pair.3.csfastq pair.4.csfastq

	= <br> Additional Information<br> =		= <br> Additional Information<br> =

Tblackw: /* Map Color Space Reads */

2009-11-20T02:07:59Z

Map Color Space Reads

← Older revision		Revision as of 22:07, 19 November 2009
Line 55:		Line 55:
	--pairedReads pair.1.csfastq pair.2.csfastq		--pairedReads pair.1.csfastq pair.2.csfastq

	The mapping results will be stored in a .sam  file named  "pair.sam", which contains reads from both files.  If multiple paired end read files are specified on the command line, KARMA will pair the 1st and 2nd files, 3rd and 4th files and etc.<br>		The mapping results will be stored in a .sam  file named  "pair.1.sam", which contains reads from both files.  If multiple paired end read files are specified on the command line, KARMA will pair the 1st and 2nd files, 3rd and 4th files and etc.<br>

	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \

Tblackw: /* A Complete Example */

2009-11-20T02:06:48Z

A Complete Example

← Older revision		Revision as of 22:06, 19 November 2009
Line 85:		Line 85:
	= A Complete Example =		= A Complete Example =

	~~ ~~ A wrap-up message for quick start mapping color space reads.<br>		A wrap-up message for quick start mapping color space reads.<br>

	Building binary genome reference and word index:<br>		Building binary genome reference and word index:<br>
Line 92:		Line 92:
	ln -s NCBI36.fa NCBI36CS.fa		ln -s NCBI36.fa NCBI36CS.fa
	karma --colorSpace --createReference --createIndex --reference NCBI36CS.fa		karma --colorSpace --createReference --createIndex --reference NCBI36CS.fa

	~~<br>~~

	Mapping color space reads:<br>		Mapping color space reads:<br>
Line 99:		Line 97:
	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace single.csfastq		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace single.csfastq
	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \
	--pairedReads ~~pair1~~.csfastq ~~pair2~~.csfastq		--pairedReads pair.1.csfastq pair.2.csfastq

	The output files are ''single.sam'' and ''~~pair1~~.sam'' and they conform ~~SAM~~ specification.		The output files are  ''single.sam''  and  ''pair.1.sam''  and they conform to the .sam format specification.

	<br>		<br>

Tblackw: /* Choose an appropriate size for word index */

2009-11-20T02:03:46Z

Choose an appropriate size for word index

← Older revision		Revision as of 22:03, 19 November 2009
Line 81:		Line 81:
	== Choose an appropriate size for word index ==		== Choose an appropriate size for word index ==

	~~Size for word~~ index ~~is sensitive to~~ mapping performance. ~~A small size of word~~ index ~~will increase~~ the number of calculation cycles for a single read and duplications of a single word. On the other side, ~~a big size will~~ require much larger memory. Please also keep in mind that appropriate size is related to your hardware architecture. For ~~practically purpose~~, we ~~found~~ size of 15 is optimal.		The length of the index words influences mapping performance.  Using short index words increases the number of calculation cycles for a single read and duplications of a single word.  On the other side, long index words require much larger memory.  Please also keep in mind that appropriate size is related to your hardware architecture.  For practical purposes, with at least 20 Gb of RAM, we find that a size of 15 is optimal.

	= A Complete Example =		= A Complete Example =

Tblackw: /* Auxiliary tools */

2009-11-20T01:58:42Z

Auxiliary tools

← Older revision		Revision as of 21:58, 19 November 2009
Line 77:		Line 77:
	== Auxiliary tools ==		== Auxiliary tools ==

	The ABI SOLiD platform generates separate FASTA and quality files named  XXX.csfasta  and  XXX\_QV.qual.  We provide a script ''solid2csfastq.py'' which converts these into a single color space FASTQ file named  XXX.csfastq.  We believe that a single color space FASTQ file simplifies post processing.<br>		The ABI SOLiD platform generates separate FASTA and quality files named  XXX.csfasta  and  XXX\_QV.qual.  We provide a script  ''solid2csfastq.py''  which converts these into a single color space FASTQ file named  XXX.csfastq.  We believe that a single color space FASTQ file simplifies post processing.<br>

	== Choose an appropriate size for word index ==		== Choose an appropriate size for word index ==

← Older revision		Revision as of 14:20, 20 November 2009
Line 5:		Line 5:
	We summarize the input data requirements as following:		We summarize the input data requirements as following:

	*~~ ~~ A binary conversion of the genome reference sequence as nucleotides (see [[#Build_Binary_Reference_Genome_and_Word_Index\|Build Binary Reference Genome and Word Index]]})		* A binary conversion of the genome reference sequence as nucleotides (see [[#Build_Binary_Reference_Genome_and_Word_Index\|Build Binary Reference Genome and Word Index]]})
	*~~ ~~ A binary conversion of the genome reference sequence as colors plus word indices in color space (see [[#Build_Binary_Reference_Genome_and_Word_Index\|Build Binary Reference Genome and Word Index]])
	*~~ ~~ Color space reads in color space FASTQ format (see [[#Input_file_requirement\|Input file requirement]] for a description)		* A binary conversion of the genome reference sequence as colors plus word indices in color space (see [[#Build_Binary_Reference_Genome_and_Word_Index\|Build Binary Reference Genome and Word Index]])
	*~~ ~~ Color space reads longer than a minimum length requirement. (see [[#Minimum_read_length_requirement\|Minimum read length requirement]])
	*~~ ~~ Specify color space parameter when starting KARMA (see [[#Map_Color_Space_Reads\|Map Color Space Reads]])~~<br>~~		* Color space reads in color space FASTQ format (see [[#Input_file_requirement\|Input file requirement]] for a description)

			* Color space reads longer than a minimum length requirement. (see [[#Minimum_read_length_requirement\|Minimum read length requirement]])

			* Specify color space parameter when starting KARMA (see [[#Map_Color_Space_Reads\|Map Color Space Reads]])

	Please note the hardware requirements for KARMA are:		Please note the hardware requirements for KARMA are:

← Older revision		Revision as of 16:19, 2 December 2009
Line 24:		Line 24:
	= Build Binary Reference Genome and Word Index =		= Build Binary Reference Genome and Word Index =

	First, build a binary version of the genome reference sequence as nucleotides (option: --createReference). Suppose that NCBI36.fa is a FASTA file which contains nucleotide sequences for all chromosomes.		First, build a binary version of the genome reference sequence as nucleotides (option: --createReference). Suppose that NCBI36.fa is a FASTA file which contains the nucleotide sequences for all chromosomes.

	The command to invoke is:		The command to invoke is:
Line 30:		Line 30:
	karma --createReference --reference NCBI36.fa		karma --createReference --reference NCBI36.fa
	<br>		<br>
	(To let KARMA map nucleotide space reads, one would use instead ''--createIndex''  to create both a binary sequence and the word index files.)<br>		(To let KARMA map nucleotide space reads, one would use instead ''--createIndex''  to create both a packed binary sequence file and the word index files.)<br>

	Second, one also needs to build color space versions of both the genome reference sequence (option: --createReference) and the word index files (option: --createIndex).  The same nucleotide FASTA file is used.  However, to avoid naming conflicts among the resulting binary files, we suggest appending "CS" to the base file name for clarity.  The command to invoke is:<br>		Second, one also needs to build color space versions of both the genome reference sequence (option: --createReference) and the word index files (option: --createIndex).  The same nucleotide FASTA file is used.  However, to avoid naming conflicts among the resulting binary files, we suggest appending "CS" to the base file name for clarity.  The command to invoke is:<br>
Line 43:		Line 43:
	= Map Color Space Reads =		= Map Color Space Reads =

	KARMA expects valid color space FASTQ files as input.  We often use the suffix .csfastq to distinguish these from nucleotide space reads.  ~~For~~ a .csfastq  file of single end color space reads named   single.csfastq,   invoke the command:<br>		KARMA expects valid color space FASTQ files as input.  We often use the suffix .csfastq to distinguish these from nucleotide space reads.  With a .csfastq   file of single end color space reads named   single.csfastq,   invoke the command:<br>

	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace single.csfastq		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace single.csfastq

	This command line specifies both the nucleotide and color space reference sequences (and the word indexes, invisibly).  The output will be written to a file in .sam~~ ~~ format named  "single.sam".<br>		This command line specifies both the nucleotide and color space reference sequences (and the word indexes, invisibly).  The output will be written to a file in .sam format named   "single.sam"  derived from the .fastq  file name.<br>
	<br>		<br>

	Multiple input files are also acceptable, e.g.<br>		Multiple input files are also acceptable and will produce multiple .sam output files, e.g.<br>

	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \
Line 60:		Line 60:
	--pairedReads pair.1.csfastq pair.2.csfastq		--pairedReads pair.1.csfastq pair.2.csfastq

	The mapping results will be stored in a .sam  file named  "pair.1.sam", which contains reads from both files.  If multiple paired end read files are specified on the command line, KARMA will pair the 1st and 2nd files, 3rd and 4th files and etc.<br>		The mapping results will be stored in a .sam  file named  "pair.1.sam", which contains reads from both files.  If multiple paired end read files are specified on the command line, KARMA will pair the 1st and 2nd files, 3rd and 4th files, etc. and write output files  "pair.1.sam", "pair.3.sam", etc.<br>

	karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \		karma --reference NCBI36.fa --csReference NCBI36CS.fa --colorSpace \
Line 78:		Line 78:
	== Minimum read length requirement ==		== Minimum read length requirement ==

	Keep in mind that KARMA requires color space reads that are at least twice as long as the index word size plus two (including the leading primer base).  (For nucleotide space, the minimum read length is twice the word size.)  For example, KARMA uses an index word size of 15 by default, so it will only map color space reads that are 32 ~~base pairs~~ or longer.<br>		Keep in mind that KARMA requires color space reads that are at least twice as long as the index word size plus two (including the leading primer base).  (For nucleotide space, the minimum read length is twice the word size.)  For example, KARMA uses an index word size of 15 by default, so it will only map color space reads that are 32 colors or longer (including the primer base).<br>

	== Auxiliary tools ==		== Auxiliary tools ==