3,045
edits
Changes
From Genome Analysis Wiki
Jump to navigationJump to search→Required Parameters
NOTE: This tool does not properly work on templates that have more than 2 segments. It does not properly match reads when more than 2 reads have the same read name.
NOTE: This tool does not properly work on templates that have more than 2 segments. It does not properly match reads when more than 2 reads have the same read name.
NOTE: Dedup cannot read from stdin since it reads the input file twice.
Potential future features:
Potential future features:
The deduper assumes that duplicates in the input BAM file are not marked. When the deduper detects a marked duplicate in the input BAM file, it will throw an error and stop. To override this behavior, use the [[#Ignore Previous Duplicate Marking (--force)|<code>--force</code>]] option; in this mode, alignments that are marked as duplicates in the input file are unmarked before the deduper begins its detection algorithm. The result is that only duplicates detected by the deduper will be marked in or removed from the output file.
The deduper assumes that duplicates in the input BAM file are not marked. When the deduper detects a marked duplicate in the input BAM file, it will throw an error and stop. To override this behavior, use the [[#Ignore Previous Duplicate Marking (--force)|<code>--force</code>]] option; in this mode, alignments that are marked as duplicates in the input file are unmarked before the deduper begins its detection algorithm. The result is that only duplicates detected by the deduper will be marked in or removed from the output file.
The handling of paired-end reads assumes that the mate information in the SAM/BAM records is accurate. If a mate is not found at the expected position, an error message is printed (once per file) indicating this error. Paired-end reads whose mate cannot be found are not marked duplicate and are not used for duplicate marking of other paired-end reads. Single-end reads with the same key as paired-end reads whose mate cannot be found are still marked as duplicate. If this error is encountered, you may want to fix the mate information and reprocess the file through the deduper. Use the [[#Treat Reads with Mates On Different Chromosomes As Single-Ended (--oneChrom)|<code>--oneChrom</code>]] option to treat reads with a mate on a different chromosome as single-ended. This option is useful if you are running the deduper on just a single chromosome.
The handling of paired-end reads assumes that the mate information in the SAM/BAM records is accurate. If a mate is not found at the expected position, an error message is printed (once per file) indicating this error. Paired-end reads whose mate cannot be found are not marked duplicate and are not used for duplicate marking of other paired-end reads. Single-end reads with the same key as paired-end reads whose mate cannot be found are still marked as duplicate. If this error is encountered, you may want to fix the mate information and reprocess the file through the deduper.
With the default settings this tool should produce similar results as Picard.
Use the [[#Treat Reads with Mates On Different Chromosomes As Single-Ended (--oneChrom)|<code>--oneChrom</code>]] option to treat reads with a mate on a different chromosome as single-ended. This option is useful if you are running the deduper on just a single chromosome. The code will use less memory with this option if mates are found on different chromosomes. (Picard does not specially handle mates on different chromosomes, so the --oneChrom option may produce different results than Picard.)
Rules:
Rules:
* Skip Unmapped Reads, they are not marked as duplicate
* Skip Unmapped Reads, they are not marked as duplicate
* Reads whose mate is unmapped are treated as single-end
* Mark a Single-End Read Duplicate (or remove it if configured to do so) if:
* Mark a Single-End Read Duplicate (or remove it if configured to do so) if:
*# A paired-end record has the same key (even if the pair is not proper/the mate is unmapped/the mate is not found)<br/>-OR-
*# A paired-end record has the same key (even if the pair is not proper/the mate is not found)<br/>-OR-
*# A single-end record has the same key and a higher base quality sum (sum of all base qualities in the record above [[#Minimum Quality for Quality Calculations (--minQual)|<code>--minBaseQual</code>]])
*# A single-end record has the same key and a higher base quality sum (sum of all base qualities in the record above [[#Minimum Quality for Quality Calculations (--minQual)|<code>--minBaseQual</code>]])
* Mark both Paired-End Reads Duplicate if:
* Mark both Paired-End Reads Duplicate if:
This code assumes that at most 1000 bases are clipped at the start of a read.
This code assumes that at most 1000 bases are clipped at the start of a read.
Deduping requires two passes through the file, so cannot read from stdin.
==Handling Recalibration==
==Handling Recalibration==
Recommended usage with Recalibrator:
Recommended usage with Recalibrator:
/usr/cluster/bin/bam dedup --recab --in ${INPUT}.bam --out ${OUTPUT}.bam --force --refFile ${REF} --dbsnp ${DBSNP} --skipFit --oneChrom --storeQualTag OQ --maxBaseQual 40
/usr/cluster/bin/bam dedup --recab --in ${INPUT}.bam --out ${OUTPUT}.bam --force --refFile ${REF} --dbsnp ${DBSNP} --oneChrom --storeQualTag OQ --maxBaseQual 40
= Usage =
= Usage =
./bam dedup --in <InputBamFile> --out <OutputBamFile> [--minQual <minPhred>] [--log <logFile>] [--oneChrom] [--rmDups] [--force] [--verbose] [--noeof] [--params] [--recab]
./bam dedup --in <InputBamFile> --out <OutputBamFile> [--minQual <minPhred>] [--log <logFile>] [--oneChrom] [--rmDups] [--force] [--excludeFlags <flag>] [--verbose] [--noeof] [--params] [--recab]
Additional Recalibration Usage is documented at [[BamUtil: recab#Usage|BamUtil: recab -> Usage]]
Additional Recalibration Usage is documented at [[BamUtil: recab#Usage|BamUtil: recab -> Usage]]
duplicates and apply this duplicate marking logic. Default is to throw errors
duplicates and apply this duplicate marking logic. Default is to throw errors
and exit when trying to run on an already mark-duplicated BAM
and exit when trying to run on an already mark-duplicated BAM
--excludeFlags <flag> : exclude reads with any of these flags set when determining or marking duplicates
by default (0xB04): exclude unmapped, secondary reads, QC failures, and supplementary reads
--verbose : Turn on verbose mode
--verbose : Turn on verbose mode
--noeof : Do not expect an EOF block on a bam file.
--noeof : Do not expect an EOF block on a bam file.
--recab : Recalibrate in addition to deduping
--recab : Recalibrate in addition to deduping
</pre>
</pre>
{{PhoneHomeParamDesc}}
Additional Recalibration Parameters are documented at [[BamUtil: recab#Parameters|BamUtil: recab -> Parameters]]
Additional Recalibration Parameters are documented at [[BamUtil: recab#Parameters|BamUtil: recab -> Parameters]]
{{inBAMInputFile}}
== Required Parameters ==
{{inBAMInputFile|noStdin=1}}
Note: The input file must be sorted by coordinate.
Note: The input file must be sorted by coordinate.
{{outBAMOutputFile}}
{{outBAMOutputFile}}
== Minimum Quality for Quality Calculations (<code>--minQual</code>) ==
== Optional Parameters==
=== Minimum Quality for Quality Calculations (<code>--minQual</code>) ===
When duplicate reads are encountered, the read with the highest quality is kept.
When duplicate reads are encountered, the read with the highest quality is kept.
To determine the quality of a read, all of the phred base quality scores above the <code>--minQual</code> value are added together. If <code>--minQual</code> is not specified, it is defaulted to 15.
To determine the quality of a read, all of the phred base quality scores above the <code>--minQual</code> value are added together. If <code>--minQual</code> is not specified, it is defaulted to 15.
== Output log & Summary Statistics FileName (<code>--log</code>) ==
=== Output log & Summary Statistics FileName (<code>--log</code>) ===
Output file name for writing logs & summary statistics.
Output file name for writing logs & summary statistics.
If this parameter is not specified, it will write to the output file specified in <code>--out</code> + ".log". Or if the output bam is written to stdout (<code>--out</code> starts with '-'), the logs will be written to stderr. If the filename after --log starts with '-' it will write to stderr.
If this parameter is not specified, it will write to the output file specified in <code>--out</code> + ".log". Or if the output bam is written to stdout (<code>--out</code> starts with '-'), the logs will be written to stderr. If the filename after --log starts with '-' it will write to stderr.
== Treat Reads with Mates On Different Chromosomes As Single-Ended (<code>--oneChrom</code>) ==
=== Treat Reads with Mates On Different Chromosomes As Single-Ended (<code>--oneChrom</code>) ===
If a read's mate is not found it will not be used for duplicate marking. If you are running on a single chromosome, all read's whose mates are on different chromosomes will not be used for duplicate marking. The <code>--oneChrom</code> option will treat reads with mates on a different chromosome as single-ended.
If a read's mate is not found it will not be used for duplicate marking. If you are running on a single chromosome, all read's whose mates are on different chromosomes will not be used for duplicate marking. The <code>--oneChrom</code> option will treat reads with mates on a different chromosome as single-ended.
== Remove Duplicates (<code>--rmDups</code>) ==
=== Remove Duplicates (<code>--rmDups</code>) ===
Instead of marking a read as duplicate in the flag, the <code>--rmDups</code> option will remove it from the output BAM file.
Instead of marking a read as duplicate in the flag, the <code>--rmDups</code> option will remove it from the output BAM file.
== Ignore Previous Duplicate Marking (<code>--force</code>) ==
=== Ignore Previous Duplicate Marking (<code>--force</code>) ===
By default the deduper will throw an error and stop if a read is already marked as duplicate. The <code>--force</code> option will removes any previous duplicate marking and marks the reads from scratch. The resulting output file will only have reads determined by the deduper marked as duplicates.
By default the deduper will throw an error and stop if a read is already marked as duplicate. The <code>--force</code> option will removes any previous duplicate marking and marks the reads from scratch. The resulting output file will only have reads determined by the deduper marked as duplicates.
== Turn on Verbose Mode (<code>--verbose</code>) ==
=== Skip Records with any of the Specified Flags (<code>--excludeFlags</code>)===
Skip records with any of the specified flags set, default 0xB04
By default skips reads with any of the following flags set:
* unmapped
* secondary alignment
* fails QC checks
* supplementary reads
Secondary (0x100) and Supplementary (0x800) reads currently must be excluded.
=== Turn on Verbose Mode (<code>--verbose</code>) ===
Turn on verbose logging to get more log messages in the log and to stderr.
Turn on verbose logging to get more log messages in the log and to stderr.
See [[BamUtil: recab]] for recalibration details.
See [[BamUtil: recab]] for recalibration details.
{{PhoneHomeParameters}}
= Return Value =
= Return Value =
Returns -1 if input parameters are invalid.
Returns -1 if input parameters are invalid.
Returns the SamStatus for the reads/writes (0 on success).
Returns the SamStatus for the reads/writes (0 on success, non-0 on failure).
