Difference between revisions of "LibStatGen: BAM"

From Genome Analysis Wiki
Jump to: navigation, search
(BAM/SAM Classes)
 
(55 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
[[Category:C++]]
 +
[[Category:libStatGen]]
 +
[[Category:libStatGen BAM]]
 +
 
= SAM/BAM File=
 
= SAM/BAM File=
  
== Read & Write BAM/SAM Executable ==
+
See the github history: https://github.com/statgen/libStatGen/commits/master/bam for a list of the most recent updates to the BAM classes.
When the pipeline is compiled, the sam/bam executable, "bam" is generated in the pipeline/bam/ directory.
 
 
 
This executable takes 2 arguments.  The first argument is the input file.  The second argument is the output file.  The executable converts the first file into the format of the second file.  So if you want to convert a Bam file to a SAM file, from the pipeline/bam/ directory you just call:
 
./bam <bamFile>.bam <newSamFile>.sam
 
Don't forget to put in the paths to the executable and your test files.
 
 
 
The software reads the beginning of the input file to determine if it is SAM/BAM. To determine the format (SAM/BAM) of the output file, the software checks the output file's extension.  If the extension is ".bam" it writes a BAM file, otherwise it writes a SAM file.
 
 
 
== Reading/Writing SAM/BAM Files In Your Program ==
 
The SamFile class allows a user to easily read/write a SAM/BAM file.
 
The methods found in this class are:
 
{| class="wikitable" style="width:100%" border="1"
 
|+ style="font-size:150%"|'''SamFile Class Methods'''
 
!  width=""|Method Name
 
!  width=""|Description
 
|-
 
| bool OpenForRead(const char* filename)
 
| Opens the specified file for reading.
 
Determines if it is a BAM/SAM file by reading the beginning of the file.
 
Returns true if successfully opened reading, false if not.
 
|-
 
| bool OpenForWrite(const char * filename)
 
| bool: true if successfully opened, false if not.
 
Opens as BAM file if the specified filename ends in .bam.  Otherwise it is opened as a SAM file.
 
Returns true if successfully opened for writing, false if not.
 
|-
 
| bool ReadHeader(SamFileHeader& header)
 
| Reads the header section from the file and stores it in the passed in header.
 
Returns true if successfully read, false if not.
 
|-
 
| bool WriteHeader(const SamFileHeader& header)
 
| Writes the specified header into the file.
 
Returns true if successfully written, false if not.
 
|- 
 
| bool ReadRecord(SamFileHeader& header, SamRecord& record)
 
| Reads the next record from the file and stores it in the passed in record.
 
Returns true if successfully read, false if not.
 
|-
 
| bool WriteRecord(SamFileHeader& header, SamRecord& record)
 
| Writes the specified record into the file.
 
Returns true if successfully written, false if not.
 
|}
 
 
 
=== Usage Example ===
 
The following example reads in a sam/bam file and writes it out as a sam/bam file.  The file format of the input sam/bam is determined by the SamFile class based on reading the type from the file.  The file format of the output sam/bam file is determined by the SamFile class based on the extension of the output file.  A ".bam" extension indicates a BAM file.  All other extensions indicate SAM files.
 
<pre>
 
int main(int argc, char ** argv)
 
{
 
  if(argc != 3)
 
  {
 
      printf("./bam <inputFile> <outputFile.sam/bam>\n");
 
      exit(-1);
 
  }
 
 
 
 
 
  SamFile samIn;
 
     
 
  samIn.OpenForRead(argv[1]);
 
 
 
  SamFile samOut;
 
 
 
  samOut.OpenForWrite(argv[2]);
 
 
 
  // Read the sam header.
 
  SamFileHeader samHeader;
 
  samIn.ReadHeader(samHeader);
 
 
 
  samOut.WriteHeader(samHeader);
 
 
 
  // Read the first sam record.
 
  SamRecord samRecord;
 
 
 
  // Keep reading records until it fails.
 
  int recordCount = 0;
 
  while (samIn.ReadRecord(samHeader, samRecord) == true)
 
  {
 
      recordCount++;
 
      samOut.WriteRecord(samHeader, samRecord);
 
  }
 
  printf("RecordCount = %d\n", recordCount);
 
}
 
</pre>
 
 
 
 
 
== Setting fields in a SAM/BAM Header ==
 
The SamRecord class contains accessors to set the header lines of a SAM/BAM header.  By using these set methods to setup the header, they can be pulled back out using the get accessors or the header can be later written to a SAM/BAM file. 
 
The methods found in the SamFileHeader class for setting fields are:
 
{| class="wikitable" style="width:100%" border="1"
 
|+ style="font-size:150%"|'''SamFileHeader Class Methods'''
 
!  width=""|Method Name
 
!  width=""|Description
 
|-
 
| bool addHeaderLine(const char* type, const char* tag, int value)
 
| Adds the type, tag, and integer value to the header.
 
Returns true if successfully added, false if not.
 
NOTE: currently, this method will only do one tag per type on a line.  If a type has multiple tags, then the whole line needs to be added at once.
 
|-
 
| bool addHeaderLine(const char* type, const char* tag, const char* value)
 
| Adds the type, tag, and const char* value to the header.
 
Returns true if successfully added, false if not.
 
NOTE: currently, this method will only do one tag per type on a line.  If a type has multiple tags, then the whole line needs to be added at once.
 
|-
 
| bool addHeaderLine(const char* headerLine)
 
| Adds the already setup/formatted headerLine to the header.  It is assumed that the line does not contain a “\n”.
 
Returns true if successfully added, false if not.
 
|-
 
|}
 
 
 
  
== Setting fields in a SAM/BAM Record ==
+
[[BAM Review Action Items|Old BAM Review Action Items]]
The SamRecord class contains accessors to set the fields of a SAM/BAM record.  They are used for creating a record that is not read from a SAM/BAM file.  By using these set methods to setup the record, they can be pulled back out using the get accessors or the record can be later written as either a SAM/BAM record. 
 
The methods found in the SamRecord class for setting fields are:
 
{| class="wikitable" style="width:100%" border="1"
 
|+ style="font-size:150%"|'''SamRecord Class Methods'''
 
!  width=""|Method Name
 
!  width=""|Description
 
|-
 
| bool setReadName(const char* readName)
 
| Sets QNAME to the passed in name.
 
Returns true if successfully set, false if not.
 
|-
 
| bool setFlag(int flag)
 
| Sets the bitwise FLAG to the passed in value.
 
Returns true if successfully set, false if not.
 
|-
 
| bool setReferenceID(int referenceID)
 
| Sets the reference sequence id.  The reference name is not currently stored.  A map to the header needs to be done to get this (which is done when writing a SAM file).  THIS is an opportunity for improvement.
 
Returns true if successfully set, false if not.
 
|-
 
| bool set1BasedPosition(int position)
 
| Sets the leftmost position.  The value passed in is 1-based (SAM formatted).  Internal processing handles switching between SAM/BAM formats when read/written.
 
Returns true if successfully set, false if not.
 
|-
 
| bool set0BasedPosition(int position)
 
| Sets the leftmost position.  The value passed in is 0-based (BAM formatted).  Internal processing handles switching between SAM/BAM formats when read/written.
 
Returns true if successfully set, false if not.
 
|-
 
|bool setMapQuality(int mapQuality)
 
| Sets the mapping quality.
 
Returns true if successfully set, false if not.
 
|-
 
| bool setCigar(const char* cigar)
 
| Sets the cigar string to the passed in CIGAR.  This is a SAM formatted CIGAR string.  Internal processing handles switching between SAM/BAM formats when read/written.
 
Returns true if successfully set, false if not.
 
|-
 
| bool setMateReferenceID(int mateReferenceID)
 
| Sets the mate reference sequence id.  The mate reference name is not currently stored.  A map to the header needs to be done to get this (which is done when writing a SAM file).  THIS is an opportunity for improvement.
 
Returns true if successfully set, false if not.
 
|-
 
| bool set1BasedMatePosition(int matePosition)
 
| Sets the leftmost mate position.  The value passed in is 1-based (SAM formatted).  Internal processing handles switching between SAM/BAM formats when read/written.
 
Returns true if successfully set, false if not.
 
|-
 
| bool set0BasedMatePosition(int matePosition)
 
| Sets the leftmost mate position.  The value passed in is 0-based (BAM formatted).  Internal processing handles switching between SAM/BAM formats when read/written.
 
Returns true if successfully set, false if not.
 
|-
 
| bool setInsertSize(int insertSize)
 
| Sets the inferred insert size.
 
Returns true if successfully set, false if not.
 
|-
 
| bool setSequence(const char* seq)
 
| Sets the sequence string to the passed in string.  This is a SAM formatted sequence string.  Internal processing handles switching between SAM/BAM formats when read/written.
 
Returns true if successfully set, false if not.
 
|-
 
| bool setQuality(const char* quality)
 
| Sets the quality string to the passed in string.  This is a SAM formatted quality string.  Internal processing handles switching between SAM/BAM formats when read/written.
 
Returns true if successfully set, false if not.
 
|-
 
|  bool addTag(const char* tag, char vtype, const char* value)
 
| Adds a tag to the record with the specified tag, vtype, and value.  Vtype can be SAM/BAM vtype.  Internal processing handles switching between SAM/BAM vtypes when read/written.
 
Returns true if successfully set, false if not.
 
|}
 
  
When set, SAM fields are validated against: [[SAM Validation Criteria]]
+
== Read & Write BAM/SAM Library Software ==
  
 +
The software reads the beginning of files opened for reading to determine if it is SAM/BAM.  To determine the format (SAM/BAM) of files open for writing, the software checks the output file's extension.  If the extension is "bam" it writes a BAM file, otherwise it writes a SAM file.
  
== Retrieving fields from a SAM/BAM Record ==
+
The library is found in statgen/lib/bam.
The SamRecord class contains accessors to access the fields of a SAM/BAM record.  They assume that the class has already been populated, either by using the set commands or by calling SamFile::ReadRecord.  Not all of the values that can be retrieved using these get accessors have set methods.  That is because they are internally calculated values if they were not read from a file.
 
  
The methods found in the SamRecord class for setting fields are:
+
=== BAM/SAM Classes ===
{| class="wikitable" style="width:100%" border="1"
+
{| style="margin: 1em 1em 1em 0; background-color: #f9f9f9; border: 1px #aaa solid; border-collapse: collapse;" border="1"
|+ style="font-size:150%"|'''SamRecord Class Get Methods'''
+
|-style="background: #f2f2f2; text-align: center;"
! width=""|Method Name
+
! Class Name !!  Description
width=""|Description
 
|-
 
| int getBlockSize()
 
| Returns the BAM block size of the record.
 
|-
 
| int getReferenceID()
 
| Returns the reference sequence id (BAM format).
 
|-
 
| int get1BasedPosition()
 
| Returns the 1-based (SAM formatted) leftmost position.
 
|-
 
| int get0BasedPosition()
 
| Returns the 0-based (BAM formatted) leftmost position.
 
|-
 
| int getReadNameLength()
 
| Returns the length of the ReadName (QNAME).
 
|-
 
| int getMapQuality()
 
| Returns the map quality.
 
|-
 
| int getBin()
 
| Returns the BAM bin for the record.
 
|-
 
| int getCigarLength()
 
| Returns the length of the CIGAR in BAM format.
 
 
|-
 
|-
| int getFlag()
+
| <code>[[C++ Class: SamFile|SamFile]]</code>
| Returns the flag.
+
| Class used for reading/writing SAM/BAM files and their headers and records.
 
|-
 
|-
| int getReadLength()
+
| <code>[[C++ Class: SamFileHeader|SamFileHeader]]</code>
| Returns the length of the read.
+
| Class used for storing the header.  Allows access for getting and setting header values when both reading & writing SAM/BAM files.
 
|-
 
|-
| int getMateReferenceID()
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamHeaderRecord.html SamHeaderRecord]</code>
| Returns the mate reference sequence id (BAM format).
+
| Class used for storing the tag/value pairs within a given header line.
 
|-
 
|-
| int get1BasedMatePosition()
+
| <code>[[C++ Class: SamRecord|SamRecord]]</code>
| Returns the 1-based (SAM formatted) mate leftmost position.
+
| Class used for storing a SAM/BAM Record.  Allows access for getting and setting record values when both reading & writing SAM/BAM files.
 
|-
 
|-
| int get0BasedMatePosition()
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamStatus.html SamStatus]</code>
| Returns the 0-based (BAM formatted) mate leftmost position.
+
| Status value used by the BAM classes for returning the status of the operations.
 
|-
 
|-
| int getInsertSize()
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamValidator.html SamValidator]</code>
| Returns the insert size.
+
| Validates a SAM/BAM Record.
 
|-
 
|-
| const char* getReadName()
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamValidationError.html SamValidationError]</code>
| Returns the SAM formatted Read Name (QNAME).
+
| Validation Error Information for a SamRecord.
 
|-
 
|-
| const char* getCigar()
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamValidationErrors.html SamValidationErrors]</code>
| Returns the SAM formatted CIGAR string.
+
| Container for ValidationErrors.
 
|-
 
|-
| const char* getSequence()
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classPileup.html Pileup]</code>
| Returns the SAM formatted Sequence string.
+
| Template for doing pileups.
 
|-
 
|-
| const char* getQuality()
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classPileupElement.html PileupElement]</code>
| Returns the SAM formatted Quality string.
+
| Base class that can be used for the elements stored in a Pileup.
 
|-
 
|-
| bool getNextSamTag(char* tag, char& vtype, void** value)
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classErrorHandler.html ErrorHandler]</code>
| Returns true if a tag was read, false if there are no more tags.
+
| Class for handling errors based on the error handling type.
For a true return value, tag is sent to the tag of the tag, vtype is set to the vtype of the tag, and value is a pointer to the value of the tag.  You will then need to use a switch to cast value to int, double, char, or String.
 
 
|-
 
|-
| bool isIntegerType(char vtype)
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classPosList.html PosList]</code>
| Returns true if the passed in vtype is of integer ('c', 'C', 's', 'S', 'i', 'I') type.
+
| Store refID/position, but does not store values < 0.
 
|-
 
|-
| bool isDoubleType(char vtype)
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamFilter.html SamFilter]</code>
| Returns true if the passed in vtype is of double ('f') type.
+
| Class for filtering a SAM/BAM record.
 
|-
 
|-
| bool isCharType(char vtype)
+
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamFlag.html SamFlag]</code>
| Returns true if the passed in vtype is of char ('A') type.
+
| Class for getting information from a SAM/BAM flag.
|-
 
| bool isStringType(char vtype)
 
| Returns true if the passed in vtype is of String ('Z') type.
 
 
|-
 
|-
 +
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamReferenceInfo.html SamReferenceInfo]</code>
 +
| Class for tracking the reference information mapping between the reference ids and the reference names.
 
|-
 
|-
 +
| <code>[http://csg.sph.umich.edu//mktrost/doxygen/current/classSamTags.html SamTags]</code>
 +
| Class for parsing/creating/operating on SAM/BAM record tags.
 
|}
 
|}
  
 +
== FAQs ==
 +
[[SAM/BAM Classes FAQs]]
  
Example of using getNextSamTag:
+
== Usage Examples ==
<pre>
+
[[Sam Library Usage Examples]]
  // record is a previously setup SamRecord.
 
  String recordString = "";
 
  char tag[3];
 
  char vtype;
 
  void* value;
 
 
 
  // While there are more tags, write them to the recordString.
 
  while(record.getNextSamTag(tag, vtype, &value) != false)
 
  {
 
      recordString += "\t";
 
      recordString += tag;
 
      recordString += ":";
 
      recordString += vtype;
 
      recordString += ":";
 
      if(record.isIntegerType(vtype))
 
      {
 
        recordString += (int)*(int*)value;
 
      }
 
      else if(record.isDoubleType(vtype))
 
      {
 
        recordString += (double)*(double*)value;
 
      }
 
      else if(record.isCharType(vtype))
 
      {
 
        recordString += (char)*(char*)value;
 
      }
 
      else
 
      {
 
        // String type.
 
        recordString += (String)*(String*)value;
 
      }
 
  }
 
  
  recordString += "\n";
 
</pre>
 
  
 +
== Programs ==
  
== Suggested Improvements/Features ==
+
BamUtil contains a set of programs that uses this library to operate on SAM & BAM files.  It includes tools for converting between SAM & BAM and validating the files.  See [[BamUtil]] for more information and a description of all the tools.
* Add optional user flag for checking if a file is sorted when it is read.  It would report an error if it is unsorted.
 

Latest revision as of 10:55, 2 February 2017


SAM/BAM File

See the github history: https://github.com/statgen/libStatGen/commits/master/bam for a list of the most recent updates to the BAM classes.

Old BAM Review Action Items

Read & Write BAM/SAM Library Software

The software reads the beginning of files opened for reading to determine if it is SAM/BAM. To determine the format (SAM/BAM) of files open for writing, the software checks the output file's extension. If the extension is "bam" it writes a BAM file, otherwise it writes a SAM file.

The library is found in statgen/lib/bam.

BAM/SAM Classes

Class Name Description
SamFile Class used for reading/writing SAM/BAM files and their headers and records.
SamFileHeader Class used for storing the header. Allows access for getting and setting header values when both reading & writing SAM/BAM files.
SamHeaderRecord Class used for storing the tag/value pairs within a given header line.
SamRecord Class used for storing a SAM/BAM Record. Allows access for getting and setting record values when both reading & writing SAM/BAM files.
SamStatus Status value used by the BAM classes for returning the status of the operations.
SamValidator Validates a SAM/BAM Record.
SamValidationError Validation Error Information for a SamRecord.
SamValidationErrors Container for ValidationErrors.
Pileup Template for doing pileups.
PileupElement Base class that can be used for the elements stored in a Pileup.
ErrorHandler Class for handling errors based on the error handling type.
PosList Store refID/position, but does not store values < 0.
SamFilter Class for filtering a SAM/BAM record.
SamFlag Class for getting information from a SAM/BAM flag.
SamReferenceInfo Class for tracking the reference information mapping between the reference ids and the reference names.
SamTags Class for parsing/creating/operating on SAM/BAM record tags.

FAQs

SAM/BAM Classes FAQs

Usage Examples

Sam Library Usage Examples


Programs

BamUtil contains a set of programs that uses this library to operate on SAM & BAM files. It includes tools for converting between SAM & BAM and validating the files. See BamUtil for more information and a description of all the tools.