LibStatGen: ASP

From Genome Analysis Wiki
Revision as of 12:13, 27 January 2012 by Mktrost (talk | contribs)
Jump to navigationJump to search


Asymmetric Pileup (ASP)

Asymmetric Pileup (ASP) is a new pileup file format that we created to replace GLF.

It is currently in initial test phase, and is not yet available for public use.


ASP File Format

ASP files are binary files consisting of 4 types of records. Every record starts with a 1-byte field that indicates the type.

Type Value Record Type Description
0 Empty indicates there are no bases for this position.
1 Position Only specifies the chromsome id/0-based position of the next record (other records do not contain a position).
2 Reference Only indicates that all bases at this position match the reference and provides the number of bases, GLH, and GLA.
3 Detailed indicates that not all bases at this position match the reference and provides the number of bases, the bases, the qualities, the cycles, the strands, and the MQs.

Only position records contain a chromosome id/position. The record after a position record has the chromosome id & position specified in the position record. All other records are assumed to just increment one position from the previous record.

The first record in a file must be a Position Only Record.


Record Type: Empty

An empty record is only 1 byte and just contains the type field.

Since non-position only records do not have a position associated with them, the position is determined by adding one to the position of the previous record.

When positions have no bases, there are two ways to deal with them.

  1. Write a new position record for the next position that has bases
  2. Write empty records to indicate those positions have no bases.

If positions that have bases are not far apart, it is preferable to write empty records rather than a new position record since Empty records should compress well and are only 1 byte.


Field Description Type Value
type Empty Record Type uint8_t 0


Record Type: Position Only

Position Only records are used to specify the chromosome ID & 0-based position of the following record. The first record in the file must be a Position Only record.

Field Description Type Value
type Position Only Record Type uint8_t 1
chromID Chromosome ID of the next record int32_t
pos 0-based position of the next record int32_t


Record Type: Reference Only

The position associated with a Reference Only record is 1 greater than the position of the previous record unless the previous record is a position record.

Field Description Type Value/Range
type Reference Only Record Type uint8_t 2
numBases Number of bases at this position uint8_t 1-255
GLH Genotype Likelihood H uint8_t 0-255
GLA Genotype Likelihood Alternate uint8_t 0-255

If a position has more than 255 bases, only the first 255 are used for calculating the GLH, GLA.

Phred Qualities that are unknown or less than 13 are not used in the Genotype Likelihood calculation, but are counted in the numBases if there is a base at the position.


Record Type: Detailed

The position associated with a Detailed record is 1 greater than the position of the previous record unless the previous record is a position record.

When a a deletion occurs at a specified position, it counts as a mismatch to the reference.

Field Description Type Value/Range
type Detailed Record Type uint8_t 2
numBases Number of bases at this position uint8_t 1-255
all Bases 4-bit encoded bases/deletions for this position

1st base is in the upper bits of 1st byte

if odd number of bases, the lower bits of the last byte are 0.

uint8_t[(numBases+1)/2] 0=A, 1=C,

2=G, 3=T,

4=N, 5=D (deletion)

allQuals All Phred Quals for this position uint8_t[numBases] 0-254

255 - unknown quality, deletion qual

allCycles 0-based position in the reads for all bases at this position uint8_t[numBases] 0-254, 255 for a deletion
allStrands Strand for all bases at this position

strand of the 1st base is in the uppermost bit of the first byte

if numBases is not a multiple of 8, the extra lower bits are set to 0

uint8_t[(numBases+7)/8] 0 - forward

1 - reverse

allMQs Mapping Qualities for all bases at this position uint8_t[numBases] 0-255


If a position has more than 255 bases, only the first 255 are used in this record.


Example

Hex dump of a record: 0x0302231d1d0201402c22

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
03 02 23 1D 1D 02 01 40 2C 22
Type = DETAILED NumBases = 2 Base1 = 2 = G, Base2 = 3 = T Qual1 = 0x1D = 29 = '>' Qual2 = 0x1D = 29 = '>' Cycle1 = 2 Cycle2 = 1 Strand1 (bit 56) = 0 = forward,

Strand2 (bit 57) = 1 = reverse,

extra bits are dummy bits = 0

MapQual1 = 0x2C = 44 MapQual2 = 0x22 = 34
0 3 0 2 2 3 1 D 1 D 0 2 0 1 4 0 2 C 2 2
0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 1 1 1 0 1 0 0 0 1 1 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 0 0 1 0


API for Reading ASP Files

You will use both an AspFileReader and an AspRecord for reading ASP files.

AspFileReader

An instance of the AspFileReader class is used to read ASP files.

include file

AspFileReader is declared in AspFile.h, so be sure to include that file. 

#include "AspFile.h"

Opening the ASP File

open opens the specified file and throws an exception if it was not successfully open. 

   // Open the asp file for reading.
    AspFileReader asp;
    asp.open("aspFileName.asp");

Reading the ASP Record

Once the file is open, there are two methods to get the next record, getNextRecord and getNextDataRecord.

Both methods return true if a record was successfully found and false on EOF or an error.

Both methods take a reference to an AspRecord as a parameter. When true is returned, the AspRecord is updated with the next record.

The AspRecord set by getNextRecord will be any type of record, Reference Only, Detailed, Empty, or Position.

The AspRecord set by getNextDataRecord will only be a Reference Only Record or a Detailed Record. It consumes any Empty Records and Position Records it finds until a Reference Only or Detailed Record is found.

If you only process one record at a time (are done with a record before reading the next one), you can loop until getNextRecord or getNextDataRecord return false, reusing the same record for each call. 

        while(asp.getNextRecord(record))
        {
            // Your record specific processing here.
        }
        // Done reading the file.

-OR- 

        while(asp.getNextDataRecord(record))
        {
            // Your record specific processing here.
        }
        // Done reading the file.

Closing the ASP File

close closes the opened ASP file. 

    asp.close();

AspRecord

Asp records are read from Asp files using getNextRecord and getNextDataRecord.

Once you have the record, use methods from the AspRecord to extract the desired information.

Determining the type of record

To determine which type of record is in your AspRecord object, use isEmptyType(), isPosType(), isRefOnlyType(), and isDetailedType(). These methods return true if the record is of that type and false if it is a different type.

Retrieve the Chromosome/Position

Regardless of the record type, getChromID() returns the chromosome ID associated with this record.

Regardless of the record type, getPosition() returns the 0-based position associated with this record.

Reference Only Record Methods

Reference Only records contain the number of bases at this position and the GLH & GLA.

  • getNumBases() returns the number of bases at this position
  • getGLH() returns the GLH of this position
  • getGLA() returns the GLA of this position
  • getLikelihood(char base1, char base2, char refBase) returns the likelihood of this position for the specified genotype.
    • When both bases match the reference base, 0 is returned.
    • If only one of the bases matches the reference base, the GLH is returned.
    • If neither base matches the reference base, the GLA is returned.

Detailed Record Methods

Detailed records contain the number of bases at this position and all of the bases, qualities, cycles, strandss, and MQs.

  • getNumBases() returns the number of bases at this position
  • getLikelihood(char base1, char base2, char refBase) returns the likelihood of this position for the specified genotype.
  • getBaseChar(int index) returns the base as a character (A,C,T,G,D,N) in the record at the specified index. The index starts at 0 and goes to numBases - 1. An out of range index returns 'N'. 'D' represents a deletion at this position.
  • getPhredQual(int index) returns the phred quality at the specified index. The index starts at 0 and goes to numBases - 1. An out of range index returns -1. An unknown quality and the quality for a deletion is -1.
  • getCharQual(int index) returns the quality represented as a character (phred+33) at the specified index. The index starts at 0 and goes to numBases - 1. An out of range index returns ' '. An unknown quality and the quality for a deletion is ' '.
  • getCycle(int index) returns the cycle of the base at this index in its original read. The index starts at 0 and goes to numBases - 1. An out of range index returns -2. The cycle for a deletion deletion is -1.
  • getStrand(int index) returns the strand of the read at this index. The index starts at 0 and goes to numBases - 1. An out of range index returns false.
  • getMQ(int index) returns the Mapping Quality of the read at this index. The index starts at 0 and goes to numBases - 1. An out of range index returns -1.
Retrieved from "http://genome.sph.umich.edu/w/index.php?title=LibStatGen:_ASP&oldid=4368"