Difference between revisions of "C++ Library: libStatGen"
(19 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
− | |||
[[Category:C++]] | [[Category:C++]] | ||
[[Category:libStatGen]] | [[Category:libStatGen]] | ||
Line 23: | Line 22: | ||
= Copyrights = | = Copyrights = | ||
− | |||
− | |||
Here are links to the copyrights for our code and some of the utilities it uses: | Here are links to the copyrights for our code and some of the utilities it uses: | ||
*[https://github.com/statgen/libStatGen/blob/master/general/COPYING GNU GENERAL PUBLIC LICENSE] and [https://github.com/statgen/libStatGen/blob/master/general/LICENSE.txt Our Copyright Note] | *[https://github.com/statgen/libStatGen/blob/master/general/COPYING GNU GENERAL PUBLIC LICENSE] and [https://github.com/statgen/libStatGen/blob/master/general/LICENSE.txt Our Copyright Note] | ||
Line 31: | Line 28: | ||
Copies of these can be found in our library under libStatGen/copyrights/. | Copies of these can be found in our library under libStatGen/copyrights/. | ||
− | = | + | = Join in libStatGen mailing list = |
− | |||
− | libStatGen | + | Please join in the [http://groups.google.com/group/libStatGen libStatGen Google Group] to ask / discuss / comment about this library. |
− | + | = Troubleshooting = | |
+ | If you are having trouble compiling any of the versions, check [[libStatGen Troubleshooting]] for help. If that does not solve your problem, email me for support. | ||
− | |||
− | = | + | = Where to Find It = |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | = | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
+ | {{ToolGitRepo|repoName=libStatGen|libStatGen=true|libBaseName=libStatGen}} | ||
− | == | + | == Releases == |
Released Versions are documented at [[libStatGen Download]] | Released Versions are documented at [[libStatGen Download]] | ||
− | |||
= What has changed = | = What has changed = | ||
Line 84: | Line 54: | ||
= Library Documentation = | = Library Documentation = | ||
Latest Doxygen documentation: | Latest Doxygen documentation: | ||
− | [http://www.sph.umich.edu/csg/mktrost/doxygen/ | + | <!-- <a href="http://csg.sph.umich.edu//abecasis/GOLD/ --> |
+ | <!-- [http://www.sph.umich.edu/csg/mktrost/doxygen/current/ Current Library Documentation in Doxygen] --> | ||
+ | [http://csg.sph.umich.edu//mktrost/doxygen/current/ Current Library Documentation in Doxygen] | ||
− | Additional documentation: | + | Additional documentation: |
* [[libStatGen: general]] - General classes for file processing and performing common tasks (used by most other libraries). | * [[libStatGen: general]] - General classes for file processing and performing common tasks (used by most other libraries). | ||
* [[libStatGen: BAM]] - Classes specific for reading/writing/analyzing SAM/BAM files. | * [[libStatGen: BAM]] - Classes specific for reading/writing/analyzing SAM/BAM files. | ||
* [[libStatGen: GLF]] - Classes specific for reading/writing/analyzing GLF files. | * [[libStatGen: GLF]] - Classes specific for reading/writing/analyzing GLF files. | ||
* [[libStatGen: FASTQ]] - Classes specific for reading/writing/analyzing FastQ files. | * [[libStatGen: FASTQ]] - Classes specific for reading/writing/analyzing FastQ files. | ||
+ | * [[libStatGen: ASP]] - Classes specific for reading/writing/analyzing ASP files. | ||
+ | * [[libStatGen: VCF]] - Classes specific for reading/writing/analyzing VCF files. | ||
= Using the Library = | = Using the Library = | ||
Line 97: | Line 71: | ||
** g++ | ** g++ | ||
** development version of zlib (zlib1g-dev on ubuntu) | ** development version of zlib (zlib1g-dev on ubuntu) | ||
− | |||
* Compiles on Linux/Unix | * Compiles on Linux/Unix | ||
Line 148: | Line 121: | ||
== Using the Library in Your Own Program == | == Using the Library in Your Own Program == | ||
− | === Starting from a Sample Program === | + | === Starting from a Sample Program (Recommended) === |
[https://github.com/statgen/SampleProgram https://github.com/statgen/SampleProgram] is a simple program demonstrating how to write a tool that uses libStatGen and can be used as a starting point for your tool. | [https://github.com/statgen/SampleProgram https://github.com/statgen/SampleProgram] is a simple program demonstrating how to write a tool that uses libStatGen and can be used as a starting point for your tool. | ||
Line 175: | Line 148: | ||
# Add your tests to the <code>test</code> directory. | # Add your tests to the <code>test</code> directory. | ||
# Update test/Makefile as appropriate for specifying how to compile/run your tests. | # Update test/Makefile as appropriate for specifying how to compile/run your tests. | ||
+ | |||
+ | |||
+ | After compiling a <code>bin</code> directory is created in the top level directory. Your executable goes in there. If you build for <code>debug</code> and/or <code>profile</code>, subdirectories for those are created under <code>bin/</code> and <code>obj</code>. | ||
Line 181: | Line 157: | ||
− | = | + | === Starting from a Sample Set of Tools === |
− | If you are | + | [https://github.com/statgen/SampleTools https://github.com/statgen/SampleTools] is a repository containing multiple programs within one directory structure. It demonstrates how to have subdirectories for each tool using libStatGen and can be used as a starting point for your set of tools. |
+ | |||
+ | SampleTools has 3 subdirectories: | ||
+ | * copyrights - contains the copyright information, add your own copyrights as necessary | ||
+ | * SampleProgram1 - a dummy demo program to show the structure for having multiple programs | ||
+ | * SampleProgram2 - a second dummy demo program to show the structure for having multiple programs | ||
+ | |||
+ | SampleProgram1 & SampleProgram2 have 2 subdirectories: | ||
+ | * src - this is where your own program code goes | ||
+ | * test - this is where your test code goes. Test code can be setup to run with <code>make test</code> to ensure the program works properly. | ||
+ | |||
+ | Upon compiling, an <code>obj</code> directory is created under <code>SampleProgram1</code> and <code>SampleProgram2</code> and a <code>bin</code> directory is created at the top level. If you build for <code>debug</code> and/or <code>profile</code>, subdirectories for those are created under <code>bin/</code> and <code>SampleProgram1(2)/obj</code>. | ||
+ | |||
+ | |||
+ | '''Using SampleTools as a starting point for your set of tools:''' | ||
+ | # Copy <code>SampleTools</code> into a directory with your toolset name (it is the starting point for your own set of tools). | ||
+ | # Update <code>ChangeLog</code>, <code>.gitignore</code>, and <code>README.txt</code> as appropriate. | ||
+ | # Add any necessary copyrights to the copyrights directory. | ||
+ | # Rename the <code>SampleProgram1</code> and <code>SampleProgram2</code> directories | ||
+ | # Create any additional directories as necessary. | ||
+ | #* Recursively copy the structure/Makefiles from <code>SampleProgram1</code>. | ||
+ | # Update <code>SUBDIRS</code> in <code>Makefile</code> as necessary. | ||
+ | # Update <code>Makefile.inc</code> | ||
+ | ## Update the <code>VERSION</code> as necessary. | ||
+ | ## Replace all occurrences of <code>SAMPLE_PROGRAM</code> with an all caps name for your toolset. | ||
+ | ##* You can then use the <code>LIB_PATH_<your toolset name></code> environment variable to specify an alternate path to libStatGen specific for your program. In most cases you will not need to do this. | ||
+ | #* No other updates to <code>Makefile.inc</code> should be necessary. | ||
+ | # For each Program you want to add: | ||
+ | ## Move into the appropriate subdirectory. | ||
+ | ##* No change should be made to the program's <code>Makefile</code> | ||
+ | ## Add your program (cpp & h files) to the <code>src</code> subdirectory. | ||
+ | ## Update src/Makefile | ||
+ | ### Set EXE to your program executable (replacing sampleProgram) | ||
+ | ### Set TOOLBASE, SRCONLY, and HDRONLY as appropriate for specifying your program file names. | ||
+ | ### Set any of the other optional settings as specified in the sample makefile. | ||
+ | ##* No other changes should be necessary to src/Makefile. | ||
+ | ## Add your tests to the <code>test</code> directory. | ||
+ | ## Update test/Makefile as appropriate for specifying how to compile/run your tests. | ||
+ | |||
+ | |||
+ | = How To Use the APIs = | ||
+ | More coming soon, see: http://genome.sph.umich.edu/wiki/Sam_Library_Usage_Examples | ||
+ | |||
+ | [[LibStatGen: ASP#API for Reading ASP Files| ASP APIs]] | ||
+ | |||
+ | [[LibStatGen: VCF#API for Reading VCF Files| VCF APIs]] |
Latest revision as of 17:21, 11 September 2021
Description
Open source, freely available (GPL license), easy to use C++ APIs
- General Operation Classes including:
- File/Stream I/O – uncompressed, BGZF, GZIP, stdin, stdout
- String processing
- Parameter Parsing
- Statistical Genetic Specific Classes including:
- Handling Common file formats – SAM/BAM, FASTQ, GLF, VCF (coming soon)
- Accessors to get/set values
- Indexed access to BAM files
- Utility classes, including:
- Cigar – interpretation and mapping between query and reference
- Pileup – structured access to data by individual reference position
- Handling Common file formats – SAM/BAM, FASTQ, GLF, VCF (coming soon)
Can be used to create your own C++ programs.
Currently the repository is recommended for Unix/Linux users with access to the GNU C++ compiler.
Copyrights
Here are links to the copyrights for our code and some of the utilities it uses:
- GNU GENERAL PUBLIC LICENSE and Our Copyright Note
- Copyright for MERSENNE TWISTER (used in Random.cpp)
- Samtools Copyright (MIT License)
Copies of these can be found in our library under libStatGen/copyrights/.
Join in libStatGen mailing list
Please join in the libStatGen Google Group to ask / discuss / comment about this library.
Troubleshooting
If you are having trouble compiling any of the versions, check libStatGen Troubleshooting for help. If that does not solve your problem, email me for support.
Where to Find It
The libStatGen repository is available both via release downloads and via github.
On github, https://github.com/statgen/libStatGen, you can both browse and download the libStatGen source code as well as explore the history of changes.
You can obtain the source either with or without git.
A copy of libStatGen is included in certain releases of some statgen tools.
Using Git To Track the Current Development Version
Clone (get your own copy)
You can create your own git clone (copy) using:
git clone https://github.com/statgen/libStatGen.git
or
git clone git://github.com/statgen/libStatGen.git
Either of these commands create a directory called libStatGen
in the current directory.
Then just cd libStatGen
and compile.
Get the latest Updates (update your copy)
To update your copy to the latest version (a major advantage of using git):
cd pathToYourCopy/libStatGen
make clean
git pull
make all
Git Refresher
If you decide to use git, but need a refresher, see How To Use Git or Notes on how to use git (if you have access)
Downloading From GitHub Without Git
If you download the latest code/version, make sure you periodically update it by downloading a newer version.
From github you can download:
- Latest Code (master branch)
- via Website
- Goto: https://github.com/statgen/libStatGen
- Click on the
Download ZIP
button on the right side panel.
- via Command Line
- via Website
- Specific Release (via a tag)
- via Website
- Goto: https://github.com/statgen/libStatGen/releases to see the available releases
- Click
zip
ortar.gz
for the desired version.
- via Command Line
wget https://github.com/statgen/libStatGen/archive/<tagName>.tar.gz
- or
wget https://github.com/statgen/libStatGen/archive/<tagName>.zip
- via Website
After downloading the file, uncompress (unzip/untar) it. The directory created will be named libStatGen-<name of version you downloaded>
.
Building
After obtaining the libStatGen repository (either by download or from github), compile the code using:
make all
Object (.o) files are compiled into the obj
directory with a subdirectory debug
and profile
for the debugging and profiling objects.
This creates the libraries, libStatGen.a, libStatGen_debug.a, libStatGen_profile.a at the top level directory.
make test
compiles for opt, debug, and profile and runs the tests (found in the test
subdirectory).
To see all make options, type make help
.
If compilation fails due to warnings being treated as errors, please contact us so we can fix the warnings. As a work-around to get it to compile, you can disable the treatment of warnings as errors by editing libStatGen/general/Makefile to remove -Werror
.
Releases
Released Versions are documented at libStatGen Download
What has changed
The pipeline
and statgen
repositories have been deprecated, so please update to our new framework.
libStatGen
is the new git repository for our library code.
There are now separate repositories for specific tools/groups of tools, allowing us to track everything separately so it is easier to follow changes that impact a specific tool or the library in general.
Library Documentation
Latest Doxygen documentation: Current Library Documentation in Doxygen
Additional documentation:
- libStatGen: general - General classes for file processing and performing common tasks (used by most other libraries).
- libStatGen: BAM - Classes specific for reading/writing/analyzing SAM/BAM files.
- libStatGen: GLF - Classes specific for reading/writing/analyzing GLF files.
- libStatGen: FASTQ - Classes specific for reading/writing/analyzing FastQ files.
- libStatGen: ASP - Classes specific for reading/writing/analyzing ASP files.
- libStatGen: VCF - Classes specific for reading/writing/analyzing VCF files.
Using the Library
Dependencies
- This software requires the following to be installed:
- g++
- development version of zlib (zlib1g-dev on ubuntu)
- Compiles on Linux/Unix
Building the Library
If you type make help, you get the build options.
Makefile help ------------- Type... To... make Compile opt make help Display this help screen make all Compile everything (opt, debug, & profile) make opt Compile optimized make debug Compile for debug make profile Compile for profile make clean Delete temporary files make test Execute tests (if there are any)
When you just type make, it will by default to make opt (optimized).
Make all indicates opt, debug, and profile.
opt creates libStatGen.a
, debug creates libStatGen_debug.a
, profile creates libStatGen_profile.a
These libraries are created in the top level libStatGen directory and can then be linked to appropriately for building tools as optimized, debugging, and/or profiling.
Under the main libStatGen repository, there are:
- bam - library code for operating on bam files.
- copyrights - copyrights for the library and any code included with it.
- fastq - library code for operating on fastq files.
- general - library code for general operations
- glf - library code for operating on glf files.
- include - after compiling, the library headers are linked here
- Makefiles - directory containing Makefiles that are used in the library and can be used for developing programs using the library
- samtools - library code used from samtools
After Compiling: libStatGen.a, libStatGen_debug.a, libStatGen_profile.a are created at the top level.
bam, fastq, general, glf, samtools
Object files are placed in an obj directory under each subdirectory with debug & profile objects in obj/debug and obj/profile.
Most also have a test directory. Tests are executed by running make test
Makefiles
This directory contains base makefiles and makefile settings that are used by the library and by programs being written to use the library.
Using the Library in Your Own Program
Starting from a Sample Program (Recommended)
https://github.com/statgen/SampleProgram is a simple program demonstrating how to write a tool that uses libStatGen and can be used as a starting point for your tool.
SampleProgram has 4 subdirectories:
- copyrights - contains the copyright information, add your own copyrights as necessary
- obj - this directory is where the object files are placed when the code is compiled (with a subdirectory for debug and profile objects)
- src - this is where your own program code goes
- test - this is where your test code goes. Test code can be setup to run with
make test
to ensure the program works properly.
Using SampleProgram as a starting point for your tool:
- Copy SampleProgram into a directory with your program name (it is the starting point for your own program).
- Update ChangeLog, .gitignore, and README.txt as appropriate.
- Add any necessary copyrights to the copyrights directory.
- No changes to Makefile should be necessary.
- Update Makefile.inc
- Update the VERSION as necessary.
- Replace all occurrences of
SAMPLE_PROGRAM
with an all caps name for your program.- You can then use the
LIB_PATH_<your program name>
environment variable to specify an alternate path to libStatGen specific for your program. In most cases you will not need to do this.
- You can then use the
- No other updates to Makefile.inc should be necessary.
- Add your program (cpp & h files) to the
src
directory. - Update src/Makefile
- Set EXE to your program executable (replacing sampleProgram)
- Set TOOLBASE, SRCONLY, and HDRONLY as appropriate for specifying your program file names.
- Set any of the other optional settings as specified in the sample makefile.
- No other changes should be necessary to src/Makefile.
- Add your tests to the
test
directory. - Update test/Makefile as appropriate for specifying how to compile/run your tests.
After compiling a bin
directory is created in the top level directory. Your executable goes in there. If you build for debug
and/or profile
, subdirectories for those are created under bin/
and obj
.
Working from Scratch
When compiling your code, be sure to include the library header files found in libStatgen/include/ and link in the appropriate library (opt: libStatGen.a, debug: libStatGen_debug.a, or profile: libStatGen_profile.a).
Starting from a Sample Set of Tools
https://github.com/statgen/SampleTools is a repository containing multiple programs within one directory structure. It demonstrates how to have subdirectories for each tool using libStatGen and can be used as a starting point for your set of tools.
SampleTools has 3 subdirectories:
- copyrights - contains the copyright information, add your own copyrights as necessary
- SampleProgram1 - a dummy demo program to show the structure for having multiple programs
- SampleProgram2 - a second dummy demo program to show the structure for having multiple programs
SampleProgram1 & SampleProgram2 have 2 subdirectories:
- src - this is where your own program code goes
- test - this is where your test code goes. Test code can be setup to run with
make test
to ensure the program works properly.
Upon compiling, an obj
directory is created under SampleProgram1
and SampleProgram2
and a bin
directory is created at the top level. If you build for debug
and/or profile
, subdirectories for those are created under bin/
and SampleProgram1(2)/obj
.
Using SampleTools as a starting point for your set of tools:
- Copy
SampleTools
into a directory with your toolset name (it is the starting point for your own set of tools). - Update
ChangeLog
,.gitignore
, andREADME.txt
as appropriate. - Add any necessary copyrights to the copyrights directory.
- Rename the
SampleProgram1
andSampleProgram2
directories - Create any additional directories as necessary.
- Recursively copy the structure/Makefiles from
SampleProgram1
.
- Recursively copy the structure/Makefiles from
- Update
SUBDIRS
inMakefile
as necessary. - Update
Makefile.inc
- Update the
VERSION
as necessary. - Replace all occurrences of
SAMPLE_PROGRAM
with an all caps name for your toolset.- You can then use the
LIB_PATH_<your toolset name>
environment variable to specify an alternate path to libStatGen specific for your program. In most cases you will not need to do this.
- You can then use the
- No other updates to
Makefile.inc
should be necessary.
- Update the
- For each Program you want to add:
- Move into the appropriate subdirectory.
- No change should be made to the program's
Makefile
- No change should be made to the program's
- Add your program (cpp & h files) to the
src
subdirectory. - Update src/Makefile
- Set EXE to your program executable (replacing sampleProgram)
- Set TOOLBASE, SRCONLY, and HDRONLY as appropriate for specifying your program file names.
- Set any of the other optional settings as specified in the sample makefile.
- No other changes should be necessary to src/Makefile.
- Add your tests to the
test
directory. - Update test/Makefile as appropriate for specifying how to compile/run your tests.
- Move into the appropriate subdirectory.
How To Use the APIs
More coming soon, see: http://genome.sph.umich.edu/wiki/Sam_Library_Usage_Examples