Difference between revisions of "Minimac3 ChangeLog"
Santy.8128 (talk | contribs) |
Santy.8128 (talk | contribs) |
||
(15 intermediate revisions by the same user not shown) | |||
Line 13: | Line 13: | ||
Minimac3 Changes: | Minimac3 Changes: | ||
-------------------------------------------------------------------------- | -------------------------------------------------------------------------- | ||
+ | |||
+ | ==Ver 2.0.1 (June 6, 2016)== | ||
+ | |||
+ | Version : 2.0.1 | ||
+ | Author : Sayantan Das | ||
+ | Date : June 6, 2016 | ||
+ | |||
+ | Major Changes: | ||
+ | 1. Reduced memory usage significantly (by 35% for HRC panel) | ||
+ | 2. Reduced run time significantly (by 20% for HRC panel) | ||
+ | 3. Added a --lowMemory version that further reduces the memory usage | ||
+ | furhter by 33% (but takes 10% more time. | ||
+ | |||
+ | Minor Changes: | ||
+ | 1. Fixed a minor bug in parameter estimation (--processReference) | ||
+ | 2. Added a draft.m3vcf file output after file compression and | ||
+ | before parameter estimation. | ||
+ | 3. Changed definition of Estimated R-square so that no estimates | ||
+ | are over 1.0. | ||
+ | 4. Added <contig> in VCF header to make bcftools readable. | ||
+ | 5. Added FILTER column for typed sites and only typed sites. | ||
+ | |||
+ | ==Ver 1.0.14 (April 15, 2016)== | ||
+ | |||
+ | Version : 1.0.13 | ||
+ | Author : Sayantan Das | ||
+ | Date : April 15, 2016 | ||
+ | |||
+ | Fixed a bug in the makefile that was missing the g++ -O4 | ||
+ | optimization flag. As a result, the openmp version was | ||
+ | 3-4 times slower than expected. Updated LibStatGen Library | ||
+ | files to remove some warnings/errors. | ||
+ | |||
+ | ==Ver 1.0.13 (October 15, 2015)== | ||
+ | |||
+ | Version : 1.0.13 | ||
+ | Author : Sayantan Das | ||
+ | Date : October 5, 2015 | ||
+ | |||
+ | Updated the format of info file to report only REF and ALT | ||
+ | alleles instead of Major and Minor allele. The ALT_Frq and | ||
+ | MAF can be compared to find minor allele. The usual .dose | ||
+ | and .hapDose file have been updated to report alternate al- | ||
+ | -llele dosage and NOT major allele dosage, like earlier ve- | ||
+ | -rsions of minimac. Also mended a bug related to variants | ||
+ | that were typed only, while using --allTypedVariants option. | ||
+ | Finally, added acknowledgements to David Hinds, for his | ||
+ | immense help in making the code faster. | ||
+ | |||
+ | ==Ver 1.0.12 (September 5, 2015)== | ||
+ | |||
+ | Version : 1.0.12 | ||
+ | Author : Sayantan Das | ||
+ | Date : September 5, 2015 | ||
+ | |||
+ | Added --allTypedVariants option which adds variants which were | ||
+ | typed ONLY to the imputed output files and imputes any missing | ||
+ | values in such variants to the MAF. Also fixed a bug with chr- | ||
+ | -omosome X only male samples. Also added --rsid option to use | ||
+ | rs IDs of variants. Updated info file RefMAF to RefAF since it | ||
+ | gives the alternate AF and NOT minor AF. | ||
+ | |||
+ | ==Ver 1.0.11 (April 21, 2015)== | ||
+ | |||
+ | Version : 1.0.11 | ||
+ | Author : Sayantan Das | ||
+ | Date : April 21, 2015 | ||
+ | |||
+ | Implemented parallel threading for processReference. Moved all | ||
+ | haplotype data from char vector to bool vector. Implemented | ||
+ | David's patch and similar other division-removal techniques. | ||
+ | Also added legacy parameter unphased for unphased output. | ||
==Ver 1.0.10 (February 27, 2015)== | ==Ver 1.0.10 (February 27, 2015)== | ||
Line 20: | Line 92: | ||
Date : February 27, 2015 | Date : February 27, 2015 | ||
− | Add parameter --log which will output a log file (BUT not print on the screen). Also added legacy parameter --myChromosome | + | Add parameter --log which will output a log file (BUT not print |
+ | on the screen). Also added legacy parameter --myChromosome whi- | ||
+ | will enable users to use special chromosomes (apart from 1-22 | ||
+ | and X) | ||
+ | |||
==Ver 1.0.9 (February 20, 2015)== | ==Ver 1.0.9 (February 20, 2015)== | ||
Line 28: | Line 104: | ||
Date : February 20, 2015 | Date : February 20, 2015 | ||
− | Removed dependcies on boost directory. Converted all double to | + | Removed dependcies on boost directory. Converted all double to fl- |
+ | oat. Had to change the main imputation formula to work well for l- | ||
+ | arge values. Also updated Rsq calulation formual to avoid underfl- | ||
+ | as a result of using float. Made minor changes to memory allocati- | ||
+ | on so as to remove all 2-d or 3-d look-ups. Updated a loop to rem- | ||
+ | ove repeat allocation, similar to minimac2. Now it is faster and | ||
+ | uses much lesser memory. | ||
==Ver 1.0.7 (January 31, 2015)== | ==Ver 1.0.7 (January 31, 2015)== | ||
Line 52: | Line 134: | ||
Date : October 3, 2014 | Date : October 3, 2014 | ||
− | Implemented chromosome X imputation (both PAR and non-PAR). Also added bgzip option to enable tabix options. | + | Implemented chromosome X imputation (both PAR and non-PAR). Also |
+ | added bgzip option to enable tabix options. | ||
==Ver 1.0.4 (September 19, 2014)== | ==Ver 1.0.4 (September 19, 2014)== | ||
Line 60: | Line 143: | ||
Date : September 19, 2014 | Date : September 19, 2014 | ||
− | Improved the chuking option for the compressed files. Updated name OPTM to M3VCF (Minimac3 VCF). Changed handle to --processReference. Added the option to add parameter estimates within M3VCF files | + | Improved the chuking option for the compressed files. Updated name |
+ | OPTM to M3VCF (Minimac3 VCF). Changed handle to --processReference. | ||
+ | Added the option to add parameter estimates within M3VCF files wh- | ||
+ | -ich can be used for later runs. Added a new handle --updateModel | ||
+ | that will allow users to update these estimates using the target | ||
+ | panel as well, if and when they think necessary. | ||
==Ver 1.0.2 (August 28, 2014)== | ==Ver 1.0.2 (August 28, 2014)== | ||
Line 68: | Line 156: | ||
Date : August 28, 2014 | Date : August 28, 2014 | ||
− | Mended a bug related to --vcfoutput option when used with --gzip option. Also added a option --format to manage handles in the FORMAT field for output VCF files. | + | Mended a bug related to --vcfoutput option when used with --gzip |
+ | option. Also added a option --format to manage handles in the | ||
+ | FORMAT field for output VCF files. | ||
==Ver 1.0.1 (August 6, 2014)== | ==Ver 1.0.1 (August 6, 2014)== | ||
Line 76: | Line 166: | ||
Date : August 6, 2014 | Date : August 6, 2014 | ||
− | Added the --window option that will allow a buffer region around chunks. Also implemented this --start, --end and --window option on OPTM files. Minimac3 will, thus, be able to handle chunks on OPTM files as well. However, it will not perform the optimal | + | Added the --window option that will allow a buffer region around |
+ | chunks. Also implemented this --start, --end and --window option | ||
+ | on OPTM files. Minimac3 will, thus, be able to handle chunks on | ||
+ | OPTM files as well. However, it will not perform the optimal all- | ||
+ | -ocation again but just extract the chunk from the given file. | ||
==Ver 1.0.0 (July 21, 2014)== | ==Ver 1.0.0 (July 21, 2014)== | ||
Line 84: | Line 178: | ||
Date : July 21, 2014 | Date : July 21, 2014 | ||
− | Added the --vcfOutput option that will output the dosage in VCF format as well. Improved it by allowing to flush out VCF | + | Added the --vcfOutput option that will output the dosage in VCF |
+ | format as well. Improved it by allowing to flush out VCF files | ||
+ | after every 200 samples to save memory. Also implemented openmp | ||
+ | to allow parallel computing. | ||
==Ver 0.7.1 (June 8, 2014)== | ==Ver 0.7.1 (June 8, 2014)== | ||
Line 92: | Line 189: | ||
Date : June 8, 2014 | Date : June 8, 2014 | ||
− | Added the --golden option that can calculate the actual R-square if the true/hard genotypes are provided in a vcf file. Made minor editions for debugging. | + | Added the --golden option that can calculate the actual R-square |
+ | if the true/hard genotypes are provided in a vcf file. Made minor | ||
+ | editions for debugging. | ||
==Ver 0.6.2 (June 8, 2014)== | ==Ver 0.6.2 (June 8, 2014)== | ||
Line 100: | Line 199: | ||
Date : June 8, 2014 | Date : June 8, 2014 | ||
− | Edited bug in VCF packing part that was giving slightly less | + | Edited bug in VCF packing part that was giving slightly less opt- |
+ | mized configurations. | ||
==Ver 0.6.1 (June 7, 2014)== | ==Ver 0.6.1 (June 7, 2014)== | ||
Line 108: | Line 208: | ||
Date : June 7, 2014 | Date : June 7, 2014 | ||
− | Worked on an important bug that gave memory overflow in large | + | Worked on an important bug that gave memory overflow in large sa- |
+ | mples. Name changed to Minimac3. Added utility for VCF output fo- | ||
+ | rmat that would enable users to easy data manipulation. | ||
==Ver 0.5.1 (April 9, 2014)== | ==Ver 0.5.1 (April 9, 2014)== | ||
Line 116: | Line 218: | ||
Date : April 9, 2014 | Date : April 9, 2014 | ||
− | Removed futher bugs. Modified to use lesser memory and run faster. The Unique.cpp file was updated according to Goncalo's packVcf | + | Removed futher bugs. Modified to use lesser memory and run faster. |
+ | The Unique.cpp file was updated according to Goncalo's packVcf so- | ||
+ | ftware. Target Data is being read more effeciently. Scaffolding | ||
+ | done locally during imputation. Changed name to Minimac2. | ||
==Ver 0.4.1 (March 14, 2014)== | ==Ver 0.4.1 (March 14, 2014)== | ||
Line 125: | Line 230: | ||
Date : March 14 2014 | Date : March 14 2014 | ||
− | Removed bug with --sample. Added lots of other options like --chr,--start, --end, --rounds, --states etc. Added data structure for variant type. Added structure so as NOT to use haplotype data after reading it once. Now the code uses the reduced haplotype structure once it has read the data. This was uses lesser memory. | + | Removed bug with --sample. Added lots of other options like --chr, |
+ | --start, --end, --rounds, --states etc. Added data structure for | ||
+ | variant type. Added structure so as NOT to use haplotype data after | ||
+ | reading it once. Now the code uses the reduced haplotype structure | ||
+ | once it has read the data. This was uses lesser memory. | ||
==Ver 0.2.1 (January 29, 2014)== | ==Ver 0.2.1 (January 29, 2014)== | ||
Line 133: | Line 242: | ||
Date : January 29 2014 | Date : January 29 2014 | ||
− | Improved on the speed in parameter estimation part by NOT | + | Improved on the speed in parameter estimation part by NOT calcula- |
+ | ting the optimal allocation everytime, instead update it from the | ||
+ | original allocation. Updated to use --refSnps for reference, use | ||
+ | --vcfTarget for target files. | ||
==Ver 0.1.1 (May 25, 2014)== | ==Ver 0.1.1 (May 25, 2014)== | ||
Line 141: | Line 253: | ||
Date : May 25 2013 | Date : May 25 2013 | ||
− | Started working on the basis of code of minimac. Implemented State Space Reduction Method for HMM calculations. Implemented the | + | Started working on the basis of code of minimac. Implemented State |
+ | Space Reduction Method for HMM calculations. Implemented the Para- | ||
+ | meter Estimation part and Optimal Allocation part. | ||
-------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
Latest revision as of 22:24, 6 June 2016
Introduction
Minimac3 is a lower memory and more computationally efficient implementation of minimac2. It is an algorithm for genotypic imputation that works on phased genotypes (say from MaCH) and is designed to handle very large reference panels in a more computationally efficient way with no loss of accuracy.
This wiki page gives users a summary of changes over the versions.
Change Log Summary
This section gives details of the changes over the different versions.
Minimac3 Changes:
Ver 2.0.1 (June 6, 2016)
Version : 2.0.1 Author : Sayantan Das Date : June 6, 2016
Major Changes: 1. Reduced memory usage significantly (by 35% for HRC panel) 2. Reduced run time significantly (by 20% for HRC panel) 3. Added a --lowMemory version that further reduces the memory usage furhter by 33% (but takes 10% more time. Minor Changes: 1. Fixed a minor bug in parameter estimation (--processReference) 2. Added a draft.m3vcf file output after file compression and before parameter estimation. 3. Changed definition of Estimated R-square so that no estimates are over 1.0. 4. Added <contig> in VCF header to make bcftools readable. 5. Added FILTER column for typed sites and only typed sites.
Ver 1.0.14 (April 15, 2016)
Version : 1.0.13 Author : Sayantan Das Date : April 15, 2016
Fixed a bug in the makefile that was missing the g++ -O4 optimization flag. As a result, the openmp version was 3-4 times slower than expected. Updated LibStatGen Library files to remove some warnings/errors.
Ver 1.0.13 (October 15, 2015)
Version : 1.0.13 Author : Sayantan Das Date : October 5, 2015
Updated the format of info file to report only REF and ALT alleles instead of Major and Minor allele. The ALT_Frq and MAF can be compared to find minor allele. The usual .dose and .hapDose file have been updated to report alternate al- -llele dosage and NOT major allele dosage, like earlier ve- -rsions of minimac. Also mended a bug related to variants that were typed only, while using --allTypedVariants option. Finally, added acknowledgements to David Hinds, for his immense help in making the code faster.
Ver 1.0.12 (September 5, 2015)
Version : 1.0.12 Author : Sayantan Das Date : September 5, 2015
Added --allTypedVariants option which adds variants which were typed ONLY to the imputed output files and imputes any missing values in such variants to the MAF. Also fixed a bug with chr- -omosome X only male samples. Also added --rsid option to use rs IDs of variants. Updated info file RefMAF to RefAF since it gives the alternate AF and NOT minor AF.
Ver 1.0.11 (April 21, 2015)
Version : 1.0.11 Author : Sayantan Das Date : April 21, 2015
Implemented parallel threading for processReference. Moved all haplotype data from char vector to bool vector. Implemented David's patch and similar other division-removal techniques. Also added legacy parameter unphased for unphased output.
Ver 1.0.10 (February 27, 2015)
Version : 1.0.10 Author : Sayantan Das Date : February 27, 2015
Add parameter --log which will output a log file (BUT not print on the screen). Also added legacy parameter --myChromosome whi- will enable users to use special chromosomes (apart from 1-22 and X)
Ver 1.0.9 (February 20, 2015)
Version : 1.0.8, 1.0.9 Author : Sayantan Das Date : February 20, 2015
Removed dependcies on boost directory. Converted all double to fl- oat. Had to change the main imputation formula to work well for l- arge values. Also updated Rsq calulation formual to avoid underfl- as a result of using float. Made minor changes to memory allocati- on so as to remove all 2-d or 3-d look-ups. Updated a loop to rem- ove repeat allocation, similar to minimac2. Now it is faster and uses much lesser memory.
Ver 1.0.7 (January 31, 2015)
Version : 1.0.7 Author : Sayantan Das Date : January 31, 2015
Fixed minor bugs and made thigsn ready for final release !!!
Ver 1.0.6 (November 4, 2014)
Version : 1.0.6 Author : Sayantan Das Date : November 4, 2014
Mended some minor bugs.
Ver 1.0.5 (October 4, 2014)
Version : 1.0.5 Author : Sayantan Das Date : October 3, 2014
Implemented chromosome X imputation (both PAR and non-PAR). Also added bgzip option to enable tabix options.
Ver 1.0.4 (September 19, 2014)
Version : 1.0.4 Author : Sayantan Das Date : September 19, 2014
Improved the chuking option for the compressed files. Updated name OPTM to M3VCF (Minimac3 VCF). Changed handle to --processReference. Added the option to add parameter estimates within M3VCF files wh- -ich can be used for later runs. Added a new handle --updateModel that will allow users to update these estimates using the target panel as well, if and when they think necessary.
Ver 1.0.2 (August 28, 2014)
Version : 1.0.2 Author : Sayantan Das Date : August 28, 2014
Mended a bug related to --vcfoutput option when used with --gzip option. Also added a option --format to manage handles in the FORMAT field for output VCF files.
Ver 1.0.1 (August 6, 2014)
Version : 1.0.1 Author : Sayantan Das Date : August 6, 2014
Added the --window option that will allow a buffer region around chunks. Also implemented this --start, --end and --window option on OPTM files. Minimac3 will, thus, be able to handle chunks on OPTM files as well. However, it will not perform the optimal all- -ocation again but just extract the chunk from the given file.
Ver 1.0.0 (July 21, 2014)
Version : 1.0.0 Author : Sayantan Das Date : July 21, 2014
Added the --vcfOutput option that will output the dosage in VCF format as well. Improved it by allowing to flush out VCF files after every 200 samples to save memory. Also implemented openmp to allow parallel computing.
Ver 0.7.1 (June 8, 2014)
Version : 0.7.1 Author : Sayantan Das Date : June 8, 2014
Added the --golden option that can calculate the actual R-square if the true/hard genotypes are provided in a vcf file. Made minor editions for debugging.
Ver 0.6.2 (June 8, 2014)
Version : 0.6.2 Author : Sayantan Das Date : June 8, 2014
Edited bug in VCF packing part that was giving slightly less opt- mized configurations.
Ver 0.6.1 (June 7, 2014)
Version : 0.6.1 Author : Sayantan Das Date : June 7, 2014
Worked on an important bug that gave memory overflow in large sa- mples. Name changed to Minimac3. Added utility for VCF output fo- rmat that would enable users to easy data manipulation.
Ver 0.5.1 (April 9, 2014)
Version : 0.5.1 Author : Sayantan Das Date : April 9, 2014
Removed futher bugs. Modified to use lesser memory and run faster. The Unique.cpp file was updated according to Goncalo's packVcf so- ftware. Target Data is being read more effeciently. Scaffolding done locally during imputation. Changed name to Minimac2.
Ver 0.4.1 (March 14, 2014)
Version : 0.4.1 Author : Sayantan Das Date : March 14 2014
Removed bug with --sample. Added lots of other options like --chr, --start, --end, --rounds, --states etc. Added data structure for variant type. Added structure so as NOT to use haplotype data after reading it once. Now the code uses the reduced haplotype structure once it has read the data. This was uses lesser memory.
Ver 0.2.1 (January 29, 2014)
Version : 0.2.1 Author : Sayantan Das Date : January 29 2014
Improved on the speed in parameter estimation part by NOT calcula-
ting the optimal allocation everytime, instead update it from the original allocation. Updated to use --refSnps for reference, use --vcfTarget for target files.
Ver 0.1.1 (May 25, 2014)
Version : 0.1.1 Author : Sayantan Das Date : May 25 2013
Started working on the basis of code of minimac. Implemented State Space Reduction Method for HMM calculations. Implemented the Para- meter Estimation part and Optimal Allocation part.
Download
Minimac3 is available as an undocumented release version. The source files (and binary executable) are available for download in Source Files and commonly used reference panels in VCF and M3VCF formats are available for download in Reference Panels.
Useful Wiki Pages
There are a few pages in this Wiki that may be useful to for Minimac3 users. Here are links to a few:
- Minimac3 Imputation Cookbook (Recommended for New Users!!)
Contact
In case of any queries and bugs please contact Sayantan Das.