Changes

From Genome Analysis Wiki
Jump to navigationJump to search

GotCloud: Versions (view source)

Revision as of 12:17, 14 May 2015

11,791 bytes added ,  12:17, 14 May 2015
→‎Version 1.16 (Full Release on 2/25/2015)
Line 1: Line 1:  +
__TOC__
 +
 
For information on installing the releases, see: [[GotCloud#Install_GotCloud_Software|Install GotCloud Software]]
 
For information on installing the releases, see: [[GotCloud#Install_GotCloud_Software|Install GotCloud Software]]
 +
 +
For information on issues/resolutions for specific versions, see: [[GotCloud:_FAQs#Version_Problems|FAQ: Version Problems]]
 +
 +
== Version 1.17 (Full Release on 5/14/2015) ==
 +
Source can be downloaded from: https://github.com/statgen/gotcloud/releases/tag/gotcloud.1.17
 +
 +
'''General'''
 +
* Add ability to run custom pipelines
 +
* Fix bug in libVcfVcfFile.cpp
 +
* Fix some compatibility issues for CentOS5
 +
 +
'''Aligner'''
 +
* Add pipelines to run just recab & QC, and just QC.
 +
* VerifyBamID
 +
** Exclude ChrX & Y
 +
 +
'''SnpCall'''
 +
 +
'''Genotype Refinement'''
 +
 +
'''Indel'''
 +
 +
'''GenomeSTRiP'''
 +
 +
 +
== Version 1.16 (Full Release on 2/25/2015) ==
 +
Source can be downloaded from: https://github.com/statgen/gotcloud/releases/tag/gotcloud.1.16
 +
 +
'''General'''
 +
* Update the default REF to hs37d5.fa (build 37 with decoy) and the default DBSNP_VCF to dbsnp version 142.
 +
** You can download an updated reference at: [[GotCloud:_Genetic_Reference_and_Resource_Files#hs37d5-db142|hs37d5-db142]] (ftp://anonymous@share.sph.umich.edu/gotcloud/ref/hs37d5-db142-v1.tgz)
 +
* Upgrade perl scripts to use <code>/usr/bin/env perl</code> instead of <code>/usr/bin/perl</code> to make it compatible with more users
 +
* Upgrade to latest versions of libStatGen and bamUtil (versions 1.0.13)
 +
** Fixes bug in calculating the MD5s for the fasta in polishBam
 +
 +
'''Aligner'''
 +
* Update default aligner to <code>bwa mem</code>
 +
** you can still use <code>bwa aln</code> (the previous default) by adding the following setting to your configuration file:
 +
**:<code>MAP_TYPE = BWA</code>
 +
* Upgrade to <code>bwa</code> version 0.7.12
 +
* No longer call <code>verifyBamID</code> with the <code>--verbose</code> option
 +
 +
 +
'''SnpCall'''
 +
 +
'''Genotype Refinement'''
 +
 +
'''Indel'''
 +
* Cleanup pipeline.pl to reduce errors in some versions of perl
 +
 +
'''GenomeSTRiP'''
 +
 +
== Version 1.15 (Full Release on 12/16/2014) ==
 +
 +
'''General'''
 +
* Rename BAM_INDEX to BAM_LIST
 +
* Change default REF_DIR
 +
* Add ref_dir and list as command-line options to all pipelines
 +
* Add bed-diff script to compare VCFs
 +
 +
'''Aligner'''
 +
* By default, create BAM_LIST
 +
* Use SAMPLE instead of MERGE_NAME if MERGE_NAME is not specified in FASTQ_LIST
 +
* No longer require fastqs to end in 'fastq.gz' or 'fastq'
 +
* Rename INDEX_FILE to FASTQ_LIST and infer all fields except FASTQ1, FASTQ2, and either SAMPLE or MERGE_NAME
 +
* Change --numcs to --numjobs and what was --numjobs to --threads
 +
* Update to latest BWA
 +
** Update aligner to pass \t instead of tabs for the RG fieldto new version of BWA
 +
* By default, no longer store OQ
 +
 +
'''SnpCall'''
 +
* Add validation that:
 +
** Each BAM has only 1 sample
 +
** BAM's sampleID matches id in BAM_LIST
 +
*** Use --ignoreSMcheck to disable this validation
 +
* Updated Exome/Targeted settings
 +
** Set TARGET_DIR and OFFSET_OFF_TARGET (0) in defaults
 +
** Remove WRITE_TARGET_LOCI and base it on whether or not UNIFORM_TARGET_BED/MULTIPLE_TARGET_MAP are set and either the loci file doesn't exist, is older than the bed, or was created by a different bed
 +
* Add validation that tabix in perl scripts succeed
 +
* Fix some bugs in glfFlex & add region option
 +
* Cleanup logs so they no longer spew to the screen
 +
* Add ext-filt option for single sample filtering
 +
* Add .OK file after vcflist file successfully created
 +
 +
'''Genotype Refinement'''
 +
* Add validation that tabix in perl scripts succeed
 +
* Add .OK file after vcflist file successfully created
 +
 +
'''Indel'''
 +
* Update default region settings
 +
* Move output directories to an "indel" folder
 +
 +
'''GenomeSTRiP'''
 +
* Add a GenomeSTRiP pipeline
 +
 +
 +
== Version 1.14 (Full Release on 8/29/2014) ==
 +
 +
'''General'''
 +
* Add initial beagle4 support (as a new pipeline)
 +
* Improve input validation
 +
** Add chromosome name consistency checks to all tools
 +
* Upgrade version of bgzf
 +
* Upgrade libStatGen to fix mergeBam issue.
 +
 +
'''Aligner'''
 +
* Cleanup reading of fastq index/info file
 +
** ignore empty lines (generates a warning)
 +
** compress extra tabs/trim white space
 +
* Validate that BWA_QUAL and BWA_THREADS settings are properly formatted
 +
 +
'''SnpCall'''
 +
* Replace glfMultiples with glfFlex
 +
* Validate format of BAM_INDEX file
 +
* Add INDEL_VCF as an alternate for INDEL_PREFIX for input indel vcfs that aren't split by chromosome.
 +
 +
'''Genotype Refinement'''
 +
* Only run beagle/thunder with more than 1 sample
 +
 +
'''Indel'''
 +
* mergeBams for a single sample as its own step (didn't work before)
 +
* Fix bug that it would fail if the list of files was too long
 +
* Add input validation
 +
* Validate format of BAM_INDEX file
 +
 +
 +
== Version 1.13 (Full Release on 7/15/2014) ==
 +
''' General '''
 +
* Cleanup runcluster
 +
* Upgrade to bamUtil v1.0.12a
 +
* Upgrade to libStatGen v1.0.12
 +
* Update README to add build instructions & wiki references
 +
 +
'''Aligner'''
 +
* Increment to latest VerifyBamID
 +
 +
'''Variant Calling'''
 +
* Update glfMultiples to handle when first glf is empty
 +
* Add check for the output file before creating the .OK file
 +
* VcfPileup - improve return codes
 +
* Write jobfiles into a sub-directory
 +
* Added a snpcall monitoring utility
 +
* VcfSplit - update to only append .gz in the vcflist if there was at least one file
 +
* Write start/stop timestamps into a logfile (generated by runcluster)
 +
 +
'''Genotype Refinement'''
 +
* Update beagle2Vcf.pl to use 255 for missing PL/PL3 values
 +
* Update vcf2Beagle and beagle2Vcf to handle biallelic indels
 +
** Still doesn't handle any multiallelic variants
 +
* Added a ldrefine test.
 +
 +
'''Indel Calling'''
 +
* Initial version of Indel Caller
 +
** Still in testing phases, if you use, please provide feedback.
 +
 +
== Version 1.12 (Full Release on 1/17/2014) ==
 +
''' General '''
 +
* GotCloud now works when installed in a bin/ directory.
 +
* Add tabix source and build & bgzip build
 +
* Add some Copyright information
 +
* Fix printing of a failed run's return code
 +
* Upgrade to latest [[LibStatGen_Download#Official Releases|libStatGen]] & [[BamUtil#Release_of_just_BamUtil_.28does_not_include_libStatGen.29|bamUtil]].  See links for version details.
 +
** Slightly newer than 1.0.10 for both - versions on 1/17/2014.
 +
** dedup & recab now ignore Secondary reads
 +
** mergeBam ignores PI header field when merging
 +
** Add PhoneHome - gotCloud applies a PhoneHome thinning (BAMUTIL_THINNING) defaulted to 10 (10% of the time bamUtil does PhoneHome)
 +
* Upgrade QPLOT to ignore secondary reads
 +
* samtools
 +
** Update samtools index to return an error code if it fails to build the index
 +
 +
'''Aligner'''
 +
* Updgrade BWA
 +
** BWA_MEM is now an option
 +
* Write timestamps to Makefile log as steps start & complete
 +
* Remove tmp files as gotCloud goes, rather than at the end.
 +
* Deprecate RUN_QPLOT & RUN_VERIFY_BAM_ID
 +
** Now the steps to run are specified in configuration.
 +
* Mosaik
 +
** Upgrade to version from Oct 29, 2013
 +
** Add premo for pre-Mosaik processing
 +
 +
'''Variant Calling'''
 +
* Update to properly handle empty VCFs
 +
* Run make with -k option to run as much as possible after a failure.
 +
* Update to allow steps to be dependent on BAMs (BAM_DEPEND) so they will rerun if a BAM has a newer timestamp.
 +
* Input Validation
 +
** Check that BAMs exist & are not empty prior to running steps that require BAMs.
 +
** Check that filters min/maxDP are numbers, not fractions.
 +
* GlfMultiples
 +
** update to use DP instead of GD and fix PL description in format field header
 +
** add region option
 +
* samtools-hybrid
 +
** fail on missing BGZF EOF indicator
 +
 +
'''Genotype Refinement'''
 +
* Add a default number of states to Thunder
 +
 +
== Version 1.11 (Full Release on 9/6/2013) ==
 +
'''Aligner'''
 +
* Remove an extra space from the Makefile for the dedup command.
 +
* Brought in latest bwa source, but it is not yet being used.
 +
 +
'''Variant Calling'''
 +
* Rename OUT_PREFIX to MAKE_BASE_NAME to specify the base filename for snpcall, ldrefine (beagle & thunder), & vc Makefiles.  The typeOfRun.Makefile is appended to MAKE_BASE_NAME.
 +
** These Makefiles all used to have the same name and would overwrite each other
 +
** --makebasename/--make_basename/--make_base_name can be specified on the command-line
 +
** Default value for MAKE_BASE_NAME is umake
 +
*** snpcall is now: $(MAKE_BASE_NAME).snpcall.Makefile (default umake.snpcall.Makefile)
 +
*** ldrefine beagle step is now: $(MAKE_BASE_NAME).beagle.Makefile (default umake.beagle.Makefile)
 +
*** ldrefine thunder step is now: $(MAKE_BASE_NAME).thunder.Makefile (default umake.thunder.Makefile)
 +
*** vc is now: $(MAKE_BASE_NAME).vc.Makefile (default umake.vc.Makefile)
 +
* Added <code>gotcloud beagle</code> and <code>gotcloud thunder</code> commands so that beagle/thunder can be called independently rather than just through <code>ldrefine</code>.
 +
* Add command-line options to <code>gotcloud vc</code> for running just certain steps rather than having to set RUN...=true in the configuration
 +
** More than one --commandToRun can be specified at once
 +
** New commands-line options:
 +
*** <code>--index</code>  (or <code>RUN_INDEX = true</code> in the configuration file)
 +
*** <code>--pileup</code>  (or <code>RUN_PILEUP = true</code> in the configuration file)
 +
*** <code>--glfMultiples</code>  (or <code>RUN_GLFMULTIPLES = true</code> in the configuration file)
 +
*** <code>--vcfPileup</code>  (or <code>RUN_VCFPILEUP = true</code> in the configuration file)
 +
*** <code>--filter</code>  (or <code>RUN_FILTER = true</code> in the configuration file)
 +
*** <code>--svm</code>  (or <code>RUN_SVM = true</code> in the configuration file)
 +
*** <code>--split</code>  (or <code>RUN_SPLIT = true</code> in the configuration file)
 +
* Cleaned up the snpcall Makefile entries for pileup.  It used to print targets/commands that were never executed.  These unused targets have now been removed
 +
 +
'''Aligner & Variant Calling'''
 +
* Remove trailing spaces from configuration values
 +
* Add MAKE_OPTS configuration value that allows users to add Makefile options to the make calls that run the pipelines.
 +
* Update gccalcstorage for better estimates and to have option to print estimates from a starting size rather than from actually input files
 +
 +
== Version 1.10 (Full Release on 8/22/2013) ==
 +
'''Aligner'''
 +
* Update gccalcstorage for better align estimates
 +
 +
'''Variant Calling'''
 +
* Add additional comments to umake.pl
 +
* Update vcf-summary to print the skipped counts
 +
* Add option to specify the REF_FAI file used by the umake (gotcloud) script for determining CHRs and their lengths.
 +
 +
'''Aligner & Variant Calling'''
 +
* Only print Configuration settings to a file if the file doesn't exist
 +
 +
== Version 1.09a (Full Release on 8/08/2013) ==
 +
'''Aligner'''
 +
* Fix relative paths
 +
* Upgrade to newest samtools (and add source)
 +
* Update gcrunsummary.pl - summary stats for the run.
 +
* Upgrade to newer Mosaik
 +
 +
'''Variant Calling'''
 +
* Fix minNS filter for odd number of samples.  It used to give a fraction and then would be ignored.
 +
 +
'''Aligner & Variant Calling'''
 +
* Cleanup phonehome script
 +
* Cleanup gotcloud script and add ability to run perf/audria for dev purposes.
 +
 +
== Version 1.08 (Full Release on 7/31/2013) ==
 +
 +
'''Aligner'''
 +
* no aligner only changes
 +
 +
'''Variant Calling'''
 +
* Add the ability to copy a glf to a different directory prior to running glfExtract or glfMultiples
 +
* Remove chromosome Y from the default CHRS.  Also allow CHRS to be set on the commandline via a comma separated list specified in --chrs
 +
* Update glfMerge to skip glf files that only have a header.
 +
* Change default FILTER_MAX_SAMPLE_DP to 1000 (from 20)
 +
* Some SVM updates
 +
* Added the vc option to gotcloud which uses the RUN_...settings to decide which steps to use.
 +
 +
'''Aligner & Variant Calling'''
 +
* Fix bug in Conf.pm that caused a failure in some versions of perl
 +
* Add the ability to set the GOTCLOUD_ROOT so you can test with an alternate align.pl/umake.pl script and still be able to access everything else from the standard gotcloud path.
 +
* Cleanup the perldoc for align/snpcall
 +
* Output all configuration settings into a file when running.
 +
* Upgrade to most current libStatGen
 +
* Compile as optimized
    
== Version 1.07 (Full Release on 7/3/2013) ==
 
== Version 1.07 (Full Release on 7/3/2013) ==
Line 51: Line 328:  
* Update to always require REF
 
* Update to always require REF
 
** this fixes bug that ''ldrefine'' was not checking REF or adding the optional prefix to it.
 
** this fixes bug that ''ldrefine'' was not checking REF or adding the optional prefix to it.
 +
    
== Version 1.05 (Full Release on 4/17/2013) ==
 
== Version 1.05 (Full Release on 4/17/2013) ==
Retrieved from "http://genome.sph.umich.edu/wiki/Special:MobileDiff/7583...13225"

Navigation menu