View source for GotCloud: FAQs

= GotCloud Frequently Asked Questions (FAQs) =

This page hopes to answer some of the FAQs about [[GotCloud]].

If you cannot find the answer to your question here, please [[GotCloud#Getting Help with GotCloud|contact us]].

== General FAQs ==

== Compiling FAQs ==
; How do I compile
: <pre>cd src/; make;</pre>


;bam_tview_curses.c<nowiki>:5:20: error: curses.h: No such file or directory</nowiki><br />bam_tview_curses.c<nowiki>:7:2: warning: #warning "_CURSES_LIB=1 but NCURSES_VERSION not defined; tview is NOT compiled"</nowiki><br />bam_tview_curses.c<nowiki>:287:2: warning: #warning "No curses library is available; tview with curses is disabled."</nowiki><br />make[2]<nowiki>:</nowiki> *** [bam_tview_curses.o] Error 1<br />make[2]<nowiki>:</nowiki> Leaving directory `/group/onel-lab/GeneticResources/bin/gotcloud/1.16/gotcloud-gotcloud.1.16/src/samtools'<br />make[1]<nowiki>:</nowiki> *** [all-recur] Error 1<br />make[1]<nowiki>:</nowiki> Leaving directory `/group/onel-lab/GeneticResources/bin/gotcloud/1.16/gotcloud-gotcloud.1.16/src/samtools'<br />make<nowiki>:</nowiki> *** [samtools] Error 2
: This error is due to curses not being installed on your machine.  To fix this error, you need to disable curses in samtools & samtools-hybrid
:* Modify: <code>gotcloud/src/samtools/Makefile</code> and <code>gotcloud/src/samtools-hybrid/Makefile</code>
:*# Replace:
:*#: <code>-D_CURSES_LIB=1</code>
:*#: with:
:*#: <code>-D_CURSES_LIB=0</code>
:*# Replace:
:*#: <code>LIBCURSES=	-lcurses # -lXCurses</code>
:*#: with:
:*#: <code>LIBCURSES=	# -lcurses # -lXCurses</code>


;mkdir -p premo/build; cd premo/build; cmake ..<br />/bin/sh<nowiki>: cmake:</nowiki> command not found<br />make<nowiki>:</nowiki> *** [premo] Error 127
: This occurs if you do not have cmake installed on your machine.
:* Since premo is not used by default, just remove premo from the build
:** premo is only used for the mosaik aligner (non-default) and for MEI calling (which is not yet implemented)
:* Modify: <code>gotcloud/src/Makefile</code>
:** Remove <code>premo</code> from the <code>EXES = </code>... line


; <nowiki>bed.cpp:62: error: call of overloaded ‘to_string(int32_t&)’ is ambiguous</nowiki><br /><nowiki>/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/basic_string.h:2604: note: candidates are: std::string std::to_string(long long int)</nowiki><br /><nowiki>/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/basic_string.h:2610: note:                 std::string std::to_string(long long unsigned int)</nowiki><br /><nowiki>/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/basic_string.h:2616: note:                 std::string std::to_string(long double)</nowiki><br /><nowiki>make[1]: *** [bed.o] Error 1</nowiki><br /><nowiki>make[1]: Leaving directory `/group/onel-lab/GeneticResources/bin/gotcloud/1.16/gotcloud-gotcloud.1.16/src/vt'</nowiki><br /><nowiki>make: *** [vt] Error 2</nowiki>
: std::to_string is defined in c++11.  However, not all compilers have updated to accept all overloads of to_string.
:* You need to cast the integer to an overload that is defined:
:** Edit line 62 of <code>gotcloud/src/vt/bed.cpp</code>
:**:From:
:**::<code>return  this->chrom + ":" + std::to_string(this->start1) + "-" + std::to_string(this->end1);</code>
:**:To:
:**::<code>return  this->chrom + ":" + std::to_string(static_cast<long long>(this->start1)) + "-" + std::to_string(static_cast<long long>(this->end1));</code>


;CMake Error<nowiki>:</nowiki> The current CMakeCache.txt directory /path1/gotcloud/src/premo/build/CMakeCache.txt is different than the directory /path2/gotcloud/src/premo/build where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt<br /><nowiki>CMake Error: The source "/path1/gotcloud/src/premo/CMakeLists.txt" does not match the source "/path2/gotcloud/src/premo/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.
</nowiki>
: This compilation error likely occurred because you moved GotCloud to a new directory and premo (one of the tools included in GotCloud) was still pointing to the old location. 
:*Remove src/premo/build/CMakeCache.txt and recompile.

== Amazon FAQs ==

== Alignment Pipeline FAQs ==
; Error "read <> mapped to 'chr1' at POS 0 to -1, flag 69,0 has BIN 4680 but should be 266824".<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"Fix it by using BAM->SAM->BAM to force a recalculation of the BIN field."<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"Fail to index the BAM file"
:This error appears to be due to BWA reference files that do not match the version of BWA used to run GotCloud align.  Run the following to regenerate the BWA reference files with the GotCloud version of BWA.
::<pre>gotcloud/bin/bwa index path/to/your/ref.fa</pre>

== Variant Calling Pipeline FAQs ==
; make<nowiki>: *** [outdir/vcfs/chr20/chr20.hardfiltered.vcf.gz.OK] Error 1</nowiki>
: Check the log file at: <code>outdir/vcfs/chr20/chr20.hardfiltered.sites.vcf.summary.log</code>
: If it says:
:: '''[tabix] the index file either does not exist or is older than the vcf file. Please reindex.<br />ERROR: vcf-summary, failed to open ''/path/to/refVcf.vcf.gz'', exit code: 1'''
: It means that your VCF file is newer than the associated index (.tbi) file.
:* If you are using a downloaded set of reference files, and the time stamps are similar, you just need to mark the .tbi file as newer (replace with the correct .tbi file):
:** <code>touch ''/path/to/refVcf.vcf.gz.tbi''</code>
:* If you are using your own reference files, and you think your vcf file is newer than your .tbi file, regenerate the .tbi file using (replace with the correct .vcf.gz file):
:** <code>gotcloud/bin/tabix -f /path/to/refVcf.vcf.gz</code>


=== Targetted/Exome ===
; Exiting due to ERROR<nowiki>:</nowiki><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;INVALID_SORT<nowiki>:</nowiki> ERROR<nowiki>:</nowiki> File is not coordinate sorted at record 1003<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;PreviousRecord was 1<nowiki>:</nowiki>15347, but this record is 1<nowiki>:</nowiki>15328

: This error occurs if you have <code>SAMTOOLS_VIEW_TARGET_ONLY = TRUE</code> in your configuration file and a read(s) in your BAM file overlaps multiple regions in your BED file.  
: To fix this, set<nowiki>:</nowiki>
:: <pre>SAMTOOLS_VIEW_TARGET_ONLY = FALSE</pre>
:Further Explanation of the issue<nowiki>:</nowiki>
::The SAMTOOLS_VIEW_TARGET_ONLY set to TRUE tells our pipeline to pass to SAMTOOLS view all of the regions in the BED file that fall into a single segment of our processing.  
::It appears that samtools view will then go through each of those regions independently and one at a time, selecting just the records that fall within that region.  
::A problem occurs if records fall within multiple regions from the BED file.  Since samtools view process the regions independently, it will write those records multiple times and they will end up out of order.

::For example<nowiki>:</nowiki>
::: If we have BED with regions chr20 20300049-20300050 and chr20 20300052-20300053
:::The resulting call to samtools view looks like this:
::: <pre>gotcloud/bin/samtools-hybrid view -q 20 -F 0x0704 bamFile.bam 20:20300049-20300050 20:20300052-20300053</pre>

:::Say there are reads at positions 20299969 and 20299994 that are each 76 bases long, they will end up being output twice by the samtools view call and in this order:
::::20299969
::::20299994
::::20299969
::::20299994


=== Does Not Run ===
;"make_path" is not exported by the <nowiki>File::Path</nowiki> module<br />Can't continue after import errors at /home/chengu/program/gotcloud/bin/umake.pl line 21<br />BEGIN failed--compilation aborted at /home/chengu/program/gotcloud/bin/umake.pl line 21.

: This error occurs if you have an older version of Perl's <code>File::Path</code> module.
::Two solutions
::# Upgrade to the latest Perl File::Path module.
::# Replace all occurrances of <code>make_path</code> in <code>gotcloud/bin/</code> and <code>gotcloud/scripts/</code> with <code>mkpath</code>:
::#* '''gotcloud/bin/umake.pl''' - required to run snpcall
::#* '''gotcloud/scripts/vcfSplitChr.pl''' - required to run snpcall
::#* gotcloud/bin/genomestrip.pl - you only need to update this one if you want to run genomestrip
::#* gotcloud/scripts/bed-diff.pl - you only need to update this one if you plan to run bed-diff.pl


== ldrefine Pipeline FAQs ==
; make<nowiki>: *** [outdir/beagle/chr#/split/bgl.#.chr#.PASS.#.vcf.gz.tbi] Error 1</nowiki>
: Check the log file at: <code>outdir/beagle/chr#/split/bgl.#.err</code>
: If it says:
:: '''Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -2'''
:::        '''at phaser.j.a(Unknown Source)<br />        at phaser.j.c(Unknown Source)<br />        at phaser.h.a(Unknown Source)<br />        at phaser.E.a(Unknown Source)<br />        at phaser.Y.a(Unknown Source)<br />        at phaser.PhaseMain.<init>(Unknown Source)<br />        at phaser.PhaseMain.main(Unknown Source)'''
: You need to change the seed for running beagle.
:* In your configuration file, set:
:** <code>BEAGLE = java -Xmx4g -jar $(BIN_DIR)/beagle.20101226.jar seed=993470 gprobs=true niterations=50 lowmem=true</code>
:* If it fails again, change the seed in the above command to something else, and rerun.

== Tutorial FAQs ==

= Version Problems =

== Prior to Version 1.16 ==
; perl is not found in <code>/usr/bin/perl</code>
: Modify all of the .pl scripts in gotcloud/, gotcloud/bin/, and gotcloud/scripts/
:* If the first line contains: <code>#!/usr/bin/perl</code>
:** Replace it with: <code>#!/usr/bin/env perl</code>
:* Also, remove <code>-w</code> if it occurs on that first line of the perl scripts.
:** If it is not already specified in the script, add the following line later in the script: <code>use warnings;</code>
: This issues has been fixed in version 1.16.

== Version 1.08 ==
; ERROR<nowiki>:</nowiki> Could not find umake/'''outputDir''' in '''currentDirectory'''
: To work around this bug, specify an absolute path for OUT_DIR/--outdir
: This is due to the aligner trying to use OUT_PREFIX to transform the output directory  to an absolute path, by prepending OUT_PREFIX.  But, OUT_PREFIX is a snpcall setting that sets the snpcall Makefile filename.  OUT_PREFIX will be deprecated in the next release.  The current OUT_PREFIX which refers to the snpcall Makefile filename will be renamed to something else.

== Version 1.07 ==

;Type of arg 1 to keys must be hash (not hash element) at gotcloud/bin/Conf.pm line 155, near "}) "<br />Compilation failed in require at gotcloud/bin/align.pl line 56.

: To fix this bug, you need to edit gotcloud/bin/Conf.pm, line 155.  This error only occurs with certain versions of perl.
:Change
:<syntaxhighlight lang="perl">foreach my $key (keys $CONF_HASH{$section}) {</syntaxhighlight>
:to
:<syntaxhighlight lang="perl">foreach my $key (keys %{$CONF_HASH{$section}}) {</syntaxhighlight>


;Illegal division by zero at gotcloud/scripts/vcf-summary line 388.<br />make<nowiki>:</nowiki> *** [vcfs/chrY/chrY.hardfiltered.vcf.gz.OK] Error 255<br />make<nowiki>:</nowiki> *** Waiting for unfinished jobs....<br />Commands finished in xxxxx secs WITH ERRORS.  Check the logs
: The current version does not gracefully handle a chromosome with no variants.
: By default, GotCloud assumes all samples are female, producing no chr Y variants, which will produce this error.

: Work arounds:
::* Disable Chromosome Y by adding the following to your configuration file:
::: <pre>CHRS = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 X</pre>
::: You may also want to disable chr X since it may be slightly misleading due to the assumption that all samples are female.
::* Specify a pedigree file that tells GotCloud which individuals are Male & which are Female by adding the following to your configuration file:
::: <pre>PED_INDEX = pedigreeFile.ped</pre>
::: GotCloud only looks at the 2nd & 5th columns of the pedigree file (white-space delimited)
:::* 2nd column is the individual ID which should match the 1st column of the BAM_INDEX file.
:::* 5th column is the sex, use: 1 for male and 2 for female.