GotCloud: FAQs

GotCloud Frequently Asked Questions (FAQs)

This page hopes to answer some of the FAQs about GotCloud.

If you cannot find the answer to your question here, please contact us.

Compiling FAQs

How do I compile

cd src/; make;

bam_tview_curses.c:5:20: error: curses.h: No such file or directory
bam_tview_curses.c:7:2: warning: #warning "_CURSES_LIB=1 but NCURSES_VERSION not defined; tview is NOT compiled"
bam_tview_curses.c:287:2: warning: #warning "No curses library is available; tview with curses is disabled."
make[2]: *** [bam_tview_curses.o] Error 1
make[2]: Leaving directory `/group/onel-lab/GeneticResources/bin/gotcloud/1.16/gotcloud-gotcloud.1.16/src/samtools'
make[1]: *** [all-recur] Error 1
make[1]: Leaving directory `/group/onel-lab/GeneticResources/bin/gotcloud/1.16/gotcloud-gotcloud.1.16/src/samtools'
make: *** [samtools] Error 2

This error is due to curses not being installed on your machine. To fix this error, you need to disable curses in samtools & samtools-hybrid

• Modify: gotcloud/src/samtools/Makefile and gotcloud/src/samtools-hybrid/Makefile
  1. Replace:

     -D_CURSES_LIB=1

     with:

     -D_CURSES_LIB=0

  2. Replace:

     LIBCURSES= -lcurses # -lXCurses

     with:

     LIBCURSES= # -lcurses # -lXCurses

mkdir -p premo/build; cd premo/build; cmake ..
/bin/sh: cmake: command not found
make: *** [premo] Error 127

This occurs if you do not have cmake installed on your machine.

• Since premo is not used by default, just remove premo from the build
  • premo is only used for the mosaik aligner (non-default) and for MEI calling (which is not yet implemented)
• Modify: gotcloud/src/Makefile
  • Remove premo from the EXES = ... line

bed.cpp:62: error: call of overloaded ‘to_string(int32_t&)’ is ambiguous
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/basic_string.h:2604: note: candidates are: std::string std::to_string(long long int)
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/basic_string.h:2610: note: std::string std::to_string(long long unsigned int)
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/basic_string.h:2616: note: std::string std::to_string(long double)
make[1]: *** [bed.o] Error 1
make[1]: Leaving directory `/group/onel-lab/GeneticResources/bin/gotcloud/1.16/gotcloud-gotcloud.1.16/src/vt'
make: *** [vt] Error 2

std::to_string is defined in c++11. However, not all compilers have updated to accept all overloads of to_string.

• You need to cast the integer to an overload that is defined:
  • Edit line 62 of gotcloud/src/vt/bed.cpp

    From:

    return this->chrom + ":" + std::to_string(this->start1) + "-" + std::to_string(this->end1);

    To:

    return this->chrom + ":" + std::to_string(static_cast<long long>(this->start1)) + "-" + std::to_string(static_cast<long long>(this->end1));

CMake Error: The current CMakeCache.txt directory /path1/gotcloud/src/premo/build/CMakeCache.txt is different than the directory /path2/gotcloud/src/premo/build where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
CMake Error: The source "/path1/gotcloud/src/premo/CMakeLists.txt" does not match the source "/path2/gotcloud/src/premo/CMakeLists.txt" used to generate cache. Re-run cmake with a different source directory.

This compilation error likely occurred because you moved GotCloud to a new directory and premo (one of the tools included in GotCloud) was still pointing to the old location.

• Remove src/premo/build/CMakeCache.txt and recompile.

Alignment Pipeline FAQs

Error "read <> mapped to 'chr1' at POS 0 to -1, flag 69,0 has BIN 4680 but should be 266824".
        "Fix it by using BAM->SAM->BAM to force a recalculation of the BIN field."
        "Fail to index the BAM file"

This error appears to be due to BWA reference files that do not match the version of BWA used to run GotCloud align. Run the following to regenerate the BWA reference files with the GotCloud version of BWA.

gotcloud/bin/bwa index path/to/your/ref.fa

Variant Calling Pipeline FAQs

make: *** [outdir/vcfs/chr20/chr20.hardfiltered.vcf.gz.OK] Error 1

Check the log file at: outdir/vcfs/chr20/chr20.hardfiltered.sites.vcf.summary.log

If it says:

[tabix] the index file either does not exist or is older than the vcf file. Please reindex.
ERROR: vcf-summary, failed to open /path/to/refVcf.vcf.gz, exit code: 1

It means that your VCF file is newer than the associated index (.tbi) file.

• If you are using a downloaded set of reference files, and the time stamps are similar, you just need to mark the .tbi file as newer (replace with the correct .tbi file):
  • touch /path/to/refVcf.vcf.gz.tbi
• If you are using your own reference files, and you think your vcf file is newer than your .tbi file, regenerate the .tbi file using (replace with the correct .vcf.gz file):
  • gotcloud/bin/tabix -f /path/to/refVcf.vcf.gz

Targetted/Exome

Exiting due to ERROR:
        INVALID_SORT: ERROR: File is not coordinate sorted at record 1003
        PreviousRecord was 1:15347, but this record is 1:15328

This error occurs if you have SAMTOOLS_VIEW_TARGET_ONLY = TRUE in your configuration file and a read(s) in your BAM file overlaps multiple regions in your BED file.

To fix this, set:

SAMTOOLS_VIEW_TARGET_ONLY = FALSE

Further Explanation of the issue:

The SAMTOOLS_VIEW_TARGET_ONLY set to TRUE tells our pipeline to pass to SAMTOOLS view all of the regions in the BED file that fall into a single segment of our processing.

It appears that samtools view will then go through each of those regions independently and one at a time, selecting just the records that fall within that region.

A problem occurs if records fall within multiple regions from the BED file. Since samtools view process the regions independently, it will write those records multiple times and they will end up out of order.

For example:

If we have BED with regions chr20 20300049-20300050 and chr20 20300052-20300053

The resulting call to samtools view looks like this:

gotcloud/bin/samtools-hybrid view -q 20 -F 0x0704 bamFile.bam 20:20300049-20300050 20:20300052-20300053

Say there are reads at positions 20299969 and 20299994 that are each 76 bases long, they will end up being output twice by the samtools view call and in this order:

20299969

20299994

20299969

20299994

Does Not Run

"make_path" is not exported by the File::Path module
Can't continue after import errors at /home/chengu/program/gotcloud/bin/umake.pl line 21
BEGIN failed--compilation aborted at /home/chengu/program/gotcloud/bin/umake.pl line 21.

This error occurs if you have an older version of Perl's File::Path module.

Two solutions

1. Upgrade to the latest Perl File::Path module.
2. Replace all occurrances of make_path in gotcloud/bin/ and gotcloud/scripts/ with mkpath:
   • gotcloud/bin/umake.pl - required to run snpcall
   • gotcloud/scripts/vcfSplitChr.pl - required to run snpcall
   • gotcloud/bin/genomestrip.pl - you only need to update this one if you want to run genomestrip
   • gotcloud/scripts/bed-diff.pl - you only need to update this one if you plan to run bed-diff.pl

ldrefine Pipeline FAQs

make: *** [outdir/beagle/chr#/split/bgl.#.chr#.PASS.#.vcf.gz.tbi] Error 1

Check the log file at: outdir/beagle/chr#/split/bgl.#.err

If it says:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -2

at phaser.j.a(Unknown Source)
at phaser.j.c(Unknown Source)
at phaser.h.a(Unknown Source)
at phaser.E.a(Unknown Source)
at phaser.Y.a(Unknown Source)
at phaser.PhaseMain.<init>(Unknown Source)
at phaser.PhaseMain.main(Unknown Source)

You need to change the seed for running beagle.

• In your configuration file, set:
  • BEAGLE = java -Xmx4g -jar $(BIN_DIR)/beagle.20101226.jar seed=993470 gprobs=true niterations=50 lowmem=true
• If it fails again, change the seed in the above command to something else, and rerun.

Version Problems

Prior to Version 1.16

perl is not found in /usr/bin/perl

Modify all of the .pl scripts in gotcloud/, gotcloud/bin/, and gotcloud/scripts/

• If the first line contains: #!/usr/bin/perl
  • Replace it with: #!/usr/bin/env perl
• Also, remove -w if it occurs on that first line of the perl scripts.
  • If it is not already specified in the script, add the following line later in the script: use warnings;

This issues has been fixed in version 1.16.

Version 1.08

ERROR: Could not find umake/outputDir in currentDirectory

To work around this bug, specify an absolute path for OUT_DIR/--outdir

This is due to the aligner trying to use OUT_PREFIX to transform the output directory to an absolute path, by prepending OUT_PREFIX. But, OUT_PREFIX is a snpcall setting that sets the snpcall Makefile filename. OUT_PREFIX will be deprecated in the next release. The current OUT_PREFIX which refers to the snpcall Makefile filename will be renamed to something else.

Version 1.07

Type of arg 1 to keys must be hash (not hash element) at gotcloud/bin/Conf.pm line 155, near "}) "
Compilation failed in require at gotcloud/bin/align.pl line 56.

To fix this bug, you need to edit gotcloud/bin/Conf.pm, line 155. This error only occurs with certain versions of perl.

Change

foreach my $key (keys $CONF_HASH{$section}) {

to

foreach my $key (keys %{$CONF_HASH{$section}}) {

Illegal division by zero at gotcloud/scripts/vcf-summary line 388.
make: *** [vcfs/chrY/chrY.hardfiltered.vcf.gz.OK] Error 255
make: *** Waiting for unfinished jobs....
Commands finished in xxxxx secs WITH ERRORS. Check the logs

The current version does not gracefully handle a chromosome with no variants.

By default, GotCloud assumes all samples are female, producing no chr Y variants, which will produce this error.

Work arounds:

• Disable Chromosome Y by adding the following to your configuration file:

CHRS = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 X

You may also want to disable chr X since it may be slightly misleading due to the assumption that all samples are female.

• Specify a pedigree file that tells GotCloud which individuals are Male & which are Female by adding the following to your configuration file:

PED_INDEX = pedigreeFile.ped

GotCloud only looks at the 2nd & 5th columns of the pedigree file (white-space delimited)

• 2nd column is the individual ID which should match the 1st column of the BAM_INDEX file.
• 5th column is the sex, use: 1 for male and 2 for female.