Changes

From Genome Analysis Wiki
Jump to navigationJump to search

GotCloud: Binary Releases (view source)

Revision as of 23:29, 7 January 2016

9,274 bytes added ,  23:29, 7 January 2016
no edit summary
Line 1: Line 1: −
'''Installing Debian Packages'''
+
Back to parent: [[GotCloud]]
 +
 
 +
 
 +
The [[GotCloud]] software has been developed and tested on Ubuntu Linux 12.04 and newer and Red Hat Enterprise Linux 6. While it should work on other Linux systems, they have not yet been tested.  It is not supported on Windows or Mac OS X.
 +
 
 +
 
 +
If you run into any problems with GotCloud, please check: [[GotCloud: FAQs]] or for problems/solutions for a specific version, [[GotCloud:_FAQs#Version_Problems|Version Problems]].
 +
 
 +
=GotCloud Binary Releases=
 +
GotCloud releases with the precompiled binary executables are available for download.
 +
 
 +
The binaries were compiled on Linux Ubuntu.
 +
 
 +
If you need the source, see [[GotCloud: Source Releases]] for information on obtaining the source.
 +
 
 +
The Binary Releases are available as:
 +
* [[#GotCloud_Debian_Packages|debian packages]] - require root access to install
 +
* [[#Tar Binary Releases|tarballs]] - anyone can untar
 +
 
 +
 
 +
The GotCloud software is distributed in several packages. These can be installed on traditional Debian systems as well as Ubuntu (where it has been tested). The software should run on all versions from Ubuntu Lucid (9.04) through Oneric (12.10). It has not been tested on other Debian systems, but it is likely it will work on those systems too. This software should work on any Unix system - only requiring recompilation from the source files.
 +
 
 +
The software is distributed in a single package including both the ''aligner'' and ''variant caller''. 
   −
Back to parent: [[GotCloud]]
+
There is also a separate package to test the installation.
    +
==GotCloud Debian Packages==
    +
If you would like to install from:
 +
* tar'd binary packages, see: [[#Tar Binary Releases|Tar Binary Releases]]
 +
* source (does not require root access), see: [[GotCloud: Source Releases]]
   −
The GotCloud software is distributed in several Debian packages. These can be installed on traditional Debian systems as well as Ubuntu (where it has been tested). The software should run on all versions from Ubuntu Lucid (9.04) through Oneric (12.10). It has not been tested on other Debian systems, but it is likely it will work on those systems too. This software should work on any Unix system - only requiring recompilation from the source files.
     −
The software is distributed in two parts, the ''aligner'' and ''umake''. Each has a package to test the installation. The following naming conventions are used:
+
=== Installing Debian Packages ===
    +
The debian files are available for download from ftp://share.sph.umich.edu/gotcloud/ :
 
<code>
 
<code>
−
   gotcloud-bin_M.m_amd64.deb         aligner and umake
+
   gotcloud-bin_latest_amd64.deb          aligner and variant caller
−
   gotcloud-test_M.n_amd64.deb         test data for software
+
  gotcloud-test_latest_amd64.deb        test data for software
 +
  M.n/gotcloud-bin_M.n_amd64.deb         version M.n of aligner and variant caller
 +
   M.n/gotcloud-test_M.n_amd64.deb       version M.n of test data for software
 
</code>
 
</code>
 +
Where 'M.n' is replaced with the specific version numbers.
 +
 +
Installing Debian packages normally requires root access, however, you <b>can</b> install as a non-root user (see below).
   −
where 'M.n' are the normal major-minor version system.
+
See [[GotCloud: Versions]] for details on the changes for each version.
   −
'''Fetch the Software Packages'''
     −
The package files are available from our public FTP site:
+
====Fetch the GotCloud Debian Packages====
    
<code>
 
<code>
Line 24: Line 54:  
   <b>mkdir debs</b>
 
   <b>mkdir debs</b>
 
   <b>cd debs</b>
 
   <b>cd debs</b>
−
   <b>wget ftp://share.sph.umich.edu/gotcloud/gotcloud\*.deb</b>
+
   <b>wget --retr-symlinks ftp://share.sph.umich.edu/gotcloud/gotcloud\*latest\*.deb</b>
 
   
 
   
 
   #  Another way:
 
   #  Another way:
Line 38: Line 68:  
   ftp> <b>cd gotcloud</b>
 
   ftp> <b>cd gotcloud</b>
 
   250 Directory successfully changed.
 
   250 Directory successfully changed.
−
   ftp> <b>mget gotcloud*.deb</b>
+
   ftp> <b>mget gotcloud*latest*.deb</b>
 
   ftp> <b>quit</b>
 
   ftp> <b>quit</b>
 
   221 Goodbye.
 
   221 Goodbye.
 
</code>
 
</code>
   −
'''Install the Software'''
+
====Install the GotCloud Debian Packages====
 +
 
 +
<b>Install Debian packages with root access</b>
   −
Installation requires one to have <b>root access</b> and uses <b>dpkg</b> which Debian users will recognize.
+
Installation normally requires one to have <b>root access</b> and uses <b>dpkg</b> which Debian users will recognize.
 
We recommend you install the first one separately so you can easily
 
We recommend you install the first one separately so you can easily
 
identify messages about prerequisite files that need to be installed:
 
identify messages about prerequisite files that need to be installed:
Line 64: Line 96:  
   #  'libssl0.9.8' is not installed, do 'sudo apt-get install libssl0.9.8'
 
   #  'libssl0.9.8' is not installed, do 'sudo apt-get install libssl0.9.8'
 
   #============================================================
 
   #============================================================
−
   ...
+
 +
   <b>sudo dpkg -i ~/debs/gotcloud-test_*_amd64.deb</b>
 
</code>
 
</code>
   −
If there were messages about missing packages, install them now
+
<b>Install Debian packages without root access</b>
 +
 
 +
Installing Debian packages generally requires root access, however, you can install as a non-root user using the following instructions (bash commands shown here):
 +
<code>
 +
tempdir=/tmp/$$.gotcloud.removeme    # Temp directory
 +
destdir=$HOME                        # Where you want GotCloud installed
 +
 +
dpkg -x [path_to_deb_file]/gotcloud-bin_*_amd64.deb $tempdir
 +
rsync -av --delete $tempdir/usr/local/gotcloud/* $destdir/gotcloud || exit 1
 +
rm -rf $tempdir
 +
 +
dpkg -x [path_to_deb_file]/gotcloud-test_*_amd64.deb $tempdir
 +
rsync -av --delete $tempdir/usr/local/gotcloud/* $destdir/gotcloud || exit 1
 +
rm -rf $tempdir
 +
 +
$destdir/gotcloud/scripts/check_requirements.sh  # Check requirements
 +
</code>
 +
 
 +
If there were messages about missing packages, install them with the commands shown by the script. Installing missing packages <b>will</b> require root access.
    
<code>
 
<code>
 
   <b>sudo apt-get update</b>
 
   <b>sudo apt-get update</b>
−
   <b>sudo apt-get install java-common default-jre make libssl0.9.8</b>
+
   <b>sudo apt-get install java-common default-jre make libssl0.9.8</b> # For example
 
</code>
 
</code>
   −
 
+
===Testing the GotCloud Installation===
−
'''Testing the Installation'''
      
We recommend that at least the first time, you install the test packages so you can
 
We recommend that at least the first time, you install the test packages so you can
Line 93: Line 143:  
   #======================================================================
 
   #======================================================================
 
   To test the GotCloud aligner, run:
 
   To test the GotCloud aligner, run:
−
     bin/gen_biopipeline.pl --test ~/testalign
+
     gotcloud align --test ~/testalign
 
   This will create/clear the output directory ~/testalign.
 
   This will create/clear the output directory ~/testalign.
 
   Test results and a log file are put in this directory.
 
   Test results and a log file are put in this directory.
 
   Results are self-checked and if errors should occur, it will be obvious.
 
   Results are self-checked and if errors should occur, it will be obvious.
 
    
 
    
−
   To test the GotCloud umake, run:
+
   To test the GotCloud snp caller, run:
−
     bin/umake.pl --test ~/testumake
+
     gotcloud snpcall --test ~/testsnp
−
   This will create/clear the output directory ~/testumake.
+
   This will create/clear the output directory ~/testsnp.
 
   Test results and a log file are put in this directory.
 
   Test results and a log file are put in this directory.
 
   Results are self-checked and if errors should occur, it will be obvious.
 
   Results are self-checked and if errors should occur, it will be obvious.
Line 106: Line 156:  
</code>
 
</code>
   −
Login as a normal user (ubuntu, not as root) and do:
+
Login as a normal user (not as root) and do:
 +
 
 +
<code>
 +
  #  Test the aligner (3 minutes +/- 1 minute)
 +
  '''/usr/local/gotcloud/bin/gotcloud align -test ~/testalign'''
 +
  Removing any previous results from: /home/myuser/testalign/aligntest
 +
  Running GOTCLOUD TEST, test log in: /home/myuser/testalign/aligntest.log
 +
  Created /home/myuser/testalign/aligntest/Makefiles/align_Sample2.Makefile
 +
  Created /home/myuser/testalign/aligntest/Makefiles/align_Sample1.Makefile
 +
  ---------------------------------------------------------------------
 +
  Waiting while samples are processed...
 +
  Processing finished in 63 secs with no errors reported
 +
  Results from DIFF will be in /home/myuser/testalign/diff_logfiles_results.txt
 +
  Successful comparison of data in '/home/myuser/testalign/aligntest' and '/usr/local/gotcloud/test/align/expected/aligntest'
 +
  Successfully ran the test case, congratulations!
 +
  '''rm -rf ~/testalign'''              # If no error
 +
 
 +
 
 +
  #  Test snp calling  (7 minutes +/-2 minute)
 +
  '''/usr/local/gotcloud/bin/gotcloud snpcall --test ~/testsnp'''
 +
  Removing any previous results from: /home/myuser/testsnp/umaketest
 +
  Running GOTCLOUD TEST, test log in: /home/myuser/testsnp/umaketest.log
 +
  Results from DIFF will be in /home/myuser/testsnp/diff_logfiles_results_umake.txt
 +
  Successful comparison of data in '/home/myuser/testsnp/umaketest' and '/usr/local/gotcloud/test/umake/expected/umaketest'
 +
  Successfully ran the test case, congratulations!
 +
  '''rm -rf ~/testsnp'''              # If no error
 +
</code>
 +
 
 +
== Tar Binary Releases ==
 +
 
 +
The tar binary releases contain the precompiled binaries and scripts necessary to run gotcloud.
 +
 
 +
If you would like to install from:
 +
* debian packages, see: [[#GotCloud Debian Packages|Debian Packages]]
 +
* source, see: [[GotCloud: Source Releases]]
 +
 
 +
=== Installing Tar Releases ===
 +
 
 +
The tar files are available for download from ftp://share.sph.umich.edu/gotcloud/ :
 +
<code>
 +
  gotcloudBin_latest.tgz          aligner and variant caller
 +
  gotcloudTest_latest.tgz        test data for software
 +
  M.n/gotcloud-bin_M.n.tar.gz    version M.n of aligner and variant caller
 +
  M.n/gotcloud-test_M.n.tar.gz    version M.n of test data for software
 +
</code>
 +
Where 'M.n' is replaced with the specific version numbers.
 +
 
 +
See [[GotCloud: Versions]] for details on the changes for each version.
 +
 
 +
====Fetch the Tar Releases====
 +
 
 +
<code>
 +
  #  The easiest way:
 +
  <b>wget --retr-symlinks ftp://share.sph.umich.edu/gotcloud/gotcloud-\*latest\*.tgz</b>
 +
 +
  #  Another way:
 +
  <b>ftp share.sph.umich.edu</b>
 +
  Connected to share.sph.umich.edu.
 +
  220 (vsFTPd 2.3.5)
 +
  Name (share.sph.umich.edu:tpg): <b>anonymous</b>
 +
  230 Login successful.
 +
  Remote system type is UNIX.
 +
  Using binary mode to transfer files.
 +
  ftp> <b>prompt</b>
 +
  Interactive mode off.
 +
  ftp> <b>cd gotcloud</b>
 +
  250 Directory successfully changed.
 +
  ftp> <b>mget gotcloud-*latest*.tgz</b>
 +
  ftp> <b>quit</b>
 +
  221 Goodbye.
 +
</code>
 +
 
 +
====Unpack the Tar Release====
    +
Unpack the realeases:
 +
tar xf gotcloud-*latest*.tgz
 +
 +
===Testing the GotCloud Release===
 +
 +
We recommend that at least the first time, you download the test packages so you can
 +
conveniently test the installation and make sure everything runs smoothly.
 +
The tests run within a few minutes and are self checking, so unless you see obvious errors,
 +
you can be reasonably sure everything is set up properly.
 +
You only need to do this once, unless you have made significant changes to your Unix system.
 +
 +
The test case results are self-checked and if errors should occur, it will be obvious.
 +
 
 
<code>
 
<code>
−
   #  Test the aligner (2 minutes +/- 1 minute)
+
   #  Test the aligner (3 minutes +/- 1 minute)
−
   '''/usr/local/gotcloud/bin/gen_biopipeline.pl -test ~/testalign'''
+
   '''gotcloud/bin/gotcloud align --test ~/testalign'''
−
   Running GOTCLOUD TEST, test log in: /home/myuser/testalign/biopipeTest.log
+
  Removing any previous results from: /home/myuser/testalign/aligntest
−
   Created /home/myuser/testalign/biopipetest/Makefiles/biopipe_Sample2.Makefile
+
   Running GOTCLOUD TEST, test log in: /home/myuser/testalign/aligntest.log
−
   Created /home/myuser/testalign/biopipetest/Makefiles/biopipe_Sample1.Makefile
+
   Created /home/myuser/testalign/aligntest/Makefiles/align_Sample2.Makefile
 +
   Created /home/myuser/testalign/aligntest/Makefiles/align_Sample1.Makefile
 
   ---------------------------------------------------------------------
 
   ---------------------------------------------------------------------
−
  Submitted 2 local commands
+
   Waiting while samples are processed...
−
   Waiting for commands to complete... . .  Commands finished in 37 secs with no errors reported
+
  Processing finished in 63 secs with no errors reported
 
   Results from DIFF will be in /home/myuser/testalign/diff_logfiles_results.txt
 
   Results from DIFF will be in /home/myuser/testalign/diff_logfiles_results.txt
−
   Successful comparison of data in '/home/myuser/testalign' and '/usr/local/gotcloud/test/align/expected'
+
   Successful comparison of data in '/home/myuser/testalign/aligntest' and 'gotcloud/test/align/expected/aligntest'
 
   Successfully ran the test case, congratulations!
 
   Successfully ran the test case, congratulations!
 
   '''rm -rf ~/testalign'''              # If no error
 
   '''rm -rf ~/testalign'''              # If no error
 
    
 
    
 
    
 
    
−
   #  Test umake (5 minutes +/-2 minute)
+
   #  Test snp calling (7 minutes +/-2 minute)
−
   '''/usr/local/gotcloud/bin/umake.pl --test ~/testumake'''
+
   '''gotcloud/bin/gotcloud snpcall --test ~/testsnp'''
−
   '''rm -rf ~/testumake'''              # If no error
+
  Removing any previous results from: /home/myuser/testsnp/umaketest
 +
  Running GOTCLOUD TEST, test log in: /home/myuser/testsnp/umaketest.log
 +
  Results from DIFF will be in /home/myuser/testsnp/diff_logfiles_results_umake.txt
 +
  Successful comparison of data in '/home/myuser/testsnp/umaketest' and 'gotcloud/test/umake/expected/umaketest'
 +
  Successfully ran the test case, congratulations!
 +
   '''rm -rf ~/testsnp'''              # If no error
 
</code>
 
</code>
   −
'''Creating Your Own Debian Package From the Source'''
+
=Creating Your Own Debian Package From the Source=
    
The process to create your own Debian package from the source files is straight-forward
 
The process to create your own Debian package from the source files is straight-forward
 
for Debian users. The source files actually contain source for the executables,
 
for Debian users. The source files actually contain source for the executables,
 
the test data and the scripts and configuration files used by the users.
 
the test data and the scripts and configuration files used by the users.
−
To source builds two debian package files - one for the test data (test) and one for the
+
These steps build two debian package files - one for the test data (test) and one for the
 
binaries and scripts (bin).
 
binaries and scripts (bin).
   −
In order to simply the build of two packages, the debian directory contains the control
+
In order to simplify the building of two packages, the debian directory contains the control files for both packages (bin and test).
−
files for both packages (bin and test).
   
This requires that the normal control files (e.g. rules) be named rules.test and rules.bin
 
This requires that the normal control files (e.g. rules) be named rules.test and rules.bin
 
and the shell script (makedeb.sh) be aware of this nuance.
 
and the shell script (makedeb.sh) be aware of this nuance.
 
In addition the shell script can make an RPM file (using alien) and will help manage
 
In addition the shell script can make an RPM file (using alien) and will help manage
 
the version for GotCloud.
 
the version for GotCloud.
 +
 +
'''Fetch the Source Including the Test Files'''
    
<code>
 
<code>
−
#   Get the source for GotCloud
+
# The easiest way:
 +
wget ftp://share.sph.umich.edu/gotcloud/gotcloudWithTest_latest.tgz
   −
#   To build a Debian package for bin
+
# Another way:
 +
ftp share.sph.umich.edu
 +
Connected to share.sph.umich.edu.
 +
220 (vsFTPd 2.3.5)
 +
Name (share.sph.umich.edu:tpg): anonymous
 +
230 Login successful.
 +
Remote system type is UNIX.
 +
Using binary mode to transfer files.
 +
ftp> prompt
 +
Interactive mode off.
 +
ftp> cd gotcloud
 +
250 Directory successfully changed.
 +
ftp> get gotcloudWithTest_latest.tgz
 +
ftp> quit
 +
221 Goodbye.
    +
#  Extract the source files
 +
cd ~/src                # Or wherever you want the source files
 +
tar xzf path/gotcloudWithTest_latest.tgz
 
</code>
 
</code>
 +
 +
 +
'''Fetch the Source Excluding the Test Files'''
 +
 +
Follow the above logic, but use: gotcloud_latest.tgz
 +
 +
'''Fetch just the Test Files'''
 +
 +
Follow the above logic, but use: gotcloudTest_latest.tgz
 +
 +
 +
===Create the Debian Package Files===
 +
 +
<code>
 +
'''cd ~/src'''                # Or wherever you put the source files
 +
'''cd gotcloud*'''
 +
 +
#  Add -replace only if rebuilding the same version
 +
'''debian/makedeb.sh  bin  ='''
 +
  dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2 ...
 +
  dpkg-buildpackage: export C...
 +
    [lines deleted]
 +
  Generate package: cd gotcloud-bin; dpkg-buildpackage -b -us -uc -rfakeroot
 +
  ===> Version '1.01 of 'gotcloud-bin' will install into 'usr/local/gotcloud'
 +
  install -d debian/tmp/DEBIAN
 +
  cp debian/control debian/postinst debian/postrm debian/tmp/DEBIAN
 +
  install -d debian/tmp/usr/share/doc/gotcloud-bin_1.01
 +
  sed -e s/99.99/1.01/g < debian/changelog > debian/changelog.new
 +
    [lines deleted]
 +
  Converting symlinks into the actual executables to be distributed
 +
    [lines deleted]
 +
  dpkg-deb: building package `gotcloud-bin' in `./gotcloud-bin_1.01_amd64.deb'.
 +
  Package file now in /home/myuser/dev/gotcloud
 +
  -rw-r--r-- 1 myuser myuser 3592780 Feb 19 11:59 gotcloud-bin_1.01_amd64.deb
 +
 +
#  Add -replace only if rebuilding the same version
 +
'''debian/makedeb.sh  test ='''
 +
  dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2 ...
 +
  dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): -D_FORTIFY_SOURCE=2
 +
    [lines deleted]
 +
  Generate package: cd gotcloud-test; dpkg-buildpackage -b -us -uc -rfakeroot
 +
  ===> Version '1.01 of 'gotcloud-test' will install into 'usr/local/gotcloud'
 +
  install -d debian/tmp/DEBIAN
 +
  cp debian/control debian/postinst debian/postrm debian/tmp/DEBIAN
 +
  install -d debian/tmp/usr/share/doc/gotcloud-test_1.01
 +
  sed -e s/99.99/1.01/g < debian/changelog > debian/changelog.new
 +
    [lines deleted]
 +
  dpkg-deb: building package `gotcloud-test' in `./gotcloud-test_1.01_amd64.deb'.
 +
  Package file now in /home/myuser/dev/gotcloud
 +
  -rw-r--r-- 1 myuser myuser 433269556 Feb 19 13:49 gotcloud-test_1.01_amd64.deb
 +
</code>
 +
 +
The deb files will be created in the directory above the debian directory  (e.g. gotcloud)
Pjvh
61

edits

Retrieved from "http://genome.sph.umich.edu/wiki/Special:MobileDiff/6460...13968"

Navigation menu