Changes

From Genome Analysis Wiki
Jump to navigationJump to search

GotCloud: Binary Releases (view source)

Revision as of 23:29, 7 January 2016

11,849 bytes added ,  23:29, 7 January 2016
no edit summary
Line 1: Line 1: −
Debian Packages
+
Back to parent: [[GotCloud]]
   −
The BioPipeline software is distributed in several Debian packages. These can be installed on traditional Debian systems as well as Ubuntu (where it has been tested). The software should run on all versions from Ubuntu Lucid (9.04) through Oneric (12.10). It has not been tested on other Debian systems, but it is likely they will work on those systems too. This software should work on any Unix system - only requiring recompilation from the source files.
     −
The software is distributed in two parts, the aligner and umake. Each has a package to test the installation. The following naming conventions are used:
+
The [[GotCloud]] software has been developed and tested on Ubuntu Linux 12.04 and newer and Red Hat Enterprise Linux 6. While it should work on other Linux systems, they have not yet been tested. It is not supported on Windows or Mac OS X.
    +
 +
If you run into any problems with GotCloud, please check: [[GotCloud: FAQs]] or for problems/solutions for a specific version, [[GotCloud:_FAQs#Version_Problems|Version Problems]].
 +
 +
=GotCloud Binary Releases=
 +
GotCloud releases with the precompiled binary executables are available for download.
 +
 +
The binaries were compiled on Linux Ubuntu.
 +
 +
If you need the source, see [[GotCloud: Source Releases]] for information on obtaining the source.
 +
 +
The Binary Releases are available as:
 +
* [[#GotCloud_Debian_Packages|debian packages]] - require root access to install
 +
* [[#Tar Binary Releases|tarballs]] - anyone can untar
 +
 +
 +
The GotCloud software is distributed in several packages. These can be installed on traditional Debian systems as well as Ubuntu (where it has been tested). The software should run on all versions from Ubuntu Lucid (9.04) through Oneric (12.10). It has not been tested on other Debian systems, but it is likely it will work on those systems too. This software should work on any Unix system - only requiring recompilation from the source files.
 +
 +
The software is distributed in a single package including both the ''aligner'' and ''variant caller''. 
 +
 +
There is also a separate package to test the installation.
 +
 +
==GotCloud Debian Packages==
 +
 +
If you would like to install from:
 +
* tar'd binary packages, see: [[#Tar Binary Releases|Tar Binary Releases]]
 +
* source (does not require root access), see: [[GotCloud: Source Releases]]
 +
 +
 +
=== Installing Debian Packages ===
 +
 +
The debian files are available for download from ftp://share.sph.umich.edu/gotcloud/ :
 
<code>
 
<code>
−
   biopipe-align_M.m_amd64.deb          aligner
+
   gotcloud-bin_latest_amd64.deb          aligner and variant caller
−
   biopipe-umake_M.n_amd64.deb         umake
+
   gotcloud-test_latest_amd64.deb         test data for software
−
   biopipe-testalign_M.n_amd64.deb
+
   M.n/gotcloud-bin_M.n_amd64.deb         version M.n of aligner and variant caller
−
   biopipe-testumake_M.n_amd64.deb
+
   M.n/gotcloud-test_M.n_amd64.deb       version M.n of test data for software
 
</code>
 
</code>
 +
Where 'M.n' is replaced with the specific version numbers.
 +
 +
Installing Debian packages normally requires root access, however, you <b>can</b> install as a non-root user (see below).
   −
The package files are available from our public FTP site:
+
See [[GotCloud: Versions]] for details on the changes for each version.
 +
 
 +
 
 +
====Fetch the GotCloud Debian Packages====
    
<code>
 
<code>
 +
  #  The easiest way:
 +
  <b>mkdir debs</b>
 +
  <b>cd debs</b>
 +
  <b>wget --retr-symlinks ftp://share.sph.umich.edu/gotcloud/gotcloud\*latest\*.deb</b>
 +
 +
  #  Another way:
 
   <b>ftp share.sph.umich.edu</b>
 
   <b>ftp share.sph.umich.edu</b>
 
   Connected to share.sph.umich.edu.
 
   Connected to share.sph.umich.edu.
 
   220 (vsFTPd 2.3.5)
 
   220 (vsFTPd 2.3.5)
−
   Name (share.sph.umich.edu:tpg): anonymous
+
   Name (share.sph.umich.edu:tpg): <b>anonymous</b>
 
   230 Login successful.
 
   230 Login successful.
 
   Remote system type is UNIX.
 
   Remote system type is UNIX.
 
   Using binary mode to transfer files.
 
   Using binary mode to transfer files.
−
   ftp> cd biopipe
+
   ftp> <b>prompt</b>
 +
  Interactive mode off.
 +
  ftp> <b>cd gotcloud</b>
 
   250 Directory successfully changed.
 
   250 Directory successfully changed.
−
   ftp> mget biopip*.deb
+
   ftp> <b>mget gotcloud*latest*.deb</b>
−
   ftp> quit
+
   ftp> <b>quit</b>
 
   221 Goodbye.
 
   221 Goodbye.
 
</code>
 
</code>
   −
Installation requires one to have root access and uses dpkg Ubuntu users will recognize. We recommend you install the first one separately so it can identify prerequisite files:
+
====Install the GotCloud Debian Packages====
 +
 
 +
<b>Install Debian packages with root access</b>
 +
 
 +
Installation normally requires one to have <b>root access</b> and uses <b>dpkg</b> which Debian users will recognize.
 +
We recommend you install the first one separately so you can easily
 +
identify messages about prerequisite files that need to be installed:
    
<code>
 
<code>
−
   <b>sudo dpkg -i debs/biopipe-align_0.6_amd64.deb</b>
+
   <b>sudo dpkg -i ~/debs/gotcloud-bin_*_amd64.deb</b>
−
   Selecting previously unselected package biopipe-align.
+
   Selecting previously unselected package gotcloud-bin.
 
   (Reading database ... 25372 files and directories currently installed.)
 
   (Reading database ... 25372 files and directories currently installed.)
−
   Unpacking biopipe-align (from .../biopipe-align_0.6_amd64.deb) ...
+
   Unpacking gotcloud-bin (from .../gotcloud-bin_1.0_amd64.deb) ...
−
   Setting up biopipe-align (0.6) ...
+
   Setting up gotcloud-bin (1.0) ...
−
  Created symlink '/usr/local/bin/gen_biopipeline.pl' to make running the aligner easier.
   
   #============================================================
 
   #============================================================
 
   #  'java' is not installed, do 'sudo apt-get install java-common default-jre'
 
   #  'java' is not installed, do 'sudo apt-get install java-common default-jre'
Line 47: Line 96:  
   #  'libssl0.9.8' is not installed, do 'sudo apt-get install libssl0.9.8'
 
   #  'libssl0.9.8' is not installed, do 'sudo apt-get install libssl0.9.8'
 
   #============================================================
 
   #============================================================
−
   #============================================================
+
−
   #  '/usr/local/biopipe/test/align' does not exist so you cannot test this install
+
  <b>sudo dpkg -i ~/debs/gotcloud-test_*_amd64.deb</b>
−
  #  Install it by doing 'sudo dpkg -i biopipe-test_*.deb'
+
</code>
−
   #============================================================
+
 
 +
<b>Install Debian packages without root access</b>
 +
 
 +
Installing Debian packages generally requires root access, however, you can install as a non-root user using the following instructions (bash commands shown here):
 +
<code>
 +
tempdir=/tmp/$$.gotcloud.removeme    # Temp directory
 +
destdir=$HOME                        # Where you want GotCloud installed
 +
 +
dpkg -x [path_to_deb_file]/gotcloud-bin_*_amd64.deb $tempdir
 +
rsync -av --delete $tempdir/usr/local/gotcloud/* $destdir/gotcloud || exit 1
 +
rm -rf $tempdir
 +
 +
dpkg -x [path_to_deb_file]/gotcloud-test_*_amd64.deb $tempdir
 +
rsync -av --delete $tempdir/usr/local/gotcloud/* $destdir/gotcloud || exit 1
 +
rm -rf $tempdir
 +
 +
$destdir/gotcloud/scripts/check_requirements.sh  # Check requirements
 +
</code>
 +
 
 +
If there were messages about missing packages, install them with the commands shown by the script. Installing missing packages <b>will</b> require root access.
 +
 
 +
<code>
 +
  <b>sudo apt-get update</b>
 +
  <b>sudo apt-get install java-common default-jre make libssl0.9.8</b>  # For example
 +
</code>
 +
 
 +
===Testing the GotCloud Installation===
 +
 
 +
We recommend that at least the first time, you install the test packages so you can
 +
conveniently test the installation and make sure everything runs smoothly.
 +
The tests run within a few minutes and are self checking, so unless you see obvious errors,
 +
you can be reasonably sure everything is set up properly.
 +
You only need to do this once, unless you have made significant changes to your Unix system.
 +
 
 +
<code> 
 +
  <b>sudo dpkg -i ~/debs/gotcloud-test*_amd64.deb</b>
 +
  Selecting previously unselected package gotcloud-test.
 +
  (Reading database ... 27376 files and directories currently installed.)
 +
  Unpacking gotcloud-test (from .../gotcloud-test_1.0_amd64.deb) ...
 +
  Setting up gotcloud-test (1.0) ...
 +
   #======================================================================
 +
   #  GotCloud README
 +
   #======================================================================
 +
  To test the GotCloud aligner, run:
 +
    gotcloud align --test ~/testalign
 +
  This will create/clear the output directory ~/testalign.
 +
  Test results and a log file are put in this directory.
 +
  Results are self-checked and if errors should occur, it will be obvious.
 +
 
 +
  To test the GotCloud snp caller, run:
 +
    gotcloud snpcall --test ~/testsnp
 +
  This will create/clear the output directory ~/testsnp.
 +
  Test results and a log file are put in this directory.
 +
  Results are self-checked and if errors should occur, it will be obvious.
 +
 
 +
</code>
 +
 
 +
Login as a normal user (not as root) and do:
 +
 
 +
<code>
 +
  #  Test the aligner (3 minutes +/- 1 minute)
 +
  '''/usr/local/gotcloud/bin/gotcloud align -test ~/testalign'''
 +
  Removing any previous results from: /home/myuser/testalign/aligntest
 +
  Running GOTCLOUD TEST, test log in: /home/myuser/testalign/aligntest.log
 +
  Created /home/myuser/testalign/aligntest/Makefiles/align_Sample2.Makefile
 +
  Created /home/myuser/testalign/aligntest/Makefiles/align_Sample1.Makefile
 +
  ---------------------------------------------------------------------
 +
  Waiting while samples are processed...
 +
  Processing finished in 63 secs with no errors reported
 +
  Results from DIFF will be in /home/myuser/testalign/diff_logfiles_results.txt
 +
  Successful comparison of data in '/home/myuser/testalign/aligntest' and '/usr/local/gotcloud/test/align/expected/aligntest'
 +
  Successfully ran the test case, congratulations!
 +
  '''rm -rf ~/testalign'''              # If no error
 +
 
 +
 
 +
  #  Test snp calling  (7 minutes +/-2 minute)
 +
  '''/usr/local/gotcloud/bin/gotcloud snpcall --test ~/testsnp'''
 +
  Removing any previous results from: /home/myuser/testsnp/umaketest
 +
  Running GOTCLOUD TEST, test log in: /home/myuser/testsnp/umaketest.log
 +
  Results from DIFF will be in /home/myuser/testsnp/diff_logfiles_results_umake.txt
 +
  Successful comparison of data in '/home/myuser/testsnp/umaketest' and '/usr/local/gotcloud/test/umake/expected/umaketest'
 +
  Successfully ran the test case, congratulations!
 +
  '''rm -rf ~/testsnp'''              # If no error
 
</code>
 
</code>
   −
If there were messages about missing packages, install them now
+
== Tar Binary Releases ==
 +
 
 +
The tar binary releases contain the precompiled binaries and scripts necessary to run gotcloud.
    +
If you would like to install from:
 +
* debian packages, see: [[#GotCloud Debian Packages|Debian Packages]]
 +
* source, see: [[GotCloud: Source Releases]]
 +
 +
=== Installing Tar Releases ===
 +
 +
The tar files are available for download from ftp://share.sph.umich.edu/gotcloud/ :
 
<code>
 
<code>
−
   <b>sudo apt-get install make libssl0.9.8 java-common default-jre</b>
+
   gotcloudBin_latest.tgz          aligner and variant caller
 +
  gotcloudTest_latest.tgz        test data for software
 +
  M.n/gotcloud-bin_M.n.tar.gz    version M.n of aligner and variant caller
 +
  M.n/gotcloud-test_M.n.tar.gz    version M.n of test data for software
 
</code>
 
</code>
 +
Where 'M.n' is replaced with the specific version numbers.
 +
 +
See [[GotCloud: Versions]] for details on the changes for each version.
 +
 +
====Fetch the Tar Releases====
 +
 +
<code>
 +
  #  The easiest way:
 +
  <b>wget --retr-symlinks ftp://share.sph.umich.edu/gotcloud/gotcloud-\*latest\*.tgz</b>
 +
 +
  #  Another way:
 +
  <b>ftp share.sph.umich.edu</b>
 +
  Connected to share.sph.umich.edu.
 +
  220 (vsFTPd 2.3.5)
 +
  Name (share.sph.umich.edu:tpg): <b>anonymous</b>
 +
  230 Login successful.
 +
  Remote system type is UNIX.
 +
  Using binary mode to transfer files.
 +
  ftp> <b>prompt</b>
 +
  Interactive mode off.
 +
  ftp> <b>cd gotcloud</b>
 +
  250 Directory successfully changed.
 +
  ftp> <b>mget gotcloud-*latest*.tgz</b>
 +
  ftp> <b>quit</b>
 +
  221 Goodbye.
 +
</code>
 +
 +
====Unpack the Tar Release====
 +
 +
Unpack the realeases:
 +
tar xf gotcloud-*latest*.tgz
   −
Finally, install the other packages. We recommend that at least the first time,
+
===Testing the GotCloud Release===
−
you install the test packages so you can conveniently test the installation and make sure everything runs smoothly.
+
 
 +
We recommend that at least the first time, you download the test packages so you can
 +
conveniently test the installation and make sure everything runs smoothly.
 
The tests run within a few minutes and are self checking, so unless you see obvious errors,
 
The tests run within a few minutes and are self checking, so unless you see obvious errors,
 
you can be reasonably sure everything is set up properly.
 
you can be reasonably sure everything is set up properly.
 +
You only need to do this once, unless you have made significant changes to your Unix system.
   −
<code>
+
The test case results are self-checked and if errors should occur, it will be obvious.
−
  <b>sudo dpkg -i biopipe-umake_M.n_amd64.deb
+
    
−
   sudo dpkg -i biopipe-testalign_0.6_amd64.deb biopipe-testumake_M.n_amd64.deb</b>
   
<code>
 
<code>
 +
  #  Test the aligner (3 minutes +/- 1 minute)
 +
  '''gotcloud/bin/gotcloud align --test ~/testalign'''
 +
  Removing any previous results from: /home/myuser/testalign/aligntest
 +
  Running GOTCLOUD TEST, test log in: /home/myuser/testalign/aligntest.log
 +
  Created /home/myuser/testalign/aligntest/Makefiles/align_Sample2.Makefile
 +
  Created /home/myuser/testalign/aligntest/Makefiles/align_Sample1.Makefile
 +
  ---------------------------------------------------------------------
 +
  Waiting while samples are processed...
 +
  Processing finished in 63 secs with no errors reported
 +
  Results from DIFF will be in /home/myuser/testalign/diff_logfiles_results.txt
 +
  Successful comparison of data in '/home/myuser/testalign/aligntest' and 'gotcloud/test/align/expected/aligntest'
 +
  Successfully ran the test case, congratulations!
 +
  '''rm -rf ~/testalign'''              # If no error
 +
 
 +
 
 +
  #  Test snp calling  (7 minutes +/-2 minute)
 +
  '''gotcloud/bin/gotcloud snpcall --test ~/testsnp'''
 +
  Removing any previous results from: /home/myuser/testsnp/umaketest
 +
  Running GOTCLOUD TEST, test log in: /home/myuser/testsnp/umaketest.log
 +
  Results from DIFF will be in /home/myuser/testsnp/diff_logfiles_results_umake.txt
 +
  Successful comparison of data in '/home/myuser/testsnp/umaketest' and 'gotcloud/test/umake/expected/umaketest'
 +
  Successfully ran the test case, congratulations!
 +
  '''rm -rf ~/testsnp'''              # If no error
 +
</code>
 +
 +
=Creating Your Own Debian Package From the Source=
   −
If you installed the test packages, test your installation. You only need to do this once,
+
The process to create your own Debian package from the source files is straight-forward
−
unless you have made signifcant changes to your Unix system.
+
for Debian users. The source files actually contain source for the executables,
−
Login as a normal user (not as root) and do:
+
the test data and the scripts and configuration files used by the users.
 +
These steps build two debian package files - one for the test data (test) and one for the
 +
binaries and scripts (bin).
 +
 
 +
In order to simplify the building of two packages, the debian directory contains the control files for both packages (bin and test).
 +
This requires that the normal control files (e.g. rules) be named rules.test and rules.bin
 +
and the shell script (makedeb.sh) be aware of this nuance.
 +
In addition the shell script can make an RPM file (using alien) and will help manage
 +
the version for GotCloud.
 +
 
 +
'''Fetch the Source Including the Test Files'''
 +
 
 +
<code>
 +
#  The easiest way:
 +
wget ftp://share.sph.umich.edu/gotcloud/gotcloudWithTest_latest.tgz
 +
 
 +
#  Another way:
 +
ftp share.sph.umich.edu
 +
Connected to share.sph.umich.edu.
 +
220 (vsFTPd 2.3.5)
 +
Name (share.sph.umich.edu:tpg): anonymous
 +
230 Login successful.
 +
Remote system type is UNIX.
 +
Using binary mode to transfer files.
 +
ftp> prompt
 +
Interactive mode off.
 +
ftp> cd gotcloud
 +
250 Directory successfully changed.
 +
ftp> get gotcloudWithTest_latest.tgz
 +
ftp> quit
 +
221 Goodbye.
 +
 
 +
#  Extract the source files
 +
cd ~/src                # Or wherever you want the source files
 +
tar xzf path/gotcloudWithTest_latest.tgz
 +
</code>
 +
 
 +
 
 +
'''Fetch the Source Excluding the Test Files'''
 +
 
 +
Follow the above logic, but use: gotcloud_latest.tgz
 +
 
 +
'''Fetch just the Test Files'''
 +
 
 +
Follow the above logic, but use: gotcloudTest_latest.tgz
 +
 
 +
 
 +
===Create the Debian Package Files===
    
<code>
 
<code>
−
  #   Test the aligner (fast, about 3 minutes)
+
'''cd ~/src'''                # Or wherever you put the source files
−
  <b>/usr/local/biopipe/bin/gen_biopipeline.pl --test ~/testalign</b>
+
'''cd gotcloud*'''
 
   
 
   
−
  #  Test umake (longer, about 15 minutes)
+
#  Add -replace only if rebuilding the same version
−
   <b>/usr/local/biopipe/bin/umake.pl --test ~/testumake</b>
+
'''debian/makedeb.sh  bin ='''
 +
  dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2 ...
 +
   dpkg-buildpackage: export C...
 +
    [lines deleted]
 +
  Generate package: cd gotcloud-bin; dpkg-buildpackage -b -us -uc -rfakeroot
 +
  ===> Version '1.01 of 'gotcloud-bin' will install into 'usr/local/gotcloud'
 +
  install -d debian/tmp/DEBIAN
 +
  cp debian/control debian/postinst debian/postrm debian/tmp/DEBIAN
 +
  install -d debian/tmp/usr/share/doc/gotcloud-bin_1.01
 +
  sed -e s/99.99/1.01/g < debian/changelog > debian/changelog.new
 +
    [lines deleted]
 +
  Converting symlinks into the actual executables to be distributed
 +
    [lines deleted]
 +
  dpkg-deb: building package `gotcloud-bin' in `./gotcloud-bin_1.01_amd64.deb'.
 +
  Package file now in /home/myuser/dev/gotcloud
 +
  -rw-r--r-- 1 myuser myuser 3592780 Feb 19 11:59 gotcloud-bin_1.01_amd64.deb
 +
 +
#  Add -replace only if rebuilding the same version
 +
'''debian/makedeb.sh  test ='''
 +
  dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2 ...
 +
  dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): -D_FORTIFY_SOURCE=2
 +
    [lines deleted]
 +
  Generate package: cd gotcloud-test; dpkg-buildpackage -b -us -uc -rfakeroot
 +
  ===> Version '1.01 of 'gotcloud-test' will install into 'usr/local/gotcloud'
 +
  install -d debian/tmp/DEBIAN
 +
  cp debian/control debian/postinst debian/postrm debian/tmp/DEBIAN
 +
  install -d debian/tmp/usr/share/doc/gotcloud-test_1.01
 +
  sed -e s/99.99/1.01/g < debian/changelog > debian/changelog.new
 +
    [lines deleted]
 +
  dpkg-deb: building package `gotcloud-test' in `./gotcloud-test_1.01_amd64.deb'.
 +
  Package file now in /home/myuser/dev/gotcloud
 +
  -rw-r--r-- 1 myuser myuser 433269556 Feb 19 13:49 gotcloud-test_1.01_amd64.deb
 
</code>
 
</code>
 +
 +
The deb files will be created in the directory above the debian directory  (e.g. gotcloud)
Pjvh
61

edits

Retrieved from "http://genome.sph.umich.edu/wiki/Special:MobileDiff/5562...13968"

Navigation menu