SeqShop: Aligning Your Own Genome, December 2014
Viewing Genetic Info
Slides on risks of viewing genetic information
Goals of This Session
Learn how to go from your FASTQ files to generate Aligned BAMs.
- Your samples have already been aligned, so we will review the steps that were done
- Workshop computers don't have enough compute to align everyone's samples during the workshop
- You will get to take home both the original FASTQs and the aligned BAMs on a USB drive
- You will get it by the end of the week if not before - it takes a while to copy 74G-111G
- 4 samples will get "modified" BAMs so both FASTQ & BAM would fit on the drive
- "binned" qualities, duplicates & unmapped reads removed
- In future all generated BAMs will be automatically binned
- "binned" qualities, duplicates & unmapped reads removed
- Those that didn't get sequenced will get a drive with NA12878 public sample on it
Login instructions for seqshop-server
Login to the seqshop-server Linux Machine
This section will appear redundantly in each session. If you are already logged in or know how to log in to the server, please skip this section
- Login to the windows machine
- The username/password for the Windows machine should be written on the right-hand monitor
- Start xming so you can open external windows on our Linux machine
- Start->Enter "Xming" in the search and select "Xming" from the program list
- Nothing will happen, but Xming was started.
- Open putty
- Start->Enter "putty" in the search and select "PuTTY" from the program list
- Configure PuTTY in the PuTTY Configuration window
- Host Name:
seqshop-server.sph.umich.edu
- Setup to allow you to open external windows:
- In the left pannel: Connection->SSH->X11
- Add a check mark in the box next to
Enable X11 forwarding
- Click
Open
- If it prompts about a key, click
OK
- Enter your provided username & password as provided
You should now be logged into a terminal on the seqshop-server and be able to access the test files.
- If you need another terminal, repeat from step 3.
Login to the seqshop Machine
So you can each run multiple jobs at once, we will have you run on 4 different machines within our seqshop setup.
- You can only access these machines after logging onto seqshop-server
3 users logon to:
ssh -X seqshop1
3 users logon to:
ssh -X seqshop2
2 users logon to:
ssh -X seqshop3
2 users logon to:
ssh -X seqshop4
First Things First
When you see Sample*/Sample#, replace it with your sample name/number
- If you are using generic data, use NA12878
Locating your FASTQs
Your FASTQ files are under ~/Sample*/fastqs
directory.
Look at your directory:
ls ~/Sample*/fastqs
ls
does a directory listing~/
means to start from your home/base directory- Using ~/ means the command will work even if you have changed directories
You will see 2 files:
- Sample#_R1.fastq.gz - 1st in pair
- Sample#_R2.fastq.gz - 2nd in pair
What did I do to run the alignment?
Created FASTQ_LIST file
I created a FASTQ_LIST file with the sample #s and fastqs.
- What columns do we need in our file that tells GotCloud about our FASTQ?
- SAMPLE
- FASTQ1
- FASTQ2
SAMPLE FASTQ1 FASTQ2 Sample# /path/to/Sample#/fastqs/Sample#_R1.fastq.gz /path/to/Sample#/fastqs/Sample#_R2.fastq.gz
You can see what your fastq.list looks like
- I took out the full path since I ran alignment in a different path:
cat ~/Sample*/fastq.list
Created a GotCloud Configuration File
After creating the FASTQ_LIST file, I created the GotCloud configuration.
- What changes from the default settings did I make?
- Use BWA instead of BWA_MEM
- Use multiple BWA_THREADS
- Set FASTQ_LIST
Look at the gotcloud.conf I setup for you
- Not "exactly" what I used. In the original gotcloud.conf, I had
- cluster settings (now blank since you won't be using a cluster for further processing)
- various number of BWA_THREADS for each sample
- full paths to:
- FASTQ_LIST (and I sometimes had multiple samples in 1 list - but each gets aligned independently)
- OUT_DIR
cat ~/Sample*/gotcloud.conf
You'll notice that this file is very similar to the one we have been using.
- Just a few modifications to run a new test on the whole genome
The settings needed for Single Sample SNP calling that we need for tomorrow are already in the gotcloud.conf (requires extra settings as the default snpcall works best for multiple samples).
Ran the Alignment
It took many threads & a couple of days to get all of the alignments complete - which is why I ran them last week.
- Used screen to run overnight.
I ran something like:
gotcloud align --conf gotcloud.conf --numjobs 4
- I set numjobs to the number of samples I was processing on that machine
Alignment Pipeline Output
FEEDBACK!
Please provide feedback on the lectures/tutorials from today.