SeqShop: Variant Calling and Filtering for SNPs Practical, June 2014

Goals of This Session

• What we want to learn
  •  How to generate filtered variant calls for SNPs from BAMs
  • Basic variant call file format (VCF)
  •  How to examine the variants at particular genomic positions
  •  How to evaluate the quality of SNP calls

Login to the seqshop-server Linux Machine

This section will appear redundantly in each session. If you are already logged in or know how to log in to the server, please skip this section

1. Login to the windows machine

• The username/password for the Windows machine should be written on the right-hand monitor

Start xming so you can open external windows on our Linux machine

• Start->Enter "Xming" in the search and select "Xming" from the program list
• Nothing will happen, but Xming was started.
View Screenshot

Open putty

• Start->Enter "putty" in the search and select "PuTTY" from the program list
View Screenshot

Configure PuTTY in the PuTTY Configuration window

• Host Name: seqshop-server.sph.umich.edu
View Screenshot



• Setup to allow you to open external windows:
• In the left pannel: Connection->SSH->X11
• Add a check mark in the box next to Enable X11 forwarding
View Screenshot



• Click Open
• If it prompts about a key, click OK

Enter your provided username & password as provided

You should now be logged into a terminal on the seqshop-server and be able to access the test files.

• If you need another terminal, repeat from step 3.

Login to the seqshop Machine

So you can each run multiple jobs at once, we will have you run on 4 different machines within our seqshop setup.

• You can only access these machines after logging onto seqshop-server

3 users logon to:

ssh -X seqshop1

3 users logon to:

ssh -X seqshop2

2 users logon to:

ssh -X seqshop3

2 users logon to:

ssh -X seqshop4

Setup your run environment

This is the same setup you did for the previous tutorial, but you need to redo it each time you log in. It will setup some environment variables to point you to:

• GotCloud program
• Tutorial input files
• Setup an output directory
  • It will leave your output directory from the previous tutorial in tact.

source /home/mktrost/seqshop/setup.txt

• You won't see any output after running source
  • It silently sets up your environment

View setup.txt

Examining GotCloud SnpCall Input files

Sequnce Alignment Files: BAM Files

Per sample BAM files contain sequence reads that are mapped to positions in the genome.

For a reminder on how to look at/read BAM files, see: SeqShop Aligment: BAM Files

Reference Files

Reference files can be downloaded with GotCloud or from other sources

• See GotCloud: Genetic Reference and Resource Files for more information on downloading/generating reference files

For GotCloud snpcall, you need:

1. Reference genome FASTA file
   • Contains the reference base for each position of each chromosome
     • Used to compare bases in sequence reads to the reference positions they mapped to
     • Identify SNPs
   • Additional information on the FASTA format: http://en.wikipedia.org/wiki/FASTA_format
2. VCF (variant call format) files with chromosomes/positions
   • indel - contains known insertions & deletions to help with filtering
   • omni - used as likely true positives for SVM filtering
   • hapmap - used as likely true positives for SVM filtering and for generating summary statistics
   • dbsnp - used for generating summary statistics

GotCloud Configuration File

We will use the same configuration file as we used yesterday in GotCloud Align.

See SeqShop: Alignment: GotCloud Configuration File for more details

• Note we want to limit snpcall to just chr22 so the configuration already has CHRS = 22 (default was 1-22 & X).

For more information on configuration, see: GotCloud snpcall: Configuration File

• Contains information on how to configure for exome/targeted sequencing

GotCloud BAM Index File

The BAM index file points GotCloud to the BAM files

• Alignment pipeline generates for you
• Tab delimited

Look at the BAM index file the alignment pipeline generated

cat ${OUT}/bam.index

What is the path to the BAM file for sample HG00640?

Answer:
• /home/YourUserName/out/bams/HG00640.recal.bam

The alignment pipeline only processed 4 samples, but for snpcall, we want to run on 62 samples.

• The other 58 samples were already aligned:

ls ${IN}/bams

Look at the BAM index for those BAMs:

less ${IN}/bams/bam.index

Remember, use 'q' to exit out of less

q

Do you notice a difference between this index and yours?

Answer:
• It doesn't have a full path to the BAM file, while your index has /home/...

• That's ok, gotcloud.conf contains the path to those BAMs

We need to add these BAMs to our index

• Append the bam.index from the pre-aligned BAMs to the one you generated from the alignment pipeline

cat ${IN}/bams/bam.index >> ${OUT}/bam.index

• ">>" will append to the file that follows it

Verify your BAM index contains the additional BAMs

 less ${IN}/bams/bam.index

Remember, use 'q' to exit out of less

q

Do you see both sets of BAMs?

Annotated Screenshot:
• If not, let me know

Run GotCloud SnpCall

Now that we have all of our input files, we need just a simple command to run:

${GC}/gotcloud snpcall --conf ${IN}/gotcloud.conf --numjobs 4 --region 22:36000000-37000000

• --numjobs tells GotCloud how many jobs to run in parallel
  • Depends on your system
• --region 22:36000000-37000000
  • The sample files are just a small region of chromosome 22, so to save time, we tell GotCloud to ignore the other regions

Curious if it started running properly? Check out this screenshot:

This should take about 5 minutes to run.

• After about 4 minutes of running, GotCloud snpcall will output some text to the screen. Don't worry, that is expected and is just output from some of the intermediate tools.
• It should end with a line like: Commands finished in 289 secs with no errors reported

If you cancelled GotCloud part way through, just rerun your GotCloud command and it will pick up where it left off.

Examining GotCloud SnpCall Output

Let's look at the output directory:

ls ${OUT}

Do you see any new files or directories?

• View Annotated Screenshot:

Let's look at the vcfs directory:

ls ${OUT}/vcfs

Just a chr22 directory, so look inside of there:

ls ${OUT}/vcfs/chr22

Can you identify the final filtered VCF and the associated summary file?

• Answer & annotated directory listing:

• Filtered VCF (SVM & hard filters): chr22.filtered.vcf.gz
• Summary file: chr22.filtered.sites.vcf.summary

Now, let's look in the split directory for the VCF with just the passing variants:

ls ${OUT}/split/chr22

Which file do you think is the one you want?

• Answer:

• chr22.filtered.PASS.vcf.gz

Filtering Summary Statistics

cat ${OUT}/vcfs/chr22/chr22.filtered.sites.vcf.summary

View Annotated Screenshot

Filtered VCF

Let's look at the filtered sites file.

less -S ${OUT}/vcfs/chr22/chr22.filtered.sites.vcf

• Scroll down until you find some variants.
  • Use space bar to jump a full page
  • Use down arrow to move down one line
• Scroll right: lots of info fields, but no per sample genotype information

What is the first filtered out variant that you find & what filter did it fail?

• Answer:

• It failed SVM filter

Remember, use 'q' to exit out of less

q

Now, let's look at the filtered file with genotypes.

zless -S ${OUT}/vcfs/chr22/chr22.filtered.vcf.gz

• Scroll down until you find some variants.
  • Use space bar to jump a full page
  • Use down arrow to move down one line
• Scroll right until you should see per sample genotype information

• View annotated screenshot:

Passing SNPs

Let's look at the file of just the pass sites:

zless -S ${OUT}/split/chr22/chr22.filtered.PASS.vcf.gz

• Scroll down: they all look like they PASS

Let's check if they are all PASS.

zcat ${OUT}/split/chr22/chr22.filtered.PASS.vcf.gz |grep -v "^#"| cut -f 7| grep -v "PASS"

It will return nothing since there are no non-passing variants in this file.

Want an explanation of this command?

• zcat ...: uncompress the zipped VCF
• '|' : this takes the output of one command and sends it as input to the next
• grep -v "^#" : exclude any lines that start with "#" - headers
• cut -f 7 : extract the FILTER column (the 7th column)
• grep -v "PASS" : exclude any rows that have a "PASS" in the FILTER column

Compare that to the filtered file we looked at before:

zcat ${OUT}/vcfs/chr22/chr22.filtered.vcf.gz |grep -v "^#"| cut -f 7| grep -v "PASS"

Do you see any filters?

• Answer

• Yes
  • It should have scrolled and you should see filters like:
    • INDEL5;SVM
    • INDEL5
    • SVM

GotCloud Genotype Refinement

To improve the quality of the genotypes, we run a genotype refinement pipeline.

This pipeline runs Beagle & thunder.

Genotype Refinement Input

The GotCloud genotype refinement pipeline takes the VCF file of PASS'ing SNPs from snpcall as input.

The bam index and the configuration file we used for GotCloud snpcall will tell GotCloud genotype refinement everything it needs to know, so no new input files need to be prepared.

Running GotCloud Genotype Refinement

Since everything is setup, just run the following command (very similar to snpcall).

${GC}/gotcloud ldrefine --conf ${IN}/gotcloud.conf --numjobs 2 --region 22:36000000-37000000

• Beagle will take about 2-3 minutes to complete
• Thunder will automatically run and will take another 3-4 minutes

Genotype Refinement Output

What's new in the output directory?

Answer
• beagle directory
 : Beagle output
• thunder directory
 : Thunder output
• umake.beagle.conf : Configuration values used for GotCloud beagle
• umake.beagle.Makefile : GNU makefile for commands run as part of GotCloud beagle
• umake.beagle.Makefile.log : Log of the GotCloud beagle run
• umake.thunder.* files : Contain the configuration & steps used in GotCloud thunder

Comparing the refined genotypes to the unrefined genotypes

TBD