Difference between revisions of "Tutorial: RAREMETAL"

From Genome Analysis Wiki
Jump to navigationJump to search
 
(32 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
[[Category:RAREMETAL]]
 +
[[Category:RAREMETALWORKER]]
 
== Useful Wiki Pages ==
 
== Useful Wiki Pages ==
  
There are a few pages in this Wiki that may be useful to RAREMETAL users. Here are links to a few:
+
There are several pages in this Wiki that may be useful to RAREMETAL users. Here are links to a few:
  
 
* The [[RAREMETAL|RAREMETAL Home Page]]
 
* The [[RAREMETAL|RAREMETAL Home Page]]
Line 7: Line 9:
 
* The [[RAREMETAL Command Reference]]
 
* The [[RAREMETAL Command Reference]]
  
* The [[RAREMETAL|RAREMETAL Documentation]]
+
* The [[RAREMETAL_Documentation|RAREMETAL Documentation]]
  
 
* The [[RAREMETAL FAQ]]
 
* The [[RAREMETAL FAQ]]
 +
 +
* The [[RAREMETALWORKER | RAREMETALWORKER documentation]]
  
 
==Introduction==
 
==Introduction==
  
In this tutorial, we will use RareMetalWorker and RareMetal to perform a simple rare variant meta analysis. [[Rare-Metal-Worker|'''rareMetalWorker''']] is a tool that generates summary statistics that can be shared to enable meta-analysis of gene-level association tests. [[rareMETAL|'''rareMETAL''']] uses the files generated by RareMetalWorker as input and perform both single variant and gene-level association meta-analysis.
+
In this tutorial, we will use [[RAREMETAL|'''RAREMETAL''']] to perform single variant and gene-level meta-analysis using summary statistics of two small studies. We will use [[RAREMETALWORKER|'''RAREMETALWORKER''']] to generate summary statistics for each study.
  
 
==STEP 1: Install Software and Download Example Data Sets==
 
==STEP 1: Install Software and Download Example Data Sets==
  
* If RAREMETAL and RAREMETALWORKER have not been installed on your local computer yet, that is the first step! Installation instructions for RAREMETAL and RAREMETALWORKER are
+
* If RAREMETAL and RAREMETALWORKER have not been installed on your local computer yet, you will first need to install them! You can download the two from here,  [[RAREMETALWORKER#Software_Download_and_Installation|'''RAREMETAL installation''']] and [[RAREMETAL#Software_Download_and_Installation|'''RAREMETALWORKER installation''']].  
available ([[Rare-Metal-Worker#Software_Download_and_Installation|RAREMETAL]] and [[Rare-Metal#Software_Download_and_Installation|RAREMETALWORKER]]).
 
  
* Then please download the [[Media:Raremetal_tutorial.tar.gz|'''tutorial package''']], including example data sets and results.  
+
* Then please download the [[Media:Raremetal_tutorial.tar.gz|'''tutorial package''']], including example data sets.  
  
** To unpack the example dataset, you can use the following two Unix commands:
+
* To unpack the example dataset, use the following two Unix commands:
  
 
   tar xvzf raremetal_tutorial.tar.gz  
 
   tar xvzf raremetal_tutorial.tar.gz  
Line 29: Line 32:
 
==STEP 2: Analyze individual samples using RAREMETALWORKER==
 
==STEP 2: Analyze individual samples using RAREMETALWORKER==
  
* The first example has 743 individuals coded as unrelated according to PED file (each person belongs to an individual family).
+
* Our first dataset has 743 unrelated individuals. Their phenotypes and relationships are described in a pair of pedigree and data files; you can get a summary of their contents using pedstats.
* there are ~1000 markers included in the VCF file.  
+
 
* To analyze this sample accounting for sample relatedness, an empirical kinship should be calculated.  
+
* These individuals have been genotyped at ~1000 markers, and these genotypes are stored in a VCF file. The VCF is a text format, so you can try to peek at the contents if you like.
* Go to $yourPath/bin/ and execute the following command:  
+
 +
* Although these individuals are "unrelated", there is the possibility that some of them are more closely related than others. In our analysis, we will calculate an empirical (i.e. data-driven) kinship (i.e. relatedness) matrix to describe the similarity between individuals.
 +
 
 +
* To analyse the first study, we execute the following command:  
 +
 
 +
  raremetalworker  --ped example1.ped --dat example1.dat --vcf example1.vcf.gz --traitName QT1 \
 +
                  --inverseNormal --makeResiduals --kinSave --kinGeno --prefix STUDY1
  
  cd raremetalworker/inputfiles
+
This command transforms phenotypes to normality (--inverseNormal), calculates trait residuals after adjusting for covariates (--makeResiduals), estimates relatedness between individuals and saves this for later use (--kinSave and --kinGeno), and even generates some PDF files summarizing results.
  
  $yourpathforRAREMETALWORKER/bin/raremetalworker  --ped example1.ped --dat example1.dat --vcf example1.vcf.gz
+
* After RAREMETALWORKER finishes running, you will see several output files:
                                --traitName QT1 --inverseNormal --makeResiduals --kinSave --kinGeno --prefix ../yourprefix.example1
 
  
* The command above allows you to estimate relatedness according to common variants genotypes with good variant calling quality, adjust for covariates, quantile normalize the residuals before further association analysis of trait QT1. The following output are generated:
+
  STUDY1.QT1.singlevar.score.txt     ## single variant statistics
  raremetalworker/yourprefix.example1.QT1.singlevar.score.txt (## summary statistics, single variant association results, QC statistics, Genomic control, etc.)
+
  STUDY1.QT1.singlevar.cov.txt       ## covariance matrices between score statistics
  raremetalworker/yourprefix.example1.QT1.singlevar.cov.txt (## covariance matrices of score statistics)
+
  STUDY1.plots.pdf                   ## QQ plots and Manhattan plots
  raremetalworker/yourprefix.example1.plots.pdf (## QQ plots and manhattan plots)
+
  STUDY1.Empirical.Kinship.gz       ## Relatedness matrix
  raremetalworker/yourprefix.example1.Empirical.Kinship.gz (##contains empirical kinship of all individuals who have at least one site genotyped, with id on top row.)
+
  STUDY1.singlevar.log               ## Log file
  raremetalworker/yourprefix.example1.singlevar.log
 
  
* Following the same strategy, example2 can also be analyzed using a similar command:
+
* We next analyse the second study using a similar command:
  
   $yourpathforRAREMETALWORKER/bin/raremetalworker  --ped example2.ped --dat example2.dat --vcf example2.vcf.gz
+
   raremetalworker  --ped example2.ped --dat example2.dat --vcf example2.vcf.gz --traitName QT1 \
                                --traitName QT1 --inverseNormal --makeResiduals --kinSave --kinGeno --prefix yourprefix.example2
+
                    --inverseNormal --makeResiduals --kinSave --kinGeno --prefix STUDY2
  
 
==STEP 3: Run RAREMETAL for Meta-Analysis==
 
==STEP 3: Run RAREMETAL for Meta-Analysis==
  
* In this step, we run RAREMETAL to meta-analyze two studies without using any of raw data.
+
* In this step, we will use RAREMETAL to combine the summary statistics we just generated.
* Before doing analysis, open raremeta/studyname.list and modify the prefix with the right path. The file should look like
+
 
   $yourpath/raremetal_tutorial/raremetalworker/output/example1.QT1
+
* We will first index the result files generated by RAREMETAL. This step relies on bgzip and tabix, two tools that allow rapid indexing and retrieval of results from compressed text files.
   $yourpath/raremetal_tutorial/raremetalworker/output/example1.QT2
+
 
Then prepare the raremetalworker results for meta-analysis using the following command:
+
  bgzip STUDY1.QT1.singlevar.score.txt
   bgzip yourprefix.example1.QT1.singlevar.score.txt
+
  tabix -c "#" -s 1 -b 2 -e 2 STUDY1.QT1.singlevar.score.txt.gz
   tabix -c "#" -s 1 -b 2 -e 2 yourprefix.example1.QT1.singlevar.score.txt.gz
+
   bgzip STUDY1.QT1.singlevar.cov.txt
   bgzip yourprefix.example1.QT1.singlevar.cov.txt
+
   tabix -c "#" -s 1 -b 2 -e 2 STUDY1.QT1.singlevar.cov.txt.gz
   tabix -c "#" -s 1 -b 2 -e 2 yourprefix.example1.QT1.singlevar.cov.txt.gz
+
 
 +
* And again for the second study:
 +
 
 +
   bgzip STUDY2.QT1.singlevar.score.txt
 +
   tabix -c "#" -s 1 -b 2 -e 2 STUDY2.QT1.singlevar.score.txt.gz
 +
   bgzip STUDY2.QT1.singlevar.cov.txt
 +
   tabix -c "#" -s 1 -b 2 -e 2 STUDY2.QT1.singlevar.cov.txt.gz
 +
 
 +
* We next create two text files that will drive the meta-analysis. The first file lists the input files with summary statistics. Let's call it summaryfiles. In most Linux workstations, you can use the command pico or nano to create this file. These should be the contents of  "summaryfiles":
 +
 
 +
  STUDY1.QT1.singlevar.score.txt.gz
 +
  STUDY2.QT1.singlevar.score.txt.gz
 +
 
 +
* The second file lists the input files with variance-covariance information between markers. Let's call it covfiles. These should be its contents:
 +
 
 +
  STUDY1.QT1.singlevar.cov.txt.gz
 +
  STUDY2.QT1.singlevar.cov.txt.gz
 +
 
 +
* Now, we are ready for meta-analysis. To perform single variant and gene-level meta-analyses all at once, use the following command:
 +
 
 +
  raremetal --summaryFiles summaryfiles --covFiles covfiles --groupFile group.file \
 +
            --SKAT --burden --MB --VT --longOutput --tabulateHits --hitsCutoff 1e-05 \
 +
            --prefix COMBINED.QT1 --hwe 1.0e-05 --callRate 0.95 -
 +
 
 +
This command filters summary statistics based on HWE p-value and variant call rate, generates single variant meta-analysis results, generates gene-level meta-analysis results using simple burden test, variable threshold test, Madson-Browning weighted burden test, and SKAT, tabulates significant genes with detailed single variant results included, and even generates some PDF files summarizing results.
 +
 
 +
* The following output will be generated:
 +
 
 +
  COMBINED.QT1.meta.plots.pdf (## QQ plots and manhattan plots)
 +
  COMBINED.QT1.meta.singlevar.results
 +
  COMBINED.QT1.meta.burden.results
 +
  COMBINED.QT1.meta.SKAT.results
 +
  COMBINED.QT1.meta.VT.results
 +
  COMBINED.QT1.meta.MB.results
 +
  COMBINED.QT1.meta.tophits.SKAT.tbl
 +
  COMBINED.QT1.meta.tophits.VT.tbl
 +
  COMBINED.QT1.meta.tophits.burden.tbl
 +
  COMBINED.QT1.meta.tophits.MB.tbl
 +
  COMBINED.QT1.raremetal.log
 +
 
 +
* It is probably a good idea to spend a few minutes reviewing these files - they have the key results for your analysis! A detailed description of output files is available  [[RAREMETAL_Documentation#Gene-level_Tests_Meta-Analysis_Output|elsewhere]].
 +
 
 +
* If you are feeling adventurous and are not yet totally confused, you can continue to explore advanced features of RAREMETAL.
 +
 
 +
* For example, RAREMETAL can carry out conditional analyses using the summary statistics in summaryfiles and covfiles.
 +
 
 +
* To carry out a conditional analysis, create a text file that specifies the variant that you want to condition on (lets call it "conditioningfile") and add "--condition conditioningfile" to the command line. The "conditioningfile" might look like this:
  
* Now, we are ready for meta-analysis. To perform single variant and four type of gene-level meta-analysis all at once, use the following command line:
+
  9:505484545:C:T
  cd raremetal
 
  $yourRAREMTALpath/bin/raremetal --studyName studyname.list --groupFile group.file --SKAT --burden --MB --VT
 
                                  --hwe 1.0e-05 --callRate 0.95 (# QC options)
 
                                  --longOutput --tabulateHits --hitsCutoff 1e-05 --prefix yourprefix.QT1 (#output options)
 
                                  --mapFile ../refFlat_hg19.txt.gz
 
  
* The following output are generated
+
* When you now run RAREMETAL with the extra options --condition conditioningfile, results will be adjusted for the variant 9:505484545:C:T.
  yourprefix.meta.plots.pdf (## QQ plots and manhattan plots from both single variant and gene-level meta-analysis with hits labeled)
 
  yourprefix.meta.singlevar.results
 
  yourprefix.meta.burden.results
 
  yourprefix.meta.SKAT.results
 
  yourprefix.meta.VT.results
 
  yourprefix.meta.MB.results
 
  yourprefix.meta.tophits.SKAT.tbl
 
  yourprefix.meta.tophits.VT.tbl
 
  yourprefix.meta.tophits.burden.tbl
 
  yourprefix.meta.tophits.MB.tbl
 
  yourprefix.raremetal.log
 
* Please refer to the RAREMETAL [[RAREMETAL_Documentation#Gene-level_Tests_Meta-Analysis_Output|'''documentation''']] for detailed description of output format.
 
* RAREMETAL also takes annotated VCF as input to parse variant grouping information. Please refer to software [[RAREMETAL_Documentation#Group_Rare_Variants_from_Annotated_VCF|'''documentation''']] for details.
 
  
* RAREMETAL allows users to output an VCF file of the super set of all variants and use their favorite annotation tool to annotate it and then come back to RAREMETAL for the gene-level meta analysis. [[RAREMETAL_Documentation#Generate_a_VCF_File_to_Annotate_Outside_of_Rare_Metal|'''--writeVCF''']] is the option to use.
+
* An alternative to generating a group.file is to provide RAREMETAL with an annotated VCF as input. For details, see  [[RAREMETAL_Documentation#Group_Rare_Variants_from_Annotated_VCF|documentation]]. This option is especially useful when you first use RAREMETAL's [[RAREMETAL_Documentation#Generate_a_VCF_File_to_Annotate_Outside_of_Rare_Metal|'''--writeVCF''']] option to create a file listing a superset of all available variants.

Latest revision as of 17:51, 16 March 2018

Useful Wiki Pages

There are several pages in this Wiki that may be useful to RAREMETAL users. Here are links to a few:

Introduction

In this tutorial, we will use RAREMETAL to perform single variant and gene-level meta-analysis using summary statistics of two small studies. We will use RAREMETALWORKER to generate summary statistics for each study.

STEP 1: Install Software and Download Example Data Sets

  • To unpack the example dataset, use the following two Unix commands:
 tar xvzf raremetal_tutorial.tar.gz 
 cd raremetal_tutorial

STEP 2: Analyze individual samples using RAREMETALWORKER

  • Our first dataset has 743 unrelated individuals. Their phenotypes and relationships are described in a pair of pedigree and data files; you can get a summary of their contents using pedstats.
  • These individuals have been genotyped at ~1000 markers, and these genotypes are stored in a VCF file. The VCF is a text format, so you can try to peek at the contents if you like.
  • Although these individuals are "unrelated", there is the possibility that some of them are more closely related than others. In our analysis, we will calculate an empirical (i.e. data-driven) kinship (i.e. relatedness) matrix to describe the similarity between individuals.
  • To analyse the first study, we execute the following command:
 raremetalworker  --ped example1.ped --dat example1.dat --vcf example1.vcf.gz --traitName QT1 \
                  --inverseNormal --makeResiduals --kinSave --kinGeno --prefix STUDY1 

This command transforms phenotypes to normality (--inverseNormal), calculates trait residuals after adjusting for covariates (--makeResiduals), estimates relatedness between individuals and saves this for later use (--kinSave and --kinGeno), and even generates some PDF files summarizing results.

  • After RAREMETALWORKER finishes running, you will see several output files:
STUDY1.QT1.singlevar.score.txt     ## single variant statistics
STUDY1.QT1.singlevar.cov.txt       ## covariance matrices between score statistics
STUDY1.plots.pdf                   ## QQ plots and Manhattan plots
STUDY1.Empirical.Kinship.gz        ## Relatedness matrix
STUDY1.singlevar.log               ## Log file
  • We next analyse the second study using a similar command:
  raremetalworker  --ped example2.ped --dat example2.dat --vcf example2.vcf.gz --traitName QT1 \
                   --inverseNormal --makeResiduals --kinSave --kinGeno --prefix STUDY2

STEP 3: Run RAREMETAL for Meta-Analysis

  • In this step, we will use RAREMETAL to combine the summary statistics we just generated.
  • We will first index the result files generated by RAREMETAL. This step relies on bgzip and tabix, two tools that allow rapid indexing and retrieval of results from compressed text files.
 bgzip STUDY1.QT1.singlevar.score.txt
 tabix -c "#" -s 1 -b 2 -e 2 STUDY1.QT1.singlevar.score.txt.gz
 bgzip STUDY1.QT1.singlevar.cov.txt
 tabix -c "#" -s 1 -b 2 -e 2 STUDY1.QT1.singlevar.cov.txt.gz
  • And again for the second study:
 bgzip STUDY2.QT1.singlevar.score.txt
 tabix -c "#" -s 1 -b 2 -e 2 STUDY2.QT1.singlevar.score.txt.gz
 bgzip STUDY2.QT1.singlevar.cov.txt
 tabix -c "#" -s 1 -b 2 -e 2 STUDY2.QT1.singlevar.cov.txt.gz
  • We next create two text files that will drive the meta-analysis. The first file lists the input files with summary statistics. Let's call it summaryfiles. In most Linux workstations, you can use the command pico or nano to create this file. These should be the contents of "summaryfiles":
 STUDY1.QT1.singlevar.score.txt.gz
 STUDY2.QT1.singlevar.score.txt.gz
  • The second file lists the input files with variance-covariance information between markers. Let's call it covfiles. These should be its contents:
 STUDY1.QT1.singlevar.cov.txt.gz
 STUDY2.QT1.singlevar.cov.txt.gz
  • Now, we are ready for meta-analysis. To perform single variant and gene-level meta-analyses all at once, use the following command:
 raremetal --summaryFiles summaryfiles --covFiles covfiles --groupFile group.file \
           --SKAT --burden --MB --VT --longOutput --tabulateHits --hitsCutoff 1e-05 \
           --prefix COMBINED.QT1 --hwe 1.0e-05 --callRate 0.95 -

This command filters summary statistics based on HWE p-value and variant call rate, generates single variant meta-analysis results, generates gene-level meta-analysis results using simple burden test, variable threshold test, Madson-Browning weighted burden test, and SKAT, tabulates significant genes with detailed single variant results included, and even generates some PDF files summarizing results.

  • The following output will be generated:
 COMBINED.QT1.meta.plots.pdf (## QQ plots and manhattan plots)
 COMBINED.QT1.meta.singlevar.results 
 COMBINED.QT1.meta.burden.results
 COMBINED.QT1.meta.SKAT.results
 COMBINED.QT1.meta.VT.results
 COMBINED.QT1.meta.MB.results
 COMBINED.QT1.meta.tophits.SKAT.tbl
 COMBINED.QT1.meta.tophits.VT.tbl
 COMBINED.QT1.meta.tophits.burden.tbl
 COMBINED.QT1.meta.tophits.MB.tbl
 COMBINED.QT1.raremetal.log
  • It is probably a good idea to spend a few minutes reviewing these files - they have the key results for your analysis! A detailed description of output files is available elsewhere.
  • If you are feeling adventurous and are not yet totally confused, you can continue to explore advanced features of RAREMETAL.
  • For example, RAREMETAL can carry out conditional analyses using the summary statistics in summaryfiles and covfiles.
  • To carry out a conditional analysis, create a text file that specifies the variant that you want to condition on (lets call it "conditioningfile") and add "--condition conditioningfile" to the command line. The "conditioningfile" might look like this:
 9:505484545:C:T
  • When you now run RAREMETAL with the extra options --condition conditioningfile, results will be adjusted for the variant 9:505484545:C:T.
  • An alternative to generating a group.file is to provide RAREMETAL with an annotated VCF as input. For details, see documentation. This option is especially useful when you first use RAREMETAL's --writeVCF option to create a file listing a superset of all available variants.