Quick NGS All-in-one data processing for Next-Generation Sequencing

Using QuickNGS as a production pipeline


Using the QuickNGS MySQL database

To make the system process your samples, the respective meta data have to be feeded into the QuickNGS database. Please access the Samples section of the database interface at http://<yourwebserver>/quickngs/db and upload a text file with each line containing the absolute path to a FastQ file on your shared file system. For each sample, you next need to specify a human-readable name along with the information whether it was processed in a paired-end run as well as the species and laboratory of origin. The Batch field aims to QC for any batch effects caused by varying processing strategies - you can e.g. fill in an abbreviation of the sequencing center's name, but can also leave the field empty. Use the Sample groups section to specify which samples are supposed to be compared by the standardized analysis. If you have not configured a web server, the MySQL tables NGSSamples, NGSSamplePairs, EnsemblSpecies, Laboratories and Institutions are the ones where the information needs to go into.


Starting an analysis run with QuickNGS

To start the analysis, there is a subfolder of the big data directory which we refer to as the stack. The files in this stack are symbolic links to the FastQ files which are to be processed by the software. If you wish to start an analysis, simply add a symbolic link into the new subdirectory which points to any FastQ file on the shared file system of your cluster environment. If you are working on paired-end sequencing files, add two symbolic links ending with _1.fq and _2.fq:

cd <datadir>/stack/new
ln -s <path_to_data>/*.fq .

If automatic invocation of the workflow has not yet been activated (see below), you need to open an additional shell and run the workflow wrapper script in it:

cd <installdir>/scripts/rnaseq
./start_RNAseq.sh

Depending on the NGS application that is used, you may instead have to run the start_miRNAseq.sh, start_ChIPseq.sh or start_WGS.sh scripts from their respective subdirectories.
If the meta data concerning the samples linked are completely available in the MySQL database, the analysis will end up with a ready-to-deliver analysis report for your collaboration partners. If a web server has been configured, they can access the results by visiting the page http://<yourwebserver>/ quickngs/<login> and following the links to the respective run numbers.


Automated invocation of workflows

The degree of automation can be further increased by using CRON jobs on the workflow scripts. As the complete workflow will be running in the background (i.e., not in an open shell window), this extremely increases the efficiency of the workflow. After metadata specification, the NGS data analysis can then be carried out by simply linking the FastQ files to the stack directory and waiting for the workflow to do its job in the background.


Status monitoring

Though all the analyses are running in the background, you still want to have an overview what the system is doing and what the current status of your analysis is. To this end, the system comprises a dynamic website for status monitoring which can be accessed any time at http://<yourwebserver>/quickngs/db/ status.php.


Maintenance of the local copies of public databases

The analyses within QuickNGS mainly rely on the Ensembl database. If nothing else is specified, the software looks for the most recent Ensembl version and downloads the corresponding data automatically from the Ensembl FTP site and the Ensembl MySQL interface. However, in some cases, it may be required to use older versions of the Ensembl database. Please use the script <installdir>/scripts/public/download_ensembl.sh for this purpose which can download and prepare the genomic information from Ensembl for any species and database version.
The miRNA-Seq analysis workflow relies on the miRBase which is also copied to a local repository. At this point, it is not possible to switch between the releases of miRBase as for Ensembl.


Previous topic: Setting up the server environment
Next topic: Troubleshooting

News

June 6th, 2017: Our paper on the cancer genome analysis platform QuickNGS Cancer has been published in Human Mutation.

September 30th, 2016: Please access the 'Multi-Layer Integration' area to combine RNA-Seq and ChIP-Seq data with an early release of our new multi-OMICS data integration platform.

June 23rd, 2016: Tools for gene set enrichment analysis using GO terms and KEGG pathways have been adopted into QuickNGS from version 1.2.2 on.

February 15th, 2016: The latest QuickNGS release now includes QuickNGS Cancer, a new platform specifically designed for cancer genome analysis.

January 4th, 2016: Bluebee High-Performance Genomics B.V., Delft, The Netherlands, have adopted QuickNGS into their cloud-based NGS analysis solution.

September 23rd, 2015: Please refer to our new FAQ to handle common problems with the QuickNGS results.

July 31th, 2015: The QuickNGS paper was published in BMC Genomics! Please cite this paper for all analyses based on the QuickNGS system.

July 11th, 2014: The first public version of the QuickNGS source code has just been released! Please click here to download the software.