Conveyor: a workflow engine for bioinformatic analyses
Linke B, Giegerich R, Goesmann A (2011)
Bioinformatics 27(7): 903-911.
Zeitschriftenaufsatz
| Veröffentlicht | Englisch
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Einrichtung
Centrum für Biotechnologie > Arbeitsgruppe A. Goesmann
Technische Fakultät > Computational Genomics
Centrum für Biotechnologie > Arbeitsgruppe R. Giegerich
Technische Fakultät > AG Praktische Informatik
Centrum für Biotechnologie > Technologieplattformen > Bioinformatics Resource Facility
Centrum für Biotechnologie > Institut für Bioinformatik
Technische Fakultät > Computational Genomics
Centrum für Biotechnologie > Arbeitsgruppe R. Giegerich
Technische Fakultät > AG Praktische Informatik
Centrum für Biotechnologie > Technologieplattformen > Bioinformatics Resource Facility
Centrum für Biotechnologie > Institut für Bioinformatik
Abstract / Bemerkung
Motivation: The rapidly increasing amounts of data available from new high-throughput methods have made data processing without automated pipelines infeasible. As was pointed out in several publications, integration of data and analytic resources into workflow systems provides a solution to this problem, simplifying the task of data analysis. Various applications for defining and running workflows in the field of bioinformatics have been proposed and published, e. g. Galaxy, Mobyle, Taverna, Pegasus or Kepler. One of the main aims of such workflow systems is to enable scientists to focus on analysing their datasets instead of taking care for data management, job management or monitoring the execution of computational tasks. The currently available workflow systems achieve this goal, but fundamentally differ in their way of executing workflows.
Results: We have developed the Conveyor software library, a multitiered generic workflow engine for composition, execution and monitoring of complex workflows. It features an open, extensible system architecture and concurrent program execution to exploit resources available on modern multicore CPU hardware. It offers the ability to build complex workflows with branches, loops and other control structures. Two example use cases illustrate the application of the versatile Conveyor engine to common bioinformatics problems.
Erscheinungsjahr
2011
Zeitschriftentitel
Bioinformatics
Band
27
Ausgabe
7
Seite(n)
903-911
ISSN
1367-4803
eISSN
1460-2059
Page URI
https://pub.uni-bielefeld.de/record/2093578
Zitieren
Linke B, Giegerich R, Goesmann A. Conveyor: a workflow engine for bioinformatic analyses. Bioinformatics. 2011;27(7):903-911.
Linke, B., Giegerich, R., & Goesmann, A. (2011). Conveyor: a workflow engine for bioinformatic analyses. Bioinformatics, 27(7), 903-911. https://doi.org/10.1093/bioinformatics/btr040
Linke, Burkhard, Giegerich, Robert, and Goesmann, Alexander. 2011. “Conveyor: a workflow engine for bioinformatic analyses”. Bioinformatics 27 (7): 903-911.
Linke, B., Giegerich, R., and Goesmann, A. (2011). Conveyor: a workflow engine for bioinformatic analyses. Bioinformatics 27, 903-911.
Linke, B., Giegerich, R., & Goesmann, A., 2011. Conveyor: a workflow engine for bioinformatic analyses. Bioinformatics, 27(7), p 903-911.
B. Linke, R. Giegerich, and A. Goesmann, “Conveyor: a workflow engine for bioinformatic analyses”, Bioinformatics, vol. 27, 2011, pp. 903-911.
Linke, B., Giegerich, R., Goesmann, A.: Conveyor: a workflow engine for bioinformatic analyses. Bioinformatics. 27, 903-911 (2011).
Linke, Burkhard, Giegerich, Robert, and Goesmann, Alexander. “Conveyor: a workflow engine for bioinformatic analyses”. Bioinformatics 27.7 (2011): 903-911.
18 Zitationen in Europe PMC
Daten bereitgestellt von Europe PubMed Central.
Comparative genomics and pathogenicity potential of members of the Pseudomonas syringae species complex on Prunus spp.
Ruinelli M, Blom J, Smits THM, Pothier JF., BMC Genomics 20(1), 2019
PMID: 30836956
Ruinelli M, Blom J, Smits THM, Pothier JF., BMC Genomics 20(1), 2019
PMID: 30836956
Temporal variation in brain transcriptome is associated with the expression of female mimicry as a sequential male alternative reproductive tactic in fish.
Cardoso SD, Gonçalves D, Goesmann A, Canário AVM, Oliveira RF., Mol Ecol 27(3), 2018
PMID: 29110358
Cardoso SD, Gonçalves D, Goesmann A, Canário AVM, Oliveira RF., Mol Ecol 27(3), 2018
PMID: 29110358
Flexible metagenome analysis using the MGX framework.
Jaenicke S, Albaum SP, Blumenkamp P, Linke B, Stoye J, Goesmann A., Microbiome 6(1), 2018
PMID: 29690922
Jaenicke S, Albaum SP, Blumenkamp P, Linke B, Stoye J, Goesmann A., Microbiome 6(1), 2018
PMID: 29690922
Comparative genomics of host adaptive traits in Xanthomonas translucens pv. graminis.
Hersemann L, Wibberg D, Blom J, Goesmann A, Widmer F, Vorhölter FJ, Kölliker R., BMC Genomics 18(1), 2017
PMID: 28056815
Hersemann L, Wibberg D, Blom J, Goesmann A, Widmer F, Vorhölter FJ, Kölliker R., BMC Genomics 18(1), 2017
PMID: 28056815
First complete genome sequence of Bacillus glycinifermentans B-27.
Stadermann KB, Blom J, Borgmeier C, Sciberras N, Herbold S, Kipker M, Meurer G, Molck S, Petri D, Pelzer S, Schneider J., J Biotechnol 257(), 2017
PMID: 28438580
Stadermann KB, Blom J, Borgmeier C, Sciberras N, Herbold S, Kipker M, Meurer G, Molck S, Petri D, Pelzer S, Schneider J., J Biotechnol 257(), 2017
PMID: 28438580
Systems and synthetic biology perspective of the versatile plant-pathogenic and polysaccharide-producing bacterium Xanthomonas campestris.
Schatschneider S, Schneider J, Blom J, Létisse F, Niehaus K, Goesmann A, Vorhölter FJ., Microbiology 163(8), 2017
PMID: 28795660
Schatschneider S, Schneider J, Blom J, Létisse F, Niehaus K, Goesmann A, Vorhölter FJ., Microbiology 163(8), 2017
PMID: 28795660
Proteorhodopsin light-enhanced growth linked to vitamin-B1 acquisition in marine Flavobacteria.
Gómez-Consarnau L, González JM, Riedel T, Jaenicke S, Wagner-Döbler I, Sañudo-Wilhelmy SA, Fuhrman JA., ISME J 10(5), 2016
PMID: 26574687
Gómez-Consarnau L, González JM, Riedel T, Jaenicke S, Wagner-Döbler I, Sañudo-Wilhelmy SA, Fuhrman JA., ISME J 10(5), 2016
PMID: 26574687
Pan-genome analysis of Aeromonas hydrophila, Aeromonas veronii and Aeromonas caviae indicates phylogenomic diversity and greater pathogenic potential for Aeromonas hydrophila.
Ghatak S, Blom J, Das S, Sanjukta R, Puro K, Mawlong M, Shakuntala I, Sen A, Goesmann A, Kumar A, Ngachan SV., Antonie Van Leeuwenhoek 109(7), 2016
PMID: 27075453
Ghatak S, Blom J, Das S, Sanjukta R, Puro K, Mawlong M, Shakuntala I, Sen A, Goesmann A, Kumar A, Ngachan SV., Antonie Van Leeuwenhoek 109(7), 2016
PMID: 27075453
Experiences with workflows for automating data-intensive bioinformatics.
Spjuth O, Bongcam-Rudloff E, Hernández GC, Forer L, Giovacchini M, Guimera RV, Kallio A, Korpelainen E, Kańduła MM, Krachunov M, Kreil DP, Kulev O, Łabaj PP, Lampa S, Pireddu L, Schönherr S, Siretskiy A, Vassilev D., Biol Direct 10(), 2015
PMID: 26282399
Spjuth O, Bongcam-Rudloff E, Hernández GC, Forer L, Giovacchini M, Guimera RV, Kallio A, Korpelainen E, Kańduła MM, Krachunov M, Kreil DP, Kulev O, Łabaj PP, Lampa S, Pireddu L, Schönherr S, Siretskiy A, Vassilev D., Biol Direct 10(), 2015
PMID: 26282399
High-Performance Integrated Virtual Environment (HIVE) Tools and Applications for Big Data Analysis.
Simonyan V, Mazumder R., Genes (Basel) 5(4), 2014
PMID: 25271953
Simonyan V, Mazumder R., Genes (Basel) 5(4), 2014
PMID: 25271953
Streaming support for data intensive cloud-based sequence analysis.
Issa SA, Kienzler R, El-Kalioby M, Tonellato PJ, Wall D, Bruggmann R, Abouelhoda M., Biomed Res Int 2013(), 2013
PMID: 23710461
Issa SA, Kienzler R, El-Kalioby M, Tonellato PJ, Wall D, Bruggmann R, Abouelhoda M., Biomed Res Int 2013(), 2013
PMID: 23710461
Bioinformatic pipelines in Python with Leaf.
Napolitano F, Mariani-Costantini R, Tagliaferri R., BMC Bioinformatics 14(), 2013
PMID: 23786315
Napolitano F, Mariani-Costantini R, Tagliaferri R., BMC Bioinformatics 14(), 2013
PMID: 23786315
Tavaxy: integrating Taverna and Galaxy workflows with cloud computing support.
Abouelhoda M, Issa SA, Ghanem M., BMC Bioinformatics 13(), 2012
PMID: 22559942
Abouelhoda M, Issa SA, Ghanem M., BMC Bioinformatics 13(), 2012
PMID: 22559942
Bacterial community shift in treated periodontitis patients revealed by ion torrent 16S rRNA gene amplicon sequencing.
Jünemann S, Prior K, Szczepanowski R, Harks I, Ehmke B, Goesmann A, Stoye J, Harmsen D., PLoS One 7(8), 2012
PMID: 22870235
Jünemann S, Prior K, Szczepanowski R, Harks I, Ehmke B, Goesmann A, Stoye J, Harmsen D., PLoS One 7(8), 2012
PMID: 22870235
The Wasp System: an open source environment for managing and analyzing genomic data.
McLellan AS, Dubin RA, Jing Q, Broin PÓ, Moskowitz D, Suzuki M, Calder RB, Hargitai J, Golden A, Greally JM., Genomics 100(6), 2012
PMID: 22944616
McLellan AS, Dubin RA, Jing Q, Broin PÓ, Moskowitz D, Suzuki M, Calder RB, Hargitai J, Golden A, Greally JM., Genomics 100(6), 2012
PMID: 22944616
A proposal for augmenting biological model construction with a semi-intelligent computational modeling assistant.
Christley S, An G., Comput Math Organ Theory 18(4), 2012
PMID: 23990750
Christley S, An G., Comput Math Organ Theory 18(4), 2012
PMID: 23990750
Personalized cloud-based bioinformatics services for research and education: use cases and the elasticHPC package.
El-Kalioby M, Abouelhoda M, Krüger J, Giegerich R, Sczyrba A, Wall DP, Tonellato P., BMC Bioinformatics 13 Suppl 17(), 2012
PMID: 23281941
El-Kalioby M, Abouelhoda M, Krüger J, Giegerich R, Sczyrba A, Wall DP, Tonellato P., BMC Bioinformatics 13 Suppl 17(), 2012
PMID: 23281941
Extending KNIME for next-generation sequencing data analysis.
Jagla B, Wiswedel B, Coppée JY., Bioinformatics 27(20), 2011
PMID: 21873641
Jagla B, Wiswedel B, Coppée JY., Bioinformatics 27(20), 2011
PMID: 21873641
20 References
Daten bereitgestellt von Europe PubMed Central.
Kepler: an extensible system for design and execution of scientific workflows
Altintas, 2004
Altintas, 2004
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs.
Altschul SF, Madden TL, Schaffer AA, Zhang J, Zhang Z, Miller W, Lipman DJ., Nucleic Acids Res. 25(17), 1997
PMID: 9254694
Altschul SF, Madden TL, Schaffer AA, Zhang J, Zhang Z, Miller W, Lipman DJ., Nucleic Acids Res. 25(17), 1997
PMID: 9254694
The Universal Protein Resource (UniProt) in 2010.
UniProt Consortium, Apweiler R, Martin MJ, O'Donovan C, Magrane M, Alam-Faruque Y, Antunes R, Barrell D, Bely B, Bingley M, Binns D, Bower L, Browne P, Chan WM, Dimmer E, Eberhardt R, Fedotov A, Foulger R, Garavelli J, Huntley R, Jacobsen J, Kleen M, Laiho K, Leinonen R, Legge D, Lin Q, Liu W, Luo J, Orchard S, Patient S, Poggioli D, Pruess M, Corbett M, di Martino G, Donnelly M, van Rensburg P, Bairoch A, Bougueleret L, Xenarios I, Altairac S, Auchincloss A, Argoud-Puy G, Axelsen K, Baratin D, Blatter MC, Boeckmann B, Bolleman J, Bollondi L, Boutet E, Quintaje SB, Breuza L, Bridge A, deCastro E, Ciapina L, Coral D, Coudert E, Cusin I, Delbard G, Doche M, Dornevil D, Roggli PD, Duvaud S, Estreicher A, Famiglietti L, Feuermann M, Gehant S, Farriol-Mathis N, Ferro S, Gasteiger E, Gateau A, Gerritsen V, Gos A, Gruaz-Gumowski N, Hinz U, Hulo C, Hulo N, James J, Jimenez S, Jungo F, Kappler T, Keller G, Lachaize C, Lane-Guermonprez L, Langendijk-Genevaux P, Lara V, Lemercier P, Lieberherr D, de Oliveira Lima T, Mangold V, Martin X, Masson P, Moinat M, Morgat A, Mottaz A, Paesano S, Pedruzzi I, Pilbout S, Pillet V, Poux S, Pozzato M, Redaschi N, Rivoire C, Roechert B, Schneider M, Sigrist C, Sonesson K, Staehli S, Stanley E, Stutz A, Sundaram S, Tognolli M, Verbregue L, Veuthey AL, Yip L, Zuletta L, Wu C, Arighi C, Arminski L, Barker W, Chen C, Chen Y, Hu ZZ, Huang H, Mazumder R, McGarvey P, Natale DA, Nchoutmboube J, Petrova N, Subramanian N, Suzek BE, Ugochukwu U, Vasudevan S, Vinayaka CR, Yeh LS, Zhang J., Nucleic Acids Res. 38(Database issue), 2009
PMID: 19843607
UniProt Consortium, Apweiler R, Martin MJ, O'Donovan C, Magrane M, Alam-Faruque Y, Antunes R, Barrell D, Bely B, Bingley M, Binns D, Bower L, Browne P, Chan WM, Dimmer E, Eberhardt R, Fedotov A, Foulger R, Garavelli J, Huntley R, Jacobsen J, Kleen M, Laiho K, Leinonen R, Legge D, Lin Q, Liu W, Luo J, Orchard S, Patient S, Poggioli D, Pruess M, Corbett M, di Martino G, Donnelly M, van Rensburg P, Bairoch A, Bougueleret L, Xenarios I, Altairac S, Auchincloss A, Argoud-Puy G, Axelsen K, Baratin D, Blatter MC, Boeckmann B, Bolleman J, Bollondi L, Boutet E, Quintaje SB, Breuza L, Bridge A, deCastro E, Ciapina L, Coral D, Coudert E, Cusin I, Delbard G, Doche M, Dornevil D, Roggli PD, Duvaud S, Estreicher A, Famiglietti L, Feuermann M, Gehant S, Farriol-Mathis N, Ferro S, Gasteiger E, Gateau A, Gerritsen V, Gos A, Gruaz-Gumowski N, Hinz U, Hulo C, Hulo N, James J, Jimenez S, Jungo F, Kappler T, Keller G, Lachaize C, Lane-Guermonprez L, Langendijk-Genevaux P, Lara V, Lemercier P, Lieberherr D, de Oliveira Lima T, Mangold V, Martin X, Masson P, Moinat M, Morgat A, Mottaz A, Paesano S, Pedruzzi I, Pilbout S, Pillet V, Poux S, Pozzato M, Redaschi N, Rivoire C, Roechert B, Schneider M, Sigrist C, Sonesson K, Staehli S, Stanley E, Stutz A, Sundaram S, Tognolli M, Verbregue L, Veuthey AL, Yip L, Zuletta L, Wu C, Arighi C, Arminski L, Barker W, Chen C, Chen Y, Hu ZZ, Huang H, Mazumder R, McGarvey P, Natale DA, Nchoutmboube J, Petrova N, Subramanian N, Suzek BE, Ugochukwu U, Vasudevan S, Vinayaka CR, Yeh LS, Zhang J., Nucleic Acids Res. 38(Database issue), 2009
PMID: 19843607
Pegasus: a framework for mapping complex scientific workflows onto distributed systems
Deelman, Scientific Program. J. 13(), 2005
Deelman, Scientific Program. J. 13(), 2005
Improved microbial gene identification with GLIMMER.
Delcher AL, Harmon D, Kasif S, White O, Salzberg SL., Nucleic Acids Res. 27(23), 1999
PMID: 10556321
Delcher AL, Harmon D, Kasif S, White O, Salzberg SL., Nucleic Acids Res. 27(23), 1999
PMID: 10556321
MUSCLE: multiple sequence alignment with high accuracy and high throughput.
Edgar RC., Nucleic Acids Res. 32(5), 2004
PMID: 15034147
Edgar RC., Nucleic Acids Res. 32(5), 2004
PMID: 15034147
Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences.
Goecks J, Nekrutenko A, Taylor J; Galaxy Team, Afgan E, Ananda G, Baker D, Blankenberg D, Chakrabarty R, Coraor N, Goecks J, Von Kuster G, Lazarus R, Li K, Nekrutenko A, Taylor J, Vincent K., Genome Biol. 11(8), 2010
PMID: 20738864
Goecks J, Nekrutenko A, Taylor J; Galaxy Team, Afgan E, Ananda G, Baker D, Blankenberg D, Chakrabarty R, Coraor N, Goecks J, Von Kuster G, Lazarus R, Li K, Nekrutenko A, Taylor J, Vincent K., Genome Biol. 11(8), 2010
PMID: 20738864
Ruffus: a lightweight Python library for computational pipelines.
Goodstadt L., Bioinformatics 26(21), 2010
PMID: 20847218
Goodstadt L., Bioinformatics 26(21), 2010
PMID: 20847218
Taverna: a tool for building and running workflows of services.
Hull D, Wolstencroft K, Stevens R, Goble C, Pocock MR, Li P, Oinn T., Nucleic Acids Res. 34(Web Server issue), 2006
PMID: 16845108
Hull D, Wolstencroft K, Stevens R, Goble C, Pocock MR, Li P, Oinn T., Nucleic Acids Res. 34(Web Server issue), 2006
PMID: 16845108
BioXSD: the common data-exchange format for everyday bioinformatics web services.
Kalas M, Puntervoll P, Joseph A, Bartaseviciute E, Topfer A, Venkataraman P, Pettifer S, Bryne JC, Ison J, Blanchet C, Rapacki K, Jonassen I., Bioinformatics 26(18), 2010
PMID: 20823319
Kalas M, Puntervoll P, Joseph A, Bartaseviciute E, Topfer A, Venkataraman P, Pettifer S, Bryne JC, Ison J, Blanchet C, Rapacki K, Jonassen I., Bioinformatics 26(18), 2010
PMID: 20823319
The Sequence Alignment/Map format and SAMtools.
Li H, Handsaker B, Wysoker A, Fennell T, Ruan J, Homer N, Marth G, Abecasis G, Durbin R; 1000 Genome Project Data Processing Subgroup., Bioinformatics 25(16), 2009
PMID: 19505943
Li H, Handsaker B, Wysoker A, Fennell T, Ruan J, Homer N, Marth G, Abecasis G, Durbin R; 1000 Genome Project Data Processing Subgroup., Bioinformatics 25(16), 2009
PMID: 19505943
Mobyle: a new full web bioinformatics framework.
Neron B, Menager H, Maufrais C, Joly N, Maupetit J, Letort S, Carrere S, Tuffery P, Letondal C., Bioinformatics 25(22), 2009
PMID: 19689959
Neron B, Menager H, Maufrais C, Joly N, Maupetit J, Letort S, Carrere S, Tuffery P, Letondal C., Bioinformatics 25(22), 2009
PMID: 19689959
Automation of in-silico data analysis processes through workflow management systems.
Romano P., Brief. Bioinformatics 9(1), 2007
PMID: 18056132
Romano P., Brief. Bioinformatics 9(1), 2007
PMID: 18056132
XML schemas for common bioinformatic data types and their application in workflow systems.
Seibel PN, Kruger J, Hartmeier S, Schwarzer K, Lowenthal K, Mersch H, Dandekar T, Giegerich R., BMC Bioinformatics 7(), 2006
PMID: 17087823
Seibel PN, Kruger J, Hartmeier S, Schwarzer K, Lowenthal K, Mersch H, Dandekar T, Giegerich R., BMC Bioinformatics 7(), 2006
PMID: 17087823
Solutions for data integration in functional genomics: a critical assessment and case study.
Smedley D, Swertz MA, Wolstencroft K, Proctor G, Zouberakis M, Bard J, Hancock JM, Schofield P., Brief. Bioinformatics 9(6), 2008
PMID: 19112082
Smedley D, Swertz MA, Wolstencroft K, Proctor G, Zouberakis M, Bard J, Hancock JM, Schofield P., Brief. Bioinformatics 9(6), 2008
PMID: 19112082
CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, position-specific gap penalties and weight matrix choice.
Thompson JD, Higgins DG, Gibson TJ., Nucleic Acids Res. 22(22), 1994
PMID: 7984417
Thompson JD, Higgins DG, Gibson TJ., Nucleic Acids Res. 22(22), 1994
PMID: 7984417
Standardization of an api for distributed resource management systems
Troeger, 2007
Troeger, 2007
Analysing scientific workflows: why workflows not only connect web services
Wassink, 2009
Wassink, 2009
Interoperability with Moby 1.0--it's better than sharing your toothbrush!
BioMoby Consortium, Wilkinson MD, Senger M, Kawas E, Bruskiewich R, Gouzy J, Noirot C, Bardou P, Ng A, Haase D, Saiz Ede A, Wang D, Gibbons F, Gordon PM, Sensen CW, Carrasco JM, Fernandez JM, Shen L, Links M, Ng M, Opushneva N, Neerincx PB, Leunissen JA, Ernst R, Twigger S, Usadel B, Good B, Wong Y, Stein L, Crosby W, Karlsson J, Royo R, Parraga I, Ramirez S, Gelpi JL, Trelles O, Pisano DG, Jimenez N, Kerhornou A, Rosset R, Zamacola L, Tarraga J, Huerta-Cepas J, Carazo JM, Dopazo J, Guigo R, Navarro A, Orozco M, Valencia A, Claros MG, Perez AJ, Aldana J, Rojano M, Fernandez-Santa Cruz R, Navas I, Schiltz G, Farmer A, Gessler D, Schoof H, Groscurth A., Brief. Bioinformatics 9(3), 2008
PMID: 18238804
BioMoby Consortium, Wilkinson MD, Senger M, Kawas E, Bruskiewich R, Gouzy J, Noirot C, Bardou P, Ng A, Haase D, Saiz Ede A, Wang D, Gibbons F, Gordon PM, Sensen CW, Carrasco JM, Fernandez JM, Shen L, Links M, Ng M, Opushneva N, Neerincx PB, Leunissen JA, Ernst R, Twigger S, Usadel B, Good B, Wong Y, Stein L, Crosby W, Karlsson J, Royo R, Parraga I, Ramirez S, Gelpi JL, Trelles O, Pisano DG, Jimenez N, Kerhornou A, Rosset R, Zamacola L, Tarraga J, Huerta-Cepas J, Carazo JM, Dopazo J, Guigo R, Navarro A, Orozco M, Valencia A, Claros MG, Perez AJ, Aldana J, Rojano M, Fernandez-Santa Cruz R, Navas I, Schiltz G, Farmer A, Gessler D, Schoof H, Groscurth A., Brief. Bioinformatics 9(3), 2008
PMID: 18238804
Export
Markieren/ Markierung löschen
Markierte Publikationen
Web of Science
Dieser Datensatz im Web of Science®Quellen
PMID: 21278189
PubMed | Europe PMC
Suchen in
Google Scholar