The BRaliBase Dent – a Tale of Benchmark Design and Interpretation

Löwes B, Chauve C, Ponty Y, Giegerich R (2017)
Briefings in Bioinformatics 18(2): 306-311.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
DOI
Autor*in
Löwes, BenediktUniBi; Chauve, Cedric; Ponty, Yann; Giegerich, RobertUniBi
Einrichtung
Technische Fakultät > Int. Graduiertenkolleg DiDy (GRK 1906)
Technische Fakultät > AG Praktische Informatik
Centrum für Biotechnologie > Graduate Center
Abstract / Bemerkung
BRaliBase is a widely used benchmark for assessing the accuracy of RNA secondary structure alignment methods. In most case studies based on the BRaliBase benchmark, one can observe a puzzling drop in accuracy in the 40%-60% sequence identity range, the so-called “BRaliBase Dent”. In the present note, we show this dent is due to a bias in the composition of the BRaliBase benchmark, namely the inclusion of a disproportionate number of tRNAs, which exhibit a very conserved secondary structure. Our analysis, aside of its interest regarding the specific case of the BRaliBase benchmark, also raises important questions regarding the design and use of benchmarks in computational biology.
Stichworte
RNA structural alignment; benchmark; RNA family database
Erscheinungsjahr
2017
Zeitschriftentitel
Briefings in Bioinformatics
Band
18
Ausgabe
2
Seite(n)
306 - 311
ISSN
1467-5463
Page URI
https://pub.uni-bielefeld.de/record/2901460

Zitieren

Löwes B, Chauve C, Ponty Y, Giegerich R. The BRaliBase Dent – a Tale of Benchmark Design and Interpretation. Briefings in Bioinformatics. 2017;18(2):306-311.
Löwes, B., Chauve, C., Ponty, Y., & Giegerich, R. (2017). The BRaliBase Dent – a Tale of Benchmark Design and Interpretation. Briefings in Bioinformatics, 18(2), 306-311. doi:10.1093/bib/bbw022
Löwes, Benedikt, Chauve, Cedric, Ponty, Yann, and Giegerich, Robert. 2017. “The BRaliBase Dent – a Tale of Benchmark Design and Interpretation”. Briefings in Bioinformatics 18 (2): 306-311.
Löwes, B., Chauve, C., Ponty, Y., and Giegerich, R. (2017). The BRaliBase Dent – a Tale of Benchmark Design and Interpretation. Briefings in Bioinformatics 18, 306-311.
Löwes, B., et al., 2017. The BRaliBase Dent – a Tale of Benchmark Design and Interpretation. Briefings in Bioinformatics, 18(2), p 306-311.
B. Löwes, et al., “The BRaliBase Dent – a Tale of Benchmark Design and Interpretation”, Briefings in Bioinformatics, vol. 18, 2017, pp. 306-311.
Löwes, B., Chauve, C., Ponty, Y., Giegerich, R.: The BRaliBase Dent – a Tale of Benchmark Design and Interpretation. Briefings in Bioinformatics. 18, 306-311 (2017).
Löwes, Benedikt, Chauve, Cedric, Ponty, Yann, and Giegerich, Robert. “The BRaliBase Dent – a Tale of Benchmark Design and Interpretation”. Briefings in Bioinformatics 18.2 (2017): 306-311.

3 Zitationen in Europe PMC

Daten bereitgestellt von Europe PubMed Central.

Essential guidelines for computational method benchmarking.
Weber LM, Saelens W, Cannoodt R, Soneson C, Hapfelmeier A, Gardner PP, Boulesteix AL, Saeys Y, Robinson MD., Genome Biol 20(1), 2019
PMID: 31221194
Freiburg RNA tools: a central online resource for RNA-focused research and teaching.
Raden M, Ali SM, Alkhnbashi OS, Busch A, Costa F, Davis JA, Eggenhofer F, Gelhausen R, Georg J, Heyne S, Hiller M, Kundu K, Kleinkauf R, Lott SC, Mohamed MM, Mattheis A, Miladi M, Richter AS, Will S, Wolff J, Wright PR, Backofen R., Nucleic Acids Res 46(w1), 2018
PMID: 29788132
TurboFold II: RNA structural alignment and secondary structure prediction informed by multiple homologs.
Tan Z, Fu Y, Sharma G, Mathews DH., Nucleic Acids Res 45(20), 2017
PMID: 29036420

23 References

Daten bereitgestellt von Europe PubMed Central.

Multiple sequence alignment modeling: methods and applications
Chatzou M, Magis C, Chang J-M., 2015
BAliBASE: a benchmark alignment database for the evaluation of multiple alignment programs.
Thompson JD, Plewniak F, Poch O., Bioinformatics 15(1), 1999
PMID: 10068696
BAliBASE 3.0: latest developments of the multiple sequence alignment benchmark.
Thompson JD, Koehl P, Ripp R, Poch O., Proteins 61(1), 2005
PMID: 16044462
Some remarks on evaluating the quality of the multiple sequence alignment based on the BAliBASE benchmark
Błażewicz J, Formanowicz P, Wojciechowski P.., 2009
A comprehensive comparison of comparative RNA structure prediction approaches.
Gardner PP, Giegerich R., BMC Bioinformatics 5(), 2004
PMID: 15458580
A benchmark of multiple sequence alignment programs upon structural RNAs.
Gardner PP, Wilm A, Washietl S., Nucleic Acids Res. 33(8), 2005
PMID: 15860779
An enhanced RNA alignment benchmark for sequence alignment programs.
Wilm A, Mainz I, Steger G., Algorithms Mol Biol 1(), 2006
PMID: 17062125
Simultaneous solution of the RNA folding, alignment and protosequence problems
Sankoff D.., 1985
Fine-tuning structural RNA alignments in the twilight zone.
Bremges A, Schirmer S, Giegerich R., BMC Bioinformatics 11(), 2010
PMID: 20433706
Accurate multiple sequence-structure alignment of RNA sequences using combinatorial optimization.
Bauer M, Klau GW, Reinert K., BMC Bioinformatics 8(), 2007
PMID: 17662141
Consensus shapes: an alternative to the Sankoff algorithm for RNA consensus structure prediction.
Reeder J, Giegerich R., Bioinformatics 21(17), 2005
PMID: 16020472
Complete probabilistic analysis of RNA shapes.
Voss B, Giegerich R, Rehmsmeier M., BMC Biol. 4(), 2006
PMID: 16480488
Pure multiple RNA secondary structure alignments: a progressive profile approach.
Hochsmann M, Voss B, Giegerich R., IEEE/ACM Trans Comput Biol Bioinform 1(1), 2004
PMID: 17048408
Fast pairwise structural RNA alignments by pruning of the dynamical programming matrix
Hull J, Torarinsson E, Gorodkin J.., 2007
Alignment of RNA base pairing probability matrices.
Hofacker IL, Bernhart SH, Stadler PF., Bioinformatics 20(14), 2004
PMID: 15073017
Specific alignment of structured RNA: stochastic grammars and sequence annealing.
Bradley RK, Pachter L, Holmes I., Bioinformatics 24(23), 2008
PMID: 18796475
A max-margin model for efficient simultaneous alignment and folding of RNA sequences.
Do CB, Foo CS, Batzoglou S., Bioinformatics 24(13), 2008
PMID: 18586747
DAFS: simultaneous aligning and folding of RNA sequences via dual decomposition.
Sato K, Kato Y, Akutsu T, Asai K, Sakakibara Y., Bioinformatics 28(24), 2012
PMID: 23060618
Inferring noncoding RNA families and classes by means of genome-scale structure-based clustering.
Will S, Reiche K, Hofacker IL, Stadler PF, Backofen R., PLoS Comput. Biol. 3(4), 2007
PMID: 17432929
SPARSE: quadratic time simultaneous alignment and folding of RNAs without sequence-based heuristics.
Will S, Otto C, Miladi M, Mohl M, Backofen R., Bioinformatics 31(15), 2015
PMID: 25838465
ExpaRNA-P: simultaneous exact pattern matching and folding of RNAs.
Otto C, Mohl M, Heyne S, Amit M, Landau GM, Backofen R, Will S., BMC Bioinformatics 15(), 2014
PMID: 25551362
Chaining sequence/structure seeds for computing RNA similarity.
Bourgeade L, Chauve C, Allali J., J. Comput. Biol. 22(3), 2015
PMID: 25768236
Rfam: an RNA family database.
Griffiths-Jones S, Bateman A, Marshall M, Khanna A, Eddy SR., Nucleic Acids Res. 31(1), 2003
PMID: 12520045
Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®
Quellen

PMID: 26984616
PubMed | Europe PMC

Suchen in

Google Scholar