Enhance Your Research Alliance (EYRA) Benchmark Platform

Supporting researchers to easily set-up benchmarks

One of the currently most well-known benchmarks for algorithm performance is ImageNet. Many challenges have been organized using this database, with the latest challenge now running on Kaggle. In various scientific disciplines there is a growing interest to benchmark algorithm performance on research data. Many algorithms are proposed in the literature, but there is a growing need to compare them on the same data, using the same metrics and ground truth to compare their performance for a specific task. Organizing these open online benchmarks, will not only increase insight into which algorithms perform best for a given task, but open up these tasks for a wider audience to test their algorithms on, which could lead to new breakthroughs in the field. In this project, the Netherlands eScience Center and SURF join forces to develop a platform (EYRA Benchmark Platform) that supports researchers to easily set-up benchmarks and apply their algorithm on benchmarks from various scientific disciplines.

The EYRA benchmark platform aims to facilitate:

  • An easy way to set-up a research benchmark

  • Cross-fertilization between scientific disciplines

  • Overview of benchmarks per scientific discipline

  • Infrastructure to run algorithms on test data in the cloud

  • Insight into algorithm performance for a research problem, beyond the benchmark leaderboard.

The EYRA benchmark platform will be an extension of the COMIC platform developed in the group of professor Bram van Ginneken (Radboud UMC)

Participating organisations

SURF
ASTRON
CLARIN
Erasmus University Medical Center
Leiden University Medical Center
Life Sciences
Life Sciences
Netherlands eScience Center
Radboud University Medical Center
Royal Netherlands Academy of Arts and Sciences
SURFsara
University of Amsterdam
University of Twente

Impact

Output

  • 1.
    Beyond the Leaderboard: Insight and Deployment Challenges to Address Research Problems
    Author(s): Adriënne Mendrik, Stephen Aylward
    Published in Machine Learning Challenges "in the wild" in 2018

Team

Contact person

Jesus Garcia Gonzalez

Jesus Garcia Gonzalez

eScience Research Engineer
Netherlands eScience Center
Mail Jesus
AM
Adriënne Mendrik
eScience Coordinator
Netherlands eScience Center
Jesus Garcia Gonzalez
Jesus Garcia Gonzalez
eScience Research Engineer
Netherlands eScience Center
AL
Annette Langedijk
MH
Mary Hester
Coordinator
SURF
PP
Pushpanjali Pawar
eScience Research Engineer
Netherlands eScience Center
TK
Tom Klaver
eScience Research Engineer
Netherlands eScience Center
Maarten van Meersbergen
Maarten van Meersbergen
eScience Research Engineer
Netherlands eScience Center

Related projects

COVID-19 Grand Challenge

Assessing the suspicion and severity of COVID-19 in a CT scan

Updated 12 months ago
Finished

CHEOPS

Verified construction of correct and optimised parallel software

Updated 16 months ago
Finished

TADPOLE-SHARE

Sharing TADPOLE’s algorithms for reuse and evaluation

Updated 12 months ago
Finished

SecConNet Smart

Smart, secure container networks for trusted big data sharing

Updated 16 months ago
Finished

AutoGraph

Automated multi-scale graph manipulation with topological and flow-based methods

Updated 11 months ago
Finished

City Cloud

From the Things to the Cloud and back

Updated 12 months ago
Finished

IMPACT

Software analytics for the monitoring and assessment of the global impact of eScience software on...

Updated 11 months ago
Finished

A methodology and ecosystem for many-core programming

Boosting the performance of current and future programs

Updated 11 months ago
Finished

Visual Storytelling of Big Imaging Data

Storytelling as a means of visual data communication

Updated 14 months ago
Finished

eStep

Developing an eScience technology platform

Updated 12 months ago
Finished

Algorithmic Geo-visualization

From theory to practice

Updated 16 months ago
Finished

DynaSlum

Data-driven modeling and decision support for slums

Updated 16 months ago
Finished

Enabling Dynamic Services

Realizing the full potential of the Dutch e-Infrastructure

Updated 12 months ago
Finished

RT SAR

An architecture for real Time big data processing for AMBER

Updated 12 months ago
Finished

ABC-MUSE

The generalization and optimization of the multi-purpose software environment

Updated 16 months ago
Finished

A Jungle Computing Approach to Large-Scale Online Forensic Analysis

Programming tools that simplify application development and deployment

Updated 12 months ago
Finished

e-Visualization of Big Data

Interactive big data visualizations

Updated 13 months ago
Finished

Massive Point Clouds for eSciences

Using point clouds to their full potential

Updated 12 months ago
Finished

Generic eScience Technologies

Making breakthroughs in data-driven research

Updated 12 months ago
Finished

eSiBayes

An eScience infrastructure for Bayesian inverse modeling

Updated 12 months ago
Finished