Due to the data explosion, there is a huge demand for big data-processing infrastructures in almost all sciences. Many of such e-infrastructures are distributed facilities that are becoming increasingly diverse (e.g., they may contain a variety of accelerators and software-defined networks). The diversity makes the infrastructures complex and difficult to use for scientists.
Nonetheless, many applications need to harness the power of these diverse distributed infrastructures to address data processing problems. Handling the complexity and diversity of distributed infrastructures thus is a key research challenge. The aim of this proposal is to create a state-of-the-art experimentation environment for such research by building the fifth generation of the Distributed ASCI Supercomputer. DAS-5 will be based on a similar concept as its successful predecessors: a collection of six cluster computers located at different institutes and integrated into a single shared testbed for experimental computer science research in the Netherlands. The new system is designed to address important new research questions, like diversity (heterogeneous computing), programmable hybrid networks, distributed resource management, virtualization, energy-efficiency, and resilience. We use many real-world applications for this research. In particular, the Netherlands eScience Center (NLeSC), COMMIT, and ASTRON have many projects that need a high-performance distributed system with modern accelerators. The participation of NLeSC in DAS-5 is new and provides NLeSC a stepping-stone system for eScience research, to prepare for larger-scale runs on production systems, aiming at both scientific breakthroughs and economical impact through links with the Top Sectors as designated by the Dutch Government. Likewise, the participation of the Dutch ICT program COMMIT in DAS-5 is new and will result in an even broader national coverage of DAS usage, especially from the University of Twente.
This project focuses on the developments that require a new system, it summarizes earlier DAS-4 research (resulting in about 40 PhD theses since Sept. 2008 that would have been hard to do without DAS) and then describes the research that will be done with DAS-5. Over 50 (PhD/Postdoc) projects already exist now that will have funding by the end of 2014 and that will use DAS-5. The new system will consist of six clusters (at VU University, the University of Amsterdam, TU Delft, Leiden University, MultimediaN, and ASTRON). Co-funding will be provided by these six sites as well as by the new participants NLeSC and COMMIT.
This project page covers two projects, DAS-5 and DAS-6.
