Header

UZH-Logo

Maintenance Infos

Elastic scaling of e-infrastructures to support data-intensive research collaborations


Nino-Ruiz, Marcos; Bayliss, Christopher; Galang, Gerson; Grazioli, Guido; Rabanal, Rosana; Tomko, Martin; Sinnott, Richard O (2014). Elastic scaling of e-infrastructures to support data-intensive research collaborations. In: Workshop on Tools and Technologies for Virtual Laboratories – TTVL 2014 / 2014 IEEE 10th International Conference on eScience, Guarujá-SP, Brazil, 20 October 2014 - 24 October 2014, 55-61.

Abstract

For many research endeavours, e-Infrastructures need to provide predictable, on-demand access to large-scale computational resources with high data availability. These need to scale with the research communities requirements and use. One example of such an e-Infrastructure is the Australian Urban Research Infrastructure Network (AURIN – www.aurin.org.au) project, which supports Australia-wide research in and across the urban and built environment. This paper describes the architecture of the AURIN infrastructure and its support for access to distributed (federated) and highly heterogeneous data sets from a wide range of providers. We present how this architecture solution leverages the intersection of high throughput computing (HTC), infrastructure as a service (IaaS) Cloud services and big data technologies including use of NoSQL resources. The driving concept in this architecture and the focus of this paper is the ability for scaling up or down depending on resource demands at any given time. This is done automatically and on demand avoiding either under- or over-utilization of resources. This resource-optimization-driven infrastructure has been designed to ensure that peak loads can be predicted and successfully coped with, as well as avoid wasting resources during non-peak times. This overall management strategy has resulted in an e-Infrastructure that provides a flexible, evolving research environment that scales with research needs, rather than providing a rigid (static) end product.

Abstract

For many research endeavours, e-Infrastructures need to provide predictable, on-demand access to large-scale computational resources with high data availability. These need to scale with the research communities requirements and use. One example of such an e-Infrastructure is the Australian Urban Research Infrastructure Network (AURIN – www.aurin.org.au) project, which supports Australia-wide research in and across the urban and built environment. This paper describes the architecture of the AURIN infrastructure and its support for access to distributed (federated) and highly heterogeneous data sets from a wide range of providers. We present how this architecture solution leverages the intersection of high throughput computing (HTC), infrastructure as a service (IaaS) Cloud services and big data technologies including use of NoSQL resources. The driving concept in this architecture and the focus of this paper is the ability for scaling up or down depending on resource demands at any given time. This is done automatically and on demand avoiding either under- or over-utilization of resources. This resource-optimization-driven infrastructure has been designed to ensure that peak loads can be predicted and successfully coped with, as well as avoid wasting resources during non-peak times. This overall management strategy has resulted in an e-Infrastructure that provides a flexible, evolving research environment that scales with research needs, rather than providing a rigid (static) end product.

Statistics

Altmetrics

Downloads

74 downloads since deposited on 03 Feb 2015
16 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), not refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Geography
Dewey Decimal Classification:910 Geography & travel
Language:English
Event End Date:24 October 2014
Deposited On:03 Feb 2015 18:39
Last Modified:16 Aug 2017 03:37
Publisher:IEEE
Publisher DOI:https://doi.org/10.1109/eScience.2014.13

Download

Download PDF  'Elastic scaling of e-infrastructures to support data-intensive research collaborations'.
Preview
Content: Published Version
Language: English
Filetype: PDF
Size: 457kB
View at publisher