UZH-Logo

Maintenance Infos

Elastic scaling of e-infrastructures to support data-intensive research collaborations


Nino-Ruiz, Marcos; Bayliss, Christopher; Galang, Gerson; Grazioli, Guido; Rabanal, Rosana; Tomko, Martin; Sinnott, Richard O (2014). Elastic scaling of e-infrastructures to support data-intensive research collaborations. In: Workshop on Tools and Technologies for Virtual Laboratories – TTVL 2014 / 2014 IEEE 10th International Conference on eScience, Guarujá-SP, Brazil, 20 October 2014 - 24 October 2014, 55-61.

Abstract

For many research endeavours, e-Infrastructures need to provide predictable, on-demand access to large-scale computational resources with high data availability. These need to scale with the research communities requirements and use. One example of such an e-Infrastructure is the Australian Urban Research Infrastructure Network (AURIN – www.aurin.org.au) project, which supports Australia-wide research in and across the urban and built environment. This paper describes the architecture of the AURIN infrastructure and its support for access to distributed (federated) and highly heterogeneous data sets from a wide range of providers. We present how this architecture solution leverages the intersection of high throughput computing (HTC), infrastructure as a service (IaaS) Cloud services and big data technologies including use of NoSQL resources. The driving concept in this architecture and the focus of this paper is the ability for scaling up or down depending on resource demands at any given time. This is done automatically and on demand avoiding either under- or over-utilization of resources. This resource-optimization-driven infrastructure has been designed to ensure that peak loads can be predicted and successfully coped with, as well as avoid wasting resources during non-peak times. This overall management strategy has resulted in an e-Infrastructure that provides a flexible, evolving research environment that scales with research needs, rather than providing a rigid (static) end product.

For many research endeavours, e-Infrastructures need to provide predictable, on-demand access to large-scale computational resources with high data availability. These need to scale with the research communities requirements and use. One example of such an e-Infrastructure is the Australian Urban Research Infrastructure Network (AURIN – www.aurin.org.au) project, which supports Australia-wide research in and across the urban and built environment. This paper describes the architecture of the AURIN infrastructure and its support for access to distributed (federated) and highly heterogeneous data sets from a wide range of providers. We present how this architecture solution leverages the intersection of high throughput computing (HTC), infrastructure as a service (IaaS) Cloud services and big data technologies including use of NoSQL resources. The driving concept in this architecture and the focus of this paper is the ability for scaling up or down depending on resource demands at any given time. This is done automatically and on demand avoiding either under- or over-utilization of resources. This resource-optimization-driven infrastructure has been designed to ensure that peak loads can be predicted and successfully coped with, as well as avoid wasting resources during non-peak times. This overall management strategy has resulted in an e-Infrastructure that provides a flexible, evolving research environment that scales with research needs, rather than providing a rigid (static) end product.

Altmetrics

Downloads

57 downloads since deposited on 03 Feb 2015
48 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), not refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Geography
Dewey Decimal Classification:910 Geography & travel
Language:English
Event End Date:24 October 2014
Deposited On:03 Feb 2015 18:39
Last Modified:05 Apr 2016 18:52
Publisher:IEEE
Publisher DOI:https://doi.org/10.1109/eScience.2014.13
Permanent URL: https://doi.org/10.5167/uzh-105663

Download

[img]
Preview
Content: Published Version
Language: English
Filetype: PDF
Size: 457kB
View at publisher

TrendTerms

TrendTerms displays relevant terms of the abstract of this publication and related documents on a map. The terms and their relations were extracted from ZORA using word statistics. Their timelines are taken from ZORA as well. The bubble size of a term is proportional to the number of documents where the term occurs. Red, orange, yellow and green colors are used for terms that occur in the current document; red indicates high interlinkedness of a term with other terms, orange, yellow and green decreasing interlinkedness. Blue is used for terms that have a relation with the terms in this document, but occur in other documents.
You can navigate and zoom the map. Mouse-hovering a term displays its timeline, clicking it yields the associated documents.

Author Collaborations