UZH-Logo

Cluster versus grid for operational generation of ATCOR’s modtran-based look up tables


Brazil, J; Richter, R; Schläpfer, D; Schaepman, M E; Itten, K I (2008). Cluster versus grid for operational generation of ATCOR’s modtran-based look up tables. Parallel Computing, 34(1):32-46.

Abstract

A critical step in the product generation of satellite or airborne earth observation data is the correction of atmospheric features. Due to the complexity of the underlying physical model and the amount of coordinated effort required to provide, verify and maintain baseline atmospheric observations, one particular scientific modelling program, modtran, whose ancestor was first released in 1972, has become a de facto basis for such processing. While this provides the basis of per-pixel physical modelling, higher-level algorithms, which rely on the output of potentially thousands of runs of modtran are required for the processing of an entire scene. The widely-used atcor family of atmospheric correction software employs the commonly-used strategy of pre-computing a large look up table (lut) of values, representing modtran input parameter variation in multiple dimensions, to allow for reasonable running times in operation. The computation of this pre-computed look up table has previously taken weeks to produce a dvd (about 4 GB) of output. The motivation for quicker turnaround was introduced when researchers at multiple institutions began collaboration on extending atcor features into more specialized applications. In this setting, a parallel implementation is investigated with the primary goals of: the parallel execution of multiple instances of modtran as opaque third-party software, the consistency of numeric results in a heterogeneous compute environment, the potential to make use of otherwise idle computing resources available to researchers located at multiple institutions, and acceptable total turnaround time. In both grid and cluster environments, parallel generation of a numerically consistent lut is shown to be possible and reduce ten days of computation time on a single, high-end processor to under two days of processing time with as little as eight commodity CPUs. Runs on up to 64 processors are investigated and the advantages and disadvantages of clusters and grids are briefly explored in reference to the their evaluation in a medium-sized collaborative project.

A critical step in the product generation of satellite or airborne earth observation data is the correction of atmospheric features. Due to the complexity of the underlying physical model and the amount of coordinated effort required to provide, verify and maintain baseline atmospheric observations, one particular scientific modelling program, modtran, whose ancestor was first released in 1972, has become a de facto basis for such processing. While this provides the basis of per-pixel physical modelling, higher-level algorithms, which rely on the output of potentially thousands of runs of modtran are required for the processing of an entire scene. The widely-used atcor family of atmospheric correction software employs the commonly-used strategy of pre-computing a large look up table (lut) of values, representing modtran input parameter variation in multiple dimensions, to allow for reasonable running times in operation. The computation of this pre-computed look up table has previously taken weeks to produce a dvd (about 4 GB) of output. The motivation for quicker turnaround was introduced when researchers at multiple institutions began collaboration on extending atcor features into more specialized applications. In this setting, a parallel implementation is investigated with the primary goals of: the parallel execution of multiple instances of modtran as opaque third-party software, the consistency of numeric results in a heterogeneous compute environment, the potential to make use of otherwise idle computing resources available to researchers located at multiple institutions, and acceptable total turnaround time. In both grid and cluster environments, parallel generation of a numerically consistent lut is shown to be possible and reduce ten days of computation time on a single, high-end processor to under two days of processing time with as little as eight commodity CPUs. Runs on up to 64 processors are investigated and the advantages and disadvantages of clusters and grids are briefly explored in reference to the their evaluation in a medium-sized collaborative project.

Citations

9 citations in Web of Science®
10 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

1 download since deposited on 08 Jan 2009
0 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Geography
Dewey Decimal Classification:910 Geography & travel
Language:English
Date:January 2008
Deposited On:08 Jan 2009 12:09
Last Modified:05 Apr 2016 12:37
Publisher:Elsevier
ISSN:0167-8191
Publisher DOI:10.1016/j.parco.2007.11.002
Permanent URL: http://doi.org/10.5167/uzh-6431

Download

[img]Filetype: PDF - Registered users only
Size: 2MB
View at publisher

TrendTerms

TrendTerms displays relevant terms of the abstract of this publication and related documents on a map. The terms and their relations were extracted from ZORA using word statistics. Their timelines are taken from ZORA as well. The bubble size of a term is proportional to the number of documents where the term occurs. Red, orange, yellow and green colors are used for terms that occur in the current document; red indicates high interlinkedness of a term with other terms, orange, yellow and green decreasing interlinkedness. Blue is used for terms that have a relation with the terms in this document, but occur in other documents.
You can navigate and zoom the map. Mouse-hovering a term displays its timeline, clicking it yields the associated documents.

Author Collaborations