Weakly supervised alignment of multisensor images

Abstract

Manifold alignment has become very popular in recent literature. Aligning data distributions prior to product generation is an appealing strategy, since it allows to provide data spaces that are more similar to each other, regardless of the subsequent use of the transformed data. We propose a methodology that finds a common representation among data spaces from different sensors using geographic image correspondences, or semantic ties. To cope with the strong deformations between the data spaces considered, we propose to add nonlinearities by expanding the input space with Gaussian Radial Basis Function (RBF) features with respect to the centroids of a partitioning of the data. Such features allow us to cope with nonlinear transformations, while keeping a simple and efficient linear formulation. The proposed method is multi-domain and does not require co-registration, rather only a partial degree of spatial overlap. We test it on a challenging problem of multisensor classification transferring a model trained on a WorldView 2 image to predict land cover of a 3-bands orthophoto and show that we can transfer the model with an accuracy comparable to the one that would have been obtained by a model trained on the target image with an image-specific ground truth.