Navigation auf zora.uzh.ch

Search ZORA

ZORA (Zurich Open Repository and Archive)

BiasBed -- Rigorous Texture Bias Evaluation

Kalischek, Nikolai; Daudt, Rodrigo Caye; Peters, Torben; Furrer, Reinhard; Wegner, Jan D; Schindler, Konrad (2022). BiasBed -- Rigorous Texture Bias Evaluation. ArXiv.org 2211.13190, Cornell University.

Abstract

The well-documented presence of texture bias in modern convolutional neural networks has led to a plethora of algorithms that promote an emphasis on shape cues, often to support generalization to new domains. Yet, common datasets, benchmarks and general model selection strategies are missing, and there is no agreed, rigorous evaluation protocol. In this paper, we investigate difficulties and limitations when training networks with reduced texture bias. In particular, we also show that proper evaluation and meaningful comparisons between methods are not trivial. We introduce BiasBed, a testbed for texture- and style-biased training, including multiple datasets and a range of existing algorithms. It comes with an extensive evaluation protocol that includes rigorous hypothesis testing to gauge the significance of the results, despite the considerable training instability of some style bias methods. Our extensive experiments, shed new light on the need for careful, statistically founded evaluation protocols for style bias (and beyond). E.g., we find that some algorithms proposed in the literature do not significantly mitigate the impact of style bias at all. With the release of BiasBed, we hope to foster a common understanding of consistent and meaningful comparisons, and consequently faster progress towards learning methods free of texture bias. Code is available at this https URL: https://github.com/D1noFuzi/BiasBed.

Additional indexing

Item Type:Working Paper
Communities & Collections:07 Faculty of Science > Institute of Mathematics
07 Faculty of Science > Institute for Computational Science
Dewey Decimal Classification:510 Mathematics
Uncontrolled Keywords:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Language:English
Date:2022
Deposited On:17 Feb 2023 13:39
Last Modified:06 Jun 2024 03:21
Series Name:ArXiv.org
Number of Pages:18
ISSN:2331-8422
Additional Information:Version 1 submittet am 23.11.2022; Version 3 am 24.03.2023 überarbeitet.
OA Status:Green
Free access at:Publisher DOI. An embargo period may apply.
Publisher DOI:https://doi.org/10.48550/arXiv.2211.13190
Download PDF  'BiasBed -- Rigorous Texture Bias Evaluation'.
Preview
  • Content: Published Version
  • Language: English
  • Description: Version 3 vom 21.03.2023
  • Licence: Creative Commons: Attribution 4.0 International (CC BY 4.0)

Metadata Export

Statistics

Citations

Dimensions.ai Metrics

Altmetrics

Downloads

4 downloads since deposited on 17 Feb 2023
5 downloads since 12 months
Detailed statistics

Authors, Affiliations, Collaborations

Similar Publications