publications

2025

1. WACV

   

   Underwater Image Enhancement and Object Detection: Are Poor Object Detection Results On Enhanced Images Due to Missing Human Labels?

   Evan Lucas, Ali Awad, Anthony Geglio, and 5 more authors

   In Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, Feb 2025

   DOI PDF

   

2023

1. TMLR

   

   Bounding generalization error with input compression: An empirical study with infinite-width networks

   Angus Galloway, Anna Golubeva, Mahmoud Salem, and 3 more authors

   Transactions on Machine Learning Research, 2023

   Abs arXiv PDF

   

   Estimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data. The ability to better predict GE based on a single training set may yield overarching DNN design principles to reduce a reliance on trial-and-error, along with other performance assessment advantages. In search of a quantity relevant to GE, we investigate the Mutual Information (MI) between the input and final layer representations, using the infinite-width DNN limit to bound MI. An existing input compression-based GE bound is used to link MI and GE. To the best of our knowledge, this represents the first empirical study of this bound. In our attempt to empirically falsify the theoretical bound, we find that it is often tight for best-performing models. Furthermore, it detects randomization of training labels in many cases, reflects test-time perturbation robustness, and works well given only few training samples. These results are promising given that input compression is broadly applicable where MI can be estimated with confidence.

2022

1. ICML

   

   Monitoring Shortcut Learning using Mutual Information

   Mohammed Adnan, Yani Ioannou, Kenyon Tsai, and 4 more authors

   In ICML Workshop on Spurious Correlations Invariance and Stability, 2022

   Abs arXiv PDF

   

   The failure of deep neural networks to generalize to out-of-distribution data is a well-known problem and raises concerns about the deployment of trained networks in safety-critical domains such as healthcare, finance and autonomous vehicles. We study a particular kind of distribution shift: shortcuts or spurious correlations in the training data. Shortcut learning is often only exposed when models are evaluated on real-world data that does not contain the same spurious correlations, posing a serious dilemma for AI practitioners to properly assess the effectiveness of a trained model for real-world applications. In this work, we propose to use the mutual information (MI) between the learned representation and the input as a metric to find where in training, the network latches onto shortcuts. Experiments demonstrate that MI can be used as a domain-agnostic metric for monitoring shortcut learning.

2. L&O: Methods

   

   Predicting Dreissenid Mussel Abundance in Nearshore Waters using Underwater Imagery and Deep Learning

   Angus Galloway, Dominique Brunet, Reza Valipour, and 5 more authors

   Limnology and Oceanography: Methods, 2022

   Abs DOI HTML PDF

   

   Accurate and cost-effective dreissenid mussel abundance maps are vital to assess their ecological roles in aquatic systems. A deep neural network (DNN) modeling framework using semantic segmentation was developed to automatically assess the abundance distribution of two invasive mussel species: zebra and quagga. DNN models were trained on images captured in Lake Erie and Lake Ontario using an underwater color imaging technique. The accuracy of the method was assessed relative to manual laboratory counts of harvested mussels, their dry biomass, and percentage live coverage estimated from fixed-size quadrats. Assessments performed on a test set collected from 2016 to 2018 show that DNN-based mussel coverage predictions explain 79% of the variance in log biomass, and 71% for log abundance (N = 125). For reference, live coverage estimated by scuba divers was transformed and found to be a better predictor of biomass (93%) and abundance (91%) (N = 725), leaving room for improvement of our automated method. When identical images were presented to eight human analysts and the DNN, the agreement in live mussel coverage prediction was 85% (N = 189). Models generalize well to diverse underwater illuminations, camera orientations, and resolutions, but are adversely impacted by occluding vegetation and suspended sediment. DNN models are an efficient and accurate solution for mapping mussel abundances at a scale that was previously impossible. The method may be integrated with other studies to assess the mussels’ impacts in a variety of aquatic ecosystems. Source code: https://github.com/AngusG/deep-learning-dreissenid and data https://doi.org/10.5683/SP3/MZEBOJ for reproducing our method are publicly available.

2019

1. ICML

   

   Batch Normalization is a Cause of Adversarial Vulnerability

   Angus Galloway, Anna Golubeva, Thomas Tanay, and 2 more authors

   In ICML Workshop on Identifying and Understanding Deep Learning Phenomena, 2019

   Abs arXiv PDF

   

   Batch normalization (BN) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and common corruptions by double-digit percentages, as we show on five standard datasets. Furthermore, we find that substituting weight decay for BN is sufficient to nullify a relationship between adversarial vulnerability and the input dimension. A recent mean-field analysis found that BN induces gradient explosion when used on multiple layers, but this cannot fully explain the vulnerability we observe, given that it occurs already for a single BN layer. We argue that the actual cause is the tilting of the decision boundary with respect to the nearest-centroid classifier along input dimensions of low variance. As a result, the constant introduced for numerical stability in the BN step acts as an important hyperparameter that can be tuned to recover some robustness at the cost of standard test accuracy. We explain this mechanism explicitly on a linear “toy“ model and show in experiments that it still holds for nonlinear real-world models.

2018

1. ICLR

   

   Attacking Binarized Neural Networks

   Angus Galloway, Graham W. Taylor, and Medhat Moussa

   In International conference on learning representations, 2018

   Abs arXiv PDF

   

   Neural networks with low-precision weights and activations offer compelling efficiency advantages over their full-precision equivalents. The two most frequently discussed benefits of quantization are reduced memory consumption, and a faster forward pass when implemented with efficient bitwise operations. We propose a third benefit of very low-precision neural networks: improved robustness against some adversarial attacks, and in the worst case, performance that is on par with full-precision models. We focus on the very low-precision case where weights and activations are both quantized to +/- 1, and note that stochastically quantizing weights in just one layer can sharply reduce the impact of iterative attacks. We observe that non-scaled binary neural networks exhibit a similar effect to the original defensive distillation procedure that led to gradient masking, and a false notion of security. We address this by conducting both black-box and white-box experiments with binary models that do not artificially mask gradients.

2017

1. CRV

   

   The Ciona17 Dataset for Semantic Segmentation of Invasive Species in a Marine Aquaculture Environment

   A. Galloway, G. W. Taylor, A. Ramsay, and 1 more author

   In Conference on Computer and Robot Vision (CRV), May 2017

   Abs DOI arXiv PDF

   

   An original dataset for semantic segmentation, Ciona17, is introduced, which to the best of the authors’ knowledge, is the first dataset of its kind with pixel-level annotations pertaining to invasive species in a marine environment. Diverse outdoor illumination, a range of object shapes, colour, and severe occlusion provide a significant real world challenge for the computer vision community. An accompanying ground-truthing tool for superpixel labeling, Truth and Crop, is also introduced. Finally, we provide a baseline using a variant of Fully Convolutional Networks, and report results in terms of the standard mean intersection over union (mIoU) metric.