Nahal Mirzaie

Training with stochastic gradient descent (SGD) at moderately large learning rates has been observed to improve robustness against spurious correlations, strong cor- relation between non-predictive features and target labels. Yet, the mechanism underlying this effect remains unclear. In this work, we identify batch size as an additional critical factor and show that robustness gains arise from the implicit regularization of SGD, which intensifies with larger learning rates and smaller batch sizes. This implicit regularization reduces reliance on spurious or short- cut features, thereby enhancing robustness while preserving accuracy. Importantly, this effect appears unique to SGD: gradient descent (GD) does not con- fer the same benefit and may even exacerbate shortcut reliance. Theoretically, we establish this phenomenon in linear models by leveraging statistical formulations of spurious correlations, proving that SGD systematically suppresses spurious feature dependence. Empirically, we demonstrate that the effect extends to deep neural networks across multiple benchmarks.

Whole Slide Image (WSI) classification with multiple instance learning (MIL) in digital pathology faces significant computational challenges. Current methods mostly rely on extensive self-supervised learning (SSL) for satisfactory performance, requiring long training periods and considerable computational resources. At the same time, no pre-training affects performance due to domain shifts from natural images to WSIs. We introduce Snuffy architecture, a novel MIL-pooling method based on sparse transformers that mitigates performance loss with limited pre-training and enables continual few-shot pre-training as a competitive option. Our sparsity pattern is tailored for pathology and is theoretically proven to be a universal approximator with the tightest probabilistic sharp bound on the number of layers for sparse transformers, to date. We demonstrate Snuffy’s effectiveness on CAMELYON16 and TCGA Lung cancer datasets, achieving superior WSI and patch-level accuracies. The code is available on https://github.com/jafarinia/snuffy.

The COVID-19 pandemic has prompted a surge in drug repurposing studies. However, many promising hits identified by modern neural networks failed in the preclinical research, which has raised concerns about the reliability of current drug discovery methods. Among studies that explore the therapeutic potential of drugs for COVID-19 treatment is RxRx19a. Its dataset was derived from High Throughput Screening (HTS) experiments conducted by the Recursion biotechnology company. Prior research on hit discovery using this dataset involved learning healthy and infected cells’ morphological features and utilizing this knowledge to estimate contaminated drugged cells’ scores. Nevertheless, models have never seen drugged cells during training, so these cells’ phenotypic features are out of their trained distribution. That being said, model estimations for treatment samples are not trusted in these methods and can lead to false positives. This work offers a first-in-field weakly-supervised drug efficiency estimation pipeline that utilizes the mixup methodology with a confidence score for its predictions. We applied our method to the RxRx19a dataset and showed that consensus between top hits predicted on different representation spaces increases using our confidence method. Further, we demonstrate that our pipeline is robust, stable, and sensitive to drug toxicity.