REVIEW 8 cited by
Gradient Matching for Domain Generalization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Gradient Matching for Domain Generalization
read the original abstract
Machine learning systems typically assume that the distributions of training and test sets match closely. However, a critical requirement of such systems in the real world is their ability to generalize to unseen domains. Here, we propose an inter-domain gradient matching objective that targets domain generalization by maximizing the inner product between gradients from different domains. Since direct optimization of the gradient inner product can be computationally prohibitive -- requires computation of second-order derivatives -- we derive a simpler first-order algorithm named Fish that approximates its optimization. We demonstrate the efficacy of Fish on 6 datasets from the Wilds benchmark, which captures distribution shift across a diverse range of modalities. Our method produces competitive results on these datasets and surpasses all baselines on 4 of them. We perform experiments on both the Wilds benchmark, which captures distribution shift in the real world, as well as datasets in DomainBed benchmark that focuses more on synthetic-to-real transfer. Our method produces competitive results on both benchmarks, demonstrating its effectiveness across a wide range of domain generalization tasks.
Forward citations
Cited by 8 Pith papers
-
Invariant Gradient Alignment for Robust Reasoning Distillation
Invariant Gradient Alignment uses Logical Isomer Sets and a Continuous Gradient Conflict Mask to tighten OOD generalization bounds and boost empirical performance over ERM in reasoning distillation.
-
Continual Learning of Domain-Invariant Representations
Introduces replay-based continual learning with sequential invariance alignment to learn domain-invariant representations, outperforming baselines on generalization to unseen domains across six datasets in vision, med...
-
Assessing Distribution Shift in Human Activity Recognition for Domain Generalization
Evaluates four distribution shifts in sensor-based HAR, finds diversity shifts dominate, and shows 28 DG methods only marginally beat ERM while releasing open benchmarks.
-
Learning Gradient-based Mixup with Extrapolation toward Flatter Minima for Domain Generalization
FGMix learns instance weights via gradient compatibilities to perform mixup with extrapolation toward flatter minima, outperforming prior DG methods on DomainBed.
-
Towards Truly Multilingual ASR: Generalizing Code-Switching ASR to Unseen Language Pairs
Merged bilingual CS-ASR models show only modest generalization to unseen language pairs, indicating limited transfer of code-switching capabilities.
-
Environment-Robust Representation Learning with Empirical Bayes
An empirical Bayes variational inference method learns environment-robust latent variables from multi-environment data for improved prediction in unseen environments.
-
Beyond Instance-Level Self-Supervision in 3D Multi-Modal Medical Imaging
A self-supervised approach uses consistent spatial relationships of anatomical structures across patients to improve 3D multi-modal medical image representations, yielding modest gains on segmentation and classificati...
-
Causal Fine-Tuning under Latent Confounded Shift
Causal Fine-Tuning decomposes BERT representations into causal and spurious parts via SCM inductive bias to improve robustness under latent confounded shifts in text classification.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.