R\'enyi Differential Privacy of the Sampled Gaussian Mechanism
read the original abstract
The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadratically, rather than linearly, with the sampling rate. Characterizing the precise privacy properties of SGM motivated development of several relaxations of the notion of differential privacy. This work unifies and fills in gaps in published results on SGM. We describe a numerically stable procedure for precise computation of SGM's R\'enyi Differential Privacy and prove a nearly tight (within a small constant factor) closed-form bound.
This paper has not been read by Pith yet.
Forward citations
Cited by 15 Pith papers
-
Privacy Amplification in Differentially Private Zeroth-Order Optimization with Hidden States
Introduces hybrid noise and novel coupling analysis to achieve the first convergent hidden-state DP bound for zeroth-order optimization.
-
Trade-off Functions for DP-SGD with Subsampling based on Random Shuffling: Tight Upper and Lower Bounds
Derives tight closed-form bounds on f-DP trade-off functions for DP-SGD with random shuffling subsampling in the high-noise regime σ ≥ √(3/ln M), plus an asymptotic convergence result to the random guessing diagonal w...
-
Trade-off Functions for DP-SGD with Subsampling based on Random Shuffling: Tight Upper and Lower Bounds
Tight closed-form bounds via Berry-Esseen show DP-SGD with random shuffling achieves near-ideal privacy (trade-off close to 1-a) for σ ≥ √(3/ln M) and large M, with δ linear in epochs restricting E to O(√M) and an asy...
-
$\alpha$-Wasserstein Mechanism for R\'{e}nyi Pufferfish Privacy
The α-Wasserstein mechanism calibrates noise for exact Rényi Pufferfish Privacy by bounding the Wasserstein metric, generalizing the W_∞ pufferfish mechanism and Rényi differential privacy results.
-
Differentially Private Model Merging
Post-processing via random selection or linear combination generates differentially private models for arbitrary privacy parameters from pre-trained models on the same dataset.
-
DPQuant: Efficient and Differentially-Private Model Training via Dynamic Quantization Scheduling
DPQuant uses epoch-wise probabilistic layer rotation and DP loss sensitivity to quantize only a changing subset of layers, reducing accuracy degradation from quantization noise in DP-SGD and delivering up to 2.21x thr...
-
AdaPrivate-TS: Private Thompson Sampling for Contextual Bandits with Privacy Amplification
AdaPrivate-TS is a private contextual bandit algorithm that interprets DP noise as increased posterior covariance and uses batched zCDP composition to achieve O(sqrt(d) log T / sqrt(rho)) privacy cost under event-leve...
-
Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models
Empirical benchmarks show distribution similarity between adaptation and pretraining data increases practical privacy leakage in DP-adapted LLMs at fixed theoretical guarantees, with LoRA providing strongest protectio...
-
Provably Communication-Efficient and Privacy-Preserving Federated Graph Neural Networks
CE-FedGNN enables federated GNN training on coupled distributed graphs via infrequent aggregated representation exchange, moving-average estimation for staleness, and metric-DP, with O(1/sqrt(T)) convergence and O(T^{...
-
DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models
DP-SelFT improves the privacy-utility trade-off for LLM fine-tuning by selecting robust layer subsets via DP synthetic data and perturbation-matched evaluation.
-
Rethinking the Security of DP-SGD: A Corrected Analysis of Differentially Private Machine Learning
DP-SGD with expected or batch averaging (EASGM or ASGM) has weaker privacy guarantees than the standard subsampled Gaussian mechanism analysis, confirmed by theoretical re-analysis and audits of libraries including Opacus.
-
INO-SGD: Addressing Utility Imbalance under Individualized Differential Privacy
INO-SGD down-weights data in each batch to improve model performance on strongly private data while satisfying individualized differential privacy constraints.
-
Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGD
Shuffled DP-SGD requires σ ≥ 1/√(2 ln M) or κ ≥ (1/√8)(1 - 1/√(4π ln M)) to limit adversarial advantage, preventing strong privacy and high utility simultaneously.
-
DP-SAPF: Saliency-Aware Parameter Fine-tuning of Public Models for Differentially Private Image Synthesis
DP-SAPF uses gradient-magnitude saliency after noise addition to select LoRA targets for DP fine-tuning of public models, reporting improved synthetic image utility and fidelity on four datasets with lower compute.
-
Differentially Private Model Merging
Post-processing via random selection or linear combination of differentially private models allows meeting arbitrary target privacy parameters without additional training.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.