pith. sign in

arxiv: 2605.05076 · v2 · pith:T44TCBXQnew · submitted 2026-05-06 · 🧮 math.ST · stat.CO· stat.ME· stat.ML· stat.TH

High-Dimensional Statistics: Reflections on Progress and Open Problems

Arian Maleki , Subhabrata Sen , Sivaraman Balakrishnan , Verena Zuber , Chao Gao , Rishabh Dudeja , Christos Thrampoulidis , Anru Zhang
show 4 more authors
Weijie Su Jason M. Klusowski Po-Ling Loh Ali Shojaie
This is my paper

Pith reviewed 2026-06-30 23:24 UTC · model grok-4.3

classification 🧮 math.ST stat.COstat.MEstat.MLstat.TH
keywords high-dimensional statisticsestimationinferencerandom matrix theoryoptimizationconcentration of measureinformation theoryopen problems
0
0 comments X

The pith

High-dimensional statistics has advanced substantially over two decades to handle complex dependent data while forming deep links to optimization, random matrix theory, and information theory.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper synthesizes representative advances in high-dimensional statistics driven by easier data collection in biology, medicine, and other domains. It shows how the field now addresses estimation and inference for datasets with rich dependency and heterogeneity that defeat classical methods. This work has created contributions to and from optimization, concentration of measure, random matrix theory, information theory, and theoretical computer science. The review highlights common themes, salient open problems, and entry-point references to guide readers through the rapid developments.

Core claim

Over the past two decades, the field of high-dimensional statistics has experienced substantial progress, driven largely by technological advances that have dramatically reduced the cost and effort for data collection and storage across a broad range of domains. Modern datasets are increasingly complex, often exhibiting rich dependency, heterogeneity, and other features that challenge traditional statistical methods. In response, high-dimensional statistics has evolved to address more sophisticated estimation and inference problems. This evolution has, in turn, fostered deep connections with and contributions to a wide range of research areas, including optimization, concentration of measure

What carries the argument

Synthesis of representative advances in estimation and inference for high-dimensional data together with identification of common themes and open problems.

Load-bearing premise

The advances chosen for synthesis are representative of the field's overall trajectory and the open problems identified are the most salient ones for guiding future work.

What would settle it

A broader community survey that demonstrates the selected advances are not representative or that different open problems are more pressing would falsify the review's synthesis.

Figures

Figures reproduced from arXiv: 2605.05076 by Ali Shojaie, Anru Zhang, Arian Maleki, Chao Gao, Christos Thrampoulidis, Jason M. Klusowski, Po-Ling Loh, Rishabh Dudeja, Sivaraman Balakrishnan, Subhabrata Sen, Verena Zuber, Weijie Su.

Figure 1
Figure 1. Figure 1: Schematic phase diagram illustrating the computational-statistical gap. The solid blue curve marks view at source ↗
Figure 2
Figure 2. Figure 2: (a) Schematic illustration of data integration from summary level data. (b) Illustration of analysis view at source ↗
Figure 3
Figure 3. Figure 3: A visual comparison of (a) one-shot averaging vs. (b) iterative optimization. Machine view at source ↗
Figure 4
Figure 4. Figure 4: Some interesting open directions in distributed learning. (a) An illustration of a sequential setting, view at source ↗
read the original abstract

Over the past two decades, the field of high-dimensional statistics has experienced substantial progress, driven largely by technological advances that have dramatically reduced the cost and effort for data collection and storage across a broad range of domains, including biology, medicine, astronomy, and the social and environmental sciences. Modern datasets are increasingly complex, often exhibiting rich dependency, heterogeneity, and other features that challenge traditional statistical methods. In response, high-dimensional statistics has evolved to address more sophisticated estimation and inference problems. This evolution has, in turn, fostered deep connections with and contributions to a wide range of research areas, including optimization, concentration of measure, random matrix theory, information theory, and theoretical computer science. Given the rapid pace of recent developments in high-dimensional statistics, our goal is to synthesize representative advances, highlight common themes and open problems, and point to important works that offer entry points into the field.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The manuscript is a reflective survey synthesizing representative advances in high-dimensional statistics over the past two decades. It attributes substantial progress to technological advances in data collection and storage across domains such as biology, medicine, astronomy, and the social sciences. The paper notes that modern datasets exhibit rich dependency and heterogeneity, prompting the field to address more sophisticated estimation and inference problems. This evolution has fostered connections with optimization, concentration of measure, random matrix theory, information theory, and theoretical computer science. The stated goal is to synthesize advances, highlight common themes and open problems, and point to entry-point works in the literature.

Significance. As a survey, the work provides a high-level narrative overview of field trajectory and interdisciplinary links. If the selected examples are balanced and the open problems are well-chosen, it can serve as a useful orientation for new researchers and a reference for common themes. The manuscript does not advance new technical claims, derivations, or empirical results; its value lies in synthesis rather than novelty of content.

minor comments (2)
  1. [Abstract / Introduction] The abstract states the goal of synthesizing 'representative advances' but does not indicate selection criteria or the number of sub-areas covered; adding a brief explicit statement on scope in the introduction would help readers assess balance.
  2. Since the paper is positioned as a reflection rather than an exhaustive review, a short concluding section that explicitly lists the open problems highlighted throughout would improve navigability and impact.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of the manuscript as a reflective survey and for the recommendation of minor revision. The report accurately captures the paper's scope and intent. No specific major comments are listed in the provided report.

Circularity Check

0 steps flagged

Survey paper with no derivations, predictions or fitted quantities

full rationale

The manuscript is explicitly a reflective survey whose goal is to synthesize representative advances and open problems. It contains no equations, no derivations, no statistical predictions, and no fitted parameters. All statements are narrative characterizations of field trajectory resting on selection of published examples; these are presented as illustrative. No load-bearing step reduces to a self-citation chain or to a definition by construction. The paper is therefore self-contained against external benchmarks with score 0.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a review paper with no new mathematical claims, so it introduces no free parameters, axioms, or invented entities beyond those already established in the prior literature it cites.

pith-pipeline@v0.9.1-grok · 5737 in / 959 out tokens · 24182 ms · 2026-06-30T23:24:15.294837+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

109 extracted references · 95 canonical work pages · 36 internal anchors

  1. [1]

    Understanding intermediate layers using linear classifier probes

    [AB16] G. Alain and Y. Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644,

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [2]

    [ADZ23] A. N. Angelopoulos, J. C. Duchi, and T. Zrnic. PPI++: Efficient prediction-powered infer- ence.arXiv preprint arXiv:2311.01453,

    work page internal anchor Pith review Pith/arXiv arXiv

  3. [3]

    Concrete Problems in AI Safety

    [AOS`16] D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Man´ e. Concrete problems in ai safety.arXiv preprint arXiv:1606.06565,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    What learning algorithm is in-context learning? Investigations with linear models

    44 [ASA`22] E. Aky¨ urek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou. What learning algorithm is in-context learning? Investigations with linear models.arXiv preprint arXiv:2211.15661,

    work page internal anchor Pith review Pith/arXiv arXiv

  5. [5]

    [BC15] R. F. Barber and E. J. Cand` es. Controlling the false discovery rate via knockoffs.The Annals of Statistics, pages 2055–2085,

    2055

  6. [6]

    Bayesian inference in high-dimensional models

    [BCG21] S. Banerjee, I. Castillo, and S. Ghosal. Bayesian inference in high-dimensional models.arXiv preprint arXiv:2101.04491,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    Bhattacharyya, C

    [BDGW25] A. Bhattacharyya, C. Daskalakis, T. Gouleakis, and Y. Wang. Learning high-dimensional Gaussians from censored data.arXiv preprint arXiv:2504.19446,

    work page arXiv

  8. [8]

    arXiv preprint arXiv:2505.17360 , year=

    [BHJK25] R. Buhai, J. Hsieh, A. Jain, and P. K. Kothari. The quasi-polynomial low-degree conjecture is false.arXiv preprint arXiv:2505.17360,

    work page arXiv

  9. [9]

    Brown, B

    [BMR`20] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877–1901,

    1901

  10. [10]

    Castillo

    [Cas14] I. Castillo. On Bayesian supremum norm contraction rates.Annals of Statistics, 42(5):2058– 2091,

    2058

  11. [11]

    Castillo

    [Cas24] I. Castillo. Bayesian nonparametric statistics, St-Flour lecture notes.arXiv preprint arXiv:2402.16422,

    work page arXiv

  12. [12]

    The high-dimensional asymptotics of first order methods with random data

    [CCM21] M. Celentano, C. Cheng, and A. Montanari. The high-dimensional asymptotics of first order methods with random data.arXiv preprint arXiv:2112.07572,

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    [CCV24] T. T. Cai, A. Chakraborty, and L. Vuursteen. Federated nonparametric hypothesis test- ing with differential privacy constraints: Optimal rates and adaptive tests.arXiv preprint arXiv:2406.06749,

    work page arXiv

  14. [14]

    [CGR18] M. Chen, C. Gao, and Z. Ren. Robust covariance and scatter matrix estimation under huber’s contamination model.The Annals of Statistics, 46(5):1932–1960,

    1932

  15. [15]

    Dynamics of transient structure in in-context linear regression transformers

    [CHFRM25] L. Carroll, J. Hoogland, M. Farrugia-Roberts, and D. Murfet. Dynamics of transient structure in in-context linear regression transformers.arXiv preprint arXiv:2501.17745,

    work page arXiv

  16. [16]

    Chaudhuri, J

    [CLC25] S. Chaudhuri, J. Li, and T. A. Courtade. Robust estimation under heterogeneous corruption rates.arXiv preprint arXiv:2508.15051,

    work page arXiv

  17. [17]

    [CLM24] X. Chen, L. Liu, and R. Mukherjee. Method-of-moments inference for GLMs and doubly robust functionals under proportional asymptotics.arXiv preprint arXiv:2408.06103,

    work page arXiv

  18. [18]

    [CLS15] E. J. Candes, X. Li, and M. Soltanolkotabi. Phase retrieval via Wirtinger flow: Theory and algorithms.IEEE Transactions on Information Theory, 61(4):1985–2007,

    1985

  19. [19]

    Celentano and A

    [CM21] M. Celentano and A. Montanari. CAD: Debiasing the Lasso with inaccurate covariate model. arXiv preprint arXiv:2107.14172,

    work page arXiv

  20. [20]

    [CMZZ25] S. Chen, T. Misiakiewicz, I. Zadik, and P. Zhang. An optimized franz-parisi criterion and its equivalence with sq lower bounds.arXiv preprint arXiv:2506.06259,

    work page arXiv

  21. [21]

    Castillo, J

    49 [CSHvdV15] I. Castillo, J. Schmidt-Hieber, and A. W. van der Vaart. Bayesian linear regression with sparse priors.The Annals of Statistics, 43(5):1986–2018,

    1986

  22. [22]

    Castillo and A

    [CvdV12] I. Castillo and A. W. van der Vaart. Needles and straw in a haystack: Posterior concentration for possibly sparse sequences.The Annals of Statistics, 40(4):2069–2101,

    2069

  23. [23]

    & Wainwright, M

    [CW23] M. Celentano and M. J. Wainwright. Challenges of the inconsistency regime: Novel debiasing methods for missing data models.arXiv preprint arXiv:2309.01362,

    work page arXiv

  24. [24]

    [CWL`25] X. Cai, W. Wang, F. Liu, T. Liu, G. Niu, and M. Sugiyama. Reinforcement learning with verifiable yet noisy rewards under imperfect verifiers.arXiv preprint arXiv:2510.00915,

    work page internal anchor Pith review Pith/arXiv arXiv

  25. [25]

    Decruyenaere, H

    [DDR`23] A. Decruyenaere, H. Dehaene, P. Rabaey, C. Polet, J. Decruyenaere, S. Vansteelandt, and T. Demeester. The real deal behind the artificial appeal: Inferential utility of tabular syn- thetic data.arXiv preprint arXiv:2312.07837,

    work page arXiv

  26. [26]

    [DFH`15] C

    arXiv:2411.04216. [DFH`15] C. Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and A. Roth. The reusable holdout: Preserving validity in adaptive data analysis.Science, 349(6248):636–638,

    work page arXiv

  27. [27]

    Diakonikolas, D

    [DKLP25b] I. Diakonikolas, D. M. Kane, S. Liu, and T. Pittas. PTF testing lower bounds for non- Gaussian component analysis.arXiv preprint arXiv:2511.19398,

    work page arXiv

  28. [28]

    Diakonikolas, D

    [DKP26] I. Diakonikolas, D. M. Kane, and T. Pittas. High-dimensional Gaussian mean estimation under realizable contamination.arXiv preprint arXiv:2603.16798,

    work page arXiv

  29. [29]

    What is the objective of reasoning with reinforcement learning? arXiv preprint arXiv:2510.13651, 2025

    [DR25] D. Davis and B. Recht. What is the objective of reasoning with reinforcement learning? arXiv preprint arXiv:2510.13651,

    work page arXiv

  30. [30]

    Toy Models of Superposition

    [EHO`22] N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wat- tenberg, and C. Olah. Toy models of superposition.arXiv preprint arXiv:2209.10652,

    work page internal anchor Pith review Pith/arXiv arXiv

  31. [31]

    Asymptotic behavior of unregularized and ridge-regularized high-dimensional robust regression estimators : rigorous results

    [EK13] N. El Karoui. Asymptotic behavior of unregularized and ridge-regularized high-dimensional robust regression estimators: Rigorous results.arXiv preprint arXiv:1311.2445,

    work page internal anchor Pith review Pith/arXiv arXiv

  32. [32]

    [FGLX25] Y. Fan, L. Gao, J. Lv, and X. Xu. Asymptotic FDR control with model-X knockoffs: Is moments matching sufficient?arXiv preprint arXiv:2502.05969,

    work page arXiv

  33. [33]

    Localizing Model Behavior with Path Patching

    53 [GDMSA23] N. Goldowsky-Dill, C. MacLeod, L. Sato, and A. Arora. Localizing model behavior with path patching.arXiv preprint arXiv:2304.05969,

    work page internal anchor Pith review Pith/arXiv arXiv

  34. [34]

    [GGHVDM19] C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten. Certified data removal from machine learning models.arXiv preprint arXiv:1911.03030,

    work page arXiv 1911

  35. [35]

    [GGWM24] Y. Guo, S. Ghosh, H. Weng, and A. Maleki. A note on the minimax risk of sparse linear regression.arXiv preprint arXiv:2405.05344,

    work page internal anchor Pith review Pith/arXiv arXiv

  36. [36]

    Ghosh, Y

    [GGWM25] S. Ghosh, Y. Guo, H. Weng, and A. Maleki. Signal-to-noise ratio aware minimax analysis of sparse linear regression.arXiv preprint arXiv:2501.13323,

    work page arXiv

  37. [37]

    Universality of first-order methods on random and deterministic matrices

    [GJKP26] N. Gorini, C. Jones, D. Kunisky, and L. Pesenti. Universality of first-order methods on random and deterministic matrices.arXiv preprint arXiv:2604.11729,

    work page internal anchor Pith review Pith/arXiv arXiv

  38. [38]

    Gan and W

    [GL23] F. Gan and W. Liang. Prediction de-correlated inference.arXiv preprint arXiv:2312.06478,

    work page arXiv

  39. [39]

    Gu and D

    [GX24] Y. Gu and D. Xia. Local prediction-powered inference.arXiv preprint arXiv:2409.18321,

    work page arXiv

  40. [40]

    [GYZ`25] D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    work page internal anchor Pith review Pith/arXiv arXiv

  41. [41]

    [Han25b] Q. Han. Long-time dynamics and universality of nonconvex gradient descent.arXiv preprint arXiv:2509.11426,

    work page arXiv

  42. [42]

    Training Compute-Optimal Large Language Models

    [HBM`22] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. Training compute-optimal large language models.arXiv preprint arXiv:2203.15556,

    work page internal anchor Pith review Pith/arXiv arXiv

  43. [43]

    [HGL`24] Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey.arXiv preprint arXiv:2403.14608,

    work page internal anchor Pith review Pith/arXiv arXiv

  44. [44]

    Hsieh, D

    [HKK`26] J. Hsieh, D. M Kane, P. K. Kothari, J. Li, S. Mohanty, and S. Tiegel. Rigorous implications of the low-degree heuristic.arXiv preprint arXiv:2601.05850,

    work page arXiv

  45. [45]

    Hu and Y

    [HL22] H. Hu and Y. M. Lu. Universality laws for high-dimensional learning with random features. IEEE Transactions on Information Theory, 69(3):1932–1964,

    1932

  46. [46]

    [HLM24] H. Hu, Y. M. Lu, and T. Misiakiewicz. Asymptotics of random feature regression beyond the linear scaling regime.arXiv preprint arXiv:2403.08160,

    work page arXiv

  47. [47]

    [HSS26] Y. Han, A. Shetty, and J. Shkrob. An empirical Bayes perspective on heteroskedastic mean estimation.arXiv preprint arXiv:2603.13499,

    work page arXiv

  48. [48]

    Heckel, M

    [HST26] R. Heckel, M. Soltanolkotabi, and C. Thramboulidis. Asymmetric prompt weighting for reinforcement learning with verifiable rewards.arXiv preprint arXiv:2602.11128,

    work page arXiv

  49. [49]

    Jayaraman and D

    [JE19] B. Jayaraman and D. Evans. Evaluating differentially private machine learning in practice. In28th USENIX Security Symposium, pages 1895–1912,

    1912

  50. [50]

    [JSM`23] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al. Mistral 7b. arxiv.arXiv preprint arXiv:2310.06825, 10:3,

    work page internal anchor Pith review Pith/arXiv arXiv

  51. [51]

    [KKR25] J. Kim, J. Kim, and E. K. Ryu. LoRA training provably converges to a low-rank global minimum or it fails loudly (but it probably won’t fail). InInternational Conference on Machine Learning, volume 2025,

    2025

  52. [52]

    Scaling Laws for Neural Language Models

    [KMH`20] J. Kaplan, S. McCandlish, T. Henighan, T. B Brown, B. Chess, R. Child, S. Gray, A. Rad- ford, J. Wu, and D. Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361,

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  53. [53]

    arXiv preprint arXiv:2503.21968 , year=

    [KS25] N. Keret and A. Shojaie. GLM inference with AI-generated synthetic data using misspecified linear regression.arXiv preprint arXiv:2503.21968,

    work page arXiv

  54. [54]

    [Kun23] D. Kunisky. Generic MANOVA limit theorems for products of projections.arXiv preprint arXiv:2301.09543,

    work page arXiv

  55. [55]

    Notes on Computational Hardness of Hypothesis Testing: Predictions using the Low-Degree Likelihood Ratio

    [KWB19] D. Kunisky, A. S. Wein, and A. S. Bandeira. Notes on computational hardness of hypothesis testing: Predictions using the low-degree likelihood ratio.CoRR, abs/1907.11636,

    work page internal anchor Pith review Pith/arXiv arXiv 1907

  56. [56]

    Luong and L

    [LC26] H. Luong and L. Chen. Why LoRA fails to forget: Regularized low-rank adaptation against backdoors in language models.arXiv preprint arXiv:2601.06305,

    work page arXiv

  57. [57]

    [LCL22a] S. Li, T. T. Cai, and H. Li. Estimation and inference with proxy data and its genetic applications.arXiv preprint arXiv:2201.03727,

    work page arXiv

  58. [58]

    Li and W

    [LF21] X. Li and W. Fithian. Whiteout: When do fixed-X knockoffs fail?arXiv preprint arXiv:2107.06388,

    work page arXiv

  59. [59]

    Tulu 3: Pushing Frontiers in Open Language Model Post-Training

    [LMP`24] N. Lambert, J. Morrison, V. Pyatkin, S. Huang, H. Ivison, F. Brahman, L. J. V. Miranda, A. Liu, N. Dziri, S. Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124,

    work page internal anchor Pith review Pith/arXiv arXiv

  60. [60]

    Li and P

    [LS23] Y. Li and P. Sur. Spectrum-aware debiasing: A modern inference framework with applications to principal components regression.arXiv preprint arXiv:2309.07810,

    work page arXiv

  61. [61]

    [LWB25] L. Lin, J. Wu, and P. L. Bartlett. Improved scaling laws in linear regression via data reuse. arXiv preprint arXiv:2506.08415,

    work page arXiv

  62. [62]

    [LWGH13] J. Liu, C. Wang, J. Gao, and J. Han. Multi-view clustering via joint nonnegative matrix factorization. InProceedings of the 2013 SIAM International Conference on Data Mining, pages 252–260. SIAM,

    2013

  63. [63]

    [LZVLP25] M. I. Letey, J. A. Zavatone-Veth, Y. M. Lu, and C. Pehlevan. Pretrain-test task alignment governs generalization in in-context learning.arXiv preprint arXiv:2509.26551,

    work page arXiv

  64. [64]

    [Mal11] C. Male. Traffic distributions and independence: Permutation invariant random matrices and the three notions of independence.arXiv preprint arXiv:1111.4662,

    work page internal anchor Pith review Pith/arXiv arXiv

  65. [65]

    Mousavi-Hosseini and M

    [MHE26] A. Mousavi-Hosseini and M. A. Erdogdu. Post-training with policy gradients: Optimality and the base model barrier.arXiv preprint arXiv:2603.06957,

    work page arXiv

  66. [66]

    The Natural Language Decathlon: Multitask Learning as Question Answering

    [MKXS18] B. McCann, Nitish S. Keskar, C. Xiong, and R. Socher. The natural language decathlon: Multitask learning as question answering.arXiv preprint arXiv:1806.08730,

    work page internal anchor Pith review Pith/arXiv arXiv

  67. [67]

    Miao and Q

    [ML24] J. Miao and Q. Lu. Task-agnostic machine learning-assisted inference.arXiv preprint arXiv:2405.20039,

    work page arXiv

  68. [68]

    Mahdavi, R

    [MLT24] S. Mahdavi, R. Liao, and C. Thrampoulidis. Revisiting the equivalence of in-context learning and gradient descent: The impact of data distribution. InICASSP 2024-2024 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing, pages 7410–7414. IEEE,

    2024

  69. [69]

    [MMW`23] J. Miao, X. Miao, Y. Wu, J. Zhao, and Q. Lu. Assumption-lean and data-adaptive post- prediction inference.arXiv preprint arXiv:2311.14220,

    work page arXiv

  70. [70]

    Ma and L

    [MP17] J. Ma and L. Ping. Orthogonal AMP.IEEE Access, 5:2020–2033,

    2020

  71. [71]

    A solvable model of neural scaling laws.arXiv preprint arXiv:2210.16859, 2022

    [MRS22] A. Maloney, D. A. Roberts, and J. Sully. A solvable model of neural scaling laws.arXiv preprint arXiv:2210.16859,

    work page arXiv

  72. [72]

    Montanari, F

    [MRSS23] A. Montanari, F. Ruan, B. Saeed, and Y. Sohn. Universality of max-margin classifiers.arXiv preprint arXiv:2310.00176,

    work page arXiv

  73. [73]

    [MVB`24] T. Ma, K. A. Verchand, T. B. Berrett, T. Wang, and R. J. Samworth. Estimation beyond missing (completely) at random.arXiv preprint arXiv:2410.10704,

    work page internal anchor Pith review Pith/arXiv arXiv

  74. [74]

    [MZH`25] A. M. Mason, V. Zuber, G. Hemani, E. Raffetti, Y. Xu, A. H. W. Chong, B. Woolf, E. Allara, D. Gill, O. Soremekun, and S. Burgess. Mendelian randomization in a multi-ancestry world: Reflections and practical advice.arXiv preprint arXiv:2510.17554,

    work page arXiv

  75. [75]

    In-context Learning and Induction Heads

    [OEN`22] C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, S. John- ston, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Ka- plan, S. McCandlish, and C. Olah. In-context learning and induction heads.arX...

    work page internal anchor Pith review Pith/arXiv arXiv

  76. [76]

    New Null Space Results and Recovery Thresholds for Matrix Rank Minimization

    [OH10] S. Oymak and B. Hassibi. New null space results and recovery thresholds for matrix rank minimization.arXiv preprint arXiv:1011.6326,

    work page internal anchor Pith review Pith/arXiv arXiv

  77. [77]

    [PGI`23] S. M. Park, K. Georgiev, A. Ilyas, G. Leclerc, and A. Madry. Trak: Attributing model behavior at scale.arXiv preprint arXiv:2303.14186,

    work page arXiv

  78. [78]

    [PLPT24] C. F. Park, E. S. Lubana, I. Pres, and H. Tanaka. Competition dynamics shape algorithmic phases of in-context learning.arXiv preprint arXiv:2412.01003,

    work page arXiv

  79. [79]

    Open Problems in Mechanistic Interpretability

    [SCB`25] L. Sharkey, B. Chughtai, J. Batson, J. Lindsey, J. Wu, L. Bushnaq, N. Goldowsky-Dill, S. Heimersheim, A. Ortega, J. Bloom, S. Biderman, A. Garriga-Alonso, A. Conmy, N. Nanda, J. Rumbelow, M. Wattenberg, N. Schoots, J. Miller, E. J. Michaud, S. Casper, M. Tegmark, W. Saunders, D. Bau, E. Todd, A. Geiger, M. Geva, J. Hoogland, D. Murfet, and T. McG...

    work page internal anchor Pith review Pith/arXiv arXiv

  80. [80]

    [SL19] W. W. Sun and L. Li. Dynamic tensor clustering.Journal of the American Statistical Association, 114(528):1894–1907,

    1907

Showing first 80 references.