REVIEW 2 major objections 1 minor 72 references
Auto-Relate recovers reliable functional relationships in tables by mining candidates then verifying them with three statistical tests.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-06-27 20:31 UTC pith:WNKFKXYY
load-bearing objection Auto-Relate unifies three relationship types under four reliability criteria and three tests, but the 6,414 ground-truth labels from 58k tables are the unverified load-bearing piece. the 2 major comments →
Auto-Relate: A Unified Approach to Discovering Reliable Functional Relationships Leveraging Statistical Tests
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Auto-Relate is a mine-then-verify framework that first generates candidate functional relationships across arithmetic, string, and dependency types and then verifies the remaining reliability criteria through a Minimality Test, a Perturbation Test, and an Independence Test. Three efficiency optimizations further prune the search space. When evaluated on a benchmark constructed from 58,679 real-world tables that contains 6,414 ground-truth functional relationships, the system records an average PR-AUC of 0.87 and exceeds the strongest competing baseline by 59 percent across all tested settings.
What carries the argument
The mine-then-verify framework that first produces candidate functional relationships and then filters them with a Minimality Test for atomicity, a Perturbation Test for stability, and an Independence Test for integrity.
Load-bearing premise
The 6,414 ground-truth functional relationships extracted from the 58,679 real-world tables are correctly labeled and representative enough that the statistical tests can separate them from spurious candidates.
What would settle it
A new collection of tables in which the functional relationships labeled as ground truth by the benchmark construction process are shown by manual inspection to be mostly spurious, or in which Auto-Relate's PR-AUC falls substantially below the reported 0.87 value.
If this is right
- Recovered functional relationships can directly support table understanding, data-quality repair, and provenance tracking without manual inspection.
- The four reliability criteria give a uniform way to judge any inter-column relationship regardless of whether it is arithmetic, string-based, or a dependency.
- The three statistical tests and the associated early-rejection bounds allow the approach to scale to tables with tens of thousands of rows.
- The released benchmark supplies a common testbed that future methods can use to measure progress on reliable relationship discovery.
Where Pith is reading between the lines
- The same verification tests could be applied to relationships discovered across multiple related tables rather than within a single table.
- Replacing the current candidate generator with a learned model might further raise recall while still relying on the statistical tests for precision.
- The stability and independence tests might also flag relationships that break under distribution shift when tables are updated over time.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces functional relationships (FRs) as a unified concept subsuming arithmetic relationships, string transformations, and functional dependencies in tables. It defines four reliability criteria (accuracy, atomicity, stability, integrity) and presents Auto-Relate, a mine-then-verify framework that generates candidates and verifies them via Minimality, Perturbation, and Independence Tests, with optimizations for efficiency. A benchmark is constructed from 58,679 real-world tables yielding 6,414 ground-truth FRs; experiments against 18 baselines report average PR-AUC of 0.87 (59% higher than the best baseline).
Significance. If the benchmark construction and labeling process can be shown to be independent and unbiased, the work would offer a practically useful advance in table understanding, data quality, and provenance by providing a statistically grounded method for reliable relationship discovery. The empirical scale of the benchmark and the three-test verification approach are positive elements.
major comments (2)
- [Benchmark construction (likely §4 or §5)] Benchmark construction section: The central performance claim (PR-AUC 0.87, 59% above baselines) rests on the 6,414 ground-truth FRs being correctly labeled and representative. The manuscript provides no description of the extraction heuristics, independent validation, or inter-annotator agreement for these labels, nor any check that the labeling process does not overlap with the proposed statistical tests. This is load-bearing for the empirical results.
- [Experiments / Evaluation] Evaluation section: No ablation study, error analysis, or per-test contribution breakdown is reported for the three statistical tests (Minimality, Perturbation, Independence). Without this, it is impossible to determine whether the reported gains are driven by the full framework or by one component, weakening the claim that the unified approach is necessary.
minor comments (1)
- [Abstract] The abstract states results 'across all settings' but does not define the settings or reference the corresponding table/figure; this should be clarified for reproducibility.
Simulated Author's Rebuttal
We thank the referee for their constructive feedback and positive assessment of the work's significance. We address each major comment below and commit to revisions where appropriate.
read point-by-point responses
-
Referee: [Benchmark construction (likely §4 or §5)] Benchmark construction section: The central performance claim (PR-AUC 0.87, 59% above baselines) rests on the 6,414 ground-truth FRs being correctly labeled and representative. The manuscript provides no description of the extraction heuristics, independent validation, or inter-annotator agreement for these labels, nor any check that the labeling process does not overlap with the proposed statistical tests. This is load-bearing for the empirical results.
Authors: We agree that additional details on the benchmark construction are necessary to substantiate the empirical claims. In the revised manuscript, we will add a dedicated subsection detailing the extraction heuristics employed to generate the ground-truth FRs, the independent validation procedures used, any inter-annotator agreement statistics, and explicit verification that the labeling process does not rely on or overlap with the Minimality, Perturbation, or Independence Tests. This revision will address concerns about potential bias or circularity in the evaluation. revision: yes
-
Referee: [Experiments / Evaluation] Evaluation section: No ablation study, error analysis, or per-test contribution breakdown is reported for the three statistical tests (Minimality, Perturbation, Independence). Without this, it is impossible to determine whether the reported gains are driven by the full framework or by one component, weakening the claim that the unified approach is necessary.
Authors: We concur that an ablation analysis would strengthen the evaluation by isolating the contribution of each statistical test. We will revise the Experiments section to include: (i) an ablation study measuring PR-AUC when each test is removed individually, (ii) a breakdown of how many candidates are filtered by each test, and (iii) an error analysis on cases where the full framework succeeds or fails. These additions will clarify the necessity of the combined approach. revision: yes
Circularity Check
No significant circularity in empirical benchmark-driven method
full rationale
The paper presents an empirical mine-then-verify framework (Minimality Test, Perturbation Test, Independence Test) evaluated on a benchmark of 6,414 ground-truth FRs extracted from 58,679 real-world tables. Performance is reported via PR-AUC comparisons to 18 baselines with no equations, derivations, or fitted parameters that reduce the reported metrics to inputs by construction. No self-citation load-bearing steps, uniqueness theorems, or ansatzes are referenced in the abstract or description. The central claims rest on external benchmark construction and statistical testing rather than self-referential reduction, making the derivation self-contained.
Axiom & Free-Parameter Ledger
read the original abstract
Tables in spreadsheets, computational notebooks, and databases often contain rich inter-column relationships. Yet these relationships are typically implicit and are often lost when tables are exported to standard formats. Recovering them can benefit downstream tasks, including table understanding, data quality improvement, and provenance analysis. However, simply mining relationships that hold on an observed table is insufficient, as many are spurious due to coincidence, redundancy, or limited data diversity. In this paper, we introduce functional relationships (FRs) as a unified notion for inter-column relationships in tables, subsuming arithmetic relationships, string transformations, and functional dependencies. We characterize FR reliability through four complementary criteria: accuracy, atomicity, stability, and integrity. Guided by these criteria, we propose Auto-Relate, a mine-then-verify framework that first generates accurate candidate FRs and then verifies the remaining reliability criteria through a Minimality Test, a Perturbation Test, and an Independence Test, respectively. To further improve efficiency, we develop three optimization strategies, including a group-by lower bound for early rejection, a closed-form speedup for arithmetic FRs, and a binomial bound for statistically guided early termination. We construct a large-scale benchmark suite from 58,679 real-world spreadsheets and relational tables, containing 6,414 ground-truth FRs spanning all three FR types. Extensive experiments against 18 baselines show that Auto-Relate consistently achieves the best performance, with an average PR-AUC of 0.87, 59% higher than the best competing baseline across all settings.
Figures
Reference graph
Works this paper leans on
-
[1]
Ilyas, Mourad Ouzzani, Paolo Papotti, Michael Stonebraker, and Nan Tang
Ziawasch Abedjan, Xu Chu, Dong Deng, Raul Castro Fernandez, Ihab F. Ilyas, Mourad Ouzzani, Paolo Papotti, Michael Stonebraker, and Nan Tang. 2016. De- tecting data errors: where are we and what needs to be done?PVLDB9, 12 (2016), 993–1004
2016
-
[2]
Ziawasch Abedjan, John Morcos, Ihab F Ilyas, Mourad Ouzzani, Paolo Papotti, and Michael Stonebraker. 2016. Dataxformer: A robust transformation discovery system. InICDE. 1134–1145
2016
-
[3]
Ziawasch Abedjan, Patrick Schulze, and Felix Naumann. 2014. DFD: Efficient functional dependency discovery. InCIKM. 949–958
2014
-
[4]
Daniel W Barowy, Emery D Berger, and Benjamin Zorn. 2018. ExceLint: au- tomatically finding spreadsheet formula errors.PACMPL2, OOPSLA (2018), 1–26
2018
-
[5]
Rohan Bavishi, Caroline Lemieux, Roy Fox, Koushik Sen, and Ion Stoica. 2019. Au- toPandas: neural-backed generators for program synthesis.PACMPL3, OOPSLA (2019), 1–27
2019
-
[6]
Laure Berti-Équille, Hazar Harmouch, Felix Naumann, Noël Novelli, and Sara- vanan Thirumuruganathan. 2018. Discovery of genuine functional dependencies from relational data with missing values.PVLDB11, 8 (2018), 880–892
2018
-
[7]
Tobias Bleifuß, Thorsten Papenbrock, Thomas Bläsius, Martin Schirneck, and Felix Naumann. 2024. Discovering functional dependencies through hitting set enumeration.PACMMOD2, 1 (2024), 1–24
2024
-
[8]
Alex Bogatu, Norman W Paton, Alvaro AA Fernandes, and Martin Koehler. 2019. Towards automatic data format transformations: data wrangling at scale.Comput. J.62, 7 (2019), 1044–1060
2019
-
[9]
Philip Bohannon, Wenfei Fan, Floris Geerts, Xibei Jia, and Anastasios Kementsi- etsidis. 2007. Conditional functional dependencies for data cleaning. InICDE. 746–755
2007
-
[10]
Peter Buneman, Sanjeev Khanna, and Tan Wang-Chiew. 2001. Why and where: A characterization of data provenance. InICDT. 316–330
2001
-
[11]
Loredana Caruccio, Vincenzo Deufemia, Felix Naumann, and Giuseppe Polese
-
[12]
Discovering Relaxed Functional Dependencies Based on Multi-Attribute Dominance.TKDE33, 9 (2021), 3212–3228
2021
-
[13]
Loredana Caruccio, Vincenzo Deufemia, and Giuseppe Polese. 2016. Relaxed functional dependencies—a survey of approaches.TKDE28, 1 (2016), 147–165
2016
-
[14]
Roger Cavallo and Michael Pittarelli. 1987. The theory of probabilistic databases.. InPVLDB. 71–81
1987
-
[15]
Jie Cheng, David A Bell, and Weiru Liu. 1997. Learning belief networks from data: An information theory based approach. InCIKM. 325–331
1997
-
[16]
Shing-Chi Cheung, Wanjun Chen, Yepang Liu, and Chang Xu. 2016. CUSTODES: automatic spreadsheet cell clustering and smell detection using strong and weak features. InICSE. 464–475
2016
-
[17]
Xu Chu, Ihab F Ilyas, Sanjay Krishnan, and Jiannan Wang. 2016. Data cleaning: Overview and emerging challenges. InSIGMOD. 2201–2206
2016
-
[18]
Xu Chu, Ihab F Ilyas, and Paolo Papotti. 2013. Discovering denial constraints. PVLDB6, 13 (2013), 1498–1509
2013
-
[19]
Yingwei Cui and Jennifer Widom. 2000. Practical lineage tracing in data ware- houses. InICDE. 367–378
2000
-
[20]
Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu. 2020. TURL: table understanding through representation learning.PVLDB14, 3 (2020), 307–319
2020
-
[21]
Xiaoou Ding, Yixing Lu, Hongzhi Wang, Chen Wang, Yida Liu, and Jianmin Wang. 2024. Dafdiscover: Robust mining algorithm for dynamic approximate functional dependencies on dirty data.PVLDB17, 11 (2024), 3484–3496
2024
-
[22]
Wenfei Fan, Floris Geerts, Xibei Jia, and Anastasios Kementsietsidis. 2008. Con- ditional functional dependencies for capturing data inconsistencies.TODS33, 2 (2008), 1–48
2008
-
[23]
Wenfei Fan, Ziyan Han, Yaoshu Wang, and Min Xie. 2022. Parallel Rule Discovery from Large Datasets by Sampling. InSIGMOD. 384–398
2022
-
[24]
Wenfei Fan, Ziyan Han, Yaoshu Wang, and Min Xie. 2023. Discovering top-k rules using subjective and objective criteria.PACMMOD1, 1 (2023), 1–29
2023
-
[25]
Wenfei Fan, Ziyan Han, Min Xie, and Guangyi Zhang. 2024. Discovering top-k relevant and diversified rules.PACMMOD2, 4 (2024), 1–28
2024
-
[26]
Chris Giannella and Edward Robertson. 2004. On approximation measures for functional dependencies.Information Systems29, 6 (2004), 483–507
2004
-
[27]
1979.Measures of association for cross classifications
Leo A Goodman and William H Kruskal. 1979.Measures of association for cross classifications. Springer
1979
-
[28]
Sumit Gulwani. 2011. Automating string processing in spreadsheets using input- output examples.ACM Sigplan Notices46, 1 (2011), 317–330
2011
-
[29]
Ziyan Han, Wanjia Chen, Yunpeng Han, Rui Mao, and Jianbin Qin. 2026. Fast Diversified Top-k Rule Discovery via User-Guided Embeddings.TKDE38, 3 (2026), 1739–1753
2026
-
[30]
William R Harris and Sumit Gulwani. 2011. Spreadsheet table transformations from examples.ACM SIGPLAN Notices46, 6 (2011), 317–328
2011
-
[31]
Yeye He, Xu Chu, Kris Ganjam, Yudian Zheng, Vivek Narasayya, and Surajit Chaudhuri. 2018. Transform-data-by-example (TDE) an extensible search engine for data transformations.PVLDB11, 10 (2018), 1165–1177
2018
-
[32]
Yka Huhtala, Juha Kärkkäinen, Pasi Porkka, and Hannu Toivonen. 1999. TANE: An efficient algorithm for discovering functional and approximate dependencies. The computer journal42, 2 (1999), 100–111
1999
-
[33]
Madelon Hulsebos, Kevin Hu, Michiel Bakker, Emanuel Zgraggen, Arvind Satya- narayan, Tim Kraska, Çagatay Demiralp, and César Hidalgo. 2019. Sherlock: A Deep Learning Approach to Semantic Data Type Detection. InSIGKDD. 1500–1508
2019
-
[34]
Ihab F Ilyas, Volker Markl, Peter Haas, Paul Brown, and Ashraf Aboulnaga. 2004. CORDS: Automatic discovery of correlations and soft functional dependencies. InSIGMOD. 647–658
2004
-
[35]
Zhongjun Jin, Michael R Anderson, Michael Cafarella, and HV Jagadish. 2017. Foofah: Transforming data by example. InSIGMOD. 683–698
2017
-
[36]
Zhongjun Jin, Michael Cafarella, HV Jagadish, Sean Kandel, Michael Minar, and Joseph M Hellerstein. 2019. CLX: Towards verifiable PBE data transformation. EDBT(2019), 265–276
2019
-
[37]
Jyrki Kivinen and Heikki Mannila. 1995. Approximate inference of functional dependencies from relations.Theoretical Computer Science149, 1 (1995), 129–149
1995
-
[38]
Patrick Koch, Konstantin Schekotihin, Dietmar Jannach, Birgit Hofer, and Franz Wotawa. 2021. Metric-based fault prediction for spreadsheets.TSE47, 10 (2021), 2195–2207
2021
-
[39]
Ioannis Koumarelas, Thorsten Papenbrock, and Felix Naumann. 2020. MDedup: Duplicate detection with matching dependencies.PVLDB13, 5 (2020), 712–725
2020
-
[40]
Sebastian Kruse and Felix Naumann. 2018. Efficient discovery of approximate dependencies.PVLDB11, 7 (2018), 759–772
2018
-
[41]
Mengran Li, Zijing Tan, Honghui Yang, and Shuai Ma. 2025. Efficient Discovery of Relaxed Functional Dependencies.PVLDB18, 7 (2025), 2044–2056
2025
-
[42]
Girija Limaye, Sunita Sarawagi, and Soumen Chakrabarti. 2010. Annotating and searching web tables using entities, types and relationships.PVLDB3, 1-2 (2010), 1338–1347
2010
-
[43]
Mohammad Mahdavi and Ziawasch Abedjan. 2020. Baran: Effective Error Cor- rection via a Unified Context Representation and Transfer Learning.PVLDB13, 11 (2020), 1948–1961
2020
-
[44]
Mohammad Mahdavi, Ziawasch Abedjan, Raul Castro Fernandez, Samuel Mad- den, Mourad Ouzzani, Michael Stonebraker, and Nan Tang. 2019. Raha: A Configuration-Free Error Detection System. InSIGMOD. 865–882
2019
-
[45]
Panagiotis Mandros, Mario Boley, and Jilles Vreeken. 2017. Discovering reliable approximate functional dependencies. InSIGKDD. 355–363
2017
-
[46]
Panagiotis Mandros, Mario Boley, and Jilles Vreeken. 2018. Discovering reliable dependencies from data: Hardness and improved algorithms. InICDM. 317–326
2018
-
[47]
Panagiotis Mandros, Mario Boley, and Jilles Vreeken. 2020. Discovering depen- dencies with reliable mutual information.KAIS62, 11 (2020), 4223–4253
2020
-
[48]
Panagiotis Mandros, David Kaltenpoth, Mario Boley, and Jilles Vreeken. 2020. Discovering functional dependencies from mixed-type data. InSIGKDD. 1404– 1414
2020
-
[49]
Felix Naumann. 2014. Data profiling revisited.SIGMOD Record42, 4 (2014), 40–49
2014
-
[50]
Noel Novelli and Rosine Cicchetti. 2001. Fun: An efficient algorithm for mining functional and embedded dependencies. InICDT. 189–203
2001
-
[51]
OpenAI. 2025. Introducing GPT-5. https://openai.com/index/introducing-gpt-5/. Accessed: 2026-04-29
2025
-
[52]
Raymond R Panko. 1998. What we know about spreadsheet errors.JOEUC10, 2 (1998), 15–21
1998
-
[53]
Thorsten Papenbrock, Jens Ehrlich, Jannik Marten, Tommy Neubert, Jan-Peer Rudolph, Martin Schönberg, Jakob Zwiener, and Felix Naumann. 2015. Functional dependency discovery: An experimental evaluation of seven algorithms.PVLDB 8, 10 (2015), 1082–1093
2015
-
[54]
Marcel Parciak, Sebastiaan Weytjens, Niel Hens, Frank Neven, Liesbet M Peeters, and Stijn Vansummeren. 2024. Measuring Approximate Functional Dependencies: a Comparative Study. InICDE. 3505–3518
2024
-
[55]
Karl Pearson. 1900. On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling.Philos. Mag.50, 302 (1900), 157–175
1900
-
[56]
Eduardo H. M. Pena, Eduardo Cunha de Almeida, and Felix Naumann. 2019. Discovery of Approximate (and Exact) Denial Constraints.PVLDB13, 3 (2019), 266–278
2019
-
[57]
Frédéric Pennerath, Panagiotis Mandros, and Jilles Vreeken. 2020. Discovering approximate functional dependencies using smoothed mutual information. In SIGKDD. 1254–1264
2020
-
[58]
Gregory Piatetsky-Shapiro and C Matheus. 1993. Measuring data dependencies in large databases. InKDD. 162–173
1993
-
[59]
David Powers. 2011. Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation.JMLT2, 1 (2011), 37–63
2011
-
[60]
Erhard Rahm, Hong Hai Do, et al. 2000. Data cleaning: Problems and current approaches.IEEE Data Eng. Bull.23, 4 (2000), 3–13
2000
-
[61]
Sanjivni Rana, Junya Ogawa, Suraj Shetiya, Senjuti Basu Roy, and Gautam Das. 2025. Anytime Algorithms for Approximate Functional Dependencies. InSIGKDD. 2386–2397. 14
2025
-
[62]
Theodoros Rekatsinas, Xu Chu, Ihab F. Ilyas, and Christopher Ré. 2017. Holo- Clean: holistic data repairs with probabilistic inference.Proc. VLDB Endow.10, 11 (Aug. 2017), 1190–1201. https://doi.org/10.14778/3137628.3137631
-
[63]
Philipp Schirmer, Thorsten Papenbrock, Ioannis Koumarelas, and Felix Naumann
-
[64]
Efficient Discovery of Matching Dependencies.TODS45, 3 (2020), 1–33
2020
-
[65]
Roee Shraga and Renée J Miller. 2023. Explaining dataset changes for semantic data versioning with Explain-Da-V.PVLDB16, 6 (2023), 1587–1600
2023
-
[66]
Yoshihiko Suhara, Jinfeng Li, Yuliang Li, Dan Zhang, Çağatay Demiralp, Chen Chen, and Wang-Chiew Tan. 2022. Annotating Columns with Pre-trained Lan- guage Models. InSIGMOD. 1493–1503
2022
-
[67]
Ziheng Wei and Sebastian Link. 2023. Towards the efficient discovery of mean- ingful functional dependencies.Information Systems116 (2023), 102224
2023
-
[68]
Edwin B Wilson. 1927. Probable inference, the law of succession, and statistical inference.J. Amer. Statist. Assoc.22, 158 (1927), 209–212
1927
-
[69]
Catharine Wyss, Chris Giannella, and Edward Robertson. 2001. FastFDs: A heuristic-driven, depth-first algorithm for mining functional dependencies from relation instances. InDaWaK. 101–110
2001
-
[70]
Junwen Yang, Yeye He, and Surajit Chaudhuri. 2021. Auto-pipeline: synthesiz- ing complex data pipelines by-target using reinforcement learning and search. PVLDB14, 11 (2021), 2563–2575
2021
-
[71]
Dan Zhang, Madelon Hulsebos, Yoshihiko Suhara, Çağatay Demiralp, Jinfeng Li, and Wang-Chiew Tan. 2020. Sato: contextual semantic type detection in tables. PVLDB13, 12 (2020), 1835–1848
2020
-
[72]
Yunjia Zhang, Zhihan Guo, and Theodoros Rekatsinas. 2020. A Statistical Per- spective on Discovering Functional Dependencies in Noisy Data. InSIGMOD. 861–876. 15
2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.