Pith. sign in

REVIEW 2 major objections 2 minor 5 references

Standard accuracy metrics like AUROC overlook critical pre-deployment failures in AI decision-support systems for healthcare and other domains.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 22:04 UTC pith:IMNJUJD2

load-bearing objection RISED supplies a workable five-dimension checklist with bootstrap stats that catches AUROC-blind issues on real cohorts, but the pass/fail calls rest on fixed thresholds whose robustness is not shown. the 2 major comments →

arxiv 2605.12895 v2 pith:IMNJUJD2 submitted 2026-05-13 cs.LG cs.AIcs.CYstat.AP

RISED: A Pre-Deployment Evaluation Framework for High-Stakes AI Decision-Support Systems, with Application to Healthcare

Rohith Reddy Bellibatlu , Manpreet Singh , Yash Jajoo , Shyamal Lakhanpal , Abhishek Israni This is my paper
classification cs.LG cs.AIcs.CYstat.AP
keywords pre-deployment evaluationAI decision supporthealthcare AIreliability assessmentinclusivitysensitivity analysisbootstrap confidence intervalsfairness diagnostics
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents RISED as a framework that evaluates AI models across five dimensions using bootstrap confidence intervals and fixed thresholds to produce structured verdicts. It argues that aggregate accuracy numbers from held-out sets provide insufficient evidence for systems whose outputs clinicians act on directly. When applied to seven cohorts over 35 years, the framework identifies cases where models pass reliability checks but fail decisively on inclusivity and sensitivity. These patterns appear in both medical and non-medical prediction tasks, showing the issues are not limited to one field.

Core claim

RISED operationalises five dimensions (Reliability, Inclusivity, Sensitivity, Equity, Deployability) through BCa bootstrap 95% confidence intervals, literature-grounded thresholds, and Holm-Bonferroni-corrected PASS / FAIL / INCONCLUSIVE verdicts, with Equity treated as a proxy-dependence diagnostic. Applied to cohorts from Diabetes 130, NHIS, NHANES, BRFSS and credit/income tasks, it shows models can pass AUROC yet fail on AUC parity gaps up to 0.262 and threshold-flip rates up to 64.2%, with failures reproduced across datasets and confirmed as data-driven rather than model-specific.

What carries the argument

The RISED evaluation procedure that converts bootstrap intervals into dimension-specific PASS/FAIL/INCONCLUSIVE verdicts using literature thresholds and multiple-testing correction.

Load-bearing premise

The chosen thresholds and operational definitions for the five dimensions are appropriate and sufficient to identify real pre-deployment risks without arbitrary cutoffs driving the reported failures.

What would settle it

A re-run of the Diabetes 130 evaluation that changes only the literature-grounded thresholds and produces entirely different PASS/FAIL verdicts on the same bootstrap intervals.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Models cleared on AUROC alone may still exhibit large subgroup performance gaps that affect different patient populations unequally.
  • High threshold sensitivity means small changes in decision cutoffs can reverse nearly half the predictions, undermining operational stability.
  • The same failure pattern appears in non-healthcare tasks, indicating the evaluation method applies beyond medicine.
  • Failures trace to data properties such as missing features after instrument changes rather than to specific model architectures.
  • Equity checks flag proxy dependence even when direct protected attributes are absent from the input.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The framework supplies the numerical evidence required by standards such as TRIPOD+AI but not prescribed by them.
  • Data collection practices that rotate instruments or drop variables can systematically trigger sensitivity failures.
  • Proxy-dependence diagnostics for Equity may need calibration against direct fairness audits in future uses.
  • Extending the same bootstrap-plus-threshold logic to continuous monitoring after deployment could catch distribution shifts earlier.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces RISED, a pre-deployment evaluation framework for high-stakes AI decision-support systems. It operationalizes five dimensions (Reliability via PSS, Inclusivity via AUC parity gap, Sensitivity via max threshold-flip rate, Equity as proxy-dependence diagnostic, Deployability) using BCa bootstrap 95% CIs, literature-grounded thresholds, and Holm-Bonferroni-corrected PASS/FAIL/INCONCLUSIVE verdicts. Applied to seven cohorts spanning healthcare (Diabetes 130, NHIS, NHANES 2021-2023, BRFSS 2024) and other domains (credit, income prediction), RISED identifies cases where AUROC passes but other dimensions fail decisively (e.g., Diabetes 130: Reliability PSS=0.0004 passes while Inclusivity gap=0.262 and Sensitivity flip rate=49.1% fail; reproduced on NHIS and BRFSS with 64.2% flip rate). Failures are attributed to data properties rather than model choice via a multi-model check. The framework is released as an open-source Python package to complement TRIPOD+AI, FUTURE-AI, and Fairlearn.

Significance. If the dimension operationalizations and thresholds prove robust to variation, RISED would supply structured numerical evidence for deployment decisions that existing aggregate metrics and guidelines do not prescribe. The multi-cohort application (n=303 to 99,492), reproduction of patterns across domains, open-source release, and explicit distinction between data-driven versus model-specific failures are concrete strengths that support reproducibility and practical utility.

major comments (2)
  1. [Abstract and §4 (cohort applications)] Abstract and results on Diabetes 130 cohort: the central claim that RISED surfaces failures invisible to AUROC rests on the specific literature-grounded cutoffs (e.g., AUC parity gap of 0.262 failing Inclusivity; max threshold-flip rate of 49.1% failing Sensitivity). The manuscript must demonstrate via sensitivity analysis that modest shifts in these cutoffs (within the cited literature ranges) do not flip the verdicts, otherwise the data-driven nature of the failures cannot be separated from threshold choice.
  2. [§3 (framework operationalization)] §3 (framework operationalization): the Holm-Bonferroni correction and PASS/FAIL/INCONCLUSIVE rules are applied uniformly across cohorts, yet no derivation or validation shows that the chosen thresholds for PSS, parity gap, and flip rate remain stable when the underlying bootstrap CIs or cohort sizes vary; this is load-bearing for the domain-agnosticity assertion.
minor comments (2)
  1. [Abstract] The abstract states that NHANES achieves INCONCLUSIVE verdicts and BRFSS produces the most severe Sensitivity failure, but does not tabulate the full set of dimension verdicts for all seven cohorts; a summary table would improve readability.
  2. [Conclusion] The open-source Python package is credited as a contribution, but the manuscript should include the repository URL or PyPI identifier to enable immediate access.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for these detailed and constructive comments, which highlight important aspects of robustness for the framework. We agree that explicit sensitivity checks will strengthen the separation of data-driven failures from threshold choice and will bolster the domain-agnosticity claim. The revised manuscript will incorporate the requested analyses in §3 and §4 (plus an appendix), while preserving the literature-grounded nature of the thresholds.

read point-by-point responses

  1. Referee: [Abstract and §4] the central claim that RISED surfaces failures invisible to AUROC rests on the specific literature-grounded cutoffs (e.g., AUC parity gap of 0.262 failing Inclusivity; max threshold-flip rate of 49.1% failing Sensitivity). The manuscript must demonstrate via sensitivity analysis that modest shifts in these cutoffs (within the cited literature ranges) do not flip the verdicts.

    Authors: We accept this point. The thresholds were selected from the cited literature ranges (e.g., parity-gap bounds from fairness studies and flip-rate tolerances from clinical decision literature), but the manuscript does not currently include a formal sensitivity sweep. In revision we will add a sensitivity analysis that perturbs each threshold by ±10–20% within the source ranges, recomputes verdicts on the Diabetes 130, NHIS, and BRFSS cohorts, and reports the fraction of verdicts that remain unchanged. Preliminary internal checks indicate that the decisive FAIL verdicts on Inclusivity and Sensitivity are stable under these perturbations; the new results and table will be placed in §4 with a brief methods note in §3. revision: yes

  2. Referee: [§3] the Holm-Bonferroni correction and PASS/FAIL/INCONCLUSIVE rules are applied uniformly across cohorts, yet no derivation or validation shows that the chosen thresholds for PSS, parity gap, and flip rate remain stable when the underlying bootstrap CIs or cohort sizes vary; this is load-bearing for the domain-agnosticity assertion.

    Authors: We agree that empirical stability across cohort sizes and CI widths is necessary to support the uniform application. The current manuscript already spans n=303 to 99,492 and reproduces the same failure pattern on multiple independent cohorts, providing indirect evidence. However, no dedicated stability experiment is reported. In revision we will add (i) a subsampling study on the largest cohorts that recomputes verdicts at reduced sample sizes matching the smaller cohorts, and (ii) a brief analytic note on the monotonicity properties of the BCa CI relative to the fixed thresholds under Holm-Bonferroni. These additions will appear in §3; we note that a full theoretical derivation of optimal thresholds would require a separate methodological paper and is outside the scope of the present work. revision: yes

Circularity Check

0 steps flagged

No circularity: framework applies external literature thresholds to computed metrics without reduction to fitted inputs or self-citation chains

full rationale

The paper defines RISED as an operational framework using BCa bootstrap CIs, literature-grounded thresholds, and Holm-Bonferroni rules to produce PASS/FAIL/INCONCLUSIVE verdicts on five dimensions. Reported results (e.g., PSS=0.0004, AUC parity gap=0.262, max flip rate=49.1%) are direct comparisons of data-derived quantities against those external thresholds on held-out cohorts. No equations or steps reduce verdicts to parameters fitted on the evaluation data itself, nor do they rely on self-citation for load-bearing uniqueness or ansatz. The multi-model check and domain-agnosticity claim rest on independent application across datasets rather than definitional equivalence. This is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

Based solely on the abstract; the framework rests on the premise that the five dimensions and literature thresholds adequately represent pre-deployment risks, but no free parameters, axioms, or invented entities are explicitly quantified or justified in the provided text.

free parameters (1)
  • literature-grounded thresholds for verdicts
    Specific cutoff values for PASS/FAIL/INCONCLUSIVE are stated to be literature-grounded but not enumerated or derived in the abstract.
axioms (1)
  • domain assumption The five dimensions Reliability, Inclusivity, Sensitivity, Equity, and Deployability are the appropriate axes for pre-deployment evaluation of high-stakes AI systems.
    The entire framework is constructed around operationalizing these five dimensions.

pith-pipeline@v0.9.1-grok · 5875 in / 1472 out tokens · 35882 ms · 2026-06-30T22:04:46.266783+00:00 · methodology

0 comments
read the original abstract

Clinical decision-support systems are expert systems whose recommendations clinicians act on directly, yet they are usually cleared on one aggregate accuracy number from a held-out test set. That number says nothing about input reliability under encoding shifts, subgroup gaps, threshold sensitivity, or operational feasibility. We present RISED, a pre-deployment evaluation framework operationalising five dimensions (Reliability, Inclusivity, Sensitivity, Equity, Deployability) through BCa bootstrap 95% confidence intervals, literature-grounded thresholds, and Holm-Bonferroni-corrected PASS / FAIL / INCONCLUSIVE verdicts; Equity is a proxy-dependence diagnostic rather than a gating test. Applied to seven cohorts spanning 35 years (n from 303 to 99,492), RISED surfaces failures invisible to AUROC: on Diabetes 130, Reliability passes by three orders of magnitude (PSS = 0.0004) while Inclusivity (AUC parity gap = 0.262) and Sensitivity (max threshold-flip rate 49.1%) fail decisively; both NHIS cohorts reproduce this. NHANES 2021-2023, with a complete feature profile, achieves INCONCLUSIVE verdicts; BRFSS 2024 produces the suite's most severe Sensitivity failure (max threshold-flip rate 64.2%) after instrument rotation removed hypertension and cholesterol. The pattern recurs on credit- and income-prediction cohorts, confirming domain-agnosticity; a multi-model check shows the failures are data-driven, not model-specific. RISED ships as an open-source Python package complementing TRIPOD+AI, FUTURE-AI, and Fairlearn with the structured numerical evidence those standards require but do not prescribe.

Figures

Figures reproduced from arXiv: 2605.12895 by Abhishek Israni, Manpreet Singh, Rohith Reddy Bellibatlu, Shyamal Lakhanpal, Yash Jajoo.

Figure 1
Figure 1. Figure 1: Reliability dimension: decision flip rates per perturbation type. Dashed line marks the 5% pass [PITH_FULL_IMAGE:figures/full_fig_p024_1.png] view at source ↗
Figure 1
Figure 1. Figure 1: RISED evaluation pipeline. Five parallel paths evaluate the model against eight gating sub-criteria [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Inclusivity dimension: subgroup AUC-ROC across race, sex, age group, and insurance subgroups. [PITH_FULL_IMAGE:figures/full_fig_p024_2.png] view at source ↗
Figure 2
Figure 2. Figure 2: Synthetic cohort — Reliability: decision flip rate per perturbation. Dashed line: 5% pass threshold. [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Sensitivity dimension: threshold flip rate sweep from [PITH_FULL_IMAGE:figures/full_fig_p025_3.png] view at source ↗
Figure 3
Figure 3. Figure 3: Synthetic cohort — Inclusivity: subgroup AUC-ROC. Age 75+ (0.923) is lowest; Race:Other [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Equity dimension: group-level need–prediction gaps using the binary outcome label as the need [PITH_FULL_IMAGE:figures/full_fig_p026_4.png] view at source ↗
Figure 4
Figure 4. Figure 4: Synthetic cohort — Sensitivity: TFR sweep [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Deployability dimension: global SHAP feature importance (rank order). Top five features: age, [PITH_FULL_IMAGE:figures/full_fig_p026_5.png] view at source ↗
Figure 5
Figure 5. Figure 5: Synthetic cohort — Equity: group need–prediction gaps (outcome-label proxy). Under the CCI [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: RISED Framework scorecard with CI-based decisions across all five dimensions for the XGBoost [PITH_FULL_IMAGE:figures/full_fig_p027_6.png] view at source ↗
Figure 6
Figure 6. Figure 6: Deployability dimension: global SHAP feature importance (mean [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: RISED scorecard for the XGBoost baseline on the [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

5 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    Angwin, J., Larson, J., Mattu, S., and Kirchner, L. (2016). Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks. ProPublica. Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., and Mooney, C. (2021). Current challenges and future opportunities for XAI in machine learning-...

    2016

  2. [2]

    E., Pompei, P., Ales, K

    Charlson, M. E., Pompei, P., Ales, K. L., and MacKenzie, C. R. (1987). A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.Journal of Chronic Diseases, 40(5):373–383. Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Singh, R., Shaban, M., and Mahmood, F. (2023). Algorithmic fairne...

    1987

  3. [3]

    Auto-downloaded by external_validation_nhis2023_diabetes.py

    Outcome used in this paper: physician-diagnosed diabetes (DIBEV_A). Auto-downloaded by external_validation_nhis2023_diabetes.py. National Center for Health Statistics (2025). National health interview survey, 2024 public-use data file (Sample Adult). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Sample Adult int...

    2025

  4. [4]

    Artificial intelligence risk management framework (AI RMF 1.0)

    National Institute of Standards and Technology (2023). Artificial intelligence risk management framework (AI RMF 1.0). Technical Report NIST AI 100-1, U.S. Department of Commerce. Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. W.-C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., and Mathur, P. (2023). Bias in artif...

    2023

  5. [5]

    Aequitas: A Bias and Fairness Audit Toolkit

    Osibogun, O. (2024). Adverse childhood experiences and suboptimal self-rated health in adulthood: Exploring effect modification by age, sex and race/ethnicity.American Journal of Health Promotion, 39(2):244–252. Paleyes, A., Urma, R.-G., and Lawrence, N. D. (2022). Challenges in deploying machine learning: A survey of case studies.ACM Computing Surveys, 5...

    work page internal anchor Pith review Pith/arXiv arXiv 2024