pith. sign in

arxiv: 2605.02010 · v2 · pith:BY5FSUMEnew · submitted 2026-05-03 · 💻 cs.AI

Reliable AI Needs to Externalize Implicit Knowledge: A Human-AI Collaboration Perspective

Hengyu Liu , Tianyi Li , Zhihong Cui , Yushuai Li , Zhangkai Wu , Torben Bach Pedersen , Kristian Torp , Christian S. Jensen This is my paper

Pith reviewed 2026-07-01 00:10 UTC · model grok-4.3

classification 💻 cs.AI
keywords reliable AIimplicit knowledgehuman validationknowledge objectsAI reliabilityexternalizationhuman-AI collaboration
0
0 comments X

The pith

Reliable AI requires infrastructure for humans to validate the implicit knowledge that current systems acquire without oversight.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AI learns from explicit sources like papers and databases as well as implicit sources like reasoning patterns and debugging steps, but only explicit knowledge can be checked against sources today. The paper argues this creates a fundamental gap because the most useful AI capabilities remain unverified. It proposes Knowledge Objects as structured artifacts that externalize implicit knowledge into inspectable, verifiable, and endorsable forms. This change in verification economics makes accumulated human validation practical, allowing reliability to improve over time through repeated endorsements.

Core claim

The paper claims that reliable AI needs infrastructure for human validation of implicit knowledge. Knowledge Objects transform verification economics by making previously unexternalized knowledge inspectable, verifiable, and endorsable, enabling accumulated human validation to improve reliability over time.

What carries the argument

Knowledge Objects, structured artifacts that externalize implicit knowledge into forms humans can inspect, verify, and endorse.

If this is right

  • AI capabilities such as reasoning and judgment become subject to human verification.
  • Human endorsements accumulate to produce cumulative gains in AI reliability.
  • The gap between what AI learns and what can be checked against sources narrows.
  • Implicit knowledge that was previously too costly to document becomes practical to capture and validate.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This externalization could be applied to capture intermediate steps in AI training pipelines for later review.
  • Similar structures might help surface unstated assumptions in other complex systems like software engineering or scientific modeling.
  • Pilot tests in narrow domains could measure whether endorsement rates rise enough to offset the added documentation effort.

Load-bearing premise

The documentation cost of implicit knowledge currently exceeds its perceived value, and structured Knowledge Objects will reduce verification costs enough to enable widespread human endorsement.

What would settle it

A deployed AI system using Knowledge Objects would show measurably higher rates of human endorsements and lower incidence of unverified biases than equivalent systems without them.

Figures

Figures reproduced from arXiv: 2605.02010 by Christian S. Jensen, Hengyu Liu, Kristian Torp, Tianyi Li, Torben Bach Pedersen, Yushuai Li, Zhangkai Wu, Zhihong Cui.

Figure 1
Figure 1. Figure 1: AI training data spans data, information, and knowledge. Within knowledge, only the explicit fraction (5–20%) is docu￾mented and verifiable; the implicit majority (80–95%) consists of undocumented patterns that drive capability but resist verification. formation: conversations, code commits, experiment logs, email threads, draft versions, meeting notes, and count￾less other digital artifacts (Dodge et al.,… view at source ↗
Figure 2
Figure 2. Figure 2: illustrates the KO-hub collaboration paradigm. AI System and Human collaborate to address tasks from the environment, generating interaction data. From this data, AI System externalizes implicit knowledge into structured Knowledge Objects. Human experts then validate these KOs, marking them as verified or flagging issues. Validated KOs accumulate and are published to Collective Human Knowledge, parts of wh… view at source ↗
Figure 2
Figure 2. Figure 2: The KO-hub collaboration paradigm. 4.3. How KOs Enable Cumulative Validation [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
read the original abstract

This position paper argues that reliable AI requires infrastructure for human validation of implicit knowledge. AI learns from both explicit knowledge (papers, documentation, structured databases) and implicit knowledge (reasoning patterns, debugging processes, intermediate steps). Implicit knowledge remains unexternalized because documentation cost exceeds perceived value -- yet AI learns from it indiscriminately, acquiring both beneficial patterns and harmful biases. Current reliability methods can only verify explicit knowledge against sources, creating a fundamental gap: the most valuable AI capabilities (reasoning, judgment, intuition) are precisely those we cannot verify. We propose Knowledge Objects (KOs) -- structured artifacts that externalize implicit knowledge into forms humans can inspect, verify, and endorse. KOs transform verification economics: what was previously too costly to verify becomes feasible, enabling accumulated human validation to improve reliability over time.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. This position paper claims that reliable AI requires infrastructure for human validation of implicit knowledge (reasoning patterns, debugging processes, intermediate steps) that current methods cannot verify because they focus only on explicit knowledge sources. It argues that implicit knowledge remains unexternalized due to high documentation costs relative to perceived value, creating a verification gap for AI's most valuable capabilities. The proposed solution is Knowledge Objects (KOs), structured artifacts that externalize this knowledge into inspectable, verifiable, and endorsable forms, thereby changing verification economics to enable cumulative human validation and improved reliability over time.

Significance. If the proposal is developed further, it could usefully frame reliability challenges in AI as a human-AI collaboration problem rather than a purely technical one, drawing attention to the distinction between explicit and implicit knowledge in training data. The conceptual distinction between verifiable explicit knowledge and unexternalized implicit knowledge may encourage new lines of work on externalization mechanisms. As a position paper without empirical data, formal models, or worked examples, its significance rests on whether the KOs concept can be operationalized in follow-on research.

major comments (1)
  1. [Abstract] Abstract (paragraph on verification gap): the assertion that 'documentation cost exceeds perceived value' for implicit knowledge is presented as a premise without references to empirical studies on knowledge externalization costs or value assessments; this assumption is load-bearing for the claim that KOs will transform verification economics sufficiently to enable widespread human endorsement.
minor comments (1)
  1. The manuscript would benefit from at least one illustrative example (even hypothetical) of an implicit knowledge pattern and its corresponding Knowledge Object to make the proposal more concrete for readers.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the detailed review and for identifying the need to strengthen the evidential basis of a key premise. We address the comment below and commit to revisions that improve the manuscript without altering its position-paper character.

read point-by-point responses

  1. Referee: [Abstract] Abstract (paragraph on verification gap): the assertion that 'documentation cost exceeds perceived value' for implicit knowledge is presented as a premise without references to empirical studies on knowledge externalization costs or value assessments; this assumption is load-bearing for the claim that KOs will transform verification economics sufficiently to enable widespread human endorsement.

    Authors: We accept the point. The statement is presented as an observed pattern rather than a newly derived empirical result, but we agree that explicit citations would make the premise more robust. In the revised manuscript we will (1) qualify the claim in the abstract to note that it draws on established findings in the knowledge-management literature and (2) add targeted references to empirical and theoretical work on the costs and incentives of externalizing tacit knowledge (e.g., studies on codification effort versus reuse value). These additions will be placed in the abstract, the verification-gap paragraph, and the discussion of verification economics, thereby supporting rather than presupposing the argument that KOs can alter those economics. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

This position paper advances a conceptual proposal for Knowledge Objects to externalize implicit knowledge without any mathematical derivations, equations, fitted parameters, or load-bearing self-citations. The central argument defines a verification gap and proposes a solution in a single narrative flow, but this is standard for perspective pieces and does not reduce any prediction or uniqueness claim to its own inputs by construction. No patterns from the enumerated circularity kinds are present, and the logic remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The claim depends on the domain assumption that implicit knowledge is both learnable by AI and currently unverifiable, plus the introduction of a new construct (KOs) without independent evidence of feasibility or impact.

axioms (1)
  • domain assumption Implicit knowledge remains unexternalized because documentation cost exceeds perceived value, creating a verification gap that current methods cannot close.
    Stated directly in the abstract as the fundamental limitation of existing reliability methods.
invented entities (1)
  • Knowledge Objects (KOs) no independent evidence
    purpose: Structured artifacts that externalize implicit knowledge into inspectable, verifiable, and endorsable forms.
    Newly introduced construct whose effectiveness is asserted without prior existence or supporting data.

pith-pipeline@v0.9.1-grok · 5691 in / 1194 out tokens · 34817 ms · 2026-07-01T00:10:49.262087+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 10 canonical work pages · 6 internal anchors

  1. [1]

    M., Gebru, T., McMillan-Major, A., and Shmitchell, S

    Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S. On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Trans- parency (FAccT), pp. 610–623,

    2021

  2. [2]

    H., Chen, S., Liu, Z., Jiang, F., and Wang, B

    Chen, G. H., Chen, S., Liu, Z., Jiang, F., and Wang, B. Humans or LLMs as the judge? a study on judgement bias. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8301–8327,

    2024

  3. [3]

    2024 , url =

    Colombo, P., Pires, T. P., Boudiaf, M., Culver, D., Melo, R., Corro, C., Martins, A. F. T., Esposito, F., Raposo, V . L., Morgado, S., and Desa, M. SaulLM-7B: A pio- neering large language model for law.arXiv preprint arXiv:2403.03883,

    work page arXiv

  4. [4]

    Doc- umenting large webtext corpora: A case study on the Colossal Clean Crawled Corpus

    Dodge, J., Sap, M., Marasovic, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., and Gardner, M. Doc- umenting large webtext corpora: A case study on the Colossal Clean Crawled Corpus. InProceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pp. 1286–1305,

    2021

  5. [5]

    Cl-bench: A benchmark for context learning

    Dou, S., Zhang, M., Yin, Z., Huang, C., Shen, Y ., Wang, J., Chen, J., Ni, Y ., Ye, J., Zhang, C., Xie, H., Hu, J., Wang, S., Wang, W., Xiao, Y ., Liu, Y ., Xu, Z., Guo, Z., Zhou, P., Gui, T., Wu, Z., Qiu, X., Zhang, Q., Huang, X., Jiang, Y .-G., Wang, D., and Yao, S. CL-bench: A benchmark for context learning.arXiv preprint arXiv:2602.03587,

    work page arXiv

  6. [6]

    Retrieval-Augmented Generation for Large Language Models: A Survey

    Gao, Y ., Xiong, Y ., Gao, X., Jia, K., Pan, J., Bi, Y ., Dai, Y ., Sun, J., Wang, M., and Wang, H. Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    A survey of confidence estimation and calibration in large language models

    Geng, J., Cai, F., Wang, Y ., Koeppl, H., Nakov, P., and Gurevych, I. A survey of confidence estimation and calibration in large language models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), pp. 6577–6595,

    2024

  8. [8]

    Memory in the Age of AI Agents

    Hu, Y ., Liu, S., Yue, Y ., Zhang, G., Liu, B., Zhu, F., Lin, J., Guo, H., Dou, S., Xi, Z., Jin, S., Tan, J., Yin, Y ., Liu, J., Zhang, Z., Sun, Z., Zhu, Y ., Sun, H., Peng, B., Cheng, Z., Fan, X., Guo, J., Yu, X., Zhou, Z., Hu, Z., Huo, J., Wang, J., Niu, Y ., Wang, Y ., Yin, Z., Hu, X., Liao, Y ., Li, Q., Wang, K., Zhou, W., Liu, Y ., Cheng, D., Zhang, ...

    work page internal anchor Pith review Pith/arXiv arXiv

  9. [9]

    Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation

    Krishna, S., Krishna, K., Mohananey, A., Schwarcz, S., Stambler, A., Upadhyay, S., and Faruqui, M. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technolo- gies (NAACL), pp. 4745–475...

    2025

  10. [10]

    Measuring Faithfulness in Chain-of-Thought Reasoning

    Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Deni- son, C., Hernandez, D., Li, D., Durmus, E., Hubinger, E., Kernion, J., Luko ˇsi¯ut˙e, K., Nguyen, K., Cheng, N., Joseph, N., Schiefer, N., Rausch, O., Larson, R., McCan- dlish, S., Kundu, S., Kadavath, S., Yang, S., Henighan, T., Maxwell, T., Telleen-Lawton, T., Hume, T., Hatfield- Dodds, Z., Kap...

    work page internal anchor Pith review Pith/arXiv arXiv

  11. [11]

    LLM -based agents suffer from hallucinations: A survey of taxonomy, methods, and directions

    Lin, X., Ning, Y ., Zhang, J., Dong, Y ., Liu, Y ., Wu, Y ., Qi, X., Sun, N., Shang, Y ., Wang, K., Cao, P., Wang, Q., Zou, L., Chen, X., Zhou, C., Wu, J., Zhang, P., Wen, Q., Pan, S., Wang, B., Cao, Y ., Chen, K., Hu, S., and Guo, L. LLM-based agents suffer from hallucinations: A survey of taxonomy, methods, and directions.arXiv preprint arXiv:2509.18970,

    work page arXiv

  12. [12]

    Liu, H., Li, T., He, Y ., Torp, K., Li, Y ., and Jensen, C. S. MH-GIN: Multi-scale heterogeneous graph-based impu- tation network for AIS data.Proceedings of the VLDB Endowment (PVLDB), 19(2):170–182, 2026a. Liu, H., Li, T., Wang, H., Torp, K., Li, Y ., Zhang, T., Ped- ersen, T. B., and Jensen, C. S. CLEAR: A knowledge- centric vessel trajectory analysis ...

    work page arXiv

  13. [13]

    MemGPT: Towards LLMs as Operating Systems

    Packer, C., Wooders, S., Lin, K., Fang, V ., Patil, S. G., and Gonzalez, J. E. MemGPT: Towards LLMs as operating systems.arXiv preprint arXiv:2310.08560,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    The Impact of AI on Developer Productivity: Evidence from GitHub Copilot

    Peng, S., Kalliamvakou, E., Cihon, P., and Demirer, M. The impact of AI on developer productivity: Evidence from GitHub Copilot.arXiv preprint arXiv:2302.06590,

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    Language models as knowl- edge bases? InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp

    Petroni, F., Rockt¨aschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y ., and Miller, A. Language models as knowl- edge bases? InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2463–2473,

    2019

  16. [16]

    V oyager: An open-ended embodied agent with large language models.Transac- tions on Machine Learning Research, 2024a

    Wang, G., Xie, Y ., Jiang, Y ., Mandlekar, A., Xiao, C., Zhu, Y ., Fan, L., and Anandkumar, A. V oyager: An open-ended embodied agent with large language models.Transac- tions on Machine Learning Research, 2024a. Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y ., Zhao, W. X., Wei, Z., and Wen, J.-R. A surve...

    2024

  17. [17]

    Simple synthetic data reduces sycophancy in large language models

    Wei, J., Tay, Y ., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Met- zler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., and Fedus, W. Emergent abilities of large language models.Transactions on Machine Learning Research, 2022a. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., C...

    work page internal anchor Pith review Pith/arXiv arXiv