pith. sign in

arxiv: 2604.26197 · v3 · pith:PPMBUTRRnew · submitted 2026-04-29 · 💻 cs.IR · cs.LG

Hierarchical Long-Term Semantic Memory for LinkedIn's Hiring Agent

Zhentao Xu , Shangjin Zhang , Emir Poyraz , Yvonne Li , Ye Jin , Xie Lu , Xiaoyang Gu , Karthik Ramgopal
show 2 more authors
Praveen Kumar Bodigutla Xiaofeng Wang
This is my paper

Pith reviewed 2026-07-01 09:05 UTC · model grok-4.3

classification 💻 cs.IR cs.LG
keywords Hierarchical Long-Term Semantic MemoryLLM agentsmemory treeretrieval F1answer correctnessLinkedIn Hiring Assistantpersonalizationlatency trade-off
0
0 comments X

The pith

HLTM's schema-aligned memory tree improves LLM hiring assistant answer correctness by more than 5% and retrieval F1 by more than 10%.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces the Hierarchical Long-Term Semantic Memory (HLTM) framework to handle five challenges in long-term memory systems for LLM agents: scalability, low-latency retrieval, privacy constraints, adaptability, and observability. HLTM structures textual data extracted from noisy longitudinal behavioral signals into a schema-aligned memory tree that stores semantic knowledge at multiple levels of granularity. This structure supports efficient ingestion, privacy-aware storage, low-latency retrieval with transparent provenance, and an adaptation mechanism for new use cases. Evaluations on LinkedIn's Hiring Assistant demonstrate the reported gains in correctness and F1 plus an improved trade-off between query and indexing latency. The framework has been deployed in production to support core personalization features in hiring workflows.

Core claim

The HLTM framework organizes textual data into a schema-aligned memory tree that captures semantic knowledge at multiple levels of granularity. This design supports scalable ingestion, privacy-aware storage, low-latency retrieval, and transparent provenance, with an adaptation mechanism to generalize across diverse use cases. In LinkedIn's Hiring Assistant, it yields more than 5% higher answer correctness and more than 10% higher retrieval F1 while advancing the query-indexing latency Pareto frontier, and has been deployed to power personalization features.

What carries the argument

schema-aligned memory tree that captures semantic knowledge at multiple levels of granularity

Load-bearing premise

The measured gains in correctness and F1 are caused by the hierarchical memory tree design itself rather than by other unstated changes in the hiring assistant implementation or by characteristics of the specific LinkedIn user data.

What would settle it

An ablation test that removes the multi-level hierarchy from the memory tree while keeping all other components fixed and measures whether the correctness and retrieval F1 improvements disappear on the same LinkedIn hiring tasks.

Figures

Figures reproduced from arXiv: 2604.26197 by Emir Poyraz, Karthik Ramgopal, Praveen Kumar Bodigutla, Shangjin Zhang, Xiaofeng Wang, Xiaoyang Gu, Xie Lu, Ye Jin, Yvonne Li, Zhentao Xu.

Figure 1
Figure 1. Figure 1: LinkedIn Hiring Assistant with HLTM: a recruiter initiates a hiring project; the hiring assistant queries HLTM in natural language to retrieve preference signals, then uses the returned information to update structured hiring require￾ments. Query Latency (s) Answer Correctness 3 0.3 0.4 4 5 6 7 8 9 10 20 0.5 0.6 0.7 0.8 HLTM (ours) HLTM (ours) view at source ↗
Figure 2
Figure 2. Figure 2: Performance–latency trade-off across evaluated view at source ↗
Figure 3
Figure 3. Figure 3: Overview of Hierarchical Long-Term Semantic Memory ( view at source ↗
Figure 4
Figure 4. Figure 4: Lossless incremental nearline indexing in view at source ↗
Figure 5
Figure 5. Figure 5: Query vs. indexing latency: HLTM advances the Pareto frontier. 2Disclaimer: Results may vary in production environments or with different datasets. 3Disclaimer: Results may vary in production environments or with different datasets. 4.6 Ablation Study and Analysis We conduct an ablation study to quantify the contributions of tree aggregation, adaptation, and each memory representation view at source ↗
Figure 6
Figure 6. Figure 6: Hyperparameter analysis shows HLTM has no early peak and quickly plateaus at small 𝑘, indicating robustness to 𝑘 beyond a small threshold. User Setup Set up environment Supervisor Planner Based on user message, chat history, and current workflow state Supervisor Planning Scenario Scenario ? Plan instruction Task List Based on task result, update plan (remaining task list) Supervisor Replanner All done, or … view at source ↗
Figure 7
Figure 7. Figure 7: HLTM’s Production Use Case in Hiring Assistant 5 Production Use Case LinkedIn Hiring Assistant (LiHA) [8] is an AI agent for recruiters, powered by LinkedIn’s dynamic talent network, that helps re￾cruiters discover and engage candidates with greater speed and scale. Architecturally, LiHA is a plan-and-execute system centered on a supervisor agent that interprets recruiter intent and orches￾trates specializ… view at source ↗
read the original abstract

Large Language Model (LLM) agents are increasingly used in real-world products, where personalized and context-aware user interactions are essential. A central enabler of such capabilities is the agent's long-term semantic memory system, which extracts implicit and explicit signals from noisy longitudinal behavioral data, stores them in a structured form, and supports low-latency retrieval. Building industrial-grade long-term memory for LLM agents raises five challenges: scalability, low-latency retrieval, privacy constraints, adaptability, and observability. We introduce the Hierarchical Long-Term Semantic Memory (HLTM) framework, which organizes textual data into a schema-aligned memory tree that captures semantic knowledge at multiple levels of granularity, enabling scalable ingestion, privacy-aware storage, low-latency retrieval, and transparent provenance; HLTM further incorporates an adaptation mechanism to generalize across diverse use cases. Extensive evaluations on LinkedIn's Hiring Assistant show that HLTM improves answer correctness by more than 5% and retrieval F1 by more than 10%, while significantly advancing the Pareto frontier between query and indexing latency. HLTM has been fully deployed in LinkedIn's Hiring Assistant to power core personalization features in production hiring workflows.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper introduces the Hierarchical Long-Term Semantic Memory (HLTM) framework for LLM agents, which structures longitudinal behavioral data into a schema-aligned memory tree supporting multi-granularity semantic knowledge. It addresses five industrial challenges (scalability, low-latency retrieval, privacy, adaptability, observability) via the tree organization plus an adaptation mechanism. Extensive evaluations on LinkedIn's Hiring Assistant are reported to yield >5% gains in answer correctness and >10% in retrieval F1, with Pareto improvements on query/indexing latency; the system is stated to be fully deployed in production for personalization features.

Significance. If the reported gains are causally attributable to the hierarchical tree and adaptation design, the work would be significant for industrial LLM agent memory systems by demonstrating a structured, privacy-aware approach that scales to production hiring workflows. The explicit production deployment provides a concrete existence proof of feasibility under real constraints, which is a strength relative to purely academic memory proposals.

major comments (2)
  1. [Abstract] Abstract: the central empirical claims (>5% correctness lift, >10% F1 lift, Pareto latency advance) are stated without any reference to baselines, statistical tests, dataset size or characteristics, exclusion criteria, or evaluation protocol. This absence makes it impossible to determine whether the numbers support the improvements or to assess their robustness.
  2. [Evaluation / deployment description] Evaluation / deployment description (the section reporting production results): no ablation (flat vs. hierarchical storage, with vs. without adaptation) or controlled A/B test description is supplied to isolate the contribution of the schema-aligned memory tree from concurrent implementation changes, user-data distribution shifts, or other unstated modifications to the Hiring Assistant. Without such isolation the causal link required by the central claim cannot be established from the internal metrics alone.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive feedback on our work describing the HLTM framework and its deployment in LinkedIn's Hiring Assistant. Below we respond point-by-point to the major comments, with a focus on clarifying evaluation aspects while respecting production constraints.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central empirical claims (>5% correctness lift, >10% F1 lift, Pareto latency advance) are stated without any reference to baselines, statistical tests, dataset size or characteristics, exclusion criteria, or evaluation protocol. This absence makes it impossible to determine whether the numbers support the improvements or to assess their robustness.

    Authors: We agree the abstract is concise and omits explicit references to the evaluation protocol, baselines, and dataset details. In the revised manuscript we will expand the abstract by one sentence to reference the internal LinkedIn Hiring Assistant evaluation (including the use of production query logs for correctness and retrieval F1 metrics, with comparisons against the prior non-hierarchical memory baseline) while preserving length limits. Full protocol, dataset characteristics, and any statistical considerations are already detailed in the evaluation section. revision: yes

  2. Referee: [Evaluation / deployment description] Evaluation / deployment description (the section reporting production results): no ablation (flat vs. hierarchical storage, with vs. without adaptation) or controlled A/B test description is supplied to isolate the contribution of the schema-aligned memory tree from concurrent implementation changes, user-data distribution shifts, or other unstated modifications to the Hiring Assistant. Without such isolation the causal link required by the central claim cannot be established from the internal metrics alone.

    Authors: We recognize the value of ablations and controlled tests for causal attribution. The manuscript reports pre- versus post-deployment metrics on the live system, which include direct measurements of the Pareto latency improvements and the >5% correctness / >10% F1 gains. However, as a production system at LinkedIn, we are unable to release detailed A/B test protocols, ablation results, or internal change logs due to proprietary and privacy constraints. We have added a limitations paragraph in the revised version discussing the inherent challenges of isolating effects in deployed industrial systems and note that the full production deployment itself provides an existence proof under real-world constraints. revision: no

standing simulated objections not resolved
  • Detailed descriptions of controlled A/B tests, ablations, or internal production change isolation cannot be supplied without violating LinkedIn's proprietary and privacy policies.

Circularity Check

0 steps flagged

No circularity; claims rest on empirical deployment metrics

full rationale

The paper introduces the HLTM framework as a system design for organizing memory in LLM agents and reports measured improvements (>5% correctness, >10% F1, Pareto latency gains) from its deployment in LinkedIn's Hiring Assistant. No derivation chain, equations, or first-principles predictions exist that could reduce to fitted inputs or self-citations by construction. The central claims are observational results from production use rather than any self-referential or fitted 'prediction' structure.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract supplies no information on free parameters, axioms, or invented entities; full text would be required to populate the ledger.

pith-pipeline@v0.9.1-grok · 5764 in / 1035 out tokens · 30474 ms · 2026-07-01T09:05:34.527040+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 11 canonical work pages · 5 internal anchors

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  2. [2]

    Anthropic. 2025. Home — Anthropic. https://www.anthropic.com/. Accessed: 2025-12-12

    2025

  3. [3]

    Sizhe Cheng, Jiaping Li, Huanchen Wang, and Yuxin Ma. 2025. Ragtrace: Un- derstanding and refining retrieval-generation dynamics in retrieval-augmented generation. InProceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. 1–20

    2025

  4. [4]

    Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Ya- dav. 2025. Mem0: Building production-ready ai agents with scalable long-term memory. InProceedings of the 28th European Conference on Artificial Intelligence. 2993–3000. doi:10.3233/FAIA251160

    work page doi:10.3233/faia251160 2025

  5. [5]

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  6. [6]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization.arXiv preprint arXiv:2404.16130(2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    GDPR.eu. [n. d.]. Complete guide to GDPR compliance. https://gdpr.eu/. Accessed: 2026-01-30

    2026

  8. [8]

    2025.Building the agentic future of recruiting: how we engineered LinkedIn’s Hiring Assistant

    Xiaoyang Gu, Xie Lu, and Daniel Hewlett. 2025.Building the agentic future of recruiting: how we engineered LinkedIn’s Hiring Assistant. LinkedIn Engi- neering. https://www.linkedin.com/blog/engineering/ai/how-we-engineered- linkedins-hiring-assistant

    2025

  9. [9]

    Changyue Jiang, Xudong Pan, Geng Hong, Chenfu Bao, and Min Yang. 2024. Rag- thief: Scalable extraction of private data from retrieval-augmented generation applications with agent-based attacks.arXiv preprint arXiv:2411.14110(2024)

    work page arXiv 2024

  10. [10]

    Bernal Jimenez Gutierrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su

  11. [11]

    Hipporag: Neurobiologically inspired long-term memory for large language models.Advances in Neural Information Processing Systems37 (2024), 59532– 59569

    2024

  12. [12]

    Ehsan Kamalloo, Nouha Dziri, Charles Clarke, and Davood Rafiei. 2023. Eval- uating open-domain question answering in the era of large language models. InProceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers). 5591–5606

    2023

  13. [13]

    Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, and Ian Fischer. 2024. A human-inspired reading agent with gist memory of very long contexts. In Proceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235). 26396–26415

    2024

  14. [14]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems33 (2020), 9459–9474

    2020

  15. [15]

    Zhiyu Li, Chenyang Xi, Chunyu Li, Ding Chen, Boyu Chen, Shichao Song, Simin Niu, Hanyu Wang, Jiawei Yang, Chen Tang, et al. 2025. Memos: A memory os for ai system.arXiv preprint arXiv:2507.03724(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  16. [16]

    Jiaqi Liu, Yaofeng Su, Peng Xia, Siwei Han, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao. 2026. SimpleMem: Efficient Lifelong Memory for LLM Agents.arXiv preprint arXiv:2601.02553(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  17. [17]

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 1906–1919. doi:10.18653/v1/2020.acl-main.173

    work page doi:10.18653/v1/2020.acl-main.173 2020

  18. [18]

    Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez. 2023. MemGPT: Towards LLMs as Operating Systems. (2023)

    2023

  19. [19]

    Alireza Rezazadeh, Zichao Li, Wei Wei, and Yujia Bao. 2024. From isolated conversations to hierarchical schemas: Dynamic tree memory representation for llms.arXiv preprint arXiv:2410.14052(2024)

    work page arXiv 2024

  20. [20]

    Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D Manning. 2024. Raptor: Recursive abstractive processing for tree-organized retrieval. InThe Twelfth International Conference on Learning Representations

    2024

  21. [21]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools.Advances in Neural Information Processing Systems36 (2023), 68539–68551

    2023

  22. [22]

    Wenyu Tao, Xiaofen Xing, Yirong Chen, Linyi Huang, and Xiangmin Xu. 2025. Treerag: Unleashing the power of hierarchical storage for enhanced knowledge retrieval in long documents. InFindings of the Association for Computational Linguistics: ACL 2025. 356–371. ��� ����� ������ ����� ����� ���� ������� �������� �� ������ ������� �� �� ���

    2025

  23. [23]

    Bing Wang, Xinnian Liang, Jian Yang, Hui Huang, Shuangzhi Wu, Peihao Wu, Lu Lu, Zejun Ma, and Zhoujun Li. 2025. SCM: Enhancing large language model with self-controlled memory framework. InProceedings of the 30th International Conference on Database Systems for Advanced Applications

    2025

  24. [24]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024. A survey on large language model based autonomous agents.Frontiers of Computer Science18, 6 (2024), 186345

    2024

  25. [25]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems35 (2022), 24824–24837

    2022

  26. [26]

    Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2025. LongMemEval: Benchmarking chat assistants on long-term interactive memory. InThe Thirteenth International Conference on Learning Representations

    2025

  27. [27]

    Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang

  28. [28]

    InAdvances in Neural Information Processing Systems

    A-Mem: Agentic memory for LLM agents. InAdvances in Neural Information Processing Systems

  29. [29]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems36 (2023), 11809–11822

    2023

  30. [30]

    Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Wang Yongji, and Jian-Guang Lou. 2023. Large language models meet nl2code: A survey. InProceedings of the 61st Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers). 7443–7464

    2023

  31. [31]

    Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shuaiqiang Wang, Dawei Yin, et al. 2024. The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag). InFindings of the Association for Computational Linguistics: ACL 2024. 4505–4524

    2024

  32. [32]

    Guibin Zhang, Muxin Fu, Guancheng Wan, Miao Yu, Kun Wang, and Shuicheng Yan. 2025. G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems. arXiv preprint arXiv:2506.07398(2025)

    work page arXiv 2025

  33. [33]

    Zeyu Zhang, Quanyu Dai, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. 2025. A survey on the memory mechanism of large language model-based agents.ACM Transactions on Information Systems 43, 6 (2025), 1–47

    2025

  34. [34]

    Yibo Zhao, Jiapeng Zhu, Ye Guo, Kangkang He, and Xiang Li. 2025. Eˆ 2GraphRAG: Streamlining Graph-based RAG for High Efficiency and Effectiveness.arXiv preprint arXiv:2505.24226(2025)

    work page arXiv 2025

  35. [35]

    Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memo- rybank: Enhancing large language models with long-term memory. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 19724–19731

    2024

  36. [36]

    Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Liu, Zhicheng Dou, and Ji-Rong Wen. 2025. Large language models for information retrieval: A survey.ACM Transactions on Information Systems44, 1 (2025), 1–54. A����Indexing-time Prompts A.1 Facet Extraction � ������ ������� � ��� ��� � ������� ��������� ����...

    2025