From Volume to Value: Preference-Aligned Memory Construction for On-Device RAG

Changmin Lee; Jaemin Kim; Taesik Gong

arxiv: 2605.18271 · v2 · pith:UBN77HXPnew · submitted 2026-05-18 · 💻 cs.CL · cs.AI· cs.IR· cs.LG

From Volume to Value: Preference-Aligned Memory Construction for On-Device RAG

Changmin Lee , Jaemin Kim , Taesik Gong This is my paper

Pith reviewed 2026-06-30 18:40 UTC · model grok-4.3

classification 💻 cs.CL cs.AIcs.IRcs.LG

keywords on-device RAGpreference alignmentmemory efficiencypersonal AI agentsretrieval latencyindex constructionLLM agentscontext filtering

0 comments

The pith

EPIC builds on-device RAG memory from user preferences alone, cutting indexing use 2404 times while lifting preference accuracy 18.79 points and slashing latency 32 times.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents EPIC as a way to run personal LLM agents on devices by treating user preferences as compact, stable context instead of storing raw data volumes. It filters incoming information to keep only preference-relevant items and then aligns the retrieval stage of RAG to those retained items. The result is far lower memory and faster responses while still following user preferences more closely on tasks such as conversations and recommendations. A sympathetic reader cares because the approach makes private, low-latency agents practical under the strict memory limits of phones and other edge hardware.

Core claim

EPIC selectively retains preference-relevant information from raw data and aligns retrieval toward preference-aligned contexts. Across four benchmarks this produces 2404 times lower indexing memory, 18.79 percentage point gains in preference-following accuracy, and 32.17 times lower retrieval latency than the strongest baseline. On-device runs keep total memory under 1 MB, deliver 5.21 to 29.35 ms per query across three platforms, and allow streaming updates when preferences drift.

What carries the argument

The preference-aligned index, which filters raw personal data to retain only preference-relevant items and directs retrieval to those items throughout the RAG pipeline.

If this is right

On-device agents can stay under 1 MB while supporting real-time queries and preference updates.
Preference alignment improves accuracy on conversations, debates, explanations, and recommendations.
Retrieval latency drops by more than 30 times compared with prior indexing methods.
The same pipeline works across multiple hardware platforms without cloud offloading.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same filtering logic could be tested on other stable personal signals such as recurring goals or interaction patterns.
Lower memory footprints might enable preference-aligned retrieval on even smaller devices like smart watches.
Preference drift handling suggests the method could maintain performance as user interests evolve over weeks or months.

Load-bearing premise

User preferences form a compact and stable personal context that can be extracted from raw data and used to align retrieval throughout the pipeline.

What would settle it

A baseline that stores full raw context or uses non-preference filtering would need to match or beat EPIC on memory size, latency, and preference accuracy across the same four benchmarks to falsify the central claim.

Figures

Figures reproduced from arXiv: 2605.18271 by Changmin Lee, Jaemin Kim, Taesik Gong.

**Figure 1.** Figure 1: Prior Method indiscriminately stores raw data, which is infeasible under tight on-device memory budgets and can yield preference-misaligned responses (left). EPIC instead retains only preference-relevant data with aligned instructions, enabling efficient retrieval and preference-aligned responses (right). Example from the PrefWiki dataset. et al., 2024; Li et al., 2024a). Prior studies on assistant usage… view at source ↗

**Figure 2.** Figure 2: Overview of EPIC’s pipeline. (i) Semantic-Based Coarse Filtering (Sec. 3.1): documents from a large corpus are first encoded and compared with user preference embeddings; only those with at least one preference-aligned match pass this stage. (ii) Preference-Aligned Fine Verification (Sec. 3.2): the Decision Module verifies textual alignment and discards unrelated documents, while the Instruction Generator … view at source ↗

**Figure 3.** Figure 3: Efficiency comparison across baselines. We report on-disk memory usage, end-to-end retrieval latency, and indexing latency (detailed results in Appendix B.6). Numbers in parentheses represent the specific values on the x-axis for each method. though query steering adds a small constant overhead, retrieval remains a single FAISS kNN search over a much smaller index. This yields consistently lower latency t… view at source ↗

**Figure 4.** Figure 4: On-device streaming data setup with random preference drift. On Jetson Orin Nano 8GB using PrefWiki, EPIC maintains higher preference-following accuracy while keeping memory nearly constant, compared to the lightweight Contriever. This indicates that instruction-centric memory construction both strengthens preference alignment and replaces bulky raw items with compact, preference-aware representations. … view at source ↗

**Figure 6.** Figure 6: Preference change events during streaming (examples). To complement [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗

**Figure 5.** Figure 5: Streaming on-device evaluation platform. NVIDIA Jetson Orin Nano 8GB used for the streaming on-device experiments [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗

read the original abstract

With the rapid emergence of personal AI agents based on Large Language Models (LLMs), implementing them on-device has become essential for privacy and responsiveness. To handle the inherently personal and context-dependent nature of real-world requests, such agents must ground their generation in device-resident personal context. However, under tight memory budgets, the core bottleneck is what to store so that retrieval remains aligned with the user. We propose EPIC (Efficient Preference-aligned Index Construction), which focuses on user preferences as a compact and stable form of personal context and integrates them throughout the RAG pipeline. EPIC selectively retains preference-relevant information from raw data and aligns retrieval toward preference-aligned contexts. Across four benchmarks covering conversations, debates, explanations, and recommendations, EPIC reduces indexing memory by 2,404 times, improves preference-following accuracy by 18.79 %p, and achieves 32.17 times lower retrieval latency over the best-performing baseline. In on-device experiments, EPIC maintains under 1 MB memory and achieves 5.21 to 29.35 ms/query latency across three platforms, while supporting streaming updates under preference drift. Our code and data are available at https://github.com/UbiquitousAILab/EPIC.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

EPIC reports big memory and latency wins for on-device RAG by indexing around user preferences, but the abstract leaves methods and stats thin.

read the letter

EPIC's main result is that treating user preferences as the compact signal for indexing and retrieval cuts memory by 2404 times, lifts preference-following accuracy by 18.79 points, and drops latency by 32 times versus the best baseline. On-device runs stay under 1 MB and 30 ms per query while handling streaming updates when preferences change.

The new piece is the end-to-end preference alignment: it filters raw data to keep only preference-relevant parts, then tunes retrieval to match. That is more integrated than typical compression or standard RAG pipelines.

The experiments run across four benchmark categories and three hardware platforms, with code and data released. That makes the numbers easier to check.

The soft spots sit in the reporting. The abstract gives no error bars, dataset splits, or exclusion rules, so the size of the gains is hard to judge without the full methods. The core assumption that preferences form a stable, selective signal is stated clearly and the paper claims support for drift, but longer real-world tests would help confirm it does not break under messier changes.

This is aimed at researchers building memory-tight personal agents on device. Anyone comparing retrieval methods for private context would find the numbers and setup useful to reference.

I would send it to peer review. The claims are concrete and the problem practical enough that referees should see the full paper and code.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes EPIC (Efficient Preference-aligned Index Construction) for on-device RAG in personal AI agents. It claims that selectively retaining preference-relevant information from raw data and aligning retrieval toward preference-aligned contexts yields a 2,404× reduction in indexing memory, an 18.79 percentage-point gain in preference-following accuracy, and 32.17× lower retrieval latency versus the best baseline across four benchmarks (conversations, debates, explanations, recommendations). On-device tests report <1 MB memory and 5.21–29.35 ms/query latency on three platforms, with support for streaming updates under preference drift. Code and data are released.

Significance. If the reported gains are reproducible, the work would be significant for on-device LLM agents by reframing memory construction around stable user preferences rather than raw volume. The explicit integration of preferences throughout the RAG pipeline and the public release of code/data are concrete strengths that enable direct verification and extension.

major comments (2)

[Abstract] Abstract: the central empirical claims (2,404× memory reduction, 18.79 %p accuracy lift, 32.17× latency reduction) are stated without error bars, number of runs, or statistical tests. Because the manuscript’s contribution is defined by these quantitative improvements over baselines, the absence of variability measures prevents assessment of whether the gains are reliable or could be explained by run-to-run variance.
[Abstract] Abstract (and § on experimental setup, per reader’s note): dataset details, exclusion criteria, and exact baseline implementations are not provided in the visible text. The four named benchmarks are central to the claim of broad applicability; without these specifics the reader cannot determine whether the preference-alignment advantage generalizes or is benchmark-specific.

minor comments (1)

[Abstract] The abstract states that EPIC “supports streaming updates under preference drift,” but the precise mechanism (e.g., incremental index update cost, drift detection threshold) is not quantified in the provided summary; a short paragraph or table entry would clarify this feature.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on reproducibility and clarity. We address each major comment below and will update the manuscript to strengthen these aspects.

read point-by-point responses

Referee: [Abstract] Abstract: the central empirical claims (2,404× memory reduction, 18.79 %p accuracy lift, 32.17× latency reduction) are stated without error bars, number of runs, or statistical tests. Because the manuscript’s contribution is defined by these quantitative improvements over baselines, the absence of variability measures prevents assessment of whether the gains are reliable or could be explained by run-to-run variance.

Authors: We agree that variability measures are essential for evaluating reliability. The experiments were run with 5 independent seeds; we omitted these details from the abstract for space. In revision we will add standard deviations as error bars, state the run count, and report paired t-test p-values for the primary comparisons, both in the abstract and in the results tables/figures. revision: yes
Referee: [Abstract] Abstract (and § on experimental setup, per reader’s note): dataset details, exclusion criteria, and exact baseline implementations are not provided in the visible text. The four named benchmarks are central to the claim of broad applicability; without these specifics the reader cannot determine whether the preference-alignment advantage generalizes or is benchmark-specific.

Authors: Section 4.1 of the full manuscript already describes the four benchmarks (conversations, debates, explanations, recommendations), their sources, sizes, construction, and exclusion criteria for preference alignment. Section 4.3 details baseline implementations, hyperparameters, and provides the public code repository link. To improve visibility we will insert a one-sentence summary of the benchmarks plus an explicit cross-reference to Section 4 into the abstract. revision: partial

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper presents EPIC as an empirical method for selective retention of preference-relevant data in on-device RAG, with all central claims (memory reduction by 2,404x, accuracy gain of 18.79%p, latency improvements) grounded in benchmark results across four datasets and on-device measurements. No derivation chain, equations, fitted parameters renamed as predictions, or self-citation load-bearing steps are present; the work is self-contained as an engineering contribution whose validity rests on external experimental falsifiability rather than internal reduction to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an applied empirical systems paper. The central claim depends on the validity of the four chosen benchmarks and the assumption that preferences can be extracted reliably; no free parameters, axioms, or invented entities are stated in the abstract.

pith-pipeline@v0.9.1-grok · 5758 in / 1146 out tokens · 35245 ms · 2026-06-30T18:40:00.549581+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Reference graph

Works this paper leans on

34 extracted references · 5 canonical work pages · 3 internal anchors

[1]

The Llama 3 Herd of Models

URLhttps://aclanthology.org/2023. emnlp-main.398/. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024. Gutierrez, B. J., Shu, Y ., Gu, Y ., Yasunaga, M., and Su, Y . HippoRAG: Neurobiologically inspired long...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2021.emnlp-main 2023
[2]

emnlp-main.243/

URLhttps://aclanthology.org/2021. emnlp-main.243/. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V ., Goyal, N., K ¨uttler, H., Lewis, M., Yih, W.-t., Rockt¨aschel, T., et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020. Li, X., Wang, S., Zeng, S., Wu, Y...

work page doi:10.18653/v1/2021.acl-long 2021
[3]

Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security

URLhttps://aclanthology.org/2021. acl-long.353/. Li, Y ., Wen, H., Wang, W., Li, X., Yuan, Y ., Liu, G., Liu, J., Xu, W., Wang, X., Sun, Y ., et al. Personal llm agents: Insights and survey about the capability, efficiency and security.arXiv preprint arXiv:2401.05459, 2024b. Mysore, S., Lu, Z., Wan, M., Yang, L., Sarrafzadeh, B., Menezes, S., Baghaee, T.,...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2024.customnlp4u-1 2021
[4]

gpt-oss-120b & gpt-oss-20b Model Card

URLhttps://aclanthology.org/2024. customnlp4u-1.16/. Neverova, N., Wolf, C., Lacey, G., Fridman, L., Chandra, D., Barbello, B., and Taylor, G. Learning human identity from motion patterns.IEEE Access, 4:1810–1820, 2016. OpenAI. gpt-oss-120b & gpt-oss-20b model card, 2025. URLhttps://arxiv.org/abs/2508.10925. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wai...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2020 2024
[5]

soft prompts

URLhttps://openreview.net/forum? id=QWunLKbBGF. Zheng, L., Chiang, W.-L., Sheng, Y ., Li, T., Zhuang, S., Wu, Z., Zhuang, Y ., Li, Z., Lin, Z., Xing, E. P., Gonzalez, J. E., Stoica, I., and Zhang, H. Lmsys-chat-1m: A large- scale real-world llm conversation dataset, 2023. Zhong, W., Guo, L., Gao, Q., Ye, H., and Wang, Y . Mem- orybank: Enhancing large lan...

2023
[6]

Either the user preference or the question is missing, so the retrieval target cannot be precisely defined
[7]

Questions rarely induce preference conflicts, making violations unlikely and the retrieval task non-discriminative
[8]

I avoid electric vehicles,

No gold labels tying (preference, question) pairs to documents that both answer the query and satisfy preferences. In light of these limitations of existing datasets, this study makes extensive use of the PrefEval benchmark (Zhao et al., 2025). A.5. PrefEval Benchmark The Explicit Preference subset of PrefEval dataset (Zhao et al., 2025) focuses on prefer...

2025
[9]

a preference statement (clear like/dislike or constraint), and
[10]

a query that can easily elicit a default answer which would violate that preference unless the model takes it into account (e.g., recommending the best compact cars for city driving, where the most top options are electric vehicles),
[11]

This subset deliberately booby-traps the obvious answer: the quickest generic response is often preference-inconsistent

optionally, a short explanation/rationale highlighting why the query is risky with respect to the preference. This subset deliberately booby-traps the obvious answer: the quickest generic response is often preference-inconsistent. Strong performance therefore requires the model to (1) recognize the explicit constraint, (2) prioritize it alongside topical ...
[12]

Preference-Unaware Violation: The LLM provides generic recommendations that contradict the user’s stated prefer- ence due to unawareness of user preference
[13]

Preference Hallucination Violation: The response fabricates or misattributes preferences, diverging from the user’s true preference and violates the true preference
[14]

Inconsistent Violation: The response acknowledges the correct preference but generates contradicting response
[15]

role: content

Unhelpful Response: The response lacks relevant recommendations or fails to address the query due to poor recall of the user’s preference. B. Experimental Details 15 From Volume to Value: Preference-Aligned Memory Construction for On-Device RAG B.1. Corpus of Preference Benchmarks This section describes the retrieval corpora used for indexing and retrieva...

work page arXiv 2025
[16]

the question directly contradicts the user’s preference, such that any answer would inherently violate the preference
[17]

the question is already perfectly aligned with the preference, such that no additional reasoning about the preference is required
[18]

For PrefRQ, since the dataset is pre-filtered to contain highly subjective questions from the Researchy Questions corpus, only conditions (1) and (2) are checked

the question has a negligible probability of violating the preference under the PrefEval data generation prompt, i.e., whenP(answer|question)≪P(answer|preference,question), indicating that even without conditioning on the preference, natural answers rarely conflict with it For PrefELI5, all three conditions are applied. For PrefRQ, since the dataset is pr...
[19]

I prefer vegetarian meals

Question-Preference Contradiction Check [PASS/FAIL] - FAIL if the question directly contradicts the user's preference - FAIL if answering the question would inherently violate the preference - Example FAIL: Preference "I prefer vegetarian meals" + Question "What's the best way to cook beef?"
[20]

I love Italian food

Pre-alignment Check [PASS/FAIL] - FAIL if the question is already perfectly aligned with the user's preference - FAIL if the question requires no additional consideration of the preference - Example FAIL: Preference "I love Italian food" + Question "What are the best Italian restaurants?"
[21]

I prefer companies that allow unlimited sick days

Low Violation Check [PASS/FAIL] - FAIL if the question has a low probability of violating the preference - FAIL if P(answer|question) << P(answer|preference, question), which means without knowing the preference, naturally answering the question rarely violates the user's preference - Example FAIL: Preference "I prefer companies that allow unlimited sick ...
[22]

Understand all user preferences thoroughly
[23]

Read the given document chunk
[24]

If the chunk contains no content relevant to any of the preferences, decide: Discard
[25]

If the chunk is relevant to any preference, decide: Keep
[26]

Always explain the reason clearly
[27]

If Keep, specify exactly which preferences the chunk aligns with
[28]

</planning_steps> <guidelines> - Do not infer unstated preferences

Output must strictly follow the XML structure and include only XML. </planning_steps> <guidelines> - Do not infer unstated preferences. - When listing <relevant_preferences>, use the exact preference texts as provided by the user, do not paraphrase or modify. </guidelines> <response_requirements> - Every output must follow strict XML format. - The <reason...
[29]

Read the user's stated preferences
[30]

Read the document chunk
[31]

Read the given reason for why this chunk was marked as relevant
[32]

Generate a clear, concise instruction that explains how to interpret or read this chunk in light of the relevant preferences
[33]

The instruction should guide readers on what aspects to focus on or what perspective to take when reading the chunk
[34]

</planning_steps> <guidelines> - The instruction is NOT a rewrite of the chunk itself, but rather guidance on how to interpret it

Output must consist of a single <instruction> XML tag. </planning_steps> <guidelines> - The instruction is NOT a rewrite of the chunk itself, but rather guidance on how to interpret it. - Focus on directing attention to preference-relevant aspects of the content. - Keep instructions concise and actionable. - Do not add information not present in the chunk...

1965

[1] [1]

The Llama 3 Herd of Models

URLhttps://aclanthology.org/2023. emnlp-main.398/. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024. Gutierrez, B. J., Shu, Y ., Gu, Y ., Yasunaga, M., and Su, Y . HippoRAG: Neurobiologically inspired long...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2021.emnlp-main 2023

[2] [2]

emnlp-main.243/

URLhttps://aclanthology.org/2021. emnlp-main.243/. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V ., Goyal, N., K ¨uttler, H., Lewis, M., Yih, W.-t., Rockt¨aschel, T., et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020. Li, X., Wang, S., Zeng, S., Wu, Y...

work page doi:10.18653/v1/2021.acl-long 2021

[3] [3]

Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security

URLhttps://aclanthology.org/2021. acl-long.353/. Li, Y ., Wen, H., Wang, W., Li, X., Yuan, Y ., Liu, G., Liu, J., Xu, W., Wang, X., Sun, Y ., et al. Personal llm agents: Insights and survey about the capability, efficiency and security.arXiv preprint arXiv:2401.05459, 2024b. Mysore, S., Lu, Z., Wan, M., Yang, L., Sarrafzadeh, B., Menezes, S., Baghaee, T.,...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2024.customnlp4u-1 2021

[4] [4]

gpt-oss-120b & gpt-oss-20b Model Card

URLhttps://aclanthology.org/2024. customnlp4u-1.16/. Neverova, N., Wolf, C., Lacey, G., Fridman, L., Chandra, D., Barbello, B., and Taylor, G. Learning human identity from motion patterns.IEEE Access, 4:1810–1820, 2016. OpenAI. gpt-oss-120b & gpt-oss-20b model card, 2025. URLhttps://arxiv.org/abs/2508.10925. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wai...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2020 2024

[5] [5]

soft prompts

URLhttps://openreview.net/forum? id=QWunLKbBGF. Zheng, L., Chiang, W.-L., Sheng, Y ., Li, T., Zhuang, S., Wu, Z., Zhuang, Y ., Li, Z., Lin, Z., Xing, E. P., Gonzalez, J. E., Stoica, I., and Zhang, H. Lmsys-chat-1m: A large- scale real-world llm conversation dataset, 2023. Zhong, W., Guo, L., Gao, Q., Ye, H., and Wang, Y . Mem- orybank: Enhancing large lan...

2023

[6] [6]

Either the user preference or the question is missing, so the retrieval target cannot be precisely defined

[7] [7]

Questions rarely induce preference conflicts, making violations unlikely and the retrieval task non-discriminative

[8] [8]

I avoid electric vehicles,

No gold labels tying (preference, question) pairs to documents that both answer the query and satisfy preferences. In light of these limitations of existing datasets, this study makes extensive use of the PrefEval benchmark (Zhao et al., 2025). A.5. PrefEval Benchmark The Explicit Preference subset of PrefEval dataset (Zhao et al., 2025) focuses on prefer...

2025

[9] [9]

a preference statement (clear like/dislike or constraint), and

[10] [10]

a query that can easily elicit a default answer which would violate that preference unless the model takes it into account (e.g., recommending the best compact cars for city driving, where the most top options are electric vehicles),

[11] [11]

This subset deliberately booby-traps the obvious answer: the quickest generic response is often preference-inconsistent

optionally, a short explanation/rationale highlighting why the query is risky with respect to the preference. This subset deliberately booby-traps the obvious answer: the quickest generic response is often preference-inconsistent. Strong performance therefore requires the model to (1) recognize the explicit constraint, (2) prioritize it alongside topical ...

[12] [12]

Preference-Unaware Violation: The LLM provides generic recommendations that contradict the user’s stated prefer- ence due to unawareness of user preference

[13] [13]

Preference Hallucination Violation: The response fabricates or misattributes preferences, diverging from the user’s true preference and violates the true preference

[14] [14]

Inconsistent Violation: The response acknowledges the correct preference but generates contradicting response

[15] [15]

role: content

Unhelpful Response: The response lacks relevant recommendations or fails to address the query due to poor recall of the user’s preference. B. Experimental Details 15 From Volume to Value: Preference-Aligned Memory Construction for On-Device RAG B.1. Corpus of Preference Benchmarks This section describes the retrieval corpora used for indexing and retrieva...

work page arXiv 2025

[16] [16]

the question directly contradicts the user’s preference, such that any answer would inherently violate the preference

[17] [17]

the question is already perfectly aligned with the preference, such that no additional reasoning about the preference is required

[18] [18]

For PrefRQ, since the dataset is pre-filtered to contain highly subjective questions from the Researchy Questions corpus, only conditions (1) and (2) are checked

the question has a negligible probability of violating the preference under the PrefEval data generation prompt, i.e., whenP(answer|question)≪P(answer|preference,question), indicating that even without conditioning on the preference, natural answers rarely conflict with it For PrefELI5, all three conditions are applied. For PrefRQ, since the dataset is pr...

[19] [19]

I prefer vegetarian meals

Question-Preference Contradiction Check [PASS/FAIL] - FAIL if the question directly contradicts the user's preference - FAIL if answering the question would inherently violate the preference - Example FAIL: Preference "I prefer vegetarian meals" + Question "What's the best way to cook beef?"

[20] [20]

I love Italian food

Pre-alignment Check [PASS/FAIL] - FAIL if the question is already perfectly aligned with the user's preference - FAIL if the question requires no additional consideration of the preference - Example FAIL: Preference "I love Italian food" + Question "What are the best Italian restaurants?"

[21] [21]

I prefer companies that allow unlimited sick days

Low Violation Check [PASS/FAIL] - FAIL if the question has a low probability of violating the preference - FAIL if P(answer|question) << P(answer|preference, question), which means without knowing the preference, naturally answering the question rarely violates the user's preference - Example FAIL: Preference "I prefer companies that allow unlimited sick ...

[22] [22]

Understand all user preferences thoroughly

[23] [23]

Read the given document chunk

[24] [24]

If the chunk contains no content relevant to any of the preferences, decide: Discard

[25] [25]

If the chunk is relevant to any preference, decide: Keep

[26] [26]

Always explain the reason clearly

[27] [27]

If Keep, specify exactly which preferences the chunk aligns with

[28] [28]

</planning_steps> <guidelines> - Do not infer unstated preferences

Output must strictly follow the XML structure and include only XML. </planning_steps> <guidelines> - Do not infer unstated preferences. - When listing <relevant_preferences>, use the exact preference texts as provided by the user, do not paraphrase or modify. </guidelines> <response_requirements> - Every output must follow strict XML format. - The <reason...

[29] [29]

Read the user's stated preferences

[30] [30]

Read the document chunk

[31] [31]

Read the given reason for why this chunk was marked as relevant

[32] [32]

Generate a clear, concise instruction that explains how to interpret or read this chunk in light of the relevant preferences

[33] [33]

The instruction should guide readers on what aspects to focus on or what perspective to take when reading the chunk

[34] [34]

</planning_steps> <guidelines> - The instruction is NOT a rewrite of the chunk itself, but rather guidance on how to interpret it

Output must consist of a single <instruction> XML tag. </planning_steps> <guidelines> - The instruction is NOT a rewrite of the chunk itself, but rather guidance on how to interpret it. - Focus on directing attention to preference-relevant aspects of the content. - Keep instructions concise and actionable. - Do not add information not present in the chunk...

1965