Pith. sign in

REVIEW 2 major objections 2 minor 68 references

ProActor uses automated opportunity window labels and GRPO reinforcement learning to improve proactive timing in task scheduling agents.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 11:48 UTC pith:HXW43BMR

load-bearing objection ProActor packages auto time-window annotation and stage-aware GRPO rewards for proactive scheduling, but the timing gains sit on unvalidated labels. the 2 major comments →

arxiv 2605.24900 v1 pith:HXW43BMR submitted 2026-05-24 cs.AI

ProActor: Timing-Aware Reinforcement Learning for Proactive Task Scheduling Agents

Lei Ding , Bin He , Chenguang Wang , Yang Liu This is my paper
classification cs.AI
keywords proactive schedulingreinforcement learningtiming optimizationopportunity time windowsautomated annotationGRPOconversational agentstask scheduling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops ProActor to shift conversational agents from reacting only to explicit commands toward anticipating user needs and acting at suitable moments. It achieves this by automatically labeling training data with ranges of good opportunity times instead of single points, defining metrics that track both timing quality and action match to references, and applying GRPO reinforcement learning with reward designs that emphasize proactiveness rubrics and stage-aware composites. Experiments on two new auto-annotated datasets show gains in timing while action consistency stays comparable to existing baselines. If the approach works, agents could handle tasks with less constant user input and more natural initiative.

Core claim

ProActor integrates a domain-agnostic automated annotation methodology that generates full opportunity time windows for scalable proactiveness RL, systematic metrics capturing timing quality and reference action alignment, and RL optimization using GRPO with RULER-based rewards incorporating proactiveness rubrics and stage-aware composite rewards. This setup produces significant improvements in proactive timing while maintaining action consistency comparable to state-of-the-art baselines on the auto-annotated datasets, with ablations confirming the role of the distinct reward variations.

What carries the argument

Automated annotation generating full opportunity time windows combined with GRPO RL optimization using RULER-based proactiveness rubrics and stage-aware composite rewards.

Load-bearing premise

The domain-agnostic automated annotation methodology produces opportunity time windows that are sufficiently accurate and unbiased to support effective proactiveness reinforcement learning without introducing systematic errors in timing or action labels.

What would settle it

If models trained on the auto-annotated data show no timing improvement or worse performance when evaluated against human-verified timing windows on the same tasks, the central claim would not hold.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Agents learn to trigger actions within learned opportunity time windows rather than at fixed points.
  • Training data creation scales without requiring manual point-in-time labels for every example.
  • RULER-based rewards with proactiveness rubrics specifically raise timing quality.
  • Stage-aware composite rewards let the system trade off timing gains against reference action alignment.
  • The ART-F infrastructure supports efficient training of large models such as 14B-parameter agents with substantial speedups.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The window-based annotation and reward approach could extend to timing decisions in non-conversational agents such as web or planning systems.
  • Success with composite rewards suggests similar reward engineering may help other RL agent tasks that balance multiple soft objectives.
  • If the auto-annotation proves reliable, it lowers the barrier to creating large datasets for proactive behavior training.
  • The method implies agents could eventually reduce the frequency of explicit user instructions in routine interactions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces ProActor, a unified framework for conversational task scheduling agents that integrates a domain-agnostic automated annotation methodology generating full opportunity time windows (instead of point labels) to enable scalable proactiveness RL via GRPO, systematic metrics for timing quality and action alignment, and various reward designs including RULER-based and stage-aware composite rewards. It also presents the ART-F infrastructure for efficient adaptive inference and DDP-based LoRA training of models like Qwen2.5-14B. Experiments on two newly auto-annotated datasets are claimed to show significant improvements in proactive timing while maintaining action consistency comparable to SOTA baselines, with ablations on reward variations.

Significance. If the auto-annotation is shown to be accurate and unbiased and the experimental results hold with proper controls, the work could advance proactive agents by providing a scalable end-to-end RL approach for anticipatory timing in task scheduling, addressing a gap in existing reactive systems. The shift to full opportunity windows and the practical ART-F training framework are notable engineering contributions that could support further research in timing-aware RL.

major comments (2)
  1. [Abstract] Abstract: the abstract asserts 'significant improvements in proactive timing' from experiments on two auto-annotated datasets but supplies no quantitative results, baseline details, statistical tests, or error analysis, leaving the central claim without visible supporting evidence.
  2. [Automated annotation methodology] Automated annotation methodology (methods section): the domain-agnostic automated annotation for opportunity time windows is presented without reported human validation, inter-annotator agreement, human-vs-auto overlap statistics, or bias/ablation checks isolating annotation error. This is load-bearing because every reported metric (timing quality, action consistency) is computed relative to these same auto-labels; without validation, GRPO gains may reflect label artifacts correlated with the reward rubrics rather than improved real-world proactiveness.
minor comments (2)
  1. [Abstract] Abstract: the claim of 'maintaining action consistency comparable to state-of-the-art (SOTA) baselines' does not name the specific baselines or metrics used for comparison.
  2. [Abstract] The paper mentions 'RULER-based rewards with proactiveness rubrics' but does not clarify in the abstract how these rubrics avoid circularity with the annotation procedure.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and indicate the revisions we will make to the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the abstract asserts 'significant improvements in proactive timing' from experiments on two auto-annotated datasets but supplies no quantitative results, baseline details, statistical tests, or error analysis, leaving the central claim without visible supporting evidence.

    Authors: We agree that the abstract would be strengthened by including quantitative support for the claims. In the revised version we will expand the abstract to report key numerical results from the experiments section (e.g., timing-quality deltas versus baselines and action-alignment scores) while preserving the overall length and readability. revision: yes

  2. Referee: [Automated annotation methodology] Automated annotation methodology (methods section): the domain-agnostic automated annotation for opportunity time windows is presented without reported human validation, inter-annotator agreement, human-vs-auto overlap statistics, or bias/ablation checks isolating annotation error. This is load-bearing because every reported metric (timing quality, action consistency) is computed relative to these same auto-labels; without validation, GRPO gains may reflect label artifacts correlated with the reward rubrics rather than improved real-world proactiveness.

    Authors: We acknowledge that the current manuscript does not report human validation of the automated annotation. Because the metrics are computed against these labels, we will add a dedicated validation subsection (including human-auto overlap statistics, a small-scale bias audit, and inter-annotator agreement on a held-out sample) to the methods or experiments section in the revision. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The provided abstract and context describe an automated annotation procedure, proactiveness metrics, and GRPO-based RL with RULER rewards derived from rubrics. No equations, self-citations, or derivation steps are quoted that reduce any reported prediction, metric, or result directly to the inputs by construction (e.g., no fitted parameter renamed as a prediction, no load-bearing self-citation chain, and no definitional equivalence between annotation outputs and evaluation targets). The central claims rest on empirical improvements measured against the generated labels rather than on definitional or fitted equivalence, making the derivation self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The abstract does not enumerate explicit free parameters, axioms, or invented entities; the framework appears to rest on standard RL assumptions and the unstated validity of the automated annotation process.

pith-pipeline@v0.9.1-grok · 5792 in / 1053 out tokens · 29326 ms · 2026-06-30T11:48:21.657776+00:00 · methodology

0 comments
read the original abstract

Proactive task-oriented agents must autonomously anticipate user needs, identify actionable opportunities, and trigger software actions at appropriate moments - fundamentally shifting from reactive systems that await explicit instructions. However, existing approaches lack generalizable end-to-end solutions for measuring and optimizing such anticipatory behaviors. This paper introduces ProActor, a unified framework for conversational task scheduling that integrates: (1) a domain-agnostic automated annotation methodology that enables scalable proactiveness reinforcement learning (RL) by generating full opportunity time windows instead of rigid point labels, (2) systematic proactiveness metrics capturing both timing quality and reference action alignment, and (3) RL optimization using GRPO with various reward designs. Our insight is that RULER-based rewards with proactiveness rubrics are crucial for improving timing quality, and that proactiveness optimization enabled by stage-aware composite rewards is key to balancing timing quality and reference action alignment. Timing-aware RL requires extensive exploration, demanding efficient infrastructure. We develop ART-F, an adaptive framework combining request-adaptive inference clusters with DDP-based training on single-node multi-GPU systems, enabling LoRA training of 4-bit Qwen2.5-14B-ProActor-Q4 with 4-8x speedups. Experiments on two newly auto-annotated datasets demonstrate significant improvements in proactive timing while maintaining action consistency comparable to state-of-the-art (SOTA) baselines. Ablations validate the effectiveness of distinct composite reward variations.

Figures

Figures reproduced from arXiv: 2605.24900 by Bin He, Chenguang Wang, Lei Ding, Yang Liu.

Figure 1
Figure 1. Figure 1: Our ProActor shifts conversational agents [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of ProActor for end-to-end proac [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Automated annotation pipeline: Heteroge [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Turn-level GRPO optimization: we sample K action candidates from the policy model πθ(u) for each dialogue turn, score action trajectories with the chosen reward, update πθ(u) to πθ(u + 1) using GRPO with reward-weighted gradients. Reward Granularity Decision In general, trajectory-level rewards, which average action feedback over an entire dialogue, are ill-suited for task scheduling: long dialogues (≥ 23 … view at source ↗
Figure 5
Figure 5. Figure 5: ART-F framework: An efficient collocated [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Effect of λmax on Adaptive RULER weighted by Max RAC (blue) and RAC (orange), showing how Average Gradient of six main metrics change with λmax ∈ {0, 0.3, 0.5, 0.75, 1.0}. 5 Conclusion We present ProActor, a unified framework for train￾ing timing-aware proactive task-scheduling agents. Our approach integrates: (1) a domain-agnostic an￾notation pipeline generating reference actions with multiple valid timin… view at source ↗
Figure 7
Figure 7. Figure 7: The annotation data pipeline unifies domain [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: A JSON snippet of Tool Catalog 1. Context Analysis: Processing the full dia￾logue to understand user intent and informa￾tion state 2. Opportunity Identification: Matching dia￾logue context against available tools to iden￾tify actionable moments 3. Parameter Extraction: Determining which tool parameters can be filled from the conver￾sation 4. Readiness Assessment: Evaluating whether sufficient information e… view at source ↗
Figure 9
Figure 9. Figure 9: Oracle Agent Configuration Threshold Number of Actions Percentage of Actions Dataset Score Dataset Score σ with EC(g) = 1.0 with EC(g) = 1.0 EC(G) EC(G) > 0.8 4 3,477 11.78% 0.1217 ± 0.2443 4.07% 3 4,982 16.88% 0.1837 ± 0.2883 6.62% 2 7,424 25.16% 0.2850 ± 0.3326 11.42% 1 12,602 42.71% 0.4682 ± 0.3499 21.10% 0 25,715 87.14% 0.8877 ± 0.1926 70.27% [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Training-free baseline Workflow: We initial [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: SFT baseline Workflow: We integrate the ArcticTraining Framework into the baseline workflow by customizing data format, plugging the evaluation at the fixed steps, and simplifying the analysis step, which is highlighted in purple color in the diagram. SFT Comparison. We also include supervised fine-tuning (SFT) as a baseline for Qwen models. However, from the design perspective, our appli￾cation domain re… view at source ↗
Figure 12
Figure 12. Figure 12: Action State Graph (ASG). The left panel illustrates the ASG overview, while the right panel presents [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: JSON representation of action states used in the Reasoning+ASG prompt to replace the action_states variable. C.4 Agent Configuration Similar to the oracle annotation agent configura￾tion (Figure B.2), we provide a YAML configura￾tion file for baseline initialization. This file spec￾ifies the LLM backend, tool catalog settings, and the prompts used for action opportunity predic￾tion and question generation… view at source ↗
Figure 14
Figure 14. Figure 14: YAML Configuration of Direct Prompting Baseline Agent states and predictions ( [PITH_FULL_IMAGE:figures/full_fig_p021_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: YAML Configuration of Reasoning Baseline [PITH_FULL_IMAGE:figures/full_fig_p022_15.png] view at source ↗
Figure 18
Figure 18. Figure 18: Web-based visualization interface for base [PITH_FULL_IMAGE:figures/full_fig_p023_18.png] view at source ↗
Figure 17
Figure 17. Figure 17: LLM Judger Configuration: Proactiveness evaluation with 5-point rating scale (effective batch size 6). We use a learning rate of 2 × 10−5 , weight decay 0.01, and a co￾sine learning-rate scheduler with 5% warmup. DeepSpeed ZeRO Stage 3 is applied to en￾able memory-efficient full-parameter training. The maximum sequence length is set to 8192 tokens. To integrate the proactiveness LLM judge, we extend the e… view at source ↗
Figure 19
Figure 19. Figure 19: Inference Phase Summary: During inference, [PITH_FULL_IMAGE:figures/full_fig_p024_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: DDP-Based Training Phase Summary: ART￾F orchestrates distributed data-parallel (DDP) training by initializing a master process and multiple worker processes, distributing training payloads—rollout trajec￾tory groups with rewards—across workers, coordinat￾ing checkpoints, and aggregating progress and results throughout training. centrally managed controller process orchestrates multiple training workers. D… view at source ↗
Figure 21
Figure 21. Figure 21: ART-F Client Workflow: The training client [PITH_FULL_IMAGE:figures/full_fig_p027_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: ART-F Client YAML Configuration [PITH_FULL_IMAGE:figures/full_fig_p028_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Our Custom RULER Setting for proactive￾ness evaluation api_key_name, base_url, and an optional custom_ruler_placeholder. If no custom rules are provided, ART-F uses the default RULER rubric inherited from ART, referred to as General RULER (Section 3.3). When custom rules are specified, they are appended to the judger prompt as additional evaluation criteria, yielding Custom RULER (Section 3.3) [PITH_FULL… view at source ↗
Figure 24
Figure 24. Figure 24: LLM Judger Prompt for Trajectory-Level Proactiveness Evaluation with Custom RULER Injec￾tion { "scores": [ { "trajectory_id": "1", "explanation": "<explanation for rating>", "score": 0.95 }, { "trajectory_id": "2", "explanation": "<explanation for rating>", "score": 0.90 }, { "trajectory_id": "3", "explanation": "<explanation for rating>", "score": 0.85 }, . . . { "trajectory_id": "N", "explanation": "<ex… view at source ↗
Figure 25
Figure 25. Figure 25: JSON Response Format Returned by the LLM Judger for Proactiveness Evaluation hybrid_ruler_weighted_max_rac_score. 2. Stage-aware weighted rewards. In contrast, this category explicitly incorporates training progress u U into reward computation by adapt￾ing metric weights according to the current training step u relative to the total number of steps U, including adaptive_metric_score, schedule_ruler_weight… view at source ↗
Figure 26
Figure 26. Figure 26: LoRA configuration used in ART-F. LoRA Setting We use LoRA with rank r=8 and scaling factor α=16, applied to attention (q, k, v, o) and MLP (gate, up, down) projections, with dropout disabled for maximal throughput, which aligns with the default ART-F PEFT setting as shown in [PITH_FULL_IMAGE:figures/full_fig_p034_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Turn-level(blue) vs Trajectory-level(orange) [PITH_FULL_IMAGE:figures/full_fig_p042_27.png] view at source ↗
Figure 29
Figure 29. Figure 29: Turn-wise(blue) vs Trajectory-wise(orange) [PITH_FULL_IMAGE:figures/full_fig_p043_29.png] view at source ↗
Figure 28
Figure 28. Figure 28: Turn-level(blue) vs Trajectory-level(orange) [PITH_FULL_IMAGE:figures/full_fig_p043_28.png] view at source ↗
Figure 30
Figure 30. Figure 30: Turn-wise(blue) vs Trajectory-wise(orange) [PITH_FULL_IMAGE:figures/full_fig_p044_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: Training curve consistency analysis across [PITH_FULL_IMAGE:figures/full_fig_p046_31.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 3 canonical work pages · 1 internal anchor

  1. [1]

    LoRA: Low-Rank Adaptation of Large Language Models

    Unsloth. https://github.com/unslothai/ unsloth. Accessed: 2025-03. Brad Hilton, Kyle Corbitt, David Corbitt, Saumya Gandhi, Angky William, Bohdan Kovalenskyi, and Andie Jones. 2025. Art: Agent reinforcement trainer. https://github.com/openpipe/art. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Che...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  2. [2]

    JinJa Templating1. Unify

  3. [3]

    Agent initialization Parallel Processing Setup Batch Annotation Processing Output Annotated Data Figure 7: The annotation data pipeline unifies domain metadata and dialogue inputs into standardized tool cat- alogs, initializes a configurable AI agent, and performs scalable batch annotation to produce structured anno- tated outputs. B.1 Metadata Specificat...

  4. [4]

    Ontology: a JSON file that defines the hierar- chical structure of actions, the workflows com- posed of these actions, and their inter-action relationships

  5. [5]

    Type specification: a JSON file that describes each supported action, including its semantic description and detailed parameter types

  6. [6]

    To derive this, we employ a heuristic approach that generates automa- tion samples from system interactions and evaluates parameter presence across all cases

    Parameter property estimation: a JSON file that determines whether each parameter is re- quired or optional. To derive this, we employ a heuristic approach that generates automa- tion samples from system interactions and evaluates parameter presence across all cases. Parameters observed in every instance are marked asrequired, while others are marked asop...

    2025

  7. [7]

    MCP (Model Context Protocol) Tools: Stan- dard MCP-compliant tools with JSON schema validation, providing native compatibility for modern LLM tool use

  8. [8]

    API Definitions: REST API endpoints with configurable headers, methods, and param- eters, enabling integration with existing ser- vices

  9. [9]

    catalog_metadata

    Custom AI Agents: Complex multi-step workflows and specialized automation rou- tines that extend beyond simple API calls. Essentially, this unified workflow converts domain-specific metadata into standardized and consistent tool catalogs that integrate seamlessly with the annotation pipeline, independent of the underlying business domain. B.1.2 Configurat...

  10. [10]

    Context Analysis: Processing the full dia- logue to understand user intent and informa- tion state

  11. [11]

    Opportunity Identification: Matching dia- logue context against available tools to iden- tify actionable moments

  12. [12]

    Parameter Extraction: Determining which tool parameters can be filled from the conver- sation

  13. [13]

    Readiness Assessment: Evaluating whether sufficient information exists to trigger each tool

  14. [14]

    gpt-4.1-mini

    Quality Scoring: Assigning confidence scores to filter high-quality annotations The oracle sees the complete conversation (past and future), providing reference annotations for what proactive actions would have been beneficial at each turn. This hindsight-enabled annotation creates realistic training targets that online agents can learn to approximate. B....

  15. [15]

    Action opportunities that could be triggered

  16. [17]

    Readiness maturity of participants

  17. [18]

    Confidence in triggering the action

  18. [19]

    {current_text}

    Current status of the action trigger Always provide output in valid JSON format following the specified schema exactly. task_prompt: | ## TASK Analyze the following dialogue and annotate turn {turn_number} ## FULL DIALOGUE CONTEXT {dialogue_context} ## CURRENT TURN TO ANNOTATE Turn {turn_number}: {current_speaker} says: "{current_text}" ## A V AILABLE ACT...

  19. [20]

    Overall coverage: Percentage of triggered actions whose oracle annotation satisfies both early-ready and transition criteria at the most lenient threshold (threshold=0)

  20. [21]

    Oracle annotation coverage: Percentage of triggered actions that have any oracle anno- tation at all (i.e., the oracle recognized the action exists)

  21. [22]

    Score consistency: Standard deviation of per- dialogue quality scores — lower means more uniform annotation quality across dialogues

  22. [23]

    Turn gap precision: Average number of turns between the oracle’s earliest ready prediction and the actual trigger — smaller means more precise timing

  23. [24]

    Phantom noise rate: Percentage of oracle- annotated action ranges that refer to actions the assistant never actually triggered — pure noise in training data

  24. [25]

    As reflected by Table 4, the annotation agent with future-viewing outperforms the setting skip- ping the future turn information

    Critical action miss rate: Miss rate for pull- up-account, the most frequent foundational action — a proxy for how well the oracle cap- tures essential workflow steps. As reflected by Table 4, the annotation agent with future-viewing outperforms the setting skip- ping the future turn information. We furtherquantify the impact on training data qualityin Ta...

  25. [26]

    C.1 Design Rationale Prompting Strategy Design.Our three prompt- ing strategies progressively externalize reasoning and memory

    to validate the necessity of RL adaptation. C.1 Design Rationale Prompting Strategy Design.Our three prompt- ing strategies progressively externalize reasoning and memory. Non-Reasoning relies solely on in- ternal model knowledge, Reasoning makes deliber- ation explicit prior to action prediction, and ASG further maintains temporal state across turns. Eac...

  26. [27]

    Analyze Input Conversation Data MCP Tooling-Style Action Catalog YAML file Agent Configuration Baseline Agent Python Tooling API

  27. [28]

    Load & Validate Data Turn-by-Turn Prediction & Evaluation Output Prediction Data Metrics & Statistics Collection Visualization & Analysis

  28. [29]

    The agent then op- erates in a turn-by-turn manner, generating predictions while collecting performance statistics

    Predict Proactiveness LLM Judger Figure 10: Training-free baseline Workflow: We initial- ize each training-free baseline agent using the unified action catalog and its configuration. The agent then op- erates in a turn-by-turn manner, generating predictions while collecting performance statistics. Beyond prede- fined metrics, an LLM-based judger evaluates...

  29. [30]

    Analyze Format Conversion MCP Tooling-Style Action Catalog YAML file Configuration ArcticTraining Python Tooling API

  30. [31]

    Load & Validate Data Evaluating Output Prediction Data Metrics & Statistics Collection Analysis

  31. [32]

    think about specific aspects

    Predict Proactiveness LLM Judger Annotated Dataset Figure 11: SFT baseline Workflow: We integrate the ArcticTraining Framework into the baseline workflow by customizing data format, plugging the evaluation at the fixed steps, and simplifying the analysis step, which is highlighted in purple color in the diagram. SFT Comparison.We also include supervised f...

  32. [33]

    YAML configuration of Direct Prompting agent baseline, refer to Figure 14

  33. [34]

    YAML configuration for Reasoning agent baseline, refer to Figure 15

  34. [35]

    openai/gpt-4.1-mini

    YAML configuration for ASG+reasoning agent baseline, refer to Figure 16 Due to page length, we omit the schema struc- tures of action opportunities and raised questions, which are similar to action opportunities specifica- tion in Y AML file 9. Besides, a proactiveness LLM judger can be configured via the llm_judger_config_file command-line argument, whic...

  35. [36]

    Action opportunities that could be triggered once ready

  36. [37]

    Required and optional inputs for each action

  37. [38]

    Parameter Readiness (readiness_maturity)

  38. [39]

    Confidence in triggering the action (trigger_confidence)

  39. [40]

    proactive_action_opportunities

    Current status of the action trigger (action_trigger_status) ## A V AILABLE ACTION OPPORTUNITIES <tools> {tool_catalog} </tools> ## RESPONSE SCHEMA {{ "proactive_action_opportunities": [...] }} task_prompt: | ## TASK Analyze the following conversation for proactive automation opportunities. Provide only the JSON response, no additional text. ## DIALOGUE C...

  40. [41]

    openai/gpt-4.1-mini

    LoRA Tuning. On 2 ×H100 GPUs, we ap- ply Low-Rank Adaptation (LoRA) (Hu et al., # Baseline Configuration File # Configuration for LLM-based actor model # LLM Settings llm: model: "openai/gpt-4.1-mini" temperature: 0.1 max_tokens: 4096 using_short_model_name: true # Tool Catalog Settings tool_catalog: use_common_tools: true custom_tools: {} # Prompt Settin...

  41. [42]

    proactiveness_evaluator

    Full Tuning. On 2 ×H200 GPUs, we fine- tune all model parameters with a micro-batch size of 1 and gradient accumulation of 6 steps # LLM Judger Configuration File for ASG Support judgers: - name: "proactiveness_evaluator" model: "openai/gpt-4.1-mini" temperature: 0.1 max_tokens: 4096 using_short_model_name: true api_key_name: <INTERNAL_API_KEY> base_url: ...

  42. [43]

    Determine if the customer explicitly requested the predicted action

  43. [44]

    reasoning

    Assess the proactiveness level — how well the prediction matches the customer’s intent without a direct request. # RATING CRITERIA The proactiveness level is calculated based on: -1 - Conversation is insufficient to judge. 1 - Explicit or almost explicit request, target action missing. 2 - Explicit request, target action present, inaccurate parameters. 3 ...

    2023

  44. [45]

    After the training comple- tion, we resume monitoring for the leftover server and scale up back to N server and pre- pare for next rollout cycle

    For Non-DDP training, we scale down the vLLM server instances to only keep the server with the lowest port number and pause server monitor before training so that GPU memory locked by N-1 servers can be utilized in the following training. After the training comple- tion, we resume monitoring for the leftover server and scale up back to N server and pre- p...

  45. [46]

    Once the training is completed, we restart the whole in- ference cluster and enter the next rollout cycle

    For DDP training, we shut down the inference cluster completely and let the DDP training manager start the training cycle. Once the training is completed, we restart the whole in- ference cluster and enter the next rollout cycle. The reason is that in ART-F’s DDP training, we trigger a multi-processes collaboration to not only resume the normal DDP traini...

  46. [47]

    Specifically, we initialize the process con- text using mp.get_context("spawn"), and use mp.Process to launch N worker processes and cre- ate the shared M Q within the same context

    setting up the DDP process group and initializ- ing N worker processes; and 2) preparing a shared asynchronous queue—referred to as the Master Queue M Q — through which worker processes report their execution progress. Specifically, we initialize the process con- text using mp.get_context("spawn"), and use mp.Process to launch N worker processes and cre- ...

  47. [48]

    We allow developers to extend the exist- ing MlflowSystemMonitor to enables more customized monitoring

    Integrated experiment tracking.Besides wandb.ai (Weights & Biases, 2025b), ART-F provides built-in MLflow integration with ex- tended system-level metrics, including GPU memory utilization, disk I/O, and network ac- tivity. We allow developers to extend the exist- ing MlflowSystemMonitor to enables more customized monitoring

  48. [49]

    Trajectory saving via W&B Weave (Weights & Biases, 2025a) is made optional to mitigate potential sensitive data leakage during training

    Customizable RULER module.We extend the original RULER to support user-defined evaluation rules (e.g., proactiveness assess- ment rubrics) and seamless integration of third-party models. Trajectory saving via W&B Weave (Weights & Biases, 2025a) is made optional to mitigate potential sensitive data leakage during training

  49. [50]

    avalanche

    Databricks compatibility enhancements. ART-F includes additional utilities to facilitate Databricks-based training work- flows (Databricks, Inc., 2025), such as dynamic server address resolution at runtime, temporary directory management for memory- mapped files, trajectory compression, and incremental checkpoint saving. Data Size ART (1×H200) ART-F (1×H2...

    2025

  50. [51]

    Dataset splitting into training, validation, and test sets is also performed at this stage

    Initialization: The client first converts the raw dataset into rollout scenarios represented as serializable Python objects compatible with Pydantic.BaseModel, enabling them to be trans- mitted to the inference servers via HTTP requests. Dataset splitting into training, validation, and test sets is also performed at this stage. Next, the client loads a YA...

    2018

  51. [52]

    Inference & Training Loop: Once the ART-F backend reaches the ready state, the client initiates parallel rollouts, with each inference server han- dling concurrent requests under semaphore-based rate limiting. During rollout, each scenario is ex- ecuted T times as specified by the client configu- ration, and each resulting trajectory is evaluated using me...

    2018

  52. [53]

    <Your project Name>

    training parameters, and prompt templates for action prediction. By externalizing these com- ponents into a declarative configuration, ART-F enables flexible experimentation and rapid system reconfiguration without modifying the client source code. In this section, we will discuss the key settings for inference cluster and DDP training. Details of the rol...

  53. [54]

    Assis- tant

    Action opportunities that could be triggered once ready ...... task_prompt: | ## CONVERSATION AT TURN {turn_idx} {dialogue_context} Figure 22: ART-F Client Y AML Configuration LLM Rollout SettingTo reduce rollout resource consumption, we support asym- metric sampling settings for training and validation during rollout, configured via sample_num_per_traini...

    2024

  54. [55]

    content":

    Weighted rewards.This category combines multiple metrics usingfixed weighting coeffi- cients specified in the client configuration. The weighting scheme is independent of training progress and does not require access to the current or total number of training steps, including weighted_max_rac_score, weighted_rac_score, hybrid_ruler_weighted_rac_score, and...

    work page arXiv 2025

  55. [56]

    Dimensional Reduction: Aggregating mul- tiple correlated metrics into interpretable in- dices

  56. [57]

    Balanced Evaluation: Using harmonic mean to prevent one dimension from dominating

  57. [58]

    Relative Ranking: Enabling fair comparison across different dataset scales and characteris- tics I.4.2 Mathematical Formulation Given a set of evaluation metrics M≡ {AC,Max_AC,Difference,PT,FTR,RAR} and a comparison group G={M 1, M2, . . . , Mn} evalu- ated on the same dataset, we compute PRI through the following steps: Step 1: Metric NormalizationFor ea...

  58. [59]

    Ensuring minimum group size of 3 models when possible

  59. [60]

    Reporting raw metric values alongside PRI scores

  60. [61]

    when to act

    Documenting outlier cases (e.g., the ABCD+ 3000/600 performance degradation) Metric Weight AssumptionsThe equal weight- ing of AC, Max_AC, and Difference in CI assumes these metrics have similar importance. Similarly, PT, FTR, and RAR receive equal weight in TI. Fu- ture work could explore learned or domain-specific weightings. I.5 Baseline Patterns (1) R...

  61. [62]

    Underfitting (LoRA): Learns the timing sig- nal (when to act) but lacks the capacity to learn the content signal (what action to pre- dict), leading to high recall but low precision

  62. [63]

    RL’s composite reward (RAC + RULER) explic- itly optimizes both dimensions simultaneously, en- abling a calibrated trade-off that static supervised learning cannot discover

    Overfitting (Full-tune): Learns the content signal (what action to predict) but fails to gen- eralize the timing signal (when to act), leading to high precision but low proactiveness, with additional formatting errors during evaluation due to catastrophic forgetting. RL’s composite reward (RAC + RULER) explic- itly optimizes both dimensions simultaneously...

  63. [64]

    ABCD+.ProActor-Q4 + Custom RULER achieves the lowest Consistency Difference (0.136±0.048 ) with solid alignment (AC = 0.426±0.015 , Max AC = 0.484±0.019 ). The Adaptive RULER variant increases action quality substantially (AC = 0.431±0.022 , Max AC = 0.586±0.044 ) while keeping the Consistency Difference at a moderate level (0.320±0.123 ), far below those...

  64. [65]

    Home Loan.Under domain shift, Adap- tive RULER exhibits an even clearer balance. Compared to Custom RULER (AC= 0.206± 0.024, Max AC = 0.234±0.021 , Difference = 0.137±0.161 ), Adaptive RULER nearly doubles alignment quality (AC = 0.395± 0.029, Max AC = 0.466±0.038 ) while in- curring only a modest increase in Consistency Difference (0.180±0.129 ). Both va...

    work page arXiv 2030

  65. [66]

    These may be pre-suggested by the automated pipeline or identified by the annotator during review

    Action Candidate Initialization.For each dialogue, the annotator receives a set of candidate actions from the action catalog. These may be pre-suggested by the automated pipeline or identified by the annotator during review. Annotators may also flag additional actions not in the initial candidate set. 10 20 30 40 50 60 Training Step 0.020 0.025 0.030 0.03...

  66. [67]

    • READY_TO_TRIGGER: All required parameters for a are available and confirmed—the action can be appropri- ately triggered at turnt

    Turn-Level Status Assignment.At each di- alogue turn t, the annotator assigns one of the following statuses to every tracked action a∈ A: • PENDING: An opportunity for a is identified (e.g., partial prerequisites ob- served), but one or more required param- eters remain unresolved or unconfirmed. • READY_TO_TRIGGER: All required parameters for a are avail...

  67. [68]

    • (Optional, if annotation budget per- mits) The fullvalid triggering window [t∗ a, tend a ], capturing the range of turns over which triggering remains appropri- ate

    Timing Record.For each action that reaches READY_TO_TRIGGER, the anno- tator records: • Theearliest ready turn t∗ a: the first turn where all prerequisites are satisfied. • (Optional, if annotation budget per- mits) The fullvalid triggering window [t∗ a, tend a ], capturing the range of turns over which triggering remains appropri- ate. Recording only the...

  68. [69]

    missing required authorization,

    Supplementary Notes.For actions that re- main in PENDING throughout the dialogue (never reaching readiness) or that are DIS- MISSED, annotators provide a brief reason (e.g., “missing required authorization,” “user declined”). L.3 Annotator Requirements • Domain familiarity: Annotators should un- derstand the task domain sufficiently to judge whether actio...