Pith Number

pith:CRNS7LGM

pith:2026:CRNS7LGMCSHJZ6IM7BF5WCESGL

not attested not anchored not stored refs pending

Beyond Fixed Benchmarks and Worst-Case Attacks: Dynamic Boundary Evaluation for Language Models

Da Yu, Haoxiang Wang, Huishuai Zhang

Evaluating language models at each model's 0.5 success probability boundary reveals capability gaps that fixed benchmarks miss.

arxiv:2605.06213 v2 · 2026-05-07 · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{CRNS7LGMCSHJZ6IM7BF5WCESGL}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

We argue that the most informative evaluation signal lies at the boundary, where the per-prompt pass probability is near 0.5 under random-sampling decoding, and propose Dynamic Boundary Evaluation (DBE), which actively locates each model's boundary and places it on a globally comparable difficulty scale.

C2weakest assumption

That the per-item difficulty labels validated across 9 reference LLMs will allow accurate placement of new models on the same scale, and that the boundary at 0.5 probability is indeed the most informative point for evaluation.

C3one line summary

Dynamic Boundary Evaluation adaptively identifies each LLM's performance boundary on a shared difficulty scale using a calibrated item bank and a search algorithm.

Receipt and verification

First computed	2026-05-27T02:05:21.280792Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

145b2faccc148e9cf90cf84bdb089232dda0f6a930a35d3aceefc0edcc31e564

Aliases

arxiv: 2605.06213 · arxiv_version: 2605.06213v2 · doi: 10.48550/arxiv.2605.06213 · pith_short_12: CRNS7LGMCSHJ · pith_short_16: CRNS7LGMCSHJZ6IM · pith_short_8: CRNS7LGM

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/CRNS7LGMCSHJZ6IM7BF5WCESGL \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 145b2faccc148e9cf90cf84bdb089232dda0f6a930a35d3aceefc0edcc31e564

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "8fc4fa04fdb801a1bec4f8acff3dfb7c2bcd45a5d807a3ca47cfc98e1e544b09",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.AI",
    "submitted_at": "2026-05-07T13:15:31Z",
    "title_canon_sha256": "a710aa693ed0ba3acae331e88927f7353b96133d743204884b058c359ec47750"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.06213",
    "kind": "arxiv",
    "version": 2
  }
}