Pith. sign in

REVIEW 2 major objections 2 minor 42 references

Professional designers rate AI graphic designs on separate axes like typography and layout, exposing gaps in existing preference models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 17:45 UTC pith:RKSCC552

load-bearing objection TASTE supplies a new multi-criteria designer preference dataset for T2I graphic design with a usable validation framework and clear MLP baseline, though the ten-designer sample limits how far the results generalize. the 2 major comments →

arxiv 2605.20731 v2 pith:RKSCC552 submitted 2026-05-20 cs.CV cs.AIstat.AP

TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Design

Haonan Zhu , Elad Hirsch , Alexandria Minetti , Allison Nulty , Purvanshi Mehta This is my paper
classification cs.CV cs.AIstat.AP
keywords preference datasetgraphic designtext-to-imagemulti-dimensional evaluationdesigner annotationsAI-generated imagespreference modeling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper releases TASTE, a dataset of designer rankings on nine criteria for images from four text-to-image models. Two groups of five designers provide the labels, and statistical tests confirm their agreement exceeds random chance. Existing VLM and T2I scorers fall short of matching the designer panel's majority votes, but a simple MLP trained on TASTE performs better and approaches the consistency of individual raters.

Core claim

TASTE supplies multi-dimensional preference data where designers rank outputs along distinct criteria including typography, aesthetics, spatial arrangement, tone, and others, plus hallucination indicators. The validation framework using Kendall's tau and majority probabilities demonstrates significant designer consensus. Benchmarking establishes that standard judges do not reach majority agreement with the panel, whereas training a small MLP head on the dataset narrows that gap substantially.

What carries the argument

The TASTE dataset of criterion-specific rankings by professional designers, validated via Kendall's τ against uniform nulls and used to train preference models.

Load-bearing premise

The annotations from the two designer cohorts capture generalizable preferences that apply to new images and models.

What would settle it

Collecting ratings from a new cohort of designers on the same or similar images and finding that their preferences show no correlation with the original TASTE labels beyond chance levels.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript releases TASTE, a multi-dimensional preference dataset for AI-generated graphic designs in which two disjoint cohorts of five professional designers each rank outputs from four text-to-image models across nine criteria plus hallucination flags. It introduces a criterion-agnostic validation framework (Kendall's τ, majority-vote probability, Condorcet cycles vs. exact iid-uniform nulls) demonstrating significant but moderate designer agreement on every criterion. Benchmarking shows off-the-shelf VLM judges and dedicated T2I scorers fail to reach majority agreement with the designer panel, while a small MLP head trained directly on TASTE narrows the gap to the single-rater ceiling.

Significance. The dataset release supplies designer-annotated, multi-axis preference signals that address the mismatch between photo-style single-verdict datasets and graphic-design evaluation needs. The validation framework and external benchmarking establish that current automated judges underperform relative to direct training on TASTE, providing a concrete baseline and falsifiable target for future preference-model work. The use of disjoint cohorts and null-model statistical tests are explicit strengths that support the data's utility if generalization holds.

major comments (2)
  1. [§3 (Validation Framework)] §3 (Validation Framework): the reported Kendall τ, majority-vote probability, and Condorcet-cycle tests reject the iid-uniform null for each criterion, but supply no cross-cohort or cross-image stability analysis; this is load-bearing for the central claim in §5 that the annotations constitute reliable training targets for the MLP.
  2. [§5 (Benchmarking)] §5 (Benchmarking): the MLP head is reported to narrow the gap to the single-rater ceiling, yet the manuscript does not describe whether the train/test split respects the disjoint-cohort structure or holds out entire image/model sets; without such a test the improvement could reflect cohort-specific or image-specific artifacts rather than generalizable designer consensus.
minor comments (2)
  1. The abstract states the validation framework and MLP improvement but omits any numerical values (e.g., median τ or majority-vote probability); adding one or two headline figures would improve readability.
  2. Tables reporting agreement metrics should include the exact sample sizes per criterion and the number of Condorcet cycles observed, to allow direct replication of the null-model comparisons.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. We address each major point below and will incorporate revisions to strengthen the validation and benchmarking sections.

read point-by-point responses

  1. Referee: [§3 (Validation Framework)] §3 (Validation Framework): the reported Kendall τ, majority-vote probability, and Condorcet-cycle tests reject the iid-uniform null for each criterion, but supply no cross-cohort or cross-image stability analysis; this is load-bearing for the central claim in §5 that the annotations constitute reliable training targets for the MLP.

    Authors: We agree that explicit cross-cohort and cross-image stability metrics would further support the claim that the annotations provide reliable training targets. The manuscript already uses two disjoint designer cohorts and reports within-cohort agreement statistics against the null model, but does not include inter-cohort comparisons. In revision we will add cross-cohort Kendall's τ, majority-vote probability, and Condorcet-cycle statistics, along with image-level stability checks (e.g., agreement on held-out images), to directly address this concern. revision: yes

  2. Referee: [§5 (Benchmarking)] §5 (Benchmarking): the MLP head is reported to narrow the gap to the single-rater ceiling, yet the manuscript does not describe whether the train/test split respects the disjoint-cohort structure or holds out entire image/model sets; without such a test the improvement could reflect cohort-specific or image-specific artifacts rather than generalizable designer consensus.

    Authors: The referee is correct that the current description of the train/test procedure in §5 is incomplete. The splits were performed at the image level while preserving the disjoint-cohort structure (no designer overlap and no shared images between cohorts), but this was not stated explicitly and no cohort-holdout results were reported. We will revise §5 to detail the splitting protocol, confirm that entire image sets and models are held out where possible, and add results under a stricter cohort-holdout evaluation to demonstrate generalizability. revision: yes

Circularity Check

0 steps flagged

No significant circularity; dataset release with external statistical validation

full rationale

The paper is a data release (TASTE annotations from two designer cohorts) paired with benchmarking of existing judges and a simple MLP trained on the new data. The validation framework applies standard non-parametric tests (Kendall τ, majority vote, Condorcet cycles) against an iid-uniform null; these are external to the data values themselves and do not reduce any reported result to a fitted quantity by construction. No equations, self-citations, or ansatzes are invoked to derive the central claims. The MLP result is an empirical baseline, not a prediction forced by the input annotations.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that professional designer rankings constitute a usable ground truth for preference modeling and that the statistical tests correctly detect non-random signal.

axioms (1)
  • domain assumption Designer annotations on the tested images and models constitute a reliable and generalizable preference signal
    The paper uses this to claim the dataset improves alignment and that the MLP result is meaningful.

pith-pipeline@v0.9.1-grok · 5763 in / 1165 out tokens · 36846 ms · 2026-06-30T17:45:20.898753+00:00 · methodology

0 comments
read the original abstract

Text-to-image models now generate graphic design at production scale, yet their supervision still comes primarily from photo-style preference datasets with a single overall verdict per comparison. Designers evaluate designs along several distinct axes (e.g., typography, layout, color harmony) that a single preference label collapses. We release \emph{TASTE} \textit{(Typography, Aesthetics, Spatial, Tone, Etc.)}, a multi-dimensional preference dataset in which two disjoint cohorts of five professional designers each ranked outputs from four current text-to-image models across nine criteria along with per-image hallucination flags. We pair the dataset with two contributions. First, a criterion-agnostic signal-validation framework based on Kendall's $\tau$, majority-vote probability, and Condorcet cycles against exact iid-uniform nulls; the analysis reveals significant but moderate designer agreement, with every TASTE criterion rejecting the random-rater null. Second, we benchmark preference models on TASTE and find that off-the-shelf VLM judges and dedicated T2I scorers fail to reach majority agreement with the designer panel, while a small MLP head trained directly on TASTE substantially narrows the gap to the single-rater ceiling, setting a baseline for future TASTE-trained preference models.

Figures

Figures reproduced from arXiv: 2605.20731 by Alexandria Minetti, Allison Nulty, Elad Hirsch, Haonan Zhu, Purvanshi Mehta.

Figure 1
Figure 1. Figure 1: Per-prompt T distribution (the mean pairwise Kendall τ across the 10 evaluator pairs) for each TASTE sub-dimension, with horizontal lines showing the median T of the three cross-domain reference anchors. Aesthetics sub-dimensions are shown in blue and Descriptions in green. All nine sub-dimensions sit close to the Sushi (food) and MovieLens (movies) medians, well below the median of HPSv2-test restricted t… view at source ↗
Figure 1
Figure 1. Figure 1: Per-prompt T distribution (the mean pairwise Kendall τ across the 10 evaluator pairs) for each TASTE sub-dimension, with horizontal lines showing the median T of the three cross-domain reference anchors. Aesthetics sub-dimensions are shown in blue and Descriptions in green. All nine sub-dimensions sit close to the Sushi (food) and MovieLens (movies) medians, well below the median of HPSv2-test restricted t… view at source ↗
Figure 2
Figure 2. Figure 2: Per-sub-dimension Condorcet cycle rate. All nine sub-dimensions sit at or below [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Per-sub-dimension mean pairwise τ, sorted descending, color-coded by cohort. Description-fidelity sub-dimensions (green) dominate the top of the ordering; aesthetic sub￾dimensions (blue) dominate the bottom. Reference anchor mean τs are shown as dashed lines. Sushi anchor; the five aesthetic criteria cluster below it, with Color Harmony at the bot￾tom. This ordering is consistent with the observation that … view at source ↗
Figure 4
Figure 4. Figure 4: Per-evaluator agreeableness on the Aesthetics (left) and Descriptions (right) co [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 12 canonical work pages · 4 internal anchors

  1. [1]

    E-comIQ-ZH: A human-aligned dataset and benchmark for fine- grained evaluation of E-commerce posters with chain-of-thought

    Anonymous et al. E-comIQ-ZH: A human-aligned dataset and benchmark for fine- grained evaluation of E-commerce posters with chain-of-thought. arXiv preprint arXiv:2602.21698, 2026

    work page arXiv 2026

  2. [2]

    FLUX: Open-source text-to-image generation models

    Black Forest Labs. FLUX: Open-source text-to-image generation models. Technical report,https://blackforestlabs.ai, 2024. 20ZHU ET AL.: TASTE: DESIGNER-ANNOTA TED PREFERENCE FOR AI DESIGN

    2024

  3. [3]

    Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons.Biometrika, 39(3/4):324–345, 1952

    1952

  4. [4]

    Rank analysis of incomplete block designs: I

    Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons.Biometrika, 39(3/4):324–345, 1952

    1952

  5. [5]

    Learning vi- sual importance for graphic designs and data visualizations

    Zoya Bylinskii, Nam Wook Kim, Peter O’Donovan, Sami Alsheikh, Spandan Madan, Hanspeter Pfister, Fredo Durand, Bryan Russell, and Aaron Hertzmann. Learning vi- sual importance for graphic designs and data visualizations. InProceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST), 2017

    2017

  6. [6]

    Seedream: Native high-resolution bilingual image generation foundation model

    ByteDance. Seedream: Native high-resolution bilingual image generation foundation model. Technical report, ByteDance, 2024

    2024

  7. [7]

    Exploring the naturalness of AI- generated images.IEEE Transactions on Circuits and Systems for Video Technology, 2024

    Zijian Chen, Wei Sun, Yuan Tian, Jun Jia, Zicheng Zhang, Jiarui Wang, Ru Huang, Xiongkuo Min, Guangtao Zhai, and Wenjun Zhang. Exploring the naturalness of AI- generated images.IEEE Transactions on Circuits and Systems for Video Technology, 2024

    2024

  8. [8]

    Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks

    Adrienne Deganutti, Elad Hirsch, Haonan Zhu, Jaejung Seol, and Purvanshi Mehta. Graphic-Design-Bench: A comprehensive benchmark for evaluating AI on graphic de- sign tasks. arXiv preprint arXiv:2604.04192, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  9. [9]

    Predicting visual importance across graphic design types

    Camilo Fosco, Vincent Casser, Amish Kumar Bedi, Peter O’Donovan, Aaron Hertz- mann, and Zoya Bylinskii. Predicting visual importance across graphic design types. arXiv preprint arXiv:2008.02912, 2020

    work page arXiv 2008

  10. [10]

    I-HallA: Evaluating image hallucination in text-to-image generation with question answering

    Sang gil Lim, Heesoo Jung, Choonghan Kim, Hyunwoo Park, Hwanhee Lee, and Pil- sung Kang. I-HallA: Evaluating image hallucination in text-to-image generation with question answering. InProceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2025

    2025

  11. [11]

    Imagen and the Nano-Banana image generator

    Google DeepMind. Imagen and the Nano-Banana image generator. Technical report, Google DeepMind, 2024

    2024

  12. [12]

    DistortBench: Benchmarking Vision Language Models on Image Distortion Identification

    Divyanshu Goyal, Akhil Eppa, and Vanya Bannihatti Kumar. DistortBench: Bench- marking vision language models on image distortion identification.arXiv preprint arXiv:2604.19966, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  13. [13]

    Maxwell Harper and Joseph A

    F. Maxwell Harper and Joseph A. Konstan. The MovieLens datasets: History and context.ACM Transactions on Interactive Intelligent Systems (TiiS), 5(4):19:1–19:19, 2015

    2015

  14. [14]

    LICA: Lay- ered image composition annotations for graphic design research

    Elad Hirsch, Shubham Yadav, Mohit Garg, and Purvanshi Mehta. LICA: Lay- ered image composition annotations for graphic design research. arXiv preprint arXiv:2603.16098, 2026

    work page arXiv 2026

  15. [15]

    VBench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2024

  16. [16]

    Nantonac collaborative filtering: Recommendation based on order responses

    Toshihiro Kamishima. Nantonac collaborative filtering: Recommendation based on order responses. Technical report, ACM SIGKDD, 2003. Sushi preference dataset

    2003

  17. [17]

    Maurice G. Kendall. A new measure of rank correlation.Biometrika, 30(1/2):81–93, 1938

    1938

  18. [18]

    Pick-a-Pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-Pic: An open dataset of user preferences for text-to-image generation. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  19. [19]

    Reliability in content analysis: Some common misconceptions and recommendations.Human Communication Research, 30(3):411–433, 2004

    Klaus Krippendorff. Reliability in content analysis: Some common misconceptions and recommendations.Human Communication Research, 30(3):411–433, 2004

    2004

  20. [20]

    SAGE, 4th edition, 2018

    Klaus Krippendorff.Content Analysis: An Introduction to Its Methodology. SAGE, 4th edition, 2018

    2018

  21. [21]

    GenAI-Bench: Evaluating and improving compositional text-to-visual generation

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, and Deva Ramanan. GenAI-Bench: Evaluating and improving compositional text-to-visual generation. InAdvances in Neu- ral Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2024

    2024

  22. [22]

    AGIQA-3K: An open database for AI-generated image quality assessment.IEEE Transactions on Circuits and Systems for Video Technology, 2023

    Chunyi Li, Zicheng Zhang, Haoning Wu, Wei Sun, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, and Weisi Lin. AGIQA-3K: An open database for AI-generated image quality assessment.IEEE Transactions on Circuits and Systems for Video Technology, 2023

    2023

  23. [23]

    AIGIQA-20K: A large database for AI-generated image quality assessment

    Chunyi Li, Tengchuan Kou, Yixuan Gao, Yuqin Cao, Wei Sun, Zicheng Zhang, Yingjie Zhou, Zhichao Zhang, Weixia Zhang, Haoning Wu, Xiaohong Liu, Xiongkuo Min, and Guangtao Zhai. AIGIQA-20K: A large database for AI-generated image quality assessment. InCVPR Workshops (NTIRE), 2024

    2024

  24. [24]

    Qwen3- vl-embedding and qwen3-vl-reranker: A unified framework for state-of-the-art multi- modal retrieval and ranking.arXiv, 2026

    Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Qwen3- vl-embedding and qwen3-vl-reranker: A unified framework for state-of-the-art multi- modal retrieval and ranking.arXiv, 2026

    2026

  25. [25]

    Rich human feedback for text-to-image generation

    Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Car- olan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, et al. Rich human feedback for text-to-image generation. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2024

    2024

  26. [26]

    A survey of multimodal hallucination evaluation and detection

    Hao Liu et al. A survey of multimodal hallucination evaluation and detection. arXiv preprint arXiv:2507.19024, 2025

    work page arXiv 2025

  27. [27]

    Duncan Luce.Individual Choice Behavior: A Theoretical Analysis

    R. Duncan Luce.Individual Choice Behavior: A Theoretical Analysis. John Wiley & Sons, 1959

    1959

  28. [28]

    HPSv3: Towards wide- spectrum human preference score

    Yuhang Ma, Xiaoshi Wu, Keqiang Sun, Hongsheng Li, et al. HPSv3: Towards wide- spectrum human preference score. InProceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), 2025

    2025

  29. [29]

    C. L. Mallows. Non-null ranking models.Biometrika, 44(1/2):114–130, 1957. 22ZHU ET AL.: TASTE: DESIGNER-ANNOTA TED PREFERENCE FOR AI DESIGN

    1957

  30. [30]

    GPT-Image and DALL·E 3: Text-to-image generation

    OpenAI. GPT-Image and DALL·E 3: Text-to-image generation. Technical report, OpenAI, 2024

    2024

  31. [31]

    Bigham, and Jason Wu

    Yi-Hao Peng, Jeffrey P. Bigham, and Jason Wu. DesignPref: Capturing personal pref- erences in visual design generation. arXiv preprint arXiv:2511.20513, 2025

    work page arXiv 2025

  32. [32]

    Plackett

    Robin L. Plackett. The analysis of permutations.Journal of the Royal Statistical Soci- ety: Series C, 24(2):193–202, 1975

    1975

  33. [33]

    Manning, Stefano Er- mon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Er- mon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  34. [34]

    ImagenWorld: Stress-testing image generation models with explainable human evaluation on open-ended real-world tasks

    Tao Sun et al. ImagenWorld: Stress-testing image generation models with explainable human evaluation on open-ended real-world tasks. arXiv preprint arXiv:2603.27862, 2026

    work page arXiv 2026

  35. [35]

    Identifying and mitigating position bias of multi-image vision-language models

    Yu Tian, Tianqi Liu, Zhiyuan Liu, Jie Yang, and Cordelia Schmid. Identifying and mitigating position bias of multi-image vision-language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. arXiv:2503.13792

    work page arXiv 2025

  36. [36]

    Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hong- sheng Li. Human Preference Score v2: A solid benchmark for evaluating human pref- erences of text-to-image synthesis. arXiv preprint arXiv:2306.09341, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  37. [37]

    ImageReward: Learning and evaluating human preferences for text-to- image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. ImageReward: Learning and evaluating human preferences for text-to- image generation. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  38. [38]

    VisionReward: Fine-grained multi- dimensional human preference learning for image and video generation

    Jiazheng Xu, Yu Huang, Jiale Cheng, Yuanming Yang, Jiajun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qinkai Li, Mingyi Zhang, et al. VisionReward: Fine-grained multi- dimensional human preference learning for image and video generation. InProceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2026

    2026

  39. [39]

    Learning multi-dimensional human preference for text-to-image generation

    Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. Learning multi-dimensional human preference for text-to-image generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), 2024

    2024

  40. [40]

    VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness

    Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Wang, Jingwen He, Fan Zhang, Yuanhan Zhang, Jingkang Yang, Yu Qiao, and Ziwei Liu. VBench-2.0: Advancing video gen- eration benchmark suite for intrinsic faithfulness. arXiv preprint arXiv:2503.21755, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  41. [41]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonza- lez, and Ion Stoica. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Bench- marks Track, 2023. ZHU ET AL.: TASTE...

    2023

  42. [42]

    From fragment to one piece: A survey on AI-driven graphic design

    Xingxing Zou, Wen Zhang, and Nanxuan Zhao. From fragment to one piece: A survey on AI-driven graphic design. arXiv preprint arXiv:2503.18641, 2025. A Statistical definitions This appendix collects the formal definitions, support sets, and null PMFs of the three signal- test statistics introduced in §4. Notation: a sample is one prompt rated byR=5 evaluato...

    work page arXiv 2025