pith. sign in

arxiv: 2603.22607 · v3 · pith:H4OOMCMGnew · submitted 2026-03-23 · 💻 cs.CV

Dress-ED: Instruction-Guided Editing for Virtual Try-On and Try-Off

Fulvio Sanguigni , Davide Lobba , Bin Ren , Marcella Cornia , Nicu Sebe , Rita Cucchiara This is my paper

Pith reviewed 2026-05-15 00:09 UTC · model grok-4.3

classification 💻 cs.CV
keywords virtual try-onvirtual try-offgarment editingtext-guided editingdiffusion modelsfashion datasetmultimodal pipelineinstruction following
0
0 comments X

The pith

Dress-ED provides the first large-scale benchmark unifying virtual try-on, virtual try-off and text-guided garment editing.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Dress-ED as a new benchmark that brings virtual try-on, virtual try-off and instruction-based garment editing into one dataset. Each entry supplies an in-shop garment image, the person wearing that garment, edited versions of both, and a natural-language instruction describing the desired change. This structure supports controllable modifications such as altering color, pattern or sleeve length within a single framework. The data is produced by an automated pipeline that uses multimodal models to understand garments, diffusion models to perform edits and language models to verify quality, yielding more than 146,000 samples across three garment categories and seven edit types. A baseline multimodal diffusion model is also presented that processes both the text instruction and visual cues to execute the edits.

Core claim

We introduce the Dress Editing Dataset (Dress-ED), the first large-scale benchmark that unifies VTON, VTOFF, and text-guided garment editing within a single framework. Each sample in Dress-ED includes an in-shop garment image, the corresponding person image wearing the garment, their edited counterparts, and a natural-language instruction of the desired modification. Built through a fully automated multimodal pipeline that integrates MLLM-based garment understanding, diffusion-based editing, and LLM-guided verification, Dress-ED comprises over 146k verified quadruplets spanning three garment categories and seven edit types, including both appearance (e.g., color, pattern, material) and the e

What carries the argument

The quadruplet consisting of in-shop garment image, person image, edited counterparts and natural-language instruction, generated by an automated pipeline of MLLM understanding, diffusion editing and LLM verification.

If this is right

  • Models trained on the dataset can perform text-guided virtual try-on and virtual try-off in one system.
  • The benchmark supports both appearance edits such as color and pattern changes and structural edits such as sleeve length and neckline adjustments.
  • A single evaluation set now exists for instruction-driven fashion synthesis tasks that were previously handled separately.
  • The proposed multimodal diffusion baseline shows how linguistic instructions and visual garment cues can be jointly processed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Interactive e-commerce tools could let users describe garment changes in plain language and see results immediately.
  • Training on this dataset may improve a model's ability to handle unseen edit instructions if the automated creation process scales cleanly.
  • Adding real-world user instructions as a test set would check whether models generalize beyond the generated data.

Load-bearing premise

The fully automated multimodal pipeline integrating MLLM-based garment understanding, diffusion-based editing, and LLM-guided verification produces accurate, high-quality data without significant artifacts or verification errors.

What would settle it

A random sample of the released quadruplets in which the edited images fail to match the supplied instructions or contain visible artifacts would falsify the benchmark's claimed reliability.

Figures

Figures reproduced from arXiv: 2603.22607 by Bin Ren, Davide Lobba, Fulvio Sanguigni, Marcella Cornia, Nicu Sebe, Rita Cucchiara.

Figure 1
Figure 1. Figure 1: We propose the Dress Editing Dataset (Dress-ED), the first benchmark for instruction-driven virtual try-on and try-off with over 146k verified samples across seven editing types, including both appearance and structural modifications. Abstract. Recent advances in Virtual Try-On (VTON) and Virtual Try-Off (VTOFF) have greatly improved photo-realistic fashion synthesis and garment reconstruction. However, ex… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the Dress-ED curation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed Dress-EM architecture for instruction-driven fashion editing. (DiT) into a cohesive architecture, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative results for edited VTON unpaired setting (first two rows) and edited VTOFF (last two rows), showing realistic and instruction-consistent edits. pose, and garment details, further confirming the effectiveness of the proposed approach. 6 Conclusion We introduced Dress-ED, the first large-scale dataset and benchmark for instruction-driven fashion editing that unifies VTON, VTOFF, and text-guided g… view at source ↗
Figure 5
Figure 5. Figure 5: Sample of label maps, masks, in-shop garment and FitDiT result. FitDiT uses bounding boxes as in￾put masks. These bounding boxes are extracted from the label maps of a clothed person extracted using a parser. A visual example is pro￾vided in [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Samples from Dress-ED across the different edit types, including the original person-garment pair, the corresponding edited outputs, and the textual instruction. Addable Details (category-specific) – Upper-body: bow (neck), pockets, chest pocket, zipper, buttons. – Lower-body: belt, drawstring. – Dresses: belt, fitted belt, pockets, drawstring, bow (neck/waist). Removable/Colorable/Replaceable Details: sam… view at source ↗
Figure 7
Figure 7. Figure 7: Editing instruction taxonomy showing the distribution of edit types in Dress-ED, covering both appearance-level changes (e.g., color, pattern, material) and structural modifications (e.g., sleeve length, neckline, shape). ditional examples from Dress-ED are provided in [PITH_FULL_IMAGE:figures/full_fig_p022_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative results for edited VTON (left, unpaired setting) and VTOFF (right), showing realistic and instruction-consistent edits. D Additional Discussion and Clarifications This section provides further explanations on several components of our pipeline. We outline the motivations behind key design choices, describe their practical implications, and discuss limitations relevant to interpreting the datase… view at source ↗
Figure 9
Figure 9. Figure 9: User study interface. suring whether models follow the intended transformation, while remaining compatible with future real-world data collection or domain adaptation. In this sense, the dataset serves as a structured foundation on top of which broader and more diverse fashion-editing scenarios can be explored. D.2 LLM-as-Judge and Evaluation Consistency GPT-5 [40] is employed solely as a non-generative ev… view at source ↗
Figure 10
Figure 10. Figure 10: Confusion matrix of our user study. To enable reproducible large-scale verification, we distill GPT-5’s judg￾ments into a smaller open-source MLLM (InternVL-3.5 [53]). Although this introduces some dependence be￾tween data construction and evalu￾ation, our final benchmark reports model-agnostic evaluation metrics (i.e., including DINO and standard perceptual scores) which remain inde￾pendent of both GPT-5… view at source ↗
Figure 9
Figure 9. Figure 9: Participants were tasked with assessing the edit correctness of triplets [PITH_FULL_IMAGE:figures/full_fig_p025_9.png] view at source ↗
read the original abstract

Recent advances in Virtual Try-On (VTON) and Virtual Try-Off (VTOFF) have greatly improved photo-realistic fashion synthesis and garment reconstruction. However, existing datasets remain static, lacking instruction-driven editing for controllable and interactive fashion generation. In this work, we introduce the Dress Editing Dataset (Dress-ED), the first large-scale benchmark that unifies VTON, VTOFF, and text-guided garment editing within a single framework. Each sample in Dress-ED includes an in-shop garment image, the corresponding person image wearing the garment, their edited counterparts, and a natural-language instruction of the desired modification. Built through a fully automated multimodal pipeline that integrates MLLM-based garment understanding, diffusion-based editing, and LLM-guided verification, Dress-ED comprises over 146k verified quadruplets spanning three garment categories and seven edit types, including both appearance (e.g., color, pattern, material) and structural (e.g., sleeve length, neckline) modifications. Based on this benchmark, we further propose a unified multimodal diffusion framework that jointly reasons over linguistic instructions and visual garment cues, serving as a strong baseline for instruction-driven VTON and VTOFF. Dataset and code available at this link: https://github.com/aimagelab/Dress-ED

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper introduces the Dress Editing Dataset (Dress-ED), the first large-scale benchmark unifying VTON, VTOFF, and text-guided garment editing. Each of the >146k samples consists of an in-shop garment image, corresponding person image, edited counterparts, and a natural-language instruction. The dataset is built via a fully automated multimodal pipeline (MLLM garment understanding + diffusion editing + LLM-guided verification) spanning three garment categories and seven edit types (appearance and structural). A unified multimodal diffusion framework is proposed as a baseline for instruction-driven tasks.

Significance. If the automated pipeline produces high-quality, accurately labeled quadruplets at this scale, Dress-ED would provide the first unified benchmark for controllable, instruction-driven virtual try-on and try-off, enabling progress on interactive multimodal fashion synthesis models.

major comments (1)
  1. [Abstract and Dataset Construction] Abstract and Dataset Construction: the claim that the 146k quadruplets are 'verified' and high-quality rests entirely on the LLM-guided verification step correctly rejecting low-fidelity edits and instruction mismatches. No quantitative human evaluation (agreement rates, error rates on a sampled subset, or false-positive acceptance of artifacts) is reported to bound the reliability of this automated verification.
minor comments (1)
  1. [Abstract] The abstract states that dataset and code will be made publicly available but provides no details on release timeline, repository, or licensing.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback. We agree that quantitative human validation would strengthen the claims about dataset quality and will incorporate the requested evaluation in the revision.

read point-by-point responses

  1. Referee: [Abstract and Dataset Construction] Abstract and Dataset Construction: the claim that the 146k quadruplets are 'verified' and high-quality rests entirely on the LLM-guided verification step correctly rejecting low-fidelity edits and instruction mismatches. No quantitative human evaluation (agreement rates, error rates on a sampled subset, or false-positive acceptance of artifacts) is reported to bound the reliability of this automated verification.

    Authors: We agree that the absence of quantitative human evaluation leaves the reliability of the LLM-guided verification step insufficiently bounded. In the revised manuscript we will add a dedicated subsection under Dataset Construction that reports a human study on a randomly sampled subset of 1,000 quadruplets. The study will include: (i) agreement rate between the LLM verifier and three independent human annotators, (ii) false-positive rate (human-accepted artifacts that the LLM incorrectly passed), and (iii) error rates stratified by edit type. We will also release the annotation protocol and sampled subset to allow reproducibility. This addition directly addresses the concern and provides empirical bounds on the automated pipeline's quality. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper's contribution is empirical: it describes construction of the Dress-ED dataset via an automated pipeline (MLLM garment understanding + diffusion editing + LLM verification) and trains a baseline multimodal diffusion model. No mathematical derivations, equations, fitted parameters, or predictions appear in the provided text. The central claims rest on the pipeline's output of 146k quadruplets rather than any self-referential reduction, self-citation load-bearing premise, or renaming of known results. Any self-citations present are incidental and not required to justify the dataset's existence or the baseline's architecture by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper's primary addition is the curated dataset and baseline rather than new theoretical constructs or parameters.

axioms (1)
  • domain assumption Pre-trained multimodal LLMs and diffusion models can reliably understand garments and generate accurate edits for verification
    The automated pipeline depends on these existing models for data generation and quality checks.

pith-pipeline@v0.9.0 · 5547 in / 1159 out tokens · 56988 ms · 2026-05-15T00:09:51.975862+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.