EGVLR: Evidence-Grounded Vision–Language Reinforcement for Anomaly Reasoning

Abstract

Multimodal large language models have recently shown promise for industrial anomaly understanding, where a model is expected not only to decide whether an object is defective, but also to localize the supporting visual evidence and explain its decision. However, existing models often rely on language priors or shortcut reasoning when facing fine-grained industrial defects. This leads to inconsistent behaviors, such as correct answers with invalid locations, hallucinated defect boxes on normal samples, or rationales that contradict the final prediction.

We propose EGVLR, an evidence-grounded vision–language reinforcement framework for industrial anomaly reasoning. EGVLR enforces a unified diagnostic response format that explicitly separates visual evidence, diagnostic logic, spatial location, and the final answer. The framework progressively teaches the model to attend to localized defect evidence, incorporate industrial-domain reasoning, and optimize answer–location–rationale consistency through geometry- and semantics-aware preference optimization. For visualization, predicted boxes can be further converted into dense masks using an off-the-shelf segmentation backend. Experiments on MMAD show that EGVLR improves evidence-grounded anomaly reasoning, with consistent gains in defect localization, reasoning-dependent question answering, and spatial false-positive control. These results suggest that explicitly coupling answers, locations, and rationales is important for reliable industrial anomaly understanding.

Method Overview

Evidence-Driven Diagnostic Protocol

EGVLR uses a unified EDDP schema across trainable stages, explicitly separating evidence, reasoning logic, location, and answer.

<think>
  <evidence>...</evidence>
  <logic>...</logic>
</think>
<location>[[x1,y1,x2,y2], ...]</location>
<answer>A/B/C/D</answer>

Each box is represented as [x1, y1, x2, y2] in the [0, 1000] coordinate system. For normal, negative-region, or non-spatial QA samples, the correct location is the empty list.

Main MMAD Results

Seven-task accuracy following the MMAD evaluation protocol; results from methods with different inference mechanisms should be interpreted with protocol differences in mind.

Stage Ablation

Reward Ablation (GS-DPO)

All variants use the Qwen3-VL-8B backbone and the same KG-IT initialization.

Qualitative Example

Qualitative comparison on a zipper anomaly example. The visual pair contains a normal reference and a query image whose teeth are pushed closer together. Baselines produce plausible but weakly grounded defect descriptions, whereas EGVLR follows the unified EDDP schema, localizes the multi-instance defect with EDDP normalized 0–1000 boxes, and predicts the correct answer.

Code Release

The official training and evaluation code is released at github.com/leolin65/EGVLR-project under the MIT License. It covers the full Stage I–III training pipeline, data-conversion scripts, and reproducible MMAD evaluation.

data_pipeline/   Stage I/II/III data conversion (synthetic + real anomaly QA,
                 domain-knowledge QA, comparative QA, dataset merging)
src/             Reward decomposition, GRPO trainer, SFT/GRPO training entrypoints
scripts/         SLURM job templates for Stage I/II/III training, LoRA merging, eval
eval/            lmms-eval task definitions for MMAD (mmad, mmad_1shot, mmad_info,
                 mmad_train) + macro-7 scoring matching the original MMAD metric
paper_fidelity/  Documented paper-vs-code gaps with tested (unapplied) fixes

The code builds on Qwen-VL-Series-Finetune and evaluates with lmms-eval. MMAD's underlying image datasets (MVTec-AD, MVTec-LOCO, VisA, GoodsAD, DS-MVTec, RealIAD, etc.) are not redistributed; obtain them from the original providers and the MMAD benchmark release.

Acknowledgements

This work was supported in part by the National Science and Technology Council, Taiwan, under grant NSTC 115-2634-F-007-003.

BibTeX

@inproceedings{lin2026egvlr,
  title     = {EGVLR: Evidence-Grounded Vision--Language Reinforcement for Anomaly Reasoning},
  author    = {Lin, Shih-Chih and Lu, Ying-Heng and Ye, Dong You and Lai, Shang-Hong},
  booktitle = {Computer Vision -- ECCV 2026},
  series    = {Lecture Notes in Computer Science},
  volume    = {17061},
  publisher = {Springer},
  year      = {2026},
  doi       = {10.1007/978-3-032-37094-5_11}
}

Published in Computer Vision – ECCV 2026, LNCS vol. 17061, Springer.