Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

VulKey uses a three-level hierarchy of repair patterns to guide LLMs in fixing vulnerabilities more accurately than prior methods.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-07-01 00:39 UTC pith:TO7IFMAX

load-bearing objection VulKey's hierarchical abstraction and two-stage pipeline give measurable AVR gains on PrimeVul and Vul4J, but the first-stage matcher lacks separate metrics so the source of the improvement stays unclear. the 2 major comments →

arxiv 2605.01769 v3 pith:TO7IFMAX submitted 2026-05-03 cs.CR cs.SE

VulKey: Automated Vulnerability Repair Guided by Domain-Specific Repair Patterns

Jia Li , Zhuangbin Chen , Yuxin Su , Michael R. Lyu This is my paper
classification cs.CR cs.SE
keywords automated vulnerability repairrepair patternsLLM-based AVRCWEpatch generationsoftware securityPrimeVulVul4J
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that large language models for automatic vulnerability repair benefit from explicit structured knowledge drawn from security sources like CWE, rather than superficial inclusion or rigid few-shot examples. It defines repair strategies through a three-level abstraction covering the flaw category, the syntactic change, and semantic key elements. This abstraction feeds a two-stage process: matching selects the right pattern for a given vulnerability, then a fine-tuned LLM generates the patch under that pattern's guidance. The result is higher repair rates on real C/C++ and Java benchmarks because the patterns supply generality and semantic detail that direct prompting lacks.

Core claim

VulKey is an LLM-based AVR framework that leverages a hierarchical abstraction of expert knowledge to guide patch generation. Our novel three-level abstraction formulates repair strategies in terms of CWE type, syntactic actions, and semantic key elements. This approach captures the essence of a security fix with greater generality than concrete examples and more semantic richness than traditional syntax-based templates. VulKey is implemented as a two-stage pipeline: first, expert knowledge matching predicts an appropriate repair pattern for the vulnerability; second, repair code generation uses a pattern-guided, fine-tuned LLM to produce secure patches. On the real-world C/C++ dataset Prime

What carries the argument

The three-level hierarchical abstraction of repair patterns (CWE type, syntactic actions, semantic key elements) that drives the two-stage expert knowledge matching followed by pattern-guided LLM patch generation.

Load-bearing premise

The first-stage matching step can correctly identify a suitable repair pattern for vulnerabilities not encountered in training.

What would settle it

A new vulnerability dataset where the matching stage selects mismatched patterns for most cases, causing overall repair accuracy to fall to or below that of a plain LLM prompt without patterns.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The two-stage separation lets pattern selection draw on expert knowledge while the generation stage focuses on producing correct code.
  • The patterns support generalization across programming languages, as shown by strong results on both C/C++ and Java benchmarks.
  • Explicit hierarchical modeling overcomes the coverage limits of few-shot examples and the shallowness of simply adding a CWE identifier to prompts.
  • Fine-tuning the LLM on pattern-guided data produces patches that align more closely with security best practices than unguided generation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same hierarchical pattern approach could be tested on non-security code changes such as general bug repair or refactoring tasks.
  • Automatically mining additional patterns from larger vulnerability repositories might reduce reliance on manual expert curation.
  • Combining the patterns with static analysis tools could provide an external check on whether generated patches actually eliminate the targeted flaw.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces VulKey, a two-stage LLM-based automatic vulnerability repair (AVR) framework. It first performs expert knowledge matching using a three-level hierarchical abstraction (CWE type, syntactic actions, semantic key elements) derived from security sources, then uses the selected pattern to guide a fine-tuned LLM for patch generation. On the PrimeVul C/C++ dataset it reports 31.5% repair accuracy (7.6% above the best baseline and outperforming VulMaster and GPT-5); it also claims state-of-the-art results on the Java Vul4J benchmark and cross-language/model generalization.

Significance. If the empirical gains are shown to stem from the hierarchical pattern mechanism rather than LLM fine-tuning alone, the work would provide concrete evidence that structured expert security knowledge can be operationalized more effectively than current prompting or few-shot baselines in AVR, addressing a recognized limitation in the field.

major comments (2)
  1. [Abstract, §3] Abstract and §3 (pipeline description): the 31.5% end-to-end accuracy on PrimeVul is presented as evidence for the value of the hierarchical abstraction, yet no separate precision/recall metric, ablation (matcher disabled), or held-out evaluation of the first-stage expert knowledge matching is reported. Without this, improvements cannot be confidently attributed to the CWE/syntactic/semantic patterns rather than the fine-tuned LLM component.
  2. [§5] §5 (experimental results): the cross-language claim on Vul4J and comparisons to VulMaster/GPT-5 rest on the same two-stage pipeline, but the manuscript provides no breakdown of how often the matcher selects a pattern that was rare or absent in the fine-tuning data, leaving open the possibility that gains are driven by patterns already well-represented in training.
minor comments (2)
  1. [§2] Notation for the three abstraction levels is introduced in the abstract but would benefit from an explicit table or figure early in §2 that maps each level to concrete examples from CWE/NVD.
  2. [§4] The manuscript should clarify the exact train/validation/test splits used for PrimeVul and whether any overlap exists between the expert patterns and the test vulnerabilities.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight important aspects of attributing performance gains to the hierarchical abstraction mechanism. We address each major comment below and commit to revisions that will strengthen the empirical support for our claims.

read point-by-point responses

  1. Referee: [Abstract, §3] Abstract and §3 (pipeline description): the 31.5% end-to-end accuracy on PrimeVul is presented as evidence for the value of the hierarchical abstraction, yet no separate precision/recall metric, ablation (matcher disabled), or held-out evaluation of the first-stage expert knowledge matching is reported. Without this, improvements cannot be confidently attributed to the CWE/syntactic/semantic patterns rather than the fine-tuned LLM component.

    Authors: We agree that the current manuscript lacks a dedicated evaluation isolating the contribution of the first-stage matcher. In the revised version, we will add precision and recall metrics for the expert knowledge matching stage evaluated on a held-out portion of the training data. We will also include an ablation experiment that disables the matcher (i.e., runs the fine-tuned LLM without pattern guidance) and directly compares it to the full VulKey pipeline on PrimeVul. These additions will be placed in §3 and §5 to allow readers to attribute gains more confidently to the hierarchical patterns. revision: yes

  2. Referee: [§5] §5 (experimental results): the cross-language claim on Vul4J and comparisons to VulMaster/GPT-5 rest on the same two-stage pipeline, but the manuscript provides no breakdown of how often the matcher selects a pattern that was rare or absent in the fine-tuning data, leaving open the possibility that gains are driven by patterns already well-represented in training.

    Authors: We acknowledge that the manuscript does not currently provide an analysis of pattern frequency in the fine-tuning data versus patterns selected at inference time. In the revision, we will add to §5 a quantitative breakdown showing, for both PrimeVul and Vul4J test instances, the distribution of selected patterns according to their occurrence count in the fine-tuning corpus. This will explicitly report how often the matcher chooses rare or unseen patterns, thereby addressing whether the reported gains could be explained solely by patterns well-represented during fine-tuning. revision: yes

Circularity Check

0 steps flagged

No circularity; empirical results on external benchmarks

full rationale

The paper reports empirical repair accuracies (31.5% on PrimeVul, SOTA on Vul4J) from a two-stage pipeline evaluated against named external baselines and datasets. No equations, self-citations, or uniqueness theorems are present that reduce any claimed result to its own inputs by construction. The first-stage matcher is described as predicting patterns, but the overall claims rest on end-to-end experimental outcomes rather than any fitted parameter renamed as prediction or self-referential derivation. This is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an applied empirical paper in software security; the abstract contains no mathematical derivations, fitted parameters, or postulated entities beyond the described method itself.

pith-pipeline@v0.9.1-grok · 5840 in / 1169 out tokens · 52878 ms · 2026-07-01T00:39:30.407649+00:00 · methodology

0 comments
read the original abstract

The increasing prevalence of software vulnerabilities highlights the need for effective Automatic Vulnerability Repair (AVR) tools. While LLM-based approaches are promising, they struggle to incorporate structured security knowledge from sources like CWE and NVD. Current methods either use this information superficially by concatenating the CWE-ID into the input prompt, yielding negligible benefits, or rely on few-shot learning with rigid, non-generalizable examples, which limits their effectiveness in real-world scenarios. To address this gap, we propose VulKey, an LLM-based AVR framework that leverages a hierarchical abstraction of expert knowledge to guide patch generation. Our novel three-level abstraction formulates repair strategies in terms of CWE type, syntactic actions, and semantic key elements. This approach captures the essence of a security fix with greater generality than concrete examples and more semantic richness than traditional syntax-based templates, overcoming the coverage limitations of prior methods. VulKey is implemented as a two-stage pipeline: first, expert knowledge matching predicts an appropriate repair pattern for the vulnerability; second, repair code generation uses a pattern-guided, fine-tuned LLM to produce secure patches. On the real-world C/C++ dataset PrimeVul, VulKey achieves 31.5% repair accuracy, surpassing the best baseline by 7.6% and outperforming leading tools such as VulMaster and GPT-5. Moreover, VulKey demonstrates cross-language and cross-model generalizability, with state-of-the-art performance on the Java benchmark Vul4J. These results underscore the importance of structured expert knowledge in advancing AVR effectiveness. Our work demonstrates that explicitly modeling and integrating expert security knowledge through hierarchical patterns is a crucial step toward building more effective and reliable AVR tools.

Figures

Figures reproduced from arXiv: 2605.01769 by Jia Li, Michael R. Lyu, Yuxin Su, Zhuangbin Chen.

Figure 1
Figure 1. Figure 1: The Vulnerability Information Tracking Systems [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Bug/Vul-fix Pair in VulKey often lacks a pre-existing failing test; the test case is typically a complex, environment-dependent exploit. More critically, a successful patch must eliminate the entire class of underlying weakness without introducing new regressions, a much higher standard that simple tests cannot guarantee. This absence of a well-defined, binary validation oracle makes automated patch genera… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of VulKey 4.1 Overview Inputs. We assume three input signals, consistent with prior AVR studies [8, 26, 39, 55]. (1) A vulnerable function 𝑋𝑖 with the buggy region marked by //bug_start and //bug_end. (2) The CWE identifier 𝑇𝑖 and name 𝑁𝑖 , produced by upstream detection/triage tools (e.g., CodeQL [28], SonarQube [47]); in our experiments we use dataset-provided labels. (3) Historical vul-fix pair… view at source ↗
Figure 4
Figure 4. Figure 4: Venn Diagram for Different Settings Results. As shown in [PITH_FULL_IMAGE:figures/full_fig_p018_4.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Signal Reshaping for GRPO in Weak-Feedback Agentic Code Repair

    cs.AI 2026-05 unverdicted novelty 5.0

    Reshaping outcome rewards, process signals, and rollout comparability in GRPO raises strict compile-and-semantic accuracy in agentic code repair from 0.385 to 0.535 under weak feedback.

Reference graph

Works this paper leans on

57 extracted references · 36 canonical work pages · cited by 1 Pith paper · 3 internal anchors

  1. [1]

    Afsah Anwar, Ahmed Abusnaina, Songqing Chen, Frank Li, and David Mohaisen. 2021. Cleaning the NVD: Compre- hensive quality assessment, improvements, and analyses.IEEE Transactions on Dependable and Secure Computing19, 6 (2021), 4255–4269. doi:10.1109/TDSC.2021.3125270

    work page doi:10.1109/tdsc.2021.3125270 2021

  2. [2]

    Guru Bhandari, Amara Naseer, and Leon Moonen. 2021. CVEfixes: automated collection of vulnerabilities and their fixes from open-source software. InProceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering. ACM, New York, NY, USA, 30–39. doi:10.1145/3475960.3475985

    work page doi:10.1145/3475960.3475985 2021

  3. [3]

    BigCode. 2024. Data Portraits. https://stack.dataportraits.org/

    2024

  4. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    2020

  5. [5]

    Díaz Ferreyra

    Quang-Cuong Bui, Riccardo Scandariato, and Nicolás E. Díaz Ferreyra. 2022. Vul4J: a dataset of reproducible Java vulnerabilities geared towards the study of program repair techniques. InProceedings of the 19th International Conference on Mining Software Repositories. Association for Computing Machinery, New York, NY, USA, 464–468. doi:10.1145/ 3524842.3528482

    work page arXiv 2022

  6. [6]

    Xiansheng Cao, Junfeng Wang, and Peng Wu. 2024. Enhancing Vulnerability Repair Through Summarization of Repair Patterns and Optimal Matching.SSRN preprint SSRN:5025384(2024). doi:10.2139/ssrn.5025384

    work page doi:10.2139/ssrn.5025384 2024

  7. [7]

    Yizheng Chen, Zhoujie Ding, Lamya Alowain, Xinyun Chen, and David Wagner. 2023. DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection. InProceedings of the 26th International Sym- posium on Research in Attacks, Intrusions and Defenses. Association for Computing Machinery, New York, NY, USA, 654–668. doi:10.1145/3...

    work page doi:10.1145/3607199.3607242 2023

  8. [8]

    Zimin Chen, Steve Kommrusch, and Martin Monperrus. 2022. Neural Transfer Learning for Repairing Security Vulnerabilities in C Code.IEEE Transactions on Software Engineering48, 9 (2022), 3582–3597. doi:10.1109/TSE.2019. 2940179

    work page doi:10.1109/tse.2019 2022

  9. [9]

    Jianlei Chi, Yu Qu, Ting Liu, Qinghua Zheng, and Heng Yin. 2020. SeqTrans: Automatic Vulnerability Fix via Sequence to Sequence Learning.arXiv preprint arXiv:2010.10805(2020)

    work page arXiv 2020

  10. [10]

    Jacob Cohen. 1960. A coefficient of agreement for nominal scales.Educational and psychological measurement20, 1 (1960), 37–46. doi:10.1177/001316446002000104

    work page doi:10.1177/001316446002000104 1960

  11. [11]

    The MITRE Corporation. 2025. Common Vulnerabilities and Exposures. https://www.cve.org/

    2025

  12. [12]

    The MITRE Corporation. 2025. Common Weakness Enumeration. https://cwe.mitre.org/. , Vol. 1, No. 1, Article . Publication date: May 2026. 22 Jia Li, Zhuangbin Chen, Yuxin Su, and Michael R. Lyu

    2025

  13. [13]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient Finetuning of Quantized LLMs. InAdvances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., Red Hook, NY, USA, 10088–10115. https://proceedings.neurips. cc/paper_fil...

    2023

  14. [14]

    Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, and Yizheng Chen. 2025. Vulnerability Detection with Code Language Models: How Far Are We?. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE Computer Society, Los Alamitos, CA, USA, 469–481. doi:10.1109...

    work page doi:10.1109/icse55347.2025.00038 2025

  15. [15]

    2024.A Global View of the CISA KEV Catalog: Prevalence and Remediation

    Ben Edwards. 2024.A Global View of the CISA KEV Catalog: Prevalence and Remediation. Technical Report. Bitsight. https://www.bitsight.com/resources/slicing-through-cisas-kev-catalog

    2024

  16. [16]

    Hugging Face. 2023. codet5p-220m. https://huggingface.co/Salesforce/codet5p-220m

    2023

  17. [17]

    Hugging Face. 2023. Quantization. https://huggingface.co/docs/transformers/main/en/main_classes/quantization

    2023

  18. [18]

    Hugging Face. 2023. starcoderbase. https://huggingface.co/bigcode/starcoderbase

    2023

  19. [19]

    Hugging Face. 2024. CodeLlama-70b-hf. https://huggingface.co/codellama/CodeLlama-70b-hf

    2024

  20. [20]

    Hugging Face. 2024. DeepSeek-Coder-V2-Lite-Base. https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite- Base

    2024

  21. [21]

    Hugging Face. 2024. Qwen2.5-Coder-32B. https://huggingface.co/Qwen/Qwen2.5-Coder-32B

    2024

  22. [22]

    Jiahao Fan, Yi Li, Shaohua Wang, and Tien N. Nguyen. 2020. A C/C++ Code Vulnerability Dataset with Code Changes and CVE Summaries. InProceedings of the 17th International Conference on Mining Software Repositories. Association for Computing Machinery, New York, NY, USA, 508–512. doi:10.1145/3379597.3387501

    work page doi:10.1145/3379597.3387501 2020

  23. [23]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, New York, NY, USA, 6491–6501. doi:10.1145/3...

    work page doi:10.1145/3637528.3671470 2024

  24. [24]

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. InFindings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, Online, 1536–1547. doi:10.18653/v1/2020....

    work page doi:10.18653/v1/2020.findings-emnlp.139 2020

  25. [25]

    Markus Freitag and Yaser Al-Onaizan. 2017. Beam Search Strategies for Neural Machine Translation. InProceedings of the First Workshop on Neural Machine Translation. Association for Computational Linguistics, Vancouver, Canada, 56–60. doi:10.18653/v1/w17-3207

    work page doi:10.18653/v1/w17-3207 2017

  26. [26]

    Michael Fu, Chakkrit Tantithamthavorn, Trung Le, Van Nguyen, and Phung Dinh. 2022. VulRepair: A T5-based Automated Software Vulnerability Repair. InProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. Association for Computing Machinery, New York, NY, USA, 935–947. doi:10.1145...

    work page doi:10.1145/3540250.3549098 2022

  27. [27]

    Qing Gao, Yingfei Xiong, Yaqing Mi, Lu Zhang, Weikun Yang, Zhaoping Zhou, Bing Xie, and Hong Mei. 2015. Safe memory-leak fixing for C programs. InProceedings of the 37th International Conference on Software Engineering - Volume 1. IEEE Press, Piscataway, NJ, USA, 459–470

    2015

  28. [28]

    GitHub. 2026. CodeQL. https://codeql.github.com/

    2026

  29. [29]

    Seongjoon Hong, Junhee Lee, Jeongsoo Lee, and Hakjoo Oh. 2020. SAVER: scalable, precise, and safe memory-error repair. InProceedings of the ACM/IEEE 42nd International Conference on Software Engineering. Association for Computing Machinery, New York, NY, USA, 271–283. doi:10.1145/3377811.3380323

    work page doi:10.1145/3377811.3380323 2020

  30. [30]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

  31. [31]

    LoRA: Low-Rank Adaptation of Large Language Models.arXiv preprint arXiv:2106.09685(2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  32. [32]

    Kai Huang, Zhengzi Xu, Su Yang, Hongyu Sun, Xuejun Li, Zheng Yan, and Yuqing Zhang. 2024. Evolving Paradigms in Automated Program Repair: Taxonomy, Challenges, and Opportunities.Comput. Surveys57, 2, Article 36 (2024), 43 pages. doi:10.1145/3696450

    work page doi:10.1145/3696450 2024

  33. [33]

    Kai Huang, Jian Zhang, Xiangxin Meng, and Yang Liu. 2025. Template-Guided Program Repair in the Era of Large Language Models. In2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE Computer Society, Los Alamitos, CA, USA, 367–379. doi:10.1109/ICSE55347.2025.00030

    work page doi:10.1109/icse55347.2025.00030 2025

  34. [34]

    J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data.Biometrics33, 1 (1977), 159–174. doi:10.2307/2529310

    work page doi:10.2307/2529310 1977

  35. [35]

    Junhee Lee, Seongjoon Hong, and Hakjoo Oh. 2018. MemFix: static analysis-based repair of memory deallocation errors for C. InProceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. Association for Computing Machinery, New York, NY, USA, 95–106. doi:10.1145/3236024...

    work page doi:10.1145/3236024.3236079 2018

  36. [36]

    Bissyandé

    Kui Liu, Anil Koyuncu, Dongsun Kim, and Tegawendé F. Bissyandé. 2019. TBar: revisiting template-based automated program repair. InProceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM, New York, NY, USA, 31–42. doi:10.1145/3293882.3330577

    work page doi:10.1145/3293882.3330577 2019

  37. [37]

    Xiangxin Meng, Xu Wang, Hongyu Zhang, Hailong Sun, and Xudong Liu. 2022. Improving fault localization and program repair with deep semantic features and transferred knowledge. InProceedings of the 44th International Conference on Software Engineering. Association for Computing Machinery, New York, NY, USA, 1169–1180. doi:10. 1145/3510003.3510147

    work page arXiv 2022

  38. [38]

    Georgios Nikitopoulos, Konstantina Dritsa, Panos Louridas, and Dimitris Mitropoulos. 2021. CrossVul: a cross-language vulnerability dataset with commit data. InProceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. Association for Computing Machinery, New York, NY, USA, 1565–156...

    work page doi:10.1145/3468264.3473122 2021

  39. [39]

    Changan Niu, Chuanyi Li, Vincent Ng, Dongxiao Chen, Jidong Ge, and Bin Luo. 2023. An Empirical Comparison of Pre-Trained Models of Source Code. InProceedings of the 45th International Conference on Software Engineering. IEEE Press, Piscataway, NJ, USA, 2136–2148. doi:10.1109/ICSE48619.2023.00180

    work page doi:10.1109/icse48619.2023.00180 2023

  40. [40]

    Yu Nong, Haoran Yang, Long Cheng, Hongxin Hu, and Haipeng Cai. 2025. APPATCH: automated adaptive prompting large language models for real-world software vulnerability patching. InProceedings of the 34th USENIX Conference on Security Symposium. USENIX Association, USA, Article 231, 20 pages

    2025

  41. [41]

    OpenAI. 2024. ChatGPT. https://openai.com/blog/chatgpt/

    2024

  42. [42]

    John W Ratcliff, David E Metzener, et al. 1988. Pattern matching: The gestalt approach.Dr. Dobbś Journal13, 7 (1988), 46

    1988

  43. [43]

    Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and beyond.Foundations and Trends®in Information Retrieval3, 4 (2009), 333–389. doi:10.1561/1500000019

    work page doi:10.1561/1500000019 2009

  44. [44]

    Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code Llama: Open Foundation Models for Code.arXiv preprint arXiv:2308.12950 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  45. [45]

    US-CERT Security. 2025. National Vulnerability Database. https://nvd.nist.gov/

    2025

  46. [46]

    Alexey Shestov, Rodion Levichev, Ravil Mussabayev, Evgeny Maslov, Anton Cheshkov, and Pavel Zadorozhny. 2024. Finetuning Large Language Models for Vulnerability Detection.arXiv preprint arXiv:2401.17010(2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  47. [47]

    Nima Shiri Harzevili, Alvine Boaye Belle, Junjie Wang, Song Wang, Zhen Ming (Jack) Jiang, and Nachiappan Nagappan

  48. [48]

    A Systematic Literature Review on Automated Software Vulnerability Detection Using Machine Learning. Comput. Surveys57, 3 (2024), 55. doi:10.1145/3699711

    work page doi:10.1145/3699711 2024

  49. [49]

    SonarSource. 2026. SonarQube. https://www.sonarsource.com/products/sonarqube/

    2026

  50. [50]

    Rijnard van Tonder and Claire Le Goues. 2018. Static automated program repair for heap properties. InProceedings of the 40th International Conference on Software Engineering. Association for Computing Machinery, New York, NY, USA, 151–162. doi:10.1145/3180155.3180250

    work page doi:10.1145/3180155.3180250 2018

  51. [51]

    Yue Wang, Weishi Wang, Shafiq Joty, and Steven C.H. Hoi. 2021. CodeT5: Identifier-aware Unified Pre-trained Encoder- Decoder Models for Code Understanding and Generation. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 8696–8708....

    work page doi:10.18653/v1/2021.emnlp-main.685 2021

  52. [52]

    Yi Wu, Nan Jiang, Hung Viet Pham, Thibaud Lutellier, Jordan Davis, Lin Tan, Petr Babkin, and Sameena Shah. 2023. How Effective Are Neural Networks for Fixing Security Vulnerabilities. InProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. Association for Computing Machinery, New York, NY, USA, 1282–1294. doi:10.114...

    work page doi:10.1145/3597926.3598135 2023

  53. [53]

    Chunqiu Steven Xia and Lingming Zhang. 2024. Automated Program Repair via Conversation: Fixing 162 out of 337 Bugs for $0.42 Each using ChatGPT. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. Association for Computing Machinery, New York, NY, USA, 819–831. doi:10.1145/3650212.3680323

    work page doi:10.1145/3650212.3680323 2024

  54. [54]

    Xin Yin, Chao Ni, Shaohua Wang, Zhenhao Li, Limin Zeng, and Xiaohu Yang. 2024. ThinkRepair: Self-Directed Automated Program Repair. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. Association for Computing Machinery, New York, NY, USA, 1274–1286. doi:10.1145/3650212.3680359

    work page doi:10.1145/3650212.3680359 2024

  55. [55]

    Jian Zhang, Chong Wang, Anran Li, Wenhan Wang, Tianlin Li, and Yang Liu. 2024. VulAdvisor: Natural Language Suggestion Generation for Software Vulnerability Repair. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. Association for Computing Machinery, New York, NY, USA, 1932–1944. doi:10. 1145/3691620.3695555

    work page arXiv 2024

  56. [56]

    Xin Zhou, Sicong Cao, Xiaobing Sun, and David Lo. 2025. Large Language Model for Vulnerability Detection and Repair: Literature Review and the Road Ahead.ACM Transactions on Software Engineering and Methodology34, 5 (2025), 1–31. doi:10.1145/3708522 , Vol. 1, No. 1, Article . Publication date: May 2026. 24 Jia Li, Zhuangbin Chen, Yuxin Su, and Michael R. Lyu

    work page doi:10.1145/3708522 2025

  57. [57]

    Xin Zhou, Kisub Kim, Bowen Xu, Donggyun Han, and David Lo. 2024. Out of Sight, Out of Mind: Better Automatic Vulnerability Repair by Broadening Input Ranges and Sources. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. Association for Computing Machinery, New York, NY, USA, 1–13. doi:10.1145/ 3597503.3639222 , Vol. 1, N...

    work page arXiv 2024