Adaptive Test Healing with LLMs and Reinforcement Learning

Publication at a glance

Paper: Adaptive Test Healing using LLM/GPT and Reinforcement Learning
Authors: Nariman Mani, Salma Attaranasl
Published in: IEEE ICSTW / AIST 2025
Persistent record: 10.1109/ICSTW64639.2025.10962516
Google Scholar: View the publication record

The research problem

Automated tests break when interfaces, selectors, data, or expected behavior evolve. Rule-based repairs are brittle, but accepting the first LLM-generated patch is unsafe because a test can pass after its original intent has been weakened.

The paper's central contribution

This paper combines the semantic capabilities of LLM/GPT models with reinforcement learning for adaptive test healing. The LLM can interpret code and failure context; the learning policy can choose actions based on feedback rather than treating every proposed edit as equally valuable.

This distinction matters for both researchers and practitioners: the contribution is a way to reason about the complete system and its decision boundaries, rather than a claim that adding one AI model solves the underlying engineering problem.

How the approach works

Failure context and test intent give the language model evidence for candidate repairs.
A reinforcement-learning formulation connects healing actions with observed rewards and penalties.
Validation feedback helps distinguish a robust repair from one that merely suppresses a symptom.

Together, these elements make the approach easier to study, reproduce, and extend. They also provide clear terms for literature searches and comparisons with related work.

Why this publication matters

The approach is relevant to automated program repair, flaky-test remediation, intelligent CI/CD, and AI-assisted quality engineering. It is particularly useful as a conceptual foundation for systems that generate several possible repairs and need a policy for selecting among them over time.

For practitioners, the larger lesson is to measure the system behavior that matters—not only a model-level score. Reliability, privacy, change over time, human review, and downstream consequences belong in the evaluation.

Research questions this work can support

This publication is relevant when investigating questions such as:

How should the core problem be represented so an AI-assisted system retains the right context?
Which technical and organizational signals provide early, actionable evidence?
How can automation improve adaptability without concealing failures or weakening safeguards?
What evaluation design captures effectiveness, safety, privacy, and long-term change?
Where should a system defer to a developer, architect, coach, clinician, or participant?

Scope and responsible interpretation

Reward design can encourage shortcuts, including assertions that are easier to satisfy. Training signals may drift as the application evolves, and generated code can create security or maintenance problems. Human-readable diffs, semantic checks, mutation testing, rollback, and conservative confidence gates remain essential.

These boundaries are opportunities for replication, comparison, and follow-on research. Readers should consult the paper itself for its precise method, datasets, experimental setup, and reported results.

How to cite this paper

Mani, N., & Attaranasl, S. (2025). Adaptive Test Healing using LLM/GPT and Reinforcement Learning. IEEE ICSTW/AIST 2025. https://doi.org/10.1109/ICSTW64639.2025.10962516

Use the 10.1109/ICSTW64639.2025.10962516 publication page to verify the latest bibliographic metadata before submission. You can also find the work through its Google Scholar record. If your research builds on the architecture, method, evaluation, or research framing described here, please cite the original publication rather than this explanatory post.

Related research on this weblog

Browse the complete Nariman Mani publications list.
Explore GraphQL-aware healing with multi-signal learning.
For adjacent work, read adaptive test healing with LLMs and reinforcement learning and privacy-by-design adaptive group assignment.