HAFixAgent
๐ Project Overview
HAFixAgent is an automated program repair (APR) agent that augments an LLM repair loop with
history-aware blame context. Given a localized fault, it uses git blame to retrieve blame
commits, extract historical context, and injects them (one of three historical heuristics) into
the repair prompt. This repository is the replication package of HAFixAgent.
It currently supports:
- Two benchmarks: Defects4J (854 Java bugs) and BugsInPy (501 Python bugs).
- Four configurations:
non-history (control), fn_all, fn_pair, fl_diff (three history representations).
- Fault localization: perfect FL (from the developer patch) and realistic SBFL (GZoltar for Java, FauxPy for Python).
- Models: DeepSeek-V3.2-Exp (primary, via OpenRouter), Qwen3-Coder and Devstral-Small-2 (locally deployed) and any OpenAI-compatible LLMs.
Authors
๐๏ธ Repository Structure
HAFixAgent/
โโโ hafix_agent/ # Core agent (dataset-agnostic)
โ โโโ agents/hafix_agent.py # Main agent, extends mini-swe-agent
โ โโโ blame/ # Blame extraction: interfaces, core, selection, context loader
โ โโโ environments/ # Docker container management
โ โโโ prompts/prompt_builder.py # History-aware prompt construction
โ โโโ utils/ # Logging, token tracking, model specs
โโโ dataset/
โ โโโ defects4j/ # Defects4J extractor, analysis, mined bug_description/
โ โโโ bugsinpy/ # BugsInPy extractor, analysis, bug discovery
โโโ config/
โ โโโ defects4j.yaml # Perfect-FL config (Java)
โ โโโ defects4j_sbfl.yaml # Realistic-SBFL config (Java)
โ โโโ bugsinpy.yaml # Perfect-FL config (Python)
โ โโโ bugsinpy_sbfl.yaml # Realistic-SBFL config (Python)
โ โโโ models/ # DeepSeek / Qwen3-Coder / Devstral model specs
โโโ evaluation/
โ โโโ run_defects4j_evaluation.py # Defects4J repair runner (console entry: hafixagent)
โ โโโ run_bugsinpy_evaluation.py # BugsInPy repair runner
โ โโโ run_sbfl_evaluation.py # SBFL repair runner (reads cached rankings)
โ โโโ run_repairagent_parallel.py # RepairAgent baseline driver
โ โโโ run_birch_parallel.py # BIRCH-feedback baseline driver
โ โโโ cache_context.py # Pre-extract blame contexts
โ โโโ sbfl/ # Self-contained FL module: GZoltar jars + FauxPy layer
โโโ analysis/ # One script per paper RQ table/figure (see below)
โโโ results/ # Curated result subset backing the paper
โโโ Dockerfile.defects4j-enhanced # Defects4J image with extra bash tools
โโโ Dockerfile.bugsinpy-enhanced # BugsInPy image with extra bash tools
โโโ pyproject.toml
๐ Environment Setup
git clone https://github.com/SAILResearch/HAFixAgent.git
cd HAFixAgent
conda create -n hafixagent python=3.11
conda activate hafixagent
pip install -e .
# Optional: SBFL fault-localization stage only (FauxPy for BugsInPy; GZoltar jars are bundled)
pip install -e ".[sbfl]"
-
LLM API. We extend mini-swe-agent and call models through
OpenRouter. The model is set in each config's model section (see config/models/). Set your key:
mini-extra config set OPENROUTER_API_KEY <OPENROUTER_API_KEY>
Any LiteLLM-supported provider works by editing the model config.
-
Defects4J Docker image.
mkdir -p vendor && cd vendor
git clone https://github.com/rjust/defects4j.git
cd defects4j && docker build -f Dockerfile -t defects4j:base .
cd ../../ && docker build -f Dockerfile.defects4j-enhanced -t defects4j:latest .
-
BugsInPy Docker image. Build the base bugsinpy_image:clean from the
BugsInPy projects, then enhance it:
docker build -f Dockerfile.bugsinpy-enhanced -t bugsinpy_image:clean .
๐ RQ0: Blame Availability
How often does buggy code have blame history, and how concentrated is it?
python analysis/analyze_rq0_blame_availability.py
๐ณ RQ1: Effectiveness, Complementarity, and Cross-Model Generalization
Run the four configurations under perfect FL on Defects4J:
# hafixagent is the console entry for python evaluation/run_defects4j_evaluation.py
# --bug-category: single_line | single_hunk | single_file_multi_hunk | multi_file_multi_hunk | all
hafixagent --bug-category all --history baseline --selector-type llm_judge --workers 4
python evaluation/run_defects4j_evaluation.py --bug-category all --history fn_all --selector-type llm_judge --workers 4
python evaluation/run_defects4j_evaluation.py --bug-category all --history fn_pair --selector-type llm_judge --workers 4
python evaluation/run_defects4j_evaluation.py --bug-category all --history fl_diff --selector-type llm_judge --workers 4
# Scope to a single category instead of all:
python evaluation/run_defects4j_evaluation.py --bug-category single_file_multi_hunk --history fl_diff --selector-type llm_judge --workers 4
# BugsInPy (same four configs)
python evaluation/run_bugsinpy_evaluation.py --bug-category all --history fl_diff --workers 4
# Cross-model: pass a model config to any runner
hafixagent --bug-category all --history fn_pair --model-config config/models/qwen3_coder_next.yaml --workers 4
Analysis (ablation tables, Venn complementarity, external baselines, significance):
python analysis/analyze_rq1_effectiveness.py # history ablation + complementarity
python analysis/analyze_rq1_external_baselines.py -b repairagent # choices: repairagent, hunk4j
python analysis/analyze_local_model_ablation.py --tag qwen3coder # cross-model ablation (--tag devstral for Devstral)
python analysis/analyze_ablation_significance.py --regime perfect # per-config and Union McNemar
๐ RQ2: Realistic Fault Localization (SBFL)
# 1. (optional) regenerate the FL cache: GZoltar (Java) / FauxPy (Python)
python evaluation/sbfl/run_fl_defects4j.py --all --workers 8
python evaluation/sbfl/run_fl_bugsinpy.py --all --workers 4
# 2. repair with SBFL-ranked top-10 locations (repeat --history for baseline, fn_all, fn_pair, fl_diff)
python evaluation/run_sbfl_evaluation.py --dataset defects4j --history fl_diff --all --workers 4
# 3. FL accuracy (top-N hit rate) and repair outcomes
python analysis/analyze_rq2_sbfl_fl_accuracy.py --dataset defects4j
python analysis/analyze_rq2_sbfl_repair.py --latex
๐ฐ RQ3: Cost and Efficiency
python analysis/analyze_rq3_cost_efficiency.py
๐ Citation
If you found this work helpful, please kindly consider citing it using the following:
HAFixAgent
@article{shi2025hafixagent,
title={HAFixAgent: History-aware program repair agent},
author={Shi, Yu and Li, Hao and Adams, Bram and Hassan, Ahmed E},
journal={arXiv preprint arXiv:2511.01047},
year={2025}
}
HAFix (Prior Foundation Work)
@article{shi2025hafix,
title={HAFix: History-Augmented Large Language Models for Bug Fixing},
author={Shi, Yu and Bangash, Abdul Ali and Fallahzadeh, Emad and Adams, Bram and Hassan, Ahmed E},
journal={arXiv preprint arXiv:2501.09135},
year={2025}
}
๐ง Contact
For questions or issues, please open a GitHub Issue.
๐ Acknowledgement