Co-authored arXiv paper mining BDD subscenario refactoring candidates across 5.3M slices using ML classifiers benchmarked against LLM judges. XGBoost classifier (F₁ = 0.891) beat a tuned rule baseline and two open-weight LLMs (GPT and Ling) at p < 10⁻⁴.
No prior work automated which recurring step subsequences in BDD test suites are worth extracting as refactoring candidates, or which of the three standard patterns (Background, reusable-scenario, shared-step) applies.
Mined every contiguous L-step window (L ∈ [2,18]) across a 339-repo corpus, keyed by paraphrase-robust cluster IDs from the foundational BDD paper. Applied SBERT/UMAP/HDBSCAN for paraphrase clustering. Trained XGBoost extraction-worthy classifier under 5-fold CV; compared against rule baseline and two open-weight LLM judges on a human-labelled 200-slice pool.
XGBoost F₁ = 0.891 (95% CI [0.852, 0.927]) beat LLM judges at F₁ ≤ 0.728 (McNemar p < 10⁻⁴). 75% / 59.5% / 11.7% of scenarios carry within-file / within-repo / cross-org refactoring candidates respectively.
Ask Bilal’s AI Clone