Impact of LSP Diagnostics on Agentic Code Translation Repair : A Controlled Comparison of Raw Compiler Output and Structured LSP Feedback
Younus Mashoor, Ebbe Karlstad · DiVA (Linnaeus University) · 2026
Automated code translation is an important tool for modernising legacy systems andcodebases, but LLM-based translation pipelines may require automated program re-pair, because so-called one-shot translations frequently fail to compile, and an itera-tive repair loop is a common approach to address this issue. Most existing pipelinesuse raw compiler error output as the feedback signal for the repair stage, while noprior work that we found has compared this against structured feedback signals likeLanguage Server Protocol (LSP) diagnostics in cross-language translation repair. Weperformed a controlled experiment using an agentic C++ to Rust translation pipelinewhere the feedback signal was the only variable changed between conditions. Con-dition A received raw compiler output, Condition B received compiler output aug-mented with LSP diagnostics from the Rust LSP server, rust-analyzer. Our experi-ment was run on 59 translation units across 8 open-source C++ projects, and we ranboth conditions with up to 8 repair iterations per repetition, producing 332 total runs.The main measured metrics were compilation success rates, number of iterations tofirst success, and wall-clock time. We compared the metrics of both conditions us-ing McNemar’s exact test on paired per-unit outcomes. We found that Condition Bcompiled 1.2 percentage points more runs than Condition A (95.2% vs 94.0%), butthe difference was not statistically significant (p = 1.000), partly because of the ceil-ing effect from 47 out of 59 translation units compiling under both conditions. Wealso found that Condition B used fewer repair iterations on 5 of 8 projects, but wasslower in wall-clock time on 5 of 8, which indicated that LSP feedback shifts costfrom repair rounds to per-iteration latency rather than reducing it. At this datasetdifficulty and model capability, raw compiler output appears to be sufficient for mostself-contained files, and the engineering cost of LSP integration is hard to justifywithout harder workloads.