The State of AI in Large Scale Automated Refactoring

LLMs are data-hungry, and when it comes to source code, more than the text is needed to make large-scale inferences about a codebase. Code has a unique structure, strict grammar, dependencies, and type information that a compiler must deterministically resolve. This information could be beneficial for AI but is invisible to the text of the source code.

For example, try to answer even a simple question about where Guava is used or where a particular logging library is used. While developers can find references in the code, the code-as-text may not have a reference to the library you are looking for. Imagine a logger instance inherited as a protected field from a base class defined in a binary dependency. The import statement that identifies which logging library that logger is coming from is IN the binary dependency, not in the text of the call site. A human would do no better in this situation.

This talk addresses improving AI accuracy for large-scale code refactoring by enhancing the data source. We’ll explore a state-of-the-art code data model called the Lossless Semantic Tree (LST) that’s part of the open-source OpenRewrite auto-refactoring project. The LST and recipes are straightforward tools to equip LLMs with the data they need to make accurate decisions.

The common excuse for inaccuracy or incompleteness in output is that LLMs will improve, but the models are good enough. What they too often need is the data to make inferences. We’ll show why when evaluating LLMs for large-scale automated refactoring:

• If it's based on text, you don't want it
• If it's based on AST, you don't want it

Speaker
Jonathan Schneider is a Java Champion, co-founder, and CEO at Miami-based Moderne, the pioneer of mass-scale auto-refactoring and codebase analysis. He founded OpenRewrite, an auto-refactoring tool, at Netflix and went on to found the Micrometer project as a member of the Spring Team. He also is the author of “SRE with Java Microservices” (O’Reilly) and “Automated Code Remediation” (O’Reilly). He is an Army veteran and two-time Bronze Star recipient.

Schedule:
6:30PM-7:00PM EST Networking
7:00PM-8:00PM EST Presentation
8:00PM-8:30PM EST Q&A Discussion