Over the past decades, we’ve abandoned the practice of measuring developers by the number of lines of code they’ve produced. We realized that it was much easier to game the system by writing bloated code with the value down instead of up. The best developers, who made the code smaller, faster, and easier to maintain, were penalized, because they were seen as producing a poor product – but the metric was wrong. Bill Atkinson, an engineer at Apple, reportedly has one reduced 2,000 lines of code in one week. He did this while doing the math of the painting six times quickly.
Today, we can generate thousands of lines of code with a single command in a large-scale language model (LLM). It can beat anyone to deliver lines of code. However, is that really the goal?
Training
Before we get to the problem of redundant lines of code, we need to understand how LLMs come to generate code with redundant lines. The answer is in the training dataset and how that dataset was found in publicly accessible places, including open repositories on Github and coding websites. These sources have no form of quality control, so the code LLMs read from is of varying quality.
While there are some code snippets that contain careful and elegant code written by the best developers and released after quality peer review, that’s not the norm. Most publicly available repositories are public domain because they are written by novice developers. They put their stores in the open, because they didn’t see much profit in what they were producing.
Early in my SharePoint software development career, I was struck by what I saw as one of the biggest problems with the sample code that was being distributed on various sites. It comes from the official templates provided by Microsoft, bypassing the RenderControl() method, which just literally writes the HTML back to the client. It would take years of application before templates were changed to CreateChildControls(), which behaved well within the ASP.NET 2.0 stack, allowing postback events. If AI was trained on SharePoint development code before 2010, it would be wrong.
In an effort to get as much training data as possible, there was little effort available to test the training data to make sure it was good training data. An LLM result produces the kind of code written by a first-year developer – and that should contact us.
Security Issues
The last decade has seen an increasing conflict between malicious attackers who want to find software bugs and software developers who are making their work harder. Early reports of the AI code say it’s going to get worse. Some of the common vulnerabilities we’ve known about for decades, including cross-site scripting, SQL injection, and log injection, are the types of vulnerabilities that AI introduces into code – and reproduces this code in the repetitive amounts produced by small developers. At a time when it is important for us to be more vigilant about security, AI cannot do it.
Maintenance Issues
Today, we have AI generating bloated code that creates maintenance problems, and we look the other way. It cannot organize code to reduce code duplication. It doesn’t matter if there are two, three, four, or more implementations of basic operations that can be made into one common operation. The code it was trained on didn’t generate the shortcuts to create the right jobs, so it can’t get there. (See Focus on Activities (from some of the writing I’ve been doing for the past few decades on how to do cool jobs that don’t seem like the AI is crawling.)
Can we code with the help of AI? Yes. Can we “vibe code”? Definitely. However, the questions we need to ask ourselves are: 1) At what cost? 2) What can we do to reduce those costs?
The answer appears to be experienced engineers who review and recode to ensure that quality and maintenance standards are met. We first wrote about how to conduct an effective code review two decades ago Painless Effective Code Review. If you need help developing an AI (or human) generated code review pattern. we can help.
