Various studies have shown that developers spend only 25% to 35% of their time actually coding. That means if a copilot boosts their coding productivity by a factor of 2, it may only save an hour of their day. Which begs the question: where is the rest of their time spent and how can AI impact those activities as well? How do we maximize the ROI on software development?
In addition to coding, a developer’s time is spent on:
- planning and design
- meetings and collaboration
- testing and debugging
- code reviews
- documentation
- learning and training
The good news is that AI can impact every aspect of a developer’s day, as well as the other members of the development team.
The first two items are closely related as most of the time spent in meetings is focused on planning and design. When it comes to documents and specifications, Agile practices espouse a philosophy of “just enough, just in time.” This philosophy impacts the quality of written documentation. Often developers are given a rough idea of the requirements and then end up in a slew of meetings to ensure the vague descriptions are understood. It would be better and more efficient to start with clear, complete specifications.
Generative AI is very good at creating requirements that include all the detail a developer needs to implement a feature. It is also good at summarizing information. By using AI to generate complete requirements, development teams can reduce the amount of meeting time and collaboration required. The developer doesn’t even have to read the full specifications. They can read a summary and then ask the AI to clarify any questions. In fact, these specifications can feed directly into the code copilot to automate the first iteration of code generation.
Testing also benefits from better documentation and Generative AI as well. Just as copilots can generate code from clear specifications, they can also generate the tests directly as well. Since the tests are generated from the same specs, they are available to validate the functionality of the code when it is complete. In fact, AI can execute the tests and summarize these results, which saves time for both the developer and the test engineer.
This brings us to code reviews. When most of the code is generated by AI, code reviews by humans will become even more important. In fact, developers will likely spend a higher percentage of their time reviewing AI generated code, since the other activities of their day will be minimized.
Much of the drudgery of code reviews can and should be eliminated however, whether the code is generated by humans or AI. Static code analysis tools can ensure the code is written according to the company standards and properly documented. Static security analysis and open source compliance tools can ensure that most of the well-known vulnerabilities are eliminated with no human intervention. As a result, the human code reviewer can focus on the high-level design. AI can help with review as well, providing the developer with an overview and guided tour of how the code was created, which further eliminates some of the documentation work that the developer is normally required to do.
The proper architecture of the code can be ensured through better specifications up front as well. When a human developer writes code, much of the architectural decisions are in their head as they are writing the code. For very complex projects, they may produce some rough documentation, but for most of their work, they wing it. With the advent of copilots, there is a need to specify the high-level design of code in a way that is easily created by the architect and understood by AI. This should be a growing area of innovation over the next year.
The net result will be a shift in the way developers spend their day. A higher percentage of their time will be spent on architectural design and code review relative to what they do today. The code review will be easier and less time consuming due to the AI assist. Functional testing will be easier since a test script will be provided before the code is even produced and less time will be spent in meetings and writing documentation.
The net impact on ROI will be an increase in value delivery to the business with a reduced investment cost per feature. There will be an improvement in the quality of changes delivered and reduced risk of down time. This improved ROI is not due solely to the adoption of a code generation copilot, but to the use of AI in all aspects of the development process, especially in the front-end planning phase. Improved ROI comes not only from the productivity improvement of developers, but from improvements in the business analyst and testers as well.
To measure the impact of Generative AI on the development process, start with the DORA metrics that we all know and love. The two most important metrics for developers are Lead Time to Change and Change Failure Rate. If changes require half the time to make it to production at the same or better failure rate, then you have doubled your developer’s productivity.
When you streamline the development process, you should be able to see an improvement in Change Failure Rate. This improvement in velocity should also mean that your team has more time available to iterate on their designs before release, resulting in more usable software and happier users. It’s difficult to measure the true value of happy users, but it should improve productivity across the board.
You may also like…
Q&A: Evaluating the ROI of AI implementation
Discerning reality from the hype around AI