Harness this week acquired Codecov, a provider of a platform that analyzes the percentage of a codebase that has been tested, from Sentry.
Brad Rydzewski, a senior vice president and general manager for Harness, said Codecov makes it simpler for DevOps teams to track testing coverage at a time when the volume of code being created in the age of artificial intelligence is exponentially increasing.
Codecov is already widely used by enterprises and maintainers to automatically run tests on any code that for one reason or another was not tested earlier in the software development lifecycle (SDLC).
Going forward, Harness plans to integrate the data that Codecov generates in real time with the Harness Software Delivery Knowledge Graph to provide deeper insights across DevSecOps workflows as AI agents are integrated into workflows.
Ultimately, the goal is to automate testing as much as possible while maintaining separation of duties between AI agents that write code and AI agents that test code, said Rydzewski. Otherwise, an AI agent that writes code when presented with data informing it that its code failed a test might decide to rewrite the test rather than fix the code it created, he noted.
The issue is that while AI tools can usually be relied on to generate a test, there still needs to be human oversight to ensure that best testing practices that are largely deterministic are being consistently followed, added Rydzewski.
In general, the rise of AI coding will force many organizations to revisit their testing processes. Many of them are not following best practices for testing code, which in the AI era will inevitably lead to issues arising later on in production environments. While best practices for testing remain largely the same in the AI era, many DevOps teams will need to focus more on fundamentals to prevent issues from arising later on that could result in regressions, outages or a security incident, said Rydzewski.
The challenge, of course, is finding a way to apply AI to both coding and testing in a way that doesn’t increase the total cost of building and deploying software to a point where it becomes economically unsustainable.
Additionally, a recent Harness survey found that while 89% of software engineering teams have seen an improvement in productivity following the adoption of AI, there are metrics not being tracked that increase total costs. For example, 81% noted that the amount of time spent reviewing code has increased. However, just under a third of their day is now consumed by AI-related tasks that existing metrics don’t surface.
A full 94% also noted technical debt, validation time, and developer burnout are not being tracked by existing productivity metrics, including time spent reviewing AI code for accuracy (53%), fixing subtle bugs from AI code (52%), explaining AI code to teammates (48%) and context switching between tools (45%).
Regardless of the level of appreciation for testing, the one thing that is certain is that all it takes is one issue in a production environment to negate a large percentage of the return on investment (ROI) that was initially hoped for when software engineering teams decided to embrace AI in the first place.
