Measure the ROI of your AI coding tools

If you’re paying for AI coding tools, someone will eventually ask what that spend is buying. Until now, answering that question meant stitching together tool invoices, headcount, and delivery data by hand.

The new ROI tab in AI impact turns that one-off analysis into a standing view. It shows what your teams ship each month and what that work costs, so you can see output and cost side by side.

The tab has three sections:

• Total cost shows a monthly chart of developer cost and AI cost.
• Throughput and cost per unit shows how much your teams ship per FTE each month — stories and epics, merged pull requests, and lines changed — and what each of those units costs, stacked into developer and AI cost.
• Team breakdown shows the same metrics for each team, with the option to drill into child teams.

AI cost is the list price of the tokens your AI coding tools consume. Developer cost is based on the average cost you set for a full-time developer and is visible to admins only.

A good place to start is the AI cost share. If AI spend is a small fraction of your total cost, the upside is in getting more out of the tools, not in trimming the tool bill. From there, read the view as a trend rather than a single month. If the AI cost share climbs while cost per unit falls, you’re trading engineering time for tool spend and getting more out of each unit of effort. If cost per line falls while PR cost rises, your pull requests are getting bigger: AI helps produce more code per change, but each change still needs to be reviewed and shipped.

Of the three units, cost per story or epic is the closest to business value: a completed issue is a customer-facing increment, although you should judge whether it still represents what it used to, as your teams change how they work.

To get started, navigate to AI tools → AI impact → ROI. You’ll need your AI coding tools connected to Swarmia, and an org admin needs to set an average yearly developer cost in settings.