A data scientist once told me her team’s worst meeting was not about a failed model. It was about a bill. Everyone arrived with a theory, nobody had a shared number, and an hour later the only agreement was that the figure looked large. The way out, she said, was not a smarter theory. It was a smaller number: the cost of one thing the team could recognise. Here are eight such units for machine learning teams, each with the question it helps you ask.
1. Cost per prediction
The classic. Divide what it costs to run a model in production by the number of answers it gave. It turns a vague “serving is expensive” into a figure you can compare across models, and it quickly shows whether a bigger model earns its keep. Ask: would a slightly less accurate model serve the same purpose for a fraction of the effort?
2. Cost per training run
Every experiment has a price in compute time and in the people waiting for it to finish. Counting the cost of one run shows how quickly small experiments add up, and why a tidy experiment habit beats a heroic one. Ask: which runs taught us something, and which only repeated an earlier lesson?
3. Cost per retrained model that reaches users
Not every trained model ships. If you divide the full spend on experiments by the models that actually reached production, you learn the true price of a release. This can be sobering and useful in equal measure. A team that builds sound deployment habits tends to see the ratio improve, which is one reason the argument in how disciplined MLOps speeds up model delivery resonates beyond engineering.
4. Cost per gigabyte of data kept alive
Datasets pile up like old paperwork. Copies, intermediate files, feature tables nobody has opened for months. Dividing storage spend by the amount that is actively used reveals how much you pay to remember things. Ask: if we deleted this tomorrow, who would notice?
5. Cost per pipeline execution
Scheduled jobs run whether or not anyone reads their output. Looking at the price of one execution next to how often its result is consumed tends to start friendly conversations about schedules. Ask: does this need to run every hour, or would every day do?
6. Cost per active user of a model-powered feature
This is the unit product managers care about. It lets the business see how a recommendation panel or a smart search box affects the margin on each customer, and it brings the data team into pricing discussions as equals. Ask: what does one happy user of this feature cost us each month?
7. Cost per idle hour
Reserved machines that nobody is using are the most expensive kind of silence. Counting the hours of idleness, not just the hours of work, makes visible what a monthly total hides. Ask: who owns this machine, and when did anyone last use it?
8. Cost per team member’s day of waiting
The least technical unit, and often the most revealing. If a slow pipeline makes five people wait an afternoon, that delay is a cost, even though no invoice lists it. Ask: what would shaving that wait do for the team’s morale and speed?
How to choose among the eight
A word of caution before you pick: a unit is only useful if the denominator is trustworthy. Counting predictions that include health checks and retries inflates the divisor and flatters the result. Spend ten minutes agreeing on what counts, and write it down.
You do not need all of them. Pick the one that matches the next decision you face. Planning a launch? Start with cost per prediction and cost per active user. Reviewing research habits? Try cost per training run. Cleaning up infrastructure? Look at storage and idle hours first. For a grounding in why the whole discipline matters, the primer on what makes machine learning projects succeed in practice is a good starting point.
Where the bill and the conversation meet
Most of these units come from cloud platforms, and cloud billing was not designed around any of them. Bridging the gap between the invoice and the unit is a skill in its own right, usually called FinOps. FinOpsSchool runs FinOps courses and certification programmes, and its classes bring engineers and finance colleagues together to practise reading cloud costs in the same terms, which is the habit these eight units depend on.
Three habits that make the units stick
First, name an owner for each unit. A number without a person attached is a decoration. Second, write the definition once, in a place everybody can read, so no two teams count differently. Third, review the figures on a calendar, not in a crisis. Reading unit costs in a calm week builds the muscle that you will need in a stressful one. Teams that want outside help in setting those routines may find the overview of working with an outside team on machine learning operations a sensible companion.
A final story
The data scientist from that bill meeting eventually asked for just one change: put the cost per prediction in the team’s weekly notes, next to accuracy. Within a month, a model that had been quietly beloved turned out to be several times more costly per answer than its leaner sibling, with no real difference in how users received it. Nobody had been careless; the number simply had not been in the room. Once it was, the decision took ten minutes.
That is the quiet power of a unit cost. It does not tell you what to do, but it makes sure that when you decide, you are deciding about the right thing.