Site reliability engineering, usually shortened to SRE, answers a question every digital business eventually faces: how reliable does our service need to be, and who decides when it is reliable enough? Without an agreed answer, engineers over-protect some systems, under-protect others, and argue with product teams about every release. This explainer describes the method in plain language, using the reference entry in the Ops Atlas on site reliability engineering for definitions and figures.
The core idea in one paragraph
SRE treats the running of a service as an engineering task. Instead of relying on goodwill and late nights, you describe what users experience, set a numeric target for it, and measure continuously. The gap between the target and perfection is a small, explicit allowance for failure. That allowance then governs how fast the team changes things: plenty left means ship boldly, little left means stabilise. Everything else in the discipline, from alert rules to staffing, supports that loop.
Four terms worth learning properly
Service level indicator
An indicator is a ratio: good events divided by valid events. If 9,990 of 10,000 valid requests succeed, the indicator is 99.9 percent. The Atlas stresses measuring it close to the user. A server that reports itself healthy tells you less than a probe that behaves like a customer. It also advises reporting speed as the share of requests quicker than a chosen limit instead of a mean, since a pleasant-looking mean can conceal a sizeable group stuck waiting.
Service level objective
An objective is the target for that indicator across a window of time. It should never be 100 percent. The entry recommends a rolling four-week window, because 28 days holds the same number of each weekday. It also offers a rule of thumb for dependencies: a component your service relies on should aim for one more nine than your own target, otherwise it will drag you below your promise.
Error budget
The budget is one minus the objective. Take a three-nines target on a 28-day window: the allowance comes to a little over forty minutes of degraded service, which is the same as one thousand bad requests in every million served. It is not an excuse to be sloppy; it is a shared currency. Product teams spend it on releases and experiments. Operations teams guard it. When it is gone, the agreed policy takes over.
Toil
Toil describes jobs done by hand, again and again, which software could handle, which react to events, and which leave nothing durable behind. It grows as the service grows. The Atlas quotes a Google rule that it should stay under fifty percent of an engineer’s week, and warns that a cap without consequences means little.
How the pieces work together: a worked example
Suppose a checkout service handles 50 million valid requests in a window and has a 99.9 percent objective. The budget is 50,000 failed requests. During the window, 31,000 requests fail. Success lands near 99.938 percent, so the objective holds, though 62 percent of the allowance has been consumed. With a third remaining, the team can continue with its planned releases while keeping an eye on the remainder.
Now suppose a bad deployment pushes the failure ratio to 0.6 percent. That is six times faster than the budget allows, a burn rate of six. At that pace, the sheet calculates that about 43 hours of budget remain. A well-designed alert would page someone well before the budget vanishes, whereas a much smaller leak would simply be ticketed for office hours.
What an alert looks like in this world
Classic alerting asks whether a metric crossed a line. Burn-rate alerting asks whether the budget is draining faster than it should. The published workbook recommendation for a 99.9 percent objective pairs a short and a long window for each severity, so brief blips are ignored and sustained problems are caught. Fast burns page a person; slow burns open tickets. That arrangement is why an on-call engineer can sleep through trivia and still be woken for a real incident.
People and process around the maths
- On-call. The Atlas proposes a ceiling of two incidents in each twelve-hour shift, with about eight people sharing a rota for one site covered around the clock.
- Postmortems. Reviews should be blameless towards individuals while firm on follow-up actions, owners and deadlines.
- Capacity. Forecast demand beyond the time it takes to add capacity, and test it under load. Spare capacity rules such as N+2 lower the safe utilisation of small clusters considerably.
- Change management. Roughly 70 percent of outages are reported to come from live changes, which is why the budget policy focuses on release pace.
Common misunderstandings
Four mistakes appear repeatedly. First, relabelling an existing operations team without changing how it works; the Atlas calls that fake SRE. Second, demanding five nines for everything regardless of cost. Third, publishing objectives without a signature from someone senior. Fourth, filling dashboards with alerts for every metric until people stop reading them. Avoiding these four puts a team well ahead of many that use the label.
SRE is also not the same as monitoring, which the entry places in a neighbouring discipline, nor is it identical to DevOps. The relationship is that SRE offers a concrete implementation of the DevOps philosophy. If you want a side-by-side treatment, see our guide to implementing DevOps and SRE together in engineering teams.
Where to start in your own organisation
Begin small and on purpose. Pick one service with real users. Write one indicator, one objective and one budget, then draft a one-page policy that states what happens when the budget is exhausted. Ask a senior stakeholder to sign it. Run for a month and look at the results before adding a second service. The Atlas suggests a quick screen before committing: is there an outage users would notice, will an executive sign, and can you sustain a rotation of the right size?
For a structured plan, our walkthrough on building an SRE roadmap for enterprise IT lays out phases, while the training guide for operations and engineering teams covers the skills people need to run it.
Summary
SRE comes down to agreeing on a user-centred measure, accepting that perfection is the wrong goal, and spending a small, known allowance for failure in a disciplined way. The vocabulary may look technical, but the logic is the same as any sensible budget: know what you have, know what you are spending, and decide in advance what to do when the money runs low.