Every operations team has a story about the time it acted on a confident claim that turned out to be half-true. Maybe a blog post promised a kernel setting would fix latency, or a conference talk made a migration sound painless. The teams that get burned less often are not smarter; they simply have a few evidence-first habits baked into how they work. Here are eight of them, written for DevOps and SRE folks who want fewer surprises and calmer on-call weeks.
1. They read the primary source before the hot take
When a cloud provider has an outage, the first wave of commentary arrives within minutes. The provider’s own post-incident report often arrives days later. Evidence-first teams wait for that report, or at least treat everything before it as speculation. The same goes for security issues: the official advisory and the project’s own changelog carry more weight than any thread summarising them.
2. They ask “where did this number come from?” out loud
Someone says a new caching layer will cut response times in half. Lovely. Where was that measured, on what workload, with which configuration? Asking the question in a friendly tone, as a team norm, removes any sense of personal challenge. Over time people start bringing the source along with the claim, which saves everyone time.
3. They reproduce before they roll out
A claim that something “works” is a hypothesis until you have seen it on your own systems. Good teams keep a cheap environment where they can try the change with realistic data. Reproduction does not need to be elaborate; a small load test or a single canary instance is often enough to confirm or puncture a promise.
A useful trick is to write the expected result before running the experiment. If the article claimed a forty percent improvement and your test shows a small one, that gap is information worth keeping. It tells you something about the original conditions, your workload, or both, and it keeps the next proposal honest about what was really observed.
4. They write down what they still don’t know
Change proposals usually describe the expected benefit. Strong ones also include a short “open questions” list. What happens under peak traffic? Has anyone tested the failure path? Naming the unknowns makes them visible to reviewers, and it stops optimism from quietly filling the gaps.
5. They separate the incident timeline from the incident story
After an outage, memories blur and a tidy narrative forms fast. Teams that learn well build the timeline from logs, alerts and chat history first, then write the explanation. It is the operational version of checking your sources. Our guide to improving incident handling with SRE practice goes deeper on running reviews that rely on records rather than recollection.
6. They treat popularity as a signal, not proof
A pattern that everyone is talking about deserves attention. It does not deserve automatic adoption. Evidence-first teams use buzz as a prompt to investigate, then judge the idea on documented results and their own experiments. If the only support for a practice is that it is fashionable, they leave it on the reading list for now.
7. They borrow the habit from other fields
Some of the clearest examples of claim-checking come from outside tech. Health information is a good one, because the gap between marketing and evidence there can be wide. People researching cosmetic procedures, for example, are surrounded by glossy promises. Best Cosmetic Hospitals, an informational site covering cosmetic surgery and medical travel, says it bases its guides on cited sources such as ASPS and the FDA and notes that its content is not medical advice. That is exactly the shape an engineer wants from a runbook or design doc: references you can follow, and an honest boundary on what the document covers.
8. They revisit decisions when new evidence lands
Being evidence-first is not a one-time check. A decision made last quarter on the best data available may need another look when a fresh incident report or release note appears. Mature teams keep their reasoning written down so revisiting it is quick. A longer-term plan helps here, and our step-by-step approach to an enterprise SRE roadmap shows one way to schedule those reviews instead of leaving them to chance.
Bonus: what these habits are not
None of this means being slow or sceptical of everything. When an official advisory confirms a severe vulnerability under active exploitation, you act. The point is that the trigger is a primary source. Evidence-first also does not mean ignoring experience; a senior engineer’s instinct is valuable data. It simply gets labelled as instinct rather than presented as measured fact.
Putting it into practice this week
You do not need to adopt all eight at once. Pick two and try them for a sprint:
- Add an “origin” line to your change template, where the author notes where any key claim came from.
- Add an “open questions” line beneath it.
Those two lines cost a minute per change and quietly reshape how the team talks about risk. After a few weeks, you will notice the conversations in review shifting from “I think” to “the changelog says” and “our test showed.” That shift, more than any single tool or framework, is what makes an operations team dependable.
Wrapping up
The common thread across all eight habits is curiosity about origins. Where did this claim start? What does the original actually say? What have we confirmed ourselves, and what are we still guessing? Teams that keep asking those questions do not avoid every mistake, but they make far fewer of the avoidable ones, and they learn faster from the rest.