A machine learning team can generate an enormous number of choices in a single afternoon: which dataset snapshot, which model family, which learning-rate range, which metric to chart, which environment to run in, where to log the results. Left open, each of those becomes a small negotiation, and the negotiations pile up. What follows is a sequence of steps to limit your choices in an ML workflow by deciding a short menu ahead of time. It is written for data scientists and platform engineers who want fewer arguments and more comparable results, and it treats the problem as an evidence question: does narrowing the menu actually improve what the team learns?
Step 1: Count the decisions you currently make
Before changing anything, observe. Over one sprint, ask each person to tally, roughly, how often they pause over a setup choice that someone else on the team already made differently. You are not measuring time to the minute; you are looking for the choices that recur and vary for no clear reason. Typical culprits include run naming, random seed handling, train-validation split logic and where artefacts are stored.
The reasoning here is analytical. A decision that varies between runs and has no bearing on the outcome is noise. A decision that does affect the outcome is a variable you want to control deliberately. The first group can be fixed by default; the second needs a labelled, intentional choice.
Step 2: Sort choices into “fixed”, “menu” and “open”
Create three buckets. Fixed choices have one answer for everyone: the experiment tracking location, the naming scheme, the way seeds are recorded. Menu choices have a short list of approved answers, for example three model families for a given task type, or two data-splitting strategies. Open choices are the ones where genuine exploration is the point, such as feature ideas or architecture variations in a research spike.
The proportions matter. If most of your list lands in “open”, you have not yet been honest about which choices matter. If everything is “fixed”, you have removed the thinking. A healthy split leaves exploration where it earns its keep and takes it away from everywhere else.
Step 3: Write each menu as a config preset
Turn the buckets into named configuration files. A preset might define the data split, the metrics reported, the logging destination and the compute profile in one place. Running an experiment then means picking a preset and supplying the few parameters that are genuinely open. This is the practical heart of the method: the menu lives in code, not in memory. For the broader background on how such automation fits into the discipline, our stage-by-stage map of MLOps for data and engineering teams lays out where each piece belongs.
Step 4: Make the preset the path of least effort
The evidence from every tooling rollout is the same: people follow the easiest route. If launching a run from a preset is a single command and launching a custom setup takes a dozen edits, most will use the preset. Put the command in the project readme, add it to the template repository and keep the number of presets small, three or four being a sensible ceiling for any one task type.
- Offer a default that most people can accept untouched.
- Show exactly where to override a value when needed.
- Print the resolved settings at the start of each run, so results are self-describing.
Step 5: Run a short comparison to test the effect
Because this is an evidence-led approach, test it. For two or three weeks, track a couple of simple signals. How often does someone ask “what settings did you use?” How often do two supposedly comparable runs turn out to differ in setup? How many runs have to be repeated because a detail was missed? You do not need rigorous statistics for a team of a handful of people, but a before-and-after tally often makes the benefit obvious. Many teams find that reproducibility questions fall sharply once the setup is a named preset rather than a copy-pasted script.
Step 6: Connect the menu to your deployment path
Experiments eventually become services. A tidy, narrow set of training presets makes the later handover easier, because the production pipeline can mirror the same few shapes. Our overview of how MLOps speeds up model deployment and supports growth explains why a consistent path from experiment to release pays off, and a habit of named presets is among the least expensive ways of reaching it.
Step 7: Handle the people side with the same discipline
It is easy to design the technical menu and forget the human one. Who covers a long-running job when its owner is away? How are absences recorded so that review rotations are fair? Teams that settle those conventions early avoid a steady drip of interruptions. One site worth a glance here is Inoroff, which describes software for attendance and absence handling in smaller and mid-sized teams, pairing leave requests and manager sign-off with work schedules. The details of how it frames them are on the Inoroff site for time-off and schedule management. A shared calendar and one written rule can do the same job, provided the routine is settled up front rather than renegotiated.
A note on small teams and solo practitioners
None of this requires a large platform group. A single practitioner can keep a folder of three config files and a short text note describing when to use each. The evidence for the approach is just as visible at that scale: the next time you return to a project after a month away, you will find that you can rerun an old experiment without reconstructing your own reasoning. That alone is worth the hour it takes to set up.
Step 8: Schedule a review and prune
Menus decay. New libraries appear, data changes, and someone discovers that a preset has been quietly ignored for months. Set a recurring review, perhaps monthly at first and then quarterly, and for each preset ask three questions. Is it used? Does it still reflect good practice? Can it be merged with a neighbour? Retire any preset that fails all three. A shrinking list is a healthy sign, a sprawling one a warning.
Worked example: a tabular classification project
Imagine a team building classification models on tabular business data. Before the exercise, every person chooses their own split, their own metric subset and their own place to save models. Comparing two experiments requires an archaeological dig through scripts.
After the exercise, the team has one fixed tracking location, one naming scheme, a menu of two splitting strategies, a menu of three baseline model families and an open slot for feature ideas. Running an experiment is a command with the preset name and a short list of overrides. When two results look different, the first question is about the open choices, since everything else is identical. That alone saves a surprising amount of discussion.
What the narrowing does not do
It does not replace judgement. A weak model stays weak, and careful validation is still required. What it does is make the differences between runs meaningful, so that when you do change something deliberately you can trust the comparison. If you are looking for guidance beyond the workflow itself, our introduction to why MLOps matters for machine learning success is a sensible companion.
Quick reference
- Observe the recurring setup choices for one sprint.
- Sort them into fixed, menu and open.
- Write the menus as named presets in code.
- Make the preset the easiest way to start a run.
- Compare before and after with a simple tally.
- Mirror the presets in your deployment path.
- Agree the people routines on the same principle.
- Review on a calendar and prune.
Teams that follow these steps usually discover that their real creativity needed fewer options than they feared. Narrowing never starves curiosity; it points it somewhere useful.