Definition
The ML workflow is the sequence of steps for taking a problem to a working model: define the problem, get the data, prepare and explore it, train and evaluate models, deploy the best one, and monitor it. It's a cycle, and you'll go back to earlier steps often.
In simple words
Building a model is like cooking for guests. Decide the menu, buy ingredients, clean and chop, cook, taste, serve, and ask how it was. Most of the work is the prep. Beginners imagine that ML is all cooking, and then find that it's mostly chopping.
Overview
- A machine-learning project follows a cycle: define the problem, collect data, prepare it, train, evaluate, deploy and monitor.
- Most time goes into understanding and preparing data, not into choosing the algorithm.
- A simple baseline should always be built first.
- The test set is used once, at the end, for an honest estimate.
- Frameworks such as CRISP-DM (1996) describe the same stages for data projects.
- Models need monitoring after launch, because the world and the data change.
Why it matters
Many ML projects fail for reasons that have nothing to do with algorithms: the wrong question, bad data, or no plan for after launch. Following the workflow makes those failures visible early.
Key terms
- Problem framing
- Turning a goal into a precise ML task with defined inputs, outputs and success metric.
- Exploratory data analysis (EDA)
- Summarising and visualising data to understand it before modelling.
- Preprocessing
- Cleaning, encoding and scaling data so a model can use it.
- Model selection
- Comparing candidate models and settings on validation data.
- Deployment
- Putting a model into a product or service where it makes real predictions.
- Monitoring and drift
- Tracking performance after deployment, as the data changes over time.
Types
The workflow has several types of stage. Each type of work produces something the next stage needs.
1. Problem definition
Stating the goal, the prediction to make, how success is measured and the constraints.
- How it works: Talk to users, decide the target and the metric, and check that ML is the right tool at all.
- Example: "Predict next-day PM2.5 in Delhi within 20 µg/m³ to warn schools".
- When to use it: Do it first and write it down; unclear goals waste the rest of the project.
2. Data collection and preparation
Gathering, cleaning, labelling and transforming data into features.
- How it works: Handle missing values, duplicates and outliers, engineer features and split data into training, validation and test sets.
- Example: Joining pollution readings with weather data and adding a festival-day feature.
- When to use it: Spend most effort here; it usually matters more than the choice of model.
3. Modelling
Training a baseline and then better models.
- How it works: Start with a simple model, try stronger ones, tune hyperparameters with cross-validation and keep a log of experiments.
- Example: A persistence baseline (tomorrow equals today), then linear regression, then gradient boosting.
- When to use it: Use pipelines so that preprocessing is fitted only on training data.
4. Evaluation
Measuring performance honestly and checking errors, fairness and robustness.
- How it works: Compare models on validation data, analyse where they fail, test the final choice once on the test set and check performance for different groups.
- Example: Finding that a model is accurate in winter but misses sudden Diwali spikes.
- When to use it: Use it to decide whether the model is good enough to deploy and what to warn users about.
5. Deployment and monitoring
Putting the model into use and watching it over time.
- How it works: Serve predictions through an app or API, log inputs and outputs, track accuracy and drift, and retrain when performance falls.
- Example: A dashboard alerting the team when weekly prediction error rises above a threshold.
- When to use it: Plan it from the start, since a model is only useful once people can rely on it.
Explained in detail
The steps and their questions
| Step | Key question | Common pitfall |
|---|---|---|
| 1. Frame the problem | What decision will the prediction support, and how will we measure success? | Optimising a metric that does not match the real goal |
| 2. Get data | Do we have enough relevant, legal and representative data? | Data that does not represent real users |
| 3. Explore and prepare | What do the data look like, and what is wrong with them? | Skipping EDA and missing errors or leakage |
| 4. Train models | Which simple baseline and which stronger models should we try? | Jumping to complex models before a baseline |
| 5. Evaluate | How well does it work on unseen data, and for every group? | Testing on training data; reporting only accuracy |
| 6. Deploy and monitor | How will it be used, checked and updated? | Assuming a model stays accurate forever |
Why it is a cycle
Each step can send you back. Exploring the data may reveal that the labels are unreliable, so you return to data collection. Evaluation may show poor results for one region, so you gather more data from there. Monitoring after deployment may show performance falling as customer behaviour changes, a problem called data drift, so you retrain. Industry surveys repeatedly find that practitioners spend far more of their time on data collection, cleaning and preparation than on choosing or tuning models.
Data leakage
Leakage happens when information that would not be available at prediction time slips into training, making results look far better than they will be in reality. Examples include scaling features using statistics computed on the whole data set before splitting, including a column that is recorded only after the outcome, such as "treatment given" when predicting diagnosis, or putting near-duplicate records in both training and test sets. A model with suspiciously high accuracy should always trigger a search for leakage.
Start simple
- Build a trivial baseline, such as predicting the average or the majority class.
- Train one simple, interpretable model, such as linear or logistic regression.
- Only then try stronger models, and keep them only if they clearly beat the simple ones on validation data.
- Record every experiment: data version, features, model, settings and results.
How it works
- Define the problem. What decision will this change, and how will you measure success? Is ML even needed?
- Get the data. Check where it comes from, and whether you may use it.
- Explore it (EDA) with charts and summaries.
- Prepare it: clean, encode, scale and split.
- Set a baseline, the simplest possible answer, so you know what to beat.
- Train and compare models.
- Evaluate on the test set, once, at the end.
- Deploy it, usually behind an API.
- Monitor it, and retrain when it degrades.
In code
The whole workflow in a few lines, including the baseline and a final test.
from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0, stratify=y)
baseline = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_train, y_train)
print("Baseline accuracy:", round(baseline.score(X_test, y_test), 3))
print("Model accuracy:", round(model.score(X_test, y_test), 3))Output
Baseline accuracy: 0.632 Model accuracy: 0.982
Common mistakes
- Jumping to the model before understanding the problem and data.
- Reusing the test set while tuning. Then it's no longer a fair exam.
- Skipping the baseline, so you can't tell whether the model helps at all.
Check yourself
Summary
- Problem, data, explore, prepare, baseline, train, evaluate, deploy, monitor.
- It's a cycle, and the prep takes most of the time.
- Beat a baseline, and touch the test set only once.
Finished reading?
Sources: Google Machine Learning Crash Course