Definition
A family wants to know whether ₹62 lakh is a fair price for a flat. Could a model trained on past sales give them a sensible estimate, and tell them how uncertain it is?
This project builds an end-to-end regression model that predicts house prices from features such as area, bedrooms, age, floor and locality. It covers exploring and cleaning the data, preparing features, establishing a baseline, comparing models with cross-validation, evaluating with MAE and RMSE on a test set, and interpreting and communicating the results responsibly.
In simple words
You will do what a property valuer does, but with data: look at many past sales, learn how size, age and location affect price, and use that to estimate a new flat's price, with an honest idea of how far off the estimate might be.
Overview
- The project predicts house prices from features such as area, rooms, location and age.
- It practises the whole workflow: explore, clean, engineer features, train, evaluate and explain.
- Prices are skewed, so a log transform of the target often helps.
- Models are compared with a baseline such as the median price.
- MAE in rupees is easy to explain to users; R² shows how much variation is explained.
- Fairness matters: location features can encode historical inequality.
Why it matters
Regression projects like this appear everywhere: forecasting sales, estimating delivery times, predicting crop yields. Working through one from start to finish teaches the full workflow and the judgement that no single algorithm provides.
Key terms
- End-to-end project
- A project covering every step from raw data to a communicated result.
- Target transformation
- Modelling log(price) instead of price, to handle skew.
- Feature engineering
- Creating useful features, such as price per square foot or age from build year.
- Feature importance
- How much each feature contributes to a model's predictions.
- Prediction interval
- A range likely to contain the true value, expressing uncertainty.
- Model card
- A short document describing a model's purpose, data, performance and limits.
Types
A house-price project can use several types of model. Comparing them is part of the project.
1. Baseline models
Simple predictions such as the overall median price or the median per locality.
- How it works: They need no learning, and every real model must beat them on validation data.
- Example: Predicting ₹65 lakh for every flat in a city where that is the median price.
- When to use it: Use them first, so you know how much the real models actually add.
2. Linear and regularised regression
Price as a weighted sum of features, possibly with ridge or lasso penalties.
- How it works: Scaled numeric features and one-hot locations feed a linear model; coefficients show the price effect of each feature.
- Example: Each extra 100 square feet adding about ₹4 lakh in one locality.
- When to use it: Use them for an explainable model and as a strong second baseline.
3. Tree ensembles
Random forests and gradient boosting.
- How it works: They capture non-linear effects and interactions, such as area mattering more in prime locations, without scaling.
- Example: Gradient boosting reducing MAE noticeably compared with linear regression.
- When to use it: Use them for the best accuracy on tabular property data. Check their predictions for very expensive houses, which may be rare in the training data and harder to predict.
4. Nearest-neighbour (comparable sales) models
Predicting from the prices of the most similar recently sold houses.
- How it works: k-NN regression on scaled features mirrors how property valuers use "comparables".
- Example: Averaging the five most similar 2-BHK flats sold in the same area last year.
- When to use it: Use them to explain predictions with real examples users can check.
5. Explaining and checking the final model
Tools that show which features drive predictions and where the model fails.
- How it works: Permutation importance, partial dependence plots and residual analysis by price band and locality reveal strengths and blind spots.
- Example: Finding the model under-predicts luxury flats and needs a separate check for them.
- When to use it: Use them before presenting the model, and state its limits clearly.
Explained in detail
Project plan
| Stage | What to do | Watch out for |
|---|---|---|
| Explore | Histograms of price and area; price against area; counts per locality | Price is right-skewed; a few luxury homes dominate |
| Clean | Fix impossible values, handle missing data, remove duplicates | Area of 15 square feet or 150,000 are errors |
| Prepare | Scale numeric features, one-hot encode locality, consider log(price) | Fit preprocessing on training data only |
| Baseline | Predict the median price for every house | Every model must beat this |
| Model | Linear regression, then a random forest or gradient boosting | Compare with the same cross-validation folds |
| Evaluate | MAE and RMSE in rupees on the test set; residual plots; errors by locality | Large errors for expensive homes |
| Communicate | A model card: purpose, data, accuracy, limits | Do not present an estimate as a fact |
Why log(price)?
House prices are strongly right-skewed: most flats cost tens of lakhs, a few cost many crores. A model trained on raw prices is dominated by those few expensive homes, because their squared errors are enormous. Modelling log(price) compresses the scale so that a 10 per cent error counts similarly for a ₹40 lakh flat and a ₹4 crore villa, and relationships with area often become more linear. After predicting, convert back with the exponential. Many published house-price models use this transformation.
Interpreting and communicating
- Linear regression coefficients show roughly how much each feature adds; forests give feature importances, which show influence but not direction.
- Report error in rupees: "typically within ₹4 lakh" is clearer than an R² of 0.82.
- Check errors by locality and price range; a model that is accurate on average may be poor for some areas.
- Give a range, not a single number, for example the predicted price ± the typical error.
- State limits: the model knows only past sales in the data, not a new metro line or a leaking roof.
Automated valuations affect real decisions about loans and purchases, so responsible use matters. In 2021 the US property company Zillow closed its home-buying business after its pricing models overestimated many homes, leading to large losses. Models are tools for informed judgement, not substitutes for it.
Keep a short project log as you go: what you tried, what changed and the scores.
How it works
- Load and explore the data; write down three observations.
- Clean errors and duplicates, then split off a test set.
- Build a pipeline and score a median baseline and two models with cross-validation.
- Evaluate the best model once on the test set in rupees.
- Write a short model card with the results and limits.
In code
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
rng = np.random.default_rng(7)
n = 600
locality = rng.choice(["central", "suburb", "outskirts"], n, p=[0.3, 0.5, 0.2])
area = rng.normal(1100, 300, n).clip(400, 2500)
age = rng.integers(0, 30, n)
rate = pd.Series(locality).map({"central": 9000, "suburb": 6000, "outskirts": 4000}).to_numpy()
price = (area * rate * (1 - 0.006 * age) * rng.lognormal(0, 0.1, n)) / 1e5 # in lakh
df = pd.DataFrame({"area": area.round(), "age": age, "locality": locality, "price": price.round(1)})
X, y = df.drop(columns="price"), np.log(df["price"])
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=0)
prep = ColumnTransformer([("num", StandardScaler(), ["area", "age"]),
("cat", OneHotEncoder(handle_unknown="ignore"), ["locality"])])
true_lakh = np.exp(y_te)
print(f"Median baseline MAE: {mean_absolute_error(true_lakh, np.full(len(y_te), np.exp(y_tr.median()))):.1f} lakh")
for name, model in [("linear", LinearRegression()), ("forest", RandomForestRegressor(300, random_state=0))]:
pipe = Pipeline([("prep", prep), ("model", model)]).fit(X_tr, y_tr)
pred_lakh = np.exp(pipe.predict(X_te))
print(f"{name:>7} MAE: {mean_absolute_error(true_lakh, pred_lakh):.1f} lakh")
flat = pd.DataFrame({"area": [950], "age": [8], "locality": ["suburb"]})
best = Pipeline([("prep", prep), ("model", LinearRegression())]).fit(X_tr, y_tr)
print(f"Estimate for a 950 sq ft, 8-year-old suburb flat: {np.exp(best.predict(flat))[0]:.1f} lakh")Output
Median baseline MAE: 20.9 lakh linear MAE: 5.4 lakh forest MAE: 5.6 lakh Estimate for a 950 sq ft, 8-year-old suburb flat: 52.5 lakh
Try it yourself
Guided project in the browser
Work through the checklist: train, test, find failures, improve the data and test again. Every result is recorded.
A fair price for the flat?
The Iyer family is offered a 950 sq ft, 8-year-old suburban flat for ₹62 lakh.
Common mistakes
- Skipping the baseline, so you cannot tell if the model helps.
- Reporting only R², with no error in rupees.
- Presenting a single predicted price as certain.
Check yourself
Summary
- An end-to-end project explores, cleans, prepares, models, evaluates and communicates.
- Use a baseline, log-transform skewed targets and compare models fairly.
- Report errors in real units with a range, and state the model's limits.
Finished reading?
Sources: scikit-learn User Guide · An Introduction to Statistical Learning (James, Witten, Hastie, Tibshirani)