Data scientist interview questions

What they're checking

Whether you understand why models fail on new data, which underlies most modelling decisions.

A strong answer covers

1. Define bias as error from over-simple assumptions, which leads to underfitting.
2. Define variance as sensitivity to the particular training data, which leads to overfitting.
3. Explain that more complex models usually lower bias and raise variance, and the aim is the lowest error on unseen data.
4. Name how you manage it in practice, with cross-validation, regularisation, more data or ensembles.

What they're checking

Whether you validate models properly and know the standard fixes.

A strong answer covers

1. Give the signal: strong performance on training data and much weaker performance on validation data.
2. Explain how you check, with a held-out set or cross-validation, split by time if the data has a time order.
3. List the fixes: simplifying the model, regularisation, more data, fewer features or early stopping.
4. Mention a case where you found and fixed overfitting, and how much the validation score changed.

What they're checking

Whether you choose metrics from the cost of mistakes, rather than defaulting to accuracy.

A strong answer covers

1. Point out that accuracy misleads here, since predicting no positives at all scores 99%.
2. Use precision, recall or their balance in F1, and the precision-recall curve rather than ROC when positives are this rare.
3. Tie the choice to the business cost, such as missing fraud versus annoying a genuine customer.
4. Add that you would tune the decision threshold for that trade-off, not leave it at 0.5.

What they're checking

Your end-to-end judgement, from defining the target to using the model.

A strong answer covers

1. Define churn and the prediction window precisely, such as no purchase in the next 60 days.
2. Build features from data available before the prediction date only, such as usage trends, support contacts and billing history.
3. Start with a simple, explainable model, then try gradient boosting, validating on a split by time.
4. Explain how it would be used, such as which customers to contact, and how you would measure the effect of acting on it.

What they're checking

Whether your work reaches real use, and what you learned getting it there.

A strong answer covers

1. Describe the problem and why a model was the right tool.
2. Explain your key choices briefly: the data, the features that mattered and the model.
3. Cover deployment and monitoring, and how you worked with engineers to ship it.
4. Give the impact in numbers, and one thing you would do differently.