Machine Learning Engineer Interview Questions

Reproducible training (versioned data + code + params), an offline eval that mirrors the production objective, then packaging the model behind a serving layer with the same feature transforms used in training to avoid train/serve skew. Add monitoring for drift and performance, a rollback path, and a shadow or canary deploy before full traffic. The notebook is maybe 20% of the work; the pipeline and monitoring are the rest.

The root cause is usually features computed differently in training vs serving. A feature store with shared transformation logic fixes most of it, plus logging serving features and comparing their distribution to training. Building a feature store is what took our iteration cycle from 2 weeks to 2 days and killed a class of silent accuracy bugs.

Common causes: train/serve skew, distribution shift since training, a leak that inflated offline metrics, or an objective mismatch — the offline metric isn't what the business cares about. I'd check for leakage first, compare feature distributions, and confirm the offline metric correlates with the online KPI. The fix is often an online A/B, not more offline tuning.

Start with the simplest thing that could work — often gradient boosting or logistic regression — as a baseline. Go deep only when the data volume, the signal (images, text, sequences), and the marginal lift justify the added cost and ops burden. A well-tuned simple model in production beats a fancy one that's hard to serve and monitor.

Track input drift (feature distributions), prediction drift, and — where labels arrive — live performance against the training baseline, with alerts on burn thresholds. I separate data-quality alerts from performance alerts. My monitoring stack caught a 9-point AUC degradation before it hit revenue, which is the whole point of monitoring.

Profile to find the bottleneck, then apply the right lever: quantization, distillation, batching, caching frequent inputs, and an efficient serving runtime (ONNX/TensorRT). For cost, spot GPUs with checkpointing and right-sizing the pipeline. I cut latency 380ms → 45ms and training cost 44% using these — measured, one lever at a time.

S: A churn model with strong AUC didn't move retention. T: Understand why. A: The predictions were accurate but un-actionable — they fired too late and didn't tell the team what to do. R: I learned to design from the decision backward: the model has to produce a timely, actionable output the business can act on, or accuracy is irrelevant. The relaunch focused on lead-time and recommended actions.

Define the task narrowly, evaluate on a real labeled set, and design a fallback for low-confidence cases rather than letting the model act unchecked. I deployed a support classifier that auto-resolved 31% of tickets but kept a measured human-review fallback for the rest — you ship the automatable slice with a safety net, not the whole thing blind.