Problem framing
Q1. Why is machine learning harder in finance than in image or text problems, and what does that imply for model choice?
Q1. Why is machine learning harder in finance than in image or text problems, and what does that imply for model choice?
- Low signal-to-noise: Most of the variation in returns is noise, so a useful model can have a very small predictive R squared. Complex models fit the noise easily.
- Non-stationarity: The relationships change with regimes, crowding and policy. A pattern from one period can mean something else later.
- Limited effective sample: Years of daily data are only a few thousand observations, and neighboring observations are strongly dependent.
- Feedback and costs: Your own trading affects prices, and costs can remove a small edge.
Q2. Design a model to predict short-horizon equity returns from a large, noisy feature set. Walk me through it.
Q2. Design a model to predict short-horizon equity returns from a large, noisy feature set. Walk me through it.
- Universe and data: point-in-time data, including delisted names, with timestamps showing when each value was known.
- Labels: define the target and the holding horizon (see Q4), with returns net of an estimated cost where possible.
- Features: economically motivated, made stationary or cross-sectionally normalized, with multicollinearity handled by grouping, regularization or dimension reduction.
- Validation: walk-forward or purged and embargoed cross-validation (Q3), with tuning kept inside the training folds.
- Evaluation: out-of-sample metrics that match the use (Q8), including realistic transaction costs and turnover.
- Monitoring: how you detect alpha decay and decide between a full retrain and an online update (Q9).
Validation and leakage
Q3. Why does k-fold cross-validation fail on financial data, and what do purging, embargo and combinatorial purged CV do?
Q3. Why does k-fold cross-validation fail on financial data, and what do purging, embargo and combinatorial purged CV do?
- Purging: remove from the training set any observation whose label window overlaps the time range of the test labels.
- Embargo: also drop a buffer of observations right after the test set, to remove leakage from serial correlation that the labels do not show.
- Combinatorial purged cross-validation (CPCV): split the data into N groups, choose k of them as the test set in every possible way, purge and embargo around each, and recombine the test predictions into several full backtest paths. With six groups and two test groups there are 15 splits and 5 paths, which gives a distribution of performance instead of a single path.
Q4. What is the triple-barrier method, and what is meta-labeling?
Q4. What is the triple-barrier method, and what is meta-labeling?
Q5. Prices are non-stationary but differencing removes the memory. What can you do about features?
Q5. Prices are non-stationary but differencing removes the memory. What can you do about features?
- Fractional differentiation: difference by a fraction between 0 and 1, and choose the smallest amount that passes a stationarity test, which keeps as much memory as possible.
- Normalize relative to context: z-scores over a rolling window, ratios to moving averages, or cross-sectional ranks at each date.
- Use economically stationary quantities: spreads, yields and valuation ratios instead of raw levels.
Models and features
Q6. Compare random forests and gradient boosting on a noisy, non-stationary return series.
Q6. Compare random forests and gradient boosting on a noisy, non-stationary return series.
- Random forest: averages many deep, decorrelated trees, which mainly reduces variance. It is fairly robust to noise and needs little tuning, but it can be too weak at capturing subtle structure.
- Gradient boosting: builds trees one after another, each correcting the previous errors, which mainly reduces bias. It is often stronger on rich data, but later rounds can fit the noise.
Q8. How do you evaluate a return-prediction model beyond accuracy?
Q8. How do you evaluate a return-prediction model beyond accuracy?
- Information coefficient: the rank correlation between predictions and subsequent returns, by date, with its stability over time and its decay across horizons.
- Portfolio results net of costs: Sharpe ratio, drawdown, turnover and capacity, with realistic spreads, fees, borrow and market impact.
- Hit rate and payoff asymmetry, because a high hit rate with small gains and rare large losses can still lose money.
- Stability across regimes, sectors and size buckets, not just in aggregate.
Production and risk
Q9. Performance is decaying live. How do you decide between a full retrain and an online update?
Q9. Performance is decaying live. How do you decide between a full retrain and an online update?
- Data or pipeline problem: a changed feed, a late field or a unit change. Fix this before touching the model.
- Distribution shift: compare live feature distributions with the training period (for example a population stability measure) and check whether the signal correlations have changed.
- Alpha decay: the edge was real but has eroded through crowding or adaptation.
Q10. When would you use deep learning in finance, and what are the risks?
Q10. When would you use deep learning in finance, and what are the risks?
Q11. Your backtest looks great. Why do you not trust it?
Q11. Your backtest looks great. Why do you not trust it?
- Leakage: through data, labels, features or tuning. Were splits purged, and was normalization fitted only on the training window?
- Number of trials: how many variants were tried? The best of many is biased upward, so adjust for the number of trials (for example with a deflated Sharpe ratio) and estimate the probability of backtest overfitting.
- Costs and capacity: realistic spreads, impact, borrow and turnover.
- Sample issues: survivorship, look-ahead in the universe and a period dominated by one regime.
- Economic story: is there a reason the edge should exist and persist?
Q12. What does model risk management look like for an ML trading model?
Q12. What does model risk management look like for an ML trading model?
- Explainability: be able to say what drives the signal, using methods such as feature attribution, and check that the drivers make sense.
- Stability: test sensitivity to small input changes, retraining dates and random seeds.
- Monitoring: input distributions, prediction distribution, live versus backtest performance and slippage, with alerts.
- Controls: position and loss limits, a fallback model or rule, and a kill switch that does not depend on the model.
- Documentation and review: data lineage, assumptions, validation results and independent review.
Related pages
- Statistical models
- Regression and backtesting
- Data engineering and systems
- Mock Interview (Quantitative Research)
Sources
- Machine learning quant interview guide, QuantVault
- Machine learning interview questions, QuantVault
- Bayesian ML, overfitting and error metrics, QuantVault
- Purged cross-validation, Wikipedia
- The 7 reasons most machine learning funds fail, Quantpedia summary of de Prado
- Cross-validation for financial data: purging and embargoing, TradingInterview
- Backtesting interview questions, TraderMath
- Quant research interview questions: 25 real examples, Quantt
- Two Sigma quant researcher interview, MyntBit
- Two Sigma interview report, Glassdoor candidate report
- Random forest versus GBDT trade-offs, Dataford

