Skip to main content
Status of this page: P0 draft. Design guidance, not a benchmark of specific libraries.

1. Definitions

  • Vectorized: compute signals, positions, and returns as array operations over the whole sample. Positions are a function of lagged signals; P&L = position(t-1) x return(t) minus costs.
  • Event-driven: replay a time-ordered stream of events (quotes, trades, order acks, timers). Strategy code reacts to each event and issues orders; a simulated exchange handles matching and fills.

2. Comparison

3. Choosing

Use vectorized for: cross-sectional alpha research, factor tests, daily rebalanced portfolios, and first-pass signal screening where costs are a function of turnover and spread. Use event-driven for: market making, execution algorithms, auction strategies, anything where order state, queue position, or latency affects P&L (queue position and fills). A common workflow: screen vectorized, then validate the survivors in an event-driven simulator with a realistic fill model, and compare the two P&L streams. A large gap indicates the vectorized cost model was too optimistic.

4. Common vectorized errors

  1. Signal uses same-bar data as the return it is traded against (off-by-one lag).
  2. Rebalance on close with a signal computed from the close.
  3. Costs as a constant bps rather than a function of size, spread, and volatility (market impact and TCA).
  4. Full-sample normalisation (mean, std, rank) leaks future information.
  5. Missing-data forward fill that carries stale prices into signals.

5. Common event-driven errors

  1. Timestamp semantics: exchange time vs receive time; using exchange time for decisions removes latency.
  2. Order-book reconstruction gaps lead to a wrong state that persists.
  3. Fill model too generous (tick simulation and fills).
  4. No feedback of own orders into the book, so impact is ignored.
  5. Time-zone, auction-phase, and holiday handling.

6. Testing the backtester itself

  • Replay a strategy with known P&L (for example a buy-and-hold) and check it matches.
  • Shuffle the time index of signals; the P&L should collapse. If not, there is leakage.
  • Delay the signal by one bar or by N milliseconds and measure the P&L decay curve.
For bias controls see Backtest bias taxonomy.

Points to verify

  • Specific library behaviours (bar timestamp conventions, default fill rules) should be checked in the version used.

Sources

  • Methodological synthesis; no single source. Bias controls referenced from the pages linked above.