Market data
Q1. How do you check the quality of a market data feed or historical price table?
Q1. How do you check the quality of a market data feed or historical price table?
- Missing values and days: Compare against the trading calendar to find missing dates.
- Duplicates and ordering: Duplicate rows for the same timestamp and symbol, and out-of-order timestamps.
- Outliers: Prices or returns outside a plausible range, zero or negative prices, abnormal volume.
- Stale prices: A long run of unchanged prices may be a halt or a feed interruption.
- Consistency: The open-high-low-close relationships (high not below open or close) and differences between sources.
- Corporate actions: Price breaks caused by splits and dividends.
Q2. What are corporate actions and how do they affect backtests?
Q2. What are corporate actions and how do they affect backtests?
Q3. How do you align trades and quotes in time, and what is a point-in-time (as-of) join?
Q3. How do you align trades and quotes in time, and what is a point-in-time (as-of) join?
- Time zone and exchange clock differences
- Timestamp precision (milliseconds, microseconds) and source latency
- Never taking a quote from after the trade, which is look-ahead bias
Q4. How do you handle time zones and trading calendars across markets?
Q4. How do you handle time zones and trading calendars across markets?
Python and SQL
Q5. Why is vectorized code faster than a Python loop, and when can you not vectorize?
Q5. Why is vectorized code faster than a Python loop, and when can you not vectorize?
Q6. Why does Python's GIL matter for research code? How do you use multiple cores?
Q6. Why does Python's GIL matter for research code? How do you use multiple cores?
Q7. What SQL techniques do you use for time-series data? Explain window functions.
Q7. What SQL techniques do you use for time-series data? Explain window functions.
- Latest row per symbol (ranking or row numbers within a partition)
- Rolling averages, cumulative sums and differences (lag and lead)
- Ranking and percentiles within a group
Q8. How do you make a data pipeline idempotent and safe to backfill?
Q8. How do you make a data pipeline idempotent and safe to backfill?
- Partition by date and overwrite the whole partition on write, or use an upsert keyed on the primary key
- Make tasks depend only on explicit inputs (date parameter, versioned data), not on the current time
- Keep raw and transformed data separately so transforms can be re-run
- Limit backfill scope, write to staging and then swap, and record versions
Linux and Docker
Q9. A research job is running slowly on a Linux server. How do you investigate?
Q9. A research job is running slowly on a Linux server. How do you investigate?
- CPU and memory: Use top or htop to see whether CPU is saturated or the system is swapping.
- Disk I/O: Use a tool such as iostat to see read and write waits.
- Network: Check whether the job is waiting on a remote data source.
- The program itself: Use a profiler to find hot spots, and strace to see whether it is blocked in system calls.
- Resource limits: Check ulimit and cgroup limits (inside containers) for CPU or memory caps.
Q10. What is the difference between a Docker image and a container? Why use Docker for research?
Q10. What is the difference between a Docker image and a container? Why use Docker for research?
Q11. Batch versus streaming for market data: how do you choose?
Q11. Batch versus streaming for market data: how do you choose?
- Batch: End-of-day backtests, research and reports. Simple to build, easy to re-run and validate, with latency measured in minutes to hours.
- Streaming: Live quotes, risk, hedging and market making. Low latency, but you must handle out-of-order data, duplicates, disconnects and backpressure, and validation and replay are harder.

