Glossary

Out of Sample

The data your strategy never saw while you built it, kept back to test if the edge is real.

Last updated: 2026-08-05

Out of sample is the slice of history you hide from yourself while building a strategy, then use once at the end to check it. The strategy gets tuned on the in sample data. The out of sample is the last exam, data it never touched, so a good result there is a lot closer to what live trading will feel like.

Why hold data back?

Because a backtest built and judged on the same data can't really fail. You'd keep tweaking until it looked perfect on those exact trades, which proves nothing. Split your history instead, say the first 70% to build on and the last 30% to test, and you get a clean read the tuning never touched. If the edge holds on the part you never optimised, it's far more likely to be real. If it dies there, you were fitting noise. Overfitting is exactly what this catches.

How much do you hold back?

Let's say you do a generic backtest with a strategy. Once you got the journal with a few hundred/thousands of trades, there's no perfect split, but 70/30 or 80/20 is common. Too little out of sample and the test is just a handful of trades, easy to pass on luck. Too much and you starved the build of data. Whatever you pick, lock it in. The moment you peek at the out of sample and go back to tweak, it stops being out of sample and quietly becomes part of the in sample. That's the one rule people break without noticing.

The catch with a single split.

One split gives you exactly one out of sample test, and one test can still get lucky. Your last 30% might have been an easy market. That's why walk forward exists, it rolls the split forward again and again so the edge has to prove itself across many windows, not just one. A single out of sample pass is the floor, not the finish line.

Frequently asked questions

It's the portion of your data a strategy was never built or optimised on, saved to test the edge honestly. A strong out of sample result is the best cheap sign a backtest might hold up live.
In sample is the data you build and tune on. Out of sample is the data you keep untouched and test on once. The in sample always looks good, that's the point of tuning. The out of sample is the number that actually counts.
Commonly 20% to 30% of your history, but it depends on how many trades that leaves. A 30% slice holding 12 trades tells you almost nothing.
It's a solid start but it can get lucky on a single stretch. Walk forward testing repeats the check across many windows. On your own trades, Quantprove runs that as a Consistency Check on the backtest and saves true out of sample for your live results in the Stability Score.

References

See your strategy’s edge change over time.

Upload a live trade log and watch your Health Score across the record. Free to start.

Start for freeHow it works

No credit card required·Swiss Made