An optimization usually ends with a ranked list. You give the optimizer a range for each number in the rules you weren't sure about, it backtests every combination, and the version at the top is the one you save. It has the best score, so it feels like an answer.
The usual worry is that the top version is overfit, tuned so closely to the dates it was tested on that it won't hold up on others. The usual fix is to test it on dates it wasn't picked on. That's worth doing, but it only tests the one version you picked. There's a question I care about more. If you'd run the same optimization a few months sooner, would it have picked the same version? If it wouldn't have, the version you saved depends as much on the dates you happened to use as on the rules.
We tried this on an opening range breakout on SPY, a strategy that buys or sells when price closes past the high or low of the first few minutes after the open. We let the optimizer choose two things, how long the opening range is and how far away the target sits, and then made it choose six times, each time on the 306 trading days before a stretch of about five months. Each pick then traded its stretch. The first pick was a 20-minute range with a target 2.5 times the range's height. Then it went to 20 minutes and 3x, back to 20 minutes and 2.5x, and then to 15 minutes and 2.5x for the last three. That last version is the one the optimizer offers you at the end, and on its five months, May 13 to Oct 2, 2026, it lost $4,770.50.
Each card is one period, with its dates, what it made or lost and the version it traded, so the moves from 20 minutes to 15 and from 2.5x to 3x and back are right there in order.
All six periods are public if you want to check them yourself.
I've saved the top version of an optimization more than once and watched it stop working within a few weeks.
Running this kind of test takes a few clicks, but reading it takes more care. It's tempting to save the last pick, when the more useful thing to look at is how the pick changed from one period to the next.
The rules leave two numbers to the optimizer
The rules are the usual ones. Mark the high and low of the opening range, the first 5 to 30 minutes after the 9:30 AM open. Go long on the first close above the range high, or short on the first close below the range low, one trade a day and no new entries after 3:00 PM. The stop is a close back through the other side of the range. The target is a close past the breakout by a set multiple of the range's height. Anything still open is closed at the end of the day, on the 3:55 PM bar.
Two of those numbers are somebody's choice: how many minutes the range covers and the target multiple. The usual version says 15 minutes and 2x. We let the optimizer pick both. The range could be 5 to 30 minutes in steps of 5, and the target 1 to 4 times the range in steps of 0.5, which makes 42 versions of the same strategy.
The optimizer ranks versions by return divided by maximum drawdown, which is the biggest drop the balance took from its peak. It ignores any version with fewer than 30 trades. Every trade put the whole balance in, and every result is after commissions and slippage. The bars are SPY 5-minute bars from Mar 28, 2023 to Oct 2, 2026.
The rules as written, 15 minutes and 2x, are public as a backtest over that whole stretch if you want to see the strategy on SPY before any optimizing.
Walk-forward asks the same question six times
Optimize over all three and a half years and the winner gets graded on the days it was picked on. Of course it looks good there. Those are the days that picked it.
Walk-forward splits history into a stretch to pick on and a stretch to trade. The optimizer picks the best version on the first stretch, trades it on the second, then slides both forward and does it again. Each trading period only ever gets a version picked before it started. Optimize in the Strategy Tester runs this walk-forward check by default on the Trader and Quant plans, and a saved result shows the same strip of periods.
Here the picking stretch was 306 trading days and each trading period was about 102. Six periods ran back to back from May 29, 2024 to Oct 2, 2026, and every pick took all 42 versions through a backtest, 252 in all.
Each pick's 306 days end where its trading period starts, and the next pick's 306 days are the same stretch moved forward by about 102 days. So two picks in a row share about 204 of their 306 days. Between one pick and the next, only a third of the history changes.
A third of the dates changing was enough to change the pick
The first period, May 29 to Oct 16, 2024, traded a 20-minute range with a 2.5x target and made $4,068.49. The second moved the target to 3x and made $4,897.64. The third went back to 2.5x and made $9,214.53, the best of the six.
Then the range changed to 15 minutes, with the target still at 2.5x, and stayed there. That version lost $862.34 from Aug 1 to Dec 18, 2025, made $7,094.94 from Dec 19, 2025 to May 12, 2026, and lost $4,770.50 from May 13 to Oct 2, 2026. Every period had 97 or 98 trades, so none of these results rests on a handful of trades.
Look at where the switch to 15 minutes happened. That pick was ranked on the 306 days before Aug 1, 2025. Those are the same days the three 20-minute picks had just traded, making $18,180.66 between them on dates they hadn't been picked on. Ranked on those days, a 15-minute range came out on top of all 42.
Swapping 102 of the 306 days was enough for a different version to rank first.
A pick that stays put isn't proof either
Once it landed on 15 minutes and 2.5x, the optimizer chose it three times in a row. That looks like the result you'd hope for. On those three periods it lost $862.34, made $7,094.94 and lost $4,770.50, which comes to $1,462.10.
The 20-minute picks made $18,180.66 on their three periods. That isn't evidence that a 20-minute range is better. They traded different months, and no version traded all six. It does show that picking the same version three times in a row didn't predict how the next five months would go.
It also matters how far the pick moved. Of 42 versions, it only ever chose three, and they sit close together: 15 or 20 minutes, 2.5x or 3x. The 2.5x target was in five of the six picks. The range length is the part it couldn't make up its mind about. If the picks had jumped from a 5-minute range to a 30-minute one and back, I wouldn't trust this strategy at all.
After 252 backtests, the optimizer finished one step from the rules we started with.
The six periods together are the number to judge
Add up the six periods and it made $19,642.76 over 585 trades, with four of the six in profit and a deepest drawdown of $7,459.99 along the way.
It's the only figure here that wasn't fitted to the dates it's measured on. It comes from the whole process rather than one version: optimize on the last 306 days, trade the winner for about five months, repeat. If you're going to trade an optimized strategy, that's the process you're signing up for, so it's the number I'd hold it to, and not the score of whatever came out on top last.
What I'd do with it
With a result like this, there are three things I'd consider.
The first is to trade the last pick, 15 minutes and 2.5x, which is what the result screen offers you. It was picked on the most recent days, which sounds like a reason to trust it, but it also makes it the version most fitted to those days, and it just lost $4,770.50 on the five months after them.
The second is to stop treating it as a choice between three versions and trade one from the patch the optimizer landed in every time, with a 2.5x target and a range of 15 or 20 minutes. You give up the claim that you found the best one, which after six tries was never much of a claim. If I were going to trade this strategy, that's what I'd do, and I'd judge it by the $19,642.76 from the six periods, not by any single version's score.
The third is not to trade it yet. In the Strategy Tester, Check on other symbols runs the final version over the same six periods on other symbols you add, none of which had any say in the pick. A breakout rule that only works on the one symbol it was optimized on has a problem that more SPY history won't fix.
The optimizer wasn't wrong about the last pick. On the 306 days it was picked on, a 15-minute range with a 2.5x target ranked first of 42. The ranking couldn't tell you that a little over a year earlier the same rules on the same symbol ranked a 20-minute range first, and that the version on top at the end lost $4,770.50 on the next five months. So before you trust an optimized version, run the optimization again on earlier dates and see whether it still comes out on top.