Backtesting, Biases, and the Bilbo Box Breakout

A plain-English walkthrough of the strategy I'm forward testing right now, followed by what I've learned about backtesting: why it matters, the ways it lies to you, and how I use AI tools to catch the lies. Updated 2026-07-14.


Who am I

I'm a former investment professional who now owns a business related to milk products. My friends jokingly call me "the milk man", so that became my handle. I don't have anything to sell you. I simply enjoy leveraging AI tools and sharing things that can help others. My past life left me with the institutional habit of demanding evidence before risking money, and today I have the freedom to publish what I find, including the failures, because the only compliance department that has to approve it is my own.

Just as important is what I am not: I am not a financial advisor, and nothing on this page or anywhere on my site (https://milkmantrades.com) is financial advice. This is research I did for my own trading, published with its receipts. Please do your own work before risking a dollar.

The Bilbo Box Breakout, in plain English

The concept was created by Saty Mahajan (satyland.com), and what follows is my specific adaptation of it. Much of what I trade and test is built on Saty's work, and I'm grateful for his mentorship. I highly recommend watching his free YouTube videos and learning to use his indicators. For this strategy, everything you need is: price, volume, the 21 exponential moving average (EMA) on the daily chart, and the free Saty Phase Oscillator indicator on the hourly chart, available on platforms like TradingView and ThinkOrSwim at no charge, courtesy of Saty.

The idea is simple. Markets alternate between quiet coiling and sharp expansion. The Phase Oscillator tells you when a stock has gone quiet: it paints the candles grey, a state Saty calls compression. Draw a box around those quiet candles. When price finally breaks out of the box with some force, the odds favor a real move getting started, and a call option is a cheap, defined-risk way to ride it.

The rules

  1. Watch about 20 liquid large-cap names (NVDA, TSLA, AMZN, META, AMD and the like) on the hourly chart.
  2. Wait for compression. When the Phase Oscillator paints grey candles, the box is the high and low of the first 5 grey candles. After the 5th, the box is frozen.
  3. Wait for a confirmed break. The signal is an hourly candle that closes both out of compression and above the box high, between 10:00 and 15:00 ET. A poke that closes back inside the box is not a signal. Long side only; shorting the downside breaks tested out at zero to negative.
  4. Check three gates, and skip the trade unless all pass: the breakout candle's volume is at least normal for that hour of the day; the stock is above its daily 21 EMA (a simple trend filter: no chasing breakouts in a downtrend); and the option you're about to buy has a tight two-sided market (bid-ask spread of 5% of mid or less).
  5. Buy one call, about one strike out of the money, expiring about 28 days out. Premium sized at 3 to 5% of the account. No adds, no averaging down.
  6. Exit on the stock's price, never on the option's price. A 5-minute close below the box low means you're out. Once the stock has moved a full daily ATR in your favor, trail it loosely (I give back up to 75% of the open gain, because that looseness is what keeps you in the rare monster runs). Whatever is still open after 10 trading days gets sold. Every premium-based stop, profit target, and "cut losers on day one" variant I tested made the results worse.

A real trade example: META

This is an actual trade from my forward-testing paper account, not a hand-picked historical chart. On the morning of Thursday July 9, 2026, META's hourly candles went grey (Phase Oscillator compression). The box is the range of the first 5 grey candles: 577.07 to 610.84. Nine grey candles later, the 2pm ET hourly candle closed at 615.61, out of compression and above the box high. All three gates passed, so that close was the signal.

META · hourly candles · Jul 7 to Jul 13, 2026 · grey = compression · shaded hours = premarket/after-hours

The trade: one August 7 $630 call (about 28 days out, roughly one strike above the 0.75 ATR target), filled at $31.18 just after the 3pm signal close. META kept climbing on Friday, gapped up Monday morning, and the retrace rule fired at 10:01am Monday: sold at $58.48, up 87.6% on the premium ($2,728 on one contract after fees). Here is the option itself, tick by tick, over those three sessions:

META Aug 7 $630 call · 5-minute mid price · the three sessions spanning the trade

To be clear about what this example is: one recent winner, shown because it makes the mechanics easy to see. The median trade loses. Winners like this one are what the losers pay for.

The results

Across 1,031 trades on 20 names from January 2019 through July 2026, priced against real historical option quotes, the strategy averaged +11.9% of premium per trade (day-clustered t = 3.56). Both halves of the record were independently profitable: +9.9% per trade in 2019 to 2022, +13.2% in 2023 to 2026.

Compounding those trades one after another from a $10,000 account at 4% of the account per trade grows it to roughly $660,000 over the 7.5 years, with a worst peak-to-trough drawdown of 43%. Log scale below; hover for the trade-by-trade path.

Equity curve · $10,000 start · 4% per trade · log scale

And a Monte Carlo view of what one forward year (about 150 trades) might look like: each simulated path resamples whole trading days from the record. The median path roughly 1.8x's the account; about 1 path in 12 ends the year down.

Monte Carlo · 10,000 paths · 150 trades · $10,000 start

Before you get excited, know what a typical trade feels like. The win rate is only 41% and the median trade loses 6.7% of premium. A small number of large winners pays for long strings of small losers. Also, while the exit rule passed a genuine out-of-sample test (chosen on 2019 to 2022 data, tested untouched on 2023 to 2026), the strike and expiration choice was made looking at the whole record, so treat the exact per-trade figure as somewhat flattered. And this is long-only momentum in megacaps: expect it to bleed in bear markets. 2022 was the worst stretch.

The full study, with every rule, chart, caveat, and the Monte Carlo detail, lives here: Bilbo Box Options, the study page. The broader Saty system it sits inside is written up in the Trading Manual.

Why I like this strategy

What happens next: forward testing

The strategy did great on a historical dataset. That is a very different thing from surviving contact with reality. Fills, slippage, halted names, fat-fingered entries, my own psychology on the fifth loser in a row: none of that lives in a simulation.

So before I allocate real capital, I'm forward testing it on a paper account for 3 months. The signals come from a live scanner running the exact rules above, and every paper trade is logged publicly at /bilbo-paper.html as it happens. If the forward record looks nothing like the backtest, that tells me something no amount of historical simulation could.

I share regular updates on this strategy on my X account: x.com/MrMilkTrading. Follow along there if you want to watch it play out in real time, warts and all.


What a backtest is, and why the biases matter more than the results

A backtest is a simulation: you write a rule down precisely enough that a computer can follow it with no judgment calls, then replay history and record what the rule would have done. The output is not a prediction. It answers a narrower question: if the future resembles this sample of the past, what should I expect from this rule?

Before a backtest, a trading idea is a story ("compression leads to expansion"). After one, it's a measurable claim with a sample size attached ("across 1,031 breakouts on 20 stocks over 7.5 years, buying the break paid +11.9% of premium per trade"). Stories can't be wrong. Claims can, and that's what makes them useful.

Why bother

  1. It kills bad ideas cheaply. Most ideas don't survive contact with data. Finding that out in an afternoon of simulation costs nothing; finding it out with real money costs the tuition.
  2. It sizes the good ones. Knowing a rule wins 41% of the time with a losing median trade changes how you size and how you emotionally prepare. One of my simulations showed the same trade stream compounding beautifully at 4% of the account and ending in ruin at 20%.
  3. It forces precision. "Buy the breakout" is not testable. "Buy the call within 5 minutes of an hourly close above the box high between 10:00 and 15:00 ET" is. Making a rule testable usually reveals you had three different strategies in your head under one name.

The catch: a backtest is easiest to cheat by accident

Every backtest is a claim about the past made by someone who already knows the past. Every bias below is a variation of that problem. A biased backtest is worse than no backtest, because it converts a hunch into false confidence with a t-statistic attached.

1. Lookahead bias is using future information to make decisions in the present. For example: select all the days where SPX ended down 1%, then run statistics on the best entry point for a short; the numbers will look very convincing, but when the setup shows up in the future you won't know whether SPX will end the day down 1% or not, so the information is essentially useless.

2. Survivorship bias is testing on the stocks that are big and alive today, which means testing on history's winners. My ticker list (NVDA, TSLA, AMD...) is exactly that, so the results are somewhat flattered by hindsight.

3. Overfitting and data mining. Test enough variants and something will look great by pure chance. The defense is discipline, not cleverness: decide the tests in advance and treat any positive result as noise until it repeats on data it has never seen.

These three are just the start. Backtesting bias is a much deeper topic with many more potential pitfalls (fill assumptions, missing baselines, cluster effects, and plenty of others), and I'll dig into more of them in future writeups.

The common thread: the goal of a backtest is not to produce a big number, it's to survive attempts to kill the number. Year splits, held-out tickers, worst-case fills, cluster-robust statistics, adversarial code review. A result that survives all of that has earned some trust. A result that hasn't is a screenshot.

Using AI tools

I use AI heavily, and I want to be transparent about exactly how. My two current favorites are Claude Code (Fable 5) and Codex (5.6 sol, at maximum reasoning effort). Both can write efficient scripts that crunch enormous amounts of data, and both can produce sophisticated, institutional-grade analysis: the kind of pipeline work that would have taken a quant team weeks now takes an evening.

But (no offense to either of them) they are just tools, and what separates great output from slop is human judgement. I start every backtest study with curiosity, asking the right questions of the right data. I end every study with skepticism and rigor, and I constantly challenge my AI models to double-check their work.

My favorite method for doing this is adversarial review: I will literally have Codex audit Claude Code's work, and vice versa, with instructions to hunt for bugs and biases rather than to be agreeable. I estimate that at least 90% of the time, I discover a major flaw or hidden bug this way. The expiry-anchoring lookahead bug above? Caught by exactly this process, before the headline number was ever computed. If you take one workflow idea from this page, take that one: never let the model that wrote the analysis be the only one that checks it.

Milkman Trades · research, not financial advice. · the Bilbo Box options study · live paper record · @MrMilkTrading on X