The RefutationMethod · The credibility engine
The Edge We Refused to Trade
Cross-sectional momentum in crypto passed every honest test we dragged it through. A perfect score and yet we're not going to trade it. This is the more useful side of quant research: what to do when the backtest gives a green light.
Most of what we publish here is a funeral. We take an idea people believe in, test it the way it deserves to be tested, and watch it die. The golden cross, MACD, overbought signals, session sweeps: all buried, with completed autopsies.
This one is different, and most people who grew up in country towns with rivers could teach you the same lesson in a couple of minutes. The lesson underneath it all actually protects your money: a strategy passing a rigorous backtest is only half of the job, and stress testing can reveal some damning evidence. But the story starts back before rail reached the regional towns of Australia, and long before sealed roads could move anything heavy overland. Coastal steamers, on the other hand, could head up and down the coasts, so the rivers became the highways to production. The furthest point upstream a loaded vessel could reliably reach was known as the 'head of navigation', and naturally, towns formed around these economic hubs.
The idea, in plain terms
Cross-sectional relative strength asks a different kind of question to most trading strategies. Instead of asking "is Bitcoin cheap?", it lines up the whole market at once and asks "which coins have been strongest lately?" Then it holds the leaders for a week, re-checks, and rotates. Back the frontrunners; let the laggards go. It is the oldest real anomaly in finance, and unlike the indicators we usually bury, it has decades of independent evidence behind it in traditional markets.
So we gave it an honest trial.
What an honest trial actually looks like
The towns sited at the 'head of navigation' had bustling economies, the "red gold", or cedarwood and floating the logs down the river to waiting vessels. Once the timber was gone, the floodplains from the cleared land turned out to be excellent farmland. Farmers grew cane on the Mary, maize and dairy on the Richmond. Fast forward to today, and the original purpose for the towns is long gone, but the towns stayed put. Pure path dependency. The only question is, how did all of these towns survive such an arduous transition to the different economies they have today? The answer is, most didn't. So this is the part worth learning, because it is where almost every retail backtest quietly cheats.
We rebuilt the cryptoverse through time and we raised the dead. The single biggest lie in crypto backtesting is survivorship bias: 'test the top 30 token charts', and you have secretly deleted every coin that used to be in the top 30, or even top 50 or top 100 for a while, and then faded away to be forgotten. We rebuilt a universe of the coins that were actually large at each point in time, including the ones that later collapsed and the ones you've probably never heard of. Terra's LUNA is in there, ranked as a giant right up to the week it fell from over a hundred US dollars to a fraction of a cent, and then delisted. Similarly, Wilcannia in western NSW was once one of Australia's busiest inland ports in the 1880s, now a struggling town of a few hundred. The list of towns just like it is long, but they're the ones that we don't see today, and they outnumber the survivors in almost any endurance endeavour.
We matched exposure. This is the trap that killed our earlier ideas. If your "edge" just holds more of a rising market, you have discovered leverage, not skill. So we never compared the strategy to cash. We compared "hold the strongest coins" against "hold the entire basket, equally weighted." Both are fully invested, all the time. Same market, same beta. The only difference is selection. Any win has to come from picking, not from being long.
We calibrated against background noise. If the basket returns 22%, is 26% really that impressive? So we ran the same machine a thousand times choosing coins at random, to measure how good a result pure luck produces. The strategy only counted if it beat the noise, not zero.
We refused to fit. One specification, chosen in advance from the published literature: a four-week formation window, weekly rebalancing, hold the top fifth. Written down before the test, and run through the lab once. No sweeping through settings to find the perfect fit (another mirage we've documented on therefutation.com).
It passed
It passed cleanly. Across eight distinct crypto eras, the strategy beat the equal-weight basket in six eras. Its realised market exposure came out at almost exactly 1.0, the same exposure as the basket itself, so this was genuinely selection and not a disguised leverage bet. It cleared the random-luck floor by more than three to one. We attacked the result four separate ways looking for the catch and it survived all four.
After this strategy survived the full lab treatment, you would expect us to fund it.
The edge was positive and broad, meaning it wouldn't be too sensitive to variation. Widen or shorten the formation window, hold more coins or fewer: the edge stays positive around an area. A robust result looks like this, a broad plateau showing an edge decaying as you move away, not a single lucky corner you would never have found in advance.
If we stopped here, this would be a very different kind of post, and a dishonest one.
Why is it sitting in the drawer?
Below is the same edge month-by-month, and this time it doesn't look as promising.
2024 is a bad year and 2018 is a bad year. The good years are enormous and the edge is real, but it arrives in lumps and it disappears, sometimes reverses, for a year at a time. You don't get to choose which year you are living in when you fund it.
And in terms of geographical settlement patterns, the perfect port sites had their drawdowns too. Flat land at the 'head of navigation' often meant you were establishing a town in the lowest point of a valley, the worst possible flood site. If you think of your portfolio drawdown as a flood level, you'd be wise to consider any recent flooding. And this is what backtests do: they say 'well going back 5 years, we were high and dry, and our drawdown was manageable', and we would bet that many of Lismore's residents thought the same before February 2022, when the flood record stood at a whopping 12.11 m, set in both 1954 and 1974. So when a 1-in-150 year, 14.4 m monster flood annihilated Lismore in February 2022, previously dry, once flood-free homes found themselves with heavy drawdowns to their physical assets. Even when people leave these towns, one rule stays with them: give the flood zone a bit of extra margin.
And now this brings us to cross-sectional relative strength's failings. When we took the strategy's own weekly returns and resampled them three thousand times to ask one question, 'in all the different timelines, how far do we draw down in each of them?', despite crypto's short history, we can simulate hundreds of years of data, and more than 1-in-20 blew the account up. The median loss was 90%.
That's not a tail we invented to be dramatic; it's the strategy's own behaviour, replayed. The selection edge does nothing to save you from it, because the strategy is long crypto the entire time, and crypto does what crypto does.
Figure 4 is the punchline in one picture. The cross-sectional relative strength (gold) finishes ahead of the basket (blue): the edge is right there, compounding. And the panel below shows the price of admission. To farm the edge you had to sit through drop after drop, the worst of them into the negative eighties, and hold your nerve through a losing year while the "edge" you were promised quietly ran backwards. The biggest issue then is that when you're down 80%, your wins are only 1/5th of the $$ size so it takes a lot of winning to get back to breakeven.
So the honest ledger reads: a real selection premium of roughly twenty-five percent a year, bolted onto a full-blooded crypto ride with ninety-percent drawdowns and whole years where the premium vanishes. For most people, and for us, that is not a trade. It is a way to blow up with slightly more steps than the next person.
The lesson worth keeping
"Does it work?" is the wrong question, or at least a badly incomplete one. This strategy works. It is not overfit, it is not survivorship, it is not disguised leverage. It passed a test most published strategies would fail.
And it is still not worth trading, because the question that actually decides whether you keep your money is not does the line go up. It is what does it cost you to hold, and can you actually hold it. A backtest reports the first and stays silent on the second. The second is where real portfolios die.
The craft is not finding a green backtest. Green backtests are cheap; we can manufacture one before lunch. The craft is knowing which green backtests to walk away from. This was one of them.
That is the whole job of this letter: not just whether an edge is real, but whether it is one you could actually hold. Most ideas we test fail outright; a rare few, like this one, pass and still are not worth the ride. The strategies that clear both bars, real and holdable, are the ones in The Survivors, for less than a dollar a day.
The Refutation publishes its own research honestly, including the ideas that survive. Nothing here is financial advice. It is a record of how we test, and how we decide. We hold no position in any strategy described above.