Quant research · model validation
Validating option models on live BTC data
Banks don't trust a pricing model until an independent team has tried to break it. I built that validation for Bitcoin options on Deribit. It fits arbitrage-free volatility surfaces, calibrates six models, rebuilds a year of option-chain history from free trade data, and tests which delta actually hedges. It updates itself every day and writes its own validation report.
- 0.13 SVI fit error, vol pts · 99% inside bid-ask
- −15% Heston hedge vs Black-Scholes
- +0.60 spot-vol correlation in Aug · −0.36 in Feb
- 23 / 23 tests vs published references
The question
A trading desk marks its options with a volatility model and hedges them with that model’s delta. If the model is wrong, both the marks and the hedges are wrong. That is why banks have model-validation teams whose job is to break models before they are used. I wanted to do that job properly, on real data anyone can check: every listed Bitcoin option on Deribit, pulled from its free public API.
The project answers three questions a validator would ask. Does the model fit the market without arbitrage? Are its parameters believable and stable? And does its delta actually hedge better than the simplest one?
1 — Fit: an arbitrage-free surface
Each day about 900 BTC options are listed across 11 expiries, from 2 days to a year. I fit raw SVI to each expiry using the quasi-explicit method, with Lee’s moment bound and the Durrleman butterfly condition, and check calendar arbitrage between expiries. As a benchmark I also fit SSVI, which is arbitrage-free by construction.
Errors are measured against the bid-ask mid. SVI prices 99% of quotes inside the bid-ask spread and shows no butterfly or calendar arbitrage.
SSVI’s guarantee has a cost. It uses one skew parameter for all maturities, so it misses BTC’s steep short-dated put skew by 1 to 2 vol points.
2 — Calibrate: stable is not the same as correct
Heston is priced with the COS method, which matches the published Fang & Oosterlee reference price to 1e-6. On BTC its best fit needs mean reversion κ of 57 to 67 and vol of vol around 900%: variance would halve every 4 to 5 days. Even then, it misprices one-week options by several vol points.
The obvious objection is noisy data. So I redrew every quote at random inside its bid-ask spread and refitted 20 to 30 times. The parameters moved by only 1–2%. The data pins them down tightly, so the misfit is model risk, not estimation noise: pure diffusion cannot produce steep short-dated crypto smiles. Over a year of daily calibrations, κ sat on the optimiser’s upper bound on 15% of days.
The textbook fix is jumps, so I added Bates (Heston plus lognormal jumps) and checked it against an independent Merton jump-diffusion price. It cuts the error from 1.45 to 0.60 vol points, but only by pricing a rare, very large crash: about one jump every 12 years, averaging −22%. The jump size, rate and dispersion lie on a ridge (correlation above 0.9 when refitted), so only their combination is pinned down. The objective also has a second basin with frequent small jumps, and from day to day the calibration switches between the two. Jumps fix the fit, not the model.
3 — The hard part: history that doesn’t exist
Testing a hedge needs a year of daily option chains, and there is no free source. Deribit’s trade history does carry each trade’s mark price, its implied vol and the index price. Its IV is computed on the expiry’s forward, so each trade gives one equation with one unknown and a unique root. The median over near-the-money trades gives the forward per expiry. Checked against live forwards, the error is under 0.5 basis points. From that I rebuilt 365 days of chain states.
4 — Hedge: which delta actually works?
For every out-of-the-money option seen on two consecutive days, the hedge error is dV − Δ·dF. I compare seven deltas using Hull & White’s gain, 1 − SSE(model)/SSE(BS). The empirical deltas are fitted only on earlier months, and the intervals come from a bootstrap over whole days.
95% intervals: sticky moneyness −24.7 to −4.4%, Heston −27.8 to −6.7%, adaptive −0.7 to +15.4%. Bartlett and min-variance intervals span zero.
The plain sticky-strike delta is hard to beat, and adding jumps changes nothing: Bates hedges exactly as badly as Heston. Sticky moneyness and Heston both assume IV rises when BTC rises; over the year it fell slightly. The deeper reason is that the spot-vol relationship keeps flipping:
Every fixed smile assumption wins in one regime and loses in the other. Bartlett and min-variance gain up to 39% in negative months and lose up to 52% in positive ones.
Only adapting helps, and only a little. Re-estimating the spot-vol slope from the last 30 days is the best delta at +8.5%, but its interval touches zero and the gain comes from a few large-move days. In the report it is a challenger to monitor, not a model to approve.
What went wrong, and how I caught it
The first backtest reported a sticky-moneyness gain of −2,250,000%. That isn’t a finding, it’s a bug. On 2-day expiries the SVI optimiser started from points sized for one-year smiles, wandered off, and produced negative total variance. The smile slope, and with it the delta, then exploded to 7,757. I scaled the starting points to each slice, kept total variance non-negative, added regression tests and reran the whole year.
A second check mattered as much. The worst 1% of option-days carry 45% of the squared hedging error, so a single SSE ranking can turn on a few crash days. The report therefore shows vega-scaled errors and the share of days each delta wins, next to the headline numbers.
The verdict
| Model | Use | Decision |
|---|---|---|
| SVI | Marking the daily smile | Approved, with arbitrage checks on every fit |
| SABR (β = 1) | Marking | Approved; Bartlett delta not approved as default hedge |
| SSVI | Arbitrage-free benchmark | Approved; not for short-dated marking |
| Heston | Pricing and hedging | Not approved |
| Bates (Heston + jumps) | Pricing and hedging | Not approved; jump risk identified only as a combination |
| Black-Scholes, sticky strike | Default delta | Approved |
| Adaptive spot-vol delta | Challenger | Monitor for six more months |
The report is generated from the data every morning, in the structure a bank uses: scope, data, conceptual soundness, implementation tests, outcome analysis, stability, a findings register and a decision per model. It also re-runs the 23 tests, which check against published references such as Fang & Oosterlee’s Heston price and Gatheral & Jacquier’s arbitrage example and an independent Merton jump-diffusion price.
A model can be precisely calibrated and still wrong. The validation’s job is to tell the two apart, and then say plainly what each model may and may not be used for.