← THE CHARTSPOD LEDGEROPEN TERMINAL →
BACKTESTING · OPTIMISATION

Parameter Optimization: Report All Trials, Not Just the Winning Curve

If you test 500 parameter combinations, the best one will look great by chance. Reporting only that one hides how many tries it took.

By ChartsPod Editorial Team · Reviewed by ChartsPod Research
Published · Updated · 5 min read

Short answer

When optimising parameters, report every trial: the number of combinations tested, the distribution of results and where the chosen setting sits within it. The best of many trials is biased upward by chance (the more you try, the better the best looks). Prefer parameter regions where neighbouring settings also perform well (a plateau), confirm on out-of-sample data, and discount the in-sample result by the number of trials.

Why the best trial misleads

Even with no real edge, testing many variations produces some that look profitable. Illustrative: 500 random strategies with zero true edge, each with 100 trades: the best few will show strong returns purely from noise.

Illustrative: share of 500 no-edge trials above thresholds (%)Illustrative: share of 500 no-edge trials above thresholds (%): Return > 0 50 %; Return > 10% 15 %; Return > 25% 2 %.ILLUSTRATIVE: SHARE OF 500 NO-EDGE TRIALS ABOVE THRESHOLDS (%)Return > 050 %Return > 10%15 %Return > 25%2 %
Ten 'great' strategies out of 500 can be pure luck. Original ChartsPod editorial graphic.

What to report

ItemWhy
Number of trialsScales the luck factor
Distribution of resultsShows typical vs best
Chosen parameters' neighboursPlateau vs spike
Out-of-sample resultGeneralisation
Costs includedComparability
Plateau vs spikePlateau vs spike. Left, Spike: Best setting far above neighbours; Small change ruins result; Likely fitted to noise. Right, Plateau: Many nearby settings work; Robust to small changes; More likely real.SPIKEBest setting far above neighboursSmall change ruins resultLikely fitted to noisePLATEAUMany nearby settings workRobust to small changesMore likely real
Prefer plateaus. Original ChartsPod editorial graphic.

Process

Honest optimisationHonest optimisation: Fix parameter ranges before testing; Log every trial; Choose from a plateau, not the peak; Test choice out-of-sample; Report trial count with results.HONEST OPTIMISATIONFix parameter ranges before testingLog every trialChoose from a plateau, not the peakTest choice out-of-sampleReport trial count with results
Transparency is the defence against self-deception. Original ChartsPod editorial graphic.

Related: Walk-Forward Crypto Testing: Lock the Rule Before the Test Window, A Profitable Crypto Backtest With 20 Trades: Is the Sample Useful?.

Example of a trial log

A trial log lists: date, parameter set, period, costs, trades, return, max drawdown and notes. After optimising a moving-average crossover across 120 combinations, the log shows a median return of +2%, a best of +31% at fast=7/slow=41, and a broad region between fast 10–20 and slow 50–80 earning 8–12%. Choosing from that region and testing out of sample is more defensible than taking the peak.

Conclusion

Report all trials and their distribution, choose robust plateaus and confirm out of sample.

Keep a log of every backtest variation you run.

Open the ChartsPod terminal

Check Parameter Optimization against a live market before relying on it.

Try it in ChartsPod

Questions readers ask

Why report all trials?

The best of many trials looks good by chance; the count shows how much.

What is a parameter plateau?

A region where neighbouring settings perform similarly well.

How do I avoid overfitting?

Fewer trials, plateau selection and out-of-sample confirmation.

Sources and methodology

Backtesting principles follow TradingView's Pine Script documentation on strategies, repainting and non-standard charts, and ChartsPod's strategies-and-backtests guide summary, as read on 17 October 2026. All numbers are illustrative and do not describe a real strategy's performance.

  1. TradingView Pine Script: Strategies — simulated fills and assumptions.
  2. ChartsPod documentation — guides for order flow, derivatives, alerts, backtests, journal and workspaces.

This article explains research tools and workflows. It is not financial advice, does not recommend buying or selling any asset and does not predict prices. Crypto derivatives carry a risk of rapid and total loss of margin.

CHARTSPOD EDITORIAL EDITION · RESEARCH AND EDUCATION ONLY · NOT FINANCIAL ADVICE.