Guarding against data-snooping. As a rule of thumb, we assume that the number of data points we need for optimizing is 252 (365 in cryptos case) times the number of free parameters your model has (not based on literature but on his experience). If a model has 3 parameters then 3 years of data is required (daily).
Out of Sample Testing. Divide your historical data into two parts. Save the second (more recent) part of the data for out of sample testing. When you build the model, optimize the parameters as well as other qualitative decisions on the first portion (the training set), but test the resulting model on the second portion (called the test set). The two sets should be roughly equal in size.
A more rigorous (albeit more computationally intensive) method of out-of-sample testing is to use moving optimization of the parameters. In this case, the parameters themselves are con- stantly adapting to the changing historical data, and data-snooping bias with respect to parameters is eliminated.
“Beware of changes in companies fundamentals that can render out of sample performance quite poor despite stellar backtest results” - Chan
“Trading a portfolio of cointegrating ETF’s can be better than pair-trading stocks” - Chan
Notes on Algo Trading, Winning Strategies and their Rational
Advanced Algorithmic Trading Book Notes