Backtesting – Part 1

Guest Post by DV 34

As promised, our guest blogger DV34 is going to post a series of articles covering his practical experience backtesting trading strategies 

This promises to be a really valuable resource. I hope you enjoy it

Part 1 – Introduction and Article Series Outline

I thought the best place to start this series on manual back testing was to outline the basic structure of how the articles will be written. They are only from my own experience, and I respect others will have different views but here goes… 

As a note right up front BACK TESTING DOES NOT GUARANTEE PROFITS, it has both pros and cons, there are traders who totally agree with it and others who question its validity and believe it neglects the emotional/ psychological side of trading.

While both sides have valid arguments, I find that for me,  back testing  allows me to understand my strategies nuances much better, and to develop a deeper understanding of the possible risks, ideal market conditions and when it is likely to fail.

It is useful to understand this as most strategies are built around trading a specific market condition – but not all market conditions. 

In back testing there is a natural tendency to use a ‘hindsight bias’ knowing what happened to skew the results to see what you want to see… not what you need to see…  this is quite important.

There are ways to try to mitigate this effect, but the bottom line is that during testing you want to make the strategy work as hard as possible to be profitable, including all costs and possibly adding slippage to literally try and break it

It is always better to have it perform better than tested going forward and err on the side of caution 

Developing a trading strategy goes something like this: 

1)         MONITORING THE MARKET/ OBSERVATIONS 

2)         TRADING IDEAS/ CONCEPTS/ BELIEFS 

3)         CHOOSING THE TOOLS 

4)         SPECIFIC RULES, CONDITIONS 

5)         TESTING & VERIFICATION 

6) …. And then back to Step 1… 

In this series I will focus on Step 5 only: TESTING & VERIFICATION, which automatically suggests that you already have a trading strategy/ rules to guide your trading decision making process… 

This series is not about developing a complete trading strategy as there are literally 1,000’s of ways of doing it, therefore Steps 1-4 are excluded. 

I broke the Testing and Verification section down into further key segments as below, this is going to serve as the framework for how the articles will be presented with a little more detail/ explanations included in each part possibly including some charts/ diagrams and examples along the way. 

4)                  TESTING & VERIFICATION 

A)    THE BASICS

i)          Why Backtest? Pro’s vs. Con’s

ii)         Embracing uncertainty – it’s all about probabilities – why they matter…

iii)        What are my objectives? Goals? Desired outcomes?

iv)        What is my risk tolerance! How can I limit or mitigate it?

v)         Using Van Tharp’s Approach – Using statistics to quantify and measure a Trading Strategy

vi)        “R” Multiples – Reward to Risk Ratios

vii)        Winning %

viii)       Expectancy (the mathematical edge)… & the Expectancy Curve

ix)         Van Tharp’s SQN – (System Quality Numbers)

x)             Expectunity (Expectancy x Opportunity)

xi)          Time of Day – Can I trade it?

xii)        Observations during testing

xiii)        Quantifying and defining a comfortable level of risk/ setting reasonable boundaries/ limits around your strategy

xiv)        Applying position sizing strategies to meet objectives 

B)        THE TESTING PROCESS

i)      Define a clear rule-based strategy that you wish to test (Mechanical or Discretionary)

ii)     Decide on a testing process, i.e. back testing (proof of concept), forward/ walk-forward testing (verification)

iii)    The types of testing

  • Back testing (Proof of concept)
  • Scrolling back over charts
  1. Trading Simulator with unseen data
  • Forward Testing (The acid test… verification on unseen data)
  1. Walk Forward Analysis – Slicing off a part of unseen data to verify the initial test
  2. Real Time Demo Testing

iv)   Collate and set up data for testing

  • Check data quality, Vital…!
  1. Are there any anomalies? large spikes, large gaps/ missing data etc
  2. Check server time of data – when do candles “close?” this can change the appearance of data dramatically
  3. Acknowledge that the test will not be 100% accurate due to daylight saving offsets and psychological biases – i.e. tendency to ‘curve fit’
  4. Include the spread in your tests

v)   Develop a record keeping method

  • Pen and paper
  • Using a spreadsheet similar to Microsoft excel
  • Develop a trading algorithm to execute the test trades for you and obtain statistics ex a programmable software package

vi)   Scroll through and execute orders on historical data – (Preferably unseen) using your rules/ strategy and record raw results

  • Don’t cheat because it will cost you much more $$ later!! Try to make it as real as possible
    • Avoid curve fitting or over optimizing a strategy to data using hindsight bias
    • Try to trade the “hard right edge” during testing as much as possible 

C) RECORDING RESULTS

  • Capture raw data, namely:
  1. Security/ Item
  2. Buy/Long or Sell/Short?
  3. Date (Entry)
  4. Time (Entry)
  5. Lots/ Units
  6. Stop Loss Price
  7. Entry Price
  8. Target Price
  9. Exit Date
  10. Exit Time
  11. Exit Price
  12. Risk in $$ (or pips)
  13. Reward in $$ (or pips)
  14. Reward to Risk Ratio of each trade
  15. Potential Reward to Risk in $$ (or pips)
  16. Wins
  17. Losses
  18. No. Days in Trade 

D) EXTRACTING KEY STATISTICS YOU WANT FROM THE TEST

  • The basic statistics
  1. The importance of exits… not entries
  2. No of Trades (Preferably 50-100+) more = better… as each trade becomes less significant in your sample
  3. Win% – All Trades
  4. “R” Multiples (Reward to Risk Ratios – positive or negative) for all trades – If scaling in or out then compile overall trade into a single “R” multiple result
  5. Avg. Reward to Risk over all Trades
  6. Expectancy over all trades
  7. Std Deviation for all “R” Multiples
  8. SQN number or Ratio of Expectancy/ Std Deviation
  9. Maximum Drawdown during test in terms of “R” Multiples

10. % Long Trades, % Short Trades (to check it is not just a bull or bear market)

11. Avg. no of days in trade (carry costs?)

12. No. opportunities over a set period (i.e. year, month, week etc)

13. Expectunity

 14. Any others not listed you feel are important

E) APPLY POSITION SIZING STRAEGY TO KEEP RISK WITHIN OBJECTIVES

  • Using Position sizing to manage your risk to an acceptable level
  1. Be aware of win% vs. drawdown, Lower win% = Larger drawdown’s

i.e. Monte Carlo Simulation

(Monte Carlo simulation is scrambling the order of trades repeatedly to try to simulate the worst case scenario and probability)

  1. Note larger position size = greater account volatility – both positive and negative

i.e. What I call the megaphone effect

  1. Note the impact of your position sizing strategy
  2. Develop worst case contingency plan

F)     VERIFICATION

  • Forward test on live markets using either a demo account or small live trial account
  • Obtain large enough sample to be statistically relevant (50-100+)
  • Compare to back tested results
  • Define limits/ action plan if strategy goes beyond expected boundaries
    • Pull plug?
    • Reassess?
    • Review all trades? 

As I mentioned earlier this draws heavily on Van Tharp’s work (which is excellent in my opinion and a must-read), and I will go through each section using my own backtest results in the coming articles 

Hope this helps and look forward to hearing any feedback 

DV34

About Ric Spooner

Over 30 years market experience - professional trader, broker, director
This entry was posted in Trading and tagged , , , . Bookmark the permalink.

2 Responses to Backtesting – Part 1

  1. jwc says:

    i guess one of the main problems is finding an easy to use, cheap (or free) and reliable platform for backtesting, which i find is something hard to come by..

    • dv34 says:

      I will note the one I use and you will see some screenshots of it soon, you can upload your own data to test and enter Parent + 2-leg oco orders on the chart, although this costs around $150-200, still I wish I bought it years earlier as it would have easily saved me that in losses!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s