Comments on: Backtesting http://www.thestockbandit.net/2006/03/08/backtesting/ Trading blog for day trading, swing trading, and trading psychology Wed, 17 Mar 2010 05:57:48 +0000 http://wordpress.org/?v=2.7 hourly 1 By: Testing 1-2-3 | TheStockBandit.net http://www.thestockbandit.net/2006/03/08/backtesting/comment-page-1/#comment-139564 Testing 1-2-3 | TheStockBandit.net Tue, 12 May 2009 12:17:42 +0000 http://www.thestockbandit.net/?p=63#comment-139564 [...] written before about my opinion of backtesting, but let’s go a different direction here and talk about just plain-Jane [...] [...] written before about my opinion of backtesting, but let’s go a different direction here and talk about just plain-Jane [...]

]]> By: EMB http://www.thestockbandit.net/2006/03/08/backtesting/comment-page-1/#comment-2028 EMB Tue, 10 Oct 2006 03:52:57 +0000 http://www.thestockbandit.net/?p=63#comment-2028 I've done a lot of work that testing both in the commodities and the stock markets, and I've seen the pitfalls. However, it is possible to avoid them with the right kind of analysis. I've been doing very well over the past two-plus years in the stock market using a strategy that I developed through backtesting the previous five years of markets. The actual results (so far, anyway) are fairly consistent with the backtesting results. So I wouldn't dismiss backtesting out of hand if I were you. I’ve done a lot of work that testing both in the commodities and the stock markets, and I’ve seen the pitfalls. However, it is possible to avoid them with the right kind of analysis. I’ve been doing very well over the past two-plus years in the stock market using a strategy that I developed through backtesting the previous five years of markets. The actual results (so far, anyway) are fairly consistent with the backtesting results. So I wouldn’t dismiss backtesting out of hand if I were you.

]]> By: john http://www.thestockbandit.net/2006/03/08/backtesting/comment-page-1/#comment-23 john Sun, 12 Mar 2006 12:18:05 +0000 http://www.thestockbandit.net/?p=63#comment-23 Backtesting should always be done with current data and across several different timeframes. I like using 90 days of data in 4 increments - 90 days ago, 60 days ago, 30 days ago and today. By doing this I am overlapping several increments and testing two totally fresh batches. If I can get consistent results in both win rate and roi across all tests I feel that I have a good, current indicator. Would that indicator have worked 5 years ago? Maybe, but, really, who cares? Markets change, chop is the norm, the 90's will probably never exist again. So if I test a system that works great from 99 to 2000 should I use it today? Only if it tests well today. Backtesting should always be done with current data and across several different timeframes. I like using 90 days of data in 4 increments - 90 days ago, 60 days ago, 30 days ago and today. By doing this I am overlapping several increments and testing two totally fresh batches. If I can get consistent results in both win rate and roi across all tests I feel that I have a good, current indicator.

Would that indicator have worked 5 years ago? Maybe, but, really, who cares? Markets change, chop is the norm, the 90’s will probably never exist again. So if I test a system that works great from 99 to 2000 should I use it today? Only if it tests well today.

]]> By: TheStockBandit.net » Seeing the Future? http://www.thestockbandit.net/2006/03/08/backtesting/comment-page-1/#comment-20 TheStockBandit.net » Seeing the Future? Thu, 09 Mar 2006 22:36:23 +0000 http://www.thestockbandit.net/?p=63#comment-20 [...] While I think it’s important to be able to locate and use chart patterns and technical analysis for trading, I sure don’t think a trader’s ability to tell the future (or backtest the past) will make him a profitable trader. Chart pattern recognition is certainly helpful to traders, but what about execution? What about psychology and knowing when to ride out a pullback versus recognizing a reversal and knowing when to bail out? Some things you just have to learn by trading. [...] [...] While I think it’s important to be able to locate and use chart patterns and technical analysis for trading, I sure don’t think a trader’s ability to tell the future (or backtest the past) will make him a profitable trader. Chart pattern recognition is certainly helpful to traders, but what about execution? What about psychology and knowing when to ride out a pullback versus recognizing a reversal and knowing when to bail out? Some things you just have to learn by trading. [...]

]]> By: Dino http://www.thestockbandit.net/2006/03/08/backtesting/comment-page-1/#comment-19 Dino Thu, 09 Mar 2006 15:27:56 +0000 http://www.thestockbandit.net/?p=63#comment-19 Interesting thoughts on backtesting. The biggest problem that I have found with backtesting is simple. Time. Many systems simply do not go back far enough. For example, let us assume that the current market that we are in (flat) is due for some type of 10% correction. What good is a backtesting program that only goes back a few months, or even a few years? NONE. I can see the significance in backtesting programs that give data going back 5 to 10 years, where someone can test there ideas in both bull and bear markets. I am currently playing around with the backtesting features in Tradestation and with Zacks. I find them interesting. No matter what results they may come up with, a trader needs to be able to read a chart and make that finally dicision before buying. Good luck all. Interesting thoughts on backtesting. The biggest problem that I have found with backtesting is simple. Time. Many systems simply do not go back far enough. For example, let us assume that the current market that we are in (flat) is due for some type of 10% correction. What good is a backtesting program that only goes back a few months, or even a few years? NONE. I can see the significance in backtesting programs that give data going back 5 to 10 years, where someone can test there ideas in both bull and bear markets. I am currently playing around with the backtesting features in Tradestation and with Zacks. I find them interesting. No matter what results they may come up with, a trader needs to be able to read a chart and make that finally dicision before buying. Good luck all.

]]> By: Michael Taylor http://www.thestockbandit.net/2006/03/08/backtesting/comment-page-1/#comment-18 Michael Taylor Thu, 09 Mar 2006 00:49:22 +0000 http://www.thestockbandit.net/?p=63#comment-18 Agree 100% with your comments. Thanks for the clarification. Now, was that the 4th red bar after or before the moving average crossover? Ha ha. :) Here's that funny forum topic I was referring to: http://www.wealth-lab.com/cgi-bin/WealthLab.DLL/topic?id=7536&start=0 Have a good week, MT Agree 100% with your comments. Thanks for the clarification.

Now, was that the 4th red bar after or before the moving average crossover? Ha ha. :)

Here’s that funny forum topic I was referring to: http://www.wealth-lab.com/cgi-bin/WealthLab.DLL/topic?id=7536&start=0

Have a good week,

MT

]]> By: Administrator http://www.thestockbandit.net/2006/03/08/backtesting/comment-page-1/#comment-17 Administrator Wed, 08 Mar 2006 23:21:29 +0000 http://www.thestockbandit.net/?p=63#comment-17 Hey Michael, Thanks for your two cents! I think that your input is certainly valid, so let me explain the point I was trying to get across. I agree with you completely in that being able to code systems to locate patterns should be done, especially by those who are able to do it (those with the ability to mathematically express just what a chart pattern like a flag looks like). Perhaps I should have clarified better what my point was.....that the weekend warrior who reads in a trading magazine at Barnes & Noble that he should backtest a strategy can go home, download the software and start tinkering that afternoon. This hypothetical trader will likely continue to adjust the inputs & variables to improve performance, and by Saturday at dinner he's got random inputs that would have tripled his account in the last month. Come Monday's opening bell, he might put it into action and start taking trading signals based on what his 3 hours generated! That's poor judgement and exactly the kind of trader I'm referring to. The glossy ads seem aimed at the guy who's been buying random stocks while the market chops around and he can't understand why his account is lower than it was on Jan. 1. So, the thought of backtesting appeals greatly to him as he wants to be able to quit his day job. I should have done better to clarify that the "backtester" I was making a jab at is the amateur part-timer who wants to know what the perfect trading system would have made him in the past, and how he might apply that towards tomorrow's session. THAT is the guy that I think is a victim of the ideal that he can put a little elbow grease into his strategy software and POOF, have a wonderful and profitable strategy. When it comes to locating patterns based on a mathematical interpretation and having a PC locate those for you, I'm all for that. I just don't think that backtesting to buy the 4th red bar after a moving average crossover with the TRIN at lows is going to make anybody any money, even if it might have worked like a charm last week! Jeff Hey Michael,

Thanks for your two cents! I think that your input is certainly valid, so let me explain the point I was trying to get across.

I agree with you completely in that being able to code systems to locate patterns should be done, especially by those who are able to do it (those with the ability to mathematically express just what a chart pattern like a flag looks like). Perhaps I should have clarified better what my point was…..that the weekend warrior who reads in a trading magazine at Barnes & Noble that he should backtest a strategy can go home, download the software and start tinkering that afternoon. This hypothetical trader will likely continue to adjust the inputs & variables to improve performance, and by Saturday at dinner he’s got random inputs that would have tripled his account in the last month. Come Monday’s opening bell, he might put it into action and start taking trading signals based on what his 3 hours generated! That’s poor judgement and exactly the kind of trader I’m referring to.

The glossy ads seem aimed at the guy who’s been buying random stocks while the market chops around and he can’t understand why his account is lower than it was on Jan. 1. So, the thought of backtesting appeals greatly to him as he wants to be able to quit his day job.

I should have done better to clarify that the “backtester” I was making a jab at is the amateur part-timer who wants to know what the perfect trading system would have made him in the past, and how he might apply that towards tomorrow’s session. THAT is the guy that I think is a victim of the ideal that he can put a little elbow grease into his strategy software and POOF, have a wonderful and profitable strategy.

When it comes to locating patterns based on a mathematical interpretation and having a PC locate those for you, I’m all for that.

I just don’t think that backtesting to buy the 4th red bar after a moving average crossover with the TRIN at lows is going to make anybody any money, even if it might have worked like a charm last week!

Jeff

]]> By: Michael Taylor http://www.thestockbandit.net/2006/03/08/backtesting/comment-page-1/#comment-16 Michael Taylor Wed, 08 Mar 2006 23:04:42 +0000 http://www.thestockbandit.net/?p=63#comment-16 I'm sure part of this post was a jab to testers like myself to get some response. Well, you're getting one. Ha ha. :) I think you're right in some ways and wrong in others. What's happening now in the retail trading world is the same thing that happened in the corporate world back in the 90's. Back in the 90's, everyone in the corporate world just threw money at technology...didn't matter what it did or whether the results or ROI was better or worse than before. If the project contained one of the buzzwords...then another million thrown. Now, things are different...the buzzwords don't mean as much...substance is more important than some slick ad or salesman. And I think retail trading is at this technological stage where everyone wants to backtest, forward test, fundamentally rank, or any other buzzword that is hot. All without knowing what any of it really means or how to evaluate the results these things can produce. An example of this is the Wealth-Lab forum has a very funny topic where new users are wanting to trade this fantastic system that peaks a bar ahead. And cannot understand why they cannot trade this peak ahead system because the backtested results are so amazing! :) Anyway, back to your main point...that your trading strategy should be as simple as can be...but not any simpler. Markets change and bring about new environments and your trading strategy should adjust. I agree 100% with all of this. But, please, keep one thing in mind...something one of my programming mentors told me back in the day. If it can be explained...it can be programmed. If you can come up with a trading strategy...it can be coded and tested...and more importantly should be coded and tested. Trading systems can follow trends. Flags, triangles, double tops, and channeling stock patterns can be coded. And all can be tested and quantified. If this can be done...shouldn't it be done? I believe it should but with one word of caution...you've got to know how to test and evaluate the results all these wonderful backtests produce. Without that...you might as well keep to your golden gut. Oh, and most importantly, you've got to be able to logically express ideas into computer code. There's an art to that as well. Some people wouldn't know the first step in converting patterns such as a double top into logic a computer can understand. But, that doesn't mean it cannot be done. :) Just my two cents...keep the change. Take care, MT I’m sure part of this post was a jab to testers like myself to get some response. Well, you’re getting one. Ha ha. :)

I think you’re right in some ways and wrong in others. What’s happening now in the retail trading world is the same thing that happened in the corporate world back in the 90’s.

Back in the 90’s, everyone in the corporate world just threw money at technology…didn’t matter what it did or whether the results or ROI was better or worse than before. If the project contained one of the buzzwords…then another million thrown. Now, things are different…the buzzwords don’t mean as much…substance is more important than some slick ad or salesman.

And I think retail trading is at this technological stage where everyone wants to backtest, forward test, fundamentally rank, or any other buzzword that is hot. All without knowing what any of it really means or how to evaluate the results these things can produce.

An example of this is the Wealth-Lab forum has a very funny topic where new users are wanting to trade this fantastic system that peaks a bar ahead. And cannot understand why they cannot trade this peak ahead system because the backtested results are so amazing! :)

Anyway, back to your main point…that your trading strategy should be as simple as can be…but not any simpler. Markets change and bring about new environments and your trading strategy should adjust. I agree 100% with all of this. But, please, keep one thing in mind…something one of my programming mentors told me back in the day. If it can be explained…it can be programmed. If you can come up with a trading strategy…it can be coded and tested…and more importantly should be coded and tested.

Trading systems can follow trends. Flags, triangles, double tops, and channeling stock patterns can be coded. And all can be tested and quantified. If this can be done…shouldn’t it be done?

I believe it should but with one word of caution…you’ve got to know how to test and evaluate the results all these wonderful backtests produce. Without that…you might as well keep to your golden gut.

Oh, and most importantly, you’ve got to be able to logically express ideas into computer code. There’s an art to that as well. Some people wouldn’t know the first step in converting patterns such as a double top into logic a computer can understand. But, that doesn’t mean it cannot be done. :)

Just my two cents…keep the change.

Take care,

MT

]]>