I don't know for sure, I would have Metaquote's people answer this question, but to me it seems to me that the strategy tester access a different "temporary" data file depending on wether or not you are connected to your broker's server. Therefore you get one or the other but the results on each data set remains the same even though it is a different result for both data file.

I have not had time, nor did I really ever care to research this any further sorry :)

I am planning to run my 6 EAs independently under the assumption that they all are profitable and are designed with from different perspectives. It is highly unlikely that all 6 EAs will experience their worse drawdown during the same period.

Patrick,

Thanks for your prompt reply. I have had lots of problems with the backtesting in MetaTrader and I have asked around. It seems that everyone in the community is having the same reliability problem; hence, Metaquotes needs to address this issue. For instance I only trade the Euro and have designed 6 EAs using 6 different trading techniques. The majorities of my EAs do not use an averaging or technical indicator as they are too prompt to curve fitting during optimization and would fail during live trading. I would test my 6 EAs on 3 different computers and would get completely different results even when the 3 computers are using the same imported 1min historical data with 90% model quality.

Computer A Computer B Computer C
EA1 very profitable No good very profitable
EA2 very profitable very profitable No good
EA3 very profitable No good No good
EA4 very profitable very profitable No good
EA5 very profitable very profitable No good
EA6 very profitable No good very profitable

Note: very profitable is defined as profit to loss ratio of 5:1, for instance my stoploss would be 10pips and my limit would be 50pips. No good is defined as money losing EA. How is it possible to be very profitable on one computer, and then not profitable at all but then 100% correllation on another EA?

Hello Trohoang,

1) Are we talking about backtesting forex or backtesting in general? If you ask simply regarding about Forex let me tell you that MetaStock backtesting is just plain useless. For the simple reason that their backtester can not enter a position in real time. What I mean is that MetaStock system tester can only use the open high low close of a bar as its entry and therefore its simulation, especially for Forex, is of little value. As you know within a 5 minute bar the price may have gone up and down 30 pips ...
Now while MetaTrader's backtesting is better, I still do not think it is fully accurate. I don't want to get in trouble with the good people over at metaquotes but my only use for the backtesting in MetaTrader is to check that my expert entered and exited trades on the right spot based on my strategy. The reason I say this is because the data used in MetaTrader is not tick historical data ... Unless you can get historical tick data from 2002 you can't really backtest your strategy and rely on the results you are getting ...

Now what I recommend to every one when wanting a simulation result, is to open a demo account, let the expert run live on it for a few weeks and that will give you some real results.

2) I believe that each currency has its own personality, however some are similar enough that a system could work on multiple currency pairs. For example EURUSD and USDJPY are so close to each other in terms of market reaction that a system working on one will most likely work on the other. This to me is important, especialy when considering risk. The saying goes "Don't put all your eggs in the same basket". Here is a practical example of this, let's first assume that you agree that the EURUSD and USDJPY are fairly similar. Let's say my system triggers a buy on EURUSD and therefore a SELL on USDJPY. One may be tempted to buy 10 lots on eurusd ... I would however buy 5 lots on EURUSD and SELL 5 lots of USDJPY. For the simple reason that you never know what will happen. What if tomorrow the EU adds a few more coutries to its union and traders perceives this as a devaluation of the EURO. There you have an economic event that will only affect the euro .. . Same idea with big events such as wars or some guy in Iraq getting shot causing oil prices to go up $3 a barrel. So yes limit the currency pairs you trade but limit your risk by trading a few different ones that are similar.

Regarding the expert idea, in theory it sounds good but it will be extremely difficult to concile all 10 strategies and define which one does what and when. It is very interresting ... Have you already found a way of combining all this? Do you have a ranking setup? How would you decide which trending system ( out of the 3 you made ) to use if the market is trending?

3) If you made an EA that worked well for 2 years, I think it is something to be proud of. I guess it will depends also on what the market did during these 2 years, was it "stable" or did it go through many changes during that time period. But the question for me remains what do we mean by "works", I know of someone who turned $80 000 into a few millions of dollars in 6 month, imagine what it will be in 1 year from now if the expert worked that long ... It is all about perception and about what one expects out of trading, if you are pleased with your results then you have something to write home about ...


Thanks for your comments :)

Patrick,

Thank you very much for sharing your thoughts with us. I have several questions for you:

1) how reliable is MetaTrader's backtesting compared to Metastock? Do you know of any tools that is reasonably reliable in backtesting? Keep in mind, that I don't use the back testing for EA development or optimization. Curve fitting is the last thing I want to do. I have completed several profitable EAs that I would like to backtest from 2002 to 2005. So far I have ran these EAs live over 4 weeks and they seems to hold their grounds. I would like to know their worse drawdown over a longer term; hence, backtesting data are much better than random generated data.

2) Regarding your comment about an ideal system adjusting to market condition i. e. if(Market is in Trend) { Use MACD system } else { use RSI system }, I agree with you. A common pitfall is curve fitting during the transition and the actual state of the marekt. What do you think about creating 10 profitable EAs on the same currency, 3 EAs are on trending, 3 EAs are on range bounding, 2 EAs on scalping and one generic? I am a big fan of only trading one single currency rather than multiple currencies trading.

3) I have come across several programmers and they mention that they have come across very few profitable EAs that work over a long period of time. If an EA works over a period of 2 years, do you think that is enough to write home about?