Strategy Optimization

The Strategy Tester allows you to test and optimize trading strategies (Expert Advisors) before using them for live trading. During testing, an Expert Advisor with initial parameters is once run on history data. During optimization, a trading strategy is run several times with different sets of parameters which allows selecting the most appropriate combination thereof.

The Strategy Tester is a multi-currency tool for testing and optimizing strategies trading multiple financial instruments. The tester automatically processes information of all symbols that are used in the trading strategy, so you do not need to manually specify the list of symbols for testing/optimization.

The Strategy Tester is multi-threaded, thus allowing to use all available computer resources. Testing and optimization are carried out using special computing agents that are installed as services on the user's computer. Agents work independently and allow parallel processing of optimization passes.

An unlimited number of remote agents can be connected to the Strategy Tester. In addition, the Strategy Tester can access the MQL5 Cloud Network. It brings together thousands of agents around the world, and this computational power is available to any user of the trading platform.

In addition to Expert Advisor testing and optimization, you can use the Strategy Tester to test the operation of custom indicators in the visual mode. This feature allows to easily test the operation of demo versions of indicators downloaded from the Market.

How to Optimize

Optimization means multiple runs of an Expert Advisor using history data with different sets of parameters, aimed at finding their best combination. During multiple runs, different combinations of the input parameters of an Expert Advisor are tested to find the best ones.

Watch the video: How to test Expert Advisors and Indicators before purchase

Watch the video to learn how to test a trading robot before you purchase it from the Market. Every product on the Market is provided with a free demo version, which can be tested in the Strategy Tester. Watch the video for details.

How to select a trading robot for testing

Click " Test" in the context menu of an Expert Advisor in the Navigator window.

The tester automatically downloads the history of required symbols from the trading platform (not from the trade server!) during the first call of the symbol data. Only the missing price history data are additionally downloaded from the trading server.

Before you start optimization of a multi-currency Expert Advisor, enable the symbols required for testing in the Market Watch. In the context menu, click " Symbols" and enable the required instruments.

Choosing Optimization Settings

Before you start optimization, select the financial instrument to test the trading robot operation on, the period and the mode.

Symbol and period

Select the main chart for testing and optimization. Symbol selection is required to provide the triggering of OnTick() events contained in Expert Advisors. Also, the selected symbol and period affect special functions in the Expert Advisor code that use current chart parameters (for example, Symbol() and Period()). In other words, the chart to which the Expert Advisor is attached should be selected here.

Date

Select the testing and optimization period. You can select one of predefined periods or set a custom time interval. To set a custom period, enter the start and end dates in the appropriate fields to the right.

The specific feature of the tester is that it additionally downloads some data preceding the specified period (to form no less than 100 bars). This is required for a more accurate testing and optimization. For example, if you test on a one-week timeframe, two additional years are downloaded.

If there is not enough history data for forming additional 100 bars (it is especially significant for the monthly and weekly timeframes), for example, when specifying a start of testing close to the start of existing history data, then the start date of testing will be automatically shifted. An appropriate message is added to the Strategy Tester journal.

Forward

This option allows you to check the results of optimization in order to avoid fitting to certain time intervals. During forward optimization, the period set in the Date field is divided into two parts in accordance with the selected forward period (a half, one third, one fourth or a custom period when you specify the forward testing start date).

Expert Advisor optimization is performed using the data of the first period. After that 10% (in the full search) or 25% (in the genetic algorithm) of best runs are selected and then tested on the forward period. The results of the best optimization runs on both periods can be compared on tabs Optimization Results and Forward Results.

Execution

The strategy tester allows you to emulate network delays during an Expert Advisor operation in order to bring testing closer to real conditions. A certain time delay is inserted between placing a trade request and its execution in the strategy tester. From the moment of sending a request till its execution, the price can change. This allows you to evaluate how trade processing speed affects the trading results.

In case of the instant execution mode, users can additionally check the EA's response to a requote from the trade server. If the difference between requested and execution prices exceeds the deviation value specified in the order, the EA receives a requote.

Please note that delays work only for trades performed by an EA (placing orders, changing stop levels, etc.). For example, if an EA uses pending orders, delays are only applied to placing an order but not to its execution (in real conditions, execution occurs at the server without a network delay).

No delay

In this mode, all orders are executed at requested prices with no requotes. The mode is used to check an EA in "perfect" conditions.

Random delay

This mode allows testing an EA in conditions that are close to real ones. The delay value is generated as follows: a number from 0 to 9 is selected randomly - this is the number of seconds for a delay; if a selected number is equal to 9, another number from the same range is selected randomly and added to the first one.

Thus, the possibility of a delay for 0-8 seconds is 90%, possibility of a 9-18 second delay is 10%.

Fixed delay

You can select one of the predefined delay values or set a custom one. The platform measures the ping to the trade server and allows you to set that value as a delay in the tester so that you are able to test a robot in the conditions that are as close to the real ones as possible.

Tick generation mode

Select one of the tick generation modes:

Every tick is the most accurate but also the slowest mode. It emulates all ticks.

Every tick based on real ticks is as close to real conditions as possible. It uses real ticks
of financial instruments accumulated by a broker. Emulation is not performed. Tick data has larger size.
Downloading it may take quite a long time during the first test.

1 minute OHLC — in this mode only 4 prices (Open, High, Low and Close) of each minute bar are emulated.

Open prices only — in this mode OHLC prices are also modeled, however only the open price is used for testing/optimization.

Math calculations — in this mode the tester does not download history data and information on symbols, as well as does not generate ticks. Only functions OnInit(), OnTester() and OnDeinit() are called. Thus a tester can be used for various mathematical calculations where the selection of parameters is required.

Initial deposit and leverage

Specify the amount of the initial deposit used for testing and optimization. The currency depends on the deposit currency of the currently connected account. Next select the leverage for testing and optimization.

Note that symbol specification does not mean that the tester will use only these history data. The tester automatically downloads information on all the symbols used in the Expert Advisor.

Before testing/optimization, all the available price data of the symbol on the main chart are automatically downloaded from the server. It may take quite a long time if the internet connection is slow.

Downloading of all data is performed once, only the missing information is downloaded during the next starts.

Only the symbols that are currently selected in the Market Watch are available for testing/optimization.

The price data of all necessary symbols are automatically downloaded from the server during testing and optimization.

Testing starts and ends at 00hr.00m.00s. of the specified dates. Thus the start date of testing/optimization is included in the testing period, while the end date is not included. Testing ends on the last tick of the previous date. Also you cannot specify the end date, which is greater than the current one. In such case, the testing anyway will be performed to the current date (not including it).

The quick optimization based on the genetic algorithm is enabled by selecting optimization criteria in the field located to the right. This field sets the parameter, based on which the most successful Expert Advisor runs are selected. The larger the value of a selected parameter, the better the result.

After setting all the parameters click "Start". This launches the process of testing and optimization.

The settings of the strategy tester are memorized as testing/optimization is started.

In case of a regular optimization stop (when you press the Stop button) all the previously calculated runs are saved. When the optimization process is resumed, it continues from the last calculated run.

Selection of Input Parameters

Input parameters allow you to control the behavior of the Expert Advisor, adapting it to different market conditions and a specific financial instrument. For example, you can explore the Expert Advisor performance with different Stop Loss and Take Profit values, different periods of the moving average used for market analysis and decision-making, etc.

Optimization is testing different values ​​and combinations of input parameters to obtain the best result.

To enable the optimization of a parameter, mark the appropriate checkbox. Next set the start and end of the range of values, as well as the step for testing. You can select one or more parameters. The total number of possible combinations is displayed beneath the list of parameters.

Parameter sets. You can at any time return to the current settings of your MQL5 program by saving a set of its parameters using a context menu:

To save the parameters as a set-file on your computer, click "Save". These files can be moved between platforms on different computers or sent to other users.

To save parameters for future use in the current platform, click "Save Version". These saved presets will be available then in the "Load Version" sub-menu. They can be applied at any time by selecting an appropriate version from the list.

Start of Optimization

To start optimization, click "Start" on the "Settings" tab. The optimization progress is displayed to the left.

Where to View the Optimization Results

Detailed results of each optimization run are displayed on the "Optimization" tab. The tab contains general testing results, including profit and the number of trades, as well as many statistical values to help assess the performance of the trading robot.

Recovery factor — this parameter displays the risk level of the strategy (the funds that are put to risk to earn the obtained profit). It is calculated as the ratio of gained profit to the maximum drawdown;

Sharpe Ratio — this parameter shows strategy efficiency and reliability. It reflects the ratio of the arithmetical mean profit for the position holding time to the standard deviation from it. In addition, this value includes the risk-free rate which is the interest on a certain bank deposit amount;

Optimized inputs — in addition to the common statistical values, values of input parameters set for this run are shown here.

Using context menu commands you can show/hide some of the above columns. For convenience, check the "Switch to Optimization Results" option: once the optimization process is complete, the Strategy Tester will automatically switch to the Results tab. The same command is available in the context menu of the Journal tab.

If optimization includes forward testing, this tab also contains the corresponding values of the optimization parameter (optimization criterion) for the back and forward tests. You can switch between results of back and forward testing using the context menu.

A double click on one of the optimization results starts Expert Advisor testing with the parameters of this run (provided that the optimization is over).

During genetic optimization one of the test runs (a population member) can have the same parameters (genes) as the previous test run. In this case, this run is not displayed on the results tab, because it has the same testing result. However, the optimization graph display all test runs to visualize the process of searching for the best result.

If a line of an optimization run has the red background, it means that an error occurred during Expert Advisor operation. An appropriate message is also added to the tester log ("tested with error").

Analysis of Optimization Results in Third-Party Software

To analyze results in third-party programs, for example, Office Excel, optimization report can be saved as a file through the " Export to XML" command of the context menu.

The numerical values of all parameters and characteristics obtained during optimization are saved as an XML file in platform_data_folder/tester/cache/. The file is named according to the following rule: ExpertName.Symbol.Period.GenerationMode.xml, Here:

During the genetic optimization, intermediate results are saved in the cache after the calculation of each generation (in a file platform_data_folder/tester/cache/*.gen). Thus the optimization process can be interrupted at any time. Even if the process of genetic optimization is interrupted as a result of an external factor (for example, a black out), the optimization will be automatically continued from the last calculated generation once you restart it. The genetic optimization cache is stored until the optimization settings are changed or the optimization process is completed.

In case of a regular optimization stop (when you press the Stop button) all the previously calculated runs are saved. When the optimization process is resumed, it continues from the last calculated run.

The Visualization of Optimization Results

The Strategy Tester in the trading platform provides a powerful visualization system for presenting optimization results. Open "Optimization graph". The tab contains several types of charts, you can switch between them using the context menu.

Zero line (plane)

All kinds of graphs, except flat have a zero line (or pane if it's a three-dimensional chart). If the balance value is used as the optimization criterion, this line usually means the initial deposit, allowing to visually separate loss-making and profitable passes. In all other cases this line is drawn on the zero value of the optimization criterion.

Graph of results and linear chart (1D)

A graph with optimization results opens by default. Each pass of an Expert Advisor with certain input parameters is displayed as a point on the graph. The number of a pass is shown on the horizontal axis, the value of the parameter that is the optimization criterion is shown on the vertical axis.

The linear chart (1D) shows the variation of the parameter selected as the optimization criterion (vertical axis) depending on one of the optimization parameters selected for the horizontal axis. To select a parameter for the horizontal axis, use the "X Axis" command in the context menu.

Flat chart (2D) and three-dimensional chart (3D)

In the two-dimensional graph mode, variations of the selected parameters used for optimization are shown on both axes. Variation of the optimization criterion is shown using the color gradient. The deeper the color, the higher the value of the optimization criterion.

In the three-dimensional visualization mode, changes of the selected parameters used for optimization are shown on the X and Y axes. Variation of the optimization criterion is displayed on the vertical Z axis and using a color gradient.

To select a parameters for the horizontal and vertical axes, use commands "X Axis" and "Y Axis" in the context menu.

3D chart management using a mouse

To move a chart, grab its central part using the left mouse button and move the cursor.

To rotate a chart around its vertical axis, grab it outside the center and move the cursor.

To rotate a chart around its horizontal axis, rotate the mouse wheel holding down the "Ctrl" key.

To zoom in/out a chart, press "Ctrl" and move the mouse cursor vertically in the central part of the chart holding down the left mouse button.

To move the zero plane, press "Ctrl" and move the mouse cursor vertically outside the central part of the chart holding down the left mouse button.

To return to the initial position of the chart, double click in its central part.

The selected part is separated from the period specified in the "Date" field. The first part is the period of back testing, and the second one is the period of forward testing.

The full optimization (slow or fast) of the Expert Advisor is conducted on the back testing period. After that 10% (in the full search) or 25% (in the genetic algorithm) of best runs are selected and then tested on the forward period.

There is a lower limit for the number of passes of forward testing. If the number of best runs is less than 256, the additional best runs are used for forward testing until their number reaches 256. If the number of all runs is less than 256, all of them participate in forward testing.

Results of back and forward testing can be compared on the "Optimization Results" (select "Forward testing results" in the context menu) and "Forward Results" tabs. The better the results coincide, the more likely it is that the Expert Advisor will show good results in real trading.

The visualization of optimization results on the forward period is available on the "Forward optimization graph" tab. To compare these results with the backtest, switch between them using the context menu.

Multithreaded Testing Using Agents

The multithreaded Strategy Tester uses all available computer resources. Testing and optimization are carried out using special computing agents that are installed as services on the user's computer. Agents work independently and calculate optimization passes in parallel.

Three types of agents are available: local, remote and cloud (MQL5 Cloud Network). Local agents are installed automatically when you install the trading platform. Their number is equal to the number of logical cores of the computer.

Open the "Agents" section in the Strategy Tester and select the type of agents you want to use for optimization.

Tips and features:

To conserve the laptop battery, you can disable local agents and use only the remote and cloud ones.

If testing/optimization is not finished manually (neither by pressing the Stop button at the settings tab nor by closing the trading platform), the processes of used local agents are not unloaded from the computer memory for 5 minutes. This feature allows avoiding delays connected with preparing the price history and starting the agent processes when re-testing/re-optimizing the same Expert Advisor at the same symbol, timeframe and time period.

Only local agents are installed together with the platform installation. They are only used in the Strategy Tester of the local platform. Remote agents that can also be connected to the global MQL5 Cloud Network can be installed only manually.

How to Speed up Optimization Using a Local Farm of Agents

You can purchase a processor with more cores, but it does not allow to multiply the number of concurrent tasks. You can create your own farm of processing agents in your local network.

How to Create a Farm of Agents

Install agents on each computer of the local network. If the platform is installed on a computer, open testing agents manager using the "Tools" menu.

Select the number of agents to be installed. They are installed based on the number of logical cores.

Enter the password for connecting to the agent.

Select a range of ports for connection.

Click Add.

After installation, the agents are available for use from other computers on the local network.

Remote agents can only be used in 64 bit systems.

To save traffic and disk space, as well as for security reasons:

messages of Expert Advisors (Print() function) and messages about trade operations are not added to the Journal;

DLL call is prohibited on remote agents.

How to Connect Agents

Open the Strategy Tester. On tab "Agents", select "Local Network Farm" and click "Add" in the context menu.

The easiest and fastest way is to automatically scan the local network for a range of IP addresses and ports. Select them and enter the agent connection password that was specified during installation.

Click "Finish" and all the found agents become available for testing.

Other options to add agents:

Add agents (by IP address or domain name) — specify the IP address or domain name of a server where agents are installed, as well as the range of ports and password for connecting to the agents.

Import from file *.mt5 — select this option, click "Next" and specify the *.mt5 file from which you want to import agents.

Agents installed on the computer using MetaTester 5 Agents Manager, can be connected as remote on the same computer. If the processor cores have extra computation power during calculations, they can take a higher load to use all the computing capacity.

How to Change Agent Settings

To change the settings, click the " Edit" command in its context menu.

The following fields are available in the settings window:

Name — the name of the agent;

Address — IP address and port for connecting to an agent, separated by a colon;

Password — password for connection;

Enable — this option allows to enable or disable the use of the agent during testing and optimization.

In settings of local agents only the option of enabling/disabling them is available.

Import and Export of Settings of Remote Agents

To make setting up of remote agents easier, the platform provides a feature for importing and exporting their settings. The files of settings have the *.mt5 extension. The import and export commands are located in the context menu of the "Agents" tab.

Files of settings have the following format: Name;Address:port;Password;Description;Enable.

Name — the name of the agent;

Address:port — IP address and port for connecting to an agent, separated by a colon;

Settings of different agents are separated from each other with line breaks.

How to Speed Up Optimization Using the MQL5 Cloud Network

The MQL5 Cloud Network allows you to quickly optimize your Expert Advisors using the power of thousands of computers. The network combines remote agents and distributes optimization tasks among them. The Strategy Tester connects to the cloud network through multiple access points, which are distributed on a territorial basis (e.g., MQL5 Cloud Europe).

Payments for the Use of the MQL5 Cloud Network

Using agents of the MQL5 Cloud Network is paid. The formula for calculating the cost is described in a separate section. The current MQL5.community account balance is displayed above the list of cloud agents.

To use MQL5 Cloud Network a user needs to have at least 1 US dollar on the MQL5.community account. Tasks are passed in packages to several access points simultaneously, and the user must be able to pay for completion of that tasks. The Network is not able to calculate the exact cost as the time and resources required for calculations cannot be estimated precisely before the start of calculations.

Enabling MQL5 Cloud Network

To use the network agents, enable them using command " Enable" in the context menu. Since the MQL5 Cloud Network is a paid service, a user must have an account at the MQL5.community website, through which all the accounting operations are performed. Account details are specified on the MQL5.community tab of the platform settings.

If you do not specify the details of your MQL5.community account before enabling the MQL5 Cloud Network agents, you will be offered to do this.

Starting Calculations Using the MQL5 Cloud Network

Like with a conventional optimization, you need to set all the testing options and Expert Advisor input parameters. On the Agents tab, you can monitor how the Strategy Tester distributes tasks to available agents. The number of available and currently used agents is displayed for each access point.

Traders may need to run hundreds of thousands of optimization passes in a reasonable time. With the multi-threaded Strategy Tester and the MQL5 Cloud Network, in one hour you can complete the calculations that would require a few days without the network. The computing power of thousands of cores is available straight on the trading platform.