After you sign in to your demo account, go to Manage API Access. There you can find your API key which we are going to use in our system to place trades. MAKE SURE YOU DON'T SHARE THIS KEY.

The code for this is and all other posts is on github and you can install it and run it pretty easily.

Update: Oanda released a new (kickass) execution engine called v20 and they have released a new (improved) API. This post has been updated in order to use the new API but if (for any reason) you want to check the old code, it is right here. You lucky you!

Connecting to Oanda needs a conf file - which you can generate using a script that Oanda provides hereor you can just create it yourself. Why would you want that? First of all when it comes to credentials (and my money), I prefer to know everything that is going on. And I don't like having to install PyYAML just to read a conf file. Feel free to use either method.

Now, prepare to be amazed. The code is straight-forward. We initialize the API:

Super easy. Don't worry about what EURUSD is or how many units we are buying or what a market order is. For now, we have placed our first trade from our laptop and we are going to build our own API to place trades. Exciting stuff!

You can read Oanda's documentation here to see what else you can do with their API and find the Python library here.Tons of examples are available from Oanda's github page here.

Coming up next, connecting to a real LIVE algotrading system, running from my RaspberryPI at home.

You'll be able to see the (almost) final program running and we'll talk more about Forex and strategies.

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

This is more of a "How to build your own algotrading strategy - the Ethereum edition" and not a "make money fast" blog post.

It is also a real example with real returns (and real production errors that cost me money) where you can see how to identify opportunities, why algotrading is awesome and why risk management can save your ass.

I get this question almost on a daily basis. How can I find a good strategy? How can I built my own? Do I need to have a PhD in mathematics? Statistics?

Newsflash: If I can write a strategy, anyone can write a strategy. Trust me on that. The only trick is to look for a simple one.

Cryptocurrencies

Update: This post has been rewritten "at least" five times (as "The DAO drama" escalated) and it is the perfect example of a strategy doing a full circle.

I started getting involved with Ethereum early on as I really liked the "run your algorithms on the blockchain" thing. When TheDAO came out, I read everything about it and loved the idea.
You don't need to understand what Ethereum, theDAO, blockchain is at this point (I promise I will ramble on a another post). The same ideas apply to Forex, Stocks even Pokemon balls.

As I said before, simple ideas turn into (simple?) strategies.
Complex strategies turn into mayhem that is impossible to backtest
and deploy without errors.

My four step - captain obvious - system

I personally, have a specific way that I work.

Step one : Identify an idea.

My idea in this case is that there are a couple of exchanges offering Ethereum and DAO tokens. What if there was an arbitrage between those?

Step two : Manually test the idea.

If something "kinda works", I am on to something. All I had to do is execute all the steps manually and write down any fees, conditions or anything that should be documented.

Step three : Automate

This algorithm is not a high frequency trading algorithm. There is a considerable amount of time risk (which you can eliminate as we'll see later on) but what I did could be done manually. The problem is that I would have to spend all time in front of my computer, checking if there is an arbitrage condition and if there was, I had to act fast and without messing up. Oh and I had to recruit five of my friends to scale this up.

Long story short, I spent Presidents' Day writing a simple program that will replay all my manual steps. The program would crash and it was not more that 100 lines of code. This is the data collection stage where I see if there is an advantage that algorithms can give me. Advantages can be:

Something that be automated and run 1000 of times per seconds or 1000 times in parallel

Something that thinks faster than I can

Something that has no feelings to screw up my system

If there is at least one or more conditions met, I will start building and rewriting the algo.

Step four : All in

I am kidding and you'll see in a bit why risk management is super important in this business.

The Arbitrage

Let's discuss a little bit about what this arbitrage was.

The idea is: "I wonder whether Kraken and Shapeshift have different prices for the same assets". This is a classic arbitrage case (Kraken and Shapeshift are "exchanges").
I could exchange DAO for ETH on Kraken, transfer ETH to Shapeshift, exchange ETH for DAO and send them back to Kraken and due to price inconsintencies I would end up with more DAO than I initially started! Risk-free money, the best kind of money.

You can make money as long as
ETHDAO from Kraken * DAOETH from ShapeShift > 1 (+ fees + gas). Very simple formula, right?

Every cycle, was a 2% to 10% return of my capital. After a while, I started hitting the limits of Shapeshift and I had to make this run in parallel.

The question is what would you do if you had an algorithm that makes you 10% of your money every 20 minutes? The stupidest thing you could do is put tons of money into it.

Hybris

If you are not familiar with the Greek word hybris, consider yourself lucky. Hybris is when you think that you are invincible, better than gods. And this is the biggest NO NO you can do in trading.

After a couple of weeks, theDAO was hacked. 160 million dollars were stolen (or should I say frozen?) and noone knew what was going to happen. For me, this happened, 10 minutes before boarding a plane to fly to New York. Hybris. Or as people in the US say: Fuck.

I was smart(lucky?) enough to have good risk management habits (thank you Forex). I never, ever, ever risk more that 2% of my capital even if it seems the best kind of deal.

Luckily, the money were "restored" and I could withdraw/convert my DAO to Ethereum (but yes, I bought in-flight WiFi to keep up with what's going on).

This whole experience is a reminder that there are always things that you cannot predict. Things that you cannot control.
This was a systematic risk and there was no way I could have seen it coming. Pushing buttons and building algorithms is not enough. Proper risk management and knowing when you need to take a chill pill is what can keep you in the game.

On the next post, I will post the whole algorithm and go line by line. I also plan to discuss a little bit more about theDAO and Ethereum. If you don't want to miss any of these and get some more additional info, feel free to sign up to the newsletter where I talk about fintech, algorithms and the markets.

By the way, if you want to make your own cryptocurrency and learn more about Ethereum, I have a great post with the code posted here.

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

Machine learning algorithms are algorithms where a machine can identify patterns in your data. Yeap, it is that simple. For example, find all the animals in this photo and draw a box around them. Also, name that animal. Crazy I know. For trading as you can imagine it is pretty similar:

"Find how can I make money based on this chart and do all the trades.
Then wash my underwear and don't mix the colored with the whites"

In order for a machine to "learn", you need to teach it what is right or wrong (supervised learning) or give it a big dataset and let it got wild (unsupervised). For identifying objects this is straight-forward but what about trading?

Playing with data

I looked around to see if there is any machine learning program that can identify S/R lines but to no avail. So I decided to write the first machine learning program in python that identifies support and resistance lines in Python. Another first! Hooray!

But how can an algorithm identify these areas? Hoooooow?
Ladies and gents (and robots), let me introduce you to MeanShift, an unsupervised algorithm that is used mostly for image recognition and is pretty trivial to setup and run (but also very slow).

The idea is that this algorithm will let me partition my data (forex ticks) into areas and then I can use the "edges" as support and resistance lines. Cool idea but does it work?

Spoiler: It works.

We analyse around 12 million datapoints of EURUSD in 2014 and a couple of months of 2015. The resistance lines are placed automagically by a machine learning algorithm.

What is really cool (and spooky) is that the algorithm pretty much nails it. NAILS it hard. It gets really spooky when we are going to use the algorithm to identify micro-structures and start scalping.

The system is able to process any kind of timeseries data (stocks, forex, gold, whatever) and it will render an html interactive chart (like the chart above) with your data and the machine generated S/L. The code is here so go crazy.

Now let's step through the code.
After you have your set of data you need to read them and clean them. Prepare for some pandas magic.

We drop the empty values (weekends) and then we resample the data to 24 hours candlesticks (ohcl). This makes it MUCH easier to plot. The grouped_data are the data that we will feed into the ml algorithm.

On the next post, we'll discuss how to make this work even better, discuss some very interesting results (can the algorithm actually predict about the future?) and start using it in our own trading. If you want to check the next article and read more about trading and investing using algorithms, signup to the newsletter.

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

Update: The Machine Learning post is going to be epic but it takes so much time to make the code presentable. Bear with me, cool things are coming (as you've read at the newsletter)

Disclaimer: THIS IS NOT TAX ADVICE.

What is really surprising is that the majority of the requests in the newsletter are:

Use tools to assist Trading

Machine Learning to optimize trades

Taxes

The first two, I can understand. Everybody wants to be a better trader. I get it. But taxes? TAXES?

This is your lucky day. Forex taxes are super easy. Seriously. By default (this is called Section 988), all your losses are going to offset your income taxes without the 3k limit per year. This is much better than stock trading where losses offset your capital gains.

But what happens to gains? WHY DO YOU CARE? WHY?The majority of the Forex traders lose money (I call it "paying tuition") the first year(s), so you are better off keeping it simple until you have a proven and consistent strategy.

Having said that, if you actually make a profit, you are taxed at the short-term capital gain (sometimes up to 40%).
The solutions to when you start making money are:

Opting out of Section 988 and get taxed under Section 1256 where 60% of profits are taxed as long term capital gains and 40% as short term (but now losses cannot offset your income). This is very good when you make money, very bad when you don't.

Start an LLC

For people that just started experimenting with Forex and algotrading, I always suggest them to stay with Section 988 (the default) and when they start making some money (consistently) or they want to go full time, talk to me :) Seriously, there are so many things that you will start doing differently when you go from the "hobby" stage to "second income" to "full-time job" that there is no reason to over-optimize this.

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

I prefer to have my strategies in a JSON format that contains the name of the strategy and some specs (like how many pips for stop_loss or take_profit etc). That way, when we'll start using an event-based backtester, we can pass the strategy through a machine learning algorithm and try to optimize it.

Next line is loading our data in. I know people don't like pickle and there other ways to load data (and we are going to talk about BColz at some point) but for now, just bare with me.

The next line is self-explanatory. We pass the historical data to our algo and we get back some stats to print.

Let's focus on the algorithm a little bit and we can discuss plotting etc at a later point.

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

What is a good algotrading system without some neat strategies to deploy?

Below there is a list of strategies that I found online (or sent to me by traders that are on the newsletter). I plan to update the list as I keep coming across to new ideas.

The concept is that as we keep diving more and more into our algotrading system, I will show you how to code and deploy these strategies. I know for sure that most of them work with minimal changes. Worst case scenario, you'll have a system to test out your assumptions.

Here is the list (and please send me any other strategy that you think it should be included):

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

Building a backtest system is actually pretty easy. Easy to screw up I mean. Even though there are tons of excellent libraries out there (and we'll go through them at some point), I always like doing this on my own in order to fine-tune it.

From all the backtesting systems I have seen, we can assume that there are two categories:

The "for-loopers"

The Event generators

Today, we'll talk about for-loopers.

The "for-loopers" are my favorite type of backtesters. They are trivial to write and super fun to expand but they have some vital flows and sadly the majority of backtesters out there is "for-loopers" (ps: I need to find a better name for this!).

How for-loopers work? Using a for loop (as you might have guessed). It is something like this:

Very simple right? This is how one backtesting system works, that runs a momentum strategy:

So, what is the problem?

Very difficult to scale (horizontally)

Needs lots of work to keep your apply_strategy() working on backtesting and production

You need to have everything in the same programming language

Let's dive into these, one by one.

Scalability.
I was experimenting a couple a weeks ago with a hill-climbing algorithm to optimize one of my strategies. It is still running. After two weeks. And I build uber-scalable systems for a living.
Why is it still running? You can use multiprocessing, Disco, producer/consumer (using ZeroMQ) or just threads to speed this up but some problems are not "embarrassing parallel" (yes, this is an actual term, not one of my made-up words). The amount of work to scale a backtester like this (especially when you want to do same machine learning on top of it) is huge. You can do it but it is the wrong way.

Production and backtesting in sync
This. The times I have been bitten by this. I can recall the lost trades where I was "hm, why I entered this trade?" or my old time favorite "WHY TRAILING STOP WAS APPLIED NOW?".

Story time: I had an idea in order to optimize my strategy, to run a backtester to see what would happen if I could put a trailing stop AFTER the trade was profitable in order to always secure profits. Backtesting worked like a charm at a 13% increase of earnings and production lost every single trade. I figured it out after my algo lost $3400 in a couple of hours (a very expensive lesson).

Keeping the apply_strategy in sync is very difficult and becomes almost impossible when you want to do it in a distributed fashion. And you don't want to have two version of your strategy that are "almost" identical. Unless you have $3400 to spare.

Using different languages
I love Python. And Erlang. And Clojure. And J. And C. And R. And Ruby (no actually I hate Ruby). I want to be able to leverage the strength of other languages in my system. I want to try out strategies in R where there are very well-tested libraries and there is a huge community behind it. I want to have Erlang to scale my code and C to crunch data. If you want to be succesful (not only in trading), you need to be able to use all the available resources without prejudices. I have learnt tons of stuff from hanging out with R developers regarding how you can delta hedge bonds and visualize them or why Sharpe ratio can be a lie. Every language has a different crowd and you want as many people pouring ideas into your system.
If you try to have apply_strategy in different language then good luck with (2).

Are you convinced now? Well, I am not trying to convince you as for-loopers is a great way to run your initial tests. It is how I started and for many strategies I don't send them down to the pipeline. A "better" way (so you can sleep at night) is the event generators.

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

Before running any live algotrading system, it is a good practice to backtest (that means run a simulation) our algorithms. Have in mind that this doesn't mean that if your system is killing it for the last 5 years/months/days it will make a profit but it is a good indicator that you might be on to something.

There are four things that we need to take into consideration when we do our backtesting:

The quality of the data

How to load them efficiently

How to built our backtesting system

Try to have our backtesting and our live system share as much code as we can

Today, we are going to focus on (1) and (2).

For Forex data, I am using GainCapital. Their data are in the form of ticks. For a free source it is good enough. I used to use Oanda's historical data service but it seems that they moved it to a premium product. Too bad. Make sure that you use GainCapital's data only for experimentation.

For any other kind of paid historical data (ETFs, stocks, options stc), I am using eoddata.com (they also have some forex historical data but I haven't used them).

and you'll get a 25MB file named EUR_USD_Week1.csv. These are data for one week for one currency pair. You can imagine the amount of data you need to process for all currencies for the last five years (hint: a lot!). But don't worry, we are going optimize this. For now, let's open the file and inspect.

&gt;headEUR_USD_Week1.csv

lTid

cDealable

CurrencyPair

RateDateTime

RateBid

RateAsk

4464650058

D

EUR/USD

2015-11-01 17:00:06.490000000

1.103380

1.103770

4464650061

D

EUR/USD

2015-11-01 17:00:06.740000000

1.103400

1.103760

4464650072

D

EUR/USD

2015-11-01 17:00:07.990000000

1.103390

1.103750

4464650083

D

EUR/USD

2015-11-01 17:00:08.990000000

1.103400

1.103750

the things that we care about is the RateDateTime, RateBid and RateAsk. As you can understade each line has a timestamp and the how much was the price to buy or sell. Formats downloaded by other services are pretty similar.

There are many ways to load these data into Python but the most preferable when it comes to data slicing and manipulating is using Pandas.
We can always use the csv library to load data (and it might be faster) but we need to do some optimizations and processing first that as you will see it is pretty easy with pandas.

Another great tool to load TONS of GBs pretty efficiently and very fast is using Bcolz, covered in a much later post (or you can read a preview if you have signed up in the newsletter.

Manipulating data using Pandas
The data we downloaded are in ticks. Unless we are building an UHFT (ultra high frequency trading) algorithm, it is much more efficient (memory, storage and processing-wise) to
"group" these ticks into seconds (or minutes or hours depending on your strategy). This will make our download scale down from 25MB to just 35KB which translate to HUGE performance and memory benefits.

Let's group all these data in 15 minutes. How? Time to fall in love with resample.

df=pandas.read_csv(filename,parse_dates={'DateTime'},index_col='DateTime',names=['Tid','Dealable','Pair','DateTime','Buy','Sell'],date_parser=parse)# let's erase the columns we don't needdeldf['Tid']deldf['Dealable']deldf['Pair']# group every 15 minutes and create an OHLC bargrouped_data=df.resample('15Min',how='ohlc')

This is called OHLC (Open High Low Close) bar for every 15 minutes. You can see now that the ticks are grouped in 15 minute segments and you have the highest and lowest point that the price reached during these 15 minutes and also the open/close for buy and sell. Pure gold! Not only you have all the information you need but now it is extremely fast to load it. You just need to save the data:

::python# save to filegrouped_data.to_pickle(filename+'-OHLC.pkl')

and then you can reuse this 35kb file.

We can write a simple momentum algorithm that checks if there was a huge movement the last 15 minutes and if that was the case, let's buy. We will dive into this in a later post.

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

First of all, why do we choose Forex for algotrading? Why don't we become millionaires trading like everybody else? Why not just buy Tesla, Amazon, Google, Facebook, Twitter and hope for the best (PS: please read the legal outro at the end of this blog post before buying any stocks).

Forex has a nice (or terrible, depending on which side of the coin you are) thing called leverage.

Leverage can be 1:10, 1:50, 1:100, 1:200, 1:1000 depending on how suicidal you are or how sketchy your broker is (don't worry, we'll talk about brokers in the next post). Let's see an example.

We want to place a trade of $1k. The logical thing is that in order to buy something that costs $1k, you need to have in your account $1k, right? Nope. You can always get a loan. Oooooooor: Enter leverage.

If we have a 1:100 leverage, we can place a trade and "control" $1k with just $10. For those that failed at math (don't be ashamed, I am one of you), $10 x 100 (leverage) = $1,000.
That means you can trade big and win big! Actually 100 times more big! The catch is that you can actually go 100 times more small. Let's have another example.

I just got my bonus ($1k) and I want to play on Forex.
Without any leverage, I can buy 1,000 Forex units that cost $1 (by the way, there are no Forex units but we'll talk about this later).
How many units can I buy with 1:50 leverage?

If you answer if 100,000, you did something wrong. The answer is 50,000 ($1,000 x 50 leverage).

There you have it. One of the reasons that we are doing Forex, is because you can lose/win big. We'll get back to leverage when we start placing trades.

There are three more exciting reasons actually that are even more awesome (dare to say awesomer?).

Forex (almost) never sleeps. The markets are open ALL DAY, six days per week. To be more exact, there is not one market but four and they are overlapping providing the "all day" effect.

Forex is very volatile and there are tons of money moving around (more than 5 trillion per day). Crazy right?

No fees on trades. You don't pay $10 per trade like you are doing with stocks. Here you pay the spread which is just a fraction of a cent (again, we'll talk about this in another post).

All these reasons (leverage, all-day, volatility, fees) make Forex the most exciting platform to build and deploy your algorithms.

Coming up next, Forex brokers. How not to be scammed before even writing a line of code.

Legal outro. This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices.
If you lose any (or all) you money because you followed any trading advices or deployed this system in production, you cannot blame this random blog (and/or me). Enjoy at your own risk.

So, I am planning to cover the basics of how to build your own trading platform, write your own strategies and go on vacations while electrons are making you money. Or zeroing your account. Either way, it is going to be fun!

The majority of the examples are going to be in Python even though there might be parts in Erlang and I'll try to keep it as easy as it can be.